2024-02-28T22:38:14.867 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-28T22:38:14.867 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi --owner scheduled_vshankar@teuthology --verbose --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603 -- /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/orig.config.yaml 2024-02-28T22:38:14.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:38:15.006 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603 branch: main description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7575603' kernel: flavor: default kdb: true sha1: bd7d00fb724897dda560070d01bc6e5c80881f15 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2226 sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPyUeAHE+pp+qsi9PwfFSyTqcvIpf+MMHyBwtJQS5LT0oRTI8OnDc5En1PxVOxfZmoOK0RXw+Ryr8x9ys2OdmY= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYrpinHrdS6obIeVRroouFicW2iMRUs07L2SfDCXTt8b/FteHe2nSgbFWebkSSZsXWqLRD9kdEZo3L9LzMCNLk= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-28_15:21:56 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-28T22:38:15.006 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa; will attempt to use it 2024-02-28T22:38:15.006 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks 2024-02-28T22:38:15.007 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-28T22:38:15.008 INFO:teuthology.task.internal:Checking packages... 2024-02-28T22:38:15.032 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1fb5d10608dccee73edd6ef8a2c7b804a097c324' 2024-02-28T22:38:15.032 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-28T22:38:15.032 INFO:teuthology.packaging:ref: None 2024-02-28T22:38:15.032 INFO:teuthology.packaging:tag: None 2024-02-28T22:38:15.032 INFO:teuthology.packaging:branch: main 2024-02-28T22:38:15.032 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:38:15.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-28T22:38:15.517 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739.g1f09b1e2 2024-02-28T22:38:15.519 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-28T22:38:15.529 INFO:teuthology.task.internal:no buildpackages task found 2024-02-28T22:38:15.529 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-28T22:38:15.577 INFO:teuthology.task.internal:Saving configuration 2024-02-28T22:38:15.597 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-28T22:38:15.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-28T22:38:15.636 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-28 22:33:06.857282', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPyUeAHE+pp+qsi9PwfFSyTqcvIpf+MMHyBwtJQS5LT0oRTI8OnDc5En1PxVOxfZmoOK0RXw+Ryr8x9ys2OdmY='} 2024-02-28T22:38:15.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-28 22:33:06.855821', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYrpinHrdS6obIeVRroouFicW2iMRUs07L2SfDCXTt8b/FteHe2nSgbFWebkSSZsXWqLRD9kdEZo3L9LzMCNLk='} 2024-02-28T22:38:15.661 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-28T22:38:15.672 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-28T22:38:15.672 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-28T22:38:15.672 INFO:teuthology.run_tasks:Running task console_log... 2024-02-28T22:38:15.768 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8317cf9160>, signals=[15]) 2024-02-28T22:38:15.768 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-28T22:38:15.780 INFO:teuthology.task.internal:Opening connections... 2024-02-28T22:38:15.780 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T22:38:15.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:38:15.890 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-02-28T22:38:15.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:38:15.977 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-28T22:38:15.985 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2024-02-28T22:38:16.004 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2024-02-28T22:38:16.004 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2024-02-28T22:38:16.061 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="9" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="9" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:38:16.062 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:38:16.063 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:38:16.063 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:38:16.063 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:38:16.063 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:38:16.064 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2024-02-28T22:38:16.083 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-02-28T22:38:16.100 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-02-28T22:38:16.101 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-02-28T22:38:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-02-28T22:38:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-02-28T22:38:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:38:16.159 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:38:16.159 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-02-28T22:38:16.191 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-28T22:38:16.885 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-28T22:38:17.640 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-28T22:38:17.641 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2024-02-28T22:38:17.645 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-02-28T22:38:17.662 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-28T22:38:17.849 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-28T22:38:17.849 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2024-02-28T22:38:17.852 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-02-28T22:38:17.955 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-28T22:38:18.016 INFO:teuthology.run_tasks:Running task kernel... 2024-02-28T22:38:18.034 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'bd7d00fb724897dda560070d01bc6e5c80881f15'} 2024-02-28T22:38:18.034 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': 'bd7d00fb724897dda560070d01bc6e5c80881f15'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': 'bd7d00fb724897dda560070d01bc6e5c80881f15'}}, timeout 300 2024-02-28T22:38:18.034 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-28T22:38:18.035 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-28T22:38:18.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T22:38:18.052 DEBUG:teuthology.task.kernel:sha1 for host.b is bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.052 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "bd7d00fb724897dda560070d01bc6e5c80881f15"... 2024-02-28T22:38:18.052 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-02-28T22:38:18.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T22:38:18.054 DEBUG:teuthology.task.kernel:sha1 for host.a is bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.054 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "bd7d00fb724897dda560070d01bc6e5c80881f15"... 2024-02-28T22:38:18.054 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-02-28T22:38:18.109 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-361.el9.x86_64 2024-02-28T22:38:18.109 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.110 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-28T22:38:18.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.114 INFO:teuthology.orchestra.run.smithi057.stdout:5.14.0-361.el9.x86_64 2024-02-28T22:38:18.114 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.114 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-28T22:38:18.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:38:18.462 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum upgrade -y linux-firmware 2024-02-28T22:38:18.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum upgrade -y linux-firmware 2024-02-28T22:38:20.740 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-02-28T22:38:20.882 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - AppStream 9.8 MB/s | 8.1 MB 00:00 2024-02-28T22:38:22.415 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.5 MB 00:00 2024-02-28T22:38:22.700 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.5 MB 00:00 2024-02-28T22:38:24.081 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-02-28T22:38:24.350 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-02-28T22:38:25.923 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-02-28T22:38:26.248 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-02-28T22:38:30.618 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.1 MB 00:00 2024-02-28T22:38:30.865 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.1 MB 00:01 2024-02-28T22:38:32.292 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-02-28T22:38:32.350 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-02-28T22:38:33.342 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2024-02-28T22:38:34.093 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-28T22:38:38.308 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-28T22:38:39.409 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-02-28T22:38:40.537 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.538 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.539 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.540 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.541 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.542 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:38:40.543 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.544 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.545 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.546 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.547 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:38:40.547 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.547 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.547 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:38:40.547 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.110 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:38:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 1 Package 2024-02-28T22:38:41.122 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:38:41.122 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 376 M 2024-02-28T22:38:41.122 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:38:41.785 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:38:41.786 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.787 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:38:41.788 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.789 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.790 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.791 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.792 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.793 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:38:41.794 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:38:42.356 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:38:42.369 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 1 Package 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 376 M 2024-02-28T22:38:42.370 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:38:54.068 INFO:teuthology.orchestra.run.smithi057.stdout:linux-firmware-20240219-143.el9.noarch.rpm 33 MB/s | 376 MB 00:11 2024-02-28T22:38:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:38:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:Total 32 MB/s | 376 MB 00:11 2024-02-28T22:38:57.282 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:38:57.396 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:38:57.396 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:39:00.743 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:39:00.743 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:39:08.151 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:39:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-02-28T22:39:08.490 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: linux-firmware-20240219-143.el9.noarch 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:39:10.069 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:39:10.409 INFO:teuthology.task.kernel:Downloading kernel bd7d00fb724897dda560070d01bc6e5c80881f15 on host.a... 2024-02-28T22:39:10.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:39:10.562 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/ 2024-02-28T22:39:10.562 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc6_gbd7d00fb7248-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-28T22:39:10.565 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi057.front.sepia.ceph.com to complete... 2024-02-28T22:39:32.334 INFO:teuthology.orchestra.run.smithi084.stdout:linux-firmware-20240219-143.el9.noarch.rpm 7.4 MB/s | 376 MB 00:51 2024-02-28T22:39:32.334 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:39:32.334 INFO:teuthology.orchestra.run.smithi084.stdout:Total 7.3 MB/s | 376 MB 00:51 2024-02-28T22:39:35.362 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:39:35.543 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:39:35.543 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:39:38.735 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:39:38.736 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:39:45.775 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:39:46.126 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-02-28T22:39:46.209 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:47.179 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:47.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware-20240219-143.el9.noarch 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:39:47.607 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:39:47.926 INFO:teuthology.task.kernel:Downloading kernel bd7d00fb724897dda560070d01bc6e5c80881f15 on host.b... 2024-02-28T22:39:47.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd7d00fb724897dda560070d01bc6e5c80881f15 2024-02-28T22:39:48.151 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/ 2024-02-28T22:39:48.152 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc6_gbd7d00fb7248-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-28T22:39:48.154 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi084.front.sepia.ceph.com to complete... 2024-02-28T22:40:08.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28 22:40:08 URL:https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/kernel-6.8.0_rc6_gbd7d00fb7248-1.x86_64.rpm [335852831/335852831] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-28T22:40:08.660 INFO:teuthology.orchestra.run.smithi057.stderr:FINISHED --2024-02-28 22:40:08-- 2024-02-28T22:40:08.660 INFO:teuthology.orchestra.run.smithi057.stderr:Total wall clock time: 58s 2024-02-28T22:40:08.660 INFO:teuthology.orchestra.run.smithi057.stderr:Downloaded: 1 files, 320M in 58s (5.56 MB/s) 2024-02-28T22:40:08.663 INFO:teuthology.task.kernel:Installing kernel bd7d00fb724897dda560070d01bc6e5c80881f15 on host.a... 2024-02-28T22:40:08.663 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-28T22:40:10.698 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying... ######################################## 2024-02-28T22:40:10.760 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing... ######################################## 2024-02-28T22:40:10.845 INFO:teuthology.orchestra.run.smithi057.stdout:Updating / installing... 2024-02-28T22:40:18.685 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-6.8.0_rc6_gbd7d00fb7248-1 ######################################## 2024-02-28T22:40:32.610 INFO:teuthology.orchestra.run.smithi057.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-28T22:40:55.734 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-28T22:40:55.735 DEBUG:teuthology.orchestra.run.smithi057:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-28T22:40:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/System.map 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/config 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-28T22:40:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-28T22:40:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu/mce 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-28T22:40:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/ansi_cprng.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/anubis.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/arc4.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_memcpy.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_pq.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_tx.ko 2024-02-28T22:40:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_xor.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/raid6test.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/authenc.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/authencesn.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blake2b_generic.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blowfish_common.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blowfish_generic.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/camellia_generic.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast5_generic.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast6_generic.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast_common.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/ccm.ko 2024-02-28T22:40:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cmac.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crc32_generic.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crypto_engine.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crypto_user.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cts.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/des_generic.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/dh_generic.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/echainiv.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/essiv.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/fcrypt.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/khazad.ko 2024-02-28T22:40:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/lz4.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/lz4hc.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/md4.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/michael_mic.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/pcbc.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/pcrypt.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/rmd160.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/seed.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/serpent_generic.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/tcrypt.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/tea.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/twofish_common.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/twofish_generic.ko 2024-02-28T22:40:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/vmac.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/wp512.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xcbc.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xor.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xxhash_generic.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/acpi_pad.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/custom_method.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/ec_sys.ko 2024-02-28T22:40:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/platform_profile.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/sbs.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/sbshc.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/video.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/acard-ahci.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/ahci_platform.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/ata_generic.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/libahci_platform.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_acpi.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ali.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_amd.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_artop.ko 2024-02-28T22:40:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_atiixp.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_atp867x.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cmd640.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cmd64x.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cypress.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_efar.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt366.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt37x.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_it8213.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_it821x.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_jmicron.ko 2024-02-28T22:40:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_marvell.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_mpiix.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_netcell.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ninja32.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ns87410.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ns87415.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_oldpiix.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_opti.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_optidma.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pcmcia.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-28T22:40:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_piccolo.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_rdc.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sch.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_serverworks.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sil680.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sis.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sl82c105.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_triflex.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_via.ko 2024-02-28T22:40:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pdc_adma.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_inic162x.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_mv.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_nv.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_promise.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_qstor.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sil.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sil24.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sis.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_svw.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sx4.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_uli.ko 2024-02-28T22:40:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_via.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_vsc.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/auxdisplay 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/auxdisplay/ks0108.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/bcma 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/bcma/bcma.ko 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/aoe 2024-02-28T22:40:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/aoe/aoe.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/brd.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/drbd 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/drbd/drbd.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/floppy.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/loop.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/mtip32xx 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/nbd.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/null_blk 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/null_blk/null_blk.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/pktcdvd.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/rbd.ko 2024-02-28T22:40:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/virtio_blk.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkback 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkfront.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/zram 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/zram/zram.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hangcheck-timer.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-28T22:40:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/via-rng.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/lp.ko 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/mwave 2024-02-28T22:40:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/mwave/mwave.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ppdev.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tlclk.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/uv_mmtimer.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/virtio_console.ko 2024-02-28T22:40:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp/ccp.ko 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel 2024-02-28T22:40:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_common 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/padlock-aes.ko 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/padlock-sha.ko 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/virtio 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dca 2024-02-28T22:40:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dca/dca.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/devfreq 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/dw 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/ioat 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/amd64_edac.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/e752x_edac.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/edac_mce_amd.ko 2024-02-28T22:40:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i3000_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i3200_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i5100_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i5400_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i7300_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i7core_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i82975x_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/ie31200_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/sb_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/x38_edac.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware/edd.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio 2024-02-28T22:40:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio/gpio-ich.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-appleir.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-aureal.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-axff.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-dr.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-elecom.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-elo.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-emsff.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gaff.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gt683r.ko 2024-02-28T22:40:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gyration.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtekff.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-hyperv.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-icade.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-keytouch.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-kye.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-lcpower.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-led.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-lenovo.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-28T22:40:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-multitouch.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-ortek.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-penmount.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-petalynx.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-picolcd.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-pl.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-plantronics.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-primax.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-rmi.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-common.ko 2024-02-28T22:40:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-saitek.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-samsung.ko 2024-02-28T22:40:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sjoy.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sony.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-speedlink.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-steelseries.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sunplus.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-tivo.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-tmff.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-topseed.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-twinhan.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-uclogic.ko 2024-02-28T22:40:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-waltop.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-wiimote.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-xinmo.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-zpff.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-zydacron.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/uhid.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/wacom.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_balloon.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_utils.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_vmbus.ko 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon 2024-02-28T22:40:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/abituguru.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/abituguru3.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ad7414.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ad7418.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adc128d818.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1025.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1026.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1029.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1031.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm9240.ko 2024-02-28T22:40:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ads7828.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7410.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7411.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7462.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7470.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7475.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7x10.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/amc6821.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/applesmc.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asb100.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asc7621.ko 2024-02-28T22:40:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/atxp1.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/coretemp.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/dme1737.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ds1621.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ds620.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/emc1403.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/emc6w201.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f71805f.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f71882fg.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f75375s.ko 2024-02-28T22:40:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/fam15h_power.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/fschmd.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/g760a.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/g762.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/gl518sm.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/gl520sm.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/i5500_temp.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/i5k_amb.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ibmaem.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ibmpex.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ina209.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ina2xx.ko 2024-02-28T22:40:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/it87.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/k10temp.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/k8temp.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lineage-pem.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm63.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm73.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm75.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm77.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm78.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm80.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm83.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm85.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm87.ko 2024-02-28T22:40:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm90.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm92.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm93.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95234.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95241.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95245.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc2945.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4151.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4215.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4222.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4245.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4260.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4261.ko 2024-02-28T22:40:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max16065.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max1619.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max1668.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max197.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6639.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6650.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6697.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/mcp3021.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6683.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6775-core.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6775.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct7802.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pc87360.ko 2024-02-28T22:40:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pc87427.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pcf8591.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-28T22:40:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/powr1220.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch5627.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch5636.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sht15.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sht21.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/shtc1.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sis5595.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47b397.ko 2024-02-28T22:40:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47m1.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47m192.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/thmc50.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp102.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp103.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp401.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp421.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/via-cputemp.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/via686a.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/vt1211.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/vt8231.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83627ehf.ko 2024-02-28T22:40:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83627hf.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83781d.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83791d.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83792d.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83793.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83795.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83l785ts.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83l786ng.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-28T22:40:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-28T22:40:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-28T22:40:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-dev.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-mux.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-smbus.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-stub.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_core.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-28T22:40:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/cxgb4 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx4 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx5 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mthca 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/ipoib 2024-02-28T22:40:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/iser 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/isert 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srp 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srpt 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input 2024-02-28T22:40:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/gameport.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/joydev.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/keyboard 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/appletouch.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/bcm5974.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/cyapatp.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/sermouse.ko 2024-02-28T22:40:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/rmi4 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/altera_ps2.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/arc_ps2.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/serio_raw.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/sparse-keymap.ko 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds 2024-02-28T22:40:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-blinkm.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-lm3530.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-lp3944.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-ss4200.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-28T22:40:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/bcache 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/bcache/bcache.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-bio-prison.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-cache-smq.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-cache.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-crypt.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-delay.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-flakey.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-log-userspace.ko 2024-02-28T22:40:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-multipath.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-queue-length.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-raid.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-round-robin.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-service-time.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-switch.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-thin-pool.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-verity.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/persistent-data 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid0.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid1.ko 2024-02-28T22:40:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid10.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid456.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptbase.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptctl.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptfc.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptlan.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptsas.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptscsih.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptspi.ko 2024-02-28T22:40:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/lpc_ich.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/lpc_sch.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/mfd-core.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/sm501.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/viperboard.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/vx855.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/wl1273-core.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/altera-stapl 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/apds9802als.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/apds990x.ko 2024-02-28T22:40:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/bh1770glc.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/cb710 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/cb710/cb710.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/at24.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/max6875.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/enclosure.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/hpilo.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/isl29003.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/isl29020.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-28T22:40:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei-me.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei-txe.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-gru 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tifm_7xx1.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tifm_core.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tsl2550.ko 2024-02-28T22:40:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_balloon.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_vmci 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/bonding 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/bonding/bonding.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/b53 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6xxx 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-28T22:40:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dummy.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/eql.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-28T22:40:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/adaptec 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/agere 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/alteon 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/altera 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-28T22:40:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/alx 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-28T22:40:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/genet 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade/bna 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-28T22:40:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco/enic 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-28T22:40:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dnet.ko 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex/benet 2024-02-28T22:40:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ethoc.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/fealnx.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000e 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/fm10k 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/i40e 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-28T22:40:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/iavf 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igb 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igbvf 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/jme.ko 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell 2024-02-28T22:40:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/micrel 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-28T22:40:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/neterion 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/nvidia 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-28T22:40:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rdc 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-28T22:40:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rocker 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sfc 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/silan 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-28T22:40:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-28T22:40:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/tehuti 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ti 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/xircom 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-28T22:40:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/geneve.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/hyperv 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ieee802154 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ifb.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ipvlan 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/macvlan.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/macvtap.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-28T22:40:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mii.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/net_failover.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/netconsole.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/nlmon.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/amd.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-28T22:40:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm87xx.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/broadcom.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/cicada.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/davicom.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/dp83640.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/et1011c.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/icplus.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/lxt.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/marvell.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/micrel.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/national.ko 2024-02-28T22:40:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/phylink.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/qsemi.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/realtek.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/smsc.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/ste10Xp.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/vitesse.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_async.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-28T22:40:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pppoe.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pppox.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pptp.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip/slhc.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip/slip.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/sungem_phy.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/tap.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_random.ko 2024-02-28T22:40:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/tun.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/veth.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/virtio_net.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vmxnet3 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vxlan 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vxlan/vxlan.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netback 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netfront.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ntb 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ntb/ntb.ko 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme 2024-02-28T22:40:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host/nvme-core.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host/nvme.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_cs.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_pc.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_serial.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/hotplug 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/pcie 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-28T22:40:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/xen-pcifront.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/i82092.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/pd6729.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl/intel 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/acerhdf.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-28T22:40:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/hdaps.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel_ips.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-28T22:40:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/wmi.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-28T22:40:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps_parport.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/pps_core.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp/ptp.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp/ptp_kvm.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-28T22:40:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-em3027.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-max6900.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-28T22:40:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-x1205.ko 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi 2024-02-28T22:40:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-9xxx.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-sas.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-xxxx.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/BusLogic.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/a100u2w.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aacraid 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/advansys.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/am53c974.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/arcmsr 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-28T22:40:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/atp870u.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/be2iscsi 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bfa 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bfa/bfa.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2fc 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2i 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ch.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/csiostor 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-28T22:40:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/dc395x.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/dmx3191d.ko 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esas2r 2024-02-28T22:40:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esp_scsi.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fnic 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fnic/fnic.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hpsa.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hptiop.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hv_storvsc.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/initio.ko 2024-02-28T22:40:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ipr.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ips.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/isci 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/isci/isci.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libfc 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libfc/libfc.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libiscsi.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libsas 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libsas/libsas.ko 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/lpfc 2024-02-28T22:40:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mpt3sas 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvsas 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvumi.ko 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pm8001 2024-02-28T22:40:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pmcraid.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla1280.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla4xxx 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/raid_class.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_debug.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-28T22:40:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ses.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/st.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/stex.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/sym53c8xx_2 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/virtio_scsi.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/wd719x.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ssb 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ssb/ssb.ko 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target 2024-02-28T22:40:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/iscsi 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/loopback 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_file.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_iblock.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_mod.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_pscsi.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_user.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/tcm_fc 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal 2024-02-28T22:40:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-28T22:40:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/ipwireless 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/n_gsm.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/n_hdlc.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/nozomi.ko 2024-02-28T22:40:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/8250 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/arc_uart.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/jsm 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/core 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/host 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio.ko 2024-02-28T22:40:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_aec.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_cif.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_pci_generic.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_sercos3.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/cdc-acm.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/usblp.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/usbtmc.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/host 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-28T22:40:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image/mdc800.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image/microtek.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/adutux.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/appledisplay.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/emi26.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/emi62.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/ezusb.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/idmouse.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/iowarrior.ko 2024-02-28T22:40:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/ldusb.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/legousbtower.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/sisusbvga 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usb3503.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usblcd.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/uss720.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/yurex.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/phy 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/phy/phy-generic.ko 2024-02-28T22:40:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/aircable.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ark3116.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ch341.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cp210x.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cyberjack.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/empeg.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-28T22:40:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/io_ti.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ipaq.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ipw.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ir-usb.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/keyspan.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mct_u232.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mos7720.ko 2024-02-28T22:40:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mos7840.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/navman.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/omninet.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/opticon.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/option.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/oti6858.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/pl2303.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/qcaux.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/qcserial.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/quatech2.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/safe_serial.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/sierra.ko 2024-02-28T22:40:55.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ssu100.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/symbolserial.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb_debug.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/visor.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/whiteheat.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage 2024-02-28T22:40:55.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/uas.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-karma.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-28T22:40:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/usb-storage.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/vfio.ko 2024-02-28T22:40:55.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_net.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_scsi.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/apple_bl.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/lcd.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio 2024-02-28T22:40:55.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_balloon.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_mmio.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_ring.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters/ds2482.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters/ds2490.ko 2024-02-28T22:40:55.916 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-28T22:40:55.917 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/wire.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/hpwdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/i6300esb.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ib700wdt.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ibmasr.ko 2024-02-28T22:40:55.918 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/it87_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/machzwd.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/nv_tco.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/softdog.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/via_wdt.ko 2024-02-28T22:40:55.919 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/wdt_pci.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/xen_wdt.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-evtchn.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-gntalloc.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-gntdev.ko 2024-02-28T22:40:55.920 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-pciback 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-privcmd.ko 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xenfs 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/9p 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/9p/9p.ko 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/affs 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/affs/affs.ko 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/befs 2024-02-28T22:40:55.921 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/befs/befs.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/binfmt_misc.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/btrfs 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/btrfs/btrfs.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cachefiles 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cachefiles/cachefiles.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ceph 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ceph/ceph.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/coda 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/coda/coda.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cramfs 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cramfs/cramfs.ko 2024-02-28T22:40:55.922 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/dlm 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/dlm/dlm.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ecryptfs 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/f2fs 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/f2fs/f2fs.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/fat.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/msdos.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/vfat.ko 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse 2024-02-28T22:40:55.923 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse/cuse.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse/fuse.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/gfs2 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/gfs2/gfs2.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfs 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfs/hfs.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfsplus 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfsplus/hfsplus.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/isofs 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/isofs/isofs.ko 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/jfs 2024-02-28T22:40:55.924 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/jfs/jfs.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/lockd 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/lockd/lockd.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/minix 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/minix/minix.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/netfs 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/netfs/netfs.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/blocklayout 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/filelayout 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-28T22:40:55.925 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/flexfilelayout 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfs.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfsv3.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfsv4.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common/grace.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common/nfs_acl.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfsd 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfsd/nfsd.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nilfs2 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nilfs2/nilfs2.ko 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls 2024-02-28T22:40:55.926 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-celtic.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-centeuro.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-croatian.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-cyrillic.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-gaelic.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-greek.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-iceland.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-inuit.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-roman.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-romanian.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-turkish.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1250.ko 2024-02-28T22:40:55.927 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1251.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1255.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp737.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp775.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp850.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp852.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp855.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp857.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp860.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp861.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp862.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp863.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp864.ko 2024-02-28T22:40:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp865.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp866.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp869.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp874.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp932.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp936.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp949.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp950.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_euc-jp.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-1.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-13.ko 2024-02-28T22:40:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-14.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-15.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-2.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-3.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-4.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-5.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-6.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-7.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-9.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-r.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-ru.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-u.ko 2024-02-28T22:40:55.930 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_utf8.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/cluster 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlm 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlmfs 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-28T22:40:55.931 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/overlayfs 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/overlayfs/overlay.ko 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/pstore 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/pstore/ramoops.ko 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/reiserfs 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/reiserfs/reiserfs.ko 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/romfs 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/romfs/romfs.ko 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/client 2024-02-28T22:40:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/client/cifs.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common/cifs_arc4.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common/cifs_md4.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/squashfs 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/squashfs/squashfs.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/sysv 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/sysv/sysv.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/udf 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/udf/udf.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ufs 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ufs/ufs.ko 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/xfs 2024-02-28T22:40:55.933 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/xfs/xfs.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/rcu 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/rcu/rcutorture.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/torture.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/trace 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/asn1_encoder.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crc-itu-t.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crc8.ko 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto 2024-02-28T22:40:55.934 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto/libarc4.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto/libdes.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/libcrc32c.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lru_cache.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4/lz4_compress.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4/lz4hc_compress.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/math 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/math/cordic.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/raid6 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/raid6/raid6_pq.ko 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/reed_solomon 2024-02-28T22:40:55.935 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_bm.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_fsm.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_kmp.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/zstd 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/zstd/zstd_compress.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/mm 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/mm/hwpoison-inject.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/6lowpan.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_dest.ko 2024-02-28T22:40:55.936 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_fragment.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_hop.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_mobility.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_routing.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_udp.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/garp.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/mrp.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/p8022.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/psnap.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/stp.ko 2024-02-28T22:40:55.937 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/8021q 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/8021q/8021q.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_fd.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_rdma.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_virtio.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/batman-adv 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/batman-adv/batman-adv.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/br_netfilter.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/bridge.ko 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter 2024-02-28T22:40:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-28T22:40:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtables.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-28T22:40:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ceph 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ceph/libceph.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core/failover.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core/pktgen.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_diag.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_ipv4.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_ipv6.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dns_resolver 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dns_resolver/dns_resolver.ko 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa 2024-02-28T22:40:55.941 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/dsa_core.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_brcm.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_dsa.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_none.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_trailer.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/6lowpan 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/ieee802154.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ah4.ko 2024-02-28T22:40:55.942 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/esp4.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/fou.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/gre.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/inet_diag.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_gre.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_tunnel.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_vti.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ipcomp.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ipip.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-28T22:40:55.943 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-28T22:40:55.944 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_bic.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_dctcp.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_diag.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_highspeed.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_htcp.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_hybla.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_illinois.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_lp.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_scalable.ko 2024-02-28T22:40:55.945 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_vegas.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_veno.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_westwood.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_yeah.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tunnel4.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/udp_diag.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/udp_tunnel.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ah6.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/esp6.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/fou6.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_tunnel.ko 2024-02-28T22:40:55.946 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_vti.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ipcomp6.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-28T22:40:55.947 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-28T22:40:55.948 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/sit.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/tunnel6.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/key 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/key/af_key.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_core.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_eth.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ip.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ip6.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_netlink.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ppp.ko 2024-02-28T22:40:55.949 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/llc 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/llc/llc.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mac802154 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mac802154/mac802154.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mpls 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mpls/mpls_gso.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-28T22:40:55.950 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-28T22:40:55.951 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-28T22:40:55.952 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conncount.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-28T22:40:55.953 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_flow_table.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_log_syslog.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_irc.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_sip.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-28T22:40:55.954 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_tables.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_log.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_chain_nat.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_compat.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_ct.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-28T22:40:55.955 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_fib.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_hash.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_limit.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_log.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_masq.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_nat.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_numgen.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_queue.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_quota.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_redir.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_reject.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_reject_inet.ko 2024-02-28T22:40:55.956 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_socket.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_tunnel.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_AUDIT.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CT.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_DSCP.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_HL.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_HMARK.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_LED.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_LOG.ko 2024-02-28T22:40:55.957 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NETMAP.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NFLOG.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_RATEEST.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_SECMARK.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TEE.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TPROXY.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TRACE.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_addrtype.ko 2024-02-28T22:40:55.958 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_bpf.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cgroup.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cluster.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_comment.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connbytes.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connlabel.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connlimit.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connmark.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_conntrack.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cpu.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_dccp.ko 2024-02-28T22:40:55.959 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_devgroup.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_dscp.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ecn.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_esp.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_hashlimit.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_helper.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_hl.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ipcomp.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_iprange.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ipvs.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_l2tp.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_length.ko 2024-02-28T22:40:55.960 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_limit.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_mac.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_mark.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_multiport.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_nat.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_nfacct.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_osf.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_owner.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_physdev.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_pkttype.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_policy.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_quota.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_rateest.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_realm.ko 2024-02-28T22:40:55.961 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_recent.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_sctp.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_set.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_socket.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_state.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_statistic.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_string.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_tcpmss.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_time.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_u32.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netlink 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netlink/netlink_diag.ko 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/nsh 2024-02-28T22:40:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/nsh/nsh.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/openvswitch.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-geneve.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-gre.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-vxlan.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/packet 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/packet/af_packet_diag.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds_rdma.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds_tcp.ko 2024-02-28T22:40:55.963 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_csum.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_gact.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_mirred.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_nat.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_pedit.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_police.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_simple.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_skbedit.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_vlan.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_basic.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_bpf.ko 2024-02-28T22:40:55.964 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_flow.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_fw.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_route.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_u32.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_cmp.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_ipset.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_meta.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_nbyte.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_text.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_u32.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_choke.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_codel.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_drr.ko 2024-02-28T22:40:55.965 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_fq.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_gred.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_hfsc.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_hhf.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_htb.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_ingress.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_mqprio.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_mqprio_lib.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_multiq.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_netem.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_pie.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_plug.ko 2024-02-28T22:40:55.966 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_prio.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_qfq.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_red.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_sfb.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_sfq.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_tbf.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_teql.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp/sctp.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp/sctp_diag.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-28T22:40:55.967 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/sunrpc.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/xprtrdma 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc/diag.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc/tipc.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/unix 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/unix/unix_diag.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock.ko 2024-02-28T22:40:55.968 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/xfrm 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/encrypted-keys 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/trusted-keys 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/trusted-keys/trusted.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt/lib 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt/lib/irqbypass.ko 2024-02-28T22:40:55.969 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.alias 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.alias.bin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.alias.bin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.bin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.modinfo 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.dep 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.dep.bin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.devname 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.order 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.softdep 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.symbols 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.symbols.bin 2024-02-28T22:40:55.970 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/vmlinuz 2024-02-28T22:40:55.975 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:55.975 DEBUG:teuthology.orchestra.run.smithi057:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout:postinstall scriptlet (using /bin/sh): 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout: /usr/bin/kernel-install add 6.8.0-rc6-gbd7d00fb7248 /lib/modules/6.8.0-rc6-gbd7d00fb7248/vmlinuz 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout:for file in vmlinuz System.map config; do 2024-02-28T22:40:56.000 INFO:teuthology.orchestra.run.smithi057.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc6-gbd7d00fb7248/${file}" "/boot/${file}-6.8.0-rc6-gbd7d00fb7248"; then 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout: cp "/lib/modules/6.8.0-rc6-gbd7d00fb7248/${file}" "/boot/${file}-6.8.0-rc6-gbd7d00fb7248" 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout: fi 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:done 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:new-kernel-pkg --remove 6.8.0-rc6-gbd7d00fb7248 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc6-gbd7d00fb7248.img 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-install remove 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-28T22:40:56.001 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/update-bootloader --remove 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:56.002 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-02-28T22:40:56.002 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.06 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 68.el9 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2024-02-28T22:40:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Group : Unspecified 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 8264254 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : builder@centos.org 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2024-02-28T22:40:56.086 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:40:56.087 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2024-02-28T22:40:56.088 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-28T22:40:56.088 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:56.088 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-28T22:40:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2024-02-28T22:40:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-28T22:40:58.170 INFO:teuthology.orchestra.run.smithi057.stderr:done 2024-02-28T22:40:58.180 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-02-28T22:40:58.195 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.ufFO25Uhaf 2024-02-28T22:40:58.196 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ufFO25Uhaf 2024-02-28T22:40:58.272 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.ufFO25Uhaf 2024-02-28T22:40:58.410 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.ufFO25Uhaf is 6KB 2024-02-28T22:40:58.459 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.ufFO25Uhaf 2024-02-28T22:40:58.474 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2024-02-28T22:40:58.542 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-28T22:40:58.543 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-28T22:40:58.543 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-28T22:40:58.543 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc6-gbd7d00fb7248.conf 2024-02-28T22:40:58.544 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:40:58.709 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2024-02-28T22:41:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28 22:41:17 URL:https://4.chacra.ceph.com/r/kernel/testing/bd7d00fb724897dda560070d01bc6e5c80881f15/centos/9/flavors/default/x86_64/kernel-6.8.0_rc6_gbd7d00fb7248-1.x86_64.rpm [335852831/335852831] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-28T22:41:17.999 INFO:teuthology.orchestra.run.smithi084.stderr:FINISHED --2024-02-28 22:41:17-- 2024-02-28T22:41:17.999 INFO:teuthology.orchestra.run.smithi084.stderr:Total wall clock time: 1m 30s 2024-02-28T22:41:17.999 INFO:teuthology.orchestra.run.smithi084.stderr:Downloaded: 1 files, 320M in 1m 29s (3.59 MB/s) 2024-02-28T22:41:18.002 INFO:teuthology.task.kernel:Installing kernel bd7d00fb724897dda560070d01bc6e5c80881f15 on host.b... 2024-02-28T22:41:18.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-28T22:41:19.939 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying... ######################################## 2024-02-28T22:41:20.000 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing... ######################################## 2024-02-28T22:41:20.080 INFO:teuthology.orchestra.run.smithi084.stdout:Updating / installing... 2024-02-28T22:41:27.642 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-6.8.0_rc6_gbd7d00fb7248-1 ######################################## 2024-02-28T22:41:28.712 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-28T22:41:28.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:41:28.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:41:42.317 INFO:teuthology.orchestra.run.smithi084.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-28T22:42:03.196 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-02-28T22:42:04.823 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-28T22:42:04.823 DEBUG:teuthology.orchestra.run.smithi084:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/System.map 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/config 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-28T22:42:04.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-28T22:42:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu/mce 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/arch/x86/kvm/kvm.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/ansi_cprng.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/anubis.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/arc4.ko 2024-02-28T22:42:04.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_memcpy.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_pq.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_tx.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/async_xor.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/async_tx/raid6test.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/authenc.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/authencesn.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blake2b_generic.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blowfish_common.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/blowfish_generic.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/camellia_generic.ko 2024-02-28T22:42:04.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast5_generic.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast6_generic.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cast_common.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/ccm.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cmac.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crc32_generic.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crypto_engine.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/crypto_user.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/cts.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/des_generic.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/dh_generic.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/echainiv.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/essiv.ko 2024-02-28T22:42:04.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/fcrypt.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/khazad.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/lz4.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/lz4hc.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/md4.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/michael_mic.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/pcbc.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/pcrypt.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/rmd160.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/seed.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/serpent_generic.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/tcrypt.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/tea.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/twofish_common.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/twofish_generic.ko 2024-02-28T22:42:04.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/vmac.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/wp512.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xcbc.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xor.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/crypto/xxhash_generic.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/acpi_pad.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/custom_method.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/ec_sys.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/platform_profile.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/sbs.ko 2024-02-28T22:42:04.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/sbshc.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/acpi/video.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/acard-ahci.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/ahci_platform.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/ata_generic.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/libahci_platform.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_acpi.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ali.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_amd.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_artop.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_atiixp.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_atp867x.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cmd640.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cmd64x.ko 2024-02-28T22:42:04.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_cypress.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_efar.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt366.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt37x.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_it8213.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_it821x.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_jmicron.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_marvell.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_mpiix.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_netcell.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ninja32.ko 2024-02-28T22:42:04.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ns87410.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_ns87415.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_oldpiix.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_opti.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_optidma.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pcmcia.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_piccolo.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_rdc.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sch.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_serverworks.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sil680.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sis.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_sl82c105.ko 2024-02-28T22:42:04.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_triflex.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pata_via.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/pdc_adma.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_inic162x.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_mv.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_nv.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_promise.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_qstor.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sil.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sil24.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sis.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_svw.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_sx4.ko 2024-02-28T22:42:04.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_uli.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_via.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ata/sata_vsc.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/auxdisplay 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/auxdisplay/ks0108.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/bcma 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/bcma/bcma.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/aoe 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/aoe/aoe.ko 2024-02-28T22:42:04.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/brd.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/drbd 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/drbd/drbd.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/floppy.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/loop.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/mtip32xx 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/nbd.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/null_blk 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/null_blk/null_blk.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/pktcdvd.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/rbd.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/virtio_blk.ko 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkback 2024-02-28T22:42:04.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/xen-blkfront.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/zram 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/block/zram/zram.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hangcheck-timer.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/via-rng.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-28T22:42:04.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/lp.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/mwave 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/mwave/mwave.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/ppdev.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tlclk.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-28T22:42:04.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/uv_mmtimer.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/char/virtio_console.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-28T22:42:04.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/ccp/ccp.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_common 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/padlock-aes.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/padlock-sha.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/virtio 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dca 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dca/dca.ko 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/devfreq 2024-02-28T22:42:04.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/dw 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/ioat 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/amd64_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/e752x_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/edac_mce_amd.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i3000_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i3200_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i5100_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i5400_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i7300_edac.ko 2024-02-28T22:42:04.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i7core_edac.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/i82975x_edac.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/ie31200_edac.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/sb_edac.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/edac/x38_edac.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware/edd.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio/gpio-ich.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid 2024-02-28T22:42:04.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-appleir.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-aureal.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-axff.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-dr.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-elecom.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-elo.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-emsff.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gaff.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gt683r.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-gyration.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-holtekff.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-hyperv.ko 2024-02-28T22:42:04.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-icade.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-keytouch.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-kye.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-lcpower.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-led.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-lenovo.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-multitouch.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-ortek.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-penmount.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-petalynx.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-picolcd.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-pl.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-plantronics.ko 2024-02-28T22:42:04.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-primax.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-rmi.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-common.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-roccat.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-saitek.ko 2024-02-28T22:42:04.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-samsung.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sjoy.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sony.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-speedlink.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-steelseries.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-sunplus.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-tivo.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-tmff.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-topseed.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-twinhan.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-uclogic.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-waltop.ko 2024-02-28T22:42:04.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-wiimote.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-xinmo.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-zpff.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/hid-zydacron.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/uhid.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hid/wacom.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_balloon.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_utils.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hv/hv_vmbus.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/abituguru.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/abituguru3.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ad7414.ko 2024-02-28T22:42:04.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ad7418.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adc128d818.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1025.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1026.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1029.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm1031.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adm9240.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ads7828.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7410.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7411.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7462.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7470.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7475.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/adt7x10.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/amc6821.ko 2024-02-28T22:42:04.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/applesmc.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asb100.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asc7621.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/atxp1.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/coretemp.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/dme1737.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ds1621.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ds620.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/emc1403.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/emc6w201.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f71805f.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f71882fg.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/f75375s.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/fam15h_power.ko 2024-02-28T22:42:04.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/fschmd.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/g760a.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/g762.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/gl518sm.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/gl520sm.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/i5500_temp.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/i5k_amb.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ibmaem.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ibmpex.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ina209.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ina2xx.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/it87.ko 2024-02-28T22:42:04.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/k10temp.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/k8temp.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lineage-pem.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm63.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm73.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm75.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm77.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm78.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm80.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm83.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm85.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm87.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm90.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm92.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm93.ko 2024-02-28T22:42:04.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95234.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95241.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/lm95245.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc2945.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4151.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4215.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4222.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4245.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4260.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/ltc4261.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max16065.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max1619.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max1668.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max197.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6639.ko 2024-02-28T22:42:04.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6650.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/max6697.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/mcp3021.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6683.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6775-core.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct6775.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/nct7802.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pc87360.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pc87427.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pcf8591.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-28T22:42:04.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/powr1220.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch5627.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch5636.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sht15.ko 2024-02-28T22:42:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sht21.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/shtc1.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/sis5595.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47b397.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47m1.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/smsc47m192.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/thmc50.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp102.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp103.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp401.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/tmp421.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/via-cputemp.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/via686a.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/vt1211.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/vt8231.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83627ehf.ko 2024-02-28T22:42:04.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83627hf.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83781d.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83791d.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83792d.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83793.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83795.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83l785ts.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/hwmon/w83l786ng.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-28T22:42:04.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-28T22:42:04.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-dev.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-mux.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-smbus.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/i2c/i2c-stub.ko 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core 2024-02-28T22:42:04.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_core.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/cxgb4 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx4 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx5 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mthca 2024-02-28T22:42:04.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/ipoib 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/iser 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/isert 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srp 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srpt 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-28T22:42:04.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/gameport/gameport.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/joydev.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/keyboard 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/appletouch.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/bcm5974.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/cyapatp.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/sermouse.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/rmi4 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio 2024-02-28T22:42:04.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/altera_ps2.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/arc_ps2.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/serio/serio_raw.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/input/sparse-keymap.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-blinkm.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-lm3530.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-lp3944.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/leds-ss4200.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-28T22:42:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/bcache 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/bcache/bcache.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-bio-prison.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-cache-smq.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-cache.ko 2024-02-28T22:42:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-crypt.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-delay.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-flakey.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-log-userspace.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-multipath.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-queue-length.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-raid.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-round-robin.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-service-time.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-switch.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-thin-pool.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/dm-verity.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/persistent-data 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid0.ko 2024-02-28T22:42:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid1.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid10.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/md/raid456.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptbase.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptctl.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptfc.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptlan.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptsas.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptscsih.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/message/fusion/mptspi.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/lpc_ich.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/lpc_sch.ko 2024-02-28T22:42:04.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/mfd-core.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/sm501.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/viperboard.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/vx855.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/mfd/wl1273-core.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/altera-stapl 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/apds9802als.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/apds990x.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/bh1770glc.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/cb710 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/cb710/cb710.ko 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom 2024-02-28T22:42:04.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/at24.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/eeprom/max6875.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/enclosure.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/hpilo.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/isl29003.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/isl29020.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei-me.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei-txe.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/mei/mei.ko 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-gru 2024-02-28T22:42:04.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tifm_7xx1.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tifm_core.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/tsl2550.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_balloon.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_vmci 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/bonding 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/bonding/bonding.ko 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/b53 2024-02-28T22:42:04.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6xxx 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/dummy.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/eql.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-28T22:42:04.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/adaptec 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/agere 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/alteon 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/altera 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-28T22:42:04.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/alx 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-28T22:42:04.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/genet 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade/bna 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-28T22:42:04.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco/enic 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink 2024-02-28T22:42:04.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/dnet.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex/benet 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ethoc.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/fealnx.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000e 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-28T22:42:04.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/fm10k 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/i40e 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/iavf 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igb 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igbvf 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-28T22:42:04.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/jme.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/micrel 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-28T22:42:04.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/neterion 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/nvidia 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-28T22:42:04.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rdc 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rocker 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-28T22:42:04.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sfc 2024-02-28T22:42:04.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-28T22:42:04.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/silan 2024-02-28T22:42:04.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-28T22:42:04.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis 2024-02-28T22:42:04.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-28T22:42:04.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/tehuti 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ti 2024-02-28T22:42:04.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/xircom 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/geneve.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/hyperv 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ieee802154 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-28T22:42:04.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ifb.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ipvlan 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/macvlan.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/macvtap.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/mii.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/net_failover.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/netconsole.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/nlmon.ko 2024-02-28T22:42:04.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/amd.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/bcm87xx.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/broadcom.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/cicada.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/davicom.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/dp83640.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/et1011c.ko 2024-02-28T22:42:04.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/icplus.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/lxt.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/marvell.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/micrel.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/national.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/phylink.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/qsemi.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/realtek.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/smsc.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/ste10Xp.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/phy/vitesse.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_async.ko 2024-02-28T22:42:04.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pppoe.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pppox.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/ppp/pptp.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip/slhc.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/slip/slip.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/sungem_phy.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/tap.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-28T22:42:04.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_random.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/tun.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/veth.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/virtio_net.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vmxnet3 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vxlan 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/vxlan/vxlan.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netback 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/net/xen-netfront.ko 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ntb 2024-02-28T22:42:04.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ntb/ntb.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host/nvme-core.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/nvme/host/nvme.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_cs.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_pc.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/parport/parport_serial.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/hotplug 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/pcie 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-28T22:42:04.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pci/xen-pcifront.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/i82092.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/pd6729.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl/intel 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/acerhdf.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-28T22:42:04.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/hdaps.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/intel_ips.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-28T22:42:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/platform/x86/wmi.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/clients/pps_parport.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/pps/pps_core.ko 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp 2024-02-28T22:42:04.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp/ptp.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ptp/ptp_kvm.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-em3027.ko 2024-02-28T22:42:04.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-max6900.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-28T22:42:04.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/rtc/rtc-x1205.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-9xxx.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-sas.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/3w-xxxx.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/BusLogic.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/a100u2w.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aacraid 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/advansys.ko 2024-02-28T22:42:04.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/am53c974.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/arcmsr 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/atp870u.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/be2iscsi 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bfa 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bfa/bfa.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2fc 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2i 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ch.ko 2024-02-28T22:42:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/csiostor 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/dc395x.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/dmx3191d.ko 2024-02-28T22:42:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esas2r 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/esp_scsi.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fnic 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/fnic/fnic.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hpsa.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hptiop.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/hv_storvsc.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/initio.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ipr.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ips.ko 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/isci 2024-02-28T22:42:04.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/isci/isci.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libfc 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libfc/libfc.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libiscsi.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libsas 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/libsas/libsas.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/lpfc 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-28T22:42:04.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mpt3sas 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvsas 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/mvumi.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pm8001 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/pmcraid.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla1280.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla4xxx 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-28T22:42:04.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/raid_class.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_debug.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/ses.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/st.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/stex.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/sym53c8xx_2 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/virtio_scsi.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-28T22:42:04.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/scsi/wd719x.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ssb 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ssb/ssb.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/iscsi 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/loopback 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_file.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_iblock.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_mod.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_pscsi.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/target_core_user.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/tcm_fc 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-28T22:42:04.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-28T22:42:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/ipwireless 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/n_gsm.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/n_hdlc.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/nozomi.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/8250 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-28T22:42:04.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/arc_uart.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/jsm 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/core 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/host 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_aec.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_cif.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_pci_generic.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/uio/uio_sercos3.ko 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb 2024-02-28T22:42:04.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/cdc-acm.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/usblp.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/class/usbtmc.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/host 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image/mdc800.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/image/microtek.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/adutux.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/appledisplay.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/emi26.ko 2024-02-28T22:42:04.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/emi62.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/ezusb.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/idmouse.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/iowarrior.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/ldusb.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/legousbtower.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/sisusbvga 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usb3503.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usblcd.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/uss720.ko 2024-02-28T22:42:04.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/misc/yurex.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/phy 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/phy/phy-generic.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/aircable.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ark3116.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ch341.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cp210x.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cyberjack.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/empeg.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-28T22:42:04.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/io_ti.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ipaq.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ipw.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ir-usb.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/keyspan.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mct_u232.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mos7720.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/mos7840.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/navman.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/omninet.ko 2024-02-28T22:42:04.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/opticon.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/option.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/oti6858.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/pl2303.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/qcaux.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/qcserial.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/quatech2.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/safe_serial.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/sierra.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ssu100.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/symbolserial.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-28T22:42:04.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb_debug.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/visor.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/whiteheat.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/uas.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-karma.ko 2024-02-28T22:42:04.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/storage/usb-storage.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-28T22:42:04.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/vfio.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_net.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/vhost/vhost_scsi.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/apple_bl.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/lcd.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_balloon.ko 2024-02-28T22:42:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_mmio.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/virtio/virtio_ring.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters/ds2482.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/masters/ds2490.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-28T22:42:04.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/w1/wire.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/hpwdt.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/i6300esb.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-28T22:42:04.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ib700wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ibmasr.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/it87_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/machzwd.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/nv_tco.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/softdog.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-28T22:42:04.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/via_wdt.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/wdt_pci.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/watchdog/xen_wdt.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-evtchn.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-gntalloc.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-gntdev.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-pciback 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xen-privcmd.ko 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xenfs 2024-02-28T22:42:04.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/9p 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/9p/9p.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/affs 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/affs/affs.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/befs 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/befs/befs.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/binfmt_misc.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/btrfs 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/btrfs/btrfs.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cachefiles 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cachefiles/cachefiles.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ceph 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ceph/ceph.ko 2024-02-28T22:42:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/coda 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/coda/coda.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cramfs 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/cramfs/cramfs.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/dlm 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/dlm/dlm.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ecryptfs 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/f2fs 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/f2fs/f2fs.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/fat.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/msdos.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fat/vfat.ko 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse 2024-02-28T22:42:04.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse/cuse.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/fuse/fuse.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/gfs2 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/gfs2/gfs2.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfs 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfs/hfs.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfsplus 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/hfsplus/hfsplus.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/isofs 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/isofs/isofs.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/jfs 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/jfs/jfs.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/lockd 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/lockd/lockd.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/minix 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/minix/minix.ko 2024-02-28T22:42:04.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/netfs 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/netfs/netfs.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/blocklayout 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/filelayout 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/flexfilelayout 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfs.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfsv3.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs/nfsv4.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common/grace.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfs_common/nfs_acl.ko 2024-02-28T22:42:04.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfsd 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nfsd/nfsd.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nilfs2 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nilfs2/nilfs2.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-celtic.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-centeuro.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-croatian.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-cyrillic.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-gaelic.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-greek.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-iceland.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-inuit.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-roman.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-romanian.ko 2024-02-28T22:42:04.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/mac-turkish.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1250.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1251.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp1255.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp737.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp775.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp850.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp852.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp855.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp857.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp860.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp861.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp862.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp863.ko 2024-02-28T22:42:04.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp864.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp865.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp866.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp869.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp874.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp932.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp936.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp949.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_cp950.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_euc-jp.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-1.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-13.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-14.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-15.ko 2024-02-28T22:42:04.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-2.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-3.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-4.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-5.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-6.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-7.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_iso8859-9.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-r.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-ru.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_koi8-u.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/nls/nls_utf8.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/cluster 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-28T22:42:04.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlm 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlmfs 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/overlayfs 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/overlayfs/overlay.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/pstore 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/pstore/ramoops.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/reiserfs 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/reiserfs/reiserfs.ko 2024-02-28T22:42:04.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/romfs 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/romfs/romfs.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/client 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/client/cifs.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common/cifs_arc4.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/smb/common/cifs_md4.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/squashfs 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/squashfs/squashfs.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/sysv 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/sysv/sysv.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/udf 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/udf/udf.ko 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ufs 2024-02-28T22:42:04.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/ufs/ufs.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/xfs 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/fs/xfs/xfs.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/rcu 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/rcu/rcutorture.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/torture.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/trace 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/asn1_encoder.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crc-itu-t.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crc8.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto/libarc4.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/crypto/libdes.ko 2024-02-28T22:42:04.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/libcrc32c.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lru_cache.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4/lz4_compress.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/lz4/lz4hc_compress.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/math 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/math/cordic.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/raid6 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/raid6/raid6_pq.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/reed_solomon 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_bm.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_fsm.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/ts_kmp.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/zstd 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/lib/zstd/zstd_compress.ko 2024-02-28T22:42:04.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/mm 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/mm/hwpoison-inject.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/6lowpan.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_dest.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_fragment.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_hop.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_mobility.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_routing.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/6lowpan/nhc_udp.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/garp.ko 2024-02-28T22:42:05.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/mrp.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/p8022.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/psnap.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/802/stp.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/8021q 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/8021q/8021q.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_fd.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_rdma.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/9p/9pnet_virtio.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/batman-adv 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/batman-adv/batman-adv.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/br_netfilter.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/bridge.ko 2024-02-28T22:42:05.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-28T22:42:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/ebtables.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ceph 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ceph/libceph.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core/failover.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/core/pktgen.ko 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp 2024-02-28T22:42:05.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_diag.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_ipv4.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dccp/dccp_ipv6.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dns_resolver 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dns_resolver/dns_resolver.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/dsa_core.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_brcm.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_dsa.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_none.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/dsa/tag_trailer.ko 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/6lowpan 2024-02-28T22:42:05.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/ieee802154.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ah4.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/esp4.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/fou.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/gre.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/inet_diag.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_gre.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_tunnel.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ip_vti.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ipcomp.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/ipip.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-28T22:42:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-28T22:42:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_bic.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_dctcp.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_diag.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_highspeed.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_htcp.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_hybla.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_illinois.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_lp.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_scalable.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_vegas.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_veno.ko 2024-02-28T22:42:05.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_westwood.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tcp_yeah.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/tunnel4.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/udp_diag.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/udp_tunnel.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ah6.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/esp6.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/fou6.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_tunnel.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ip6_vti.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/ipcomp6.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-28T22:42:05.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-28T22:42:05.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/sit.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/tunnel6.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/key 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/key/af_key.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_core.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_eth.ko 2024-02-28T22:42:05.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ip.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ip6.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_netlink.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/l2tp/l2tp_ppp.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/llc 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/llc/llc.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mac802154 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mac802154/mac802154.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mpls 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/mpls/mpls_gso.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-28T22:42:05.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs 2024-02-28T22:42:05.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conncount.ko 2024-02-28T22:42:05.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-28T22:42:05.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_flow_table.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_log_syslog.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_irc.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_sip.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nf_tables.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_log.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-28T22:42:05.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_chain_nat.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_compat.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_ct.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_fib.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_hash.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_limit.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_log.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_masq.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_nat.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_numgen.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_queue.ko 2024-02-28T22:42:05.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_quota.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_redir.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_reject.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_reject_inet.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_socket.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/nft_tunnel.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_AUDIT.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_CT.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_DSCP.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_HL.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_HMARK.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-28T22:42:05.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_LED.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_LOG.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NETMAP.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NFLOG.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_RATEEST.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_SECMARK.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TEE.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TPROXY.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_TRACE.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_addrtype.ko 2024-02-28T22:42:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_bpf.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cgroup.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cluster.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_comment.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connbytes.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connlabel.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connlimit.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_connmark.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_conntrack.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_cpu.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_dccp.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_devgroup.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_dscp.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ecn.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_esp.ko 2024-02-28T22:42:05.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_hashlimit.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_helper.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_hl.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ipcomp.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_iprange.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_ipvs.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_l2tp.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_length.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_limit.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_mac.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_mark.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_multiport.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_nat.ko 2024-02-28T22:42:05.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_nfacct.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_osf.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_owner.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_physdev.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_pkttype.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_policy.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_quota.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_rateest.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_realm.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_recent.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_sctp.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_set.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_socket.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_state.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_statistic.ko 2024-02-28T22:42:05.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_string.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_tcpmss.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_time.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netfilter/xt_u32.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netlink 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/netlink/netlink_diag.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/nsh 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/nsh/nsh.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/openvswitch.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-geneve.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-gre.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/openvswitch/vport-vxlan.ko 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/packet 2024-02-28T22:42:05.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/packet/af_packet_diag.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds_rdma.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/rds/rds_tcp.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_csum.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_gact.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_mirred.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_nat.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_pedit.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_police.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_simple.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_skbedit.ko 2024-02-28T22:42:05.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/act_vlan.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_basic.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_bpf.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_flow.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_fw.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_route.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/cls_u32.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_cmp.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_ipset.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_meta.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_nbyte.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_text.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/em_u32.ko 2024-02-28T22:42:05.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_choke.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_codel.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_drr.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_fq.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_gred.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_hfsc.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_hhf.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_htb.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_ingress.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_mqprio.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_mqprio_lib.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_multiq.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_netem.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_pie.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_plug.ko 2024-02-28T22:42:05.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_prio.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_qfq.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_red.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_sfb.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_sfq.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_tbf.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sched/sch_teql.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp/sctp.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sctp/sctp_diag.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-28T22:42:05.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/sunrpc.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/xprtrdma 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc/diag.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/tipc/tipc.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/unix 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/unix/unix_diag.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/xfrm 2024-02-28T22:42:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/encrypted-keys 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/trusted-keys 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/security/keys/trusted-keys/trusted.ko 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt/lib 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/kernel/virt/lib/irqbypass.ko 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.alias 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.alias.bin 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.alias.bin 2024-02-28T22:42:05.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.bin 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.builtin.modinfo 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.dep 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.dep.bin 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.devname 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.order 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.softdep 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.symbols 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/modules.symbols.bin 2024-02-28T22:42:05.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-rc6-gbd7d00fb7248/vmlinuz 2024-02-28T22:42:05.033 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:05.033 DEBUG:teuthology.orchestra.run.smithi084:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout:postinstall scriptlet (using /bin/sh): 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout: /usr/bin/kernel-install add 6.8.0-rc6-gbd7d00fb7248 /lib/modules/6.8.0-rc6-gbd7d00fb7248/vmlinuz 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout:for file in vmlinuz System.map config; do 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc6-gbd7d00fb7248/${file}" "/boot/${file}-6.8.0-rc6-gbd7d00fb7248"; then 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout: cp "/lib/modules/6.8.0-rc6-gbd7d00fb7248/${file}" "/boot/${file}-6.8.0-rc6-gbd7d00fb7248" 2024-02-28T22:42:05.055 INFO:teuthology.orchestra.run.smithi084.stdout: fi 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:done 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:new-kernel-pkg --remove 6.8.0-rc6-gbd7d00fb7248 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc6-gbd7d00fb7248.img 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-install remove 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/update-bootloader --remove 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:05.056 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-02-28T22:42:05.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2024-02-28T22:42:05.143 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2024-02-28T22:42:05.143 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2024-02-28T22:42:05.143 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2024-02-28T22:42:05.143 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 68.el9 2024-02-28T22:42:05.143 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8264254 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2024-02-28T22:42:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:42:05.145 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2024-02-28T22:42:05.146 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-28T22:42:05.146 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:05.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-28T22:42:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-02-28T22:42:07.087 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-28T22:42:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-02-28T22:42:07.133 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-02-28T22:42:07.147 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.nPF0lm2Dn5 2024-02-28T22:42:07.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nPF0lm2Dn5 2024-02-28T22:42:07.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.nPF0lm2Dn5 2024-02-28T22:42:07.349 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.nPF0lm2Dn5 is 6KB 2024-02-28T22:42:07.397 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.nPF0lm2Dn5 2024-02-28T22:42:07.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2024-02-28T22:42:07.476 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-28T22:42:07.476 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-28T22:42:07.476 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-28T22:42:07.476 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc6-gbd7d00fb7248.conf 2024-02-28T22:42:07.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:42:07.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-02-28T22:42:12.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:42:12.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:42:37.655 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-28T22:42:37.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:42:37.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:43:12.240 DEBUG:teuthology.orchestra.remote:timed out 2024-02-28T22:43:24.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:43:24.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:43:24.743 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-02-28T22:43:25.450 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:43:25.450 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-rc6-gbd7d00fb7248"... 2024-02-28T22:43:25.450 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-02-28T22:43:25.507 INFO:teuthology.orchestra.run.smithi057.stdout:6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:43:25.507 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc6-gbd7d00fb7248 vs 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:43:25.507 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-28T22:43:25.507 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-28T22:43:26.508 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-28T22:43:26.508 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-28T22:43:26.664 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2024-02-28T22:43:26.727 DEBUG:teuthology.parallel:result is None 2024-02-28T22:43:37.659 DEBUG:teuthology.orchestra.remote:timed out 2024-02-28T22:43:46.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:43:46.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:43:49.724 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-02-28T22:44:01.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:44:01.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:44:02.235 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-02-28T22:44:02.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:44:02.912 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-rc6-gbd7d00fb7248"... 2024-02-28T22:44:02.912 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-02-28T22:44:02.952 INFO:teuthology.orchestra.run.smithi084.stdout:6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:44:02.953 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc6-gbd7d00fb7248 vs 6.8.0-rc6-gbd7d00fb7248 2024-02-28T22:44:02.953 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-28T22:44:02.953 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-28T22:44:03.954 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-28T22:44:03.954 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-28T22:44:04.070 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2024-02-28T22:44:04.112 DEBUG:teuthology.parallel:result is None 2024-02-28T22:44:04.113 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-28T22:44:04.120 INFO:teuthology.task.internal:Creating test directory... 2024-02-28T22:44:04.120 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-28T22:44:04.128 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-28T22:44:04.178 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-28T22:44:04.185 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-28T22:44:04.207 INFO:teuthology.task.internal:Creating archive directory... 2024-02-28T22:44:04.208 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-28T22:44:04.215 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-28T22:44:04.281 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-28T22:44:04.288 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-28T22:44:04.288 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-28T22:44:04.296 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-28T22:44:04.367 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-28T22:44:04.404 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-28T22:44:04.408 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-28T22:44:04.454 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-28T22:44:04.460 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-28T22:44:04.472 INFO:teuthology.task.internal:Configuring sudo... 2024-02-28T22:44:04.472 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-28T22:44:04.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-28T22:44:04.574 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-28T22:44:04.583 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-28T22:44:04.583 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-28T22:44:04.591 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-28T22:44:04.641 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-28T22:44:04.728 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-28T22:44:04.849 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-28T22:44:04.939 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-28T22:44:05.042 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:44:05.042 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-28T22:44:05.137 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-28T22:44:05.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-28T22:44:05.318 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-28T22:44:05.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-28T22:44:05.480 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:44:05.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-28T22:44:05.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2024-02-28T22:44:05.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-02-28T22:44:05.698 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-28T22:44:05.804 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-28T22:44:06.147 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-28T22:44:06.155 INFO:teuthology.task.internal:Starting timer... 2024-02-28T22:44:06.155 INFO:teuthology.run_tasks:Running task pcp... 2024-02-28T22:44:06.179 INFO:teuthology.run_tasks:Running task selinux... 2024-02-28T22:44:06.211 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-28T22:44:06.211 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2024-02-28T22:44:06.353 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2024-02-28T22:44:06.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-02-28T22:44:06.505 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-02-28T22:44:06.512 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-28T22:44:06.513 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2024-02-28T22:44:06.571 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2024-02-28T22:44:06.571 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-02-28T22:44:06.660 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-02-28T22:44:06.660 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive'} 2024-02-28T22:44:06.661 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-28T22:44:06.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T22:44:06.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-28T22:44:06.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T22:44:06.902 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-28T22:44:06.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2024-02-28T22:44:06.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-02-28T22:44:07.140 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-28T22:44:07.150 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-28T22:44:07.167 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-28T22:44:07.168 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-28T22:50:27.412 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com')] 2024-02-28T22:50:27.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:50:27.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:50:27.569 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-02-28T22:50:27.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-02-28T22:50:27.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:50:27.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:50:27.868 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-02-28T22:50:28.014 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-02-28T22:50:28.014 INFO:teuthology.run_tasks:Running task clock... 2024-02-28T22:50:28.027 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-28T22:50:28.027 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-28T22:50:28.028 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-28T22:50:28.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-28T22:50:28.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-28T22:50:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-28T22:50:28.138 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-28T22:50:28.185 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-28T22:50:28.191 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-28T22:50:28.275 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-02-28T22:50:28.292 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2024-02-28T22:50:28.313 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-02-28T22:50:28.337 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2024-02-28T22:50:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-28T22:50:28.391 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-28T22:50:28.415 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-28T22:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-28T22:50:28.546 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-02-28T22:50:28.555 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-28T22:50:28.555 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-02-28T22:50:28.555 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.555 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.555 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.556 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.581 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-28T22:50:28.591 INFO:teuthology.run_tasks:Running task install... 2024-02-28T22:50:28.600 DEBUG:teuthology.task.install:project ceph 2024-02-28T22:50:28.600 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}} 2024-02-28T22:50:28.600 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'} 2024-02-28T22:50:28.600 INFO:teuthology.task.install:Using flavor: default 2024-02-28T22:50:28.608 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-28T22:50:28.608 INFO:teuthology.task.install:extra packages: [] 2024-02-28T22:50:28.609 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-28T22:50:28.609 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:50:28.609 INFO:teuthology.packaging:ref: None 2024-02-28T22:50:28.609 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:50:28.609 INFO:teuthology.packaging:branch: None 2024-02-28T22:50:28.609 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:50:29.277 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> None 2024-02-28T22:50:29.277 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-28T22:50:31.537 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-28T22:50:31.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-28T22:50:31.540 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-28T22:50:31.540 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:50:31.540 INFO:teuthology.packaging:ref: None 2024-02-28T22:50:31.540 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:50:31.540 INFO:teuthology.packaging:branch: None 2024-02-28T22:50:31.540 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:50:31.541 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-28T22:50:31.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-28T22:50:31.688 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-28T22:50:31.688 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-28T22:50:31.721 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-28T22:50:31.721 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:50:31.721 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-28T22:50:31.830 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-28T22:50:31.830 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:50:31.830 INFO:teuthology.packaging:ref: None 2024-02-28T22:50:31.830 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:50:31.830 INFO:teuthology.packaging:branch: None 2024-02-28T22:50:31.830 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:50:31.830 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-28T22:50:31.905 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-28T22:50:31.906 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-28T22:50:31.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-28T22:50:31.931 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:50:31.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-28T22:50:31.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-28T22:50:32.032 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-28T22:50:32.032 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:50:32.032 INFO:teuthology.packaging:ref: None 2024-02-28T22:50:32.032 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:50:32.032 INFO:teuthology.packaging:branch: None 2024-02-28T22:50:32.032 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:50:32.032 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-28T22:50:32.105 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-28T22:50:32.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-28T22:50:32.202 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2024-02-28T22:50:32.207 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2024-02-28T22:50:32.302 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-28T22:50:32.393 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-02-28T22:50:32.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-02-28T22:50:32.602 INFO:teuthology.orchestra.run.smithi057.stdout:74 files removed 2024-02-28T22:50:32.645 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw 2024-02-28T22:50:32.743 INFO:teuthology.orchestra.run.smithi084.stdout:74 files removed 2024-02-28T22:50:32.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw 2024-02-28T22:50:33.345 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 465 kB/s | 76 kB 00:00 2024-02-28T22:50:33.447 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 480 kB/s | 76 kB 00:00 2024-02-28T22:50:33.553 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 58 kB/s | 9.3 kB 00:00 2024-02-28T22:50:33.648 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 61 kB/s | 9.3 kB 00:00 2024-02-28T22:50:33.753 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 13 kB/s | 2.2 kB 00:00 2024-02-28T22:50:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 14 kB/s | 2.2 kB 00:00 2024-02-28T22:50:34.467 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-02-28T22:50:35.037 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 7.0 MB/s | 8.1 MB 00:01 2024-02-28T22:50:37.163 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.5 MB 00:00 2024-02-28T22:50:37.285 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - BaseOS 2.0 MB/s | 2.5 MB 00:01 2024-02-28T22:50:39.547 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-02-28T22:50:39.716 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:02 2024-02-28T22:50:41.569 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-02-28T22:50:43.884 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-02-28T22:50:50.634 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-02-28T22:50:52.393 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-02-28T22:50:54.440 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 710 kB/s | 6.1 MB 00:08 2024-02-28T22:50:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-02-28T22:50:58.651 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 20 MB 00:02 2024-02-28T22:51:04.710 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-02-28T22:51:05.325 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 1.6 MB/s | 20 MB 00:12 2024-02-28T22:51:07.457 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:51:07.457 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:51:07.457 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:51:07.457 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.458 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.459 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:51:07.460 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:51:07.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.465 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.466 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:07.466 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.466 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.466 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:07.466 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:08.168 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:51:08.179 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:08.179 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-28T22:51:08.180 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-28T22:51:08.181 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:51:08.181 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-28T22:51:08.181 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-28T22:51:08.181 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-28T22:51:08.181 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-28T22:51:08.182 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-28T22:51:08.183 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-28T22:51:08.184 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-28T22:51:08.184 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-28T22:51:08.184 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-28T22:51:08.184 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-28T22:51:08.184 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-28T22:51:08.185 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-28T22:51:08.185 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-28T22:51:08.185 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-28T22:51:08.185 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:08.186 INFO:teuthology.orchestra.run.smithi057.stdout:Install 28 Packages 2024-02-28T22:51:08.187 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 2 Packages 2024-02-28T22:51:08.187 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:08.188 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 53 M 2024-02-28T22:51:08.188 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:51:09.036 INFO:teuthology.orchestra.run.smithi057.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 15 MB/s | 5.2 MB 00:00 2024-02-28T22:51:09.061 INFO:teuthology.orchestra.run.smithi057.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 987 kB/s | 24 kB 00:00 2024-02-28T22:51:09.112 INFO:teuthology.orchestra.run.smithi057.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 13 MB/s | 653 kB 00:00 2024-02-28T22:51:09.162 INFO:teuthology.orchestra.run.smithi057.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 9.4 MB/s | 474 kB 00:00 2024-02-28T22:51:09.312 INFO:teuthology.orchestra.run.smithi057.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 12 MB/s | 7.6 MB 00:00 2024-02-28T22:51:09.337 INFO:teuthology.orchestra.run.smithi057.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-02-28T22:51:09.371 INFO:teuthology.orchestra.run.smithi057.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 3.6 MB/s | 119 kB 00:00 2024-02-28T22:51:09.405 INFO:teuthology.orchestra.run.smithi057.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 155 kB 00:00 2024-02-28T22:51:09.447 INFO:teuthology.orchestra.run.smithi057.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-02-28T22:51:09.489 INFO:teuthology.orchestra.run.smithi057.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-02-28T22:51:09.514 INFO:teuthology.orchestra.run.smithi057.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 4.0 MB/s | 99 kB 00:00 2024-02-28T22:51:09.639 INFO:teuthology.orchestra.run.smithi057.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-02-28T22:51:09.690 INFO:teuthology.orchestra.run.smithi057.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 613 kB/s | 107 kB 00:00 2024-02-28T22:51:09.740 INFO:teuthology.orchestra.run.smithi057.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 922 kB/s | 45 kB 00:00 2024-02-28T22:51:09.790 INFO:teuthology.orchestra.run.smithi057.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-02-28T22:51:09.874 INFO:teuthology.orchestra.run.smithi057.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-02-28T22:51:09.899 INFO:teuthology.orchestra.run.smithi057.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 292 kB 00:00 2024-02-28T22:51:10.133 INFO:teuthology.orchestra.run.smithi057.stdout:(18/30): ceph-common-18.2.0-0.el9.x86_64.rpm 13 MB/s | 18 MB 00:01 2024-02-28T22:51:10.225 INFO:teuthology.orchestra.run.smithi057.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 121 kB/s | 42 kB 00:00 2024-02-28T22:51:10.592 INFO:teuthology.orchestra.run.smithi057.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 160 kB/s | 111 kB 00:00 2024-02-28T22:51:10.676 INFO:teuthology.orchestra.run.smithi057.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 315 kB/s | 26 kB 00:00 2024-02-28T22:51:10.709 INFO:teuthology.orchestra.run.smithi057.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 615 kB/s | 309 kB 00:00 2024-02-28T22:51:10.768 INFO:teuthology.orchestra.run.smithi057.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 534 kB/s | 49 kB 00:00 2024-02-28T22:51:10.793 INFO:teuthology.orchestra.run.smithi057.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 814 kB/s | 67 kB 00:00 2024-02-28T22:51:10.893 INFO:teuthology.orchestra.run.smithi057.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-28T22:51:11.002 INFO:teuthology.orchestra.run.smithi057.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-02-28T22:51:11.053 INFO:teuthology.orchestra.run.smithi057.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-02-28T22:51:11.075 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-02-28T22:51:11.170 INFO:teuthology.orchestra.run.smithi057.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-02-28T22:51:11.245 INFO:teuthology.orchestra.run.smithi057.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 14 MB/s | 3.3 MB 00:00 2024-02-28T22:51:11.312 INFO:teuthology.orchestra.run.smithi057.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-02-28T22:51:11.322 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:11.323 INFO:teuthology.orchestra.run.smithi057.stdout:Total 17 MB/s | 53 MB 00:03 2024-02-28T22:51:11.560 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:51:11.648 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:51:11.648 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:51:12.172 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:51:12.172 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.676 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:51:13.677 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:51:13.678 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:51:13.679 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.680 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.681 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.682 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.683 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.684 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.685 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.685 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:13.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:13.881 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:51:13.997 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-28T22:51:14.037 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-28T22:51:14.143 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-28T22:51:14.332 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:51:14.352 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:14.352 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:51:14.352 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-28T22:51:14.353 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-28T22:51:14.354 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout:Install 28 Packages 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 2 Packages 2024-02-28T22:51:14.355 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:14.356 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 53 M 2024-02-28T22:51:14.356 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:51:14.650 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-28T22:51:14.699 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:14.826 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:14.933 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-28T22:51:14.959 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:15.027 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:15.115 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-28T22:51:15.209 INFO:teuthology.orchestra.run.smithi084.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 17 MB/s | 5.2 MB 00:00 2024-02-28T22:51:15.234 INFO:teuthology.orchestra.run.smithi084.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 974 kB/s | 24 kB 00:00 2024-02-28T22:51:15.276 INFO:teuthology.orchestra.run.smithi084.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 16 MB/s | 653 kB 00:00 2024-02-28T22:51:15.319 INFO:teuthology.orchestra.run.smithi084.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-02-28T22:51:15.324 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-28T22:51:15.468 INFO:teuthology.orchestra.run.smithi084.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 13 MB/s | 7.6 MB 00:00 2024-02-28T22:51:15.475 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-28T22:51:15.493 INFO:teuthology.orchestra.run.smithi084.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-02-28T22:51:15.504 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:15.527 INFO:teuthology.orchestra.run.smithi084.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 3.6 MB/s | 119 kB 00:00 2024-02-28T22:51:15.561 INFO:teuthology.orchestra.run.smithi084.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 155 kB 00:00 2024-02-28T22:51:15.603 INFO:teuthology.orchestra.run.smithi084.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-02-28T22:51:15.608 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:15.645 INFO:teuthology.orchestra.run.smithi084.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-02-28T22:51:15.680 INFO:teuthology.orchestra.run.smithi084.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-02-28T22:51:15.795 INFO:teuthology.orchestra.run.smithi084.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-02-28T22:51:15.799 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-28T22:51:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-28T22:51:15.884 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-28T22:51:15.922 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-28T22:51:15.962 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-28T22:51:16.015 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-28T22:51:16.061 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-28T22:51:16.070 INFO:teuthology.orchestra.run.smithi084.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 16 MB/s | 18 MB 00:01 2024-02-28T22:51:16.221 INFO:teuthology.orchestra.run.smithi084.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 197 kB/s | 107 kB 00:00 2024-02-28T22:51:16.363 INFO:teuthology.orchestra.run.smithi084.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 283 kB/s | 160 kB 00:00 2024-02-28T22:51:16.413 INFO:teuthology.orchestra.run.smithi084.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 133 kB/s | 45 kB 00:00 2024-02-28T22:51:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-28T22:51:16.494 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-28T22:51:16.600 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-28T22:51:16.634 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-28T22:51:16.703 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-28T22:51:16.755 INFO:teuthology.orchestra.run.smithi084.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 124 kB/s | 42 kB 00:00 2024-02-28T22:51:16.930 INFO:teuthology.orchestra.run.smithi084.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 636 kB/s | 111 kB 00:00 2024-02-28T22:51:17.013 INFO:teuthology.orchestra.run.smithi084.stdout:(19/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.6 MB/s | 309 kB 00:00 2024-02-28T22:51:17.039 INFO:teuthology.orchestra.run.smithi084.stdout:(20/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 433 kB/s | 292 kB 00:00 2024-02-28T22:51:17.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-28T22:51:17.197 INFO:teuthology.orchestra.run.smithi084.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-02-28T22:51:17.213 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-28T22:51:17.305 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-28T22:51:17.331 INFO:teuthology.orchestra.run.smithi084.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2024-02-28T22:51:17.397 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-28T22:51:17.433 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-28T22:51:17.439 INFO:teuthology.orchestra.run.smithi084.stdout:(23/30): librdkafka-1.6.1-102.el9.x86_64.rpm 544 kB/s | 662 kB 00:01 2024-02-28T22:51:17.465 INFO:teuthology.orchestra.run.smithi084.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 507 kB/s | 67 kB 00:00 2024-02-28T22:51:17.590 INFO:teuthology.orchestra.run.smithi084.stdout:(25/30): libarrow-9.0.0-10.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-02-28T22:51:17.715 INFO:teuthology.orchestra.run.smithi084.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-02-28T22:51:17.741 INFO:teuthology.orchestra.run.smithi084.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 697 kB/s | 192 kB 00:00 2024-02-28T22:51:17.824 INFO:teuthology.orchestra.run.smithi084.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-02-28T22:51:17.933 INFO:teuthology.orchestra.run.smithi084.stdout:(29/30): librbd1-18.2.0-0.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-02-28T22:51:18.008 INFO:teuthology.orchestra.run.smithi084.stdout:(30/30): librados2-18.2.0-0.el9.x86_64.rpm 11 MB/s | 3.3 MB 00:00 2024-02-28T22:51:18.013 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:18.014 INFO:teuthology.orchestra.run.smithi084.stdout:Total 14 MB/s | 53 MB 00:03 2024-02-28T22:51:18.242 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:51:18.321 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:51:18.321 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:51:18.839 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:51:18.840 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:51:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:20.022 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:20.359 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:51:20.466 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-28T22:51:20.499 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-28T22:51:20.592 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-28T22:51:20.692 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:20.784 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-28T22:51:20.962 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-28T22:51:20.963 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-28T22:51:20.963 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-28T22:51:20.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:20.973 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-28T22:51:21.012 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:21.018 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:21.119 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:21.213 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-28T22:51:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:21.329 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:21.372 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-28T22:51:21.446 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-28T22:51:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-28T22:51:21.574 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:21.667 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:21.851 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-28T22:51:21.884 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-28T22:51:21.935 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-28T22:51:22.004 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-28T22:51:22.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-28T22:51:22.101 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-28T22:51:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-28T22:51:22.546 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-28T22:51:22.579 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-28T22:51:22.693 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-28T22:51:22.728 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-28T22:51:22.800 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-28T22:51:23.280 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-28T22:51:23.316 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-28T22:51:23.408 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-28T22:51:23.507 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-28T22:51:23.543 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-28T22:51:25.981 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:26.014 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:26.669 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-28T22:51:26.709 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-28T22:51:26.853 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-28T22:51:26.853 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-28T22:51:26.853 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-28T22:51:26.853 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:26.902 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:32.637 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /root 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2024-02-28T22:51:32.638 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:33.530 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-28T22:51:33.601 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:33.607 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-28T22:51:33.760 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-28T22:51:33.760 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-28T22:51:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-28T22:51:35.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-28T22:51:35.374 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-28T22:51:36.500 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:36.500 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:36.500 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-28T22:51:36.501 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:36.503 INFO:teuthology.orchestra.run.smithi057.stdout: re2-1:20211101-3.el9.x86_64 2024-02-28T22:51:36.503 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-28T22:51:36.503 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:36.503 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:51:36.972 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-test 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-02-28T22:51:37.684 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:37.857 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:33 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:51:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:51:38.185 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:51:38.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:51:38.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:51:38.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.189 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.190 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.191 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:38.192 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.193 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:38.194 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.194 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.194 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.194 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:38.286 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-28T22:51:38.287 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout:Install 6 Packages 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:38.288 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 40 M 2024-02-28T22:51:38.289 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 167 M 2024-02-28T22:51:38.289 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:51:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-28T22:51:38.616 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-28T22:51:38.616 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:51:38.616 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-28T22:51:38.616 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-28T22:51:38.616 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-28T22:51:38.617 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:38.621 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-28T22:51:38.679 INFO:teuthology.orchestra.run.smithi057.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.0 MB/s | 161 kB 00:00 2024-02-28T22:51:38.740 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-28T22:51:38.740 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:39.338 INFO:teuthology.orchestra.run.smithi057.stdout:(2/6): ceph-test-18.2.0-0.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2024-02-28T22:51:39.746 INFO:teuthology.orchestra.run.smithi057.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 905 kB/s | 305 kB 00:00 2024-02-28T22:51:39.830 INFO:teuthology.orchestra.run.smithi057.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 773 kB/s | 64 kB 00:00 2024-02-28T22:51:39.863 INFO:teuthology.orchestra.run.smithi057.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 140 kB/s | 187 kB 00:01 2024-02-28T22:51:40.172 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:40.172 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-28T22:51:40.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-28T22:51:40.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-28T22:51:40.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-28T22:51:40.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-28T22:51:40.205 INFO:teuthology.orchestra.run.smithi057.stdout:(6/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 143 kB/s | 218 kB 00:01 2024-02-28T22:51:40.209 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:40.210 INFO:teuthology.orchestra.run.smithi057.stdout:Total 21 MB/s | 40 MB 00:01 2024-02-28T22:51:40.367 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:51:40.425 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:51:40.426 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:51:40.842 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:51:40.843 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:51:40.975 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-28T22:51:40.975 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:40.975 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-02-28T22:51:40.975 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.975 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-28T22:51:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-28T22:51:40.977 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2024-02-28T22:51:40.978 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-28T22:51:40.978 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:40.978 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:51:41.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-test 2024-02-28T22:51:41.489 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:51:41.573 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-02-28T22:51:41.632 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-02-28T22:51:41.690 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-02-28T22:51:41.765 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-02-28T22:51:41.793 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-28T22:51:42.246 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:31 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:51:42.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.567 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.568 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.569 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:42.637 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:51:42.638 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:42.638 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:51:42.638 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:42.638 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:51:42.638 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout:Install 6 Packages 2024-02-28T22:51:42.639 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:42.640 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 40 M 2024-02-28T22:51:42.640 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 167 M 2024-02-28T22:51:42.640 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:51:43.524 INFO:teuthology.orchestra.run.smithi084.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.0 MB/s | 161 kB 00:00 2024-02-28T22:51:44.174 INFO:teuthology.orchestra.run.smithi084.stdout:(2/6): ceph-test-18.2.0-0.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2024-02-28T22:51:44.208 INFO:teuthology.orchestra.run.smithi084.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 222 kB/s | 187 kB 00:00 2024-02-28T22:51:44.366 INFO:teuthology.orchestra.run.smithi084.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 259 kB/s | 218 kB 00:00 2024-02-28T22:51:44.616 INFO:teuthology.orchestra.run.smithi084.stdout:(5/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 156 kB/s | 64 kB 00:00 2024-02-28T22:51:44.883 INFO:teuthology.orchestra.run.smithi084.stdout:(6/6): socat-1.7.4.1-5.el9.x86_64.rpm 431 kB/s | 305 kB 00:00 2024-02-28T22:51:44.887 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:44.888 INFO:teuthology.orchestra.run.smithi084.stdout:Total 18 MB/s | 40 MB 00:02 2024-02-28T22:51:45.038 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:51:45.091 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:51:45.092 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:51:45.517 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:51:45.518 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:51:46.103 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-28T22:51:46.165 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:51:46.205 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-28T22:51:46.243 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-02-28T22:51:46.293 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-02-28T22:51:46.333 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-02-28T22:51:46.400 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-02-28T22:51:46.428 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-28T22:51:48.223 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-28T22:51:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-28T22:51:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-28T22:51:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-02-28T22:51:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-02-28T22:51:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-02-28T22:51:48.614 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:48.615 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:51:48.925 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph 2024-02-28T22:51:49.852 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:45 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:51:50.179 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:51:50.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:51:50.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:50.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.187 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.188 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:50.259 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-28T22:51:50.262 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-28T22:51:50.263 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-28T22:51:50.264 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout:Install 33 Packages 2024-02-28T22:51:50.265 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:50.266 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 30 M 2024-02-28T22:51:50.266 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 109 M 2024-02-28T22:51:50.266 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:51:50.546 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-28T22:51:50.640 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-28T22:51:50.826 INFO:teuthology.orchestra.run.smithi057.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 36 kB/s | 6.4 kB 00:00 2024-02-28T22:51:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 5.9 MB/s | 1.4 MB 00:00 2024-02-28T22:51:50.951 INFO:teuthology.orchestra.run.smithi057.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 7.0 MB/s | 2.1 MB 00:00 2024-02-28T22:51:50.985 INFO:teuthology.orchestra.run.smithi057.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.2 MB/s | 240 kB 00:00 2024-02-28T22:51:51.110 INFO:teuthology.orchestra.run.smithi057.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-02-28T22:51:51.319 INFO:teuthology.orchestra.run.smithi057.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 761 kB/s | 253 kB 00:00 2024-02-28T22:51:51.385 INFO:teuthology.orchestra.run.smithi057.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-02-28T22:51:51.619 INFO:teuthology.orchestra.run.smithi057.stdout:(8/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 24 MB/s | 18 MB 00:00 2024-02-28T22:51:51.695 INFO:teuthology.orchestra.run.smithi057.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 2.4 MB/s | 1.2 MB 00:00 2024-02-28T22:51:51.795 INFO:teuthology.orchestra.run.smithi057.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 331 kB/s | 135 kB 00:00 2024-02-28T22:51:51.853 INFO:teuthology.orchestra.run.smithi057.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.4 MB/s | 215 kB 00:00 2024-02-28T22:51:51.912 INFO:teuthology.orchestra.run.smithi057.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 428 kB/s | 125 kB 00:00 2024-02-28T22:51:51.962 INFO:teuthology.orchestra.run.smithi057.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 322 kB/s | 35 kB 00:00 2024-02-28T22:51:52.021 INFO:teuthology.orchestra.run.smithi057.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 775 kB/s | 172 kB 00:00 2024-02-28T22:51:52.046 INFO:teuthology.orchestra.run.smithi057.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 501 kB/s | 42 kB 00:00 2024-02-28T22:51:52.072 INFO:teuthology.orchestra.run.smithi057.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 322 kB/s | 51 kB 00:00 2024-02-28T22:51:52.422 INFO:teuthology.orchestra.run.smithi057.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 109 kB/s | 43 kB 00:00 2024-02-28T22:51:52.506 INFO:teuthology.orchestra.run.smithi057.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 131 kB/s | 11 kB 00:00 2024-02-28T22:51:52.531 INFO:teuthology.orchestra.run.smithi057.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 355 kB/s | 172 kB 00:00 2024-02-28T22:51:52.589 INFO:teuthology.orchestra.run.smithi057.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 214 kB/s | 18 kB 00:00 2024-02-28T22:51:52.615 INFO:teuthology.orchestra.run.smithi057.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 280 kB/s | 23 kB 00:00 2024-02-28T22:51:52.656 INFO:teuthology.orchestra.run.smithi057.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 614 kB/s | 358 kB 00:00 2024-02-28T22:51:52.682 INFO:teuthology.orchestra.run.smithi057.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 213 kB/s | 19 kB 00:00 2024-02-28T22:51:52.707 INFO:teuthology.orchestra.run.smithi057.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 215 kB/s | 20 kB 00:00 2024-02-28T22:51:52.741 INFO:teuthology.orchestra.run.smithi057.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 558 kB/s | 46 kB 00:00 2024-02-28T22:51:52.816 INFO:teuthology.orchestra.run.smithi057.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 221 kB/s | 16 kB 00:00 2024-02-28T22:51:52.868 INFO:teuthology.orchestra.run.smithi057.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 498 kB/s | 79 kB 00:00 2024-02-28T22:51:52.874 INFO:teuthology.orchestra.run.smithi057.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-02-28T22:51:52.908 INFO:teuthology.orchestra.run.smithi057.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 987 kB/s | 90 kB 00:00 2024-02-28T22:51:52.909 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-28T22:51:52.909 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-28T22:51:52.909 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-28T22:51:52.910 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-02-28T22:51:52.910 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-02-28T22:51:52.910 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-02-28T22:51:52.934 INFO:teuthology.orchestra.run.smithi057.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 392 kB/s | 36 kB 00:00 2024-02-28T22:51:52.967 INFO:teuthology.orchestra.run.smithi057.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-02-28T22:51:53.026 INFO:teuthology.orchestra.run.smithi057.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 219 kB/s | 20 kB 00:00 2024-02-28T22:51:53.116 INFO:teuthology.orchestra.run.smithi057.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.0 MB/s | 427 kB 00:00 2024-02-28T22:51:53.130 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:53.130 INFO:teuthology.orchestra.run.smithi057.stdout:Total 10 MB/s | 30 MB 00:02 2024-02-28T22:51:53.259 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-02-28T22:51:53.259 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:53.259 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:51:53.259 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-28T22:51:53.260 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-28T22:51:53.260 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-28T22:51:53.260 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:53.260 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:51:53.515 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:51:53.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph 2024-02-28T22:51:53.607 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:51:53.607 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:51:54.075 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:51:54.076 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:51:54.471 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:43 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:51:54.794 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:51:54.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:51:54.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:51:54.801 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.802 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.803 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.803 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:51:54.880 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-28T22:51:54.883 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-28T22:51:54.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-28T22:51:54.885 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout:Install 33 Packages 2024-02-28T22:51:54.886 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:51:54.887 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 30 M 2024-02-28T22:51:54.887 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 109 M 2024-02-28T22:51:54.887 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:51:55.476 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:51:55.542 INFO:teuthology.orchestra.run.smithi084.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 43 kB/s | 6.4 kB 00:00 2024-02-28T22:51:55.546 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-28T22:51:55.587 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-28T22:51:55.601 INFO:teuthology.orchestra.run.smithi084.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 6.9 MB/s | 1.4 MB 00:00 2024-02-28T22:51:55.627 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-28T22:51:55.668 INFO:teuthology.orchestra.run.smithi084.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 7.6 MB/s | 2.1 MB 00:00 2024-02-28T22:51:55.701 INFO:teuthology.orchestra.run.smithi084.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.1 MB/s | 240 kB 00:00 2024-02-28T22:51:55.775 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-28T22:51:55.818 INFO:teuthology.orchestra.run.smithi084.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-02-28T22:51:55.881 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-28T22:51:55.910 INFO:teuthology.orchestra.run.smithi084.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-02-28T22:51:55.961 INFO:teuthology.orchestra.run.smithi084.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-02-28T22:51:56.011 INFO:teuthology.orchestra.run.smithi084.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-02-28T22:51:56.038 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-28T22:51:56.069 INFO:teuthology.orchestra.run.smithi084.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 4.9 MB/s | 1.2 MB 00:00 2024-02-28T22:51:56.089 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-28T22:51:56.095 INFO:teuthology.orchestra.run.smithi084.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-02-28T22:51:56.126 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-02-28T22:51:56.153 INFO:teuthology.orchestra.run.smithi084.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-02-28T22:51:56.172 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-02-28T22:51:56.223 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-02-28T22:51:56.290 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-02-28T22:51:56.354 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-02-28T22:51:56.397 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-02-28T22:51:56.416 INFO:teuthology.orchestra.run.smithi084.stdout:(12/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-02-28T22:51:56.454 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-02-28T22:51:56.589 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-02-28T22:51:56.595 INFO:teuthology.orchestra.run.smithi084.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 79 kB/s | 35 kB 00:00 2024-02-28T22:51:56.746 INFO:teuthology.orchestra.run.smithi084.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 265 kB/s | 172 kB 00:00 2024-02-28T22:51:56.763 INFO:teuthology.orchestra.run.smithi084.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 148 kB/s | 51 kB 00:00 2024-02-28T22:51:56.784 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-02-28T22:51:56.855 INFO:teuthology.orchestra.run.smithi084.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-02-28T22:51:56.880 INFO:teuthology.orchestra.run.smithi084.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 325 kB/s | 43 kB 00:00 2024-02-28T22:51:56.888 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-02-28T22:51:56.930 INFO:teuthology.orchestra.run.smithi084.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 217 kB/s | 11 kB 00:00 2024-02-28T22:51:56.955 INFO:teuthology.orchestra.run.smithi084.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 897 kB/s | 172 kB 00:00 2024-02-28T22:51:56.956 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-02-28T22:51:56.972 INFO:teuthology.orchestra.run.smithi084.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 429 kB/s | 18 kB 00:00 2024-02-28T22:51:57.006 INFO:teuthology.orchestra.run.smithi084.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 468 kB/s | 23 kB 00:00 2024-02-28T22:51:57.006 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-02-28T22:51:57.023 INFO:teuthology.orchestra.run.smithi084.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 392 kB/s | 19 kB 00:00 2024-02-28T22:51:57.056 INFO:teuthology.orchestra.run.smithi084.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-02-28T22:51:57.073 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-02-28T22:51:57.073 INFO:teuthology.orchestra.run.smithi084.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 296 kB/s | 20 kB 00:00 2024-02-28T22:51:57.090 INFO:teuthology.orchestra.run.smithi084.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 696 kB/s | 46 kB 00:00 2024-02-28T22:51:57.124 INFO:teuthology.orchestra.run.smithi084.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2024-02-28T22:51:57.141 INFO:teuthology.orchestra.run.smithi084.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 331 kB/s | 16 kB 00:00 2024-02-28T22:51:57.151 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-02-28T22:51:57.174 INFO:teuthology.orchestra.run.smithi084.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-02-28T22:51:57.199 INFO:teuthology.orchestra.run.smithi084.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-02-28T22:51:57.216 INFO:teuthology.orchestra.run.smithi084.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 478 kB/s | 36 kB 00:00 2024-02-28T22:51:57.250 INFO:teuthology.orchestra.run.smithi084.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-28T22:51:57.267 INFO:teuthology.orchestra.run.smithi084.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 403 kB/s | 20 kB 00:00 2024-02-28T22:51:57.350 INFO:teuthology.orchestra.run.smithi084.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.8 MB/s | 427 kB 00:00 2024-02-28T22:51:57.355 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:51:57.356 INFO:teuthology.orchestra.run.smithi084.stdout:Total 12 MB/s | 30 MB 00:02 2024-02-28T22:51:57.422 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-02-28T22:51:57.485 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-02-28T22:51:57.577 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-02-28T22:51:57.697 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-02-28T22:51:57.731 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:51:57.757 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-02-28T22:51:57.816 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:51:57.816 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:51:57.914 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-02-28T22:51:58.108 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-28T22:51:58.140 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-28T22:51:58.229 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:51:58.286 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:51:58.286 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:51:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:51:59.728 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-28T22:51:59.765 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-28T22:51:59.808 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-28T22:51:59.947 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-28T22:51:59.966 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-28T22:52:00.044 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-28T22:52:00.060 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:00.117 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-28T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-28T22:52:00.205 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-02-28T22:52:00.243 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-02-28T22:52:00.286 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-02-28T22:52:00.350 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-02-28T22:52:00.415 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-02-28T22:52:00.454 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-02-28T22:52:00.512 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-02-28T22:52:00.566 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-28T22:52:00.638 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:00.639 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-02-28T22:52:00.817 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-02-28T22:52:00.867 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-28T22:52:00.916 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-02-28T22:52:00.941 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-28T22:52:00.941 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:00.941 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-28T22:52:00.941 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-28T22:52:00.942 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-28T22:52:00.942 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:00.978 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-02-28T22:52:01.010 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-28T22:52:01.021 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-02-28T22:52:01.084 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-02-28T22:52:01.120 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-28T22:52:01.120 INFO:teuthology.orchestra.run.smithi057.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-28T22:52:01.120 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:01.170 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-02-28T22:52:01.418 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-02-28T22:52:01.475 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-02-28T22:52:01.564 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-02-28T22:52:01.683 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-02-28T22:52:01.722 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-02-28T22:52:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-02-28T22:52:02.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-28T22:52:02.080 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-28T22:52:02.170 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-28T22:52:02.171 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:02.171 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-28T22:52:02.171 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-28T22:52:02.171 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-28T22:52:02.171 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-02-28T22:52:03.043 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-28T22:52:03.044 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-28T22:52:03.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-28T22:52:03.046 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-28T22:52:03.773 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-28T22:52:03.850 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-28T22:52:03.850 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:03.851 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-28T22:52:03.851 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-28T22:52:03.851 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-28T22:52:03.851 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-28T22:52:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:04.100 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-28T22:52:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply-3.11-14.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-28T22:52:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:04.103 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:04.331 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-28T22:52:04.401 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-28T22:52:04.402 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:04.402 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-28T22:52:04.402 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-28T22:52:04.402 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-28T22:52:04.402 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:04.527 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-base 2024-02-28T22:52:04.615 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-28T22:52:04.684 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-28T22:52:04.684 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:04.684 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-28T22:52:04.684 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-28T22:52:04.684 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-28T22:52:04.685 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:04.772 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-28T22:52:04.895 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-28T22:52:04.895 INFO:teuthology.orchestra.run.smithi084.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-28T22:52:04.895 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:05.448 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:01 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:05.770 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:05.771 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.772 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:05.773 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:05.774 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:05.775 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.776 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.777 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.778 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.779 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:05.780 INFO:teuthology.orchestra.run.smithi057.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:05.845 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:05.846 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:52:05.846 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:05.919 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install cephadm 2024-02-28T22:52:06.745 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-28T22:52:06.745 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-02-28T22:52:06.746 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-28T22:52:06.747 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-28T22:52:06.748 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-28T22:52:06.748 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-28T22:52:06.748 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-28T22:52:06.748 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-28T22:52:06.748 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-28T22:52:06.749 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-28T22:52:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:02 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:07.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:07.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.127 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.128 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.129 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.130 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.131 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:07.131 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.131 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.131 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.131 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:07.196 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:07.196 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:52:07.197 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:07.198 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 209 k 2024-02-28T22:52:07.198 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 200 k 2024-02-28T22:52:07.198 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:07.271 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.8 MB/s | 209 kB 00:00 2024-02-28T22:52:07.271 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:07.272 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.7 MB/s | 209 kB 00:00 2024-02-28T22:52:07.272 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:07.279 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:07.279 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:07.282 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:07.283 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:07.372 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:07.600 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-28T22:52:07.601 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-28T22:52:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-28T22:52:07.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-28T22:52:07.603 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-28T22:52:07.603 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:07.603 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:07.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-base 2024-02-28T22:52:08.050 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:08.364 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:08.807 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:57 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:09.098 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:09.209 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:09.209 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:09.209 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:09.209 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.209 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.210 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:09.211 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:09.212 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:09.213 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.214 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:09.215 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.216 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.217 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:09.218 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:09.278 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:09.279 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:52:09.279 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:09.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install cephadm 2024-02-28T22:52:09.380 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-immutable-object-cache 2024-02-28T22:52:10.212 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:59 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:06 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.517 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.518 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.519 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:10.520 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.521 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.522 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.523 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.524 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.525 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.526 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.586 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:10.587 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:52:10.587 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 209 k 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 200 k 2024-02-28T22:52:10.588 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:52:10.616 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:10.616 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.617 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:10.618 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:10.619 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.620 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:10.621 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.622 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.623 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.624 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.625 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:10.669 INFO:teuthology.orchestra.run.smithi084.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.5 MB/s | 209 kB 00:00 2024-02-28T22:52:10.669 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:10.670 INFO:teuthology.orchestra.run.smithi084.stdout:Total 2.5 MB/s | 209 kB 00:00 2024-02-28T22:52:10.670 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:52:10.676 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:52:10.676 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:52:10.679 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:52:10.679 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:52:10.693 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:10.693 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:52:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:10.695 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 142 k 2024-02-28T22:52:10.695 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 399 k 2024-02-28T22:52:10.695 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:52:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-02-28T22:52:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-02-28T22:52:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:10.779 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:10.779 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:10.841 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:10.841 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:11.009 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:11.086 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:11.165 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:11.165 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:11.165 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-28T22:52:11.165 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:11.977 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:12.321 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-28T22:52:12.321 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:12.321 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:52:12.321 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-28T22:52:12.321 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:12.322 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:12.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-immutable-object-cache 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:12.740 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:13.037 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr 2024-02-28T22:52:13.520 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:02 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:13.828 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:13.828 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:13.828 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:13.828 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.828 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:13.829 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.830 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:13.831 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.832 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.833 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.834 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:13.835 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.836 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:13.837 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.837 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.837 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.837 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:13.900 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:13.901 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 142 k 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 399 k 2024-02-28T22:52:13.902 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:52:13.949 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:09 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:13.965 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.3 MB/s | 142 kB 00:00 2024-02-28T22:52:13.965 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:13.966 INFO:teuthology.orchestra.run.smithi084.stdout:Total 2.2 MB/s | 142 kB 00:00 2024-02-28T22:52:13.966 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:52:13.975 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:52:13.975 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:52:14.033 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:52:14.033 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:52:14.196 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:52:14.258 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:14.271 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:14.272 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.273 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.274 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.275 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.276 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.277 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.278 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.279 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:14.279 INFO:teuthology.orchestra.run.smithi057.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:14.329 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:14.329 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:52:14.329 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-28T22:52:14.329 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:14.345 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:14.346 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:52:14.346 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:14.424 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-dashboard 2024-02-28T22:52:15.321 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:11 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:15.648 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:15.648 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:15.648 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:15.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:15.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:15.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.659 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:15.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:15.708 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:52:15.708 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:15.709 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:15.727 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:15.728 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:15.728 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:52:15.728 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Installing weak dependencies: 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Install 7 Packages 2024-02-28T22:52:15.730 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:15.730 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 2.0 M 2024-02-28T22:52:15.730 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 8.1 M 2024-02-28T22:52:15.730 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:16.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr 2024-02-28T22:52:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 108 kB/s | 23 kB 00:00 2024-02-28T22:52:16.323 INFO:teuthology.orchestra.run.smithi057.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 64 kB/s | 15 kB 00:00 2024-02-28T22:52:16.473 INFO:teuthology.orchestra.run.smithi057.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 4.5 MB/s | 1.7 MB 00:00 2024-02-28T22:52:16.515 INFO:teuthology.orchestra.run.smithi057.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 214 kB/s | 41 kB 00:00 2024-02-28T22:52:16.540 INFO:teuthology.orchestra.run.smithi057.stdout:(5/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 39 kB/s | 9.0 kB 00:00 2024-02-28T22:52:16.565 INFO:teuthology.orchestra.run.smithi057.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 343 kB/s | 31 kB 00:00 2024-02-28T22:52:16.632 INFO:teuthology.orchestra.run.smithi057.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.6 MB/s | 188 kB 00:00 2024-02-28T22:52:16.640 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:16.641 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-02-28T22:52:16.642 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:16.670 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:16.671 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:16.809 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:16.809 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:16.885 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:05 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:17.190 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:17.190 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:17.191 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.192 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:17.193 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.194 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:17.195 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.196 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:17.197 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:17.198 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.199 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.199 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.199 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:17.237 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:17.259 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:17.260 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:52:17.260 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:17.309 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-28T22:52:17.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-dashboard 2024-02-28T22:52:17.360 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-28T22:52:17.465 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-28T22:52:17.516 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-28T22:52:17.566 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-28T22:52:17.888 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-28T22:52:17.928 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-28T22:52:18.175 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:07 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:18.480 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.481 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:18.482 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.483 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.484 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.485 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.486 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.487 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:18.488 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.489 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:18.553 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-28T22:52:18.554 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout:Install 7 Packages 2024-02-28T22:52:18.555 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:18.556 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 2.0 M 2024-02-28T22:52:18.556 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 8.1 M 2024-02-28T22:52:18.556 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:52:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-28T22:52:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-28T22:52:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-28T22:52:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-28T22:52:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-28T22:52:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-28T22:52:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-28T22:52:18.784 INFO:teuthology.orchestra.run.smithi084.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 167 kB/s | 23 kB 00:00 2024-02-28T22:52:18.809 INFO:teuthology.orchestra.run.smithi084.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 89 kB/s | 15 kB 00:00 2024-02-28T22:52:18.860 INFO:teuthology.orchestra.run.smithi084.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 7.9 MB/s | 1.7 MB 00:00 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-28T22:52:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:18.971 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:19.043 INFO:teuthology.orchestra.run.smithi084.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 35 kB/s | 9.0 kB 00:00 2024-02-28T22:52:19.102 INFO:teuthology.orchestra.run.smithi084.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 194 kB/s | 31 kB 00:00 2024-02-28T22:52:19.128 INFO:teuthology.orchestra.run.smithi084.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 129 kB/s | 41 kB 00:00 2024-02-28T22:52:19.177 INFO:teuthology.orchestra.run.smithi084.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.4 MB/s | 188 kB 00:00 2024-02-28T22:52:19.187 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:19.187 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.2 MB/s | 2.0 MB 00:00 2024-02-28T22:52:19.188 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:52:19.215 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:52:19.215 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:52:19.236 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-28T22:52:19.347 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:52:19.347 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:52:19.740 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:52:19.787 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-28T22:52:19.834 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-28T22:52:19.925 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-28T22:52:19.968 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-28T22:52:20.014 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-28T22:52:20.115 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:16 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:20.337 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-28T22:52:20.372 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-28T22:52:20.445 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:20.447 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.448 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.449 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.450 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:20.451 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:20.452 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.453 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.454 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:20.455 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.455 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.455 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.455 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:20.525 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-28T22:52:20.527 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout:Install 12 Packages 2024-02-28T22:52:20.528 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:20.529 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 38 M 2024-02-28T22:52:20.529 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 209 M 2024-02-28T22:52:20.529 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-28T22:52:21.014 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-28T22:52:21.090 INFO:teuthology.orchestra.run.smithi057.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 29 MB/s | 7.4 MB 00:00 2024-02-28T22:52:21.123 INFO:teuthology.orchestra.run.smithi057.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 676 kB/s | 195 kB 00:00 2024-02-28T22:52:21.207 INFO:teuthology.orchestra.run.smithi057.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 283 kB/s | 33 kB 00:00 2024-02-28T22:52:21.257 INFO:teuthology.orchestra.run.smithi057.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 371 kB/s | 18 kB 00:00 2024-02-28T22:52:21.299 INFO:teuthology.orchestra.run.smithi057.stdout:(5/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-02-28T22:52:21.324 INFO:teuthology.orchestra.run.smithi057.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 534 kB/s | 35 kB 00:00 2024-02-28T22:52:21.399 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:21.400 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:21.416 INFO:teuthology.orchestra.run.smithi057.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-02-28T22:52:21.508 INFO:teuthology.orchestra.run.smithi057.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.8 MB/s | 3.0 MB 00:00 2024-02-28T22:52:21.567 INFO:teuthology.orchestra.run.smithi057.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.3 MB/s | 250 kB 00:00 2024-02-28T22:52:21.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-28T22:52:21.717 INFO:teuthology.orchestra.run.smithi057.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-02-28T22:52:22.042 INFO:teuthology.orchestra.run.smithi057.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-02-28T22:52:22.292 INFO:teuthology.orchestra.run.smithi057.stdout:(12/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.8 MB/s | 4.7 MB 00:00 2024-02-28T22:52:22.293 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:22.294 INFO:teuthology.orchestra.run.smithi057.stdout:Total 22 MB/s | 38 MB 00:01 2024-02-28T22:52:22.567 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:11 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:22.878 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:22.879 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.880 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:22.881 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.882 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:22.883 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.884 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.885 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.886 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.887 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:22.953 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-28T22:52:22.955 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout:Install 12 Packages 2024-02-28T22:52:22.956 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:22.957 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 38 M 2024-02-28T22:52:22.957 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 209 M 2024-02-28T22:52:22.957 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:52:22.960 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:23.042 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:23.043 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:23.474 INFO:teuthology.orchestra.run.smithi084.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 30 MB/s | 7.4 MB 00:00 2024-02-28T22:52:23.575 INFO:teuthology.orchestra.run.smithi084.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 329 kB/s | 33 kB 00:00 2024-02-28T22:52:23.600 INFO:teuthology.orchestra.run.smithi084.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 520 kB/s | 195 kB 00:00 2024-02-28T22:52:23.642 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:23.643 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:23.683 INFO:teuthology.orchestra.run.smithi084.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-02-28T22:52:23.709 INFO:teuthology.orchestra.run.smithi084.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 169 kB/s | 18 kB 00:00 2024-02-28T22:52:23.817 INFO:teuthology.orchestra.run.smithi084.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 265 kB/s | 35 kB 00:00 2024-02-28T22:52:24.059 INFO:teuthology.orchestra.run.smithi084.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-02-28T22:52:24.251 INFO:teuthology.orchestra.run.smithi084.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.4 MB/s | 3.0 MB 00:00 2024-02-28T22:52:24.584 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:24.676 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-02-28T22:52:24.743 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-02-28T22:52:24.899 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-02-28T22:52:24.944 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-28T22:52:25.118 INFO:teuthology.orchestra.run.smithi084.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 289 kB/s | 250 kB 00:00 2024-02-28T22:52:25.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-28T22:52:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-28T22:52:25.331 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-28T22:52:26.217 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-28T22:52:26.356 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-28T22:52:27.966 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-28T22:52:29.184 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-28T22:52:29.214 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-02-28T22:52:31.770 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-02-28T22:52:31.771 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-02-28T22:52:31.771 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-02-28T22:52:32.209 INFO:teuthology.orchestra.run.smithi084.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 641 kB/s | 5.1 MB 00:08 2024-02-28T22:52:32.274 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-02-28T22:52:32.274 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:32.274 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:32.274 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-28T22:52:32.274 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:32.275 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:32.722 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-rook 2024-02-28T22:52:33.342 INFO:teuthology.orchestra.run.smithi084.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 504 kB/s | 4.7 MB 00:09 2024-02-28T22:52:33.634 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:29 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:33.974 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.975 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:33.976 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.977 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.978 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:33.979 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.980 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.981 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.982 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:33.983 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:34.054 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-28T22:52:34.056 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout:Install 13 Packages 2024-02-28T22:52:34.057 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:34.058 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 2.2 M 2024-02-28T22:52:34.058 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 26 M 2024-02-28T22:52:34.058 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:34.364 INFO:teuthology.orchestra.run.smithi057.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 569 kB/s | 47 kB 00:00 2024-02-28T22:52:34.548 INFO:teuthology.orchestra.run.smithi057.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 72 kB/s | 19 kB 00:00 2024-02-28T22:52:34.573 INFO:teuthology.orchestra.run.smithi057.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-02-28T22:52:34.907 INFO:teuthology.orchestra.run.smithi057.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 444 kB/s | 159 kB 00:00 2024-02-28T22:52:34.940 INFO:teuthology.orchestra.run.smithi057.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 761 kB/s | 279 kB 00:00 2024-02-28T22:52:34.957 INFO:teuthology.orchestra.run.smithi057.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 375 kB/s | 222 kB 00:00 2024-02-28T22:52:35.016 INFO:teuthology.orchestra.run.smithi057.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 495 kB/s | 54 kB 00:00 2024-02-28T22:52:35.291 INFO:teuthology.orchestra.run.smithi057.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 45 kB/s | 15 kB 00:00 2024-02-28T22:52:35.316 INFO:teuthology.orchestra.run.smithi057.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 86 kB/s | 32 kB 00:00 2024-02-28T22:52:35.458 INFO:teuthology.orchestra.run.smithi057.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 419 kB/s | 59 kB 00:00 2024-02-28T22:52:35.519 INFO:teuthology.orchestra.run.smithi057.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 511 kB/s | 203 kB 00:00 2024-02-28T22:52:35.592 INFO:teuthology.orchestra.run.smithi057.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 677 kB/s | 90 kB 00:00 2024-02-28T22:52:35.675 INFO:teuthology.orchestra.run.smithi057.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.7 MB/s | 1.0 MB 00:00 2024-02-28T22:52:35.682 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:35.683 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-02-28T22:52:35.909 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:35.958 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:35.958 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:36.123 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:36.124 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:36.819 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:37.051 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-28T22:52:37.136 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-28T22:52:37.221 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-28T22:52:37.257 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-28T22:52:37.314 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-28T22:52:37.465 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-28T22:52:37.611 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-02-28T22:52:37.679 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-28T22:52:38.661 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-28T22:52:38.728 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-28T22:52:38.762 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-28T22:52:38.818 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-28T22:52:38.846 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-28T22:52:39.779 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-28T22:52:39.780 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-28T22:52:40.272 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-28T22:52:40.272 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:40.272 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-28T22:52:40.273 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-28T22:52:40.274 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:40.274 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:40.630 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-cephadm 2024-02-28T22:52:41.532 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:37 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.860 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:41.861 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:41.862 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:41.863 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.864 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:41.865 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.866 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.867 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.868 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:41.869 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.869 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.869 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.869 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:41.940 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:41.940 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:Install 6 Packages 2024-02-28T22:52:41.942 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 7.0 M 2024-02-28T22:52:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 33 M 2024-02-28T22:52:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:42.119 INFO:teuthology.orchestra.run.smithi057.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.8 MB/s | 127 kB 00:00 2024-02-28T22:52:42.336 INFO:teuthology.orchestra.run.smithi057.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 862 kB/s | 248 kB 00:00 2024-02-28T22:52:42.719 INFO:teuthology.orchestra.run.smithi057.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 151 kB/s | 58 kB 00:00 2024-02-28T22:52:42.778 INFO:teuthology.orchestra.run.smithi057.stdout:(4/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 831 kB/s | 548 kB 00:00 2024-02-28T22:52:42.903 INFO:teuthology.orchestra.run.smithi057.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 279 kB/s | 51 kB 00:00 2024-02-28T22:52:43.603 INFO:teuthology.orchestra.run.smithi057.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-02-28T22:52:43.604 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:43.605 INFO:teuthology.orchestra.run.smithi057.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-02-28T22:52:43.707 INFO:teuthology.orchestra.run.smithi084.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 895 kB/s | 16 MB 00:18 2024-02-28T22:52:43.707 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:43.708 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.8 MB/s | 38 MB 00:20 2024-02-28T22:52:43.738 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:43.778 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:43.778 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:43.943 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:43.944 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:44.355 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:52:44.387 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:44.434 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:52:44.434 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:52:44.599 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-28T22:52:44.670 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-28T22:52:45.007 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:52:45.007 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:52:45.399 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-28T22:52:45.507 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-28T22:52:45.584 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-28T22:52:45.617 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-28T22:52:45.916 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:52:45.991 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-02-28T22:52:46.041 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-02-28T22:52:46.179 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-02-28T22:52:46.224 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-28T22:52:46.469 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-28T22:52:46.491 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-28T22:52:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-28T22:52:46.610 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-28T22:52:46.886 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:47.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-fuse 2024-02-28T22:52:47.450 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-28T22:52:47.597 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-28T22:52:48.136 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:44 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:48.461 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:48.462 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:48.463 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:48.464 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.465 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.467 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:48.468 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.469 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:48.470 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.471 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.472 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.473 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:48.537 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:48.539 INFO:teuthology.orchestra.run.smithi057.stdout:Install 2 Packages 2024-02-28T22:52:48.540 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:48.540 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 915 k 2024-02-28T22:52:48.540 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 2.7 M 2024-02-28T22:52:48.540 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:49.005 INFO:teuthology.orchestra.run.smithi057.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 5.1 MB/s | 835 kB 00:00 2024-02-28T22:52:49.030 INFO:teuthology.orchestra.run.smithi057.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-02-28T22:52:49.032 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:49.033 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.8 MB/s | 915 kB 00:00 2024-02-28T22:52:49.073 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:49.126 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-28T22:52:49.147 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:49.148 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:49.262 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:49.262 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:49.552 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:49.736 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-28T22:52:49.822 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-28T22:52:50.371 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-28T22:52:50.402 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-28T22:52:50.781 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-28T22:52:50.781 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-28T22:52:51.188 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-28T22:52:51.188 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:51.188 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:51.189 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-28T22:52:51.189 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:51.189 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:51.517 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librados-devel 2024-02-28T22:52:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:48 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:52.743 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:52.743 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.744 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.745 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:52.746 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:52.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:52.821 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:52.822 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 126 k 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 456 k 2024-02-28T22:52:52.823 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:52.894 INFO:teuthology.orchestra.run.smithi057.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 126 kB 00:00 2024-02-28T22:52:52.895 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:52.895 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.7 MB/s | 126 kB 00:00 2024-02-28T22:52:52.895 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:52.902 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:52.902 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-02-28T22:52:52.940 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-02-28T22:52:52.972 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:52.972 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:53.146 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:53.250 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:53.443 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-02-28T22:52:53.443 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-28T22:52:53.444 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-28T22:52:53.445 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:53.445 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:52:53.834 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:53.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-rook 2024-02-28T22:52:54.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:54.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:54.335 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:54.335 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:54.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:54.335 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:54.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install libcephfs2 2024-02-28T22:52:54.755 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:43 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:52:55.067 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.142 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-28T22:52:55.144 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout:Install 13 Packages 2024-02-28T22:52:55.145 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:52:55.146 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 2.2 M 2024-02-28T22:52:55.146 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 26 M 2024-02-28T22:52:55.146 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:52:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:51 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:55.747 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:55.747 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:55.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:55.747 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.747 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.748 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:55.749 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.750 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:55.751 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:55.752 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.753 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.754 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:55.755 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.756 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.757 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.758 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.759 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.760 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:55.761 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.762 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:55.763 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.764 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.764 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.764 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:55.765 INFO:teuthology.orchestra.run.smithi057.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:52:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:55.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install libcephfs-devel 2024-02-28T22:52:56.037 INFO:teuthology.orchestra.run.smithi084.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 456 kB/s | 47 kB 00:00 2024-02-28T22:52:56.137 INFO:teuthology.orchestra.run.smithi084.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 126 kB/s | 26 kB 00:00 2024-02-28T22:52:56.154 INFO:teuthology.orchestra.run.smithi084.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 87 kB/s | 19 kB 00:00 2024-02-28T22:52:56.312 INFO:teuthology.orchestra.run.smithi084.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 809 kB/s | 222 kB 00:00 2024-02-28T22:52:56.354 INFO:teuthology.orchestra.run.smithi084.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 797 kB/s | 159 kB 00:00 2024-02-28T22:52:56.413 INFO:teuthology.orchestra.run.smithi084.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-02-28T22:52:56.438 INFO:teuthology.orchestra.run.smithi084.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 429 kB/s | 54 kB 00:00 2024-02-28T22:52:56.655 INFO:teuthology.orchestra.run.smithi084.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-02-28T22:52:56.680 INFO:teuthology.orchestra.run.smithi084.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 99 kB/s | 32 kB 00:00 2024-02-28T22:52:56.795 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:52 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:56.947 INFO:teuthology.orchestra.run.smithi084.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 222 kB/s | 59 kB 00:00 2024-02-28T22:52:56.972 INFO:teuthology.orchestra.run.smithi084.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 380 kB/s | 203 kB 00:00 2024-02-28T22:52:57.116 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:57.116 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.117 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:57.118 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.119 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.120 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.121 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.122 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.123 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.124 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:57.125 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.126 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:57.194 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 30 k 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 137 k 2024-02-28T22:52:57.196 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:52:57.206 INFO:teuthology.orchestra.run.smithi084.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 348 kB/s | 90 kB 00:00 2024-02-28T22:52:57.248 INFO:teuthology.orchestra.run.smithi084.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.7 MB/s | 1.0 MB 00:00 2024-02-28T22:52:57.248 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:57.249 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.0 MB/s | 2.2 MB 00:02 2024-02-28T22:52:57.281 INFO:teuthology.orchestra.run.smithi057.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 358 kB/s | 30 kB 00:00 2024-02-28T22:52:57.282 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:52:57.282 INFO:teuthology.orchestra.run.smithi057.stdout:Total 351 kB/s | 30 kB 00:00 2024-02-28T22:52:57.282 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:52:57.287 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:52:57.287 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:52:57.328 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:52:57.328 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:52:57.462 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:52:57.468 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:52:57.513 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:52:57.514 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:52:57.632 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:57.671 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:52:57.672 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:52:58.106 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:58.338 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:52:58.411 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:52:58.411 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:58.412 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:52:58.412 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-28T22:52:58.412 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:52:58.412 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:58.535 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-28T22:52:58.596 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-28T22:52:58.625 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librados2 2024-02-28T22:52:58.669 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-28T22:52:58.710 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-28T22:52:58.756 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-28T22:52:58.881 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-28T22:52:59.010 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-02-28T22:52:59.072 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-28T22:52:59.521 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:55 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:52:59.846 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.847 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:52:59.848 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.849 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.850 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.851 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.852 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.853 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.854 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:52:59.855 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:52:59.922 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:52:59.923 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:52:59.923 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:52:59.946 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-28T22:52:59.989 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librbd1 2024-02-28T22:53:00.008 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-28T22:53:00.066 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-28T22:53:00.122 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-28T22:53:00.151 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-28T22:53:00.854 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:56 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:01.029 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-28T22:53:01.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-28T22:53:01.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-28T22:53:01.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-28T22:53:01.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:01.178 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:01.179 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:01.180 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.181 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:01.182 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.183 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.184 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.185 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:01.186 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:01.254 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:01.255 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:53:01.255 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:01.325 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rados 2024-02-28T22:53:01.496 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-28T22:53:01.496 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:01.496 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:01.496 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:01.497 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:01.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-cephadm 2024-02-28T22:53:02.210 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:58 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:02.537 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:02.537 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.538 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.539 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:02.540 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.541 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:02.542 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.543 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.544 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:02.545 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:02.546 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:02.615 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:02.617 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:53:02.617 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:02.689 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rgw 2024-02-28T22:53:02.730 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:51 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:03.037 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:03.038 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.039 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.040 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.041 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.042 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.043 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.044 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.045 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.046 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.108 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:03.109 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:03.109 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-02-28T22:53:03.109 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:03.109 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout:Install 6 Packages 2024-02-28T22:53:03.110 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:03.111 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 7.0 M 2024-02-28T22:53:03.111 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 33 M 2024-02-28T22:53:03.111 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:03.323 INFO:teuthology.orchestra.run.smithi084.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.6 MB/s | 127 kB 00:00 2024-02-28T22:53:03.440 INFO:teuthology.orchestra.run.smithi084.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-02-28T22:53:03.548 INFO:teuthology.orchestra.run.smithi084.stdout:(3/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 2.4 MB/s | 548 kB 00:00 2024-02-28T22:53:03.554 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:59 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:03.632 INFO:teuthology.orchestra.run.smithi084.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 617 kB/s | 51 kB 00:00 2024-02-28T22:53:03.657 INFO:teuthology.orchestra.run.smithi084.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 266 kB/s | 58 kB 00:00 2024-02-28T22:53:03.882 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:03.882 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:03.882 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:03.882 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.882 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.883 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.884 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:03.891 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:03.960 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:03.962 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:53:03.962 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:04.046 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-cephfs 2024-02-28T22:53:04.932 INFO:teuthology.orchestra.run.smithi084.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.5 MB/s | 6.0 MB 00:01 2024-02-28T22:53:04.933 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:04.934 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-02-28T22:53:04.953 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:00 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:05.063 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:05.101 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:05.101 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:05.259 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:05.260 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:05.283 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:05.283 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:05.283 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:05.284 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.285 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:05.286 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.287 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.288 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:05.289 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.290 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.291 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.292 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.293 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.294 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:05.294 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:05.361 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:05.362 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:53:05.363 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:05.441 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rbd 2024-02-28T22:53:05.671 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:05.871 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-28T22:53:05.931 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-28T22:53:06.318 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:02 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:06.617 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-28T22:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:06.651 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:06.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:06.652 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.652 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.653 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:06.654 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:06.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.655 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:06.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.655 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.656 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:06.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.656 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.657 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.658 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.659 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:06.660 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:06.663 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:06.720 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-28T22:53:06.723 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:06.724 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-02-28T22:53:06.724 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:06.792 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-28T22:53:06.803 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-fuse 2024-02-28T22:53:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-28T22:53:07.666 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:03 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:07.674 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-28T22:53:07.674 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-28T22:53:07.675 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-28T22:53:07.675 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-28T22:53:07.675 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-28T22:53:07.675 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-28T22:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.993 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:07.994 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.995 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:07.996 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.997 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:08.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-28T22:53:08.040 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:08.041 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:08.064 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:08.065 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 86 k 2024-02-28T22:53:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 230 k 2024-02-28T22:53:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:53:08.143 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-02-28T22:53:08.143 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:08.144 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-02-28T22:53:08.144 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:53:08.152 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:53:08.153 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:53:08.221 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:53:08.221 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:53:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:53:08.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-fuse 2024-02-28T22:53:08.499 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:09.064 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:09.252 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:58 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:09.373 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:09.558 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:09.559 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.560 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.561 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.562 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:09.563 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.564 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.565 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.566 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.567 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-28T22:53:09.630 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout:Install 2 Packages 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:09.631 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 915 k 2024-02-28T22:53:09.632 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 2.7 M 2024-02-28T22:53:09.632 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:09.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-mirror 2024-02-28T22:53:09.885 INFO:teuthology.orchestra.run.smithi084.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 7.8 MB/s | 835 kB 00:00 2024-02-28T22:53:10.060 INFO:teuthology.orchestra.run.smithi084.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 287 kB/s | 80 kB 00:00 2024-02-28T22:53:10.062 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:10.063 INFO:teuthology.orchestra.run.smithi084.stdout:Total 2.1 MB/s | 915 kB 00:00 2024-02-28T22:53:10.102 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:10.173 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:10.173 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:10.278 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:10.278 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:10.592 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:06 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-28T22:53:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-28T22:53:10.915 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:10.915 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:10.916 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.917 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.918 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:10.919 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.920 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.921 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.922 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.923 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.924 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:10.992 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 3.0 M 2024-02-28T22:53:10.993 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 12 M 2024-02-28T22:53:10.994 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:53:11.096 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-02-28T22:53:11.096 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:11.097 INFO:teuthology.orchestra.run.smithi057.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-02-28T22:53:11.097 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:53:11.108 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:53:11.108 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:53:11.197 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:53:11.197 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:53:11.676 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:53:11.690 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-28T22:53:11.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-28T22:53:11.755 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:11.834 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:11.835 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:53:11.835 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-28T22:53:11.835 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-28T22:53:11.835 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-28T22:53:11.835 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:12.073 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-28T22:53:12.073 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:12.074 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-28T22:53:12.074 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:12.425 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados-devel 2024-02-28T22:53:13.329 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:02 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:13.346 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:13.636 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:13.636 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:13.636 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:13.636 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:13.637 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.638 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:13.639 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.640 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:13.641 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.642 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:13.643 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:13.644 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.645 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.646 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.647 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.648 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.649 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.650 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:13.651 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.652 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.653 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:13.670 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-nbd 2024-02-28T22:53:13.709 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:13.710 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:53:13.711 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:13.711 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 126 k 2024-02-28T22:53:13.711 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 456 k 2024-02-28T22:53:13.711 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:13.788 INFO:teuthology.orchestra.run.smithi084.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.6 MB/s | 126 kB 00:00 2024-02-28T22:53:13.788 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:13.789 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.6 MB/s | 126 kB 00:00 2024-02-28T22:53:13.789 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:13.796 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:13.796 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:13.866 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:13.867 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:14.033 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:14.130 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:14.562 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:10 ago on Wed 28 Feb 2024 10:51:04 PM UTC. 2024-02-28T22:53:14.665 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.885 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.886 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.887 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-02-28T22:53:14.888 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.889 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.890 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: conflicting requests 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.891 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.892 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:14.893 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.894 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.894 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.894 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:14.961 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:14.962 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 169 k 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 490 k 2024-02-28T22:53:14.963 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:15.006 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:15.040 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.2 MB/s | 169 kB 00:00 2024-02-28T22:53:15.043 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:15.043 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.1 MB/s | 169 kB 00:00 2024-02-28T22:53:15.043 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-02-28T22:53:15.051 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-02-28T22:53:15.051 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-02-28T22:53:15.121 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-02-28T22:53:15.121 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-02-28T22:53:15.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs2 2024-02-28T22:53:15.309 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-02-28T22:53:15.417 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:16.093 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:16.183 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:05 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:16.446 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-02-28T22:53:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:53:16.447 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.491 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.492 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.493 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:16.494 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.495 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.496 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.497 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.498 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.499 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.500 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.501 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:16.501 INFO:teuthology.orchestra.run.smithi084.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:16.563 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:16.564 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:16.564 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:16.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs-devel 2024-02-28T22:53:16.806 DEBUG:teuthology.parallel:result is None 2024-02-28T22:53:17.483 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:06 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:17.792 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.793 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:17.794 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.795 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.796 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:17.797 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.798 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.799 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.800 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:17.867 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:17.867 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 30 k 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 137 k 2024-02-28T22:53:17.868 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:17.940 INFO:teuthology.orchestra.run.smithi084.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 427 kB/s | 30 kB 00:00 2024-02-28T22:53:17.941 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:17.941 INFO:teuthology.orchestra.run.smithi084.stdout:Total 415 kB/s | 30 kB 00:00 2024-02-28T22:53:17.942 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:17.946 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:17.946 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:17.985 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:17.986 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:18.118 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:18.267 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:18.736 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:19.032 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:19.032 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:19.249 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados2 2024-02-28T22:53:20.114 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:09 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.422 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.423 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:20.424 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:20.425 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:20.426 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.427 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:20.428 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.429 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.430 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:20.431 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:20.492 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:20.493 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:20.493 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:20.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librbd1 2024-02-28T22:53:21.418 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:10 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:21.724 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.725 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:21.726 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:21.727 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:21.728 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:21.729 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.730 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.731 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.732 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:21.733 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:21.794 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:21.795 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:21.796 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:21.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rados 2024-02-28T22:53:22.743 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:11 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:23.068 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.069 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:23.070 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:23.071 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.072 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.073 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.074 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:23.075 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.076 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:23.077 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:23.143 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:23.144 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:23.144 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:23.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rgw 2024-02-28T22:53:24.078 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:13 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:24.386 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:24.387 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.388 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.389 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.390 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:24.391 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.392 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.393 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.394 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.395 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:24.396 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:24.459 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:24.460 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:24.461 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:24.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-cephfs 2024-02-28T22:53:25.380 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:14 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.687 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.688 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.689 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.690 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:25.691 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:25.692 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.693 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:25.694 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.695 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:25.696 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:25.761 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:25.762 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:25.762 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:25.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rbd 2024-02-28T22:53:26.699 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:15 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:27.005 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:27.005 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:27.005 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:27.006 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.007 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:27.008 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.009 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.010 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.011 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.012 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.013 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:27.014 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-28T22:53:27.078 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:27.079 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-02-28T22:53:27.079 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:27.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-fuse 2024-02-28T22:53:28.014 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:17 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:28.322 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:28.322 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:28.322 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:28.322 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.322 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.324 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.325 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:28.326 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.327 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.328 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.329 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.330 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.331 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:28.331 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.331 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.331 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.331 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:28.393 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:28.394 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:28.395 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:53:28.395 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:28.395 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 86 k 2024-02-28T22:53:28.395 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 230 k 2024-02-28T22:53:28.395 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:28.468 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-02-28T22:53:28.468 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:28.469 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-02-28T22:53:28.469 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:28.477 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:28.477 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:28.542 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:28.542 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:28.694 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:28.806 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:29.322 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:29.638 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:29.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-mirror 2024-02-28T22:53:30.812 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:19 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.120 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:31.121 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:31.122 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.123 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:31.124 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.125 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:31.126 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.127 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.128 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:31.193 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:31.193 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 3.0 M 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 12 M 2024-02-28T22:53:31.194 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:31.303 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-02-28T22:53:31.304 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:31.304 INFO:teuthology.orchestra.run.smithi084.stdout:Total 27 MB/s | 3.0 MB 00:00 2024-02-28T22:53:31.304 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:31.316 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:31.316 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:31.398 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:31.399 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:31.859 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:31.993 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:31.993 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-28T22:53:31.993 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-28T22:53:31.993 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-28T22:53:31.993 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-28T22:53:31.994 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:33.428 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:33.428 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:33.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-nbd 2024-02-28T22:53:34.603 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:23 ago on Wed 28 Feb 2024 10:51:11 PM UTC. 2024-02-28T22:53:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:Modular dependency problems: 2024-02-28T22:53:34.914 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-28T22:53:34.914 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 1: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240221162200:20125149.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 2: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 3: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 4: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 5: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 6: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 7: conflicting requests 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.915 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 8: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 9: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 10: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 11: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 12: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 13: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 14: conflicting requests 2024-02-28T22:53:34.916 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 15: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 16: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 17: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 18: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 19: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 20: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 21: conflicting requests 2024-02-28T22:53:34.917 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 22: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 23: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 24: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 25: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 26: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 27: conflicting requests 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.918 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 28: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 29: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 30: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 31: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 32: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 33: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 34: conflicting requests 2024-02-28T22:53:34.919 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 35: conflicting requests 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 36: conflicting requests 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 37: cannot install the best candidate for the job 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 38: conflicting requests 2024-02-28T22:53:34.920 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 39: cannot install the best candidate for the job 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 40: cannot install the best candidate for the job 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.921 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 41: cannot install the best candidate for the job 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 42: cannot install the best candidate for the job 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 43: cannot install the best candidate for the job 2024-02-28T22:53:34.922 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: Problem 44: cannot install the best candidate for the job 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.923 INFO:teuthology.orchestra.run.smithi084.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-28T22:53:34.986 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 169 k 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 490 k 2024-02-28T22:53:34.987 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-02-28T22:53:35.055 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.5 MB/s | 169 kB 00:00 2024-02-28T22:53:35.056 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:53:35.056 INFO:teuthology.orchestra.run.smithi084.stdout:Total 2.4 MB/s | 169 kB 00:00 2024-02-28T22:53:35.057 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-02-28T22:53:35.065 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-02-28T22:53:35.066 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-02-28T22:53:35.132 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-02-28T22:53:35.132 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-02-28T22:53:35.310 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-02-28T22:53:35.414 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:36.021 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:53:36.371 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-02-28T22:53:36.660 DEBUG:teuthology.parallel:result is None 2024-02-28T22:53:36.661 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:53:36.661 INFO:teuthology.packaging:ref: None 2024-02-28T22:53:36.661 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:53:36.661 INFO:teuthology.packaging:branch: None 2024-02-28T22:53:36.661 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:53:36.661 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-28T22:53:36.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-28T22:53:36.808 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-28T22:53:36.883 INFO:teuthology.orchestra.run.smithi057.stdout:18.2.0-0.el9 2024-02-28T22:53:36.884 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-28T22:53:36.884 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-28T22:53:36.886 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-28T22:53:36.886 INFO:teuthology.packaging:ref: None 2024-02-28T22:53:36.886 INFO:teuthology.packaging:tag: v18.2.0 2024-02-28T22:53:36.886 INFO:teuthology.packaging:branch: None 2024-02-28T22:53:36.886 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:53:36.886 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-28T22:53:36.886 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-28T22:53:37.219 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-28T22:53:37.288 INFO:teuthology.orchestra.run.smithi084.stdout:18.2.0-0.el9 2024-02-28T22:53:37.288 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-28T22:53:37.288 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-28T22:53:37.290 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-28T22:53:37.290 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:53:37.290 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-28T22:53:37.398 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:53:37.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-28T22:53:37.494 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-28T22:53:37.495 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:53:37.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2024-02-28T22:53:37.602 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-28T22:53:37.704 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:53:37.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-02-28T22:53:37.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-28T22:53:37.890 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-28T22:53:37.890 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:53:37.891 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-28T22:53:37.978 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-28T22:53:38.111 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:53:38.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-28T22:53:38.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-28T22:53:38.349 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-28T22:53:38.349 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:53:38.350 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2024-02-28T22:53:38.437 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-28T22:53:38.568 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:53:38.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-02-28T22:53:38.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-28T22:53:38.756 INFO:teuthology.run_tasks:Running task print... 2024-02-28T22:53:38.765 INFO:teuthology.task.print:**** done install task... 2024-02-28T22:53:38.766 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-28T22:53:38.905 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'} 2024-02-28T22:53:38.905 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-02-28T22:53:38.905 INFO:tasks.cephadm:Cluster fsid is 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:53:38.906 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-28T22:53:38.906 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-28T22:53:38.906 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi057': '172.21.15.57', 'mon.smithi084': '172.21.15.84'} 2024-02-28T22:53:38.906 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-28T22:53:38.906 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2024-02-28T22:53:39.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-02-28T22:53:39.101 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-02-28T22:53:39.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T22:53:39.466 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77042/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1670-g1fb5d106', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1670.g1fb5d106'}, 'url': 'https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-23 17:21:14.966217', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-28T22:53:40.843 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 5f501fe72171f96fdc52b866d0d69821e8192eb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-02-28T22:53:40.845 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/5f501fe72171f96fdc52b866d0d69821e8192eb9/centos/9/x86_64/flavors/default/cephadm 2024-02-28T22:53:40.845 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/5f501fe72171f96fdc52b866d0d69821e8192eb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:41.983 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-r--r--. 1 ubuntu ubuntu 214738 Feb 28 22:53 /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:41.984 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/5f501fe72171f96fdc52b866d0d69821e8192eb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:42.853 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 214738 Feb 28 22:53 /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:42.854 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:42.917 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-28T22:53:42.987 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-02-28T22:53:42.987 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-28T22:53:42.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-28T22:53:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-28T22:53:44.231 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-28T22:54:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-28T22:54:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-02-28T22:54:10.977 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-02-28T22:54:11.314 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T22:54:11.349 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2024-02-28T22:54:11.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-02-28T22:54:11.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2024-02-28T22:54:11.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-02-28T22:54:11.755 INFO:tasks.cephadm:Writing seed config... 2024-02-28T22:54:11.756 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-28T22:54:11.756 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-28T22:54:11.757 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-28T22:54:11.758 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-28T22:54:11.759 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-28T22:54:11.760 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:54:11.760 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-28T22:54:11.819 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3640fe00-d68c-11ee-95c3-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-28T22:54:11.819 DEBUG:teuthology.orchestra.run.smithi057:mon.smithi057> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service 2024-02-28T22:54:11.867 INFO:tasks.cephadm:Bootstrapping... 2024-02-28T22:54:11.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-28T22:54:12.119 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-02-28T22:54:12.120 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', '3640fe00-d68c-11ee-95c3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2024-02-28T22:54:12.181 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-28T22:54:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-28T22:54:12.182 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2024-02-28T22:54:12.233 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-28T22:54:12.233 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2024-02-28T22:54:12.233 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2024-02-28T22:54:12.251 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-28T22:54:12.251 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-28T22:54:12.267 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-28T22:54:12.267 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:54:12.287 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-02-28T22:54:12.309 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-02-28T22:54:12.309 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-02-28T22:54:12.309 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2024-02-28T22:54:12.370 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-28T22:54:12.370 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-28T22:54:12.371 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2024-02-28T22:54:12.371 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2024-02-28T22:54:12.389 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-28T22:54:12.389 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-28T22:54:12.411 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-28T22:54:12.411 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:54:12.436 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-02-28T22:54:12.458 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-02-28T22:54:12.459 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-02-28T22:54:12.459 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2024-02-28T22:54:12.459 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:12.459 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 140013102605072 on /run/cephadm/3640fe00-d68c-11ee-95c3-87774f69a715.lock 2024-02-28T22:54:12.460 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 140013102605072 acquired on /run/cephadm/3640fe00-d68c-11ee-95c3-87774f69a715.lock 2024-02-28T22:54:12.461 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 3300 ... 2024-02-28T22:54:12.462 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2024-02-28T22:54:12.462 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:3300, 172.21.15.57:6789], mon addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-02-28T22:54:12.477 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.57 metric 100 2024-02-28T22:54:12.477 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2024-02-28T22:54:12.490 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-28T22:54:12.490 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-28T22:54:12.490 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-02-28T22:54:12.502 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-28T22:54:12.505 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-02-28T22:54:12.505 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-02-28T22:54:12.505 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-28T22:54:12.508 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-28T22:54:12.511 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-02-28T22:54:13.485 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-28T22:54:13.486 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-28T22:54:14.670 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-28T22:54:14.670 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-28T22:54:14.671 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2024-02-28T22:54:15.727 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2024-02-28T22:54:15.727 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2024-02-28T22:54:16.840 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCYud9l8iwHDRAArBMSP0B9/Y/OBHvWuxQdSA== 2024-02-28T22:54:17.960 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCZud9lsfnCDhAAH19U5hO81ZWzpwlpMuRKGg== 2024-02-28T22:54:18.988 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCaud9lW8CwFRAACQ+1ELbLqoxDcgYIekdl6Q== 2024-02-28T22:54:18.989 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for smithi057 [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = pacific 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:20.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-28T22:54:20.060 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:54:20.060 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2024-02-28T22:54:21.178 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.smithi057 on 2024-02-28T22:54:21.569 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-28T22:54:21.824 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-28T22:54:22.134 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target. 2024-02-28T22:54:22.134 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target. 2024-02-28T22:54:22.505 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057 2024-02-28T22:54:22.505 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service: Unit ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service not loaded. 2024-02-28T22:54:22.795 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target.wants/ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service → /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715@.service. 2024-02-28T22:54:23.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 podman[28710]: 2024-02-28 22:54:23.194782223 +0000 UTC m=+0.142895142 container create c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-28T22:54:23.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 podman[28710]: 2024-02-28 22:54:23.489768781 +0000 UTC m=+0.437881733 container init c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0) 2024-02-28T22:54:23.617 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-28T22:54:23.617 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-02-28T22:54:23.634 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-28T22:54:23.634 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:54:23.634 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-02-28T22:54:23.634 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-02-28T22:54:23.635 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2024-02-28T22:54:23.635 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2024-02-28T22:54:23.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 podman[28710]: 2024-02-28 22:54:23.50051896 +0000 UTC m=+0.448631851 container start c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, RELEASE=HEAD) 2024-02-28T22:54:23.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 bash[28710]: c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 2024-02-28T22:54:23.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 ceph-mon[28728]: mkfs 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:23.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 systemd[1]: Started Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T22:54:23.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:23 smithi057 ceph-mon[28728]: mon.smithi057 is new leader, mons smithi057 in quorum (ranks 0) 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: mon.smithi057 is new leader, mons smithi057 in quorum (ranks 0) 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: monmap e1: 1 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: fsmap 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: osdmap e1: 0 total, 0 up, 0 in 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: mgrmap e1: no daemons active 2024-02-28T22:54:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:24 smithi057 ceph-mon[28728]: from='client.? 172.21.15.57:0/1330271518' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-28T22:54:33.187 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2024-02-28T22:54:33.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:33.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-28T22:54:33.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi057 (age 0.74427s) 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:33.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.230+0000 7f851f400700 1 Processor -- start 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.230+0000 7f851f400700 1 -- start start 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.230+0000 7f851f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.231+0000 7f851f400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851810f1f0 con 0x7f8518080210 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.231+0000 7f851ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.232+0000 7f851ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40538/0 (socket says 172.21.15.57:40538) 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.232+0000 7f851ca00700 1 -- 172.21.15.57:0/3757085627 learned_addr learned my addr 172.21.15.57:0/3757085627 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.232+0000 7f851ca00700 1 -- 172.21.15.57:0/3757085627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851810f330 con 0x7f8518080210 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.232+0000 7f851ca00700 1 --2- 172.21.15.57:0/3757085627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f8508009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4e325a325f3c59 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f8517e00700 1 -- 172.21.15.57:0/3757085627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8508004180 con 0x7f8518080210 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f8517e00700 1 -- 172.21.15.57:0/3757085627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080042e0 con 0x7f8518080210 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f8517e00700 1 -- 172.21.15.57:0/3757085627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85080036a0 con 0x7f8518080210 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f851f400700 1 -- 172.21.15.57:0/3757085627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 msgr2=0x7f851810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:33.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f851f400700 1 --2- 172.21.15.57:0/3757085627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f8508009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f851f400700 1 -- 172.21.15.57:0/3757085627 shutdown_connections 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f851f400700 1 --2- 172.21.15.57:0/3757085627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851810ecb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.233+0000 7f851f400700 1 -- 172.21.15.57:0/3757085627 >> 172.21.15.57:0/3757085627 conn(0x7f851807b200 msgr2=0x7f851807d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.234+0000 7f851f400700 1 -- 172.21.15.57:0/3757085627 shutdown_connections 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.234+0000 7f851f400700 1 -- 172.21.15.57:0/3757085627 wait complete. 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851f400700 1 Processor -- start 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851f400700 1 -- start start 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851f400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851819f9d0 con 0x7f8518080210 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40550/0 (socket says 172.21.15.57:40550) 2024-02-28T22:54:33.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.235+0000 7f851ca00700 1 -- 172.21.15.57:0/1330271518 learned_addr learned my addr 172.21.15.57:0/1330271518 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.236+0000 7f851ca00700 1 -- 172.21.15.57:0/1330271518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8508009550 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.236+0000 7f851ca00700 1 --2- 172.21.15.57:0/1330271518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8508004000 tx=0x7f8508004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.236+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8508003ce0 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508020070 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851819fbd0 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8508011630 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85181a0070 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8508022030 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.237+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8508025d00 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.238+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85181992a0 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.241+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8508025480 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.325+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f851802cc30 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.326+0000 7f8516000700 1 -- 172.21.15.57:0/1330271518 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+324 (secure 0 0 0) 0x7f8508011c20 con 0x7f8518080210 2024-02-28T22:54:33.198 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 msgr2=0x7f851819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 --2- 172.21.15.57:0/1330271518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8508004000 tx=0x7f8508004750 comp rx=0 tx=0).stop 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 shutdown_connections 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 --2- 172.21.15.57:0/1330271518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8518080210 0x7f851819f490 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 >> 172.21.15.57:0/1330271518 conn(0x7f851807b200 msgr2=0x7f851807db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 shutdown_connections 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:24.327+0000 7f851f400700 1 -- 172.21.15.57:0/1330271518 wait complete. 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2024-02-28T22:54:33.199 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-28T22:54:34.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.787+0000 7ff722000700 1 Processor -- start 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.787+0000 7ff722000700 1 -- start start 2024-02-28T22:54:34.525 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.788+0000 7ff722000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.788+0000 7ff722000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c108360 con 0x7ff71c107a10 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.788+0000 7ff720c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.789+0000 7ff720c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46254/0 (socket says 172.21.15.57:46254) 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.789+0000 7ff720c00700 1 -- 172.21.15.57:0/3306017030 learned_addr learned my addr 172.21.15.57:0/3306017030 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.789+0000 7ff720c00700 1 -- 172.21.15.57:0/3306017030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c1084a0 con 0x7ff71c107a10 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.790+0000 7ff720c00700 1 --2- 172.21.15.57:0/3306017030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098a0 tx=0x7ff710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e92aa04e3671c0d server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.790+0000 7ff71b400700 1 -- 172.21.15.57:0/3306017030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff710004180 con 0x7ff71c107a10 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.790+0000 7ff71b400700 1 -- 172.21.15.57:0/3306017030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100042e0 con 0x7ff71c107a10 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.790+0000 7ff71b400700 1 -- 172.21.15.57:0/3306017030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7100036a0 con 0x7ff71c107a10 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 -- 172.21.15.57:0/3306017030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 msgr2=0x7ff71c107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 --2- 172.21.15.57:0/3306017030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098a0 tx=0x7ff710009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 -- 172.21.15.57:0/3306017030 shutdown_connections 2024-02-28T22:54:34.526 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 --2- 172.21.15.57:0/3306017030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c107e20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 -- 172.21.15.57:0/3306017030 >> 172.21.15.57:0/3306017030 conn(0x7ff71c07ae80 msgr2=0x7ff71c07b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 -- 172.21.15.57:0/3306017030 shutdown_connections 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.791+0000 7ff722000700 1 -- 172.21.15.57:0/3306017030 wait complete. 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.792+0000 7ff722000700 1 Processor -- start 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff722000700 1 -- start start 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff722000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff722000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c19b3f0 con 0x7ff71c107a10 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff720c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff720c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46270/0 (socket says 172.21.15.57:46270) 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.793+0000 7ff720c00700 1 -- 172.21.15.57:0/603047840 learned_addr learned my addr 172.21.15.57:0/603047840 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff720c00700 1 -- 172.21.15.57:0/603047840 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009550 con 0x7ff71c107a10 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff720c00700 1 --2- 172.21.15.57:0/603047840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff71c108740 tx=0x7ff710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff710003ce0 con 0x7ff71c107a10 2024-02-28T22:54:34.527 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710020070 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff710011630 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.794+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c19b5f0 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.795+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c19ba90 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.795+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff710022030 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.795+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff710025d40 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.796+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c194710 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.799+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff710025480 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.876+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff71c062380 con 0x7ff71c107a10 2024-02-28T22:54:34.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.887+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff71001b450 con 0x7ff71c107a10 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.887+0000 7ff719600700 1 -- 172.21.15.57:0/603047840 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7ff710036020 con 0x7ff71c107a10 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.889+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 msgr2=0x7ff71c19aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.889+0000 7ff722000700 1 --2- 172.21.15.57:0/603047840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff71c108740 tx=0x7ff710004750 comp rx=0 tx=0).stop 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.889+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 shutdown_connections 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.889+0000 7ff722000700 1 --2- 172.21.15.57:0/603047840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c107a10 0x7ff71c19aeb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.889+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 >> 172.21.15.57:0/603047840 conn(0x7ff71c07ae80 msgr2=0x7ff71c18f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.890+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 shutdown_connections 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:33.890+0000 7ff722000700 1 -- 172.21.15.57:0/603047840 wait complete. 2024-02-28T22:54:34.529 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2024-02-28T22:54:34.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:34 smithi057 ceph-mon[28728]: from='client.? 172.21.15.57:0/603047840' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-28T22:54:34.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:34 smithi057 ceph-mon[28728]: from='client.? 172.21.15.57:0/603047840' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-28T22:54:35.762 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:35 smithi057 ceph-mon[28728]: from='client.? 172.21.15.57:0/1494018407' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:54:35.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.092+0000 7f4741a00700 1 Processor -- start 2024-02-28T22:54:35.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.092+0000 7f4741a00700 1 -- start start 2024-02-28T22:54:35.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.093+0000 7f4741a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:35.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.093+0000 7f4741a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c108520 con 0x7f473c107bd0 2024-02-28T22:54:35.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.093+0000 7f473aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:35.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.094+0000 7f473aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51972/0 (socket says 172.21.15.57:51972) 2024-02-28T22:54:35.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.094+0000 7f473aa00700 1 -- 172.21.15.57:0/1659696964 learned_addr learned my addr 172.21.15.57:0/1659696964 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:35.782 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.094+0000 7f473aa00700 1 -- 172.21.15.57:0/1659696964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c108660 con 0x7f473c107bd0 2024-02-28T22:54:35.782 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.094+0000 7f473aa00700 1 --2- 172.21.15.57:0/1659696964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f472c0098a0 tx=0x7f472c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3295334cf889f3a server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:35.782 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4739600700 1 -- 172.21.15.57:0/1659696964 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c004180 con 0x7f473c107bd0 2024-02-28T22:54:35.782 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4739600700 1 -- 172.21.15.57:0/1659696964 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c0042e0 con 0x7f473c107bd0 2024-02-28T22:54:35.783 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4739600700 1 -- 172.21.15.57:0/1659696964 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c011a50 con 0x7f473c107bd0 2024-02-28T22:54:35.783 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4741a00700 1 -- 172.21.15.57:0/1659696964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 msgr2=0x7f473c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:35.783 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4741a00700 1 --2- 172.21.15.57:0/1659696964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f472c0098a0 tx=0x7f472c009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:35.784 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4741a00700 1 -- 172.21.15.57:0/1659696964 shutdown_connections 2024-02-28T22:54:35.784 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4741a00700 1 --2- 172.21.15.57:0/1659696964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c107fe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:35.784 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.095+0000 7f4741a00700 1 -- 172.21.15.57:0/1659696964 >> 172.21.15.57:0/1659696964 conn(0x7f473c07b020 msgr2=0x7f473c07b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:35.784 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.096+0000 7f4741a00700 1 -- 172.21.15.57:0/1659696964 shutdown_connections 2024-02-28T22:54:35.785 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.096+0000 7f4741a00700 1 -- 172.21.15.57:0/1659696964 wait complete. 2024-02-28T22:54:35.785 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f4741a00700 1 Processor -- start 2024-02-28T22:54:35.785 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f4741a00700 1 -- start start 2024-02-28T22:54:35.785 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f4741a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:35.786 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f4741a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19bea0 con 0x7f473c107bd0 2024-02-28T22:54:35.786 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f473aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:35.786 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.097+0000 7f473aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51978/0 (socket says 172.21.15.57:51978) 2024-02-28T22:54:35.786 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.098+0000 7f473aa00700 1 -- 172.21.15.57:0/1494018407 learned_addr learned my addr 172.21.15.57:0/1494018407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:35.787 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.098+0000 7f473aa00700 1 -- 172.21.15.57:0/1494018407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c009550 con 0x7f473c107bd0 2024-02-28T22:54:35.787 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.098+0000 7f473aa00700 1 --2- 172.21.15.57:0/1494018407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f472c003de0 tx=0x7f472c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:35.787 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c003930 con 0x7f473c107bd0 2024-02-28T22:54:35.787 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c003a90 con 0x7f473c107bd0 2024-02-28T22:54:35.788 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c019730 con 0x7f473c107bd0 2024-02-28T22:54:35.788 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c19c0a0 con 0x7f473c107bd0 2024-02-28T22:54:35.788 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c19c540 con 0x7f473c107bd0 2024-02-28T22:54:35.788 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f472c019960 con 0x7f473c107bd0 2024-02-28T22:54:35.789 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.099+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f472c024b20 con 0x7f473c107bd0 2024-02-28T22:54:35.789 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.100+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c10c060 con 0x7f473c107bd0 2024-02-28T22:54:35.789 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.103+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f472c042b00 con 0x7f473c107bd0 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.182+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f473c109ac0 con 0x7f473c107bd0 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.182+0000 7f4733400700 1 -- 172.21.15.57:0/1494018407 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f472c031030 con 0x7f473c107bd0 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 msgr2=0x7f473c19b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 --2- 172.21.15.57:0/1494018407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f472c003de0 tx=0x7f472c011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 shutdown_connections 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 --2- 172.21.15.57:0/1494018407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f473c107bd0 0x7f473c19b960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:35.790 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 >> 172.21.15.57:0/1494018407 conn(0x7f473c07b020 msgr2=0x7f473c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:35.791 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 shutdown_connections 2024-02-28T22:54:35.791 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:35.184+0000 7f4741a00700 1 -- 172.21.15.57:0/1494018407 wait complete. 2024-02-28T22:54:35.791 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2024-02-28T22:54:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:35 smithi057 systemd[1]: Stopping Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T22:54:36.661 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[28724]: 2024-02-28T22:54:36.322+0000 7f2705600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi057 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T22:54:36.661 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[28724]: 2024-02-28T22:54:36.322+0000 7f2705600700 -1 mon.smithi057@0(leader) e1 *** Got Signal Terminated *** 2024-02-28T22:54:36.662 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 podman[28971]: 2024-02-28 22:54:36.395286985 +0000 UTC m=+0.201928618 container died c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-02-28T22:54:36.940 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 podman[28971]: 2024-02-28 22:54:36.782773386 +0000 UTC m=+0.589415059 container cleanup c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, io.buildah.version=1.29.1) 2024-02-28T22:54:36.940 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 bash[28971]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057 2024-02-28T22:54:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:36 smithi057 podman[28985]: 2024-02-28 22:54:36.94183146 +0000 UTC m=+0.528838876 container remove c73aded981e536e8c7c1efe539dff65d6313d6351b3747b8792e6c7838648663 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.license=GPLv2) 2024-02-28T22:54:37.765 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service: Deactivated successfully. 2024-02-28T22:54:37.765 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 systemd[1]: Stopped Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T22:54:37.765 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service: Consumed 1.968s CPU time. 2024-02-28T22:54:37.765 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 systemd[1]: Starting Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T22:54:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 podman[29078]: 2024-02-28 22:54:37.672452246 +0000 UTC m=+0.035198607 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-02-28T22:54:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:37 smithi057 podman[29078]: 2024-02-28 22:54:37.792801356 +0000 UTC m=+0.155547743 container create 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0) 2024-02-28T22:54:39.699 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 podman[29078]: 2024-02-28 22:54:39.593423372 +0000 UTC m=+1.956169793 container init 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 podman[29078]: 2024-02-28 22:54:39.606789024 +0000 UTC m=+1.969535375 container start 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: set uid:gid to 167:167 (ceph:ceph) 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: pidfile_write: ignore empty --pid-file 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: load: jerasure load: lrc 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: RocksDB version: 7.9.2 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Git sha 0 2024-02-28T22:54:39.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: DB SUMMARY 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: DB Session ID: WGUM5EV1AQL0AOHD2E9W 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: CURRENT file: CURRENT 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: IDENTITY file: IDENTITY 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi057/store.db dir, Total Num: 1, files: 000008.sst 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi057/store.db: 000009.log size: 98401 ; 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.error_if_exists: 0 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.create_if_missing: 0 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.paranoid_checks: 1 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-28T22:54:39.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.env: 0x56347a93a720 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.fs: PosixFileSystem 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.info_log: 0x56347d7512c0 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_file_opening_threads: 16 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.statistics: (nil) 2024-02-28T22:54:39.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.use_fsync: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_log_file_size: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.keep_log_file_num: 1000 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.recycle_log_file_num: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_fallocate: 1 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_mmap_reads: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_mmap_writes: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.use_direct_reads: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.create_missing_column_families: 0 2024-02-28T22:54:39.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.db_log_dir: 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.wal_dir: 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.advise_random_on_open: 1 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.db_write_buffer_size: 0 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.write_buffer_manager: 0x56347c9c23c0 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-28T22:54:39.850 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.rate_limiter: (nil) 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.wal_recovery_mode: 2 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enable_thread_tracking: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enable_pipelined_write: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.unordered_write: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.row_cache: None 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.wal_filter: None 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_ingest_behind: 0 2024-02-28T22:54:39.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.two_write_queues: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.manual_wal_flush: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.wal_compression: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.atomic_flush: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.log_readahead_size: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.best_efforts_recovery: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.allow_data_in_errors: 0 2024-02-28T22:54:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.db_host_id: __hostname__ 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_background_jobs: 2 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_background_compactions: -1 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_subcompactions: 1 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_total_wal_size: 0 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-28T22:54:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_open_files: -1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bytes_per_sync: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_readahead_size: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_background_flushes: -1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Compression algorithms supported: 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kZSTD supported: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kXpressCompression supported: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kLZ4HCCompression supported: 1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kZlibCompression supported: 1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kSnappyCompression supported: 1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kLZ4Compression supported: 1 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: kBZip2Compression supported: 0 2024-02-28T22:54:39.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi057/store.db/MANIFEST-000010 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.merge_operator: 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_filter: None 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_filter_factory: None 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.sst_partitioner_factory: None 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56347d7513c0) 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: cache_index_and_filter_blocks: 1 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-28T22:54:39.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: pin_top_level_index_and_filter: 1 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_type: 0 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: data_block_index_type: 0 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_shortening: 1 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: checksum: 4 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: no_block_cache: 0 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache: 0x56347ca63350 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_name: BinnedLRUCache 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_options: 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: capacity : 536870912 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: num_shard_bits : 4 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: strict_capacity_limit : 0 2024-02-28T22:54:39.856 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: high_pri_pool_ratio: 0.000 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_compressed: (nil) 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: persistent_cache: (nil) 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_size: 4096 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_size_deviation: 10 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_restart_interval: 16 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_block_restart_interval: 1 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: metadata_block_size: 4096 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: partition_filters: 0 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: use_delta_encoding: 1 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: filter_policy: bloomfilter 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: whole_key_filtering: 1 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: verify_compression: 0 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: read_amp_bytes_per_bit: 0 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: format_version: 5 2024-02-28T22:54:39.857 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: enable_index_compression: 1 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_align: 0 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: max_auto_readahead_size: 262144 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: prepopulate_block_cache: 0 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: initial_auto_readahead_size: 8192 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.write_buffer_size: 33554432 2024-02-28T22:54:39.858 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_write_buffer_number: 2 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression: NoCompression 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression: Disabled 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.prefix_extractor: nullptr 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.num_levels: 7 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-28T22:54:39.859 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.level: 32767 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.strategy: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-28T22:54:39.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.enabled: false 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.target_file_size_base: 67108864 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-28T22:54:39.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.arena_block_size: 1048576 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.disable_auto_compactions: 0 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-28T22:54:39.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.table_properties_collectors: 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.inplace_update_support: 0 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.bloom_locality: 0 2024-02-28T22:54:39.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.max_successive_merges: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.paranoid_file_checks: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.force_consistency_checks: 1 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.report_bg_io_stats: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.ttl: 2592000 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enable_blob_files: false 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.min_blob_size: 0 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_file_size: 268435456 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-28T22:54:39.864 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.blob_file_starting_level: 0 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi057/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c66d57c8-b843-4c5a-aecf-6c5cd94005f7 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709160879666917, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709160879670102, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 91274, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13388, "raw_average_key_size": 49, "raw_value_size": 83962, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709160879, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c66d57c8-b843-4c5a-aecf-6c5cd94005f7", "db_session_id": "WGUM5EV1AQL0AOHD2E9W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709160879670336, "job": 1, "event": "recovery_finished"} 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi057/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56347cb00000 2024-02-28T22:54:39.865 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: DB pointer 0x56347caec000 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** DB Stats ** 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** Compaction Stats [default] ** 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T22:54:39.866 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: L0 2/0 92.94 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Sum 2/0 92.94 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** Compaction Stats [default] ** 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-28T22:54:39.867 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative compaction: 0.00 GB write, 8.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval compaction: 0.00 GB write, 8.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Block cache BinnedLRUCache@0x56347ca63350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: starting mon.smithi057 rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi057 fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???) e1 preinit fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:39.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).mds e1 new map 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).mds e1 print_map 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: e1 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: legacy client fscid: -1 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: No filesystems configured 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expand map: {default=false} 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta from 'false' to 'false' 2024-02-28T22:54:39.869 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expanded map: {default=false} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expand map: {default=info} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta from 'info' to 'info' 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expanded map: {default=info} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expand map: {default=daemon} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expanded map: {default=daemon} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expand map: {default=debug} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta from 'debug' to 'debug' 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: expand_channel_meta expanded map: {default=debug} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 bash[29078]: 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 systemd[1]: Started Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mon.smithi057 is new leader, mons smithi057 in quorum (ranks 0) 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: monmap e1: 1 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2024-02-28T22:54:39.870 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: fsmap 2024-02-28T22:54:39.871 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: osdmap e1: 0 total, 0 up, 0 in 2024-02-28T22:54:39.871 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:39 smithi057 ceph-mon[29096]: mgrmap e1: no daemons active 2024-02-28T22:54:41.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.336+0000 7f032ee00700 1 Processor -- start 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.337+0000 7f032ee00700 1 -- start start 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.337+0000 7f032ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.337+0000 7f032ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032807a7d0 con 0x7f032807bd90 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.338+0000 7f0327e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.338+0000 7f0327e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51992/0 (socket says 172.21.15.57:51992) 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.338+0000 7f0327e00700 1 -- 172.21.15.57:0/3830706210 learned_addr learned my addr 172.21.15.57:0/3830706210 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:41.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.338+0000 7f0327e00700 1 -- 172.21.15.57:0/3830706210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032807a910 con 0x7f032807bd90 2024-02-28T22:54:41.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f0327e00700 1 --2- 172.21.15.57:0/3830706210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0318009870 tx=0x7f0318009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5952c3bdac183125 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:41.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f0326a00700 1 -- 172.21.15.57:0/3830706210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0318005c30 con 0x7f032807bd90 2024-02-28T22:54:41.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f0326a00700 1 -- 172.21.15.57:0/3830706210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0318005d90 con 0x7f032807bd90 2024-02-28T22:54:41.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f0326a00700 1 -- 172.21.15.57:0/3830706210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0318006030 con 0x7f032807bd90 2024-02-28T22:54:41.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f032ee00700 1 -- 172.21.15.57:0/3830706210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 msgr2=0x7f032807a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.339+0000 7f032ee00700 1 --2- 172.21.15.57:0/3830706210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0318009870 tx=0x7f0318009b80 comp rx=0 tx=0).stop 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.340+0000 7f032ee00700 1 -- 172.21.15.57:0/3830706210 shutdown_connections 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.340+0000 7f032ee00700 1 --2- 172.21.15.57:0/3830706210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032807a290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.340+0000 7f032ee00700 1 -- 172.21.15.57:0/3830706210 >> 172.21.15.57:0/3830706210 conn(0x7f0328101020 msgr2=0x7f0328103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.340+0000 7f032ee00700 1 -- 172.21.15.57:0/3830706210 shutdown_connections 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.340+0000 7f032ee00700 1 -- 172.21.15.57:0/3830706210 wait complete. 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.341+0000 7f032ee00700 1 Processor -- start 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.341+0000 7f032ee00700 1 -- start start 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.341+0000 7f032ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.341+0000 7f032ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819be50 con 0x7f032807bd90 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.342+0000 7f0327e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.342+0000 7f0327e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52006/0 (socket says 172.21.15.57:52006) 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.342+0000 7f0327e00700 1 -- 172.21.15.57:0/4191629232 learned_addr learned my addr 172.21.15.57:0/4191629232 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:41.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.342+0000 7f0327e00700 1 -- 172.21.15.57:0/4191629232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318009500 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.342+0000 7f0327e00700 1 --2- 172.21.15.57:0/4191629232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f03180038f0 tx=0x7f03180061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0318004310 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0318004470 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0318011c90 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032819c050 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032819c4f0 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0318018070 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.343+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f03180249f0 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.344+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0328062380 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.347+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0318016070 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.425+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f032819f240 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.431+0000 7f0324c00700 1 -- 172.21.15.57:0/4191629232 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0318024bb0 con 0x7f032807bd90 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 msgr2=0x7f032819b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 --2- 172.21.15.57:0/4191629232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f03180038f0 tx=0x7f03180061e0 comp rx=0 tx=0).stop 2024-02-28T22:54:41.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 shutdown_connections 2024-02-28T22:54:41.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 --2- 172.21.15.57:0/4191629232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f032807bd90 0x7f032819b910 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:41.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 >> 172.21.15.57:0/4191629232 conn(0x7f0328101020 msgr2=0x7f03281026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:41.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 shutdown_connections 2024-02-28T22:54:41.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:40.433+0000 7f032ee00700 1 -- 172.21.15.57:0/4191629232 wait complete. 2024-02-28T22:54:41.080 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-28T22:54:41.122 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-28T22:54:41.122 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2024-02-28T22:54:41.122 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-28T22:54:41.123 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-28T22:54:41.123 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8443 ... 2024-02-28T22:54:41.566 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mgr.smithi057.zrrldj 2024-02-28T22:54:41.566 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mgr.smithi057.zrrldj.service: Unit ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mgr.smithi057.zrrldj.service not loaded. 2024-02-28T22:54:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:41 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4191629232' entity='client.admin' 2024-02-28T22:54:41.829 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715.target.wants/ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mgr.smithi057.zrrldj.service → /etc/systemd/system/ceph-3640fe00-d68c-11ee-95c3-87774f69a715@.service. 2024-02-28T22:54:42.649 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-28T22:54:42.649 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-02-28T22:54:42.667 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-28T22:54:42.668 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:54:42.668 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-02-28T22:54:42.668 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-02-28T22:54:42.692 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-28T22:54:42.692 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-02-28T22:54:42.713 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-28T22:54:42.714 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:54:42.714 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-02-28T22:54:42.715 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-02-28T22:54:42.715 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2024-02-28T22:54:42.715 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2024-02-28T22:54:43.739 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:43 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1039862881' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3640fe00-d68c-11ee-95c3-87774f69a715", 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-02-28T22:54:44.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "smithi057" 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-28T22:54:44.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:44.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-28T22:54:44.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-28T22:54:44.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-28T22:54:44.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-28T22:54:44.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-02-28T22:54:23.586006+0000", 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.288+0000 7fb8e3e00700 1 Processor -- start 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.288+0000 7fb8e3e00700 1 -- start start 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.288+0000 7fb8e3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.288+0000 7fb8e3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40744b0 con 0x7fb8e4104160 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.289+0000 7fb8e2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.289+0000 7fb8e2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52026/0 (socket says 172.21.15.57:52026) 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.289+0000 7fb8e2a00700 1 -- 172.21.15.57:0/783317160 learned_addr learned my addr 172.21.15.57:0/783317160 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.290+0000 7fb8e2a00700 1 -- 172.21.15.57:0/783317160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40745f0 con 0x7fb8e4104160 2024-02-28T22:54:44.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.290+0000 7fb8e2a00700 1 --2- 172.21.15.57:0/783317160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098a0 tx=0x7fb8d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b72c285e8b78d03 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.290+0000 7fb8e1600700 1 -- 172.21.15.57:0/783317160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d4004180 con 0x7fb8e4104160 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.290+0000 7fb8e1600700 1 -- 172.21.15.57:0/783317160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8d40042e0 con 0x7fb8e4104160 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.290+0000 7fb8e1600700 1 -- 172.21.15.57:0/783317160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d4011a50 con 0x7fb8e4104160 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 -- 172.21.15.57:0/783317160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 msgr2=0x7fb8e4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 --2- 172.21.15.57:0/783317160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098a0 tx=0x7fb8d4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 -- 172.21.15.57:0/783317160 shutdown_connections 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 --2- 172.21.15.57:0/783317160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e4104570 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 -- 172.21.15.57:0/783317160 >> 172.21.15.57:0/783317160 conn(0x7fb8e40ffd30 msgr2=0x7fb8e4102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 -- 172.21.15.57:0/783317160 shutdown_connections 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.291+0000 7fb8e3e00700 1 -- 172.21.15.57:0/783317160 wait complete. 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.292+0000 7fb8e3e00700 1 Processor -- start 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.292+0000 7fb8e3e00700 1 -- start start 2024-02-28T22:54:44.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e419bcc0 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52042/0 (socket says 172.21.15.57:52042) 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e2a00700 1 -- 172.21.15.57:0/1039862881 learned_addr learned my addr 172.21.15.57:0/1039862881 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.293+0000 7fb8e2a00700 1 -- 172.21.15.57:0/1039862881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009550 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8e2a00700 1 --2- 172.21.15.57:0/1039862881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4003de0 tx=0x7fb8d4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d4003930 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8d4003a90 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d40196c0 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e419bec0 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e419c360 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.294+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb8d4019c30 con 0x7fb8e4104160 2024-02-28T22:54:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.295+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb8d402c3e0 con 0x7fb8e4104160 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.295+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e4194fc0 con 0x7fb8e4104160 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.298+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb8d4042b00 con 0x7fb8e4104160 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.377+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb8e402cc60 con 0x7fb8e4104160 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.378+0000 7fb8db400700 1 -- 172.21.15.57:0/1039862881 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb8d4031020 con 0x7fb8e4104160 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 msgr2=0x7fb8e419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 --2- 172.21.15.57:0/1039862881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4003de0 tx=0x7fb8d4011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 shutdown_connections 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 --2- 172.21.15.57:0/1039862881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4104160 0x7fb8e419b780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 >> 172.21.15.57:0/1039862881 conn(0x7fb8e40ffd30 msgr2=0x7fb8e418fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 shutdown_connections 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:43.379+0000 7fb8e3e00700 1 -- 172.21.15.57:0/1039862881 wait complete. 2024-02-28T22:54:44.074 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2024-02-28T22:54:47.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:46 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2105377387' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3640fe00-d68c-11ee-95c3-87774f69a715", 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "smithi057" 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-28T22:54:47.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-28T22:54:47.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:47.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-28T22:54:47.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-28T22:54:47.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-28T22:54:47.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-28T22:54:47.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-02-28T22:54:23.586006+0000", 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:54:47.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.671+0000 7f86e1800700 1 Processor -- start 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.671+0000 7f86e1800700 1 -- start start 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.672+0000 7f86e1800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.672+0000 7f86e1800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc079720 con 0x7f86dc07ace0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.672+0000 7f86daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.672+0000 7f86daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35810/0 (socket says 172.21.15.57:35810) 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.673+0000 7f86daa00700 1 -- 172.21.15.57:0/3601752426 learned_addr learned my addr 172.21.15.57:0/3601752426 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.673+0000 7f86daa00700 1 -- 172.21.15.57:0/3601752426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc079860 con 0x7f86dc07ace0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.673+0000 7f86daa00700 1 --2- 172.21.15.57:0/3601752426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0098a0 tx=0x7f86cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbb87b1daa6a3bc server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86da000700 1 -- 172.21.15.57:0/3601752426 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86cc004180 con 0x7f86dc07ace0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86da000700 1 -- 172.21.15.57:0/3601752426 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86cc0042e0 con 0x7f86dc07ace0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86da000700 1 -- 172.21.15.57:0/3601752426 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86cc011a50 con 0x7f86dc07ace0 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 -- 172.21.15.57:0/3601752426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 msgr2=0x7f86dc0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 --2- 172.21.15.57:0/3601752426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0098a0 tx=0x7f86cc009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:47.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 -- 172.21.15.57:0/3601752426 shutdown_connections 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 --2- 172.21.15.57:0/3601752426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc0791e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 -- 172.21.15.57:0/3601752426 >> 172.21.15.57:0/3601752426 conn(0x7f86dc100fd0 msgr2=0x7f86dc103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.674+0000 7f86e1800700 1 -- 172.21.15.57:0/3601752426 shutdown_connections 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.675+0000 7f86e1800700 1 -- 172.21.15.57:0/3601752426 wait complete. 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.675+0000 7f86e1800700 1 Processor -- start 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86e1800700 1 -- start start 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86e1800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86e1800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19be90 con 0x7f86dc07ace0 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35826/0 (socket says 172.21.15.57:35826) 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.676+0000 7f86daa00700 1 -- 172.21.15.57:0/2105377387 learned_addr learned my addr 172.21.15.57:0/2105377387 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86daa00700 1 -- 172.21.15.57:0/2105377387 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc009550 con 0x7f86dc07ace0 2024-02-28T22:54:47.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86daa00700 1 --2- 172.21.15.57:0/2105377387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86cc003de0 tx=0x7f86cc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86cc003930 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86cc003a90 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc19c090 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86cc0196c0 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.677+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc19c530 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.678+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86cc019c30 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.678+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f86cc02c3a0 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.678+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc062380 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.681+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f86cc01abc0 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.762+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f86dc19f0b0 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.763+0000 7f86d3400700 1 -- 172.21.15.57:0/2105377387 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f86cc005a20 con 0x7f86dc07ace0 2024-02-28T22:54:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 msgr2=0x7f86dc19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 --2- 172.21.15.57:0/2105377387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86cc003de0 tx=0x7f86cc011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 shutdown_connections 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 --2- 172.21.15.57:0/2105377387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86dc07ace0 0x7f86dc19b950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 >> 172.21.15.57:0/2105377387 conn(0x7f86dc100fd0 msgr2=0x7f86dc1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 shutdown_connections 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:46.764+0000 7f86e1800700 1 -- 172.21.15.57:0/2105377387 wait complete. 2024-02-28T22:54:47.477 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2024-02-28T22:54:50.780 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:50 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1770089584' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-28T22:54:50.791 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:50.792 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:54:50.792 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3640fe00-d68c-11ee-95c3-87774f69a715", 2024-02-28T22:54:50.792 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-02-28T22:54:50.792 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-28T22:54:50.793 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-28T22:54:50.793 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-28T22:54:50.794 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "smithi057" 2024-02-28T22:54:50.795 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:50.795 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-02-28T22:54:50.795 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-28T22:54:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.802 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-28T22:54:50.803 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-02-28T22:54:23.586006+0000", 2024-02-28T22:54:50.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.073+0000 7feb14800700 1 Processor -- start 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.074+0000 7feb14800700 1 -- start start 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.074+0000 7feb14800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.074+0000 7feb14800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c074510 con 0x7feb0c1042e0 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.075+0000 7feb11e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.075+0000 7feb11e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35840/0 (socket says 172.21.15.57:35840) 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.075+0000 7feb11e00700 1 -- 172.21.15.57:0/1455009804 learned_addr learned my addr 172.21.15.57:0/1455009804 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.075+0000 7feb11e00700 1 -- 172.21.15.57:0/1455009804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c074650 con 0x7feb0c1042e0 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb11e00700 1 --2- 172.21.15.57:0/1455009804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb000098a0 tx=0x7feb00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39662b505fb374f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:50.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb10a00700 1 -- 172.21.15.57:0/1455009804 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb00004180 con 0x7feb0c1042e0 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb10a00700 1 -- 172.21.15.57:0/1455009804 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb000042e0 con 0x7feb0c1042e0 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb10a00700 1 -- 172.21.15.57:0/1455009804 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb00011a50 con 0x7feb0c1042e0 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb14800700 1 -- 172.21.15.57:0/1455009804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 msgr2=0x7feb0c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.076+0000 7feb14800700 1 --2- 172.21.15.57:0/1455009804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb000098a0 tx=0x7feb00009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.077+0000 7feb14800700 1 -- 172.21.15.57:0/1455009804 shutdown_connections 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.077+0000 7feb14800700 1 --2- 172.21.15.57:0/1455009804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c106700 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.077+0000 7feb14800700 1 -- 172.21.15.57:0/1455009804 >> 172.21.15.57:0/1455009804 conn(0x7feb0c0fff10 msgr2=0x7feb0c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.077+0000 7feb14800700 1 -- 172.21.15.57:0/1455009804 shutdown_connections 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.077+0000 7feb14800700 1 -- 172.21.15.57:0/1455009804 wait complete. 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.078+0000 7feb14800700 1 Processor -- start 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.078+0000 7feb14800700 1 -- start start 2024-02-28T22:54:50.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.078+0000 7feb14800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.078+0000 7feb14800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1a0290 con 0x7feb0c1042e0 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.078+0000 7feb11e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.079+0000 7feb11e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35856/0 (socket says 172.21.15.57:35856) 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.079+0000 7feb11e00700 1 -- 172.21.15.57:0/1770089584 learned_addr learned my addr 172.21.15.57:0/1770089584 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.079+0000 7feb11e00700 1 -- 172.21.15.57:0/1770089584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00009550 con 0x7feb0c1042e0 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.079+0000 7feb11e00700 1 --2- 172.21.15.57:0/1770089584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb00003de0 tx=0x7feb00011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:50.809 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.080+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb00003930 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.080+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb00003a90 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.080+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb000196c0 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.080+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c1a0490 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.080+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1a0930 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.081+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feb0001a430 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.081+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feb00024410 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.081+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c107890 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.084+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7feb00019e10 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.163+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feb0c107a80 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.164+0000 7feb06a00700 1 -- 172.21.15.57:0/1770089584 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7feb0001a7d0 con 0x7feb0c1042e0 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 msgr2=0x7feb0c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 --2- 172.21.15.57:0/1770089584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb00003de0 tx=0x7feb00011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 shutdown_connections 2024-02-28T22:54:50.810 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 --2- 172.21.15.57:0/1770089584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb0c1042e0 0x7feb0c19fd50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:50.811 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 >> 172.21.15.57:0/1770089584 conn(0x7feb0c0fff10 msgr2=0x7feb0c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:50.811 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 shutdown_connections 2024-02-28T22:54:50.811 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:50.166+0000 7feb14800700 1 -- 172.21.15.57:0/1770089584 wait complete. 2024-02-28T22:54:50.811 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (3/15)... 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: Activating manager daemon smithi057.zrrldj 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: mgrmap e2: smithi057.zrrldj(active, starting, since 0.0112121s) 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: Manager daemon smithi057.zrrldj is now available 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' 2024-02-28T22:54:52.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' 2024-02-28T22:54:52.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:51 smithi057 ceph-mon[29096]: from='mgr.14100 172.21.15.57:0/3319940811' entity='mgr.smithi057.zrrldj' 2024-02-28T22:54:53.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:53 smithi057 ceph-mon[29096]: mgrmap e3: smithi057.zrrldj(active, since 1.02235s) 2024-02-28T22:54:54.437 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:54 smithi057 ceph-mon[29096]: mgrmap e4: smithi057.zrrldj(active, since 2s) 2024-02-28T22:54:54.437 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:54 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3414634992' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3640fe00-d68c-11ee-95c3-87774f69a715", 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-28T22:54:54.448 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "smithi057" 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-28T22:54:54.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:54.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-28T22:54:54.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-28T22:54:54.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-28T22:54:54.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-28T22:54:54.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-02-28T22:54:23.586006+0000", 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.477+0000 7fd9f2000700 1 Processor -- start 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.477+0000 7fd9f2000700 1 -- start start 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.477+0000 7fd9f2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:54.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.477+0000 7fd9f2000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec080220 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.478+0000 7fd9f0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.478+0000 7fd9f0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35912/0 (socket says 172.21.15.57:35912) 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.478+0000 7fd9f0c00700 1 -- 172.21.15.57:0/1437922892 learned_addr learned my addr 172.21.15.57:0/1437922892 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.478+0000 7fd9f0c00700 1 -- 172.21.15.57:0/1437922892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec080360 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9f0c00700 1 --2- 172.21.15.57:0/1437922892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00098a0 tx=0x7fd9e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe313291271ce96f server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9eb400700 1 -- 172.21.15.57:0/1437922892 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e0004180 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9eb400700 1 -- 172.21.15.57:0/1437922892 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9e00042e0 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9eb400700 1 -- 172.21.15.57:0/1437922892 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e0011a50 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9f2000700 1 -- 172.21.15.57:0/1437922892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 msgr2=0x7fd9ec07fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.479+0000 7fd9f2000700 1 --2- 172.21.15.57:0/1437922892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00098a0 tx=0x7fd9e0009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.480+0000 7fd9f2000700 1 -- 172.21.15.57:0/1437922892 shutdown_connections 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.480+0000 7fd9f2000700 1 --2- 172.21.15.57:0/1437922892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec07fce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.480+0000 7fd9f2000700 1 -- 172.21.15.57:0/1437922892 >> 172.21.15.57:0/1437922892 conn(0x7fd9ec07ae80 msgr2=0x7fd9ec07d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:54.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.480+0000 7fd9f2000700 1 -- 172.21.15.57:0/1437922892 shutdown_connections 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.480+0000 7fd9f2000700 1 -- 172.21.15.57:0/1437922892 wait complete. 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.481+0000 7fd9f2000700 1 Processor -- start 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.481+0000 7fd9f2000700 1 -- start start 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.481+0000 7fd9f2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.481+0000 7fd9f2000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1a00b0 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.482+0000 7fd9f0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.482+0000 7fd9f0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35922/0 (socket says 172.21.15.57:35922) 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.482+0000 7fd9f0c00700 1 -- 172.21.15.57:0/3414634992 learned_addr learned my addr 172.21.15.57:0/3414634992 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.482+0000 7fd9f0c00700 1 -- 172.21.15.57:0/3414634992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e0009550 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.482+0000 7fd9f0c00700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.483+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e0003970 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.483+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9e0003ad0 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.483+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e0024750 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.483+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec1a02b0 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.483+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec1a0750 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.484+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec10dc80 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.484+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fd9e001a430 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.485+0000 7fd9e9600700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 0x7fd9d403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.485+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd9e004bc70 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.485+0000 7fd9ebe00700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 0x7fd9d403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.486+0000 7fd9ebe00700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 0x7fd9d403a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.491+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd9e0018530 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.797+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd9ec080360 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.798+0000 7fd9e9600700 1 -- 172.21.15.57:0/3414634992 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fd9ec080360 con 0x7fd9ec07f8d0 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 msgr2=0x7fd9d403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 0x7fd9d403a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).stop 2024-02-28T22:54:54.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 msgr2=0x7fd9ec19fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e0011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 shutdown_connections 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fd9d40383f0 0x7fd9d403a8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 --2- 172.21.15.57:0/3414634992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9ec07f8d0 0x7fd9ec19fb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 >> 172.21.15.57:0/3414634992 conn(0x7fd9ec07ae80 msgr2=0x7fd9ec07c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.801+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 shutdown_connections 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:53.802+0000 7fd9f2000700 1 -- 172.21.15.57:0/3414634992 wait complete. 2024-02-28T22:54:54.465 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2024-02-28T22:54:56.061 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:55 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2559728565' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-28T22:54:56.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.103+0000 7f9350600700 1 Processor -- start 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.103+0000 7f9350600700 1 -- start start 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.103+0000 7f9350600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.103+0000 7f9350600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480796e0 con 0x7f934807aca0 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.104+0000 7f934dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.104+0000 7f934dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51068/0 (socket says 172.21.15.57:51068) 2024-02-28T22:54:56.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.104+0000 7f934dc00700 1 -- 172.21.15.57:0/1773064005 learned_addr learned my addr 172.21.15.57:0/1773064005 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.104+0000 7f934dc00700 1 -- 172.21.15.57:0/1773064005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348079820 con 0x7f934807aca0 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.105+0000 7f934dc00700 1 --2- 172.21.15.57:0/1773064005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c07d950be83304 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.105+0000 7f9347e00700 1 -- 172.21.15.57:0/1773064005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c004180 con 0x7f934807aca0 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.105+0000 7f9347e00700 1 -- 172.21.15.57:0/1773064005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f933c0042e0 con 0x7f934807aca0 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.105+0000 7f9347e00700 1 -- 172.21.15.57:0/1773064005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c011a50 con 0x7f934807aca0 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 -- 172.21.15.57:0/1773064005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 msgr2=0x7f93480791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 --2- 172.21.15.57:0/1773064005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 -- 172.21.15.57:0/1773064005 shutdown_connections 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 --2- 172.21.15.57:0/1773064005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f93480791a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 -- 172.21.15.57:0/1773064005 >> 172.21.15.57:0/1773064005 conn(0x7f9348100fd0 msgr2=0x7f9348103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 -- 172.21.15.57:0/1773064005 shutdown_connections 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.106+0000 7f9350600700 1 -- 172.21.15.57:0/1773064005 wait complete. 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.108+0000 7f9350600700 1 Processor -- start 2024-02-28T22:54:56.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.108+0000 7f9350600700 1 -- start start 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.108+0000 7f9350600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.108+0000 7f9350600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934819bec0 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.109+0000 7f934dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.109+0000 7f934dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51084/0 (socket says 172.21.15.57:51084) 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.109+0000 7f934dc00700 1 -- 172.21.15.57:0/2559728565 learned_addr learned my addr 172.21.15.57:0/2559728565 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.109+0000 7f934dc00700 1 -- 172.21.15.57:0/2559728565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c009550 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f934dc00700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f933c003de0 tx=0x7f933c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c0038b0 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f933c003a10 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c019600 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f9350600700 1 -- 172.21.15.57:0/2559728565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934819c0c0 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.110+0000 7f9350600700 1 -- 172.21.15.57:0/2559728565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934819c560 con 0x7f934807aca0 2024-02-28T22:54:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.112+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93240052f0 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.112+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f933c01a430 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.112+0000 7f9346000700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 0x7f933003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.112+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f933c04bee0 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.112+0000 7f934d200700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 0x7f933003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.113+0000 7f934d200700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 0x7f933003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9334006fd0 tx=0x7f9334006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.117+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f933c018b00 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.340+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9324005f40 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.341+0000 7f9346000700 1 -- 172.21.15.57:0/2559728565 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f933c018b00 con 0x7f934807aca0 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.344+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 msgr2=0x7f933003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.344+0000 7f933b400700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 0x7f933003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9334006fd0 tx=0x7f9334006e40 comp rx=0 tx=0).stop 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.344+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 msgr2=0x7f934819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.344+0000 7f933b400700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f933c003de0 tx=0x7f933c0047c0 comp rx=0 tx=0).stop 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 shutdown_connections 2024-02-28T22:54:56.081 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f93300383a0 0x7f933003a850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:56.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 --2- 172.21.15.57:0/2559728565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934807aca0 0x7f934819b980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:56.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 >> 172.21.15.57:0/2559728565 conn(0x7f9348100fd0 msgr2=0x7f93481026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:56.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 shutdown_connections 2024-02-28T22:54:56.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:55.345+0000 7f933b400700 1 -- 172.21.15.57:0/2559728565 wait complete. 2024-02-28T22:54:56.082 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2024-02-28T22:54:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4195479611' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-28T22:54:57.764 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.731+0000 7fcb40800700 1 Processor -- start 2024-02-28T22:54:57.764 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.731+0000 7fcb40800700 1 -- start start 2024-02-28T22:54:57.764 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.732+0000 7fcb40800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:57.764 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.732+0000 7fcb40800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3810f1f0 con 0x7fcb38080210 2024-02-28T22:54:57.764 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.732+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.732+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51086/0 (socket says 172.21.15.57:51086) 2024-02-28T22:54:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.732+0000 7fcb3de00700 1 -- 172.21.15.57:0/3735128162 learned_addr learned my addr 172.21.15.57:0/3735128162 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.733+0000 7fcb3de00700 1 -- 172.21.15.57:0/3735128162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3810f330 con 0x7fcb38080210 2024-02-28T22:54:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.733+0000 7fcb3de00700 1 --2- 172.21.15.57:0/3735128162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098a0 tx=0x7fcb2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6750c98234a3448 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.733+0000 7fcb3ca00700 1 -- 172.21.15.57:0/3735128162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb2c004180 con 0x7fcb38080210 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb3ca00700 1 -- 172.21.15.57:0/3735128162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c0042e0 con 0x7fcb38080210 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb3ca00700 1 -- 172.21.15.57:0/3735128162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb2c011a50 con 0x7fcb38080210 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb40800700 1 -- 172.21.15.57:0/3735128162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 msgr2=0x7fcb3810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb40800700 1 --2- 172.21.15.57:0/3735128162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098a0 tx=0x7fcb2c009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb40800700 1 -- 172.21.15.57:0/3735128162 shutdown_connections 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.734+0000 7fcb40800700 1 --2- 172.21.15.57:0/3735128162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3810ecb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.735+0000 7fcb40800700 1 -- 172.21.15.57:0/3735128162 >> 172.21.15.57:0/3735128162 conn(0x7fcb3807b200 msgr2=0x7fcb3807d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.735+0000 7fcb40800700 1 -- 172.21.15.57:0/3735128162 shutdown_connections 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.735+0000 7fcb40800700 1 -- 172.21.15.57:0/3735128162 wait complete. 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.736+0000 7fcb40800700 1 Processor -- start 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.736+0000 7fcb40800700 1 -- start start 2024-02-28T22:54:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.736+0000 7fcb40800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.736+0000 7fcb40800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381a02f0 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.737+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.737+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51090/0 (socket says 172.21.15.57:51090) 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.737+0000 7fcb3de00700 1 -- 172.21.15.57:0/4195479611 learned_addr learned my addr 172.21.15.57:0/4195479611 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.737+0000 7fcb3de00700 1 -- 172.21.15.57:0/4195479611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009550 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.737+0000 7fcb3de00700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009520 tx=0x7fcb2c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.738+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb2c003a80 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.738+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c003be0 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.738+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb381a04f0 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.738+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb2c0197b0 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.738+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb381a0990 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.739+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fcb2c019a20 con 0x7fcb38080210 2024-02-28T22:54:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.739+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb381100b0 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.739+0000 7fcb32a00700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 0x7fcb2003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.740+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb2c04c4b0 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.740+0000 7fcb3d400700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 0x7fcb2003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.740+0000 7fcb3d400700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 0x7fcb2003a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006fd0 tx=0x7fcb28006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:56.745+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcb2c01ac30 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.027+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fcb381a3730 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.086+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fcb2c01a950 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.089+0000 7fcb32a00700 1 -- 172.21.15.57:0/4195479611 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fcb2c019cd0 con 0x7fcb38080210 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 msgr2=0x7fcb2003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 0x7fcb2003a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006fd0 tx=0x7fcb28006e40 comp rx=0 tx=0).stop 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 msgr2=0x7fcb3819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:57.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009520 tx=0x7fcb2c011ab0 comp rx=0 tx=0).stop 2024-02-28T22:54:57.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 shutdown_connections 2024-02-28T22:54:57.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7fcb200383a0 0x7fcb2003a850 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:57.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 --2- 172.21.15.57:0/4195479611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb38080210 0x7fcb3819fdb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:57.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 >> 172.21.15.57:0/4195479611 conn(0x7fcb3807b200 msgr2=0x7fcb3807d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:57.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 shutdown_connections 2024-02-28T22:54:57.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:57.092+0000 7fcb40800700 1 -- 172.21.15.57:0/4195479611 wait complete. 2024-02-28T22:54:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:58 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4195479611' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-28T22:54:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:58 smithi057 ceph-mon[29096]: mgrmap e5: smithi057.zrrldj(active, since 5s) 2024-02-28T22:54:59.241 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:54:59 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4260040132' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-28T22:54:59.274 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "smithi057.zrrldj", 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.326+0000 7f25ffe00700 1 Processor -- start 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.326+0000 7f25ffe00700 1 -- start start 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.326+0000 7f25ffe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.326+0000 7f25ffe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f80796e0 con 0x7f25f807aca0 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.327+0000 7f25fd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.327+0000 7f25fd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51116/0 (socket says 172.21.15.57:51116) 2024-02-28T22:54:59.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.327+0000 7f25fd400700 1 -- 172.21.15.57:0/3215779065 learned_addr learned my addr 172.21.15.57:0/3215779065 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.327+0000 7f25fd400700 1 -- 172.21.15.57:0/3215779065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8079820 con 0x7f25f807aca0 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.328+0000 7f25fd400700 1 --2- 172.21.15.57:0/3215779065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7810a2ab3840cea server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.328+0000 7f25f7e00700 1 -- 172.21.15.57:0/3215779065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec004180 con 0x7f25f807aca0 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.328+0000 7f25f7e00700 1 -- 172.21.15.57:0/3215779065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25ec0042e0 con 0x7f25f807aca0 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.328+0000 7f25f7e00700 1 -- 172.21.15.57:0/3215779065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec011a50 con 0x7f25f807aca0 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 -- 172.21.15.57:0/3215779065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 msgr2=0x7f25f80791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 --2- 172.21.15.57:0/3215779065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 -- 172.21.15.57:0/3215779065 shutdown_connections 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 --2- 172.21.15.57:0/3215779065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f80791a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 -- 172.21.15.57:0/3215779065 >> 172.21.15.57:0/3215779065 conn(0x7f25f8100f90 msgr2=0x7f25f81033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 -- 172.21.15.57:0/3215779065 shutdown_connections 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.329+0000 7f25ffe00700 1 -- 172.21.15.57:0/3215779065 wait complete. 2024-02-28T22:54:59.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.330+0000 7f25ffe00700 1 Processor -- start 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.330+0000 7f25ffe00700 1 -- start start 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.330+0000 7f25ffe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.330+0000 7f25ffe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f819be40 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25fd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25fd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51130/0 (socket says 172.21.15.57:51130) 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25fd400700 1 -- 172.21.15.57:0/4260040132 learned_addr learned my addr 172.21.15.57:0/4260040132 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25fd400700 1 -- 172.21.15.57:0/4260040132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009550 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25fd400700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25ec000c00 tx=0x7f25ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.331+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec003a00 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.332+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25ec003b60 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.332+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec01a800 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.332+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f819c040 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.332+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f819c4e0 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f25ec024440 con 0x7f25f807aca0 2024-02-28T22:54:59.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f810bc50 con 0x7f25f807aca0 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25f6000700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f25ec04bf90 con 0x7f25f807aca0 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25fca00700 1 -- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.333+0000 7f25fca00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.338+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f25ec024bb0 con 0x7f25f807aca0 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.534+0000 7f25fca00700 1 -- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.534+0000 7f25fca00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.609+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f25f819f270 con 0x7f25f807aca0 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.610+0000 7f25f6000700 1 -- 172.21.15.57:0/4260040132 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f25ec018350 con 0x7f25f807aca0 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 msgr2=0x7f25f819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:54:59.278 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25ec000c00 tx=0x7f25ec0047c0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 shutdown_connections 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 --2- 172.21.15.57:0/4260040132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25f807aca0 0x7f25f819b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 >> 172.21.15.57:0/4260040132 conn(0x7f25f8100f90 msgr2=0x7f25f8102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 shutdown_connections 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:58.613+0000 7f25ffe00700 1 -- 172.21.15.57:0/4260040132 wait complete. 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-02-28T22:54:59.279 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: Active manager daemon smithi057.zrrldj restarted 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: Activating manager daemon smithi057.zrrldj 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: osdmap e2: 0 total, 0 up, 0 in 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: mgrmap e6: smithi057.zrrldj(active, starting, since 0.0135576s) 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T22:55:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T22:55:06.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T22:55:06.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: Manager daemon smithi057.zrrldj is now available 2024-02-28T22:55:06.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:06.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:05 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:07.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: Found migration_current of "None". Setting to last migration. 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:06 smithi057 ceph-mon[29096]: mgrmap e7: smithi057.zrrldj(active, since 1.0186s) 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.928+0000 7f1c67e00700 1 Processor -- start 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.929+0000 7f1c67e00700 1 -- start start 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.929+0000 7f1c67e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.929+0000 7f1c67e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68079720 con 0x7f1c6807ace0 2024-02-28T22:55:07.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.930+0000 7f1c66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.930+0000 7f1c66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51146/0 (socket says 172.21.15.57:51146) 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.930+0000 7f1c66a00700 1 -- 172.21.15.57:0/2404247610 learned_addr learned my addr 172.21.15.57:0/2404247610 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.930+0000 7f1c66a00700 1 -- 172.21.15.57:0/2404247610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68079860 con 0x7f1c6807ace0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.930+0000 7f1c66a00700 1 --2- 172.21.15.57:0/2404247610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c580098a0 tx=0x7f1c58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81806cea26ded25 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.931+0000 7f1c65600700 1 -- 172.21.15.57:0/2404247610 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c58004180 con 0x7f1c6807ace0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.931+0000 7f1c65600700 1 -- 172.21.15.57:0/2404247610 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c580042e0 con 0x7f1c6807ace0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.931+0000 7f1c65600700 1 -- 172.21.15.57:0/2404247610 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c58011a50 con 0x7f1c6807ace0 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.931+0000 7f1c67e00700 1 -- 172.21.15.57:0/2404247610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 msgr2=0x7f1c680791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.931+0000 7f1c67e00700 1 --2- 172.21.15.57:0/2404247610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c580098a0 tx=0x7f1c58009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.932+0000 7f1c67e00700 1 -- 172.21.15.57:0/2404247610 shutdown_connections 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.932+0000 7f1c67e00700 1 --2- 172.21.15.57:0/2404247610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c680791e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.932+0000 7f1c67e00700 1 -- 172.21.15.57:0/2404247610 >> 172.21.15.57:0/2404247610 conn(0x7f1c68100e50 msgr2=0x7f1c68103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.932+0000 7f1c67e00700 1 -- 172.21.15.57:0/2404247610 shutdown_connections 2024-02-28T22:55:07.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.932+0000 7f1c67e00700 1 -- 172.21.15.57:0/2404247610 wait complete. 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c67e00700 1 Processor -- start 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c67e00700 1 -- start start 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c67e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c67e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6819bcf0 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51158/0 (socket says 172.21.15.57:51158) 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.933+0000 7f1c66a00700 1 -- 172.21.15.57:0/3129005346 learned_addr learned my addr 172.21.15.57:0/3129005346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c66a00700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009550 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c66a00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c58003de0 tx=0x7f1c58011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c58003930 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c58003a90 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c67e00700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6819bef0 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.934+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c580196c0 con 0x7f1c6807ace0 2024-02-28T22:55:07.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.935+0000 7f1c67e00700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6819c390 con 0x7f1c6807ace0 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f1c58019930 con 0x7f1c6807ace0 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c5f400700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c4c03b000 con 0x7f1c4c038440 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c5804c200 con 0x7f1c6807ace0 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c66000700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:54:59.936+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:00.137+0000 7f1c66000700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:00.137+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:00.538+0000 7f1c66000700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:00.538+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:01.339+0000 7f1c66000700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:01.339+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:02.941+0000 7f1c66000700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3145473765 2024-02-28T22:55:07.773 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:02.941+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:05.928+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7f1c5801abc0 con 0x7f1c6807ace0 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:05.929+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 msgr2=0x7f1c4c03a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:05.929+0000 7f1c5f400700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.934+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f1c5804d980 con 0x7f1c6807ace0 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.935+0000 7f1c5f400700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 0x7f1c4c03e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.935+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c4c03b000 con 0x7f1c4c03bd10 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.937+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 0x7f1c4c03e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.940+0000 7f1c66000700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 0x7f1c4c03e0f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003a10 tx=0x7f1c540092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.942+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f1c4c03b000 con 0x7f1c4c03bd10 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.953+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1c40002600 con 0x7f1c4c03bd10 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.954+0000 7f1c5f400700 1 -- 172.21.15.57:0/3129005346 <== mgr.14120 v2:172.21.15.57:6800/935476336 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1c40002600 con 0x7f1c4c03bd10 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 msgr2=0x7f1c4c03e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 0x7f1c4c03e0f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003a10 tx=0x7f1c540092b0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.774 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 msgr2=0x7f1c6819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c58003de0 tx=0x7f1c58011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 shutdown_connections 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1c4c03bd10 0x7f1c4c03e0f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c6807ace0 0x7f1c6819b7b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 --2- 172.21.15.57:0/3129005346 >> [v2:172.21.15.57:6800/3145473765,v1:172.21.15.57:6801/3145473765] conn(0x7f1c4c038440 0x7f1c4c03a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.955+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 >> 172.21.15.57:0/3129005346 conn(0x7f1c68100e50 msgr2=0x7f1c68102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.956+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 shutdown_connections 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:06.956+0000 7f1c5cc00700 1 -- 172.21.15.57:0/3129005346 wait complete. 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2024-02-28T22:55:07.775 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2024-02-28T22:55:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:07 smithi057 ceph-mon[29096]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-28T22:55:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:07 smithi057 ceph-mon[29096]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-28T22:55:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:07 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:07] ENGINE Bus STARTING 2024-02-28T22:55:08.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:07 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:07] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T22:55:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:07] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T22:55:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:07] ENGINE Bus STARTED 2024-02-28T22:55:09.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: mgrmap e8: smithi057.zrrldj(active, since 2s) 2024-02-28T22:55:09.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:09.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:08 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:09.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.390+0000 7f212b000700 1 Processor -- start 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.390+0000 7f212b000700 1 -- start start 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.391+0000 7f212b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.391+0000 7f212b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240796c0 con 0x7f212407ac80 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.391+0000 7f2123e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.392+0000 7f2123e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43202/0 (socket says 172.21.15.57:43202) 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.392+0000 7f2123e00700 1 -- 172.21.15.57:0/3433994355 learned_addr learned my addr 172.21.15.57:0/3433994355 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.392+0000 7f2123e00700 1 -- 172.21.15.57:0/3433994355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124079800 con 0x7f212407ac80 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f2123e00700 1 --2- 172.21.15.57:0/3433994355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e80 tx=0x7f211400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130ad016099bbd8b server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f2123400700 1 -- 172.21.15.57:0/3433994355 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211400e070 con 0x7f212407ac80 2024-02-28T22:55:09.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f2123400700 1 -- 172.21.15.57:0/3433994355 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2114003e90 con 0x7f212407ac80 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f2123400700 1 -- 172.21.15.57:0/3433994355 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2114004380 con 0x7f212407ac80 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f212b000700 1 -- 172.21.15.57:0/3433994355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 msgr2=0x7f2124079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.393+0000 7f212b000700 1 --2- 172.21.15.57:0/3433994355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e80 tx=0x7f211400b330 comp rx=0 tx=0).stop 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.394+0000 7f212b000700 1 -- 172.21.15.57:0/3433994355 shutdown_connections 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.394+0000 7f212b000700 1 --2- 172.21.15.57:0/3433994355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f2124079180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.394+0000 7f212b000700 1 -- 172.21.15.57:0/3433994355 >> 172.21.15.57:0/3433994355 conn(0x7f2124100fd0 msgr2=0x7f2124103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.394+0000 7f212b000700 1 -- 172.21.15.57:0/3433994355 shutdown_connections 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.394+0000 7f212b000700 1 -- 172.21.15.57:0/3433994355 wait complete. 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.395+0000 7f212b000700 1 Processor -- start 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.395+0000 7f212b000700 1 -- start start 2024-02-28T22:55:09.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.395+0000 7f212b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.395+0000 7f212b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419be60 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.396+0000 7f2123e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.396+0000 7f2123e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43210/0 (socket says 172.21.15.57:43210) 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.396+0000 7f2123e00700 1 -- 172.21.15.57:0/2106375946 learned_addr learned my addr 172.21.15.57:0/2106375946 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.396+0000 7f2123e00700 1 -- 172.21.15.57:0/2106375946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114009b30 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.396+0000 7f2123e00700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f2114004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.397+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211400e040 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.397+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2114003cf0 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.397+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211401b770 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.397+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212419c060 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.397+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212419c560 con 0x7f212407ac80 2024-02-28T22:55:09.410 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.398+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212410bc70 con 0x7f212407ac80 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.398+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2114019040 con 0x7f212407ac80 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.398+0000 7f2121600700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 0x7f210803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.399+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f211404bca0 con 0x7f212407ac80 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.399+0000 7f211b400700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 0x7f210803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.399+0000 7f211b400700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 0x7f210803a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.404+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f21140239c0 con 0x7f212407ac80 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.639+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2124103320 con 0x7f2108038440 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.656+0000 7f2121600700 1 -- 172.21.15.57:0/2106375946 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2124103320 con 0x7f2108038440 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.659+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 msgr2=0x7f210803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.659+0000 7f212b000700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 0x7f210803a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:09.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.659+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 msgr2=0x7f212419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.659+0000 7f212b000700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f2114004920 comp rx=0 tx=0).stop 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 shutdown_connections 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2108038440 0x7f210803a8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 --2- 172.21.15.57:0/2106375946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f212407ac80 0x7f212419b920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 >> 172.21.15.57:0/2106375946 conn(0x7f2124100fd0 msgr2=0x7f2124102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 shutdown_connections 2024-02-28T22:55:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:08.660+0000 7f212b000700 1 -- 172.21.15.57:0/2106375946 wait complete. 2024-02-28T22:55:10.243 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:09 smithi057 ceph-mon[29096]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:10.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.984+0000 7f1221a00700 1 Processor -- start 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.984+0000 7f1221a00700 1 -- start start 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.985+0000 7f1221a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.985+0000 7f1221a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c108530 con 0x7f121c107be0 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.985+0000 7f121aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.985+0000 7f121aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43222/0 (socket says 172.21.15.57:43222) 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.985+0000 7f121aa00700 1 -- 172.21.15.57:0/1640800216 learned_addr learned my addr 172.21.15.57:0/1640800216 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.986+0000 7f121aa00700 1 -- 172.21.15.57:0/1640800216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c108670 con 0x7f121c107be0 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.987+0000 7f121aa00700 1 --2- 172.21.15.57:0/1640800216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f120c0098a0 tx=0x7f120c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4721797f30da6b45 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.987+0000 7f121a000700 1 -- 172.21.15.57:0/1640800216 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120c004180 con 0x7f121c107be0 2024-02-28T22:55:10.951 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.987+0000 7f121a000700 1 -- 172.21.15.57:0/1640800216 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f120c0042e0 con 0x7f121c107be0 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.987+0000 7f121a000700 1 -- 172.21.15.57:0/1640800216 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120c011a50 con 0x7f121c107be0 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 -- 172.21.15.57:0/1640800216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 msgr2=0x7f121c107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 --2- 172.21.15.57:0/1640800216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f120c0098a0 tx=0x7f120c009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 -- 172.21.15.57:0/1640800216 shutdown_connections 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 --2- 172.21.15.57:0/1640800216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c107ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 -- 172.21.15.57:0/1640800216 >> 172.21.15.57:0/1640800216 conn(0x7f121c07b200 msgr2=0x7f121c07b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 -- 172.21.15.57:0/1640800216 shutdown_connections 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.988+0000 7f1221a00700 1 -- 172.21.15.57:0/1640800216 wait complete. 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.989+0000 7f1221a00700 1 Processor -- start 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.989+0000 7f1221a00700 1 -- start start 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.990+0000 7f1221a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:10.952 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.990+0000 7f1221a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c19beb0 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.990+0000 7f121aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.990+0000 7f121aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43236/0 (socket says 172.21.15.57:43236) 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.990+0000 7f121aa00700 1 -- 172.21.15.57:0/622524598 learned_addr learned my addr 172.21.15.57:0/622524598 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.991+0000 7f121aa00700 1 -- 172.21.15.57:0/622524598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009550 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.991+0000 7f121aa00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f120c003de0 tx=0x7f120c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.992+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120c003930 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.992+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f120c003a90 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.992+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c19c0b0 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.992+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120c019730 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.992+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c19c550 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.994+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f120c0199d0 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.994+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c062380 con 0x7f121c107be0 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.994+0000 7f1213400700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 0x7f11fc03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:10.953 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.994+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f120c04c740 con 0x7f121c107be0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.995+0000 7f1213e00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 0x7f11fc03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:09.995+0000 7f1213e00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 0x7f11fc03a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.000+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f120c023370 con 0x7f121c107be0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.222+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f121c106c60 con 0x7f11fc0383f0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.224+0000 7f1213400700 1 -- 172.21.15.57:0/622524598 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f121c106c60 con 0x7f11fc0383f0 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 msgr2=0x7f11fc03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 0x7f11fc03a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 msgr2=0x7f121c19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f120c003de0 tx=0x7f120c011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 shutdown_connections 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f11fc0383f0 0x7f11fc03a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 --2- 172.21.15.57:0/622524598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f121c107be0 0x7f121c19b970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 >> 172.21.15.57:0/622524598 conn(0x7f121c07b200 msgr2=0x7f121c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:10.955 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 shutdown_connections 2024-02-28T22:55:10.955 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:10.227+0000 7f1221a00700 1 -- 172.21.15.57:0/622524598 wait complete. 2024-02-28T22:55:10.955 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2024-02-28T22:55:11.480 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:11 smithi057 ceph-mon[29096]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:12.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:12 smithi057 ceph-mon[29096]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:12.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:12 smithi057 ceph-mon[29096]: Generating ssh key... 2024-02-28T22:55:13.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.524+0000 7f75b0600700 1 Processor -- start 2024-02-28T22:55:13.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.524+0000 7f75b0600700 1 -- start start 2024-02-28T22:55:13.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.525+0000 7f75b0600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:13.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.525+0000 7f75b0600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80796c0 con 0x7f75a807ac80 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.525+0000 7f75adc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.525+0000 7f75adc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43244/0 (socket says 172.21.15.57:43244) 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.525+0000 7f75adc00700 1 -- 172.21.15.57:0/1803525723 learned_addr learned my addr 172.21.15.57:0/1803525723 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.526+0000 7f75adc00700 1 -- 172.21.15.57:0/1803525723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8079800 con 0x7f75a807ac80 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.526+0000 7f75adc00700 1 --2- 172.21.15.57:0/1803525723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098a0 tx=0x7f759c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fd136e6a197c85 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75a7e00700 1 -- 172.21.15.57:0/1803525723 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f759c004180 con 0x7f75a807ac80 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75a7e00700 1 -- 172.21.15.57:0/1803525723 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f759c0042e0 con 0x7f75a807ac80 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75a7e00700 1 -- 172.21.15.57:0/1803525723 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f759c011a50 con 0x7f75a807ac80 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75b0600700 1 -- 172.21.15.57:0/1803525723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 msgr2=0x7f75a8079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75b0600700 1 --2- 172.21.15.57:0/1803525723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098a0 tx=0x7f759c009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75b0600700 1 -- 172.21.15.57:0/1803525723 shutdown_connections 2024-02-28T22:55:13.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75b0600700 1 --2- 172.21.15.57:0/1803525723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a8079180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.527+0000 7f75b0600700 1 -- 172.21.15.57:0/1803525723 >> 172.21.15.57:0/1803525723 conn(0x7f75a8100fd0 msgr2=0x7f75a8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.528+0000 7f75b0600700 1 -- 172.21.15.57:0/1803525723 shutdown_connections 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.528+0000 7f75b0600700 1 -- 172.21.15.57:0/1803525723 wait complete. 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.529+0000 7f75b0600700 1 Processor -- start 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.529+0000 7f75b0600700 1 -- start start 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.529+0000 7f75b0600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.529+0000 7f75b0600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a819be80 con 0x7f75a807ac80 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.529+0000 7f75adc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75adc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43246/0 (socket says 172.21.15.57:43246) 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75adc00700 1 -- 172.21.15.57:0/1557372308 learned_addr learned my addr 172.21.15.57:0/1557372308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75adc00700 1 -- 172.21.15.57:0/1557372308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009550 con 0x7f75a807ac80 2024-02-28T22:55:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75adc00700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f759c003de0 tx=0x7f759c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f759c003930 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f759c003a90 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f759c0196c0 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.530+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a819c080 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.531+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a819c520 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.532+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a810bc70 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.532+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f759c01abc0 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.532+0000 7f75a6000700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 0x7f759003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.532+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f759c04c4b0 con 0x7f75a807ac80 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.533+0000 7f75ad200700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 0x7f759003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:13.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.533+0000 7f75ad200700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 0x7f759003a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7594006fd0 tx=0x7f7594006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.538+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f759c010770 con 0x7f75a807ac80 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:11.760+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f75a8103310 con 0x7f7590038440 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.592+0000 7f75a6000700 1 -- 172.21.15.57:0/1557372308 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f75a8103310 con 0x7f7590038440 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 msgr2=0x7f759003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 0x7f759003a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7594006fd0 tx=0x7f7594006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 msgr2=0x7f75a819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f759c003de0 tx=0x7f759c011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 shutdown_connections 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f7590038440 0x7f759003a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 --2- 172.21.15.57:0/1557372308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75a807ac80 0x7f75a819b940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 >> 172.21.15.57:0/1557372308 conn(0x7f75a8100fd0 msgr2=0x7f75a81026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.595+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 shutdown_connections 2024-02-28T22:55:13.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:12.596+0000 7f75b0600700 1 -- 172.21.15.57:0/1557372308 wait complete. 2024-02-28T22:55:13.707 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:13 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:13.707 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:13 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmPBU8HH15blr49NXBrFs0hBCGaBdpz5Msl6Nja7XP7J1IaHJTcloCgjxSMWC3Y8TGn6llq7bru5xa4LPusRS6j7lFl6aOtydln49W86jkWnJ8RYQmZ7DNbaLI2j6gpgDOhWHlXZtaFgxKoCXBXmLYxuTd4Xtl2fMQqZbjBHzyeO4UpmEiLaGrpE4LmQarb2nlC0/4V8kp56cGzK6b208Q+dBsEvoABbFj8cqjnGNH0rK52/BAXVzGET+L4SWD6Q7qkevIIqA16OF+f0qcj7hPoNyPO0Up55V7r34G4M9SXZCwwHYY8p7eFA4evinVxmkduBzgiZnBdSiiqymbvbmoVBTWgZ+ZNWD+hLyyL3q/RxbJjTOBYkkIvNInTAZeP7WKG/jh8h80z7PUKTlSyC87ubcs1+aFsRJownGucz2S9s5bnoI/4mO9c8rQtejeKEeSDE5cxf7R5/GRpizos3XUlMQhp+a1QzLrrzFxotjWjlRJtDCSg6zcmrn5tb5BCpk= ceph-3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.007+0000 7fbe08c00700 1 Processor -- start 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.007+0000 7fbe08c00700 1 -- start start 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.008+0000 7fbe08c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.008+0000 7fbe08c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040744f0 con 0x7fbe04106330 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.008+0000 7fbe03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.009+0000 7fbe03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43256/0 (socket says 172.21.15.57:43256) 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.009+0000 7fbe03400700 1 -- 172.21.15.57:0/758049299 learned_addr learned my addr 172.21.15.57:0/758049299 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.009+0000 7fbe03400700 1 -- 172.21.15.57:0/758049299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe04074630 con 0x7fbe04106330 2024-02-28T22:55:14.928 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.009+0000 7fbe03400700 1 --2- 172.21.15.57:0/758049299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40098a0 tx=0x7fbdf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adebb2c8e2a148b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe02000700 1 -- 172.21.15.57:0/758049299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdf4004180 con 0x7fbe04106330 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe02000700 1 -- 172.21.15.57:0/758049299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdf40042e0 con 0x7fbe04106330 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe02000700 1 -- 172.21.15.57:0/758049299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdf4011a50 con 0x7fbe04106330 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe08c00700 1 -- 172.21.15.57:0/758049299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 msgr2=0x7fbe04108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe08c00700 1 --2- 172.21.15.57:0/758049299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40098a0 tx=0x7fbdf4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe08c00700 1 -- 172.21.15.57:0/758049299 shutdown_connections 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe08c00700 1 --2- 172.21.15.57:0/758049299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04108710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.010+0000 7fbe08c00700 1 -- 172.21.15.57:0/758049299 >> 172.21.15.57:0/758049299 conn(0x7fbe040ffd30 msgr2=0x7fbe04102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.011+0000 7fbe08c00700 1 -- 172.21.15.57:0/758049299 shutdown_connections 2024-02-28T22:55:14.929 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.011+0000 7fbe08c00700 1 -- 172.21.15.57:0/758049299 wait complete. 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe08c00700 1 Processor -- start 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe08c00700 1 -- start start 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe08c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe08c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04199ac0 con 0x7fbe04106330 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.012+0000 7fbe03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43260/0 (socket says 172.21.15.57:43260) 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbe03400700 1 -- 172.21.15.57:0/2694148931 learned_addr learned my addr 172.21.15.57:0/2694148931 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbe03400700 1 -- 172.21.15.57:0/2694148931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf4009550 con 0x7fbe04106330 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbe03400700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4003de0 tx=0x7fbdf4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdf4003930 con 0x7fbe04106330 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdf4003a90 con 0x7fbe04106330 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.013+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdf40196c0 con 0x7fbe04106330 2024-02-28T22:55:14.930 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.014+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe04199cc0 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.014+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0419a160 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.015+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fbdf401abc0 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.015+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe041098a0 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.015+0000 7fbdfbe00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 0x7fbde803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.015+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbdf404c4b0 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.016+0000 7fbe02a00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 0x7fbde803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.016+0000 7fbe02a00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 0x7fbde803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0006fd0 tx=0x7fbdf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.021+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbdf4010770 con 0x7fbe04106330 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.244+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe04102050 con 0x7fbde8038440 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.246+0000 7fbdfbe00700 1 -- 172.21.15.57:0/2694148931 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fbe04102050 con 0x7fbde8038440 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.248+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 msgr2=0x7fbde803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 0x7fbde803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0006fd0 tx=0x7fbdf0006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:14.931 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 msgr2=0x7fbe04199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4003de0 tx=0x7fbdf4011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 shutdown_connections 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fbde8038440 0x7fbde803a8f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 --2- 172.21.15.57:0/2694148931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe04106330 0x7fbe04199580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 >> 172.21.15.57:0/2694148931 conn(0x7fbe040ffd30 msgr2=0x7fbe041013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 shutdown_connections 2024-02-28T22:55:14.932 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:14.249+0000 7fbe08c00700 1 -- 172.21.15.57:0/2694148931 wait complete. 2024-02-28T22:55:14.933 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-28T22:55:14.933 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2024-02-28T22:55:14.933 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2024-02-28T22:55:15.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:15 smithi057 ceph-mon[29096]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:16.945 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:16 smithi057 ceph-mon[29096]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:17 smithi057 ceph-mon[29096]: Deploying cephadm binary to smithi057 2024-02-28T22:55:20.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.511+0000 7fe6c8400700 1 Processor -- start 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.511+0000 7fe6c8400700 1 -- start start 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.511+0000 7fe6c8400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.511+0000 7fe6c8400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0108530 con 0x7fe6c0107be0 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.512+0000 7fe6c5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.512+0000 7fe6c5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41862/0 (socket says 172.21.15.57:41862) 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.512+0000 7fe6c5a00700 1 -- 172.21.15.57:0/1198706760 learned_addr learned my addr 172.21.15.57:0/1198706760 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.512+0000 7fe6c5a00700 1 -- 172.21.15.57:0/1198706760 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0108670 con 0x7fe6c0107be0 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.512+0000 7fe6c5a00700 1 --2- 172.21.15.57:0/1198706760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098a0 tx=0x7fe6b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c3ce51bd2e5427 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:20.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.513+0000 7fe6bfe00700 1 -- 172.21.15.57:0/1198706760 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6b4004180 con 0x7fe6c0107be0 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.513+0000 7fe6bfe00700 1 -- 172.21.15.57:0/1198706760 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b40042e0 con 0x7fe6c0107be0 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.513+0000 7fe6bfe00700 1 -- 172.21.15.57:0/1198706760 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6b4011a50 con 0x7fe6c0107be0 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.513+0000 7fe6c8400700 1 -- 172.21.15.57:0/1198706760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 msgr2=0x7fe6c0107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.513+0000 7fe6c8400700 1 --2- 172.21.15.57:0/1198706760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098a0 tx=0x7fe6b4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.514+0000 7fe6c8400700 1 -- 172.21.15.57:0/1198706760 shutdown_connections 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.514+0000 7fe6c8400700 1 --2- 172.21.15.57:0/1198706760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c0107ff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.514+0000 7fe6c8400700 1 -- 172.21.15.57:0/1198706760 >> 172.21.15.57:0/1198706760 conn(0x7fe6c007b200 msgr2=0x7fe6c007b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.514+0000 7fe6c8400700 1 -- 172.21.15.57:0/1198706760 shutdown_connections 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.514+0000 7fe6c8400700 1 -- 172.21.15.57:0/1198706760 wait complete. 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c8400700 1 Processor -- start 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c8400700 1 -- start start 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c8400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:20.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c8400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c019bed0 con 0x7fe6c0107be0 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41868/0 (socket says 172.21.15.57:41868) 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.515+0000 7fe6c5a00700 1 -- 172.21.15.57:0/2994001183 learned_addr learned my addr 172.21.15.57:0/2994001183 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6c5a00700 1 -- 172.21.15.57:0/2994001183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4009550 con 0x7fe6c0107be0 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6c5a00700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4003de0 tx=0x7fe6b4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6b4003930 con 0x7fe6c0107be0 2024-02-28T22:55:20.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b4003a90 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6b4019730 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.516+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c019c0d0 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.517+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c019c570 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.518+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c0062380 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.518+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fe6b4019960 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.518+0000 7fe6be000700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 0x7fe6a803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.518+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe6b404c520 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.519+0000 7fe6c5000700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 0x7fe6a803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.519+0000 7fe6c5000700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 0x7fe6a803a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac006fd0 tx=0x7fe6ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.524+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe6b4010770 con 0x7fe6c0107be0 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:15.752+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6c0106c80 con 0x7fe6a8038440 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.235+0000 7fe6be000700 1 -- 172.21.15.57:0/2994001183 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe6c0106c80 con 0x7fe6a8038440 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 msgr2=0x7fe6a803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:20.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 0x7fe6a803a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac006fd0 tx=0x7fe6ac006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 msgr2=0x7fe6c019b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4003de0 tx=0x7fe6b4011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 shutdown_connections 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fe6a8038440 0x7fe6a803a8f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 --2- 172.21.15.57:0/2994001183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6c0107be0 0x7fe6c019b990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.238+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 >> 172.21.15.57:0/2994001183 conn(0x7fe6c007b200 msgr2=0x7fe6c0106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.239+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 shutdown_connections 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:20.239+0000 7fe6c8400700 1 -- 172.21.15.57:0/2994001183 wait complete. 2024-02-28T22:55:20.916 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying mon service with default placement... 2024-02-28T22:55:21.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:21 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:21.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:21 smithi057 ceph-mon[29096]: Added host smithi057 2024-02-28T22:55:21.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:21 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:22.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-28T22:55:22.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.822+0000 7f2fcbe00700 1 Processor -- start 2024-02-28T22:55:22.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.822+0000 7f2fcbe00700 1 -- start start 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.822+0000 7f2fcbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.822+0000 7f2fcbe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40796c0 con 0x7f2fc407ac80 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.823+0000 7f2fc9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.823+0000 7f2fc9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41880/0 (socket says 172.21.15.57:41880) 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.823+0000 7f2fc9400700 1 -- 172.21.15.57:0/1730658096 learned_addr learned my addr 172.21.15.57:0/1730658096 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.823+0000 7f2fc9400700 1 -- 172.21.15.57:0/1730658096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc4079800 con 0x7f2fc407ac80 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.824+0000 7f2fc9400700 1 --2- 172.21.15.57:0/1730658096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80098a0 tx=0x7f2fb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1848edffd6df2690 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.824+0000 7f2fc3e00700 1 -- 172.21.15.57:0/1730658096 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fb8004180 con 0x7f2fc407ac80 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.824+0000 7f2fc3e00700 1 -- 172.21.15.57:0/1730658096 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb80042e0 con 0x7f2fc407ac80 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.824+0000 7f2fc3e00700 1 -- 172.21.15.57:0/1730658096 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fb8011a50 con 0x7f2fc407ac80 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 -- 172.21.15.57:0/1730658096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 msgr2=0x7f2fc4079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/1730658096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80098a0 tx=0x7f2fb8009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 -- 172.21.15.57:0/1730658096 shutdown_connections 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/1730658096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc4079180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 -- 172.21.15.57:0/1730658096 >> 172.21.15.57:0/1730658096 conn(0x7f2fc4100fd0 msgr2=0x7f2fc4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 -- 172.21.15.57:0/1730658096 shutdown_connections 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.825+0000 7f2fcbe00700 1 -- 172.21.15.57:0/1730658096 wait complete. 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.826+0000 7f2fcbe00700 1 Processor -- start 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.826+0000 7f2fcbe00700 1 -- start start 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.827+0000 7f2fcbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.827+0000 7f2fcbe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc419be80 con 0x7f2fc407ac80 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.827+0000 7f2fc9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.827+0000 7f2fc9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41896/0 (socket says 172.21.15.57:41896) 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.827+0000 7f2fc9400700 1 -- 172.21.15.57:0/4087612531 learned_addr learned my addr 172.21.15.57:0/4087612531 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fc9400700 1 -- 172.21.15.57:0/4087612531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009550 con 0x7f2fc407ac80 2024-02-28T22:55:22.995 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fc9400700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8003de0 tx=0x7f2fb8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fb8003930 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc419c080 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb8003a90 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fb80196c0 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.828+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc419c520 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.829+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2fb8019970 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.830+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc410bc70 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.830+0000 7f2fc2000700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 0x7f2fac03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.830+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2fb804c490 con 0x7f2fc407ac80 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.830+0000 7f2fc8a00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 0x7f2fac03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.831+0000 7f2fc8a00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 0x7f2fac03a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:22.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:21.836+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2fb802c960 con 0x7f2fc407ac80 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.077+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fc4103310 con 0x7f2fac038440 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.088+0000 7f2fc2000700 1 -- 172.21.15.57:0/4087612531 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2fc4103310 con 0x7f2fac038440 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.090+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 msgr2=0x7f2fac03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.090+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 0x7f2fac03a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 msgr2=0x7f2fc419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8003de0 tx=0x7f2fb8011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 shutdown_connections 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f2fac038440 0x7f2fac03a8f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 --2- 172.21.15.57:0/4087612531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fc407ac80 0x7f2fc419b940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 >> 172.21.15.57:0/4087612531 conn(0x7f2fc4100fd0 msgr2=0x7f2fc41026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 shutdown_connections 2024-02-28T22:55:22.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:22.091+0000 7f2fcbe00700 1 -- 172.21.15.57:0/4087612531 wait complete. 2024-02-28T22:55:22.998 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying mgr service with default placement... 2024-02-28T22:55:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:23 smithi057 ceph-mon[29096]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:23 smithi057 ceph-mon[29096]: Saving service mon spec with placement count:5 2024-02-28T22:55:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:23 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:24.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-28T22:55:24.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.651+0000 7fb78ce00700 1 Processor -- start 2024-02-28T22:55:24.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.651+0000 7fb78ce00700 1 -- start start 2024-02-28T22:55:24.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.652+0000 7fb78ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:24.804 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.652+0000 7fb78ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788074510 con 0x7fb7881042e0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.653+0000 7fb785e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.653+0000 7fb785e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41902/0 (socket says 172.21.15.57:41902) 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.653+0000 7fb785e00700 1 -- 172.21.15.57:0/913123635 learned_addr learned my addr 172.21.15.57:0/913123635 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.653+0000 7fb785e00700 1 -- 172.21.15.57:0/913123635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788074650 con 0x7fb7881042e0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.653+0000 7fb785e00700 1 --2- 172.21.15.57:0/913123635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb7780098a0 tx=0x7fb778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed2c2616267dbcb server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.654+0000 7fb785400700 1 -- 172.21.15.57:0/913123635 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb778004180 con 0x7fb7881042e0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.654+0000 7fb785400700 1 -- 172.21.15.57:0/913123635 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7780042e0 con 0x7fb7881042e0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.654+0000 7fb785400700 1 -- 172.21.15.57:0/913123635 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb778011a50 con 0x7fb7881042e0 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.654+0000 7fb78ce00700 1 -- 172.21.15.57:0/913123635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 msgr2=0x7fb788106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.654+0000 7fb78ce00700 1 --2- 172.21.15.57:0/913123635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb7780098a0 tx=0x7fb778009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.655+0000 7fb78ce00700 1 -- 172.21.15.57:0/913123635 shutdown_connections 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.655+0000 7fb78ce00700 1 --2- 172.21.15.57:0/913123635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb788106700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.655+0000 7fb78ce00700 1 -- 172.21.15.57:0/913123635 >> 172.21.15.57:0/913123635 conn(0x7fb7880fff10 msgr2=0x7fb788102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:24.805 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.655+0000 7fb78ce00700 1 -- 172.21.15.57:0/913123635 shutdown_connections 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.655+0000 7fb78ce00700 1 -- 172.21.15.57:0/913123635 wait complete. 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.656+0000 7fb78ce00700 1 Processor -- start 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.656+0000 7fb78ce00700 1 -- start start 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.656+0000 7fb78ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.656+0000 7fb78ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881a0290 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb785e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb785e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41916/0 (socket says 172.21.15.57:41916) 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb785e00700 1 -- 172.21.15.57:0/4116434695 learned_addr learned my addr 172.21.15.57:0/4116434695 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb785e00700 1 -- 172.21.15.57:0/4116434695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778009550 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb785e00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb778003de0 tx=0x7fb778011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.657+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb778003930 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.658+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb778003a90 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.658+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7780196c0 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.658+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7881a0490 con 0x7fb7881042e0 2024-02-28T22:55:24.806 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.658+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7881a0930 con 0x7fb7881042e0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.659+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb77801abc0 con 0x7fb7881042e0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.659+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb788107890 con 0x7fb7881042e0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.659+0000 7fb77ea00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 0x7fb76c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.659+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb77804c400 con 0x7fb7881042e0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.660+0000 7fb77fe00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 0x7fb76c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.660+0000 7fb77fe00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 0x7fb76c03a8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb770006fd0 tx=0x7fb770006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.665+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb778010770 con 0x7fb7881042e0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.900+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb78810a0f0 con 0x7fb76c0383f0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.912+0000 7fb77ea00700 1 -- 172.21.15.57:0/4116434695 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb78810a0f0 con 0x7fb76c0383f0 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.915+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 msgr2=0x7fb76c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.915+0000 7fb78ce00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 0x7fb76c03a8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb770006fd0 tx=0x7fb770006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.915+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 msgr2=0x7fb78819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.915+0000 7fb78ce00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb778003de0 tx=0x7fb778011ab0 comp rx=0 tx=0).stop 2024-02-28T22:55:24.807 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.915+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 shutdown_connections 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.916+0000 7fb78ce00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fb76c0383f0 0x7fb76c03a8a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.916+0000 7fb78ce00700 1 --2- 172.21.15.57:0/4116434695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7881042e0 0x7fb78819fd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.916+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 >> 172.21.15.57:0/4116434695 conn(0x7fb7880fff10 msgr2=0x7fb788101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.916+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 shutdown_connections 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:23.916+0000 7fb78ce00700 1 -- 172.21.15.57:0/4116434695 wait complete. 2024-02-28T22:55:24.808 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying crash service with default placement... 2024-02-28T22:55:25.249 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:24 smithi057 ceph-mon[29096]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:25.249 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:24 smithi057 ceph-mon[29096]: Saving service mgr spec with placement count:2 2024-02-28T22:55:25.249 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:24 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:25.249 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:24 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:25 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:25 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:26.432 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-28T22:55:26.432 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.412+0000 7f19ea400700 1 Processor -- start 2024-02-28T22:55:26.432 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.412+0000 7f19ea400700 1 -- start start 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.413+0000 7f19ea400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.413+0000 7f19ea400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4079700 con 0x7f19e407acc0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.413+0000 7f19e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.413+0000 7f19e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58100/0 (socket says 172.21.15.57:58100) 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.413+0000 7f19e3400700 1 -- 172.21.15.57:0/3871031467 learned_addr learned my addr 172.21.15.57:0/3871031467 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.414+0000 7f19e3400700 1 -- 172.21.15.57:0/3871031467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4079840 con 0x7f19e407acc0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.414+0000 7f19e3400700 1 --2- 172.21.15.57:0/3871031467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19d40098a0 tx=0x7f19d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72ef07b38a4236a server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.414+0000 7f19e2a00700 1 -- 172.21.15.57:0/3871031467 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d4004180 con 0x7f19e407acc0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.414+0000 7f19e2a00700 1 -- 172.21.15.57:0/3871031467 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19d40042e0 con 0x7f19e407acc0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.414+0000 7f19e2a00700 1 -- 172.21.15.57:0/3871031467 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d4011af0 con 0x7f19e407acc0 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 -- 172.21.15.57:0/3871031467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 msgr2=0x7f19e40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 --2- 172.21.15.57:0/3871031467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19d40098a0 tx=0x7f19d4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 -- 172.21.15.57:0/3871031467 shutdown_connections 2024-02-28T22:55:26.433 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 --2- 172.21.15.57:0/3871031467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e40791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 -- 172.21.15.57:0/3871031467 >> 172.21.15.57:0/3871031467 conn(0x7f19e4100fd0 msgr2=0x7f19e4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 -- 172.21.15.57:0/3871031467 shutdown_connections 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.415+0000 7f19ea400700 1 -- 172.21.15.57:0/3871031467 wait complete. 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.416+0000 7f19ea400700 1 Processor -- start 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.416+0000 7f19ea400700 1 -- start start 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19ea400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19ea400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419bf00 con 0x7f19e407acc0 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58110/0 (socket says 172.21.15.57:58110) 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19e3400700 1 -- 172.21.15.57:0/3350954698 learned_addr learned my addr 172.21.15.57:0/3350954698 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.417+0000 7f19e3400700 1 -- 172.21.15.57:0/3350954698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009550 con 0x7f19e407acc0 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19e3400700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19d4011670 tx=0x7f19d4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d40039f0 con 0x7f19e407acc0 2024-02-28T22:55:26.434 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19d4003b50 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d4024690 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e419c100 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.418+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e419c5a0 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.419+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f19d401ab00 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.419+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e410bcf0 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.420+0000 7f19e0c00700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 0x7f19c803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.420+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f19d404c320 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.420+0000 7f19daa00700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 0x7f19c803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.421+0000 7f19daa00700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 0x7f19c803a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.426+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f19d4018920 con 0x7f19e407acc0 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.670+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f19e4103310 con 0x7f19c8038440 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.676+0000 7f19e0c00700 1 -- 172.21.15.57:0/3350954698 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f19e4103310 con 0x7f19c8038440 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 msgr2=0x7f19c803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:26.435 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 0x7f19c803a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 msgr2=0x7f19e419b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19d4011670 tx=0x7f19d4011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 shutdown_connections 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f19c8038440 0x7f19c803a8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 --2- 172.21.15.57:0/3350954698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19e407acc0 0x7f19e419b9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 >> 172.21.15.57:0/3350954698 conn(0x7f19e4100fd0 msgr2=0x7f19e41026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.679+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 shutdown_connections 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:25.680+0000 7f19ea400700 1 -- 172.21.15.57:0/3350954698 wait complete. 2024-02-28T22:55:26.436 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying ceph-exporter service with default placement... 2024-02-28T22:55:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:26 smithi057 ceph-mon[29096]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:26 smithi057 ceph-mon[29096]: Saving service crash spec with placement * 2024-02-28T22:55:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:26 smithi057 ceph-mon[29096]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-02-28T22:55:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.553+0000 7ff9a2200700 1 Processor -- start 2024-02-28T22:55:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.554+0000 7ff9a2200700 1 -- start start 2024-02-28T22:55:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.554+0000 7ff9a2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.554+0000 7ff9a2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c108510 con 0x7ff99c107bc0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.555+0000 7ff99b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.555+0000 7ff99b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58122/0 (socket says 172.21.15.57:58122) 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.555+0000 7ff99b400700 1 -- 172.21.15.57:0/1658717240 learned_addr learned my addr 172.21.15.57:0/1658717240 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.555+0000 7ff99b400700 1 -- 172.21.15.57:0/1658717240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c108650 con 0x7ff99c107bc0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.555+0000 7ff99b400700 1 --2- 172.21.15.57:0/1658717240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e80 tx=0x7ff98c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5d68965f8693ef server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.556+0000 7ff99aa00700 1 -- 172.21.15.57:0/1658717240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff98c00e070 con 0x7ff99c107bc0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.556+0000 7ff99aa00700 1 -- 172.21.15.57:0/1658717240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff98c00b8c0 con 0x7ff99c107bc0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.556+0000 7ff99aa00700 1 -- 172.21.15.57:0/1658717240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff98c0044d0 con 0x7ff99c107bc0 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.556+0000 7ff9a2200700 1 -- 172.21.15.57:0/1658717240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 msgr2=0x7ff99c107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.556+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1658717240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e80 tx=0x7ff98c00b330 comp rx=0 tx=0).stop 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.557+0000 7ff9a2200700 1 -- 172.21.15.57:0/1658717240 shutdown_connections 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.557+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1658717240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c107fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:28.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.557+0000 7ff9a2200700 1 -- 172.21.15.57:0/1658717240 >> 172.21.15.57:0/1658717240 conn(0x7ff99c07b200 msgr2=0x7ff99c07b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.557+0000 7ff9a2200700 1 -- 172.21.15.57:0/1658717240 shutdown_connections 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.557+0000 7ff9a2200700 1 -- 172.21.15.57:0/1658717240 wait complete. 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.558+0000 7ff9a2200700 1 Processor -- start 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.558+0000 7ff9a2200700 1 -- start start 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.558+0000 7ff9a2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.558+0000 7ff9a2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c19bdf0 con 0x7ff99c107bc0 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.559+0000 7ff99b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.559+0000 7ff99b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58134/0 (socket says 172.21.15.57:58134) 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.559+0000 7ff99b400700 1 -- 172.21.15.57:0/1875542234 learned_addr learned my addr 172.21.15.57:0/1875542234 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.559+0000 7ff99b400700 1 -- 172.21.15.57:0/1875542234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009b30 con 0x7ff99c107bc0 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.559+0000 7ff99b400700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff98c000f80 tx=0x7ff98c0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.560+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff98c00e040 con 0x7ff99c107bc0 2024-02-28T22:55:28.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.560+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff98c00bd00 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.560+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff98c012770 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.560+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c19bff0 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.560+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c19c4f0 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.561+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff98c019040 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.561+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c10c050 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.561+0000 7ff998c00700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 0x7ff98003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.561+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff98c04bce0 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.562+0000 7ff992a00700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 0x7ff98003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.562+0000 7ff992a00700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 0x7ff98003a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff984006fd0 tx=0x7ff984006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.569+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff98c0129a0 con 0x7ff99c107bc0 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.807+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7ff99c106c70 con 0x7ff980038440 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.815+0000 7ff998c00700 1 -- 172.21.15.57:0/1875542234 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7ff99c106c70 con 0x7ff980038440 2024-02-28T22:55:28.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.817+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 msgr2=0x7ff98003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.817+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 0x7ff98003a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff984006fd0 tx=0x7ff984006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.817+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 msgr2=0x7ff99c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.817+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff98c000f80 tx=0x7ff98c0049f0 comp rx=0 tx=0).stop 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 shutdown_connections 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7ff980038440 0x7ff98003a8f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 --2- 172.21.15.57:0/1875542234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99c107bc0 0x7ff99c19b8b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 >> 172.21.15.57:0/1875542234 conn(0x7ff99c07b200 msgr2=0x7ff99c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 shutdown_connections 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:27.818+0000 7ff9a2200700 1 -- 172.21.15.57:0/1875542234 wait complete. 2024-02-28T22:55:28.572 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying prometheus service with default placement... 2024-02-28T22:55:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:28 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:28 smithi057 ceph-mon[29096]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:28 smithi057 ceph-mon[29096]: Saving service ceph-exporter spec with placement * 2024-02-28T22:55:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:28 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:28 smithi057 ceph-mon[29096]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:30.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-28T22:55:30.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.174+0000 7fdf9f000700 1 Processor -- start 2024-02-28T22:55:30.074 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.174+0000 7fdf9f000700 1 -- start start 2024-02-28T22:55:30.075 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.175+0000 7fdf9f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:30.075 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.175+0000 7fdf9f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf98079cf0 con 0x7fdf98106540 2024-02-28T22:55:30.076 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.175+0000 7fdf97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:30.076 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.175+0000 7fdf97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58140/0 (socket says 172.21.15.57:58140) 2024-02-28T22:55:30.076 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.175+0000 7fdf97e00700 1 -- 172.21.15.57:0/3424784407 learned_addr learned my addr 172.21.15.57:0/3424784407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:30.076 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.176+0000 7fdf97e00700 1 -- 172.21.15.57:0/3424784407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98079e30 con 0x7fdf98106540 2024-02-28T22:55:30.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.176+0000 7fdf97e00700 1 --2- 172.21.15.57:0/3424784407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009e80 tx=0x7fdf8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7b5abfc745358 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:30.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.176+0000 7fdf96a00700 1 -- 172.21.15.57:0/3424784407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf8800e070 con 0x7fdf98106540 2024-02-28T22:55:30.077 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf96a00700 1 -- 172.21.15.57:0/3424784407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf8800b8c0 con 0x7fdf98106540 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf96a00700 1 -- 172.21.15.57:0/3424784407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf880044d0 con 0x7fdf98106540 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf9f000700 1 -- 172.21.15.57:0/3424784407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 msgr2=0x7fdf98108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf9f000700 1 --2- 172.21.15.57:0/3424784407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009e80 tx=0x7fdf8800b330 comp rx=0 tx=0).stop 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf9f000700 1 -- 172.21.15.57:0/3424784407 shutdown_connections 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.177+0000 7fdf9f000700 1 --2- 172.21.15.57:0/3424784407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf98108920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.178+0000 7fdf9f000700 1 -- 172.21.15.57:0/3424784407 >> 172.21.15.57:0/3424784407 conn(0x7fdf980fff00 msgr2=0x7fdf98102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:30.078 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.178+0000 7fdf9f000700 1 -- 172.21.15.57:0/3424784407 shutdown_connections 2024-02-28T22:55:30.079 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.178+0000 7fdf9f000700 1 -- 172.21.15.57:0/3424784407 wait complete. 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.179+0000 7fdf9f000700 1 Processor -- start 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.179+0000 7fdf9f000700 1 -- start start 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.179+0000 7fdf9f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.179+0000 7fdf9f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9819be40 con 0x7fdf98106540 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.179+0000 7fdf97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.180+0000 7fdf97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58144/0 (socket says 172.21.15.57:58144) 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.180+0000 7fdf97e00700 1 -- 172.21.15.57:0/2073784111 learned_addr learned my addr 172.21.15.57:0/2073784111 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.180+0000 7fdf97e00700 1 -- 172.21.15.57:0/2073784111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88009b30 con 0x7fdf98106540 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.180+0000 7fdf97e00700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009e50 tx=0x7fdf880049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.181+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf8800e040 con 0x7fdf98106540 2024-02-28T22:55:30.080 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.181+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf8800bcc0 con 0x7fdf98106540 2024-02-28T22:55:30.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.181+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf88012650 con 0x7fdf98106540 2024-02-28T22:55:30.082 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.181+0000 7fdf9f000700 1 -- 172.21.15.57:0/2073784111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9819c040 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.181+0000 7fdf9f000700 1 -- 172.21.15.57:0/2073784111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9819c540 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.182+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf98109ac0 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.182+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fdf88019040 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.183+0000 7fdf94c00700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 0x7fdf7c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.183+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdf8804bf20 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.183+0000 7fdf97400700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 0x7fdf7c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.184+0000 7fdf97400700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 0x7fdf7c03a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.188+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdf8801b430 con 0x7fdf98106540 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.425+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf98109c50 con 0x7fdf7c038400 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.431+0000 7fdf94c00700 1 -- 172.21.15.57:0/2073784111 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fdf98109c50 con 0x7fdf7c038400 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.433+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 msgr2=0x7fdf7c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.433+0000 7fdf8e000700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 0x7fdf7c03a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:30.083 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 msgr2=0x7fdf9819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:30.084 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009e50 tx=0x7fdf880049f0 comp rx=0 tx=0).stop 2024-02-28T22:55:30.084 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 shutdown_connections 2024-02-28T22:55:30.084 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdf7c038400 0x7fdf7c03a8b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:30.084 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 --2- 172.21.15.57:0/2073784111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf98106540 0x7fdf9819b900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:30.084 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 >> 172.21.15.57:0/2073784111 conn(0x7fdf980fff00 msgr2=0x7fdf98100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:30.085 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 shutdown_connections 2024-02-28T22:55:30.085 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:29.434+0000 7fdf8e000700 1 -- 172.21.15.57:0/2073784111 wait complete. 2024-02-28T22:55:30.085 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying grafana service with default placement... 2024-02-28T22:55:30.641 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:30 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:30.641 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:30 smithi057 ceph-mon[29096]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:30.641 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:30 smithi057 ceph-mon[29096]: Saving service prometheus spec with placement count:1 2024-02-28T22:55:30.642 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:30 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:30.642 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:30 smithi057 ceph-mon[29096]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:31.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:31 smithi057 ceph-mon[29096]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:31.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:31 smithi057 ceph-mon[29096]: Saving service grafana spec with placement count:1 2024-02-28T22:55:31.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:31 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:31.989 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-28T22:55:31.990 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.642+0000 7f257ce00700 1 Processor -- start 2024-02-28T22:55:31.990 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.643+0000 7f257ce00700 1 -- start start 2024-02-28T22:55:31.990 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.643+0000 7f257ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:31.990 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.643+0000 7f257ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2578074510 con 0x7f25781064f0 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.644+0000 7f2575e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.644+0000 7f2575e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58150/0 (socket says 172.21.15.57:58150) 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.644+0000 7f2575e00700 1 -- 172.21.15.57:0/1456979068 learned_addr learned my addr 172.21.15.57:0/1456979068 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.645+0000 7f2575e00700 1 -- 172.21.15.57:0/1456979068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578074650 con 0x7f25781064f0 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.645+0000 7f2575e00700 1 --2- 172.21.15.57:0/1456979068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25680098a0 tx=0x7f2568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8043255fef8afb36 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.645+0000 7f2574a00700 1 -- 172.21.15.57:0/1456979068 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2568004180 con 0x7f25781064f0 2024-02-28T22:55:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.645+0000 7f2574a00700 1 -- 172.21.15.57:0/1456979068 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f25680042e0 con 0x7f25781064f0 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.645+0000 7f2574a00700 1 -- 172.21.15.57:0/1456979068 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2568011af0 con 0x7f25781064f0 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.646+0000 7f257ce00700 1 -- 172.21.15.57:0/1456979068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 msgr2=0x7f25781088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.646+0000 7f257ce00700 1 --2- 172.21.15.57:0/1456979068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25680098a0 tx=0x7f2568009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.646+0000 7f257ce00700 1 -- 172.21.15.57:0/1456979068 shutdown_connections 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.646+0000 7f257ce00700 1 --2- 172.21.15.57:0/1456979068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f25781088d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.646+0000 7f257ce00700 1 -- 172.21.15.57:0/1456979068 >> 172.21.15.57:0/1456979068 conn(0x7f25780fff10 msgr2=0x7f2578102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.647+0000 7f257ce00700 1 -- 172.21.15.57:0/1456979068 shutdown_connections 2024-02-28T22:55:31.993 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.647+0000 7f257ce00700 1 -- 172.21.15.57:0/1456979068 wait complete. 2024-02-28T22:55:31.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.648+0000 7f257ce00700 1 Processor -- start 2024-02-28T22:55:31.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.648+0000 7f257ce00700 1 -- start start 2024-02-28T22:55:31.996 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.648+0000 7f257ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.648+0000 7f257ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257819be80 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.649+0000 7f2575e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.649+0000 7f2575e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58154/0 (socket says 172.21.15.57:58154) 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.649+0000 7f2575e00700 1 -- 172.21.15.57:0/659516563 learned_addr learned my addr 172.21.15.57:0/659516563 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.649+0000 7f2575e00700 1 -- 172.21.15.57:0/659516563 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568009550 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f2575e00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2568011670 tx=0x7f2568011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25680039f0 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257819c080 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2568003b50 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25680247c0 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.650+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257819c520 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.652+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f256801a430 con 0x7f25781064f0 2024-02-28T22:55:31.997 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.652+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578109ad0 con 0x7f25781064f0 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.653+0000 7f256ea00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 0x7f255c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.653+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f256804c400 con 0x7f25781064f0 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.654+0000 7f2575400700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 0x7f255c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.654+0000 7f2575400700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 0x7f255c03a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.659+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2568018b40 con 0x7f25781064f0 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.896+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2578101400 con 0x7f255c038490 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.910+0000 7f256ea00700 1 -- 172.21.15.57:0/659516563 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2578101400 con 0x7f255c038490 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 msgr2=0x7f255c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 0x7f255c03a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 msgr2=0x7f257819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2568011670 tx=0x7f2568011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 shutdown_connections 2024-02-28T22:55:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f255c038490 0x7f255c03a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:31.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 --2- 172.21.15.57:0/659516563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25781064f0 0x7f257819b940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:31.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.913+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 >> 172.21.15.57:0/659516563 conn(0x7f25780fff10 msgr2=0x7f2578100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:31.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.914+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 shutdown_connections 2024-02-28T22:55:31.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:30.914+0000 7f257ce00700 1 -- 172.21.15.57:0/659516563 wait complete. 2024-02-28T22:55:31.999 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying node-exporter service with default placement... 2024-02-28T22:55:33.038 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:32 smithi057 ceph-mon[29096]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:34.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-28T22:55:34.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.039+0000 7f182a800700 1 Processor -- start 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.040+0000 7f182a800700 1 -- start start 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.040+0000 7f182a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.040+0000 7f182a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1824108530 con 0x7f1824107be0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.041+0000 7f1823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.041+0000 7f1823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58168/0 (socket says 172.21.15.57:58168) 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.041+0000 7f1823400700 1 -- 172.21.15.57:0/2510619331 learned_addr learned my addr 172.21.15.57:0/2510619331 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.041+0000 7f1823400700 1 -- 172.21.15.57:0/2510619331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824108670 con 0x7f1824107be0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.042+0000 7f1823400700 1 --2- 172.21.15.57:0/2510619331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ddb92807db4f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.042+0000 7f1822a00700 1 -- 172.21.15.57:0/2510619331 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1814004180 con 0x7f1824107be0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.042+0000 7f1822a00700 1 -- 172.21.15.57:0/2510619331 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f18140042e0 con 0x7f1824107be0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.042+0000 7f1822a00700 1 -- 172.21.15.57:0/2510619331 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1814011af0 con 0x7f1824107be0 2024-02-28T22:55:34.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 -- 172.21.15.57:0/2510619331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 msgr2=0x7f1824107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 --2- 172.21.15.57:0/2510619331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 -- 172.21.15.57:0/2510619331 shutdown_connections 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 --2- 172.21.15.57:0/2510619331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f1824107ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 -- 172.21.15.57:0/2510619331 >> 172.21.15.57:0/2510619331 conn(0x7f182407b200 msgr2=0x7f182407b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 -- 172.21.15.57:0/2510619331 shutdown_connections 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.043+0000 7f182a800700 1 -- 172.21.15.57:0/2510619331 wait complete. 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.044+0000 7f182a800700 1 Processor -- start 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.044+0000 7f182a800700 1 -- start start 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f182a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f182a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419bef0 con 0x7f1824107be0 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f1823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f1823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58174/0 (socket says 172.21.15.57:58174) 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f1823400700 1 -- 172.21.15.57:0/472653676 learned_addr learned my addr 172.21.15.57:0/472653676 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:34.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.045+0000 7f1823400700 1 -- 172.21.15.57:0/472653676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009550 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f1823400700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1814011670 tx=0x7f1814011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1814003a40 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1814003ba0 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182419c0f0 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1814024790 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.046+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182419c590 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.047+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f181401a430 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.047+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1824062380 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.048+0000 7f1820c00700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 0x7f180803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.048+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f181404c4d0 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.048+0000 7f181be00700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 0x7f180803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.049+0000 7f181be00700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 0x7f180803a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f180c006fd0 tx=0x7f180c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.053+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f181401ac50 con 0x7f1824107be0 2024-02-28T22:55:34.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.286+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1824106c60 con 0x7f1808038440 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.295+0000 7f1820c00700 1 -- 172.21.15.57:0/472653676 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1824106c60 con 0x7f1808038440 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 msgr2=0x7f180803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 0x7f180803a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f180c006fd0 tx=0x7f180c006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 msgr2=0x7f182419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1814011670 tx=0x7f1814011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 shutdown_connections 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f1808038440 0x7f180803a8f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 --2- 172.21.15.57:0/472653676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1824107be0 0x7f182419b9b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.299+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 >> 172.21.15.57:0/472653676 conn(0x7f182407b200 msgr2=0x7f1824106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.300+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 shutdown_connections 2024-02-28T22:55:34.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:33.300+0000 7f182a800700 1 -- 172.21.15.57:0/472653676 wait complete. 2024-02-28T22:55:34.047 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying alertmanager service with default placement... 2024-02-28T22:55:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:34 smithi057 ceph-mon[29096]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:34 smithi057 ceph-mon[29096]: Saving service node-exporter spec with placement * 2024-02-28T22:55:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:34 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:34 smithi057 ceph-mon[29096]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:35.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-28T22:55:35.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.670+0000 7f411aa00700 1 Processor -- start 2024-02-28T22:55:35.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.670+0000 7f411aa00700 1 -- start start 2024-02-28T22:55:35.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.671+0000 7f411aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:35.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.671+0000 7f411aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114074510 con 0x7f41141064f0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.671+0000 7f4113e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.671+0000 7f4113e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58186/0 (socket says 172.21.15.57:58186) 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.671+0000 7f4113e00700 1 -- 172.21.15.57:0/4065915007 learned_addr learned my addr 172.21.15.57:0/4065915007 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.672+0000 7f4113e00700 1 -- 172.21.15.57:0/4065915007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114074650 con 0x7f41141064f0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.672+0000 7f4113e00700 1 --2- 172.21.15.57:0/4065915007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e80 tx=0x7f410400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7c6992f2008027 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.673+0000 7f4113400700 1 -- 172.21.15.57:0/4065915007 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f410400e070 con 0x7f41141064f0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.673+0000 7f4113400700 1 -- 172.21.15.57:0/4065915007 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f410400b8c0 con 0x7f41141064f0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.673+0000 7f4113400700 1 -- 172.21.15.57:0/4065915007 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41040044d0 con 0x7f41141064f0 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.673+0000 7f411aa00700 1 -- 172.21.15.57:0/4065915007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 msgr2=0x7f41141088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.673+0000 7f411aa00700 1 --2- 172.21.15.57:0/4065915007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e80 tx=0x7f410400b330 comp rx=0 tx=0).stop 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.674+0000 7f411aa00700 1 -- 172.21.15.57:0/4065915007 shutdown_connections 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.674+0000 7f411aa00700 1 --2- 172.21.15.57:0/4065915007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f41141088d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:35.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.674+0000 7f411aa00700 1 -- 172.21.15.57:0/4065915007 >> 172.21.15.57:0/4065915007 conn(0x7f41140fff10 msgr2=0x7f4114102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.674+0000 7f411aa00700 1 -- 172.21.15.57:0/4065915007 shutdown_connections 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.674+0000 7f411aa00700 1 -- 172.21.15.57:0/4065915007 wait complete. 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.675+0000 7f411aa00700 1 Processor -- start 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.675+0000 7f411aa00700 1 -- start start 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.675+0000 7f411aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.675+0000 7f411aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114199c40 con 0x7f41141064f0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.675+0000 7f4113e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4113e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58200/0 (socket says 172.21.15.57:58200) 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4113e00700 1 -- 172.21.15.57:0/2649498690 learned_addr learned my addr 172.21.15.57:0/2649498690 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4113e00700 1 -- 172.21.15.57:0/2649498690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009b30 con 0x7f41141064f0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4113e00700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4104004940 tx=0x7f4104004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f410400e040 con 0x7f41141064f0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f410400bc40 con 0x7f41141064f0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.676+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4114199e40 con 0x7f41141064f0 2024-02-28T22:55:35.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.677+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4104012670 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.677+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411419a340 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.678+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4114109ad0 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.678+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f4104019040 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.678+0000 7f4111600700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 0x7f40f803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.678+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f410404bcb0 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.679+0000 7f410b400700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 0x7f40f803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.679+0000 7f410b400700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 0x7f40f803a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.683+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f410401a570 con 0x7f41141064f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.916+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f4114101400 con 0x7f40f80383f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.928+0000 7f4111600700 1 -- 172.21.15.57:0/2649498690 <== mgr.14120 v2:172.21.15.57:6800/935476336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f4114101400 con 0x7f40f80383f0 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 msgr2=0x7f40f803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 0x7f40f803a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:35.660 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 msgr2=0x7f4114199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4104004940 tx=0x7f4104004a20 comp rx=0 tx=0).stop 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 shutdown_connections 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f40f80383f0 0x7f40f803a8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 --2- 172.21.15.57:0/2649498690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41141064f0 0x7f4114199700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.931+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 >> 172.21.15.57:0/2649498690 conn(0x7f41140fff10 msgr2=0x7f4114100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.932+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 shutdown_connections 2024-02-28T22:55:35.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:34.932+0000 7f411aa00700 1 -- 172.21.15.57:0/2649498690 wait complete. 2024-02-28T22:55:36.238 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:35 smithi057 ceph-mon[29096]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:36.238 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:35 smithi057 ceph-mon[29096]: Saving service alertmanager spec with placement count:1 2024-02-28T22:55:36.239 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:35 smithi057 ceph-mon[29096]: from='mgr.14120 172.21.15.57:0/2134911916' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:37.134 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:36 smithi057 ceph-mon[29096]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:37.134 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:36 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4205047534' entity='client.admin' 2024-02-28T22:55:37.150 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.268+0000 7f99bf000700 1 Processor -- start 2024-02-28T22:55:37.150 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.268+0000 7f99bf000700 1 -- start start 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.269+0000 7f99bf000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.269+0000 7f99bf000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8108920 con 0x7f99b8074450 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.269+0000 7f99bdc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.269+0000 7f99bdc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54716/0 (socket says 172.21.15.57:54716) 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.269+0000 7f99bdc00700 1 -- 172.21.15.57:0/2921614851 learned_addr learned my addr 172.21.15.57:0/2921614851 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.270+0000 7f99bdc00700 1 -- 172.21.15.57:0/2921614851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8108a60 con 0x7f99b8074450 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.270+0000 7f99bdc00700 1 --2- 172.21.15.57:0/2921614851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009e80 tx=0x7f99ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0a9e38c00b520e server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.270+0000 7f99b7e00700 1 -- 172.21.15.57:0/2921614851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac00e070 con 0x7f99b8074450 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.270+0000 7f99b7e00700 1 -- 172.21.15.57:0/2921614851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99ac00b8c0 con 0x7f99b8074450 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.270+0000 7f99b7e00700 1 -- 172.21.15.57:0/2921614851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac0044d0 con 0x7f99b8074450 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.271+0000 7f99bf000700 1 -- 172.21.15.57:0/2921614851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 msgr2=0x7f99b81083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.271+0000 7f99bf000700 1 --2- 172.21.15.57:0/2921614851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009e80 tx=0x7f99ac00b330 comp rx=0 tx=0).stop 2024-02-28T22:55:37.151 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.271+0000 7f99bf000700 1 -- 172.21.15.57:0/2921614851 shutdown_connections 2024-02-28T22:55:37.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.271+0000 7f99bf000700 1 --2- 172.21.15.57:0/2921614851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b81083e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:37.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.271+0000 7f99bf000700 1 -- 172.21.15.57:0/2921614851 >> 172.21.15.57:0/2921614851 conn(0x7f99b80ffd30 msgr2=0x7f99b8102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:37.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.272+0000 7f99bf000700 1 -- 172.21.15.57:0/2921614851 shutdown_connections 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.272+0000 7f99bf000700 1 -- 172.21.15.57:0/2921614851 wait complete. 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.273+0000 7f99bf000700 1 Processor -- start 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.273+0000 7f99bf000700 1 -- start start 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.273+0000 7f99bf000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.273+0000 7f99bf000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b819bcf0 con 0x7f99b8074450 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.274+0000 7f99bdc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.274+0000 7f99bdc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54726/0 (socket says 172.21.15.57:54726) 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.274+0000 7f99bdc00700 1 -- 172.21.15.57:0/4205047534 learned_addr learned my addr 172.21.15.57:0/4205047534 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.274+0000 7f99bdc00700 1 -- 172.21.15.57:0/4205047534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac009b30 con 0x7f99b8074450 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.274+0000 7f99bdc00700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99ac004070 tx=0x7f99ac0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.275+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac00e040 con 0x7f99b8074450 2024-02-28T22:55:37.156 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.275+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99ac00bc40 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.275+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac0125e0 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.275+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b819bef0 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.275+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b819c3f0 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.276+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f99ac019040 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.276+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b8062380 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.277+0000 7f99b6000700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 0x7f99a003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.277+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99ac04bd40 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.277+0000 7f99bd200700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 0x7f99a003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.278+0000 7f99bd200700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 0x7f99a003a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99a4006fd0 tx=0x7f99a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.282+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f99ac01a570 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.499+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f99b819ee60 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.507+0000 7f99b6000700 1 -- 172.21.15.57:0/4205047534 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f99ac01f0d0 con 0x7f99b8074450 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.510+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 msgr2=0x7f99a003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:37.157 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.510+0000 7f99bf000700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 0x7f99a003a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99a4006fd0 tx=0x7f99a4006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.510+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 msgr2=0x7f99b819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.510+0000 7f99bf000700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99ac004070 tx=0x7f99ac0049f0 comp rx=0 tx=0).stop 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 shutdown_connections 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f99a00383b0 0x7f99a003a860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 --2- 172.21.15.57:0/4205047534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99b8074450 0x7f99b819b7b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 >> 172.21.15.57:0/4205047534 conn(0x7f99b80ffd30 msgr2=0x7f99b81013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 shutdown_connections 2024-02-28T22:55:37.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:36.511+0000 7f99bf000700 1 -- 172.21.15.57:0/4205047534 wait complete. 2024-02-28T22:55:38.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.771+0000 7fdceca00700 1 Processor -- start 2024-02-28T22:55:38.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.771+0000 7fdceca00700 1 -- start start 2024-02-28T22:55:38.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.772+0000 7fdceca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.772+0000 7fdceca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8074510 con 0x7fdce81064f0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.772+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.772+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54734/0 (socket says 172.21.15.57:54734) 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.772+0000 7fdce5e00700 1 -- 172.21.15.57:0/720500974 learned_addr learned my addr 172.21.15.57:0/720500974 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.773+0000 7fdce5e00700 1 -- 172.21.15.57:0/720500974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8074650 con 0x7fdce81064f0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.773+0000 7fdce5e00700 1 --2- 172.21.15.57:0/720500974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80098a0 tx=0x7fdcd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7401ccd2748f999 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.773+0000 7fdce5400700 1 -- 172.21.15.57:0/720500974 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcd8004180 con 0x7fdce81064f0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.773+0000 7fdce5400700 1 -- 172.21.15.57:0/720500974 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcd80042e0 con 0x7fdce81064f0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.773+0000 7fdce5400700 1 -- 172.21.15.57:0/720500974 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcd8011af0 con 0x7fdce81064f0 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.774+0000 7fdceca00700 1 -- 172.21.15.57:0/720500974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 msgr2=0x7fdce81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.774+0000 7fdceca00700 1 --2- 172.21.15.57:0/720500974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80098a0 tx=0x7fdcd8009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.775+0000 7fdceca00700 1 -- 172.21.15.57:0/720500974 shutdown_connections 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.775+0000 7fdceca00700 1 --2- 172.21.15.57:0/720500974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce81088d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.775+0000 7fdceca00700 1 -- 172.21.15.57:0/720500974 >> 172.21.15.57:0/720500974 conn(0x7fdce80fff10 msgr2=0x7fdce8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.775+0000 7fdceca00700 1 -- 172.21.15.57:0/720500974 shutdown_connections 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.775+0000 7fdceca00700 1 -- 172.21.15.57:0/720500974 wait complete. 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdceca00700 1 Processor -- start 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdceca00700 1 -- start start 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdceca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdceca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce819bec0 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54742/0 (socket says 172.21.15.57:54742) 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.776+0000 7fdce5e00700 1 -- 172.21.15.57:0/193047215 learned_addr learned my addr 172.21.15.57:0/193047215 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdce5e00700 1 -- 172.21.15.57:0/193047215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8009550 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdce5e00700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8011670 tx=0x7fdcd8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcd80039f0 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8003b50 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcd80247c0 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.777+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce819c0c0 con 0x7fdce81064f0 2024-02-28T22:55:38.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.778+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce819c560 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.779+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fdcd801ac00 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.779+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce8109ad0 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.779+0000 7fdcdf400700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 0x7fdccc03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.779+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcd804c380 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.780+0000 7fdcdd400700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 0x7fdccc03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.780+0000 7fdcdd400700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 0x7fdccc03a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006fd0 tx=0x7fdcd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:37.785+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdcd8010380 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.007+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fdce819f3a0 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.015+0000 7fdcdf400700 1 -- 172.21.15.57:0/193047215 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fdcd8018b40 con 0x7fdce81064f0 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.018+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 msgr2=0x7fdccc03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.018+0000 7fdceca00700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 0x7fdccc03a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006fd0 tx=0x7fdcd0006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.018+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 msgr2=0x7fdce819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.018+0000 7fdceca00700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8011670 tx=0x7fdcd8011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 shutdown_connections 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7fdccc038440 0x7fdccc03a8f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:38.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 --2- 172.21.15.57:0/193047215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce81064f0 0x7fdce819b980 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:38.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 >> 172.21.15.57:0/193047215 conn(0x7fdce80fff10 msgr2=0x7fdce81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:38.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 shutdown_connections 2024-02-28T22:55:38.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:38.019+0000 7fdceca00700 1 -- 172.21.15.57:0/193047215 wait complete. 2024-02-28T22:55:38.659 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2024-02-28T22:55:39.165 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:39 smithi057 ceph-mon[29096]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:55:39.165 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:39 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/193047215' entity='client.admin' 2024-02-28T22:55:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:40 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3060809006' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-28T22:55:40.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.323+0000 7f645aa00700 1 Processor -- start 2024-02-28T22:55:40.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.323+0000 7f645aa00700 1 -- start start 2024-02-28T22:55:40.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.323+0000 7f645aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:40.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.323+0000 7f645aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645410f1f0 con 0x7f6454080210 2024-02-28T22:55:40.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.324+0000 7f6453e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.324+0000 7f6453e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54744/0 (socket says 172.21.15.57:54744) 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.324+0000 7f6453e00700 1 -- 172.21.15.57:0/383457299 learned_addr learned my addr 172.21.15.57:0/383457299 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.324+0000 7f6453e00700 1 -- 172.21.15.57:0/383457299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645410f330 con 0x7f6454080210 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.324+0000 7f6453e00700 1 --2- 172.21.15.57:0/383457299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64440098a0 tx=0x7f6444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d5bb40449b2db4 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f6453400700 1 -- 172.21.15.57:0/383457299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6444004180 con 0x7f6454080210 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f6453400700 1 -- 172.21.15.57:0/383457299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64440042e0 con 0x7f6454080210 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f6453400700 1 -- 172.21.15.57:0/383457299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6444011af0 con 0x7f6454080210 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f645aa00700 1 -- 172.21.15.57:0/383457299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 msgr2=0x7f645410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:40.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f645aa00700 1 --2- 172.21.15.57:0/383457299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64440098a0 tx=0x7f6444009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:40.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f645aa00700 1 -- 172.21.15.57:0/383457299 shutdown_connections 2024-02-28T22:55:40.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f645aa00700 1 --2- 172.21.15.57:0/383457299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645410ecb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:40.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.325+0000 7f645aa00700 1 -- 172.21.15.57:0/383457299 >> 172.21.15.57:0/383457299 conn(0x7f645407b200 msgr2=0x7f645407d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:40.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.326+0000 7f645aa00700 1 -- 172.21.15.57:0/383457299 shutdown_connections 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.326+0000 7f645aa00700 1 -- 172.21.15.57:0/383457299 wait complete. 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f645aa00700 1 Processor -- start 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f645aa00700 1 -- start start 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f645aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f645aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541a0310 con 0x7f6454080210 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f6453e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f6453e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54760/0 (socket says 172.21.15.57:54760) 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.327+0000 7f6453e00700 1 -- 172.21.15.57:0/3060809006 learned_addr learned my addr 172.21.15.57:0/3060809006 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f6453e00700 1 -- 172.21.15.57:0/3060809006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009550 con 0x7f6454080210 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f6453e00700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6444011670 tx=0x7f6444011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64440039f0 con 0x7f6454080210 2024-02-28T22:55:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6444003b50 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64541a0510 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.328+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64440247c0 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.329+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64541a09b0 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.330+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f644401a430 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.330+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64541100b0 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.330+0000 7f6451600700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 0x7f643803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.330+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f644404c370 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.330+0000 7f644b400700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 0x7f643803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.331+0000 7f644b400700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 0x7f643803a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f643c00ad30 tx=0x7f643c0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.336+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6444018b40 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:39.615+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f64541a37f0 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.037+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f644402d910 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.040+0000 7f6451600700 1 -- 172.21.15.57:0/3060809006 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f644401acb0 con 0x7f6454080210 2024-02-28T22:55:40.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 msgr2=0x7f643803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 0x7f643803a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f643c00ad30 tx=0x7f643c0093f0 comp rx=0 tx=0).stop 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 msgr2=0x7f645419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6444011670 tx=0x7f6444011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 shutdown_connections 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f6438038440 0x7f643803a8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 --2- 172.21.15.57:0/3060809006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6454080210 0x7f645419fdd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 >> 172.21.15.57:0/3060809006 conn(0x7f645407b200 msgr2=0x7f645407d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 shutdown_connections 2024-02-28T22:55:40.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:40.044+0000 7f645aa00700 1 -- 172.21.15.57:0/3060809006 wait complete. 2024-02-28T22:55:41.236 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:41 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3060809006' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-28T22:55:41.236 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:41 smithi057 ceph-mon[29096]: mgrmap e9: smithi057.zrrldj(active, since 34s) 2024-02-28T22:55:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:42 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2511640081' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-28T22:55:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:55:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-28T22:55:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-28T22:55:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "smithi057.zrrldj", 2024-02-28T22:55:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.502+0000 7f63f3e00700 1 Processor -- start 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.502+0000 7f63f3e00700 1 -- start start 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.503+0000 7f63f3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.503+0000 7f63f3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4106150 con 0x7f63f4105800 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.503+0000 7f63f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.503+0000 7f63f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54792/0 (socket says 172.21.15.57:54792) 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.503+0000 7f63f2a00700 1 -- 172.21.15.57:0/3463568610 learned_addr learned my addr 172.21.15.57:0/3463568610 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.504+0000 7f63f2a00700 1 -- 172.21.15.57:0/3463568610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4106290 con 0x7f63f4105800 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.504+0000 7f63f2a00700 1 --2- 172.21.15.57:0/3463568610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63e40098a0 tx=0x7f63e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e023ee3b2ecc1a server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.504+0000 7f63f1600700 1 -- 172.21.15.57:0/3463568610 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e4004180 con 0x7f63f4105800 2024-02-28T22:55:42.479 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.504+0000 7f63f1600700 1 -- 172.21.15.57:0/3463568610 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63e40042e0 con 0x7f63f4105800 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.504+0000 7f63f1600700 1 -- 172.21.15.57:0/3463568610 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e4011af0 con 0x7f63f4105800 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.505+0000 7f63f3e00700 1 -- 172.21.15.57:0/3463568610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 msgr2=0x7f63f4105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.505+0000 7f63f3e00700 1 --2- 172.21.15.57:0/3463568610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63e40098a0 tx=0x7f63e4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.505+0000 7f63f3e00700 1 -- 172.21.15.57:0/3463568610 shutdown_connections 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.505+0000 7f63f3e00700 1 --2- 172.21.15.57:0/3463568610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f4105c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.505+0000 7f63f3e00700 1 -- 172.21.15.57:0/3463568610 >> 172.21.15.57:0/3463568610 conn(0x7f63f4100e50 msgr2=0x7f63f4103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.506+0000 7f63f3e00700 1 -- 172.21.15.57:0/3463568610 shutdown_connections 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.506+0000 7f63f3e00700 1 -- 172.21.15.57:0/3463568610 wait complete. 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f3e00700 1 Processor -- start 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f3e00700 1 -- start start 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f419bcf0 con 0x7f63f4105800 2024-02-28T22:55:42.480 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54798/0 (socket says 172.21.15.57:54798) 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.507+0000 7f63f2a00700 1 -- 172.21.15.57:0/2511640081 learned_addr learned my addr 172.21.15.57:0/2511640081 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63f2a00700 1 -- 172.21.15.57:0/2511640081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4009550 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63f2a00700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63e4011670 tx=0x7f63e4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e4003a40 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63e4003ba0 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e4024790 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.508+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f419bef0 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.509+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f419c390 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.510+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f63e401a430 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.510+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f4062380 con 0x7f63f4105800 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.511+0000 7f63eb400700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 0x7f63d803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.511+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f63e404c5b0 con 0x7f63f4105800 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.511+0000 7f63f2000700 1 -- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 msgr2=0x7f63d803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.511+0000 7f63f2000700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 0x7f63d803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.516+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f63e401ac50 con 0x7f63f4105800 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.712+0000 7f63f2000700 1 -- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 msgr2=0x7f63d803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.712+0000 7f63f2000700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 0x7f63d803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.792+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f63f419f520 con 0x7f63f4105800 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.793+0000 7f63eb400700 1 -- 172.21.15.57:0/2511640081 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f63e404bf20 con 0x7f63f4105800 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 msgr2=0x7f63d803a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 0x7f63d803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 msgr2=0x7f63f419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63e4011670 tx=0x7f63e4011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 shutdown_connections 2024-02-28T22:55:42.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f63d8038440 0x7f63d803a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 --2- 172.21.15.57:0/2511640081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63f4105800 0x7f63f419b7b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 >> 172.21.15.57:0/2511640081 conn(0x7f63f4100e50 msgr2=0x7f63f41024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 shutdown_connections 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:41.797+0000 7f63f3e00700 1 -- 172.21.15.57:0/2511640081 wait complete. 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-02-28T22:55:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2024-02-28T22:55:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: Active manager daemon smithi057.zrrldj restarted 2024-02-28T22:55:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: Activating manager daemon smithi057.zrrldj 2024-02-28T22:55:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: osdmap e3: 0 total, 0 up, 0 in 2024-02-28T22:55:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: mgrmap e10: smithi057.zrrldj(active, starting, since 0.0130392s) 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T22:55:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:49 smithi057 ceph-mon[29096]: Manager daemon smithi057.zrrldj is now available 2024-02-28T22:55:50.250 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:50 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:50.251 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:50 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:55:50.251 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:50 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T22:55:50.251 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:50 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T22:55:50.251 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:50 smithi057 ceph-mon[29096]: mgrmap e11: smithi057.zrrldj(active, since 1.0136s) 2024-02-28T22:55:51.138 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:51 smithi057 ceph-mon[29096]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-28T22:55:51.138 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:51 smithi057 ceph-mon[29096]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-28T22:55:51.138 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:51 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:51.152 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-02-28T22:55:51.153 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-28T22:55:51.153 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-28T22:55:51.153 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-02-28T22:55:51.153 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.069+0000 7f3ef5800700 1 Processor -- start 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.069+0000 7f3ef5800700 1 -- start start 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.070+0000 7f3ef5800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.070+0000 7f3ef5800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00744d0 con 0x7f3ef0104320 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.071+0000 7f3eeea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.071+0000 7f3eeea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54812/0 (socket says 172.21.15.57:54812) 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.071+0000 7f3eeea00700 1 -- 172.21.15.57:0/4180293695 learned_addr learned my addr 172.21.15.57:0/4180293695 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.071+0000 7f3eeea00700 1 -- 172.21.15.57:0/4180293695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0074610 con 0x7f3ef0104320 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3eeea00700 1 --2- 172.21.15.57:0/4180293695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00098a0 tx=0x7f3ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aac73aa350b74dd server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3eed600700 1 -- 172.21.15.57:0/4180293695 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0004180 con 0x7f3ef0104320 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3eed600700 1 -- 172.21.15.57:0/4180293695 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3ee00042e0 con 0x7f3ef0104320 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3eed600700 1 -- 172.21.15.57:0/4180293695 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0011af0 con 0x7f3ef0104320 2024-02-28T22:55:51.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3ef5800700 1 -- 172.21.15.57:0/4180293695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 msgr2=0x7f3ef0104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.072+0000 7f3ef5800700 1 --2- 172.21.15.57:0/4180293695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00098a0 tx=0x7f3ee0009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.073+0000 7f3ef5800700 1 -- 172.21.15.57:0/4180293695 shutdown_connections 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.073+0000 7f3ef5800700 1 --2- 172.21.15.57:0/4180293695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef0104730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.073+0000 7f3ef5800700 1 -- 172.21.15.57:0/4180293695 >> 172.21.15.57:0/4180293695 conn(0x7f3ef00fff10 msgr2=0x7f3ef0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.073+0000 7f3ef5800700 1 -- 172.21.15.57:0/4180293695 shutdown_connections 2024-02-28T22:55:51.158 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.073+0000 7f3ef5800700 1 -- 172.21.15.57:0/4180293695 wait complete. 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.074+0000 7f3ef5800700 1 Processor -- start 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.074+0000 7f3ef5800700 1 -- start start 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.074+0000 7f3ef5800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.074+0000 7f3ef5800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019bea0 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.075+0000 7f3eeea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.075+0000 7f3eeea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54818/0 (socket says 172.21.15.57:54818) 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.075+0000 7f3eeea00700 1 -- 172.21.15.57:0/2128379001 learned_addr learned my addr 172.21.15.57:0/2128379001 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.075+0000 7f3eeea00700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0009550 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.075+0000 7f3eeea00700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0011670 tx=0x7f3ee0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.076+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee00039f0 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.076+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3ee0003b50 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.076+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee00247c0 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.076+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef019c100 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.076+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef019ef00 con 0x7f3ef0104320 2024-02-28T22:55:51.159 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.077+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f3ee001ac00 con 0x7f3ef0104320 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.077+0000 7f3ee7400700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.077+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ed403b050 con 0x7f3ed4038490 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.077+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3ee004c4a0 con 0x7f3ef0104320 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.077+0000 7f3eee000700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.078+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:55:51.160 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.279+0000 7f3eee000700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.279+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.680+0000 7f3eee000700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:43.680+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:44.481+0000 7f3eee000700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:44.481+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:46.083+0000 7f3eee000700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/935476336 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:46.083+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:48.985+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7f3ee002d770 con 0x7f3ef0104320 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:48.985+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 msgr2=0x7f3ed403a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:55:51.163 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:48.985+0000 7f3ee7400700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.987+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f3ee004de10 con 0x7f3ef0104320 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.987+0000 7f3ee7400700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 0x7f3ed403e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.987+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ed403b050 con 0x7f3ed403bef0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.987+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 0x7f3ed403e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.989+0000 7f3eee000700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 0x7f3ed403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3edc003a10 tx=0x7f3edc009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.990+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f3ed403b050 con 0x7f3ed403bef0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.997+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3ef019cca0 con 0x7f3ed403bef0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ee7400700 1 -- 172.21.15.57:0/2128379001 <== mgr.14164 v2:172.21.15.57:6800/2764732365 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f3ef019cca0 con 0x7f3ed403bef0 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 msgr2=0x7f3ed403e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 0x7f3ed403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3edc003a10 tx=0x7f3edc009270 comp rx=0 tx=0).stop 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 msgr2=0x7f3ef019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0011670 tx=0x7f3ee0011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:51.164 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 shutdown_connections 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ef0104320 0x7f3ef019b960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f3ed403bef0 0x7f3ed403e2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 --2- 172.21.15.57:0/2128379001 >> [v2:172.21.15.57:6800/935476336,v1:172.21.15.57:6801/935476336] conn(0x7f3ed4038490 0x7f3ed403a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.998+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 >> 172.21.15.57:0/2128379001 conn(0x7f3ef00fff10 msgr2=0x7f3ef018fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.999+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 shutdown_connections 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:49.999+0000 7f3ef5800700 1 -- 172.21.15.57:0/2128379001 wait complete. 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2024-02-28T22:55:51.165 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2024-02-28T22:55:52.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:52 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:51] ENGINE Bus STARTING 2024-02-28T22:55:52.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:52 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:51] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T22:55:52.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:52 smithi057 ceph-mon[29096]: mgrmap e12: smithi057.zrrldj(active, since 2s) 2024-02-28T22:55:52.762 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.763+0000 7f82ede00700 1 Processor -- start 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.763+0000 7f82ede00700 1 -- start start 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.763+0000 7f82ede00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.764+0000 7f82ede00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8106310 con 0x7f82e81059c0 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.764+0000 7f82e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:52.763 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.764+0000 7f82e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36404/0 (socket says 172.21.15.57:36404) 2024-02-28T22:55:52.768 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.764+0000 7f82e6a00700 1 -- 172.21.15.57:0/3358309007 learned_addr learned my addr 172.21.15.57:0/3358309007 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:52.768 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.765+0000 7f82e6a00700 1 -- 172.21.15.57:0/3358309007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8106450 con 0x7f82e81059c0 2024-02-28T22:55:52.768 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.765+0000 7f82e6a00700 1 --2- 172.21.15.57:0/3358309007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82d80098a0 tx=0x7f82d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1f087bb8525281 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.765+0000 7f82e5600700 1 -- 172.21.15.57:0/3358309007 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d8004180 con 0x7f82e81059c0 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.765+0000 7f82e5600700 1 -- 172.21.15.57:0/3358309007 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82d80042e0 con 0x7f82e81059c0 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.765+0000 7f82e5600700 1 -- 172.21.15.57:0/3358309007 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d8011af0 con 0x7f82e81059c0 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 -- 172.21.15.57:0/3358309007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 msgr2=0x7f82e8105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 --2- 172.21.15.57:0/3358309007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82d80098a0 tx=0x7f82d8009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 -- 172.21.15.57:0/3358309007 shutdown_connections 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 --2- 172.21.15.57:0/3358309007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e8105dd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 -- 172.21.15.57:0/3358309007 >> 172.21.15.57:0/3358309007 conn(0x7f82e8100fd0 msgr2=0x7f82e8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 -- 172.21.15.57:0/3358309007 shutdown_connections 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.766+0000 7f82ede00700 1 -- 172.21.15.57:0/3358309007 wait complete. 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.767+0000 7f82ede00700 1 Processor -- start 2024-02-28T22:55:52.769 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82ede00700 1 -- start start 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82ede00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82ede00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e819be70 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36416/0 (socket says 172.21.15.57:36416) 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.768+0000 7f82e6a00700 1 -- 172.21.15.57:0/4194161308 learned_addr learned my addr 172.21.15.57:0/4194161308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82e6a00700 1 -- 172.21.15.57:0/4194161308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8009550 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82e6a00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82d8011670 tx=0x7f82d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d8003a40 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82d8003ba0 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e819c070 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.769+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d8024820 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.770+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e819c510 con 0x7f82e81059c0 2024-02-28T22:55:52.770 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.770+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f82d801ac70 con 0x7f82e81059c0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.771+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8062380 con 0x7f82e81059c0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.771+0000 7f82eca00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 0x7f82cc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.771+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82d804c300 con 0x7f82e81059c0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.771+0000 7f82e6000700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 0x7f82cc03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.772+0000 7f82e6000700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 0x7f82cc03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006fd0 tx=0x7f82d4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.777+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f82d8018760 con 0x7f82e81059c0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:51.920+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f82d801a950 con 0x7f82e81059c0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.006+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f82e81024d0 con 0x7f82cc0382e0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.091+0000 7f82eca00700 1 -- 172.21.15.57:0/4194161308 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f82e81024d0 con 0x7f82cc0382e0 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 msgr2=0x7f82cc03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 0x7f82cc03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006fd0 tx=0x7f82d4006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 msgr2=0x7f82e819b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82d8011670 tx=0x7f82d8011aa0 comp rx=0 tx=0).stop 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 shutdown_connections 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f82cc0382e0 0x7f82cc03a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 --2- 172.21.15.57:0/4194161308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81059c0 0x7f82e819b930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.094+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 >> 172.21.15.57:0/4194161308 conn(0x7f82e8100fd0 msgr2=0x7f82e8101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.095+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 shutdown_connections 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:52.095+0000 7f82ede00700 1 -- 172.21.15.57:0/4194161308 wait complete. 2024-02-28T22:55:52.772 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:51] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: [28/Feb/2024:22:55:51] ENGINE Bus STARTED 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:53.291 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:53 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:54 smithi057 ceph-mon[29096]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:55:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:54 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jX/rxEQoSPfPiZHK1noXxu0i75J1T7dEGtaOXGWfMzqwtzNxp1ITe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709160953, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.448+0000 7f55ebc00700 1 Processor -- start 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.449+0000 7f55ebc00700 1 -- start start 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.449+0000 7f55ebc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.449+0000 7f55ebc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e4106360 con 0x7f55e4079cd0 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.450+0000 7f55e9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.450+0000 7f55e9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36426/0 (socket says 172.21.15.57:36426) 2024-02-28T22:55:55.218 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.450+0000 7f55e9200700 1 -- 172.21.15.57:0/218472392 learned_addr learned my addr 172.21.15.57:0/218472392 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.450+0000 7f55e9200700 1 -- 172.21.15.57:0/218472392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e410aeb0 con 0x7f55e4079cd0 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.451+0000 7f55e9200700 1 --2- 172.21.15.57:0/218472392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d80098a0 tx=0x7f55d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce1c5784152df55 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.451+0000 7f55e3400700 1 -- 172.21.15.57:0/218472392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55d8004180 con 0x7f55e4079cd0 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.451+0000 7f55e3400700 1 -- 172.21.15.57:0/218472392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55d80042e0 con 0x7f55e4079cd0 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.451+0000 7f55e3400700 1 -- 172.21.15.57:0/218472392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55d8011af0 con 0x7f55e4079cd0 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 -- 172.21.15.57:0/218472392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 msgr2=0x7f55e410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 --2- 172.21.15.57:0/218472392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d80098a0 tx=0x7f55d8009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 -- 172.21.15.57:0/218472392 shutdown_connections 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 --2- 172.21.15.57:0/218472392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e4079cd0 0x7f55e410a970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 -- 172.21.15.57:0/218472392 >> 172.21.15.57:0/218472392 conn(0x7f55e40fff00 msgr2=0x7f55e4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 -- 172.21.15.57:0/218472392 shutdown_connections 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.452+0000 7f55ebc00700 1 -- 172.21.15.57:0/218472392 wait complete. 2024-02-28T22:55:55.219 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.454+0000 7f55ebc00700 1 Processor -- start 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.454+0000 7f55ebc00700 1 -- start start 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.454+0000 7f55ebc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.454+0000 7f55ebc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419c270 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.455+0000 7f55e9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.455+0000 7f55e9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36438/0 (socket says 172.21.15.57:36438) 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.455+0000 7f55e9200700 1 -- 172.21.15.57:0/2845438201 learned_addr learned my addr 172.21.15.57:0/2845438201 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.455+0000 7f55e9200700 1 -- 172.21.15.57:0/2845438201 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009550 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.455+0000 7f55e9200700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55d8011670 tx=0x7f55d8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.456+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55d80039f0 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.456+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55d8003b50 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.456+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55d80247c0 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.456+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e419c470 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.456+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e419f0d0 con 0x7f55e419b920 2024-02-28T22:55:55.220 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.457+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e4079cd0 con 0x7f55e419b920 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.458+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f55d801ac00 con 0x7f55e419b920 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.458+0000 7f55e1600700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 0x7f55cc03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.458+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55d804c5a0 con 0x7f55e419b920 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.459+0000 7f55e3e00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 0x7f55cc03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.459+0000 7f55e3e00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 0x7f55cc03a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55d0006fd0 tx=0x7f55d0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.468+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f55d8010380 con 0x7f55e419b920 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.711+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f55e40621a0 con 0x7f55cc038490 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.971+0000 7f55e1600700 1 -- 172.21.15.57:0/2845438201 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f55e40621a0 con 0x7f55cc038490 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 msgr2=0x7f55cc03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 0x7f55cc03a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55d0006fd0 tx=0x7f55d0006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 msgr2=0x7f55e419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55d8011670 tx=0x7f55d8011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 shutdown_connections 2024-02-28T22:55:55.221 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f55cc038490 0x7f55cc03a940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:55.222 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 --2- 172.21.15.57:0/2845438201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55e419b920 0x7f55e419bd30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:55.222 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 >> 172.21.15.57:0/2845438201 conn(0x7f55e40fff00 msgr2=0x7f55e4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:55.222 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 shutdown_connections 2024-02-28T22:55:55.222 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:53.974+0000 7f55ebc00700 1 -- 172.21.15.57:0/2845438201 wait complete. 2024-02-28T22:55:55.222 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.825+0000 7f8c3f200700 1 Processor -- start 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.826+0000 7f8c3f200700 1 -- start start 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.826+0000 7f8c3f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.826+0000 7f8c3f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40108940 con 0x7f8c40079c70 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.826+0000 7f8c3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:56.707 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.827+0000 7f8c3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43018/0 (socket says 172.21.15.57:43018) 2024-02-28T22:55:56.708 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.827+0000 7f8c3de00700 1 -- 172.21.15.57:0/694709631 learned_addr learned my addr 172.21.15.57:0/694709631 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:56.708 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.827+0000 7f8c3de00700 1 -- 172.21.15.57:0/694709631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40108a80 con 0x7f8c40079c70 2024-02-28T22:55:56.708 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.827+0000 7f8c3de00700 1 --2- 172.21.15.57:0/694709631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009e80 tx=0x7f8c3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a3701f63a1f33c server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:56.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.827+0000 7f8c3ca00700 1 -- 172.21.15.57:0/694709631 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3000e070 con 0x7f8c40079c70 2024-02-28T22:55:56.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3ca00700 1 -- 172.21.15.57:0/694709631 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c3000b8c0 con 0x7f8c40079c70 2024-02-28T22:55:56.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3ca00700 1 -- 172.21.15.57:0/694709631 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c300044d0 con 0x7f8c40079c70 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 -- 172.21.15.57:0/694709631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 msgr2=0x7f8c40108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 --2- 172.21.15.57:0/694709631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009e80 tx=0x7f8c3000b330 comp rx=0 tx=0).stop 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 -- 172.21.15.57:0/694709631 shutdown_connections 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 --2- 172.21.15.57:0/694709631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40108400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 -- 172.21.15.57:0/694709631 >> 172.21.15.57:0/694709631 conn(0x7f8c400ffd10 msgr2=0x7f8c40102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 -- 172.21.15.57:0/694709631 shutdown_connections 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.828+0000 7f8c3f200700 1 -- 172.21.15.57:0/694709631 wait complete. 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3f200700 1 Processor -- start 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3f200700 1 -- start start 2024-02-28T22:55:56.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:56.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40106290 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43032/0 (socket says 172.21.15.57:43032) 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.830+0000 7f8c3de00700 1 -- 172.21.15.57:0/4263341603 learned_addr learned my addr 172.21.15.57:0/4263341603 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c3de00700 1 -- 172.21.15.57:0/4263341603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009b30 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c3de00700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c30004940 tx=0x7f8c30004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3000e040 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c3000bc40 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c401043a0 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.831+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c30012670 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.832+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c401048a0 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.833+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f8c30019040 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.833+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40062380 con 0x7f8c40079c70 2024-02-28T22:55:56.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.833+0000 7f8c36a00700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 0x7f8c2403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.833+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c3004bd50 con 0x7f8c40079c70 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.833+0000 7f8c3d400700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 0x7f8c2403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.834+0000 7f8c3d400700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 0x7f8c2403a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006fd0 tx=0x7f8c2c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.839+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8c3000bdb0 con 0x7f8c40079c70 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:55.979+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8c30018d40 con 0x7f8c40079c70 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.062+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8c40104bf0 con 0x7f8c40079c70 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.063+0000 7f8c36a00700 1 -- 172.21.15.57:0/4263341603 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8c3001f020 con 0x7f8c40079c70 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.066+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 msgr2=0x7f8c2403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.066+0000 7f8c3f200700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 0x7f8c2403a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006fd0 tx=0x7f8c2c006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.066+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 msgr2=0x7f8c40105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.066+0000 7f8c3f200700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c30004940 tx=0x7f8c30004a20 comp rx=0 tx=0).stop 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 shutdown_connections 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8c24038490 0x7f8c2403a940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:56.715 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 --2- 172.21.15.57:0/4263341603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c40079c70 0x7f8c40105d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:56.716 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 >> 172.21.15.57:0/4263341603 conn(0x7f8c400ffd10 msgr2=0x7f8c40100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:56.716 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 shutdown_connections 2024-02-28T22:55:56.716 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:56.067+0000 7f8c3f200700 1 -- 172.21.15.57:0/4263341603 wait complete. 2024-02-28T22:55:56.743 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-28T22:55:56.743 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-02-28T22:55:56.766 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-28T22:55:56.766 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-02-28T22:55:56.767 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-02-28T22:55:56.767 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057.front.sepia.ceph.com:8443/ 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout: Password: c8bmtxfsum 2024-02-28T22:55:56.775 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:56.776 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config directory 2024-02-28T22:55:56.778 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling autotune for osd_memory_target 2024-02-28T22:55:57.234 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:56 smithi057 ceph-mon[29096]: mgrmap e13: smithi057.zrrldj(active, since 7s) 2024-02-28T22:55:57.235 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:55:56 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4263341603' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.403+0000 7f66d9c00700 1 Processor -- start 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.404+0000 7f66d9c00700 1 -- start start 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.404+0000 7f66d9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.404+0000 7f66d9c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4108510 con 0x7f66d4107bc0 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.405+0000 7f66d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.405+0000 7f66d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43046/0 (socket says 172.21.15.57:43046) 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.405+0000 7f66d2a00700 1 -- 172.21.15.57:0/2466079311 learned_addr learned my addr 172.21.15.57:0/2466079311 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.405+0000 7f66d2a00700 1 -- 172.21.15.57:0/2466079311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d4108650 con 0x7f66d4107bc0 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.405+0000 7f66d2a00700 1 --2- 172.21.15.57:0/2466079311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098a0 tx=0x7f66c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b7eb7186a88c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:58.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.406+0000 7f66d2000700 1 -- 172.21.15.57:0/2466079311 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66c4004180 con 0x7f66d4107bc0 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.406+0000 7f66d2000700 1 -- 172.21.15.57:0/2466079311 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66c40042e0 con 0x7f66d4107bc0 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.406+0000 7f66d2000700 1 -- 172.21.15.57:0/2466079311 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66c4011af0 con 0x7f66d4107bc0 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.406+0000 7f66d9c00700 1 -- 172.21.15.57:0/2466079311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 msgr2=0x7f66d4107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.406+0000 7f66d9c00700 1 --2- 172.21.15.57:0/2466079311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098a0 tx=0x7f66c4009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.407+0000 7f66d9c00700 1 -- 172.21.15.57:0/2466079311 shutdown_connections 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.407+0000 7f66d9c00700 1 --2- 172.21.15.57:0/2466079311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d4107fd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.407+0000 7f66d9c00700 1 -- 172.21.15.57:0/2466079311 >> 172.21.15.57:0/2466079311 conn(0x7f66d4103550 msgr2=0x7f66d4105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.407+0000 7f66d9c00700 1 -- 172.21.15.57:0/2466079311 shutdown_connections 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.407+0000 7f66d9c00700 1 -- 172.21.15.57:0/2466079311 wait complete. 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.408+0000 7f66d9c00700 1 Processor -- start 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.408+0000 7f66d9c00700 1 -- start start 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.408+0000 7f66d9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.408+0000 7f66d9c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419bec0 con 0x7f66d4107bc0 2024-02-28T22:55:58.282 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.408+0000 7f66d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.409+0000 7f66d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43056/0 (socket says 172.21.15.57:43056) 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.409+0000 7f66d2a00700 1 -- 172.21.15.57:0/1697816308 learned_addr learned my addr 172.21.15.57:0/1697816308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.409+0000 7f66d2a00700 1 -- 172.21.15.57:0/1697816308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009550 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.409+0000 7f66d2a00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c4011670 tx=0x7f66c4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.409+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66c4003a40 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.410+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66c4003ba0 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.410+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d419c0c0 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.410+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66c4024790 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.410+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d419c560 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.411+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f66c401a430 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.411+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d4062380 con 0x7f66d4107bc0 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.411+0000 7f66cbe00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 0x7f66b803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:58.283 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.411+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f66c404c740 con 0x7f66d4107bc0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.412+0000 7f66ca000700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 0x7f66b803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.412+0000 7f66ca000700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 0x7f66b803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ad30 tx=0x7f66bc0093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.417+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f66c401ac50 con 0x7f66d4107bc0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.636+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f66d419f460 con 0x7f66d4107bc0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.637+0000 7f66cbe00700 1 -- 172.21.15.57:0/1697816308 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f66c4018b40 con 0x7f66d4107bc0 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 msgr2=0x7f66b803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 0x7f66b803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ad30 tx=0x7f66bc0093f0 comp rx=0 tx=0).stop 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 msgr2=0x7f66d419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c4011670 tx=0x7f66c4011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 shutdown_connections 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f66b8038580 0x7f66b803aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 --2- 172.21.15.57:0/1697816308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4107bc0 0x7f66d419b980 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:58.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 >> 172.21.15.57:0/1697816308 conn(0x7f66d4103550 msgr2=0x7f66d4105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:58.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 shutdown_connections 2024-02-28T22:55:58.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:57.640+0000 7f66d9c00700 1 -- 172.21.15.57:0/1697816308 wait complete. 2024-02-28T22:55:59.900 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.926+0000 7f0a89e00700 1 Processor -- start 2024-02-28T22:55:59.900 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.926+0000 7f0a89e00700 1 -- start start 2024-02-28T22:55:59.901 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.926+0000 7f0a89e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:59.901 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.926+0000 7f0a89e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8410f1f0 con 0x7f0a84080210 2024-02-28T22:55:59.901 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.927+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:59.901 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.927+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43070/0 (socket says 172.21.15.57:43070) 2024-02-28T22:55:59.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.927+0000 7f0a82a00700 1 -- 172.21.15.57:0/2594546115 learned_addr learned my addr 172.21.15.57:0/2594546115 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:59.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.927+0000 7f0a82a00700 1 -- 172.21.15.57:0/2594546115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8410f330 con 0x7f0a84080210 2024-02-28T22:55:59.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a82a00700 1 --2- 172.21.15.57:0/2594546115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a740098a0 tx=0x7f0a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1017b4e4c927c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:59.902 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a81600700 1 -- 172.21.15.57:0/2594546115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a74004180 con 0x7f0a84080210 2024-02-28T22:55:59.903 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a81600700 1 -- 172.21.15.57:0/2594546115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a740042e0 con 0x7f0a84080210 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a81600700 1 -- 172.21.15.57:0/2594546115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a74011af0 con 0x7f0a84080210 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a89e00700 1 -- 172.21.15.57:0/2594546115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 msgr2=0x7f0a8410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.928+0000 7f0a89e00700 1 --2- 172.21.15.57:0/2594546115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a740098a0 tx=0x7f0a74009bb0 comp rx=0 tx=0).stop 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.929+0000 7f0a89e00700 1 -- 172.21.15.57:0/2594546115 shutdown_connections 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.929+0000 7f0a89e00700 1 --2- 172.21.15.57:0/2594546115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8410ecb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.929+0000 7f0a89e00700 1 -- 172.21.15.57:0/2594546115 >> 172.21.15.57:0/2594546115 conn(0x7f0a8407b200 msgr2=0x7f0a8407d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.929+0000 7f0a89e00700 1 -- 172.21.15.57:0/2594546115 shutdown_connections 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.929+0000 7f0a89e00700 1 -- 172.21.15.57:0/2594546115 wait complete. 2024-02-28T22:55:59.904 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.930+0000 7f0a89e00700 1 Processor -- start 2024-02-28T22:55:59.905 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.930+0000 7f0a89e00700 1 -- start start 2024-02-28T22:55:59.906 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.930+0000 7f0a89e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:59.906 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.930+0000 7f0a89e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841a0310 con 0x7f0a84080210 2024-02-28T22:55:59.906 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.931+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:59.906 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.931+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43078/0 (socket says 172.21.15.57:43078) 2024-02-28T22:55:59.906 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.931+0000 7f0a82a00700 1 -- 172.21.15.57:0/3649334780 learned_addr learned my addr 172.21.15.57:0/3649334780 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.931+0000 7f0a82a00700 1 -- 172.21.15.57:0/3649334780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74009550 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.931+0000 7f0a82a00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a74011670 tx=0x7f0a74011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.932+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a740039f0 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.932+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a74003b50 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.932+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a740247c0 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.932+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a841a0510 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.932+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841a09b0 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.933+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a841100b0 con 0x7f0a84080210 2024-02-28T22:55:59.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.933+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f0a7401ac00 con 0x7f0a84080210 2024-02-28T22:55:59.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.933+0000 7f0a88a00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 0x7f0a6803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:55:59.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.934+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a7404c640 con 0x7f0a84080210 2024-02-28T22:55:59.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.934+0000 7f0a82000700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 0x7f0a6803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:55:59.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.935+0000 7f0a82000700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 0x7f0a6803a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006fd0 tx=0x7f0a70006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:55:59.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:58.939+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a7401a430 con 0x7f0a84080210 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.276+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f0a841102d0 con 0x7f0a84080210 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.284+0000 7f0a88a00700 1 -- 172.21.15.57:0/3649334780 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0a74018b40 con 0x7f0a84080210 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 msgr2=0x7f0a6803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 0x7f0a6803a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006fd0 tx=0x7f0a70006e40 comp rx=0 tx=0).stop 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 msgr2=0x7f0a8419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a74011670 tx=0x7f0a74011b50 comp rx=0 tx=0).stop 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 shutdown_connections 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f0a68038530 0x7f0a6803a9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 --2- 172.21.15.57:0/3649334780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a84080210 0x7f0a8419fdd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 >> 172.21.15.57:0/3649334780 conn(0x7f0a8407b200 msgr2=0x7f0a8407d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 shutdown_connections 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-02-28T22:55:59.287+0000 7f0a89e00700 1 -- 172.21.15.57:0/3649334780 wait complete. 2024-02-28T22:55:59.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.911 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:55:59.912 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2024-02-28T22:56:00.032 INFO:tasks.cephadm:Fetching config... 2024-02-28T22:56:00.032 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:56:00.032 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-28T22:56:00.084 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-28T22:56:00.084 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:56:00.084 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-28T22:56:00.166 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-28T22:56:00.166 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:56:00.166 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/keyring of=/dev/stdout 2024-02-28T22:56:00.304 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-28T22:56:00.305 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:56:00.305 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-28T22:56:00.361 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-28T22:56:00.361 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmPBU8HH15blr49NXBrFs0hBCGaBdpz5Msl6Nja7XP7J1IaHJTcloCgjxSMWC3Y8TGn6llq7bru5xa4LPusRS6j7lFl6aOtydln49W86jkWnJ8RYQmZ7DNbaLI2j6gpgDOhWHlXZtaFgxKoCXBXmLYxuTd4Xtl2fMQqZbjBHzyeO4UpmEiLaGrpE4LmQarb2nlC0/4V8kp56cGzK6b208Q+dBsEvoABbFj8cqjnGNH0rK52/BAXVzGET+L4SWD6Q7qkevIIqA16OF+f0qcj7hPoNyPO0Up55V7r34G4M9SXZCwwHYY8p7eFA4evinVxmkduBzgiZnBdSiiqymbvbmoVBTWgZ+ZNWD+hLyyL3q/RxbJjTOBYkkIvNInTAZeP7WKG/jh8h80z7PUKTlSyC87ubcs1+aFsRJownGucz2S9s5bnoI/4mO9c8rQtejeKEeSDE5cxf7R5/GRpizos3XUlMQhp+a1QzLrrzFxotjWjlRJtDCSg6zcmrn5tb5BCpk= ceph-3640fe00-d68c-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-28T22:56:00.476 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmPBU8HH15blr49NXBrFs0hBCGaBdpz5Msl6Nja7XP7J1IaHJTcloCgjxSMWC3Y8TGn6llq7bru5xa4LPusRS6j7lFl6aOtydln49W86jkWnJ8RYQmZ7DNbaLI2j6gpgDOhWHlXZtaFgxKoCXBXmLYxuTd4Xtl2fMQqZbjBHzyeO4UpmEiLaGrpE4LmQarb2nlC0/4V8kp56cGzK6b208Q+dBsEvoABbFj8cqjnGNH0rK52/BAXVzGET+L4SWD6Q7qkevIIqA16OF+f0qcj7hPoNyPO0Up55V7r34G4M9SXZCwwHYY8p7eFA4evinVxmkduBzgiZnBdSiiqymbvbmoVBTWgZ+ZNWD+hLyyL3q/RxbJjTOBYkkIvNInTAZeP7WKG/jh8h80z7PUKTlSyC87ubcs1+aFsRJownGucz2S9s5bnoI/4mO9c8rQtejeKEeSDE5cxf7R5/GRpizos3XUlMQhp+a1QzLrrzFxotjWjlRJtDCSg6zcmrn5tb5BCpk= ceph-3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:56:00.503 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:00 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3649334780' entity='client.admin' 2024-02-28T22:56:00.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmPBU8HH15blr49NXBrFs0hBCGaBdpz5Msl6Nja7XP7J1IaHJTcloCgjxSMWC3Y8TGn6llq7bru5xa4LPusRS6j7lFl6aOtydln49W86jkWnJ8RYQmZ7DNbaLI2j6gpgDOhWHlXZtaFgxKoCXBXmLYxuTd4Xtl2fMQqZbjBHzyeO4UpmEiLaGrpE4LmQarb2nlC0/4V8kp56cGzK6b208Q+dBsEvoABbFj8cqjnGNH0rK52/BAXVzGET+L4SWD6Q7qkevIIqA16OF+f0qcj7hPoNyPO0Up55V7r34G4M9SXZCwwHYY8p7eFA4evinVxmkduBzgiZnBdSiiqymbvbmoVBTWgZ+ZNWD+hLyyL3q/RxbJjTOBYkkIvNInTAZeP7WKG/jh8h80z7PUKTlSyC87ubcs1+aFsRJownGucz2S9s5bnoI/4mO9c8rQtejeKEeSDE5cxf7R5/GRpizos3XUlMQhp+a1QzLrrzFxotjWjlRJtDCSg6zcmrn5tb5BCpk= ceph-3640fe00-d68c-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-28T22:56:00.672 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmPBU8HH15blr49NXBrFs0hBCGaBdpz5Msl6Nja7XP7J1IaHJTcloCgjxSMWC3Y8TGn6llq7bru5xa4LPusRS6j7lFl6aOtydln49W86jkWnJ8RYQmZ7DNbaLI2j6gpgDOhWHlXZtaFgxKoCXBXmLYxuTd4Xtl2fMQqZbjBHzyeO4UpmEiLaGrpE4LmQarb2nlC0/4V8kp56cGzK6b208Q+dBsEvoABbFj8cqjnGNH0rK52/BAXVzGET+L4SWD6Q7qkevIIqA16OF+f0qcj7hPoNyPO0Up55V7r34G4M9SXZCwwHYY8p7eFA4evinVxmkduBzgiZnBdSiiqymbvbmoVBTWgZ+ZNWD+hLyyL3q/RxbJjTOBYkkIvNInTAZeP7WKG/jh8h80z7PUKTlSyC87ubcs1+aFsRJownGucz2S9s5bnoI/4mO9c8rQtejeKEeSDE5cxf7R5/GRpizos3XUlMQhp+a1QzLrrzFxotjWjlRJtDCSg6zcmrn5tb5BCpk= ceph-3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:56:00.736 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-28T22:56:01.120 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:56:03.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 -- 172.21.15.57:0/1639124065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 msgr2=0x7fb2a4075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1639124065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a4075b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb294009910 tx=0x7fb294009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 -- 172.21.15.57:0/1639124065 shutdown_connections 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1639124065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a4075b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 -- 172.21.15.57:0/1639124065 >> 172.21.15.57:0/1639124065 conn(0x7fb2a40fd700 msgr2=0x7fb2a40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 -- 172.21.15.57:0/1639124065 shutdown_connections 2024-02-28T22:56:03.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.982+0000 7fb2aa000700 1 -- 172.21.15.57:0/1639124065 wait complete. 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.984+0000 7fb2aa000700 1 Processor -- start 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.984+0000 7fb2aa000700 1 -- start start 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.984+0000 7fb2aa000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.984+0000 7fb2aa000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419bae0 con 0x7fb2a4075700 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.985+0000 7fb2a3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.985+0000 7fb2a3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43100/0 (socket says 172.21.15.57:43100) 2024-02-28T22:56:03.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.985+0000 7fb2a3400700 1 -- 172.21.15.57:0/1446293905 learned_addr learned my addr 172.21.15.57:0/1446293905 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.985+0000 7fb2a3400700 1 -- 172.21.15.57:0/1446293905 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940095f0 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.985+0000 7fb2a3400700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb2940092e0 tx=0x7fb294009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.986+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb29401d070 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.986+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb29400be10 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.986+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb29400f460 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.986+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a419bce0 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.986+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a419c180 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.987+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a41950f0 con 0x7fb2a4075700 2024-02-28T22:56:03.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.987+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fb294022470 con 0x7fb2a4075700 2024-02-28T22:56:03.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.987+0000 7fb2a0c00700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 0x7fb28803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:03.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.988+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb29404c510 con 0x7fb2a4075700 2024-02-28T22:56:03.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.988+0000 7fb29be00700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 0x7fb28803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:03.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.988+0000 7fb29be00700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 0x7fb28803a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006fd0 tx=0x7fb28c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:03.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:03.993+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb29400b960 con 0x7fb2a4075700 2024-02-28T22:56:04.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.218+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fb2a402cc30 con 0x7fb2a4075700 2024-02-28T22:56:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.229+0000 7fb2a0c00700 1 -- 172.21.15.57:0/1446293905 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb294027070 con 0x7fb2a4075700 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.232+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 msgr2=0x7fb28803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.232+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 0x7fb28803a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006fd0 tx=0x7fb28c006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.232+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 msgr2=0x7fb2a419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.232+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb2940092e0 tx=0x7fb294009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.232+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 shutdown_connections 2024-02-28T22:56:04.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.233+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fb2880384e0 0x7fb28803a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:04.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.233+0000 7fb2aa000700 1 --2- 172.21.15.57:0/1446293905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a4075700 0x7fb2a419b5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:04.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.233+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 >> 172.21.15.57:0/1446293905 conn(0x7fb2a40fd700 msgr2=0x7fb2a40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:04.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.233+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 shutdown_connections 2024-02-28T22:56:04.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:04.233+0000 7fb2aa000700 1 -- 172.21.15.57:0/1446293905 wait complete. 2024-02-28T22:56:05.199 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-28T22:56:05.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1446293905' entity='client.admin' 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:56:05.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T22:56:05.535 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:56:05.535 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:05 smithi057 ceph-mon[29096]: Deploying daemon ceph-exporter.smithi057 on smithi057 2024-02-28T22:56:06.785 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 -- 172.21.15.57:0/3288332573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 msgr2=0x7fa9bc102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/3288332573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc102cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009910 tx=0x7fa9b0009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 -- 172.21.15.57:0/3288332573 shutdown_connections 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/3288332573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc102cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 -- 172.21.15.57:0/3288332573 >> 172.21.15.57:0/3288332573 conn(0x7fa9bc0fa2b0 msgr2=0x7fa9bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 -- 172.21.15.57:0/3288332573 shutdown_connections 2024-02-28T22:56:09.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.308+0000 7fa9c3c00700 1 -- 172.21.15.57:0/3288332573 wait complete. 2024-02-28T22:56:09.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.309+0000 7fa9c3c00700 1 Processor -- start 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.309+0000 7fa9c3c00700 1 -- start start 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.310+0000 7fa9c3c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.310+0000 7fa9c3c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1976c0 con 0x7fa9bc100910 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.310+0000 7fa9c1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.310+0000 7fa9c1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36916/0 (socket says 172.21.15.57:36916) 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.310+0000 7fa9c1200700 1 -- 172.21.15.57:0/4283003833 learned_addr learned my addr 172.21.15.57:0/4283003833 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:56:09.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9c1200700 1 -- 172.21.15.57:0/4283003833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00095f0 con 0x7fa9bc100910 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9c1200700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00092e0 tx=0x7fa9b0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b001d070 con 0x7fa9bc100910 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc1978c0 con 0x7fa9bc100910 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9b000be10 con 0x7fa9bc100910 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.311+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b000f460 con 0x7fa9bc100910 2024-02-28T22:56:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.312+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc197d60 con 0x7fa9bc100910 2024-02-28T22:56:09.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.313+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc190f00 con 0x7fa9bc100910 2024-02-28T22:56:09.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.313+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fa9b000f6d0 con 0x7fa9bc100910 2024-02-28T22:56:09.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.313+0000 7fa9b9600700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 0x7fa9a403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:09.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.313+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9b004c780 con 0x7fa9bc100910 2024-02-28T22:56:09.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.314+0000 7fa9bbe00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 0x7fa9a403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:09.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.314+0000 7fa9bbe00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 0x7fa9a403a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:09.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.319+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa9b0005c00 con 0x7fa9bc100910 2024-02-28T22:56:09.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.547+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9bc061220 con 0x7fa9a40384e0 2024-02-28T22:56:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.555+0000 7fa9b9600700 1 -- 172.21.15.57:0/4283003833 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa9bc061220 con 0x7fa9a40384e0 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.558+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 msgr2=0x7fa9a403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.558+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 0x7fa9a403a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.558+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 msgr2=0x7fa9bc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.558+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00092e0 tx=0x7fa9b0009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 shutdown_connections 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa9a40384e0 0x7fa9a403a990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 --2- 172.21.15.57:0/4283003833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9bc100910 0x7fa9bc197180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 >> 172.21.15.57:0/4283003833 conn(0x7fa9bc0fa2b0 msgr2=0x7fa9bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 shutdown_connections 2024-02-28T22:56:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:09.559+0000 7fa9c3c00700 1 -- 172.21.15.57:0/4283003833 wait complete. 2024-02-28T22:56:10.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:10.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:10.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:10.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:10.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:10.224 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:56:10.224 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-28T22:56:10.224 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:56:10.224 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:10 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:10.265 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi084 2024-02-28T22:56:10.265 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:56:10.265 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.conf 2024-02-28T22:56:10.323 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:56:10.323 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:56:10.418 INFO:tasks.cephadm:Adding host smithi084 to orchestrator... 2024-02-28T22:56:10.418 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch host add smithi084 2024-02-28T22:56:11.552 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:11 smithi057 ceph-mon[29096]: Deploying daemon crash.smithi057 on smithi057 2024-02-28T22:56:11.552 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:11 smithi057 ceph-mon[29096]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:56:11.708 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:56:12.398 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:12 smithi057 ceph-mon[29096]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 -- 172.21.15.57:0/291259607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 msgr2=0x7f9080102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 --2- 172.21.15.57:0/291259607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080102c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9074009910 tx=0x7f9074009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 -- 172.21.15.57:0/291259607 shutdown_connections 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 --2- 172.21.15.57:0/291259607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080102c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 -- 172.21.15.57:0/291259607 >> 172.21.15.57:0/291259607 conn(0x7f90800fa2b0 msgr2=0x7f90800fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 -- 172.21.15.57:0/291259607 shutdown_connections 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.075+0000 7f9087c00700 1 -- 172.21.15.57:0/291259607 wait complete. 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.076+0000 7f9087c00700 1 Processor -- start 2024-02-28T22:56:14.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9087c00700 1 -- start start 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9087c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9087c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9080100750 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9085200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9085200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36928/0 (socket says 172.21.15.57:36928) 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.077+0000 7f9085200700 1 -- 172.21.15.57:0/1413853570 learned_addr learned my addr 172.21.15.57:0/1413853570 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f9085200700 1 -- 172.21.15.57:0/1413853570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740095f0 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f9085200700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9074005b40 tx=0x7f9074004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f907401d070 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90800fe860 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f907400be10 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f907400f460 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.078+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90800fed00 con 0x7f90801008b0 2024-02-28T22:56:14.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.079+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f90740227e0 con 0x7f90801008b0 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.079+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9080062380 con 0x7f90801008b0 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.080+0000 7f907d600700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 0x7f906803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.080+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f907404c450 con 0x7f90801008b0 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.080+0000 7f907fe00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 0x7f906803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.081+0000 7f907fe00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 0x7f906803a9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f906c006fd0 tx=0x7f906c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:14.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.085+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f907402ba00 con 0x7f90801008b0 2024-02-28T22:56:14.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:14.320+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}) v1 -- 0x7f90800fef40 con 0x7f9068038530 2024-02-28T22:56:14.466 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:14 smithi057 ceph-mon[29096]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:14.466 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:14 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:15.815 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: Deploying daemon node-exporter.smithi057 on smithi057 2024-02-28T22:56:15.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:15 smithi057 ceph-mon[29096]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:17.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:16 smithi057 ceph-mon[29096]: Deploying cephadm binary to smithi084 2024-02-28T22:56:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:17 smithi057 ceph-mon[29096]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:19.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.214+0000 7f907d600700 1 -- 172.21.15.57:0/1413853570 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f90800fef40 con 0x7f9068038530 2024-02-28T22:56:19.215 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi084' with addr '172.21.15.84' 2024-02-28T22:56:19.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 msgr2=0x7f906803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 0x7f906803a9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f906c006fd0 tx=0x7f906c006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 msgr2=0x7f9080100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9074005b40 tx=0x7f9074004740 comp rx=0 tx=0).stop 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 shutdown_connections 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9068038530 0x7f906803a9e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 --2- 172.21.15.57:0/1413853570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90801008b0 0x7f9080100210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 >> 172.21.15.57:0/1413853570 conn(0x7f90800fa2b0 msgr2=0x7f90800faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 shutdown_connections 2024-02-28T22:56:19.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:19.218+0000 7f9087c00700 1 -- 172.21.15.57:0/1413853570 wait complete. 2024-02-28T22:56:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:20 smithi057 ceph-mon[29096]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:20 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:20 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:20.402 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-28T22:56:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:56:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:21 smithi057 ceph-mon[29096]: Added host smithi084 2024-02-28T22:56:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:21 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:21 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:21 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:21 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:22.531 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:22 smithi057 ceph-mon[29096]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:22.531 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:22 smithi057 ceph-mon[29096]: Deploying daemon alertmanager.smithi057 on smithi057 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 -- 172.21.15.57:0/1175102782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 msgr2=0x7fa2ec0feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/1175102782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec0feb40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 -- 172.21.15.57:0/1175102782 shutdown_connections 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/1175102782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec0feb40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 -- 172.21.15.57:0/1175102782 >> 172.21.15.57:0/1175102782 conn(0x7fa2ec0fa2c0 msgr2=0x7fa2ec0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 -- 172.21.15.57:0/1175102782 shutdown_connections 2024-02-28T22:56:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.315+0000 7fa2f2c00700 1 -- 172.21.15.57:0/1175102782 wait complete. 2024-02-28T22:56:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.317+0000 7fa2f2c00700 1 Processor -- start 2024-02-28T22:56:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.317+0000 7fa2f2c00700 1 -- start start 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.317+0000 7fa2f2c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.317+0000 7fa2f2c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec1976d0 con 0x7fa2ec0fe730 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.318+0000 7fa2ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.318+0000 7fa2ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38934/0 (socket says 172.21.15.57:38934) 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.318+0000 7fa2ebe00700 1 -- 172.21.15.57:0/2382627221 learned_addr learned my addr 172.21.15.57:0/2382627221 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:56:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.318+0000 7fa2ebe00700 1 -- 172.21.15.57:0/2382627221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0095f0 con 0x7fa2ec0fe730 2024-02-28T22:56:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.318+0000 7fa2ebe00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0092e0 tx=0x7fa2dc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.319+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2dc01d070 con 0x7fa2ec0fe730 2024-02-28T22:56:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.319+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc00be10 con 0x7fa2ec0fe730 2024-02-28T22:56:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.319+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec1978d0 con 0x7fa2ec0fe730 2024-02-28T22:56:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.319+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2dc00f460 con 0x7fa2ec0fe730 2024-02-28T22:56:23.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.319+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec197d70 con 0x7fa2ec0fe730 2024-02-28T22:56:23.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.320+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fa2dc022470 con 0x7fa2ec0fe730 2024-02-28T22:56:23.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.320+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec190d00 con 0x7fa2ec0fe730 2024-02-28T22:56:23.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.321+0000 7fa2e8c00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 0x7fa2d003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:23.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.321+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2dc04c520 con 0x7fa2ec0fe730 2024-02-28T22:56:23.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.321+0000 7fa2eb400700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 0x7fa2d003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:23.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.322+0000 7fa2eb400700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 0x7fa2d003a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.326+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa2dc00b960 con 0x7fa2ec0fe730 2024-02-28T22:56:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.558+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2ec061190 con 0x7fa2d0038490 2024-02-28T22:56:23.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.560+0000 7fa2e8c00700 1 -- 172.21.15.57:0/2382627221 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fa2ec061190 con 0x7fa2d0038490 2024-02-28T22:56:23.559 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T22:56:23.559 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}] 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.563+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 msgr2=0x7fa2d003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.563+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 0x7fa2d003a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.563+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 msgr2=0x7fa2ec197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.563+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0092e0 tx=0x7fa2dc009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 shutdown_connections 2024-02-28T22:56:23.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa2d0038490 0x7fa2d003a940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 --2- 172.21.15.57:0/2382627221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2ec0fe730 0x7fa2ec197190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 >> 172.21.15.57:0/2382627221 conn(0x7fa2ec0fa2c0 msgr2=0x7fa2ec0faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 shutdown_connections 2024-02-28T22:56:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:23.564+0000 7fa2f2c00700 1 -- 172.21.15.57:0/2382627221 wait complete. 2024-02-28T22:56:24.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:24 smithi057 ceph-mon[29096]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:24.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:24 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:24.646 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-28T22:56:24.646 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd crush tunables default 2024-02-28T22:56:25.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:25 smithi057 ceph-mon[29096]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-28T22:56:25.610 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:56:26.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:26 smithi057 ceph-mon[29096]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:28.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:28 smithi057 ceph-mon[29096]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 -- 172.21.15.57:0/3261235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 msgr2=0x7ff1641065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 --2- 172.21.15.57:0/3261235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff1641065f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff158009910 tx=0x7ff158009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 -- 172.21.15.57:0/3261235348 shutdown_connections 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 --2- 172.21.15.57:0/3261235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff1641065f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 -- 172.21.15.57:0/3261235348 >> 172.21.15.57:0/3261235348 conn(0x7ff1640751e0 msgr2=0x7ff1640775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 -- 172.21.15.57:0/3261235348 shutdown_connections 2024-02-28T22:56:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.026+0000 7ff16a600700 1 -- 172.21.15.57:0/3261235348 wait complete. 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff16a600700 1 Processor -- start 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff16a600700 1 -- start start 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff16a600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff16a600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419b8b0 con 0x7ff1641061e0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff169200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff169200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43442/0 (socket says 172.21.15.57:43442) 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.028+0000 7ff169200700 1 -- 172.21.15.57:0/3366637541 learned_addr learned my addr 172.21.15.57:0/3366637541 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff169200700 1 -- 172.21.15.57:0/3366637541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580095f0 con 0x7ff1641061e0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff169200700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff158005b40 tx=0x7ff158004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff15801d070 con 0x7ff1641061e0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff15800be10 con 0x7ff1641061e0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff15800f460 con 0x7ff1641061e0 2024-02-28T22:56:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.029+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16419bab0 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.030+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16419bf50 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.031+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164194ec0 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.031+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ff1580227e0 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.032+0000 7ff161600700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 0x7ff14c03a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.032+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff15804c3e0 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.032+0000 7ff163e00700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 0x7ff14c03a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.033+0000 7ff163e00700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 0x7ff14c03a9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff150006fd0 tx=0x7ff150006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.037+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff15802ba00 con 0x7ff1641061e0 2024-02-28T22:56:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:29.267+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff16402cc30 con 0x7ff1641061e0 2024-02-28T22:56:30.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.106+0000 7ff161600700 1 -- 172.21.15.57:0/3366637541 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff158027070 con 0x7ff1641061e0 2024-02-28T22:56:30.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.110+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 msgr2=0x7ff14c03a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:30.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.110+0000 7ff16a600700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 0x7ff14c03a9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff150006fd0 tx=0x7ff150006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:30.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.110+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 msgr2=0x7ff16419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:30.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.110+0000 7ff16a600700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff158005b40 tx=0x7ff158004740 comp rx=0 tx=0).stop 2024-02-28T22:56:30.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 shutdown_connections 2024-02-28T22:56:30.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff14c0384f0 0x7ff14c03a9a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:30.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 --2- 172.21.15.57:0/3366637541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641061e0 0x7ff16419b370 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:30.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 >> 172.21.15.57:0/3366637541 conn(0x7ff1640751e0 msgr2=0x7ff164075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:30.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 shutdown_connections 2024-02-28T22:56:30.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:56:30.111+0000 7ff16a600700 1 -- 172.21.15.57:0/3366637541 wait complete. 2024-02-28T22:56:30.111 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2024-02-28T22:56:30.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:30 smithi057 ceph-mon[29096]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:30.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:30 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3366637541' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-28T22:56:30.988 INFO:tasks.cephadm:Adding mon.smithi057 on smithi057 2024-02-28T22:56:30.989 INFO:tasks.cephadm:Adding mon.smithi084 on smithi084 2024-02-28T22:56:30.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch apply mon '2;smithi057:172.21.15.57=smithi057;smithi084:172.21.15.84=smithi084' 2024-02-28T22:56:31.304 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:31 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3366637541' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-28T22:56:31.304 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:31 smithi057 ceph-mon[29096]: osdmap e4: 0 total, 0 up, 0 in 2024-02-28T22:56:31.358 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:31.457 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:32.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:32 smithi057 ceph-mon[29096]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:33.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 -- 172.21.15.84:0/2904397154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 msgr2=0x7fa928102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 --2- 172.21.15.84:0/2904397154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa928102cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009910 tx=0x7fa91c009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 -- 172.21.15.84:0/2904397154 shutdown_connections 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 --2- 172.21.15.84:0/2904397154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa928102cf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 -- 172.21.15.84:0/2904397154 >> 172.21.15.84:0/2904397154 conn(0x7fa9280fa2b0 msgr2=0x7fa9280fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 -- 172.21.15.84:0/2904397154 shutdown_connections 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.378+0000 7fa92f800700 1 -- 172.21.15.84:0/2904397154 wait complete. 2024-02-28T22:56:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.379+0000 7fa92f800700 1 Processor -- start 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.380+0000 7fa92f800700 1 -- start start 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.380+0000 7fa92f800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.380+0000 7fa92f800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281976f0 con 0x7fa928100910 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.380+0000 7fa92ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.381+0000 7fa92ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:54526/0 (socket says 172.21.15.84:54526) 2024-02-28T22:56:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.381+0000 7fa92ce00700 1 -- 172.21.15.84:0/88627267 learned_addr learned my addr 172.21.15.84:0/88627267 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.381+0000 7fa92ce00700 1 -- 172.21.15.84:0/88627267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0095f0 con 0x7fa928100910 2024-02-28T22:56:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa92ce00700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005b40 tx=0x7fa91c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa91c01d070 con 0x7fa928100910 2024-02-28T22:56:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9281978f0 con 0x7fa928100910 2024-02-28T22:56:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928197d90 con 0x7fa928100910 2024-02-28T22:56:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa91c005c00 con 0x7fa928100910 2024-02-28T22:56:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.382+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa91c00fc30 con 0x7fa928100910 2024-02-28T22:56:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.384+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908005320 con 0x7fa928100910 2024-02-28T22:56:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.384+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fa91c00f5f0 con 0x7fa928100910 2024-02-28T22:56:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.384+0000 7fa925600700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 0x7fa91003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.384+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa91c04c160 con 0x7fa928100910 2024-02-28T22:56:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.385+0000 7fa927e00700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 0x7fa91003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.386+0000 7fa927e00700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 0x7fa91003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.400+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa91c00b960 con 0x7fa928100910 2024-02-28T22:56:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.625+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 --> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi057:172.21.15.57=smithi057;smithi084:172.21.15.84=smithi084", "target": ["mon-mgr", ""]}) v1 -- 0x7fa908000ca0 con 0x7fa910038490 2024-02-28T22:56:33.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.636+0000 7fa925600700 1 -- 172.21.15.84:0/88627267 <== mgr.14164 v2:172.21.15.57:6800/2764732365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa908000ca0 con 0x7fa910038490 2024-02-28T22:56:33.635 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mon update... 2024-02-28T22:56:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.638+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 msgr2=0x7fa91003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.638+0000 7fa92f800700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 0x7fa91003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 msgr2=0x7fa9281971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005b40 tx=0x7fa91c004740 comp rx=0 tx=0).stop 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 shutdown_connections 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa910038490 0x7fa91003a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 --2- 172.21.15.84:0/88627267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928100910 0x7fa9281971b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 >> 172.21.15.84:0/88627267 conn(0x7fa9280fa2b0 msgr2=0x7fa9280faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 shutdown_connections 2024-02-28T22:56:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:33.639+0000 7fa92f800700 1 -- 172.21.15.84:0/88627267 wait complete. 2024-02-28T22:56:34.270 DEBUG:teuthology.orchestra.run.smithi084:mon.smithi084> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi084.service 2024-02-28T22:56:34.278 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:34.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:34 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:34.768 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi057:172.21.15.57=smithi057;smithi084:172.21.15.84=smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: Saving service mon spec with placement smithi057:172.21.15.57=smithi057;smithi084:172.21.15.84=smithi084;count:2 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-28T22:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-28T22:56:35.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:35.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:35 smithi057 ceph-mon[29096]: Deploying daemon grafana.smithi057 on smithi057 2024-02-28T22:56:36.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:36 smithi057 ceph-mon[29096]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 -- 172.21.15.84:0/1102764432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 msgr2=0x7f6bb0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 --2- 172.21.15.84:0/1102764432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb0102b20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009910 tx=0x7f6ba4009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 -- 172.21.15.84:0/1102764432 shutdown_connections 2024-02-28T22:56:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 --2- 172.21.15.84:0/1102764432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb0102b20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 -- 172.21.15.84:0/1102764432 >> 172.21.15.84:0/1102764432 conn(0x7f6bb00fa0e0 msgr2=0x7f6bb00fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 -- 172.21.15.84:0/1102764432 shutdown_connections 2024-02-28T22:56:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.571+0000 7f6bb6400700 1 -- 172.21.15.84:0/1102764432 wait complete. 2024-02-28T22:56:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.573+0000 7f6bb6400700 1 Processor -- start 2024-02-28T22:56:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.573+0000 7f6bb6400700 1 -- start start 2024-02-28T22:56:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.573+0000 7f6bb6400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.573+0000 7f6bb6400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01005e0 con 0x7f6bb0100740 2024-02-28T22:56:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.574+0000 7f6bb5000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.574+0000 7f6bb5000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:54544/0 (socket says 172.21.15.84:54544) 2024-02-28T22:56:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.574+0000 7f6bb5000700 1 -- 172.21.15.84:0/3607271831 learned_addr learned my addr 172.21.15.84:0/3607271831 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:36.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.575+0000 7f6bb5000700 1 -- 172.21.15.84:0/3607271831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40095f0 con 0x7f6bb0100740 2024-02-28T22:56:36.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.575+0000 7f6bb5000700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40092e0 tx=0x7f6ba4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:36.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.575+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ba401d070 con 0x7f6bb0100740 2024-02-28T22:56:36.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.576+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb00fe6f0 con 0x7f6bb0100740 2024-02-28T22:56:36.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.576+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6ba400be10 con 0x7f6bb0100740 2024-02-28T22:56:36.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.576+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb00feb90 con 0x7f6bb0100740 2024-02-28T22:56:36.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.576+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ba400f460 con 0x7f6bb0100740 2024-02-28T22:56:36.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.577+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0062380 con 0x7f6bb0100740 2024-02-28T22:56:36.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.578+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f6ba4022470 con 0x7f6bb0100740 2024-02-28T22:56:36.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.578+0000 7f6bad600700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 0x7f6b9803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:36.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.578+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ba404c630 con 0x7f6bb0100740 2024-02-28T22:56:36.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.579+0000 7f6bafe00700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 0x7f6b9803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:36.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.580+0000 7f6bafe00700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 0x7f6b9803a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006fd0 tx=0x7f6b9c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:36.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.595+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6ba400b960 con 0x7f6bb0100740 2024-02-28T22:56:36.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.889+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6bb018e310 con 0x7f6bb0100740 2024-02-28T22:56:36.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.890+0000 7f6bad600700 1 -- 172.21.15.84:0/3607271831 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6ba4027020 con 0x7f6bb0100740 2024-02-28T22:56:36.890 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:36.890 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:36.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 msgr2=0x7f6b9803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:36.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 0x7f6b9803a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006fd0 tx=0x7f6b9c006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 msgr2=0x7f6bb01000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40092e0 tx=0x7f6ba4009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 shutdown_connections 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f6b980384e0 0x7f6b9803a990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 --2- 172.21.15.84:0/3607271831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb0100740 0x7f6bb01000a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 >> 172.21.15.84:0/3607271831 conn(0x7f6bb00fa0e0 msgr2=0x7f6bb00fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:36.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.893+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 shutdown_connections 2024-02-28T22:56:36.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:36.894+0000 7f6bb6400700 1 -- 172.21.15.84:0/3607271831 wait complete. 2024-02-28T22:56:36.894 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:56:37.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:37 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3607271831' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:56:38.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:38 smithi057 ceph-mon[29096]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:38.925 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:38.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:39.300 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:39.394 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:40 smithi057 ceph-mon[29096]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 -- 172.21.15.84:0/2714909318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 msgr2=0x7f3410106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 --2- 172.21.15.84:0/2714909318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f3410106850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3400009910 tx=0x7f3400009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 -- 172.21.15.84:0/2714909318 shutdown_connections 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 --2- 172.21.15.84:0/2714909318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f3410106850 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 -- 172.21.15.84:0/2714909318 >> 172.21.15.84:0/2714909318 conn(0x7f34100756e0 msgr2=0x7f3410077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 -- 172.21.15.84:0/2714909318 shutdown_connections 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.405+0000 7f3417c00700 1 -- 172.21.15.84:0/2714909318 wait complete. 2024-02-28T22:56:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.406+0000 7f3417c00700 1 Processor -- start 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3417c00700 1 -- start start 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3417c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3417c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341019bae0 con 0x7f3410106440 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3415200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3415200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:54558/0 (socket says 172.21.15.84:54558) 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.407+0000 7f3415200700 1 -- 172.21.15.84:0/3582970533 learned_addr learned my addr 172.21.15.84:0/3582970533 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.408+0000 7f3415200700 1 -- 172.21.15.84:0/3582970533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000095f0 con 0x7f3410106440 2024-02-28T22:56:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.408+0000 7f3415200700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3400005950 tx=0x7f3400009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.409+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f340001d070 con 0x7f3410106440 2024-02-28T22:56:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.409+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341019bce0 con 0x7f3410106440 2024-02-28T22:56:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.409+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f340000be10 con 0x7f3410106440 2024-02-28T22:56:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.409+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f340000f460 con 0x7f3410106440 2024-02-28T22:56:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.409+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341019c180 con 0x7f3410106440 2024-02-28T22:56:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.410+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410195110 con 0x7f3410106440 2024-02-28T22:56:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.411+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f34000227c0 con 0x7f3410106440 2024-02-28T22:56:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.411+0000 7f340e000700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 0x7f33f003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.411+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f340004c450 con 0x7f3410106440 2024-02-28T22:56:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.412+0000 7f3407e00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 0x7f33f003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.413+0000 7f3407e00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 0x7f33f003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006fd0 tx=0x7f33f8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:41.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.434+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f340002bba0 con 0x7f3410106440 2024-02-28T22:56:41.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.733+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f341002cc30 con 0x7f3410106440 2024-02-28T22:56:41.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.734+0000 7f340e000700 1 -- 172.21.15.84:0/3582970533 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3400027030 con 0x7f3410106440 2024-02-28T22:56:41.734 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:41.734 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:41.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 msgr2=0x7f33f003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:41.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 0x7f33f003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006fd0 tx=0x7f33f8006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 msgr2=0x7f341019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3400005950 tx=0x7f3400009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 shutdown_connections 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f33f0038490 0x7f33f003a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 --2- 172.21.15.84:0/3582970533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3410106440 0x7f341019b5a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 >> 172.21.15.84:0/3582970533 conn(0x7f34100756e0 msgr2=0x7f34100773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 shutdown_connections 2024-02-28T22:56:41.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:41.737+0000 7f3417c00700 1 -- 172.21.15.84:0/3582970533 wait complete. 2024-02-28T22:56:41.738 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:56:42.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:42 smithi057 ceph-mon[29096]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:42.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:42 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3582970533' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:56:43.742 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:43.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:44.119 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:44.212 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:44.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:44 smithi057 ceph-mon[29096]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:46.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.041+0000 7ffb10c00700 1 -- 172.21.15.84:0/322227926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 msgr2=0x7ffb0c1066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:46.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.041+0000 7ffb10c00700 1 --2- 172.21.15.84:0/322227926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c1066c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009910 tx=0x7ffafc009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:46.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.042+0000 7ffb10c00700 1 -- 172.21.15.84:0/322227926 shutdown_connections 2024-02-28T22:56:46.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.042+0000 7ffb10c00700 1 --2- 172.21.15.84:0/322227926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c1066c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:46.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.042+0000 7ffb10c00700 1 -- 172.21.15.84:0/322227926 >> 172.21.15.84:0/322227926 conn(0x7ffb0c075410 msgr2=0x7ffb0c077840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:46.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.042+0000 7ffb10c00700 1 -- 172.21.15.84:0/322227926 shutdown_connections 2024-02-28T22:56:46.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.042+0000 7ffb10c00700 1 -- 172.21.15.84:0/322227926 wait complete. 2024-02-28T22:56:46.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.043+0000 7ffb10c00700 1 Processor -- start 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.043+0000 7ffb10c00700 1 -- start start 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.043+0000 7ffb10c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.043+0000 7ffb10c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c19b920 con 0x7ffb0c1062b0 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.044+0000 7ffb0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.044+0000 7ffb0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:48948/0 (socket says 172.21.15.84:48948) 2024-02-28T22:56:46.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.044+0000 7ffb0b400700 1 -- 172.21.15.84:0/669661409 learned_addr learned my addr 172.21.15.84:0/669661409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:46.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.044+0000 7ffb0b400700 1 -- 172.21.15.84:0/669661409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0095f0 con 0x7ffb0c1062b0 2024-02-28T22:56:46.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.045+0000 7ffb0b400700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffafc005950 tx=0x7ffafc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:46.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.045+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffafc01d070 con 0x7ffb0c1062b0 2024-02-28T22:56:46.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.045+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c19bb20 con 0x7ffb0c1062b0 2024-02-28T22:56:46.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.045+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffafc00be10 con 0x7ffb0c1062b0 2024-02-28T22:56:46.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.045+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffafc00f460 con 0x7ffb0c1062b0 2024-02-28T22:56:46.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.046+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c19bfc0 con 0x7ffb0c1062b0 2024-02-28T22:56:46.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.047+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c194f40 con 0x7ffb0c1062b0 2024-02-28T22:56:46.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.047+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ffafc0227c0 con 0x7ffb0c1062b0 2024-02-28T22:56:46.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.048+0000 7ffb03e00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 0x7ffaf003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:46.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.048+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffafc04cc90 con 0x7ffb0c1062b0 2024-02-28T22:56:46.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.048+0000 7ffb0aa00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 0x7ffaf003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.050+0000 7ffb0aa00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 0x7ffaf003a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8006fd0 tx=0x7ffaf8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:46.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.057+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffafc02ba00 con 0x7ffb0c1062b0 2024-02-28T22:56:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:45 smithi057 ceph-mon[29096]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:46.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.355+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffb0c02cc30 con 0x7ffb0c1062b0 2024-02-28T22:56:46.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.356+0000 7ffb03e00700 1 -- 172.21.15.84:0/669661409 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ffafc027030 con 0x7ffb0c1062b0 2024-02-28T22:56:46.356 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:46.356 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:46.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 msgr2=0x7ffaf003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:46.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 0x7ffaf003a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8006fd0 tx=0x7ffaf8006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 msgr2=0x7ffb0c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffafc005950 tx=0x7ffafc009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 shutdown_connections 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ffaf00384e0 0x7ffaf003a990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 --2- 172.21.15.84:0/669661409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb0c1062b0 0x7ffb0c19b3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 >> 172.21.15.84:0/669661409 conn(0x7ffb0c075410 msgr2=0x7ffb0c077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 shutdown_connections 2024-02-28T22:56:46.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:46.359+0000 7ffb10c00700 1 -- 172.21.15.84:0/669661409 wait complete. 2024-02-28T22:56:46.360 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:56:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:46 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/669661409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:56:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:47 smithi057 ceph-mon[29096]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:48.118 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:48.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:48.593 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:50.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:50 smithi057 ceph-mon[29096]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 -- 172.21.15.84:0/2718718294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 msgr2=0x7fc62c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 --2- 172.21.15.84:0/2718718294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c102280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009910 tx=0x7fc61c009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 -- 172.21.15.84:0/2718718294 shutdown_connections 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 --2- 172.21.15.84:0/2718718294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c102280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 -- 172.21.15.84:0/2718718294 >> 172.21.15.84:0/2718718294 conn(0x7fc62c0fd530 msgr2=0x7fc62c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 -- 172.21.15.84:0/2718718294 shutdown_connections 2024-02-28T22:56:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.410+0000 7fc62be00700 1 -- 172.21.15.84:0/2718718294 wait complete. 2024-02-28T22:56:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.412+0000 7fc62be00700 1 Processor -- start 2024-02-28T22:56:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.412+0000 7fc62be00700 1 -- start start 2024-02-28T22:56:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.412+0000 7fc62be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.412+0000 7fc62be00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c197540 con 0x7fc62c101e70 2024-02-28T22:56:50.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.413+0000 7fc62aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:50.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.413+0000 7fc62aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:48978/0 (socket says 172.21.15.84:48978) 2024-02-28T22:56:50.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.413+0000 7fc62aa00700 1 -- 172.21.15.84:0/470388440 learned_addr learned my addr 172.21.15.84:0/470388440 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:50.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.414+0000 7fc62aa00700 1 -- 172.21.15.84:0/470388440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c0095f0 con 0x7fc62c101e70 2024-02-28T22:56:50.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.414+0000 7fc62aa00700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0092e0 tx=0x7fc61c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:50.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.414+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc61c01d070 con 0x7fc62c101e70 2024-02-28T22:56:50.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.415+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc61c00be10 con 0x7fc62c101e70 2024-02-28T22:56:50.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.415+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c197740 con 0x7fc62c101e70 2024-02-28T22:56:50.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.415+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc61c00f460 con 0x7fc62c101e70 2024-02-28T22:56:50.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.415+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c197be0 con 0x7fc62c101e70 2024-02-28T22:56:50.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.416+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c190ad0 con 0x7fc62c101e70 2024-02-28T22:56:50.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.417+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc61c022470 con 0x7fc62c101e70 2024-02-28T22:56:50.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.417+0000 7fc623400700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 0x7fc61003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:50.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.417+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc61c04c360 con 0x7fc62c101e70 2024-02-28T22:56:50.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.418+0000 7fc62a000700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 0x7fc61003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:50.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.419+0000 7fc62a000700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 0x7fc61003a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc618006fd0 tx=0x7fc618006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:50.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.434+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc61c00b960 con 0x7fc62c101e70 2024-02-28T22:56:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.732+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc62c02cc30 con 0x7fc62c101e70 2024-02-28T22:56:50.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.733+0000 7fc623400700 1 -- 172.21.15.84:0/470388440 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc61c027030 con 0x7fc62c101e70 2024-02-28T22:56:50.732 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:50.733 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:50.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 msgr2=0x7fc61003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:50.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 0x7fc61003a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc618006fd0 tx=0x7fc618006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 msgr2=0x7fc62c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0092e0 tx=0x7fc61c009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 shutdown_connections 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fc6100384e0 0x7fc61003a990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 --2- 172.21.15.84:0/470388440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc62c101e70 0x7fc62c197000 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 >> 172.21.15.84:0/470388440 conn(0x7fc62c0fd530 msgr2=0x7fc62c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 shutdown_connections 2024-02-28T22:56:50.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:50.736+0000 7fc62be00700 1 -- 172.21.15.84:0/470388440 wait complete. 2024-02-28T22:56:50.737 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:56:51.432 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:51 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/470388440' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:56:52.364 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:52 smithi057 ceph-mon[29096]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:52.387 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:52.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:52.867 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:54.314 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:54 smithi057 ceph-mon[29096]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 -- 172.21.15.84:0/1022936052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 msgr2=0x7f8dc4106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 --2- 172.21.15.84:0/1022936052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc4106850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009910 tx=0x7f8db4009c20 comp rx=0 tx=0).stop 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 -- 172.21.15.84:0/1022936052 shutdown_connections 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 --2- 172.21.15.84:0/1022936052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc4106850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 -- 172.21.15.84:0/1022936052 >> 172.21.15.84:0/1022936052 conn(0x7f8dc40756e0 msgr2=0x7f8dc4077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 -- 172.21.15.84:0/1022936052 shutdown_connections 2024-02-28T22:56:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.726+0000 7f8dcac00700 1 -- 172.21.15.84:0/1022936052 wait complete. 2024-02-28T22:56:54.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.727+0000 7f8dcac00700 1 Processor -- start 2024-02-28T22:56:54.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dcac00700 1 -- start start 2024-02-28T22:56:54.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dcac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:54.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dcac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc419bae0 con 0x7f8dc4106440 2024-02-28T22:56:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dc3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dc3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:35036/0 (socket says 172.21.15.84:35036) 2024-02-28T22:56:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.728+0000 7f8dc3e00700 1 -- 172.21.15.84:0/2911831856 learned_addr learned my addr 172.21.15.84:0/2911831856 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.729+0000 7f8dc3e00700 1 -- 172.21.15.84:0/2911831856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40095f0 con 0x7f8dc4106440 2024-02-28T22:56:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.729+0000 7f8dc3e00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8db4005950 tx=0x7f8db4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:54.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.731+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8db401d070 con 0x7f8dc4106440 2024-02-28T22:56:54.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.731+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8db400be10 con 0x7f8dc4106440 2024-02-28T22:56:54.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.731+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8db400f460 con 0x7f8dc4106440 2024-02-28T22:56:54.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.731+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc419bce0 con 0x7f8dc4106440 2024-02-28T22:56:54.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.732+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc419c180 con 0x7f8dc4106440 2024-02-28T22:56:54.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.733+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8db40227c0 con 0x7f8dc4106440 2024-02-28T22:56:54.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.733+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4195110 con 0x7f8dc4106440 2024-02-28T22:56:54.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.733+0000 7f8dc0c00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 0x7f8da803a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:54.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.733+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8db404c420 con 0x7f8dc4106440 2024-02-28T22:56:54.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.733+0000 7f8dc3400700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 0x7f8da803a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:54.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.734+0000 7f8dc3400700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 0x7f8da803a9d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006fd0 tx=0x7f8db0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:54.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:54.743+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8db4005c00 con 0x7f8dc4106440 2024-02-28T22:56:55.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.040+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8dc402cc30 con 0x7f8dc4106440 2024-02-28T22:56:55.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.041+0000 7f8dc0c00700 1 -- 172.21.15.84:0/2911831856 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8db4027020 con 0x7f8dc4106440 2024-02-28T22:56:55.041 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:55.041 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 msgr2=0x7f8da803a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 0x7f8da803a9d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006fd0 tx=0x7f8db0006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 msgr2=0x7f8dc419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8db4005950 tx=0x7f8db4009e60 comp rx=0 tx=0).stop 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 shutdown_connections 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8da8038520 0x7f8da803a9d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 --2- 172.21.15.84:0/2911831856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dc4106440 0x7f8dc419b5a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 >> 172.21.15.84:0/2911831856 conn(0x7f8dc40756e0 msgr2=0x7f8dc40773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 shutdown_connections 2024-02-28T22:56:55.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:55.044+0000 7f8dcac00700 1 -- 172.21.15.84:0/2911831856 wait complete. 2024-02-28T22:56:55.045 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:56:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/2911831856' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:56:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:55 smithi057 ceph-mon[29096]: Deploying daemon prometheus.smithi057 on smithi057 2024-02-28T22:56:56.747 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:56:56.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:56:57.109 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:57.202 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:56:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:56:58 smithi057 ceph-mon[29096]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:56:59.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 -- 172.21.15.84:0/1982764017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 msgr2=0x7feb1c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 --2- 172.21.15.84:0/1982764017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c100230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb1000b0a0 tx=0x7feb1000b3b0 comp rx=0 tx=0).stop 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 -- 172.21.15.84:0/1982764017 shutdown_connections 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 --2- 172.21.15.84:0/1982764017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c100230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 -- 172.21.15.84:0/1982764017 >> 172.21.15.84:0/1982764017 conn(0x7feb1c0fb3d0 msgr2=0x7feb1c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 -- 172.21.15.84:0/1982764017 shutdown_connections 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.036+0000 7feb23c00700 1 -- 172.21.15.84:0/1982764017 wait complete. 2024-02-28T22:56:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.037+0000 7feb23c00700 1 Processor -- start 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.037+0000 7feb23c00700 1 -- start start 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.038+0000 7feb23c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.038+0000 7feb23c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c1954c0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.038+0000 7feb21200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.038+0000 7feb21200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:35042/0 (socket says 172.21.15.84:35042) 2024-02-28T22:56:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.038+0000 7feb21200700 1 -- 172.21.15.84:0/3463388806 learned_addr learned my addr 172.21.15.84:0/3463388806 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:56:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.039+0000 7feb21200700 1 -- 172.21.15.84:0/3463388806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009d00 con 0x7feb1c0ffe20 2024-02-28T22:56:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.039+0000 7feb21200700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb1000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:59.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.040+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb1000e040 con 0x7feb1c0ffe20 2024-02-28T22:56:59.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.040+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb1000bdc0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.040+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb1001da40 con 0x7feb1c0ffe20 2024-02-28T22:56:59.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.040+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1c1956c0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.040+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1c195ae0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.041+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1c18ea80 con 0x7feb1c0ffe20 2024-02-28T22:56:59.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.042+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7feb1001d480 con 0x7feb1c0ffe20 2024-02-28T22:56:59.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.043+0000 7feb19600700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 0x7feb0403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:56:59.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.043+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb1004cdc0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.043+0000 7feb1be00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 0x7feb0403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:56:59.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.045+0000 7feb1be00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 0x7feb0403a9e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feb08006fd0 tx=0x7feb08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:56:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.058+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7feb1001a3b0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.356+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feb1c062380 con 0x7feb1c0ffe20 2024-02-28T22:56:59.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.357+0000 7feb19600700 1 -- 172.21.15.84:0/3463388806 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7feb1001a3b0 con 0x7feb1c0ffe20 2024-02-28T22:56:59.357 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:56:59.357 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:56:59.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 msgr2=0x7feb0403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:59.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 0x7feb0403a9e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feb08006fd0 tx=0x7feb08006e40 comp rx=0 tx=0).stop 2024-02-28T22:56:59.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 msgr2=0x7feb1c194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:56:59.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb1000bb10 comp rx=0 tx=0).stop 2024-02-28T22:56:59.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 shutdown_connections 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7feb04038530 0x7feb0403a9e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 --2- 172.21.15.84:0/3463388806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb1c0ffe20 0x7feb1c194f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 >> 172.21.15.84:0/3463388806 conn(0x7feb1c0fb3d0 msgr2=0x7feb1c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 shutdown_connections 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:56:59.360+0000 7feb23c00700 1 -- 172.21.15.84:0/3463388806 wait complete. 2024-02-28T22:56:59.360 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:00.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:00 smithi057 ceph-mon[29096]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:00.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:00 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:00.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:00 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3463388806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:00.997 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:00.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:01.401 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:01.497 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:02 smithi057 ceph-mon[29096]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:03.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 -- 172.21.15.84:0/2662143069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 msgr2=0x7ff9081066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 --2- 172.21.15.84:0/2662143069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff9081066c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009910 tx=0x7ff8f8009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 -- 172.21.15.84:0/2662143069 shutdown_connections 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 --2- 172.21.15.84:0/2662143069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff9081066c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 -- 172.21.15.84:0/2662143069 >> 172.21.15.84:0/2662143069 conn(0x7ff908075410 msgr2=0x7ff908077840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 -- 172.21.15.84:0/2662143069 shutdown_connections 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.367+0000 7ff90cc00700 1 -- 172.21.15.84:0/2662143069 wait complete. 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.368+0000 7ff90cc00700 1 Processor -- start 2024-02-28T22:57:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.368+0000 7ff90cc00700 1 -- start start 2024-02-28T22:57:03.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.369+0000 7ff90cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:03.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.369+0000 7ff90cc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819b920 con 0x7ff9081062b0 2024-02-28T22:57:03.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.369+0000 7ff907400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:03.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.369+0000 7ff907400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:56390/0 (socket says 172.21.15.84:56390) 2024-02-28T22:57:03.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.369+0000 7ff907400700 1 -- 172.21.15.84:0/4015384107 learned_addr learned my addr 172.21.15.84:0/4015384107 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:03.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.370+0000 7ff907400700 1 -- 172.21.15.84:0/4015384107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80095f0 con 0x7ff9081062b0 2024-02-28T22:57:03.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.370+0000 7ff907400700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005950 tx=0x7ff8f8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:03.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.371+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f801d070 con 0x7ff9081062b0 2024-02-28T22:57:03.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.371+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90819bb20 con 0x7ff9081062b0 2024-02-28T22:57:03.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.371+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90819bfc0 con 0x7ff9081062b0 2024-02-28T22:57:03.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.371+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff8f800be10 con 0x7ff9081062b0 2024-02-28T22:57:03.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.371+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f800f460 con 0x7ff9081062b0 2024-02-28T22:57:03.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.372+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8005320 con 0x7ff9081062b0 2024-02-28T22:57:03.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.372+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ff8f800b960 con 0x7ff9081062b0 2024-02-28T22:57:03.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.373+0000 7ff8ffe00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 0x7ff8ec03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:03.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.373+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff8f804c200 con 0x7ff9081062b0 2024-02-28T22:57:03.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.374+0000 7ff906a00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 0x7ff8ec03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:03.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.375+0000 7ff906a00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 0x7ff8ec03a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4006fd0 tx=0x7ff8f4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:03.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.389+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff8f800f5c0 con 0x7ff9081062b0 2024-02-28T22:57:03.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.683+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff8e8005190 con 0x7ff9081062b0 2024-02-28T22:57:03.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.685+0000 7ff8ffe00700 1 -- 172.21.15.84:0/4015384107 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff8f8027020 con 0x7ff9081062b0 2024-02-28T22:57:03.683 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:03.683 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:03.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 msgr2=0x7ff8ec03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:03.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 0x7ff8ec03a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4006fd0 tx=0x7ff8f4006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:03.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 msgr2=0x7ff90819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:03.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005950 tx=0x7ff8f8009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 shutdown_connections 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7ff8ec0384e0 0x7ff8ec03a990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 --2- 172.21.15.84:0/4015384107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9081062b0 0x7ff90819b3e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 >> 172.21.15.84:0/4015384107 conn(0x7ff908075410 msgr2=0x7ff908077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 shutdown_connections 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:03.688+0000 7ff90cc00700 1 -- 172.21.15.84:0/4015384107 wait complete. 2024-02-28T22:57:03.687 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:04 smithi057 ceph-mon[29096]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:04 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/4015384107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:05.415 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:05.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:05.794 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:05.892 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:05 smithi057 ceph-mon[29096]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.701+0000 7f9df2c00700 1 -- 172.21.15.84:0/3580273123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 msgr2=0x7f9dec102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.701+0000 7f9df2c00700 1 --2- 172.21.15.84:0/3580273123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec102cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009910 tx=0x7f9ddc009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.702+0000 7f9df2c00700 1 -- 172.21.15.84:0/3580273123 shutdown_connections 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.702+0000 7f9df2c00700 1 --2- 172.21.15.84:0/3580273123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec102cf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.702+0000 7f9df2c00700 1 -- 172.21.15.84:0/3580273123 >> 172.21.15.84:0/3580273123 conn(0x7f9dec0fa2b0 msgr2=0x7f9dec0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.702+0000 7f9df2c00700 1 -- 172.21.15.84:0/3580273123 shutdown_connections 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.702+0000 7f9df2c00700 1 -- 172.21.15.84:0/3580273123 wait complete. 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.703+0000 7f9df2c00700 1 Processor -- start 2024-02-28T22:57:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.703+0000 7f9df2c00700 1 -- start start 2024-02-28T22:57:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.704+0000 7f9df2c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.704+0000 7f9df2c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1976c0 con 0x7f9dec100910 2024-02-28T22:57:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.704+0000 7f9debe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.704+0000 7f9debe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:56418/0 (socket says 172.21.15.84:56418) 2024-02-28T22:57:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.704+0000 7f9debe00700 1 -- 172.21.15.84:0/154958807 learned_addr learned my addr 172.21.15.84:0/154958807 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.705+0000 7f9debe00700 1 -- 172.21.15.84:0/154958807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0095f0 con 0x7f9dec100910 2024-02-28T22:57:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9debe00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0092e0 tx=0x7f9ddc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ddc01d070 con 0x7f9dec100910 2024-02-28T22:57:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec1978c0 con 0x7f9dec100910 2024-02-28T22:57:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc00be10 con 0x7f9dec100910 2024-02-28T22:57:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ddc00f460 con 0x7f9dec100910 2024-02-28T22:57:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.706+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec197d60 con 0x7f9dec100910 2024-02-28T22:57:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.708+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec190f00 con 0x7f9dec100910 2024-02-28T22:57:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.708+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f9ddc022470 con 0x7f9dec100910 2024-02-28T22:57:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.709+0000 7f9de9600700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 0x7f9dcc03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.709+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ddc04c560 con 0x7f9dec100910 2024-02-28T22:57:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.709+0000 7f9de3e00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 0x7f9dcc03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.710+0000 7f9de3e00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 0x7f9dcc03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:07.725+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9ddc00b960 con 0x7f9dec100910 2024-02-28T22:57:08.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.027+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9dec02cc30 con 0x7f9dec100910 2024-02-28T22:57:08.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.028+0000 7f9de9600700 1 -- 172.21.15.84:0/154958807 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9ddc027070 con 0x7f9dec100910 2024-02-28T22:57:08.026 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:08.027 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.030+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 msgr2=0x7f9dcc03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 0x7f9dcc03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 msgr2=0x7f9dec197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0092e0 tx=0x7f9ddc009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 shutdown_connections 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f9dcc038490 0x7f9dcc03a940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 --2- 172.21.15.84:0/154958807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dec100910 0x7f9dec197180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:08.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 >> 172.21.15.84:0/154958807 conn(0x7f9dec0fa2b0 msgr2=0x7f9dec0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:08.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 shutdown_connections 2024-02-28T22:57:08.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:08.031+0000 7f9df2c00700 1 -- 172.21.15.84:0/154958807 wait complete. 2024-02-28T22:57:08.030 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:08 smithi057 ceph-mon[29096]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:08 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/154958807' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:09.714 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:09.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:10.186 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:10 smithi057 ceph-mon[29096]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 -- 172.21.15.84:0/225785641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 msgr2=0x7f8eac102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/225785641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac102430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009910 tx=0x7f8e9c009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 -- 172.21.15.84:0/225785641 shutdown_connections 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/225785641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac102430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 -- 172.21.15.84:0/225785641 >> 172.21.15.84:0/225785641 conn(0x7f8eac0fd700 msgr2=0x7f8eac0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 -- 172.21.15.84:0/225785641 shutdown_connections 2024-02-28T22:57:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.906+0000 7f8eb1a00700 1 -- 172.21.15.84:0/225785641 wait complete. 2024-02-28T22:57:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.907+0000 7f8eb1a00700 1 Processor -- start 2024-02-28T22:57:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eb1a00700 1 -- start start 2024-02-28T22:57:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eb1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eb1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1976e0 con 0x7f8eac102020 2024-02-28T22:57:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eaaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eaaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:56444/0 (socket says 172.21.15.84:56444) 2024-02-28T22:57:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.908+0000 7f8eaaa00700 1 -- 172.21.15.84:0/1081486639 learned_addr learned my addr 172.21.15.84:0/1081486639 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.909+0000 7f8eaaa00700 1 -- 172.21.15.84:0/1081486639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0095f0 con 0x7f8eac102020 2024-02-28T22:57:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.909+0000 7f8eaaa00700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005950 tx=0x7f8e9c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.909+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e9c01d070 con 0x7f8eac102020 2024-02-28T22:57:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.910+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8e9c00be10 con 0x7f8eac102020 2024-02-28T22:57:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.910+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e9c00f460 con 0x7f8eac102020 2024-02-28T22:57:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.910+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac1978e0 con 0x7f8eac102020 2024-02-28T22:57:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.910+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac197d80 con 0x7f8eac102020 2024-02-28T22:57:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.911+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac190d00 con 0x7f8eac102020 2024-02-28T22:57:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.911+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8e9c0227c0 con 0x7f8eac102020 2024-02-28T22:57:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.911+0000 7f8ea3400700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 0x7f8e9003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.911+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e9c04c510 con 0x7f8eac102020 2024-02-28T22:57:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.912+0000 7f8eaa000700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 0x7f8e9003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.912+0000 7f8eaa000700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 0x7f8e9003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006fd0 tx=0x7f8e98006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:11.919+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8e9c02bba0 con 0x7f8eac102020 2024-02-28T22:57:12.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.220+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8eac02cc30 con 0x7f8eac102020 2024-02-28T22:57:12.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.222+0000 7f8ea3400700 1 -- 172.21.15.84:0/1081486639 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8e9c027030 con 0x7f8eac102020 2024-02-28T22:57:12.220 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:12.220 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:12.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 msgr2=0x7f8e9003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 0x7f8e9003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006fd0 tx=0x7f8e98006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 msgr2=0x7f8eac1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005950 tx=0x7f8e9c009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 shutdown_connections 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f8e90038490 0x7f8e9003a940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 --2- 172.21.15.84:0/1081486639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eac102020 0x7f8eac1971a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 >> 172.21.15.84:0/1081486639 conn(0x7f8eac0fd700 msgr2=0x7f8eac0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 shutdown_connections 2024-02-28T22:57:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:12.225+0000 7f8eb1a00700 1 -- 172.21.15.84:0/1081486639 wait complete. 2024-02-28T22:57:12.225 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:12.348 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:12 smithi057 ceph-mon[29096]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:13 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/1081486639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:13 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:13 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:13 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:13 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-28T22:57:13.861 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:13.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:14.238 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:14 smithi057 ceph-mon[29096]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:14 smithi057 ceph-mon[29096]: from='mgr.14164 172.21.15.57:0/3030901290' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-28T22:57:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:14 smithi057 ceph-mon[29096]: mgrmap e14: smithi057.zrrldj(active, since 84s) 2024-02-28T22:57:14.330 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:16.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 -- 172.21.15.84:0/3200341333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 msgr2=0x7f1ae0100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 --2- 172.21.15.84:0/3200341333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0100060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009910 tx=0x7f1ad0009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 -- 172.21.15.84:0/3200341333 shutdown_connections 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 --2- 172.21.15.84:0/3200341333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0100060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 -- 172.21.15.84:0/3200341333 >> 172.21.15.84:0/3200341333 conn(0x7f1ae00fb2c0 msgr2=0x7f1ae00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 -- 172.21.15.84:0/3200341333 shutdown_connections 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.167+0000 7f1adf200700 1 -- 172.21.15.84:0/3200341333 wait complete. 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.168+0000 7f1adf200700 1 Processor -- start 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.169+0000 7f1adf200700 1 -- start start 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.169+0000 7f1adf200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.169+0000 7f1adf200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01974c0 con 0x7f1ae00ffc50 2024-02-28T22:57:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.170+0000 7f1adde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:16.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.170+0000 7f1adde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:37044/0 (socket says 172.21.15.84:37044) 2024-02-28T22:57:16.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.170+0000 7f1adde00700 1 -- 172.21.15.84:0/3220138646 learned_addr learned my addr 172.21.15.84:0/3220138646 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:16.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.170+0000 7f1adde00700 1 -- 172.21.15.84:0/3220138646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00095f0 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1adde00700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00092e0 tx=0x7f1ad0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad001d070 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ad000be10 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad000f460 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae01976c0 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.171+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae0197b60 con 0x7f1ae00ffc50 2024-02-28T22:57:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.172+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae0190cb0 con 0x7f1ae00ffc50 2024-02-28T22:57:16.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.173+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f1ad0022470 con 0x7f1ae00ffc50 2024-02-28T22:57:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.173+0000 7f1ad6a00700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 0x7f1ac403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.173+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ad004c530 con 0x7f1ae00ffc50 2024-02-28T22:57:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.173+0000 7f1add400700 1 -- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 msgr2=0x7f1ac403a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/2764732365 2024-02-28T22:57:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.174+0000 7f1add400700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 0x7f1ac403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:57:16.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.190+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1ad002ba00 con 0x7f1ae00ffc50 2024-02-28T22:57:16.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.375+0000 7f1add400700 1 -- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 msgr2=0x7f1ac403a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/2764732365 2024-02-28T22:57:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.375+0000 7f1add400700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 0x7f1ac403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:57:16.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.483+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ae002cc30 con 0x7f1ae00ffc50 2024-02-28T22:57:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.484+0000 7f1ad6a00700 1 -- 172.21.15.84:0/3220138646 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1ad0027070 con 0x7f1ae00ffc50 2024-02-28T22:57:16.482 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:16.482 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 msgr2=0x7f1ac403a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:57:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 0x7f1ac403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 msgr2=0x7f1ae0196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00092e0 tx=0x7f1ad0009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 shutdown_connections 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7f1ac4038530 0x7f1ac403a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 --2- 172.21.15.84:0/3220138646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ae00ffc50 0x7f1ae0196f80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 >> 172.21.15.84:0/3220138646 conn(0x7f1ae00fb2c0 msgr2=0x7f1ae00fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 shutdown_connections 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:16.487+0000 7f1adf200700 1 -- 172.21.15.84:0/3220138646 wait complete. 2024-02-28T22:57:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:16.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:16 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3220138646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:18.173 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:18.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:18.554 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:18.653 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:20.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 -- 172.21.15.84:0/3028354959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 msgr2=0x7fa050100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 --2- 172.21.15.84:0/3028354959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050100060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa040009910 tx=0x7fa040009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 -- 172.21.15.84:0/3028354959 shutdown_connections 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 --2- 172.21.15.84:0/3028354959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050100060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 -- 172.21.15.84:0/3028354959 >> 172.21.15.84:0/3028354959 conn(0x7fa0500fb260 msgr2=0x7fa0500fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 -- 172.21.15.84:0/3028354959 shutdown_connections 2024-02-28T22:57:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.586+0000 7fa054a00700 1 -- 172.21.15.84:0/3028354959 wait complete. 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.587+0000 7fa054a00700 1 Processor -- start 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.587+0000 7fa054a00700 1 -- start start 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.587+0000 7fa054a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.587+0000 7fa054a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050075030 con 0x7fa0500ffc50 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.588+0000 7fa04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.588+0000 7fa04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:37068/0 (socket says 172.21.15.84:37068) 2024-02-28T22:57:20.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.588+0000 7fa04f400700 1 -- 172.21.15.84:0/2513420423 learned_addr learned my addr 172.21.15.84:0/2513420423 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:20.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.589+0000 7fa04f400700 1 -- 172.21.15.84:0/2513420423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400095f0 con 0x7fa0500ffc50 2024-02-28T22:57:20.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.589+0000 7fa04f400700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa040005950 tx=0x7fa040009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.590+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa04001d070 con 0x7fa0500ffc50 2024-02-28T22:57:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.590+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa050073140 con 0x7fa0500ffc50 2024-02-28T22:57:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.590+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa04000be10 con 0x7fa0500ffc50 2024-02-28T22:57:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.590+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0500735e0 con 0x7fa0500ffc50 2024-02-28T22:57:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.590+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa04000f460 con 0x7fa0500ffc50 2024-02-28T22:57:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.591+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050062380 con 0x7fa0500ffc50 2024-02-28T22:57:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.591+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7fa0400227c0 con 0x7fa0500ffc50 2024-02-28T22:57:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.592+0000 7fa047e00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 0x7fa03403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.592+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa04004c390 con 0x7fa0500ffc50 2024-02-28T22:57:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.592+0000 7fa04ea00700 1 -- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 msgr2=0x7fa03403a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/2764732365 2024-02-28T22:57:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.592+0000 7fa04ea00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 0x7fa03403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:57:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.607+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa04000b960 con 0x7fa0500ffc50 2024-02-28T22:57:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.793+0000 7fa04ea00700 1 -- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 msgr2=0x7fa03403a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/2764732365 2024-02-28T22:57:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.793+0000 7fa04ea00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 0x7fa03403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-28T22:57:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.906+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa05018e310 con 0x7fa0500ffc50 2024-02-28T22:57:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.907+0000 7fa047e00700 1 -- 172.21.15.84:0/2513420423 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa040027070 con 0x7fa0500ffc50 2024-02-28T22:57:20.906 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:20.906 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.910+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 msgr2=0x7fa03403a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:57:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.910+0000 7fa054a00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 0x7fa03403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 msgr2=0x7fa050074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa040005950 tx=0x7fa040009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 shutdown_connections 2024-02-28T22:57:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:6800/2764732365,v1:172.21.15.57:6801/2764732365] conn(0x7fa0340384e0 0x7fa03403a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 --2- 172.21.15.84:0/2513420423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0500ffc50 0x7fa050074af0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 >> 172.21.15.84:0/2513420423 conn(0x7fa0500fb260 msgr2=0x7fa0500fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 shutdown_connections 2024-02-28T22:57:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:20.911+0000 7fa054a00700 1 -- 172.21.15.84:0/2513420423 wait complete. 2024-02-28T22:57:20.911 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:20 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/2513420423' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:22.579 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:22.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:22.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: Active manager daemon smithi057.zrrldj restarted 2024-02-28T22:57:22.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: Activating manager daemon smithi057.zrrldj 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: osdmap e5: 0 total, 0 up, 0 in 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: mgrmap e15: smithi057.zrrldj(active, starting, since 0.00836176s) 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T22:57:22.697 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:22 smithi057 ceph-mon[29096]: Manager daemon smithi057.zrrldj is now available 2024-02-28T22:57:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:23.053 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T22:57:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:23 smithi057 ceph-mon[29096]: mgrmap e16: smithi057.zrrldj(active, since 1.02112s) 2024-02-28T22:57:24.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 -- 172.21.15.84:0/1634446855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 msgr2=0x7f88c0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:24.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 --2- 172.21.15.84:0/1634446855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0100230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009910 tx=0x7f88b0009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 -- 172.21.15.84:0/1634446855 shutdown_connections 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 --2- 172.21.15.84:0/1634446855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0100230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 -- 172.21.15.84:0/1634446855 >> 172.21.15.84:0/1634446855 conn(0x7f88c00fb3d0 msgr2=0x7f88c00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 -- 172.21.15.84:0/1634446855 shutdown_connections 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.783+0000 7f88c6800700 1 -- 172.21.15.84:0/1634446855 wait complete. 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.784+0000 7f88c6800700 1 Processor -- start 2024-02-28T22:57:24.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.784+0000 7f88c6800700 1 -- start start 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.784+0000 7f88c6800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.784+0000 7f88c6800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01976b0 con 0x7f88c00ffe20 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.785+0000 7f88bf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.785+0000 7f88bf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:45966/0 (socket says 172.21.15.84:45966) 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.785+0000 7f88bf400700 1 -- 172.21.15.84:0/4234243143 learned_addr learned my addr 172.21.15.84:0/4234243143 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:24.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.785+0000 7f88bf400700 1 -- 172.21.15.84:0/4234243143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00095f0 con 0x7f88c00ffe20 2024-02-28T22:57:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.785+0000 7f88bf400700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88b00092e0 tx=0x7f88b0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.786+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88b001d070 con 0x7f88c00ffe20 2024-02-28T22:57:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.786+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88b000be10 con 0x7f88c00ffe20 2024-02-28T22:57:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.786+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88b000f460 con 0x7f88c00ffe20 2024-02-28T22:57:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.786+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c01978b0 con 0x7f88c00ffe20 2024-02-28T22:57:24.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.786+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0197d50 con 0x7f88c00ffe20 2024-02-28T22:57:24.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.787+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 45005+0+0 (secure 0 0 0) 0x7f88b0022470 con 0x7f88c00ffe20 2024-02-28T22:57:24.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.787+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0190e80 con 0x7f88c00ffe20 2024-02-28T22:57:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.788+0000 7f88b7e00700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 0x7f88a403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.788+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f88b004c180 con 0x7f88c00ffe20 2024-02-28T22:57:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.788+0000 7f88bea00700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 0x7f88a403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.789+0000 7f88bea00700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 0x7f88a403a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006fd0 tx=0x7f88ac006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:24.794+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f88b000b960 con 0x7f88c00ffe20 2024-02-28T22:57:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:24 smithi057 ceph-mon[29096]: [28/Feb/2024:22:57:22] ENGINE Bus STARTING 2024-02-28T22:57:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:24 smithi057 ceph-mon[29096]: [28/Feb/2024:22:57:22] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T22:57:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:24 smithi057 ceph-mon[29096]: [28/Feb/2024:22:57:23] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T22:57:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:24 smithi057 ceph-mon[29096]: [28/Feb/2024:22:57:23] ENGINE Bus STARTED 2024-02-28T22:57:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:24 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:25.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.091+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f88c01008b0 con 0x7f88c00ffe20 2024-02-28T22:57:25.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.092+0000 7f88b7e00700 1 -- 172.21.15.84:0/4234243143 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f88c01008b0 con 0x7f88c00ffe20 2024-02-28T22:57:25.091 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:25.091 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 msgr2=0x7f88a403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 0x7f88a403a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006fd0 tx=0x7f88ac006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 msgr2=0x7f88c0197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88b00092e0 tx=0x7f88b0009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 shutdown_connections 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f88a4038380 0x7f88a403a830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 --2- 172.21.15.84:0/4234243143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88c00ffe20 0x7f88c0197170 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 >> 172.21.15.84:0/4234243143 conn(0x7f88c00fb3d0 msgr2=0x7f88c00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 shutdown_connections 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:25.095+0000 7f88c6800700 1 -- 172.21.15.84:0/4234243143 wait complete. 2024-02-28T22:57:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:25 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/4234243143' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:25 smithi057 ceph-mon[29096]: mgrmap e17: smithi057.zrrldj(active, since 2s) 2024-02-28T22:57:26.723 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:26.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:27.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-28T22:57:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:29 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:29 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.769+0000 7fcdc1e00700 1 -- 172.21.15.84:0/742043640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 msgr2=0x7fcdbc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.769+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/742043640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc0731e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009910 tx=0x7fcdac009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.769+0000 7fcdc1e00700 1 -- 172.21.15.84:0/742043640 shutdown_connections 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.769+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/742043640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc0731e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.769+0000 7fcdc1e00700 1 -- 172.21.15.84:0/742043640 >> 172.21.15.84:0/742043640 conn(0x7fcdbc0fb3d0 msgr2=0x7fcdbc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.770+0000 7fcdc1e00700 1 -- 172.21.15.84:0/742043640 shutdown_connections 2024-02-28T22:57:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.770+0000 7fcdc1e00700 1 -- 172.21.15.84:0/742043640 wait complete. 2024-02-28T22:57:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.771+0000 7fcdc1e00700 1 Processor -- start 2024-02-28T22:57:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.771+0000 7fcdc1e00700 1 -- start start 2024-02-28T22:57:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.771+0000 7fcdc1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.771+0000 7fcdc1e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1976b0 con 0x7fcdbc074d80 2024-02-28T22:57:30.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.772+0000 7fcdbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:30.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.772+0000 7fcdbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:45980/0 (socket says 172.21.15.84:45980) 2024-02-28T22:57:30.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.772+0000 7fcdbaa00700 1 -- 172.21.15.84:0/2520640291 learned_addr learned my addr 172.21.15.84:0/2520640291 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:30.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.773+0000 7fcdbaa00700 1 -- 172.21.15.84:0/2520640291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0095f0 con 0x7fcdbc074d80 2024-02-28T22:57:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.773+0000 7fcdbaa00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0092e0 tx=0x7fcdac009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.774+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdac01d070 con 0x7fcdbc074d80 2024-02-28T22:57:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.774+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc1978b0 con 0x7fcdbc074d80 2024-02-28T22:57:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.774+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcdac00be10 con 0x7fcdbc074d80 2024-02-28T22:57:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.774+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdac00f460 con 0x7fcdbc074d80 2024-02-28T22:57:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.774+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc197d50 con 0x7fcdbc074d80 2024-02-28T22:57:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.775+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc190d00 con 0x7fcdbc074d80 2024-02-28T22:57:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.775+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fcdac022470 con 0x7fcdbc074d80 2024-02-28T22:57:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.776+0000 7fcdc0a00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 0x7fcda003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.776+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcdac04c4d0 con 0x7fcdbc074d80 2024-02-28T22:57:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.776+0000 7fcdba000700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 0x7fcda003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.778+0000 7fcdba000700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 0x7fcda003aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006fd0 tx=0x7fcda8006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:30.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:30.795+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcdac00b960 con 0x7fcdbc074d80 2024-02-28T22:57:31.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.101+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcdbc02cc30 con 0x7fcdbc074d80 2024-02-28T22:57:31.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.102+0000 7fcdc0a00700 1 -- 172.21.15.84:0/2520640291 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fcdac027030 con 0x7fcdbc074d80 2024-02-28T22:57:31.100 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:31.100 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 msgr2=0x7fcda003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 0x7fcda003aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006fd0 tx=0x7fcda8006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 msgr2=0x7fcdbc197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0092e0 tx=0x7fcdac009e60 comp rx=0 tx=0).stop 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 shutdown_connections 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fcda00385d0 0x7fcda003aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 --2- 172.21.15.84:0/2520640291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdbc074d80 0x7fcdbc197170 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:31.104 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 >> 172.21.15.84:0/2520640291 conn(0x7fcdbc0fb3d0 msgr2=0x7fcdbc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:31.104 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 shutdown_connections 2024-02-28T22:57:31.104 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:31.105+0000 7fcdc1e00700 1 -- 172.21.15.84:0/2520640291 wait complete. 2024-02-28T22:57:31.104 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:57:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T22:57:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:30 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T22:57:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:32 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:32 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/2520640291' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:32 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:32 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:57:32.815 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:32.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:33 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:57:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:33 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T22:57:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:33 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:33 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 -- 172.21.15.84:0/4229576298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 msgr2=0x7f38101024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 --2- 172.21.15.84:0/4229576298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101024a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3804009910 tx=0x7f3804009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 -- 172.21.15.84:0/4229576298 shutdown_connections 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 --2- 172.21.15.84:0/4229576298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101024a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 -- 172.21.15.84:0/4229576298 >> 172.21.15.84:0/4229576298 conn(0x7f38100fd990 msgr2=0x7f38100ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 -- 172.21.15.84:0/4229576298 shutdown_connections 2024-02-28T22:57:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.047+0000 7f3816600700 1 -- 172.21.15.84:0/4229576298 wait complete. 2024-02-28T22:57:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.049+0000 7f3816600700 1 Processor -- start 2024-02-28T22:57:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.049+0000 7f3816600700 1 -- start start 2024-02-28T22:57:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.049+0000 7f3816600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.049+0000 7f3816600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3810197920 con 0x7f38101020d0 2024-02-28T22:57:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.050+0000 7f3815200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.050+0000 7f3815200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:47232/0 (socket says 172.21.15.84:47232) 2024-02-28T22:57:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.050+0000 7f3815200700 1 -- 172.21.15.84:0/235210912 learned_addr learned my addr 172.21.15.84:0/235210912 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.050+0000 7f3815200700 1 -- 172.21.15.84:0/235210912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040095f0 con 0x7f38101020d0 2024-02-28T22:57:35.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f3815200700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38040092e0 tx=0x7f3804004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:35.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f380401d070 con 0x7f38101020d0 2024-02-28T22:57:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3810197b20 con 0x7f38101020d0 2024-02-28T22:57:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f380400be10 con 0x7f38101020d0 2024-02-28T22:57:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3804022e10 con 0x7f38101020d0 2024-02-28T22:57:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.051+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3810197f40 con 0x7f38101020d0 2024-02-28T22:57:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.052+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3810062380 con 0x7f38101020d0 2024-02-28T22:57:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.053+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f3804022470 con 0x7f38101020d0 2024-02-28T22:57:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.053+0000 7f380d600700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 0x7f37f803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.053+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f380404c480 con 0x7f38101020d0 2024-02-28T22:57:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.054+0000 7f380fe00700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 0x7f37f803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:35.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.055+0000 7f380fe00700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 0x7f37f803a9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006fd0 tx=0x7f37fc006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:35.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.069+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f380402a540 con 0x7f38101020d0 2024-02-28T22:57:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T22:57:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:57:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T22:57:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:57:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:34 smithi057 ceph-mon[29096]: Deploying daemon ceph-exporter.smithi084 on smithi084 2024-02-28T22:57:35.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.375+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3810109e00 con 0x7f38101020d0 2024-02-28T22:57:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.376+0000 7f380d600700 1 -- 172.21.15.84:0/235210912 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3804027030 con 0x7f38101020d0 2024-02-28T22:57:35.375 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:35.375 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 msgr2=0x7f37f803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 0x7f37f803a9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006fd0 tx=0x7f37fc006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 msgr2=0x7f38101973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38040092e0 tx=0x7f3804004c80 comp rx=0 tx=0).stop 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 shutdown_connections 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f37f8038530 0x7f37f803a9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 --2- 172.21.15.84:0/235210912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38101020d0 0x7f38101973e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.378+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 >> 172.21.15.84:0/235210912 conn(0x7f38100fd990 msgr2=0x7f38101062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.379+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 shutdown_connections 2024-02-28T22:57:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:35.379+0000 7f3816600700 1 -- 172.21.15.84:0/235210912 wait complete. 2024-02-28T22:57:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:35 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/235210912' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:37.733 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:37.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:57:38.310 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:57:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:39 smithi057 ceph-mon[29096]: Deploying daemon crash.smithi084 on smithi084 2024-02-28T22:57:40.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.924+0000 7f9887200700 1 -- 172.21.15.84:0/3913970418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 msgr2=0x7f9888101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:40.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.924+0000 7f9887200700 1 --2- 172.21.15.84:0/3913970418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f9888101200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9878009910 tx=0x7f9878009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:40.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.924+0000 7f9887200700 1 -- 172.21.15.84:0/3913970418 shutdown_connections 2024-02-28T22:57:40.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.924+0000 7f9887200700 1 --2- 172.21.15.84:0/3913970418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f9888101200 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.924+0000 7f9887200700 1 -- 172.21.15.84:0/3913970418 >> 172.21.15.84:0/3913970418 conn(0x7f98880fa690 msgr2=0x7f98880fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.925+0000 7f9887200700 1 -- 172.21.15.84:0/3913970418 shutdown_connections 2024-02-28T22:57:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.925+0000 7f9887200700 1 -- 172.21.15.84:0/3913970418 wait complete. 2024-02-28T22:57:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.926+0000 7f9887200700 1 Processor -- start 2024-02-28T22:57:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.926+0000 7f9887200700 1 -- start start 2024-02-28T22:57:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.927+0000 7f9887200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.927+0000 7f9887200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888100a60 con 0x7f9888100e30 2024-02-28T22:57:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.927+0000 7f9885e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.927+0000 7f9885e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:47262/0 (socket says 172.21.15.84:47262) 2024-02-28T22:57:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.927+0000 7f9885e00700 1 -- 172.21.15.84:0/3195247185 learned_addr learned my addr 172.21.15.84:0/3195247185 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.928+0000 7f9885e00700 1 -- 172.21.15.84:0/3195247185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780095f0 con 0x7f9888100e30 2024-02-28T22:57:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.928+0000 7f9885e00700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9878005b40 tx=0x7f9878004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.929+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f987801d070 con 0x7f9888100e30 2024-02-28T22:57:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.929+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98780044d0 con 0x7f9888100e30 2024-02-28T22:57:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.929+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98880ff030 con 0x7f9888100e30 2024-02-28T22:57:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.929+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9878003930 con 0x7f9888100e30 2024-02-28T22:57:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.929+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98880ff450 con 0x7f9888100e30 2024-02-28T22:57:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.930+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9888107340 con 0x7f9888100e30 2024-02-28T22:57:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.931+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f987800b960 con 0x7f9888100e30 2024-02-28T22:57:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.931+0000 7f987ea00700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 0x7f986c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.931+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f987804ca20 con 0x7f9888100e30 2024-02-28T22:57:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.932+0000 7f9885400700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 0x7f986c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:40.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.933+0000 7f9885400700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 0x7f986c03aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9874006fd0 tx=0x7f9874006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:40.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:40.946+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f987800e9d0 con 0x7f9888100e30 2024-02-28T22:57:41.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.246+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9888062380 con 0x7f9888100e30 2024-02-28T22:57:41.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.248+0000 7f987ea00700 1 -- 172.21.15.84:0/3195247185 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f987800e3e0 con 0x7f9888100e30 2024-02-28T22:57:41.247 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:41.247 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.250+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 msgr2=0x7f986c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.250+0000 7f9887200700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 0x7f986c03aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9874006fd0 tx=0x7f9874006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.250+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 msgr2=0x7f98880fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.250+0000 7f9887200700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9878005b40 tx=0x7f9878004c80 comp rx=0 tx=0).stop 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 shutdown_connections 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f986c038580 0x7f986c03aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 --2- 172.21.15.84:0/3195247185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9888100e30 0x7f98880fea90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 >> 172.21.15.84:0/3195247185 conn(0x7f98880fa690 msgr2=0x7f9888104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 shutdown_connections 2024-02-28T22:57:41.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:41.251+0000 7f9887200700 1 -- 172.21.15.84:0/3195247185 wait complete. 2024-02-28T22:57:41.251 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:41.454 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:41 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3195247185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:43.247 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:43.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:43.628 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:43.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:44 smithi057 ceph-mon[29096]: Deploying daemon node-exporter.smithi084 on smithi084 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.556+0000 7f5abf600700 1 -- 172.21.15.84:0/4083298377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 msgr2=0x7f5ab810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.556+0000 7f5abf600700 1 --2- 172.21.15.84:0/4083298377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab810c820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009910 tx=0x7f5aac009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.556+0000 7f5abf600700 1 -- 172.21.15.84:0/4083298377 shutdown_connections 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.556+0000 7f5abf600700 1 --2- 172.21.15.84:0/4083298377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab810c820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.556+0000 7f5abf600700 1 -- 172.21.15.84:0/4083298377 >> 172.21.15.84:0/4083298377 conn(0x7f5ab8075c40 msgr2=0x7f5ab8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.557+0000 7f5abf600700 1 -- 172.21.15.84:0/4083298377 shutdown_connections 2024-02-28T22:57:45.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.557+0000 7f5abf600700 1 -- 172.21.15.84:0/4083298377 wait complete. 2024-02-28T22:57:45.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.558+0000 7f5abf600700 1 Processor -- start 2024-02-28T22:57:45.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.558+0000 7f5abf600700 1 -- start start 2024-02-28T22:57:45.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.559+0000 7f5abf600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:45.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.559+0000 7f5abf600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a1c50 con 0x7f5ab810c450 2024-02-28T22:57:45.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.559+0000 7f5abcc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:45.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.559+0000 7f5abcc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:52746/0 (socket says 172.21.15.84:52746) 2024-02-28T22:57:45.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.559+0000 7f5abcc00700 1 -- 172.21.15.84:0/939441249 learned_addr learned my addr 172.21.15.84:0/939441249 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:45.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.560+0000 7f5abcc00700 1 -- 172.21.15.84:0/939441249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0095f0 con 0x7f5ab810c450 2024-02-28T22:57:45.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.560+0000 7f5abcc00700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0092e0 tx=0x7f5aac004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:45.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.561+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5aac01d070 con 0x7f5ab810c450 2024-02-28T22:57:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.561+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81a1df0 con 0x7f5ab810c450 2024-02-28T22:57:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.561+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5aac00be10 con 0x7f5ab810c450 2024-02-28T22:57:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.561+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81a2210 con 0x7f5ab810c450 2024-02-28T22:57:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.561+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5aac022e10 con 0x7f5ab810c450 2024-02-28T22:57:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.562+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8062380 con 0x7f5ab810c450 2024-02-28T22:57:45.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.563+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f5aac00b960 con 0x7f5ab810c450 2024-02-28T22:57:45.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.563+0000 7f5ab5600700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 0x7f5aa003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:45.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.563+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5aac04c4b0 con 0x7f5ab810c450 2024-02-28T22:57:45.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.564+0000 7f5ab7e00700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 0x7f5aa003a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.565+0000 7f5ab7e00700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 0x7f5aa003a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006fd0 tx=0x7f5aa4006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:45.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.578+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5aac00f460 con 0x7f5ab810c450 2024-02-28T22:57:45.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:45 smithi057 ceph-mon[29096]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.873+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5ab819cca0 con 0x7f5ab810c450 2024-02-28T22:57:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.874+0000 7f5ab5600700 1 -- 172.21.15.84:0/939441249 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5aac027020 con 0x7f5ab810c450 2024-02-28T22:57:45.874 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:45.874 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 msgr2=0x7f5aa003a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 0x7f5aa003a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006fd0 tx=0x7f5aa4006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 msgr2=0x7f5ab81a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0092e0 tx=0x7f5aac004c80 comp rx=0 tx=0).stop 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 shutdown_connections 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5aa0038530 0x7f5aa003a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 --2- 172.21.15.84:0/939441249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab810c450 0x7f5ab81a1710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.877+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 >> 172.21.15.84:0/939441249 conn(0x7f5ab8075c40 msgr2=0x7f5ab80768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.878+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 shutdown_connections 2024-02-28T22:57:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:45.878+0000 7f5abf600700 1 -- 172.21.15.84:0/939441249 wait complete. 2024-02-28T22:57:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:46 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/939441249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:47 smithi057 ceph-mon[29096]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:47.823 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:47.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:48.204 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:49 smithi057 ceph-mon[29096]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:50.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 -- 172.21.15.84:0/2591942426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 msgr2=0x7f8648102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 --2- 172.21.15.84:0/2591942426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648102490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8638009910 tx=0x7f8638009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 -- 172.21.15.84:0/2591942426 shutdown_connections 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 --2- 172.21.15.84:0/2591942426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648102490 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 -- 172.21.15.84:0/2591942426 >> 172.21.15.84:0/2591942426 conn(0x7f86480fd900 msgr2=0x7f86480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 -- 172.21.15.84:0/2591942426 shutdown_connections 2024-02-28T22:57:50.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.405+0000 7f8647e00700 1 -- 172.21.15.84:0/2591942426 wait complete. 2024-02-28T22:57:50.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.406+0000 7f8647e00700 1 Processor -- start 2024-02-28T22:57:50.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.406+0000 7f8647e00700 1 -- start start 2024-02-28T22:57:50.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.406+0000 7f8647e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:50.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.406+0000 7f8647e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86481978c0 con 0x7f86481020c0 2024-02-28T22:57:50.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.407+0000 7f8646a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:50.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.407+0000 7f8646a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:52758/0 (socket says 172.21.15.84:52758) 2024-02-28T22:57:50.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.407+0000 7f8646a00700 1 -- 172.21.15.84:0/757316225 learned_addr learned my addr 172.21.15.84:0/757316225 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:50.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.408+0000 7f8646a00700 1 -- 172.21.15.84:0/757316225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380095f0 con 0x7f86481020c0 2024-02-28T22:57:50.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.408+0000 7f8646a00700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86380092e0 tx=0x7f8638004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:50.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.409+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f863801d070 con 0x7f86481020c0 2024-02-28T22:57:50.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.409+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8648197ac0 con 0x7f86481020c0 2024-02-28T22:57:50.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.409+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648197ee0 con 0x7f86481020c0 2024-02-28T22:57:50.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.409+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f863800be10 con 0x7f86481020c0 2024-02-28T22:57:50.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.409+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f863800fdf0 con 0x7f86481020c0 2024-02-28T22:57:50.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.410+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8648062380 con 0x7f86481020c0 2024-02-28T22:57:50.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.410+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f863800f6b0 con 0x7f86481020c0 2024-02-28T22:57:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.411+0000 7f863f400700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 0x7f862c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:50.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.411+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f863804c4b0 con 0x7f86481020c0 2024-02-28T22:57:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.411+0000 7f8646000700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 0x7f862c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:50.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.413+0000 7f8646000700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 0x7f862c03aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8634006fd0 tx=0x7f8634006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:50.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.426+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f863802ba00 con 0x7f86481020c0 2024-02-28T22:57:50.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.739+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8648109e20 con 0x7f86481020c0 2024-02-28T22:57:50.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.740+0000 7f863f400700 1 -- 172.21.15.84:0/757316225 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8638027030 con 0x7f86481020c0 2024-02-28T22:57:50.740 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:50.740 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 msgr2=0x7f862c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 0x7f862c03aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8634006fd0 tx=0x7f8634006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 msgr2=0x7f8648197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86380092e0 tx=0x7f8638004c80 comp rx=0 tx=0).stop 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 shutdown_connections 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f862c038580 0x7f862c03aa30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 --2- 172.21.15.84:0/757316225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86481020c0 0x7f8648197380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 >> 172.21.15.84:0/757316225 conn(0x7f86480fd900 msgr2=0x7f86481062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:50.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.744+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 shutdown_connections 2024-02-28T22:57:50.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:50.745+0000 7f8647e00700 1 -- 172.21.15.84:0/757316225 wait complete. 2024-02-28T22:57:50.745 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:51.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:50 smithi057 ceph-mon[29096]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:51.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:50 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/757316225' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:53.122 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:53.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:53.743 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:57:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-28T22:57:53.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:57:53.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:57:53.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:53 smithi057 ceph-mon[29096]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T22:57:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:55 smithi057 ceph-mon[29096]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.940+0000 7f9d4ac00700 1 -- 172.21.15.84:0/1504240809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 msgr2=0x7f9d441025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.940+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/1504240809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d441025e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009910 tx=0x7f9d34009c20 comp rx=0 tx=0).stop 2024-02-28T22:57:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.941+0000 7f9d4ac00700 1 -- 172.21.15.84:0/1504240809 shutdown_connections 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.941+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/1504240809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d441025e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.941+0000 7f9d4ac00700 1 -- 172.21.15.84:0/1504240809 >> 172.21.15.84:0/1504240809 conn(0x7f9d440fda90 msgr2=0x7f9d440ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.941+0000 7f9d4ac00700 1 -- 172.21.15.84:0/1504240809 shutdown_connections 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.941+0000 7f9d4ac00700 1 -- 172.21.15.84:0/1504240809 wait complete. 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.942+0000 7f9d4ac00700 1 Processor -- start 2024-02-28T22:57:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.942+0000 7f9d4ac00700 1 -- start start 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.942+0000 7f9d4ac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.942+0000 7f9d4ac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d44197a60 con 0x7f9d44102210 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.943+0000 7f9d43e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.943+0000 7f9d43e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.84:41210/0 (socket says 172.21.15.84:41210) 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.943+0000 7f9d43e00700 1 -- 172.21.15.84:0/3468776196 learned_addr learned my addr 172.21.15.84:0/3468776196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:57:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.944+0000 7f9d43e00700 1 -- 172.21.15.84:0/3468776196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340095f0 con 0x7f9d44102210 2024-02-28T22:57:56.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.944+0000 7f9d43e00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005b40 tx=0x7f9d34004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:56.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.945+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d3401d070 con 0x7f9d44102210 2024-02-28T22:57:56.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.945+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d44197c60 con 0x7f9d44102210 2024-02-28T22:57:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.945+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d44198020 con 0x7f9d44102210 2024-02-28T22:57:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.945+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9d3400be10 con 0x7f9d44102210 2024-02-28T22:57:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.945+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d34022b60 con 0x7f9d44102210 2024-02-28T22:57:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.946+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24005320 con 0x7f9d44102210 2024-02-28T22:57:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.947+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f9d34022570 con 0x7f9d44102210 2024-02-28T22:57:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.947+0000 7f9d40c00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 0x7f9d2803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:57:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.948+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9d3404c380 con 0x7f9d44102210 2024-02-28T22:57:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.948+0000 7f9d43400700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 0x7f9d2803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:57:56.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.949+0000 7f9d43400700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 0x7f9d2803aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:57:56.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:56.964+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9d3400b960 con 0x7f9d44102210 2024-02-28T22:57:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.265+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9d24005190 con 0x7f9d44102210 2024-02-28T22:57:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.266+0000 7f9d40c00700 1 -- 172.21.15.84:0/3468776196 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9d34015350 con 0x7f9d44102210 2024-02-28T22:57:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:57:57.265 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:54:19.443109Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-28T22:57:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.269+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 msgr2=0x7f9d2803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.269+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 0x7f9d2803aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30006e40 comp rx=0 tx=0).stop 2024-02-28T22:57:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.269+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 msgr2=0x7f9d44197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.269+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005b40 tx=0x7f9d34004c80 comp rx=0 tx=0).stop 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 shutdown_connections 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f9d28038580 0x7f9d2803aa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 --2- 172.21.15.84:0/3468776196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d44102210 0x7f9d44197520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 >> 172.21.15.84:0/3468776196 conn(0x7f9d440fda90 msgr2=0x7f9d441063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 shutdown_connections 2024-02-28T22:57:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:57:57.270+0000 7f9d4ac00700 1 -- 172.21.15.84:0/3468776196 wait complete. 2024-02-28T22:57:57.270 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-02-28T22:57:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:57:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3468776196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:57:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:58 smithi057 ceph-mon[29096]: Deploying daemon mon.smithi084 on smithi084 2024-02-28T22:57:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:57:58.948 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-28T22:57:58.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-28T22:57:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:57:59 smithi057 ceph-mon[29096]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:57:59.986 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:00 smithi057 ceph-mon[29096]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.242 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 podman[34786]: 2024-02-28 22:58:02.195476375 +0000 UTC m=+0.292832838 container create 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 podman[34786]: 2024-02-28 22:58:02.499526006 +0000 UTC m=+0.596882449 container init 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD) 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 podman[34786]: 2024-02-28 22:58:02.506905173 +0000 UTC m=+0.604261621 container start 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: set uid:gid to 167:167 (ceph:ceph) 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pidfile_write: ignore empty --pid-file 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: load: jerasure load: lrc 2024-02-28T22:58:02.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: RocksDB version: 7.9.2 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Git sha 0 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: DB SUMMARY 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: DB Session ID: R1MC72CJKD9IC857VIVC 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: CURRENT file: CURRENT 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: IDENTITY file: IDENTITY 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi084/store.db dir, Total Num: 0, files: 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi084/store.db: 000004.log size: 511 ; 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.error_if_exists: 0 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.create_if_missing: 0 2024-02-28T22:58:02.765 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.paranoid_checks: 1 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.env: 0x5576383e9720 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.fs: PosixFileSystem 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.info_log: 0x557639c612c0 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_file_opening_threads: 16 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.statistics: (nil) 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.use_fsync: 0 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_log_file_size: 0 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.keep_log_file_num: 1000 2024-02-28T22:58:02.766 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.recycle_log_file_num: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_fallocate: 1 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_mmap_reads: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_mmap_writes: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.use_direct_reads: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.create_missing_column_families: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.db_log_dir: 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.wal_dir: 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-28T22:58:02.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.advise_random_on_open: 1 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.db_write_buffer_size: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.write_buffer_manager: 0x557638ed23c0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.rate_limiter: (nil) 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.wal_recovery_mode: 2 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enable_thread_tracking: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enable_pipelined_write: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.unordered_write: 0 2024-02-28T22:58:02.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.row_cache: None 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.wal_filter: None 2024-02-28T22:58:02.769 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_ingest_behind: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.two_write_queues: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.manual_wal_flush: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.wal_compression: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.atomic_flush: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.log_readahead_size: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.best_efforts_recovery: 0 2024-02-28T22:58:02.771 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.allow_data_in_errors: 0 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.db_host_id: __hostname__ 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_background_jobs: 2 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_background_compactions: -1 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_subcompactions: 1 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_total_wal_size: 0 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-28T22:58:02.772 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_open_files: -1 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bytes_per_sync: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_readahead_size: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_background_flushes: -1 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Compression algorithms supported: 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kZSTD supported: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kXpressCompression supported: 0 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kLZ4HCCompression supported: 1 2024-02-28T22:58:02.773 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kZlibCompression supported: 1 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kSnappyCompression supported: 1 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kLZ4Compression supported: 1 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: kBZip2Compression supported: 0 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi084/store.db/MANIFEST-000005 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.merge_operator: 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_filter: None 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_filter_factory: None 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.sst_partitioner_factory: None 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-28T22:58:02.774 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557639c613c0) 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_type: 0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: data_block_index_type: 0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_shortening: 1 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: checksum: 4 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: no_block_cache: 0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache: 0x557638f731f0 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-02-28T22:58:02.775 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_options: 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: capacity : 536870912 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: num_shard_bits : 4 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: strict_capacity_limit : 0 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_compressed: (nil) 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: persistent_cache: (nil) 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_size: 4096 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_size_deviation: 10 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_restart_interval: 16 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_block_restart_interval: 1 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: metadata_block_size: 4096 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: partition_filters: 0 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: use_delta_encoding: 1 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: filter_policy: bloomfilter 2024-02-28T22:58:02.776 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: whole_key_filtering: 1 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: verify_compression: 0 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: format_version: 5 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: enable_index_compression: 1 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_align: 0 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: max_auto_readahead_size: 262144 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: prepopulate_block_cache: 0 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: initial_auto_readahead_size: 8192 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.write_buffer_size: 33554432 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_write_buffer_number: 2 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression: NoCompression 2024-02-28T22:58:02.777 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression: Disabled 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.prefix_extractor: nullptr 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.num_levels: 7 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-28T22:58:02.778 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.level: 32767 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.strategy: 0 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.enabled: false 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-28T22:58:02.779 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.target_file_size_base: 67108864 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-28T22:58:02.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.arena_block_size: 1048576 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.disable_auto_compactions: 0 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-28T22:58:02.781 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.table_properties_collectors: 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.inplace_update_support: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.bloom_locality: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.max_successive_merges: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.paranoid_file_checks: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.force_consistency_checks: 1 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.report_bg_io_stats: 0 2024-02-28T22:58:02.782 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.ttl: 2592000 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enable_blob_files: false 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.min_blob_size: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_file_size: 268435456 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.blob_file_starting_level: 0 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-28T22:58:02.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi084/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ea462b4a-01a8-46aa-8522-ef4f18e31a70 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161082553501, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161082554927, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709161082, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ea462b4a-01a8-46aa-8522-ef4f18e31a70", "db_session_id": "R1MC72CJKD9IC857VIVC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161082555132, "job": 1, "event": "recovery_finished"} 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi084/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557639010000 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: DB pointer 0x557638ffc000 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-28T22:58:02.784 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** DB Stats ** 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** Compaction Stats [default] ** 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-28T22:58:02.785 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** Compaction Stats [default] ** 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-28T22:58:02.786 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Block cache BinnedLRUCache@0x557638f731f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084 does not exist in monmap, will attempt to join an existing cluster 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: using public_addr v2:172.21.15.84:0/0 -> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: starting mon.smithi084 rank -1 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi084 fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(???) e0 preinit fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).mds e1 new map 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).mds e1 print_map 2024-02-28T22:58:02.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: e1 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: legacy client fscid: -1 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: No filesystems configured 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-28T22:58:02.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:58:02.789 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/2520640291' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.790 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Deploying daemon ceph-exporter.smithi084 on smithi084 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/235210912' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:58:02.791 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Deploying daemon crash.smithi084 on smithi084 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/3195247185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Deploying daemon node-exporter.smithi084 on smithi084 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/939441249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/757316225' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T22:58:02.793 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/3468776196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: Deploying daemon mon.smithi084 on smithi084 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: mon.smithi084@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-02-28T22:58:02.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expand map: {default=false} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta from 'false' to 'false' 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expanded map: {default=false} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expand map: {default=info} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta from 'info' to 'info' 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expanded map: {default=info} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expand map: {default=daemon} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expanded map: {default=daemon} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expand map: {default=debug} 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta from 'debug' to 'debug' 2024-02-28T22:58:02.795 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 ceph-mon[34826]: expand_channel_meta expanded map: {default=debug} 2024-02-28T22:58:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 bash[34786]: 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae 2024-02-28T22:58:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:02 smithi084 systemd[1]: Started Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: mon.smithi057 calling monitor election 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: mon.smithi084 calling monitor election 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T22:58:08.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: monmap e2: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: fsmap 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: osdmap e5: 0 total, 0 up, 0 in 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: mgrmap e17: smithi057.zrrldj(active, since 45s) 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis started 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: overall HEALTH_OK 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T22:58:08.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T22:58:08.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:08.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:08.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: mon.smithi057 calling monitor election 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: mon.smithi084 calling monitor election 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T22:58:08.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: monmap e2: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: fsmap 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: osdmap e5: 0 total, 0 up, 0 in 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: mgrmap e17: smithi057.zrrldj(active, since 45s) 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis started 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: overall HEALTH_OK 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3147156629' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:08.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:08.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 -- 172.21.15.84:0/3747190508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38a8006220 msgr2=0x7f38a80065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 --2- 172.21.15.84:0/3747190508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38a8006220 0x7f38a80065f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38c40741b0 tx=0x7f38b400fe90 comp rx=0 tx=0).stop 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 -- 172.21.15.84:0/3747190508 shutdown_connections 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 --2- 172.21.15.84:0/3747190508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38a8006220 0x7f38a80065f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 -- 172.21.15.84:0/3747190508 >> 172.21.15.84:0/3747190508 conn(0x7f38c40fb970 msgr2=0x7f38c40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 -- 172.21.15.84:0/3747190508 shutdown_connections 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.743+0000 7f38c9200700 1 -- 172.21.15.84:0/3747190508 wait complete. 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.745+0000 7f38c9200700 1 Processor -- start 2024-02-28T22:58:08.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.746+0000 7f38c9200700 1 -- start start 2024-02-28T22:58:08.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.746+0000 7f38c9200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:08.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.746+0000 7f38c9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 0x7f38c419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.746+0000 7f38c9200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c4198260 con 0x7f38c4197c50 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.746+0000 7f38c9200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419b4c0 con 0x7f38a8006220 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55260/0 (socket says 172.21.15.84:55260) 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 -- 172.21.15.84:0/72638804 learned_addr learned my addr 172.21.15.84:0/72638804 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 -- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 msgr2=0x7f38c4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 -- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 msgr2=0x7f38c4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1e00700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-28T22:58:08.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.747+0000 7f38c1400700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 0x7f38c419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:08.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.748+0000 7f38c1400700 1 -- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 msgr2=0x7f38c4197680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-28T22:58:08.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.748+0000 7f38c1400700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:08.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.748+0000 7f38c1400700 1 -- 172.21.15.84:0/72638804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b400f8e0 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.748+0000 7f38c1400700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 0x7f38c419af80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38b000b4d0 tx=0x7f38b000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.749+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b0011800 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.749+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c419b7a0 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.749+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c419bcf0 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.749+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38b0011e10 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.749+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b00193b0 con 0x7f38c4197c50 2024-02-28T22:58:08.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.750+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8005e10 con 0x7f38c4197c50 2024-02-28T22:58:08.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.752+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f38b0011960 con 0x7f38c4197c50 2024-02-28T22:58:08.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.753+0000 7f38baa00700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 0x7f38a406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:08.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.753+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f38b008a6c0 con 0x7f38c4197c50 2024-02-28T22:58:08.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.754+0000 7f38c1e00700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 0x7f38a406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:08.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.755+0000 7f38c1e00700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 0x7f38a406ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001eb0 tx=0x7f38b4012a30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:08.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:08.766+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f38b0055e60 con 0x7f38c4197c50 2024-02-28T22:58:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: mgrmap e18: smithi057.zrrldj(active, since 45s), standbys: smithi084.fhkzis 2024-02-28T22:58:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T22:58:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:58:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:09.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.072+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38a8001d10 con 0x7f38c4197c50 2024-02-28T22:58:09.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: mgrmap e18: smithi057.zrrldj(active, since 45s), standbys: smithi084.fhkzis 2024-02-28T22:58:09.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T22:58:09.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:09.072 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:58:09.072 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T22:58:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.074+0000 7f38baa00700 1 -- 172.21.15.84:0/72638804 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7f38b0059480 con 0x7f38c4197c50 2024-02-28T22:58:09.072 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-02-28T22:58:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","modified":"2024-02-28T22:58:02.611729Z","created":"2024-02-28T22:54:19.443109Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi057","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi084","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 msgr2=0x7f38a406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 0x7f38a406ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001eb0 tx=0x7f38b4012a30 comp rx=0 tx=0).stop 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 msgr2=0x7f38c419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 0x7f38c419af80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38b000b4d0 tx=0x7f38b000b890 comp rx=0 tx=0).stop 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 shutdown_connections 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38a406c750 0x7f38a406ec00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38a8006220 0x7f38c4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 --2- 172.21.15.84:0/72638804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38c4197c50 0x7f38c419af80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.076+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 >> 172.21.15.84:0/72638804 conn(0x7f38c40fb970 msgr2=0x7f38c41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:58:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.077+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 shutdown_connections 2024-02-28T22:58:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T22:58:09.077+0000 7f38c9200700 1 -- 172.21.15.84:0/72638804 wait complete. 2024-02-28T22:58:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 2 2024-02-28T22:58:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:09 smithi057 ceph-mon[29096]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:09 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/72638804' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:09 smithi084 ceph-mon[34826]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:09 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/72638804' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-28T22:58:10.517 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-28T22:58:10.518 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph config generate-minimal-conf 2024-02-28T22:58:10.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:10 smithi057 ceph-mon[29096]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:10.946 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:58:11.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:10 smithi084 ceph-mon[34826]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T22:58:12.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:12 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T22:58:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:12 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T22:58:13.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.017+0000 7fbbd2200700 1 -- 172.21.15.57:0/2391374026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 msgr2=0x7fbbcc0ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:13.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.017+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2391374026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc0ff390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009910 tx=0x7fbbbc009c20 comp rx=0 tx=0).stop 2024-02-28T22:58:13.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.018+0000 7fbbd2200700 1 -- 172.21.15.57:0/2391374026 shutdown_connections 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.018+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2391374026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc0ff390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.018+0000 7fbbd2200700 1 -- 172.21.15.57:0/2391374026 >> 172.21.15.57:0/2391374026 conn(0x7fbbcc074af0 msgr2=0x7fbbcc074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.018+0000 7fbbd2200700 1 -- 172.21.15.57:0/2391374026 shutdown_connections 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.018+0000 7fbbd2200700 1 -- 172.21.15.57:0/2391374026 wait complete. 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.019+0000 7fbbd2200700 1 Processor -- start 2024-02-28T22:58:13.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.019+0000 7fbbd2200700 1 -- start start 2024-02-28T22:58:13.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbd2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:13.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbd2200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbcc100f10 0x7fbbcc101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:13.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbd2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1a1820 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbd2200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1a1960 con 0x7fbbcc100f10 2024-02-28T22:58:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbcb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbcb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35030/0 (socket says 172.21.15.57:35030) 2024-02-28T22:58:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.020+0000 7fbbcb400700 1 -- 172.21.15.57:0/2298846751 learned_addr learned my addr 172.21.15.57:0/2298846751 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbcb400700 1 -- 172.21.15.57:0/2298846751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbcc100f10 msgr2=0x7fbbcc101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbcb400700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbcc100f10 0x7fbbcc101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbcb400700 1 -- 172.21.15.57:0/2298846751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0095f0 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbcb400700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006010 tx=0x7fbbbc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbbc01d070 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc005140 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc1a1aa0 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.021+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbbc017440 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.022+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc1a1f90 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.023+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc062380 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.024+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7fbbbc00f460 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.025+0000 7fbbc8c00700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 0x7fbbb006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:13.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.025+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbbbc08bad0 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.025+0000 7fbbc3e00700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 0x7fbbb006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.026+0000 7fbbc3e00700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 0x7fbbb006ec60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40098e0 tx=0x7fbbb4006dd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:13.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.030+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbbbc057270 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.254+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbbcc1a22f0 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.255+0000 7fbbc8c00700 1 -- 172.21.15.57:0/2298846751 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fbbbc026090 con 0x7fbbcc0fefc0 2024-02-28T22:58:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:58:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2024-02-28T22:58:13.254 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T22:58:13.254 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.257+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 msgr2=0x7fbbb006ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.257+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 0x7fbbb006ec60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40098e0 tx=0x7fbbb4006dd0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.257+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 msgr2=0x7fbbcc1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.257+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006010 tx=0x7fbbbc00bda0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 shutdown_connections 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbbb006c7b0 0x7fbbb006ec60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbcc0fefc0 0x7fbbcc1009d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 --2- 172.21.15.57:0/2298846751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbcc100f10 0x7fbbcc101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:13.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 >> 172.21.15.57:0/2298846751 conn(0x7fbbcc074af0 msgr2=0x7fbbcc067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:58:13.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 shutdown_connections 2024-02-28T22:58:13.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:13.258+0000 7fbbd2200700 1 -- 172.21.15.57:0/2298846751 wait complete. 2024-02-28T22:58:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:13 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2298846751' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:58:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T22:58:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:13 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2298846751' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:14.055 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-28T22:58:14.056 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:58:14.056 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2024-02-28T22:58:14.152 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:58:14.152 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:58:14.284 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:58:14.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-02-28T22:58:14.387 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:58:14.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-28T22:58:14.525 INFO:tasks.cephadm:Deploying OSDs... 2024-02-28T22:58:14.525 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T22:58:14.525 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2024-02-28T22:58:14.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-28T22:58:14.586 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2024-02-28T22:58:14.639 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-28T22:58:14.639 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:14.639 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 636 Links: 1 2024-02-28T22:58:14.639 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:14.639 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:14.640 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-02-28 22:57:11.575568664 +0000 2024-02-28T22:58:14.640 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-02-28 22:49:54.391881453 +0000 2024-02-28T22:58:14.640 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-02-28 22:49:54.391881453 +0000 2024-02-28T22:58:14.640 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-02-28 22:49:54.391881453 +0000 2024-02-28T22:58:14.640 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-28T22:58:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-02-28T22:58:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-02-28T22:58:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000335975 s, 1.5 MB/s 2024-02-28T22:58:14.713 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-28T22:58:14.766 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 655 Links: 1 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-02-28 22:57:11.576568648 +0000 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-02-28 22:49:54.978872541 +0000 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-02-28 22:49:54.978872541 +0000 2024-02-28T22:58:14.817 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-02-28 22:49:54.978872541 +0000 2024-02-28T22:58:14.817 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-28T22:58:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:14 smithi084 ceph-mon[34826]: Reconfiguring mon.smithi057 (unknown last config time)... 2024-02-28T22:58:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:14 smithi084 ceph-mon[34826]: Reconfiguring daemon mon.smithi057 on smithi057 2024-02-28T22:58:14.898 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:14 smithi057 ceph-mon[29096]: Reconfiguring mon.smithi057 (unknown last config time)... 2024-02-28T22:58:14.898 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:14 smithi057 ceph-mon[29096]: Reconfiguring daemon mon.smithi057 on smithi057 2024-02-28T22:58:14.908 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-02-28T22:58:14.908 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-02-28T22:58:14.908 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.00040988 s, 1.2 MB/s 2024-02-28T22:58:14.913 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-28T22:58:14.972 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2024-02-28T22:58:15.018 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-28T22:58:15.018 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-02-28 22:57:11.577568633 +0000 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-02-28 22:49:55.590863251 +0000 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-02-28 22:49:55.590863251 +0000 2024-02-28T22:58:15.019 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-02-28 22:49:55.590863251 +0000 2024-02-28T22:58:15.019 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-28T22:58:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-02-28T22:58:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-02-28T22:58:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000564138 s, 908 kB/s 2024-02-28T22:58:15.158 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-28T22:58:15.211 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 694 Links: 1 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-02-28 22:57:11.578568618 +0000 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-02-28 22:49:56.190854142 +0000 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-02-28 22:49:56.190854142 +0000 2024-02-28T22:58:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-02-28 22:49:56.190854142 +0000 2024-02-28T22:58:15.260 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-28T22:58:15.358 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-02-28T22:58:15.359 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-02-28T22:58:15.359 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000482116 s, 1.1 MB/s 2024-02-28T22:58:15.366 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-28T22:58:15.424 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T22:58:15.425 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-02-28T22:58:15.482 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-28T22:58:15.482 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-02-28T22:58:15.535 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-02-28 22:58:01.001536469 +0000 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-02-28 22:49:44.064415627 +0000 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-02-28 22:49:44.064415627 +0000 2024-02-28T22:58:15.536 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-02-28 22:49:44.064415627 +0000 2024-02-28T22:58:15.536 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-28T22:58:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-02-28T22:58:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-02-28T22:58:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000424863 s, 1.2 MB/s 2024-02-28T22:58:15.625 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-28T22:58:15.673 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-02-28 22:58:01.001536469 +0000 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-02-28 22:49:44.618406264 +0000 2024-02-28T22:58:15.720 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-02-28 22:49:44.618406264 +0000 2024-02-28T22:58:15.721 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-02-28 22:49:44.618406264 +0000 2024-02-28T22:58:15.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-28T22:58:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:15 smithi057 ceph-mon[29096]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:15 smithi084 ceph-mon[34826]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-02-28T22:58:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-02-28T22:58:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000519135 s, 986 kB/s 2024-02-28T22:58:15.858 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-28T22:58:15.914 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-02-28 22:58:01.002536452 +0000 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-02-28 22:49:45.184396698 +0000 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-02-28 22:49:45.184396698 +0000 2024-02-28T22:58:15.960 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-02-28 22:49:45.184396698 +0000 2024-02-28T22:58:15.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-28T22:58:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-02-28T22:58:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-02-28T22:58:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000329041 s, 1.6 MB/s 2024-02-28T22:58:16.096 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-28T22:58:16.152 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-02-28T22:58:16.202 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-28T22:58:16.202 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-28T22:58:16.202 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-02-28T22:58:16.202 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T22:58:16.202 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-02-28T22:58:16.203 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-02-28 22:58:01.003536435 +0000 2024-02-28T22:58:16.203 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-02-28 22:49:45.748387166 +0000 2024-02-28T22:58:16.203 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-02-28 22:49:45.748387166 +0000 2024-02-28T22:58:16.203 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-02-28 22:49:45.748387166 +0000 2024-02-28T22:58:16.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-28T22:58:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-02-28T22:58:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-02-28T22:58:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000342932 s, 1.5 MB/s 2024-02-28T22:58:16.295 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-28T22:58:16.349 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2024-02-28T22:58:16.349 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: Reconfiguring mgr.smithi057.zrrldj (unknown last config time)... 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:16.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:16 smithi057 ceph-mon[29096]: Reconfiguring daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: Reconfiguring mgr.smithi057.zrrldj (unknown last config time)... 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:16 smithi084 ceph-mon[34826]: Reconfiguring daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T22:58:17.684 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:58:17.838 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:17 smithi057 ceph-mon[29096]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:17 smithi084 ceph-mon[34826]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: Reconfiguring ceph-exporter.smithi057 (monmap changed)... 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:18.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:18 smithi057 ceph-mon[29096]: Reconfiguring daemon ceph-exporter.smithi057 on smithi057 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: Reconfiguring ceph-exporter.smithi057 (monmap changed)... 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:18 smithi084 ceph-mon[34826]: Reconfiguring daemon ceph-exporter.smithi057 on smithi057 2024-02-28T22:58:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:19 smithi057 ceph-mon[29096]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:19 smithi084 ceph-mon[34826]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:21.215 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-28T22:58:21.215 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-28T22:58:21.215 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-02-28T22:58:21.216 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-02-28T22:58:21.216 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0369215 s, 284 MB/s 2024-02-28T22:58:21.216 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: Reconfiguring crash.smithi057 (monmap changed)... 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: Reconfiguring daemon crash.smithi057 on smithi057 2024-02-28T22:58:21.456 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:21 smithi057 ceph-mon[29096]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:21.637 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:21.637 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:21.637 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: Reconfiguring crash.smithi057 (monmap changed)... 2024-02-28T22:58:21.637 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:58:21.637 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:21.638 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: Reconfiguring daemon crash.smithi057 on smithi057 2024-02-28T22:58:21.638 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:21 smithi084 ceph-mon[34826]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:22.360 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2024-02-28T22:58:22.977 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:58:23.815 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:23 smithi057 ceph-mon[29096]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:23.815 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:23.815 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:23.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:23 smithi084 ceph-mon[34826]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:24.795 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:24 smithi057 ceph-mon[29096]: Reconfiguring alertmanager.smithi057 (dependencies changed)... 2024-02-28T22:58:24.795 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:24 smithi057 ceph-mon[29096]: Reconfiguring daemon alertmanager.smithi057 on smithi057 2024-02-28T22:58:24.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:24 smithi084 ceph-mon[34826]: Reconfiguring alertmanager.smithi057 (dependencies changed)... 2024-02-28T22:58:24.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:24 smithi084 ceph-mon[34826]: Reconfiguring daemon alertmanager.smithi057 on smithi057 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 -- 172.21.15.57:0/193881580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188100100 msgr2=0x7f2188100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 --2- 172.21.15.57:0/193881580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188100100 0x7f2188100510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f217c009910 tx=0x7f217c009c20 comp rx=0 tx=0).stop 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 -- 172.21.15.57:0/193881580 shutdown_connections 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 --2- 172.21.15.57:0/193881580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188101300 0x7f2188101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 --2- 172.21.15.57:0/193881580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188100100 0x7f2188100510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 -- 172.21.15.57:0/193881580 >> 172.21.15.57:0/193881580 conn(0x7f21880fb6b0 msgr2=0x7f21880fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 -- 172.21.15.57:0/193881580 shutdown_connections 2024-02-28T22:58:25.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.330+0000 7f218ea00700 1 -- 172.21.15.57:0/193881580 wait complete. 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.331+0000 7f218ea00700 1 Processor -- start 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218ea00700 1 -- start start 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218ea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188100100 0x7f2188197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218ea00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2188198660 con 0x7f2188101300 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218ea00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881987a0 con 0x7f2188100100 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45578/0 (socket says 172.21.15.57:45578) 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.332+0000 7f218cc00700 1 -- 172.21.15.57:0/359765562 learned_addr learned my addr 172.21.15.57:0/359765562 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:58:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f218d600700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188100100 0x7f2188197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:25.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f218cc00700 1 -- 172.21.15.57:0/359765562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188100100 msgr2=0x7f2188197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:58:25.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f218cc00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188100100 0x7f2188197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:58:25.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f218cc00700 1 -- 172.21.15.57:0/359765562 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0095f0 con 0x7f2188101300 2024-02-28T22:58:25.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f218cc00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f217400b4b0 tx=0x7f217400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.333+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2174011800 con 0x7f2188101300 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.334+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2174011e10 con 0x7f2188101300 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.334+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f217400f2e0 con 0x7f2188101300 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.334+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218819d250 con 0x7f2188101300 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.334+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218819d820 con 0x7f2188101300 2024-02-28T22:58:25.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.335+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21881053e0 con 0x7f2188101300 2024-02-28T22:58:25.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.336+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f2174010380 con 0x7f2188101300 2024-02-28T22:58:25.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.337+0000 7f2186000700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 0x7f217006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:58:25.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.337+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f217408a360 con 0x7f2188101300 2024-02-28T22:58:25.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.338+0000 7f218d600700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 0x7f217006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:58:25.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.338+0000 7f218d600700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 0x7f217006eb00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f217c006010 tx=0x7f217c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:58:25.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.342+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2174059640 con 0x7f2188101300 2024-02-28T22:58:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:58:25.575+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2188061190 con 0x7f217006c650 2024-02-28T22:58:25.762 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:25 smithi057 ceph-mon[29096]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:25 smithi084 ceph-mon[34826]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:26.717 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:26 smithi057 ceph-mon[29096]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:58:26.717 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:58:26.717 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:58:26.717 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:26 smithi084 ceph-mon[34826]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:58:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:58:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:58:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:27 smithi057 ceph-mon[29096]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:27 smithi084 ceph-mon[34826]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:29 smithi057 ceph-mon[29096]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:29.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:29 smithi084 ceph-mon[34826]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:30 smithi057 ceph-mon[29096]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:31.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:30 smithi084 ceph-mon[34826]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:32.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:32 smithi057 ceph-mon[29096]: Reconfiguring grafana.smithi057 (dependencies changed)... 2024-02-28T22:58:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:32 smithi057 ceph-mon[29096]: Reconfiguring daemon grafana.smithi057 on smithi057 2024-02-28T22:58:32.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:32 smithi084 ceph-mon[34826]: Reconfiguring grafana.smithi057 (dependencies changed)... 2024-02-28T22:58:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:32 smithi084 ceph-mon[34826]: Reconfiguring daemon grafana.smithi057 on smithi057 2024-02-28T22:58:33.694 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:33 smithi057 ceph-mon[29096]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:33.694 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:33 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1915373201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "515f39d8-af53-46ca-a205-a49b706e5a64"}]: dispatch 2024-02-28T22:58:33.695 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:33 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1915373201' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "515f39d8-af53-46ca-a205-a49b706e5a64"}]': finished 2024-02-28T22:58:33.695 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:33 smithi057 ceph-mon[29096]: osdmap e6: 1 total, 0 up, 1 in 2024-02-28T22:58:33.695 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:33 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:58:33.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:33 smithi084 ceph-mon[34826]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:33.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:33 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1915373201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "515f39d8-af53-46ca-a205-a49b706e5a64"}]: dispatch 2024-02-28T22:58:33.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:33 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1915373201' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "515f39d8-af53-46ca-a205-a49b706e5a64"}]': finished 2024-02-28T22:58:33.787 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:33 smithi084 ceph-mon[34826]: osdmap e6: 1 total, 0 up, 1 in 2024-02-28T22:58:33.788 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:33 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:58:35.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:34 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/962140524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:58:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:34 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/962140524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:58:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:35 smithi057 ceph-mon[29096]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:35 smithi084 ceph-mon[34826]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:37 smithi057 ceph-mon[29096]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:37 smithi084 ceph-mon[34826]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:39 smithi057 ceph-mon[29096]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:39 smithi084 ceph-mon[34826]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:40 smithi057 ceph-mon[29096]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:41.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:40 smithi084 ceph-mon[34826]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:43.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:43 smithi057 ceph-mon[29096]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:43.821 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:43 smithi084 ceph-mon[34826]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:45.699 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:45 smithi057 ceph-mon[29096]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:45.699 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:45 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:45.699 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:45 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:45 smithi084 ceph-mon[34826]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:45 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:45 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:46.581 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:46 smithi057 ceph-mon[29096]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T22:58:46.581 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:46 smithi057 ceph-mon[29096]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T22:58:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:46 smithi084 ceph-mon[34826]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T22:58:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:46 smithi084 ceph-mon[34826]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T22:58:47.660 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:47 smithi057 ceph-mon[29096]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:47.660 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T22:58:47.660 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:47 smithi084 ceph-mon[34826]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T22:58:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:48 smithi057 ceph-mon[29096]: Deploying daemon osd.0 on smithi057 2024-02-28T22:58:48.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:48 smithi084 ceph-mon[34826]: Deploying daemon osd.0 on smithi057 2024-02-28T22:58:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:49 smithi057 ceph-mon[29096]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:49.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:49 smithi084 ceph-mon[34826]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:51.274 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:50 smithi057 ceph-mon[29096]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:50 smithi084 ceph-mon[34826]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:52.756 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:53.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:58:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:53 smithi057 ceph-mon[29096]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:58:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:53.856 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:53 smithi084 ceph-mon[34826]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:53.856 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:53.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:53.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T22:58:53.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:54.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:54 smithi084 ceph-mon[34826]: Reconfiguring ceph-exporter.smithi084 (monmap changed)... 2024-02-28T22:58:54.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:54 smithi084 ceph-mon[34826]: Reconfiguring daemon ceph-exporter.smithi084 on smithi084 2024-02-28T22:58:55.016 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:54 smithi057 ceph-mon[29096]: Reconfiguring ceph-exporter.smithi084 (monmap changed)... 2024-02-28T22:58:55.016 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:54 smithi057 ceph-mon[29096]: Reconfiguring daemon ceph-exporter.smithi084 on smithi084 2024-02-28T22:58:55.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:55 smithi084 ceph-mon[34826]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:55.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:55.689 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:55.689 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:58:55.689 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:55.807 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:55 smithi057 ceph-mon[29096]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:55.808 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:55.808 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:55.808 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T22:58:55.808 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:56.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:56 smithi057 ceph-mon[29096]: Reconfiguring crash.smithi084 (monmap changed)... 2024-02-28T22:58:56.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:56 smithi057 ceph-mon[29096]: Reconfiguring daemon crash.smithi084 on smithi084 2024-02-28T22:58:57.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:56 smithi084 ceph-mon[34826]: Reconfiguring crash.smithi084 (monmap changed)... 2024-02-28T22:58:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:56 smithi084 ceph-mon[34826]: Reconfiguring daemon crash.smithi084 on smithi084 2024-02-28T22:58:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: Reconfiguring mgr.smithi084.fhkzis (monmap changed)... 2024-02-28T22:58:57.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:58:57.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:58:57.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: Reconfiguring mgr.smithi084.fhkzis (monmap changed)... 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T22:58:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: Reconfiguring daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T22:58:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:58:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T22:58:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:58 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: Reconfiguring daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T22:58:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:58 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:59 smithi057 ceph-mon[29096]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:00.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:59 smithi057 ceph-mon[29096]: Reconfiguring mon.smithi084 (monmap changed)... 2024-02-28T22:59:00.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:58:59 smithi057 ceph-mon[29096]: Reconfiguring daemon mon.smithi084 on smithi084 2024-02-28T22:59:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:59 smithi084 ceph-mon[34826]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:59 smithi084 ceph-mon[34826]: Reconfiguring mon.smithi084 (monmap changed)... 2024-02-28T22:59:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:58:59 smithi084 ceph-mon[34826]: Reconfiguring daemon mon.smithi084 on smithi084 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-02-28T22:59:01.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi057.front.sepia.ceph.com:3000"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi057.front.sepia.ceph.com:3000"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi057.front.sepia.ceph.com:9095"}]: dispatch 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:01 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-02-28T22:59:01.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi057.front.sepia.ceph.com:3000"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi057.front.sepia.ceph.com:3000"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi057.front.sepia.ceph.com:9095"}]: dispatch 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:01.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:01 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:02.640 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:02 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T22:59:02.640 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:02 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi057.front.sepia.ceph.com:9095"}]: dispatch 2024-02-28T22:59:02.750 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:02 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T22:59:02.750 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:02 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi057.front.sepia.ceph.com:9095"}]: dispatch 2024-02-28T22:59:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:03 smithi057 ceph-mon[29096]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:03 smithi084 ceph-mon[34826]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:04.729 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:04 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:04.729 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:04 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:04.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:04 smithi057 ceph-mon[29096]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-28T22:59:04.730 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2024-02-28T22:59:04.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.730+0000 7f2186000700 1 -- 172.21.15.57:0/359765562 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2188061190 con 0x7f217006c650 2024-02-28T22:59:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 msgr2=0x7f217006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 0x7f217006eb00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f217c006010 tx=0x7f217c00b540 comp rx=0 tx=0).stop 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 msgr2=0x7f2188198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f217400b4b0 tx=0x7f217400b870 comp rx=0 tx=0).stop 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 shutdown_connections 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f217006c650 0x7f217006eb00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2188100100 0x7f2188197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 --2- 172.21.15.57:0/359765562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2188101300 0x7f2188198040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 >> 172.21.15.57:0/359765562 conn(0x7f21880fb6b0 msgr2=0x7f2188102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 shutdown_connections 2024-02-28T22:59:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:04.733+0000 7f218ea00700 1 -- 172.21.15.57:0/359765562 wait complete. 2024-02-28T22:59:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:04 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:04 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:04 smithi084 ceph-mon[34826]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-28T22:59:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: osdmap e7: 1 total, 0 up, 1 in 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:05.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:05 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: osdmap e7: 1 total, 0 up, 1 in 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:05 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:06.127 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.0.service 2024-02-28T22:59:06.134 INFO:tasks.cephadm:Deploying osd.1 on smithi057 with /dev/vg_nvme/lv_3... 2024-02-28T22:59:06.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-28T22:59:07.032 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:59:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:06 smithi057 ceph-mon[29096]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T22:59:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:06 smithi057 ceph-mon[29096]: osdmap e8: 1 total, 0 up, 1 in 2024-02-28T22:59:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:06 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:06 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:06 smithi084 ceph-mon[34826]: from='osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T22:59:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:06 smithi084 ceph-mon[34826]: osdmap e8: 1 total, 0 up, 1 in 2024-02-28T22:59:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:06 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:06 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T22:59:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] boot 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: osdmap e9: 1 total, 1 up, 1 in 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: osd.0 [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] boot 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: osdmap e9: 1 total, 1 up, 1 in 2024-02-28T22:59:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T22:59:08.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:08.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:08 smithi057 ceph-mon[29096]: osdmap e10: 1 total, 1 up, 1 in 2024-02-28T22:59:09.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:08 smithi084 ceph-mon[34826]: osdmap e10: 1 total, 1 up, 1 in 2024-02-28T22:59:10.017 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:09 smithi057 ceph-mon[29096]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:10.139 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:09 smithi084 ceph-mon[34826]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0371978 s, 282 MB/s 2024-02-28T22:59:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-02-28T22:59:11.151 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2024-02-28T22:59:11.565 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:59:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:11 smithi057 ceph-mon[29096]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:11 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:11 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:11.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:11 smithi084 ceph-mon[34826]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:11.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:11 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:11 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:13.678 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:13 smithi057 ceph-mon[29096]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:13.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:13 smithi084 ceph-mon[34826]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:14.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 -- 172.21.15.57:0/1021540828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d01036d0 msgr2=0x7f00d0103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:14.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 --2- 172.21.15.57:0/1021540828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d01036d0 0x7f00d0103b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009910 tx=0x7f00bc009c20 comp rx=0 tx=0).stop 2024-02-28T22:59:14.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 -- 172.21.15.57:0/1021540828 shutdown_connections 2024-02-28T22:59:14.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 --2- 172.21.15.57:0/1021540828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d01036d0 0x7f00d0103b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:14.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 --2- 172.21.15.57:0/1021540828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0102430 0x7f00d0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:14.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 -- 172.21.15.57:0/1021540828 >> 172.21.15.57:0/1021540828 conn(0x7f00d00fda80 msgr2=0x7f00d00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:59:14.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 -- 172.21.15.57:0/1021540828 shutdown_connections 2024-02-28T22:59:14.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.195+0000 7f00d5a00700 1 -- 172.21.15.57:0/1021540828 wait complete. 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.196+0000 7f00d5a00700 1 Processor -- start 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00d5a00700 1 -- start start 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00d5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00d5a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d01036d0 0x7f00d01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00d5a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01987d0 con 0x7f00d0102430 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00d5a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0198940 con 0x7f00d01036d0 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00cea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00cea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33652/0 (socket says 172.21.15.57:33652) 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.197+0000 7f00cea00700 1 -- 172.21.15.57:0/2072558764 learned_addr learned my addr 172.21.15.57:0/2072558764 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00ce000700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d01036d0 0x7f00d01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:14.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00cea00700 1 -- 172.21.15.57:0/2072558764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d01036d0 msgr2=0x7f00d01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00cea00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d01036d0 0x7f00d01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00cea00700 1 -- 172.21.15.57:0/2072558764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0095f0 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00cea00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c8a0 tx=0x7f00c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c000cea0 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00c0004830 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c00055b0 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.198+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d019d3d0 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.199+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d019d920 con 0x7f00d0102430 2024-02-28T22:59:14.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.200+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d010af10 con 0x7f00d0102430 2024-02-28T22:59:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.201+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f00c0007ad0 con 0x7f00d0102430 2024-02-28T22:59:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.202+0000 7f00c7400700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 0x7f00b406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.202+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f00c008a6e0 con 0x7f00d0102430 2024-02-28T22:59:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.202+0000 7f00ce000700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 0x7f00b406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.203+0000 7f00ce000700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 0x7f00b406eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f00d01991a0 tx=0x7f00bc00b580 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:59:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.207+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f00c0055c60 con 0x7f00d0102430 2024-02-28T22:59:14.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:14.440+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f00d0108020 con 0x7f00b406c600 2024-02-28T22:59:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:14 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:59:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:14 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:59:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:14 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:14 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:59:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:14 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:59:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:14 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:15 smithi057 ceph-mon[29096]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:59:15.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:15 smithi057 ceph-mon[29096]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:15 smithi084 ceph-mon[34826]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:59:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:15 smithi084 ceph-mon[34826]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:18 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi057 2024-02-28T22:59:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:18 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi057 to 11847M 2024-02-28T22:59:18.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:18 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi057 2024-02-28T22:59:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:18 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi057 to 11847M 2024-02-28T22:59:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:19 smithi057 ceph-mon[29096]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:19 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4059612881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6"}]: dispatch 2024-02-28T22:59:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:19 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4059612881' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6"}]': finished 2024-02-28T22:59:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:19 smithi057 ceph-mon[29096]: osdmap e11: 2 total, 1 up, 2 in 2024-02-28T22:59:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:19 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:19 smithi084 ceph-mon[34826]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:19 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/4059612881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6"}]: dispatch 2024-02-28T22:59:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:19 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/4059612881' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6"}]': finished 2024-02-28T22:59:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:19 smithi084 ceph-mon[34826]: osdmap e11: 2 total, 1 up, 2 in 2024-02-28T22:59:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:19 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:21 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2843529797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:59:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:21 smithi057 ceph-mon[29096]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:21 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2843529797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:59:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:21 smithi084 ceph-mon[34826]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:22.925 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:23.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:25 smithi057 ceph-mon[29096]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:25 smithi084 ceph-mon[34826]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:27.736 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:27 smithi057 ceph-mon[29096]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:27 smithi084 ceph-mon[34826]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:29 smithi057 ceph-mon[29096]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:29 smithi084 ceph-mon[34826]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:30.798 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T22:59:30.798 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:30.798 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:30 smithi057 ceph-mon[29096]: Deploying daemon osd.1 on smithi057 2024-02-28T22:59:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T22:59:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:30 smithi084 ceph-mon[34826]: Deploying daemon osd.1 on smithi057 2024-02-28T22:59:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:31 smithi057 ceph-mon[29096]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:31.981 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:31 smithi084 ceph-mon[34826]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:33.962 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:33 smithi057 ceph-mon[29096]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:33.981 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:33 smithi084 ceph-mon[34826]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:36.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:35 smithi057 ceph-mon[29096]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:36.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:35 smithi084 ceph-mon[34826]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:37.642 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:37 smithi057 ceph-mon[29096]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:37 smithi084 ceph-mon[34826]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:39.055 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:39.055 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:39.055 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:39.055 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:38 smithi057 ceph-mon[29096]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:39.214 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:39.214 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:39.214 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:39.214 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:38 smithi084 ceph-mon[34826]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:41.920 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:41 smithi057 ceph-mon[29096]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:43.498 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:41 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:43.498 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:41 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:43.499 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 1 on host 'smithi057' 2024-02-28T22:59:43.499 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:41 smithi084 ceph-mon[34826]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:43.500 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:41 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:43.500 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:41 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:43.500 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:42 smithi084 ceph-mon[34826]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.995+0000 7f00c7400700 1 -- 172.21.15.57:0/2072558764 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f00d0108020 con 0x7f00b406c600 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 msgr2=0x7f00b406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 0x7f00b406eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f00d01991a0 tx=0x7f00bc00b580 comp rx=0 tx=0).stop 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 msgr2=0x7f00d0197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c8a0 tx=0x7f00c000cc60 comp rx=0 tx=0).stop 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 shutdown_connections 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f00b406c600 0x7f00b406eab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00d0102430 0x7f00d0197c90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 --2- 172.21.15.57:0/2072558764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d01036d0 0x7f00d01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:42.999+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 >> 172.21.15.57:0/2072558764 conn(0x7f00d00fda80 msgr2=0x7f00d0106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:43.000+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 shutdown_connections 2024-02-28T22:59:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:43.000+0000 7f00d5a00700 1 -- 172.21.15.57:0/2072558764 wait complete. 2024-02-28T22:59:43.502 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:42 smithi057 ceph-mon[29096]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-28T22:59:43.791 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:43.791 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-28T22:59:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: osdmap e12: 2 total, 1 up, 2 in 2024-02-28T22:59:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T22:59:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:44.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: osdmap e12: 2 total, 1 up, 2 in 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:43 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:44.510 DEBUG:teuthology.orchestra.run.smithi057:osd.1> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.1.service 2024-02-28T22:59:44.516 INFO:tasks.cephadm:Deploying osd.2 on smithi057 with /dev/vg_nvme/lv_2... 2024-02-28T22:59:44.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-28T22:59:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:59:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:44 smithi057 ceph-mon[29096]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T22:59:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:44 smithi057 ceph-mon[29096]: osdmap e13: 2 total, 1 up, 2 in 2024-02-28T22:59:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:44 smithi057 ceph-mon[29096]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:44 smithi084 ceph-mon[34826]: from='osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T22:59:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:44 smithi084 ceph-mon[34826]: osdmap e13: 2 total, 1 up, 2 in 2024-02-28T22:59:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:44 smithi084 ceph-mon[34826]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-28T22:59:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:45 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T22:59:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:45 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T22:59:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:45 smithi057 ceph-mon[29096]: osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] boot 2024-02-28T22:59:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:45 smithi057 ceph-mon[29096]: osdmap e14: 2 total, 2 up, 2 in 2024-02-28T22:59:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:45 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:45 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T22:59:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:45 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T22:59:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:45 smithi084 ceph-mon[34826]: osd.1 [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] boot 2024-02-28T22:59:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:45 smithi084 ceph-mon[34826]: osdmap e14: 2 total, 2 up, 2 in 2024-02-28T22:59:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:45 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T22:59:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:46 smithi057 ceph-mon[29096]: osdmap e15: 2 total, 2 up, 2 in 2024-02-28T22:59:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:46 smithi057 ceph-mon[29096]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:47.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:46 smithi084 ceph-mon[34826]: osdmap e15: 2 total, 2 up, 2 in 2024-02-28T22:59:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:46 smithi084 ceph-mon[34826]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-28T22:59:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-28T22:59:48.402 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-02-28T22:59:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-02-28T22:59:48.403 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0374566 s, 280 MB/s 2024-02-28T22:59:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-02-28T22:59:49.173 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2024-02-28T22:59:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:49 smithi057 ceph-mon[29096]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:49 smithi084 ceph-mon[34826]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:50.584 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T22:59:51.200 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:50 smithi057 ceph-mon[29096]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:51.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:51 smithi084 ceph-mon[34826]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:52.804 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 -- 172.21.15.57:0/4049862130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 msgr2=0x7fc864073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 --2- 172.21.15.57:0/4049862130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864073c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc850009940 tx=0x7fc850009c50 comp rx=0 tx=0).stop 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 -- 172.21.15.57:0/4049862130 shutdown_connections 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 --2- 172.21.15.57:0/4049862130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864073c20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 --2- 172.21.15.57:0/4049862130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc8640731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 -- 172.21.15.57:0/4049862130 >> 172.21.15.57:0/4049862130 conn(0x7fc8640fb7f0 msgr2=0x7fc8640fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 -- 172.21.15.57:0/4049862130 shutdown_connections 2024-02-28T22:59:53.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.133+0000 7fc86a600700 1 -- 172.21.15.57:0/4049862130 wait complete. 2024-02-28T22:59:53.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.134+0000 7fc86a600700 1 Processor -- start 2024-02-28T22:59:53.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc86a600700 1 -- start start 2024-02-28T22:59:53.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc86a600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:53.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc86a600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc864198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:53.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc86a600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864198700 con 0x7fc8640737b0 2024-02-28T22:59:53.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc86a600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864198870 con 0x7fc864074d80 2024-02-28T22:59:53.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc863400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:53.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc863400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57422/0 (socket says 172.21.15.57:57422) 2024-02-28T22:59:53.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.135+0000 7fc863400700 1 -- 172.21.15.57:0/2828875002 learned_addr learned my addr 172.21.15.57:0/2828875002 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc862a00700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc864198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc863400700 1 -- 172.21.15.57:0/2828875002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 msgr2=0x7fc864198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc863400700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc864198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc863400700 1 -- 172.21.15.57:0/2828875002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500095f0 con 0x7fc8640737b0 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc862a00700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc864198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc863400700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc85400c960 tx=0x7fc85400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc854007a30 con 0x7fc8640737b0 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc854007b90 con 0x7fc8640737b0 2024-02-28T22:59:53.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc854018630 con 0x7fc8640737b0 2024-02-28T22:59:53.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.136+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86419d2c0 con 0x7fc8640737b0 2024-02-28T22:59:53.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.137+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86419d810 con 0x7fc8640737b0 2024-02-28T22:59:53.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.138+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86410af80 con 0x7fc8640737b0 2024-02-28T22:59:53.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.139+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7fc854007d00 con 0x7fc8640737b0 2024-02-28T22:59:53.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.140+0000 7fc85be00700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 0x7fc84806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T22:59:53.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.140+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fc85408acc0 con 0x7fc8640737b0 2024-02-28T22:59:53.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.140+0000 7fc862a00700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 0x7fc84806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T22:59:53.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.141+0000 7fc862a00700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 0x7fc84806eac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc850005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T22:59:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.145+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc854091080 con 0x7fc8640737b0 2024-02-28T22:59:53.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T22:59:53.376+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc864107740 con 0x7fc84806c610 2024-02-28T22:59:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.051 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T22:59:54.052 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi057 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi057 to 5923M 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:54.900 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T22:59:54.900 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi057 2024-02-28T22:59:54.900 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi057 to 5923M 2024-02-28T22:59:54.901 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T22:59:54.901 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T22:59:54.901 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:54.901 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T22:59:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:55 smithi057 ceph-mon[29096]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:55.944 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:55 smithi084 ceph-mon[34826]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:57.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:57.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:57.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:57.746 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/945482018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4e7051f-4c2a-482b-adef-90640ca366bf"}]: dispatch 2024-02-28T22:59:57.746 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/945482018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4e7051f-4c2a-482b-adef-90640ca366bf"}]': finished 2024-02-28T22:59:57.746 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: osdmap e16: 3 total, 2 up, 3 in 2024-02-28T22:59:57.746 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T22:59:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T22:59:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T22:59:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/945482018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4e7051f-4c2a-482b-adef-90640ca366bf"}]: dispatch 2024-02-28T22:59:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/945482018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4e7051f-4c2a-482b-adef-90640ca366bf"}]': finished 2024-02-28T22:59:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: osdmap e16: 3 total, 2 up, 3 in 2024-02-28T22:59:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T22:59:59.061 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:58 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2461055587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:59:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:58 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2461055587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T22:59:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 22:59:59 smithi057 ceph-mon[29096]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 22:59:59 smithi084 ceph-mon[34826]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:01.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:00 smithi057 ceph-mon[29096]: overall HEALTH_OK 2024-02-28T23:00:01.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:00 smithi057 ceph-mon[29096]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:01.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:00 smithi084 ceph-mon[34826]: overall HEALTH_OK 2024-02-28T23:00:01.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:00 smithi084 ceph-mon[34826]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:02.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:02.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:02.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:03 smithi057 ceph-mon[29096]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:03 smithi084 ceph-mon[34826]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:05 smithi057 ceph-mon[29096]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:05.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:05 smithi084 ceph-mon[34826]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:07 smithi057 ceph-mon[29096]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:00:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:07.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:07 smithi084 ceph-mon[34826]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:00:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:08.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:08 smithi057 ceph-mon[29096]: Deploying daemon osd.2 on smithi057 2024-02-28T23:00:08.676 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:08 smithi084 ceph-mon[34826]: Deploying daemon osd.2 on smithi057 2024-02-28T23:00:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:09 smithi057 ceph-mon[29096]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:09 smithi084 ceph-mon[34826]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:11 smithi057 ceph-mon[29096]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:11.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:11 smithi084 ceph-mon[34826]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:13 smithi057 ceph-mon[29096]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:13 smithi084 ceph-mon[34826]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:15.722 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:15 smithi057 ceph-mon[29096]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:15 smithi084 ceph-mon[34826]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:16.814 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:16.814 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:16.814 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:17 smithi057 ceph-mon[29096]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:17.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:17 smithi084 ceph-mon[34826]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:19 smithi057 ceph-mon[29096]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:19.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:19 smithi084 ceph-mon[34826]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:20.774 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:20 smithi057 ceph-mon[29096]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-28T23:00:20.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:20 smithi084 ceph-mon[34826]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-28T23:00:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.868+0000 7fc85be00700 1 -- 172.21.15.57:0/2828875002 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc864107740 con 0x7fc84806c610 2024-02-28T23:00:20.869 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 2 on host 'smithi057' 2024-02-28T23:00:20.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 msgr2=0x7fc84806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:20.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 0x7fc84806eac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc850005fb0 comp rx=0 tx=0).stop 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 msgr2=0x7fc864197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc85400c960 tx=0x7fc85400cd20 comp rx=0 tx=0).stop 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 shutdown_connections 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc84806c610 0x7fc84806eac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8640737b0 0x7fc864197c30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 --2- 172.21.15.57:0/2828875002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864074d80 0x7fc864198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 >> 172.21.15.57:0/2828875002 conn(0x7fc8640fb7f0 msgr2=0x7fc864106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:00:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 shutdown_connections 2024-02-28T23:00:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:00:20.872+0000 7fc86a600700 1 -- 172.21.15.57:0/2828875002 wait complete. 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: osdmap e17: 3 total, 2 up, 3 in 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:21.938 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:21 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:21.987 DEBUG:teuthology.orchestra.run.smithi057:osd.2> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.2.service 2024-02-28T23:00:21.995 INFO:tasks.cephadm:Deploying osd.3 on smithi084 with /dev/vg_nvme/lv_4... 2024-02-28T23:00:21.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-28T23:00:22.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: osdmap e17: 3 total, 2 up, 3 in 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:21 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:22.385 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:00:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:22 smithi057 ceph-mon[29096]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T23:00:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:22 smithi057 ceph-mon[29096]: osdmap e18: 3 total, 2 up, 3 in 2024-02-28T23:00:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:22 smithi084 ceph-mon[34826]: from='osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-02-28T23:00:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:22 smithi084 ceph-mon[34826]: osdmap e18: 3 total, 2 up, 3 in 2024-02-28T23:00:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] boot 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: osdmap e19: 3 total, 3 up, 3 in 2024-02-28T23:00:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T23:00:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T23:00:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-28T23:00:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: osd.2 [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] boot 2024-02-28T23:00:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: osdmap e19: 3 total, 3 up, 3 in 2024-02-28T23:00:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:00:24.803 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-28T23:00:24.803 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-28T23:00:24.803 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-02-28T23:00:24.803 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-02-28T23:00:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0360725 s, 291 MB/s 2024-02-28T23:00:24.804 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 2024-02-28T23:00:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-02-28T23:00:25.432 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:25 smithi084 ceph-mon[34826]: osdmap e20: 3 total, 3 up, 3 in 2024-02-28T23:00:25.432 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:25 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:25.432 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:25 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:25.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-02-28T23:00:25.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:25 smithi057 ceph-mon[29096]: osdmap e20: 3 total, 3 up, 3 in 2024-02-28T23:00:25.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:25 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:25 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:25.862 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:00:26.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:26 smithi084 ceph-mon[34826]: pgmap v99: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:26.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-28T23:00:26.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-28T23:00:26.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:26 smithi084 ceph-mon[34826]: osdmap e21: 3 total, 3 up, 3 in 2024-02-28T23:00:26.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:26 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-28T23:00:26.532 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:26 smithi057 ceph-mon[29096]: pgmap v99: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:26.532 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-28T23:00:26.532 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-28T23:00:26.532 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:26 smithi057 ceph-mon[29096]: osdmap e21: 3 total, 3 up, 3 in 2024-02-28T23:00:26.532 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:26 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-28T23:00:27.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:27 smithi084 ceph-mon[34826]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:27.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:27 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-28T23:00:27.783 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:27 smithi084 ceph-mon[34826]: osdmap e22: 3 total, 3 up, 3 in 2024-02-28T23:00:27.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:27 smithi057 ceph-mon[29096]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:27.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:27 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-28T23:00:27.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:27 smithi057 ceph-mon[29096]: osdmap e22: 3 total, 3 up, 3 in 2024-02-28T23:00:27.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:27 smithi057 sudo[57014]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-28T23:00:27.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:27 smithi057 sudo[57014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:27.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:27 smithi057 sudo[57014]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:27.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:27 smithi057 sudo[57014]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:27.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 -- 172.21.15.84:0/2600810881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f601036d0 msgr2=0x7f3f60103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 --2- 172.21.15.84:0/2600810881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f601036d0 0x7f3f60103b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009910 tx=0x7f3f4c009c20 comp rx=0 tx=0).stop 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 -- 172.21.15.84:0/2600810881 shutdown_connections 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 --2- 172.21.15.84:0/2600810881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f601036d0 0x7f3f60103b40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 --2- 172.21.15.84:0/2600810881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60102430 0x7f3f60102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 -- 172.21.15.84:0/2600810881 >> 172.21.15.84:0/2600810881 conn(0x7f3f600fda80 msgr2=0x7f3f600ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 -- 172.21.15.84:0/2600810881 shutdown_connections 2024-02-28T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.953+0000 7f3f68200700 1 -- 172.21.15.84:0/2600810881 wait complete. 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.954+0000 7f3f68200700 1 Processor -- start 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.954+0000 7f3f68200700 1 -- start start 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f68200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f68200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f601036d0 0x7f3f601981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f68200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601987d0 con 0x7f3f601036d0 2024-02-28T23:00:27.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f68200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60198940 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39598/0 (socket says 172.21.15.84:39598) 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 -- 172.21.15.84:0/2470250453 learned_addr learned my addr 172.21.15.84:0/2470250453 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f64e00700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f601036d0 0x7f3f601981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 -- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f601036d0 msgr2=0x7f3f601981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f601036d0 0x7f3f601981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.955+0000 7f3f65800700 1 -- 172.21.15.84:0/2470250453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0095f0 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f65800700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d8d0 tx=0x7f3f5400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f54009940 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f54010460 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5400f5f0 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6019d3d0 con 0x7f3f60102430 2024-02-28T23:00:27.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.956+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6019d920 con 0x7f3f60102430 2024-02-28T23:00:27.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.957+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6010af10 con 0x7f3f60102430 2024-02-28T23:00:27.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.959+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f3f540105d0 con 0x7f3f60102430 2024-02-28T23:00:27.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.960+0000 7f3f5e000700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 0x7f3f4806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:00:27.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.960+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f3f5408ae60 con 0x7f3f60102430 2024-02-28T23:00:27.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.960+0000 7f3f64e00700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 0x7f3f4806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:00:27.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.961+0000 7f3f64e00700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 0x7f3f4806eac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f601991a0 tx=0x7f3f4c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:00:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:27.967+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3f54055ee0 con 0x7f3f60102430 2024-02-28T23:00:28.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:28.192+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f60108020 con 0x7f3f4806c610 2024-02-28T23:00:28.439 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57038]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-28T23:00:28.439 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:28.439 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57038]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:28.439 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57038]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:28 smithi057 ceph-mon[29096]: osdmap e23: 3 total, 3 up, 3 in 2024-02-28T23:00:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:00:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:00:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:28.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:28 smithi084 ceph-mon[34826]: osdmap e23: 3 total, 3 up, 3 in 2024-02-28T23:00:28.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:00:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:00:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:29.139 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57055]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-28T23:00:29.139 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57055]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:29.139 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57055]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:29.139 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:28 smithi057 sudo[57055]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:29.579+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3f540562b0 con 0x7f3f60102430 2024-02-28T23:00:29.810 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:29 smithi084 ceph-mon[34826]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:00:29.810 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:29 smithi084 ceph-mon[34826]: pgmap v104: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:29 smithi057 ceph-mon[29096]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:00:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:29 smithi057 ceph-mon[29096]: pgmap v104: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:29.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:29 smithi057 sudo[57464]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-28T23:00:29.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:29 smithi057 sudo[57464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:29.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:29 smithi057 sudo[57464]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:29.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:00:29 smithi057 sudo[57464]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:30.318 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57467]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-28T23:00:30.318 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57467]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:30.318 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57467]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:30.319 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57467]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:30.668 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: mgrmap e19: smithi057.zrrldj(active, since 3m), standbys: smithi084.fhkzis 2024-02-28T23:00:30.668 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.668 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.668 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.669 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:30.847 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57568]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-28T23:00:30.847 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:30.847 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57568]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:30.847 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:00:30 smithi057 sudo[57568]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:30.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: mgrmap e19: smithi057.zrrldj(active, since 3m), standbys: smithi084.fhkzis 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:30.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:31.528 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57657]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-28T23:00:31.528 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:31.528 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57657]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:31.528 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57657]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57677]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-28T23:00:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57677]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:31 smithi057 sudo[57677]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:32.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38654]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-28T23:00:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38654]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38654]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38654]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi057 2024-02-28T23:00:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi057 to 3949M 2024-02-28T23:00:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: pgmap v105: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/1993095453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]': finished 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: osdmap e24: 4 total, 3 up, 4 in 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:00:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:32 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38657]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38657]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 sudo[38657]: pam_unix(sudo:session): session closed for user root 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi057 2024-02-28T23:00:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi057 to 3949M 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: pgmap v105: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/1993095453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]: dispatch 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]: dispatch 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14bf7f3a-d281-4d38-9e96-1f7a76d3d526"}]': finished 2024-02-28T23:00:33.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: osdmap e24: 4 total, 3 up, 4 in 2024-02-28T23:00:33.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:00:33.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:00:33.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:32 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:33 smithi057 ceph-mon[29096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-28T23:00:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:33 smithi057 ceph-mon[29096]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:33 smithi057 ceph-mon[29096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-28T23:00:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:33 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:33 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/2286409628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:00:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:34 smithi084 ceph-mon[34826]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-28T23:00:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:34 smithi084 ceph-mon[34826]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:34 smithi084 ceph-mon[34826]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-28T23:00:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:34 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:34 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/2286409628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:00:35.273 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:35 smithi057 ceph-mon[29096]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:35.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:35 smithi084 ceph-mon[34826]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:37.291 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.291 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.291 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:37.291 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:37.292 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.292 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:36 smithi084 ceph-mon[34826]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 168 KiB/s wr, 11 op/s 2024-02-28T23:00:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:36 smithi057 ceph-mon[29096]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 168 KiB/s wr, 11 op/s 2024-02-28T23:00:38.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:38.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:39.151 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:38 smithi084 ceph-mon[34826]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 117 KiB/s wr, 8 op/s 2024-02-28T23:00:39.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:38 smithi057 ceph-mon[29096]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 117 KiB/s wr, 8 op/s 2024-02-28T23:00:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:41 smithi084 ceph-mon[34826]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 90 KiB/s wr, 6 op/s 2024-02-28T23:00:41.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:41 smithi057 ceph-mon[29096]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 90 KiB/s wr, 6 op/s 2024-02-28T23:00:42.501 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:42 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:00:42.502 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:42 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:42.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:42 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:00:42.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:42 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:43.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:43 smithi084 ceph-mon[34826]: Deploying daemon osd.3 on smithi084 2024-02-28T23:00:43.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:43 smithi084 ceph-mon[34826]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-02-28T23:00:43.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:43 smithi057 ceph-mon[29096]: Deploying daemon osd.3 on smithi084 2024-02-28T23:00:43.534 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:43 smithi057 ceph-mon[29096]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-02-28T23:00:45.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:45 smithi057 ceph-mon[29096]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-28T23:00:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:45 smithi084 ceph-mon[34826]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-28T23:00:47.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:47 smithi084 ceph-mon[34826]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-28T23:00:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:47 smithi057 ceph-mon[29096]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-28T23:00:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:49 smithi084 ceph-mon[34826]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:49 smithi057 ceph-mon[29096]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:51 smithi057 ceph-mon[29096]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:51.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:51 smithi084 ceph-mon[34826]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:52.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:00:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:53 smithi084 ceph-mon[34826]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:53 smithi084 ceph-mon[34826]: from='osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:00:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:53 smithi084 ceph-mon[34826]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:00:53.922 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:53 smithi057 ceph-mon[29096]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:53.923 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:53 smithi057 ceph-mon[29096]: from='osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:00:53.923 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:53 smithi057 ceph-mon[29096]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:00:54.093 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 3 on host 'smithi084' 2024-02-28T23:00:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.093+0000 7f3f5e000700 1 -- 172.21.15.84:0/2470250453 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3f60108020 con 0x7f3f4806c610 2024-02-28T23:00:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 msgr2=0x7f3f4806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 0x7f3f4806eac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f601991a0 tx=0x7f3f4c00b540 comp rx=0 tx=0).stop 2024-02-28T23:00:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 msgr2=0x7f3f60197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:00:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d8d0 tx=0x7f3f5400dc90 comp rx=0 tx=0).stop 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 shutdown_connections 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3f4806c610 0x7f3f4806eac0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f60102430 0x7f3f60197c90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 --2- 172.21.15.84:0/2470250453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f601036d0 0x7f3f601981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 >> 172.21.15.84:0/2470250453 conn(0x7f3f600fda80 msgr2=0x7f3f60106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.097+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 shutdown_connections 2024-02-28T23:00:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:00:54.098+0000 7f3f68200700 1 -- 172.21.15.84:0/2470250453 wait complete. 2024-02-28T23:00:54.732 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: osdmap e25: 4 total, 3 up, 4 in 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:54.733 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:54.776 DEBUG:teuthology.orchestra.run.smithi084:osd.3> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.3.service 2024-02-28T23:00:54.780 INFO:tasks.cephadm:Deploying osd.4 on smithi084 with /dev/vg_nvme/lv_3... 2024-02-28T23:00:54.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-28T23:00:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: osdmap e25: 4 total, 3 up, 4 in 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:56.014 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:56.014 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:00:56.014 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: osdmap e26: 4 total, 3 up, 4 in 2024-02-28T23:00:56.014 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.015 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.015 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] boot 2024-02-28T23:00:56.015 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: osdmap e27: 4 total, 4 up, 4 in 2024-02-28T23:00:56.015 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:55 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-28T23:00:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:00:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: osdmap e26: 4 total, 3 up, 4 in 2024-02-28T23:00:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: osd.3 [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] boot 2024-02-28T23:00:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: osdmap e27: 4 total, 4 up, 4 in 2024-02-28T23:00:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:55 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:00:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:00:57.421 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:57 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T23:00:57.421 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:57 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T23:00:57.421 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:57 smithi084 ceph-mon[34826]: osdmap e28: 4 total, 4 up, 4 in 2024-02-28T23:00:57.421 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:57 smithi084 ceph-mon[34826]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:00:57.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:57 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T23:00:57.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:57 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T23:00:57.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:57 smithi057 ceph-mon[29096]: osdmap e28: 4 total, 4 up, 4 in 2024-02-28T23:00:57.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:57 smithi057 ceph-mon[29096]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:00:58.551 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:58 smithi084 ceph-mon[34826]: osdmap e29: 4 total, 4 up, 4 in 2024-02-28T23:00:58.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:58 smithi057 ceph-mon[29096]: osdmap e29: 4 total, 4 up, 4 in 2024-02-28T23:00:59.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: osdmap e30: 4 total, 4 up, 4 in 2024-02-28T23:00:59.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:00:59.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi084 2024-02-28T23:00:59.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi084 to 14915M 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:00:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: osdmap e30: 4 total, 4 up, 4 in 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi084 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi084 to 14915M 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:00:59.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:00:59.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:00:59.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:00:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-28T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-28T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-02-28T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-02-28T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368377 s, 285 MB/s 2024-02-28T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 2024-02-28T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-02-28T23:01:00.474 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:00 smithi084 ceph-mon[34826]: osdmap e31: 4 total, 4 up, 4 in 2024-02-28T23:01:00.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-02-28T23:01:00.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:00 smithi057 ceph-mon[29096]: osdmap e31: 4 total, 4 up, 4 in 2024-02-28T23:01:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:01:01.311 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:01 smithi084 ceph-mon[34826]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:01 smithi057 ceph-mon[29096]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 -- 172.21.15.84:0/913081387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 msgr2=0x7f9530102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 --2- 172.21.15.84:0/913081387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530102ef0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9520009910 tx=0x7f9520009c20 comp rx=0 tx=0).stop 2024-02-28T23:01:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 -- 172.21.15.84:0/913081387 shutdown_connections 2024-02-28T23:01:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 --2- 172.21.15.84:0/913081387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 0x7f9530105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 --2- 172.21.15.84:0/913081387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530102ef0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:02.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 -- 172.21.15.84:0/913081387 >> 172.21.15.84:0/913081387 conn(0x7f95300fa4d0 msgr2=0x7f95300fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:01:02.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.896+0000 7f9536200700 1 -- 172.21.15.84:0/913081387 shutdown_connections 2024-02-28T23:01:02.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.897+0000 7f9536200700 1 -- 172.21.15.84:0/913081387 wait complete. 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 Processor -- start 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 -- start start 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 0x7f95300feac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300ff000 con 0x7f9530103430 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9536200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300ff140 con 0x7f9530100b10 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9534e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9534e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38556/0 (socket says 172.21.15.84:38556) 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.898+0000 7f9534e00700 1 -- 172.21.15.84:0/3895481488 learned_addr learned my addr 172.21.15.84:0/3895481488 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9527e00700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 0x7f95300feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9534e00700 1 -- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 msgr2=0x7f95300feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9534e00700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 0x7f95300feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9534e00700 1 -- 172.21.15.84:0/3895481488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95200095f0 con 0x7f9530100b10 2024-02-28T23:01:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9534e00700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9520005b40 tx=0x7f952000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952001d070 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f952000f460 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95300ff3c0 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.899+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9520021640 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.900+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95300ff8b0 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.901+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953018fa40 con 0x7f9530100b10 2024-02-28T23:01:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.902+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f952000fa90 con 0x7f9530100b10 2024-02-28T23:01:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.903+0000 7f952e000700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 0x7f951406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.903+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f952008c630 con 0x7f9530100b10 2024-02-28T23:01:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.904+0000 7f9527e00700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 0x7f951406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.905+0000 7f9527e00700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 0x7f951406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9518005950 tx=0x7f95180058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:01:02.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:02.909+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f952005ada0 con 0x7f9530100b10 2024-02-28T23:01:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:03.131+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9530061190 con 0x7f951406c680 2024-02-28T23:01:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:03 smithi057 ceph-mon[29096]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-28T23:01:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:01:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:01:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:03 smithi084 ceph-mon[34826]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-28T23:01:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:01:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:01:03.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:04.969 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:04 smithi084 ceph-mon[34826]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:01:05.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:04 smithi057 ceph-mon[29096]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:01:05.619 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:05 smithi084 ceph-mon[34826]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-28T23:01:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:05 smithi057 ceph-mon[29096]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-28T23:01:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:06 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/1682376288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]: dispatch 2024-02-28T23:01:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:06 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]: dispatch 2024-02-28T23:01:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:06 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]': finished 2024-02-28T23:01:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:06 smithi057 ceph-mon[29096]: osdmap e32: 5 total, 4 up, 5 in 2024-02-28T23:01:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:06 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:06 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/1682376288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]: dispatch 2024-02-28T23:01:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:06 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]: dispatch 2024-02-28T23:01:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:06 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fbb5108-2bfb-45d1-b31f-affe0d662fb8"}]': finished 2024-02-28T23:01:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:06 smithi084 ceph-mon[34826]: osdmap e32: 5 total, 4 up, 5 in 2024-02-28T23:01:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:06 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:07 smithi057 ceph-mon[29096]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-28T23:01:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:07 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3943951563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:01:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:08.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:07 smithi084 ceph-mon[34826]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-28T23:01:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:07 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/3943951563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:01:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:09 smithi057 ceph-mon[29096]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-28T23:01:10.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:09 smithi084 ceph-mon[34826]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-28T23:01:11.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:11 smithi084 ceph-mon[34826]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-28T23:01:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:11 smithi057 ceph-mon[29096]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-28T23:01:13.720 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:13 smithi084 ceph-mon[34826]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:13 smithi057 ceph-mon[29096]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:15 smithi084 ceph-mon[34826]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:15 smithi057 ceph-mon[29096]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:16.681 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:01:16.681 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:16.681 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:16 smithi084 ceph-mon[34826]: Deploying daemon osd.4 on smithi084 2024-02-28T23:01:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:01:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:16 smithi057 ceph-mon[29096]: Deploying daemon osd.4 on smithi084 2024-02-28T23:01:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:17 smithi084 ceph-mon[34826]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:17 smithi057 ceph-mon[29096]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:19.792 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:19 smithi084 ceph-mon[34826]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:19 smithi057 ceph-mon[29096]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:21 smithi057 ceph-mon[29096]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:21 smithi084 ceph-mon[34826]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:22.730 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:23.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:23 smithi084 ceph-mon[34826]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:23 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:23 smithi057 ceph-mon[29096]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:23 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:25.912 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:25 smithi084 ceph-mon[34826]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:25 smithi057 ceph-mon[29096]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:27.795 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 4 on host 'smithi084' 2024-02-28T23:01:27.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.797+0000 7f952e000700 1 -- 172.21.15.84:0/3895481488 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9530061190 con 0x7f951406c680 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.799+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 msgr2=0x7f951406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.799+0000 7f9536200700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 0x7f951406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9518005950 tx=0x7f95180058e0 comp rx=0 tx=0).stop 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.799+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 msgr2=0x7f9530100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.799+0000 7f9536200700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9520005b40 tx=0x7f952000be20 comp rx=0 tx=0).stop 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 shutdown_connections 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f951406c680 0x7f951406eb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9530100b10 0x7f9530100470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 --2- 172.21.15.84:0/3895481488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9530103430 0x7f95300feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 >> 172.21.15.84:0/3895481488 conn(0x7f95300fa4d0 msgr2=0x7f95300fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:01:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 shutdown_connections 2024-02-28T23:01:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:27.800+0000 7f9536200700 1 -- 172.21.15.84:0/3895481488 wait complete. 2024-02-28T23:01:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:27 smithi057 ceph-mon[29096]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:27 smithi057 ceph-mon[29096]: from='osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:01:28.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:27 smithi057 ceph-mon[29096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:01:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:27 smithi084 ceph-mon[34826]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:27 smithi084 ceph-mon[34826]: from='osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:01:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:27 smithi084 ceph-mon[34826]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:01:28.526 DEBUG:teuthology.orchestra.run.smithi084:osd.4> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.4.service 2024-02-28T23:01:28.532 INFO:tasks.cephadm:Deploying osd.5 on smithi084 with /dev/vg_nvme/lv_2... 2024-02-28T23:01:28.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: osdmap e33: 5 total, 4 up, 5 in 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:28.830 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[45444]: 2024-02-28T23:01:28.630+0000 7f9e6da00700 -1 osd.4 0 waiting for initial osdmap 2024-02-28T23:01:28.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:01:28 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[45444]: 2024-02-28T23:01:28.640+0000 7f9e69200700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:01:28.924 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:01:29.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: osdmap e33: 5 total, 4 up, 5 in 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: osdmap e34: 5 total, 4 up, 5 in 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] boot 2024-02-28T23:01:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:29 smithi057 ceph-mon[29096]: osdmap e35: 5 total, 5 up, 5 in 2024-02-28T23:01:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-28T23:01:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:01:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: osdmap e34: 5 total, 4 up, 5 in 2024-02-28T23:01:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:30.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:30.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: osd.4 [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] boot 2024-02-28T23:01:30.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:29 smithi084 ceph-mon[34826]: osdmap e35: 5 total, 5 up, 5 in 2024-02-28T23:01:30.738 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:30 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T23:01:30.738 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:30 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T23:01:30.738 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:30 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:30 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T23:01:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:30 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T23:01:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:30 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0361256 s, 290 MB/s 2024-02-28T23:01:31.713 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-02-28T23:01:31.805 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:31 smithi084 ceph-mon[34826]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:31.805 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:31 smithi084 ceph-mon[34826]: osdmap e36: 5 total, 5 up, 5 in 2024-02-28T23:01:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:31 smithi057 ceph-mon[29096]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:31 smithi057 ceph-mon[29096]: osdmap e36: 5 total, 5 up, 5 in 2024-02-28T23:01:32.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-02-28T23:01:33.312 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:01:33.764 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:33 smithi084 ceph-mon[34826]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:33 smithi057 ceph-mon[29096]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:35.999 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:35.999 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:35.999 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:35.999 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.000 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:36.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:01:36.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:36.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 -- 172.21.15.84:0/3882721679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 msgr2=0x7fe8f4102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 --2- 172.21.15.84:0/3882721679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4102840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009910 tx=0x7fe8e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 -- 172.21.15.84:0/3882721679 shutdown_connections 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 --2- 172.21.15.84:0/3882721679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 0x7fe8f4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 --2- 172.21.15.84:0/3882721679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4102840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 -- 172.21.15.84:0/3882721679 >> 172.21.15.84:0/3882721679 conn(0x7fe8f40fda80 msgr2=0x7fe8f40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 -- 172.21.15.84:0/3882721679 shutdown_connections 2024-02-28T23:01:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.169+0000 7fe8f9800700 1 -- 172.21.15.84:0/3882721679 wait complete. 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.170+0000 7fe8f9800700 1 Processor -- start 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.170+0000 7fe8f9800700 1 -- start start 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f9800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f9800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 0x7fe8f41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f9800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41987d0 con 0x7fe8f41036d0 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f9800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4198940 con 0x7fe8f4102430 2024-02-28T23:01:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37498/0 (socket says 172.21.15.84:37498) 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 -- 172.21.15.84:0/1504805538 learned_addr learned my addr 172.21.15.84:0/1504805538 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8ea000700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 0x7fe8f41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 -- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 msgr2=0x7fe8f41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 0x7fe8f41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.171+0000 7fe8f2a00700 1 -- 172.21.15.84:0/1504805538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40095f0 con 0x7fe8f4102430 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8f2a00700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4005b40 tx=0x7fe8e400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e401d070 con 0x7fe8f4102430 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe8e400f460 con 0x7fe8f4102430 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f419d370 con 0x7fe8f4102430 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e4021640 con 0x7fe8f4102430 2024-02-28T23:01:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.172+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f419d860 con 0x7fe8f4102430 2024-02-28T23:01:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.173+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f410af10 con 0x7fe8f4102430 2024-02-28T23:01:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.174+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe8e400fa90 con 0x7fe8f4102430 2024-02-28T23:01:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.175+0000 7fe8ebe00700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 0x7fe8d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:01:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.175+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fe8e408c670 con 0x7fe8f4102430 2024-02-28T23:01:36.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.176+0000 7fe8ea000700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 0x7fe8d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:01:36.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.177+0000 7fe8ea000700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 0x7fe8d806eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005950 tx=0x7fe8dc0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:01:36.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.180+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe8e4057220 con 0x7fe8f4102430 2024-02-28T23:01:36.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:01:36.406+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8f4108020 con 0x7fe8d806c680 2024-02-28T23:01:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi084 2024-02-28T23:01:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi084 to 7457M 2024-02-28T23:01:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:01:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:01:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:01:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:36 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi084 2024-02-28T23:01:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi084 to 7457M 2024-02-28T23:01:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-28T23:01:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-28T23:01:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:01:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:36 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:37 smithi084 ceph-mon[34826]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:01:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:37 smithi084 ceph-mon[34826]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:37 smithi057 ceph-mon[29096]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:01:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:37 smithi057 ceph-mon[29096]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:39 smithi057 ceph-mon[29096]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:40.112 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:39 smithi084 ceph-mon[34826]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:40 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/3916020247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]: dispatch 2024-02-28T23:01:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:40 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]: dispatch 2024-02-28T23:01:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:40 smithi084 ceph-mon[34826]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]': finished 2024-02-28T23:01:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:40 smithi084 ceph-mon[34826]: osdmap e37: 6 total, 5 up, 6 in 2024-02-28T23:01:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:01:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:40 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/3916020247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]: dispatch 2024-02-28T23:01:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:40 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]: dispatch 2024-02-28T23:01:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:40 smithi057 ceph-mon[29096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bd03e97-0647-4187-b440-1e175d92b72e"}]': finished 2024-02-28T23:01:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:40 smithi057 ceph-mon[29096]: osdmap e37: 6 total, 5 up, 6 in 2024-02-28T23:01:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:01:41.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:41 smithi057 ceph-mon[29096]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:41.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:41 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/4128410512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:01:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:41 smithi084 ceph-mon[34826]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:41 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/4128410512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-28T23:01:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:43 smithi057 ceph-mon[29096]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:43 smithi084 ceph-mon[34826]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:45 smithi084 ceph-mon[34826]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:45 smithi057 ceph-mon[29096]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:47.971 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:47 smithi084 ceph-mon[34826]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:47 smithi057 ceph-mon[29096]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:49.996 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:49 smithi057 ceph-mon[29096]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:49.996 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:01:49.996 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:49 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:49 smithi084 ceph-mon[34826]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:01:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:49 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:01:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:50 smithi084 ceph-mon[34826]: Deploying daemon osd.5 on smithi084 2024-02-28T23:01:51.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:50 smithi057 ceph-mon[29096]: Deploying daemon osd.5 on smithi084 2024-02-28T23:01:51.995 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:51 smithi084 ceph-mon[34826]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:51 smithi057 ceph-mon[29096]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:53.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:01:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:53 smithi057 ceph-mon[29096]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:53 smithi084 ceph-mon[34826]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:55 smithi057 ceph-mon[29096]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:55 smithi084 ceph-mon[34826]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:57 smithi057 ceph-mon[29096]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:57 smithi084 ceph-mon[34826]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:01:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:01:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:01:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:01:59 smithi084 ceph-mon[34826]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:01:59 smithi057 ceph-mon[29096]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:01 smithi057 ceph-mon[29096]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:01.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:01 smithi084 ceph-mon[34826]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:02.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.045+0000 7fe8ebe00700 1 -- 172.21.15.84:0/1504805538 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe8f4108020 con 0x7fe8d806c680 2024-02-28T23:02:02.044 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 5 on host 'smithi084' 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 msgr2=0x7fe8d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 0x7fe8d806eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005950 tx=0x7fe8dc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 msgr2=0x7fe8f4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4005b40 tx=0x7fe8e400be20 comp rx=0 tx=0).stop 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 shutdown_connections 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe8d806c680 0x7fe8d806eb30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8f4102430 0x7fe8f4197c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 --2- 172.21.15.84:0/1504805538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f41036d0 0x7fe8f41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.048+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 >> 172.21.15.84:0/1504805538 conn(0x7fe8f40fda80 msgr2=0x7fe8f4106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.049+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 shutdown_connections 2024-02-28T23:02:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:02.049+0000 7fe8f9800700 1 -- 172.21.15.84:0/1504805538 wait complete. 2024-02-28T23:02:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:02 smithi057 ceph-mon[29096]: from='osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:02:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:02 smithi057 ceph-mon[29096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:02:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:02.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:02 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:02 smithi084 ceph-mon[34826]: from='osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:02:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:02 smithi084 ceph-mon[34826]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:02:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:02 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:02.863 DEBUG:teuthology.orchestra.run.smithi084:osd.5> sudo journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.5.service 2024-02-28T23:02:02.869 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-28T23:02:02.870 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-28T23:02:03.305 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: osdmap e38: 6 total, 5 up, 6 in 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:03.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[50238]: 2024-02-28T23:02:03.256+0000 7f9874200700 -1 osd.5 0 waiting for initial osdmap 2024-02-28T23:02:03.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:02:03 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[50238]: 2024-02-28T23:02:03.275+0000 7f986aa00700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: osdmap e38: 6 total, 5 up, 6 in 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:02:03.494 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-28T23:02:03.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:03.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:03 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:04.509 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:04 smithi057 ceph-mon[29096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:02:04.509 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:04 smithi057 ceph-mon[29096]: osdmap e39: 6 total, 5 up, 6 in 2024-02-28T23:02:04.510 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:04 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:04.510 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:04 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:04.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:04 smithi084 ceph-mon[34826]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-02-28T23:02:04.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:04 smithi084 ceph-mon[34826]: osdmap e39: 6 total, 5 up, 6 in 2024-02-28T23:02:04.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:04 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:04.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:04 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: purged_snaps scrub starts 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: purged_snaps scrub ok 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] boot 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: osdmap e40: 6 total, 6 up, 6 in 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:05 smithi084 ceph-mon[34826]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 -- 172.21.15.57:0/632964013 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 msgr2=0x7f6624107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 --2- 172.21.15.57:0/632964013 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f6624107e90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6610009910 tx=0x7f6610009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 -- 172.21.15.57:0/632964013 shutdown_connections 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 --2- 172.21.15.57:0/632964013 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f6624107e90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 --2- 172.21.15.57:0/632964013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 0x7f6624106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.488+0000 7f6629000700 1 -- 172.21.15.57:0/632964013 >> 172.21.15.57:0/632964013 conn(0x7f6624075970 msgr2=0x7f6624077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:05.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.489+0000 7f6629000700 1 -- 172.21.15.57:0/632964013 shutdown_connections 2024-02-28T23:02:05.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.489+0000 7f6629000700 1 -- 172.21.15.57:0/632964013 wait complete. 2024-02-28T23:02:05.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.490+0000 7f6629000700 1 Processor -- start 2024-02-28T23:02:05.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.490+0000 7f6629000700 1 -- start start 2024-02-28T23:02:05.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6629000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 0x7f662419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:05.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6629000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:05.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6629000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419cc40 con 0x7f6624107a40 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6629000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419cd80 con 0x7f6624106840 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56350/0 (socket says 172.21.15.57:56350) 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 -- 172.21.15.57:0/483443413 learned_addr learned my addr 172.21.15.57:0/483443413 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621e00700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 0x7f662419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 -- 172.21.15.57:0/483443413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 msgr2=0x7f662419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 0x7f662419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.491+0000 7f6621400700 1 -- 172.21.15.57:0/483443413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100095f0 con 0x7f6624107a40 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f6621400700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6610005b40 tx=0x7f661000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661001d070 con 0x7f6624107a40 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f661000f460 con 0x7f6624107a40 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6610021640 con 0x7f6624107a40 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66241a17d0 con 0x7f6624107a40 2024-02-28T23:02:05.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.492+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66241a1cc0 con 0x7f6624107a40 2024-02-28T23:02:05.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.494+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662410f390 con 0x7f6624107a40 2024-02-28T23:02:05.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.495+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f661000fa90 con 0x7f6624107a40 2024-02-28T23:02:05.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.496+0000 7f661aa00700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 0x7f660806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:05.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.496+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f661008c820 con 0x7f6624107a40 2024-02-28T23:02:05.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.497+0000 7f6621e00700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 0x7f660806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:05.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.497+0000 7f6621e00700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 0x7f660806eb80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6614005950 tx=0x7f66140058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:05.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.502+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f661005acb0 con 0x7f6624107a40 2024-02-28T23:02:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: purged_snaps scrub starts 2024-02-28T23:02:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: purged_snaps scrub ok 2024-02-28T23:02:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: osd.5 [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] boot 2024-02-28T23:02:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: osdmap e40: 6 total, 6 up, 6 in 2024-02-28T23:02:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:02:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:05 smithi057 ceph-mon[29096]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.726+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f662410f550 con 0x7f6624107a40 2024-02-28T23:02:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.727+0000 7f661aa00700 1 -- 172.21.15.57:0/483443413 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f661005a840 con 0x7f6624107a40 2024-02-28T23:02:05.725 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:02:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 msgr2=0x7f660806eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 0x7f660806eb80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6614005950 tx=0x7f66140058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 msgr2=0x7f662419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6610005b40 tx=0x7f661000be20 comp rx=0 tx=0).stop 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 shutdown_connections 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f660806c6d0 0x7f660806eb80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6624106840 0x7f662419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 --2- 172.21.15.57:0/483443413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6624107a40 0x7f662419c620 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.730+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 >> 172.21.15.57:0/483443413 conn(0x7f6624075970 msgr2=0x7f662410ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.731+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 shutdown_connections 2024-02-28T23:02:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:05.731+0000 7f6629000700 1 -- 172.21.15.57:0/483443413 wait complete. 2024-02-28T23:02:06.453 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:06 smithi057 ceph-mon[29096]: osdmap e41: 6 total, 6 up, 6 in 2024-02-28T23:02:06.453 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:06 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/483443413' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-28T23:02:06.517 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1709161324,"num_in_osds":6,"osd_in_since":1709161299,"num_remapped_pgs":0} 2024-02-28T23:02:06.517 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-28T23:02:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:06 smithi084 ceph-mon[34826]: osdmap e41: 6 total, 6 up, 6 in 2024-02-28T23:02:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:06 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/483443413' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-28T23:02:06.953 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:07.425 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:07 smithi057 ceph-mon[29096]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:07.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:07 smithi084 ceph-mon[34826]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:08.526 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: Detected new or changed devices on smithi084 2024-02-28T23:02:08.526 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.526 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi084 to 4971M 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:02:08.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi084 2024-02-28T23:02:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi084 to 4971M 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:02:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 -- 172.21.15.57:0/2042890981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a30100e80 msgr2=0x7f7a30104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 --2- 172.21.15.57:0/2042890981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a30100e80 0x7f7a30104e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009910 tx=0x7f7a18009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 -- 172.21.15.57:0/2042890981 shutdown_connections 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 --2- 172.21.15.57:0/2042890981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a30100e80 0x7f7a30104e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 --2- 172.21.15.57:0/2042890981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a301004e0 0x7f7a301008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 -- 172.21.15.57:0/2042890981 >> 172.21.15.57:0/2042890981 conn(0x7f7a300fbda0 msgr2=0x7f7a300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 -- 172.21.15.57:0/2042890981 shutdown_connections 2024-02-28T23:02:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.048+0000 7f7a35800700 1 -- 172.21.15.57:0/2042890981 wait complete. 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.049+0000 7f7a35800700 1 Processor -- start 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.049+0000 7f7a35800700 1 -- start start 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a35800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a35800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a30100e80 0x7f7a30198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a35800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30198d50 con 0x7f7a301004e0 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a35800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019cae0 con 0x7f7a30100e80 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56370/0 (socket says 172.21.15.57:56370) 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a2ea00700 1 -- 172.21.15.57:0/2951073254 learned_addr learned my addr 172.21.15.57:0/2951073254 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.050+0000 7f7a26000700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a30100e80 0x7f7a30198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a2ea00700 1 -- 172.21.15.57:0/2951073254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a30100e80 msgr2=0x7f7a30198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a2ea00700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a30100e80 0x7f7a30198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a2ea00700 1 -- 172.21.15.57:0/2951073254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180095f0 con 0x7f7a301004e0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a2ea00700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a200060b0 tx=0x7f7a2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a2000dda0 con 0x7f7a301004e0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a2000f040 con 0x7f7a301004e0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3019cdc0 con 0x7f7a301004e0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a20014790 con 0x7f7a301004e0 2024-02-28T23:02:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.051+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3019d310 con 0x7f7a301004e0 2024-02-28T23:02:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.053+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a301045d0 con 0x7f7a301004e0 2024-02-28T23:02:09.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.054+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7a200148f0 con 0x7f7a301004e0 2024-02-28T23:02:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.055+0000 7f7a27e00700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 0x7f7a1406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.055+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7a2008b0b0 con 0x7f7a301004e0 2024-02-28T23:02:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.055+0000 7f7a26000700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 0x7f7a1406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.056+0000 7f7a26000700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 0x7f7a1406eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a30199750 tx=0x7f7a1801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.060+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7a2004f720 con 0x7f7a301004e0 2024-02-28T23:02:09.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.283+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3004f2a0 con 0x7f7a301004e0 2024-02-28T23:02:09.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.284+0000 7f7a27e00700 1 -- 172.21.15.57:0/2951073254 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11286 (secure 0 0 0) 0x7f7a20055ae0 con 0x7f7a301004e0 2024-02-28T23:02:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:02:09.283 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":41,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:02:05.259124+0000","last_up_change":"2024-02-28T23:02:04.242665+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6803","nonce":119728824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6805","nonce":119728824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6809","nonce":119728824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6807","nonce":119728824}]},"public_addr":"172.21.15.57:6803/119728824","cluster_addr":"172.21.15.57:6805/119728824","heartbeat_back_addr":"172.21.15.57:6809/119728824","heartbeat_front_addr":"172.21.15.57:6807/119728824","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6811","nonce":313179796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6813","nonce":313179796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6817","nonce":313179796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6815","nonce":313179796}]},"public_addr":"172.21.15.57:6811/313179796","cluster_addr":"172.21.15.57:6813/313179796","heartbeat_back_addr":"172.21.15.57:6817/313179796","heartbeat_front_addr":"172.21.15.57:6815/313179796","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6819","nonce":2667584111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6821","nonce":2667584111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6825","nonce":2667584111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6823","nonce":2667584111}]},"public_addr":"172.21.15.57:6819/2667584111","cluster_addr":"172.21.15.57:6821/2667584111","heartbeat_back_addr":"172.21.15.57:6825/2667584111","heartbeat_front_addr":"172.21.15.57:6823/2667584111","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6801","nonce":3040593930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6803","nonce":3040593930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6807","nonce":3040593930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6805","nonce":3040593930}]},"public_addr":"172.21.15.84:6801/3040593930","cluster_addr":"172.21.15.84:6803/3040593930","heartbeat_back_addr":"172.21.15.84:6807/3040593930","heartbeat_front_addr":"172.21.15.84:6805/3040593930","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6809","nonce":3561819927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6811","nonce":3561819927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6815","nonce":3561819927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6813","nonce":3561819927}]},"public_addr":"172.21.15.84:6809/3561819927","cluster_addr":"172.21.15.84:6811/3561819927","heartbeat_back_addr":"172.21.15.84:6815/3561819927","heartbeat_front_addr":"172.21.15.84:6813/3561819927","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6817","nonce":3141452916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6819","nonce":3141452916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6823","nonce":3141452916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6821","nonce":3141452916}]},"public_addr":"172.21.15.84:6817/3141452916","cluster_addr":"172.21.15.84:6819/3141452916","heartbeat_back_addr":"172.21.15.84:6823/3141452916","heartbeat_front_addr":"172.21.15.84:6821/3141452916","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.287+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 msgr2=0x7f7a1406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.287+0000 7f7a35800700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 0x7f7a1406eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a30199750 tx=0x7f7a1801a040 comp rx=0 tx=0).stop 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.287+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 msgr2=0x7f7a30198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.287+0000 7f7a35800700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a200060b0 tx=0x7f7a2000d3b0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 shutdown_connections 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7a1406c680 0x7f7a1406eb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a301004e0 0x7f7a30198130 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 --2- 172.21.15.57:0/2951073254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a30100e80 0x7f7a30198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:09.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 >> 172.21.15.57:0/2951073254 conn(0x7f7a300fbda0 msgr2=0x7f7a300fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:09.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 shutdown_connections 2024-02-28T23:02:09.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:09.288+0000 7f7a35800700 1 -- 172.21.15.57:0/2951073254 wait complete. 2024-02-28T23:02:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:09 smithi057 ceph-mon[29096]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:09 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2951073254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:09.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:09 smithi084 ceph-mon[34826]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:09.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:09 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2951073254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:10.124 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-28T23:00:25.160671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-28T23:02:10.124 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-28T23:02:10.558 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:11 smithi057 ceph-mon[29096]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:11.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:11 smithi084 ceph-mon[34826]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:12.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 -- 172.21.15.57:0/231195390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc073a00 msgr2=0x7f1fdc110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/231195390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc073a00 0x7f1fdc110d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009910 tx=0x7f1fc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 -- 172.21.15.57:0/231195390 shutdown_connections 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/231195390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc073a00 0x7f1fdc110d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/231195390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 -- 172.21.15.57:0/231195390 >> 172.21.15.57:0/231195390 conn(0x7f1fdc0fbda0 msgr2=0x7f1fdc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 -- 172.21.15.57:0/231195390 shutdown_connections 2024-02-28T23:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.863+0000 7f1fe2a00700 1 -- 172.21.15.57:0/231195390 wait complete. 2024-02-28T23:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.864+0000 7f1fe2a00700 1 Processor -- start 2024-02-28T23:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.864+0000 7f1fe2a00700 1 -- start start 2024-02-28T23:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc073a00 0x7f1fdc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc19d050 con 0x7f1fdc0730f0 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc1a0de0 con 0x7f1fdc073a00 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56384/0 (socket says 172.21.15.57:56384) 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 -- 172.21.15.57:0/1229395418 learned_addr learned my addr 172.21.15.57:0/1229395418 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdb400700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc073a00 0x7f1fdc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 -- 172.21.15.57:0/1229395418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc073a00 msgr2=0x7f1fdc19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc073a00 0x7f1fdc19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:12.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 -- 172.21.15.57:0/1229395418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80095f0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fdbe00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00d8d0 tx=0x7f1fcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fcc009940 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1fcc010460 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fcc00f5f0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc1a10c0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.865+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc1a1610 con 0x7f1fdc0730f0 2024-02-28T23:02:12.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.866+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc10e4d0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.867+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1fcc0105d0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.868+0000 7f1fd8c00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 0x7f1fc006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.868+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1fcc08b5e0 con 0x7f1fdc0730f0 2024-02-28T23:02:12.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.869+0000 7f1fdb400700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 0x7f1fc006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:12.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.869+0000 7f1fdb400700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 0x7f1fc006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc19da50 tx=0x7f1fc801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:12.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:12.873+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1fcc055f60 con 0x7f1fdc0730f0 2024-02-28T23:02:13.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.081+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fdc1a1830 con 0x7f1fdc0730f0 2024-02-28T23:02:13.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.082+0000 7f1fd8c00700 1 -- 172.21.15.57:0/1229395418 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fcc059580 con 0x7f1fdc0730f0 2024-02-28T23:02:13.081 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-02-28T23:02:13.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 msgr2=0x7f1fc006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:13.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 0x7f1fc006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc19da50 tx=0x7f1fc801a040 comp rx=0 tx=0).stop 2024-02-28T23:02:13.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 msgr2=0x7f1fdc19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:13.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00d8d0 tx=0x7f1fcc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:02:13.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 shutdown_connections 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f1fc006c680 0x7f1fc006eb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fdc0730f0 0x7f1fdc19c430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 --2- 172.21.15.57:0/1229395418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fdc073a00 0x7f1fdc19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 >> 172.21.15.57:0/1229395418 conn(0x7f1fdc0fbda0 msgr2=0x7f1fdc102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 shutdown_connections 2024-02-28T23:02:13.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:13.085+0000 7f1fe2a00700 1 -- 172.21.15.57:0/1229395418 wait complete. 2024-02-28T23:02:13.807 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:13 smithi057 ceph-mon[29096]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:13.807 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:13 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1229395418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:02:13.858 INFO:tasks.cephadm:Setting up client nodes... 2024-02-28T23:02:13.859 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-28T23:02:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:13 smithi084 ceph-mon[34826]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:13 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1229395418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:02:14.286 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:15 smithi057 ceph-mon[29096]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:15 smithi084 ceph-mon[34826]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:16.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 -- 172.21.15.57:0/1533804547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c100350 msgr2=0x7f3c5c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:16.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 --2- 172.21.15.57:0/1533804547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c100350 0x7f3c5c1007a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009910 tx=0x7f3c48009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:16.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 -- 172.21.15.57:0/1533804547 shutdown_connections 2024-02-28T23:02:16.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 --2- 172.21.15.57:0/1533804547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c100350 0x7f3c5c1007a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 --2- 172.21.15.57:0/1533804547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 -- 172.21.15.57:0/1533804547 >> 172.21.15.57:0/1533804547 conn(0x7f3c5c0fa700 msgr2=0x7f3c5c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 -- 172.21.15.57:0/1533804547 shutdown_connections 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.387+0000 7f3c63800700 1 -- 172.21.15.57:0/1533804547 wait complete. 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.389+0000 7f3c63800700 1 Processor -- start 2024-02-28T23:02:16.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.389+0000 7f3c63800700 1 -- start start 2024-02-28T23:02:16.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c63800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:16.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c63800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c100350 0x7f3c5c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:16.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c63800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c101c70 con 0x7f3c5c0ff150 2024-02-28T23:02:16.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c63800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c101db0 con 0x7f3c5c100350 2024-02-28T23:02:16.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44782/0 (socket says 172.21.15.57:44782) 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 -- 172.21.15.57:0/1306708240 learned_addr learned my addr 172.21.15.57:0/1306708240 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c5be00700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c100350 0x7f3c5c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 -- 172.21.15.57:0/1306708240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c100350 msgr2=0x7f3c5c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c100350 0x7f3c5c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.390+0000 7f3c60e00700 1 -- 172.21.15.57:0/1306708240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480095f0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c60e00700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000d900 tx=0x7f3c5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c500041f0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c50004350 con 0x7f3c5c0ff150 2024-02-28T23:02:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c102090 con 0x7f3c5c0ff150 2024-02-28T23:02:16.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c50003da0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.391+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c19db00 con 0x7f3c5c0ff150 2024-02-28T23:02:16.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.393+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c107ca0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.393+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3c50009730 con 0x7f3c5c0ff150 2024-02-28T23:02:16.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.394+0000 7f3c59600700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 0x7f3c4406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:16.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.394+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3c5008b750 con 0x7f3c5c0ff150 2024-02-28T23:02:16.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.395+0000 7f3c5be00700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 0x7f3c4406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.395+0000 7f3c5be00700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 0x7f3c4406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c48006010 tx=0x7f3c4800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:16.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.399+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3c50059b60 con 0x7f3c5c0ff150 2024-02-28T23:02:16.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.710+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3c5c19e0b0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.722+0000 7f3c59600700 1 -- 172.21.15.57:0/1306708240 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3c500596f0 con 0x7f3c5c0ff150 2024-02-28T23:02:16.721 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2024-02-28T23:02:16.721 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQB4u99lD7J1KhAA6J8H1Ud0ctWqdY2c5p8D+g== 2024-02-28T23:02:16.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 msgr2=0x7f3c4406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:16.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 0x7f3c4406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c48006010 tx=0x7f3c4800b540 comp rx=0 tx=0).stop 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 msgr2=0x7f3c5c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000d900 tx=0x7f3c5000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 shutdown_connections 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3c4406c680 0x7f3c4406eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1030e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 --2- 172.21.15.57:0/1306708240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5c100350 0x7f3c5c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.725+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 >> 172.21.15.57:0/1306708240 conn(0x7f3c5c0fa700 msgr2=0x7f3c5c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.726+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 shutdown_connections 2024-02-28T23:02:16.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:16.726+0000 7f3c63800700 1 -- 172.21.15.57:0/1306708240 wait complete. 2024-02-28T23:02:17.395 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T23:02:17.395 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-28T23:02:17.395 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-28T23:02:17.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-28T23:02:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:02:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:17 smithi057 ceph-mon[29096]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:17 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1306708240' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:17 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1306708240' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-28T23:02:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:17 smithi084 ceph-mon[34826]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:17 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1306708240' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:17 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1306708240' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-28T23:02:19.818 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:19 smithi084 ceph-mon[34826]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:19.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 -- 172.21.15.84:0/3799017476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 msgr2=0x7fa8fc101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:19.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 --2- 172.21.15.84:0/3799017476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc101710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009910 tx=0x7fa8e8009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:19.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 -- 172.21.15.84:0/3799017476 shutdown_connections 2024-02-28T23:02:19.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 --2- 172.21.15.84:0/3799017476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc101710 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:19.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 --2- 172.21.15.84:0/3799017476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc0fe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:19.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.996+0000 7fa902e00700 1 -- 172.21.15.84:0/3799017476 >> 172.21.15.84:0/3799017476 conn(0x7fa8fc0f9c30 msgr2=0x7fa8fc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:19.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.997+0000 7fa902e00700 1 -- 172.21.15.84:0/3799017476 shutdown_connections 2024-02-28T23:02:19.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.997+0000 7fa902e00700 1 -- 172.21.15.84:0/3799017476 wait complete. 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 Processor -- start 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 -- start start 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc1985d0 con 0x7fa8fc0fe2e0 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.998+0000 7fa902e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc198710 con 0x7fa8fc1012c0 2024-02-28T23:02:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48086/0 (socket says 172.21.15.84:48086) 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 -- 172.21.15.84:0/4198589579 learned_addr learned my addr 172.21.15.84:0/4198589579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901a00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 -- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 msgr2=0x7fa8fc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 -- 172.21.15.84:0/4198589579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80095f0 con 0x7fa8fc1012c0 2024-02-28T23:02:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901a00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:19.999+0000 7fa901000700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006010 tx=0x7fa8e800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.000+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e801d070 con 0x7fa8fc1012c0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.000+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8e800f460 con 0x7fa8fc1012c0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.000+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc19d160 con 0x7fa8fc1012c0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.000+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e8021640 con 0x7fa8fc1012c0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.000+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc19d680 con 0x7fa8fc1012c0 2024-02-28T23:02:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.001+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc1053a0 con 0x7fa8fc1012c0 2024-02-28T23:02:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.002+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa8e800f5d0 con 0x7fa8fc1012c0 2024-02-28T23:02:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.003+0000 7fa8fa000700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 0x7fa8e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.003+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa8e808c8f0 con 0x7fa8fc1012c0 2024-02-28T23:02:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.003+0000 7fa901a00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 0x7fa8e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.004+0000 7fa901a00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 0x7fa8e406eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005950 tx=0x7fa8f00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:20.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.008+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa8e805ada0 con 0x7fa8fc1012c0 2024-02-28T23:02:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:19 smithi057 ceph-mon[29096]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:20.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.302+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa8fc02d070 con 0x7fa8fc1012c0 2024-02-28T23:02:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.314+0000 7fa8fa000700 1 -- 172.21.15.84:0/4198589579 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa8e805a930 con 0x7fa8fc1012c0 2024-02-28T23:02:20.312 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2024-02-28T23:02:20.312 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQB8u99ljB0zEhAATjqijHQk+zA0TpX/9pDAEA== 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.316+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 msgr2=0x7fa8e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.316+0000 7fa902e00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 0x7fa8e406eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005950 tx=0x7fa8f00058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.316+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 msgr2=0x7fa8fc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.316+0000 7fa902e00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006010 tx=0x7fa8e800be20 comp rx=0 tx=0).stop 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 shutdown_connections 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa8e406c680 0x7fa8e406eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8fc0fe2e0 0x7fa8fc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 --2- 172.21.15.84:0/4198589579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8fc1012c0 0x7fa8fc197fb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 >> 172.21.15.84:0/4198589579 conn(0x7fa8fc0f9c30 msgr2=0x7fa8fc1024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 shutdown_connections 2024-02-28T23:02:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:02:20.317+0000 7fa902e00700 1 -- 172.21.15.84:0/4198589579 wait complete. 2024-02-28T23:02:21.137 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T23:02:21.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-28T23:02:21.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-28T23:02:21.292 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-28T23:02:21.293 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-28T23:02:21.293 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph mgr dump --format=json 2024-02-28T23:02:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:21 smithi084 ceph-mon[34826]: from='client.? 172.21.15.84:0/4198589579' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:21 smithi084 ceph-mon[34826]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:21 smithi084 ceph-mon[34826]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-28T23:02:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:21 smithi084 ceph-mon[34826]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:21 smithi057 ceph-mon[29096]: from='client.? 172.21.15.84:0/4198589579' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:21 smithi057 ceph-mon[29096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-28T23:02:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:21 smithi057 ceph-mon[29096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-28T23:02:21.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:21 smithi057 ceph-mon[29096]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:21.734 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:22.955 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:23.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:23.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 -- 172.21.15.57:0/760638092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 msgr2=0x7f421c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 --2- 172.21.15.57:0/760638092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c075650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f420c009910 tx=0x7f420c009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 -- 172.21.15.57:0/760638092 shutdown_connections 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 --2- 172.21.15.57:0/760638092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 0x7f421c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 --2- 172.21.15.57:0/760638092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c075650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 -- 172.21.15.57:0/760638092 >> 172.21.15.57:0/760638092 conn(0x7f421c0fdf90 msgr2=0x7f421c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 -- 172.21.15.57:0/760638092 shutdown_connections 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.844+0000 7f4221c00700 1 -- 172.21.15.57:0/760638092 wait complete. 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.845+0000 7f4221c00700 1 Processor -- start 2024-02-28T23:02:23.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.845+0000 7f4221c00700 1 -- start start 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f4221c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f4221c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 0x7f421c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f4221c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c19d1c0 con 0x7f421c075280 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f4221c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1a0f50 con 0x7f421c075b90 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44806/0 (socket says 172.21.15.57:44806) 2024-02-28T23:02:23.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 -- 172.21.15.57:0/1769247371 learned_addr learned my addr 172.21.15.57:0/1769247371 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421a000700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 0x7f421c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 -- 172.21.15.57:0/1769247371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 msgr2=0x7f421c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 0x7f421c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.846+0000 7f421aa00700 1 -- 172.21.15.57:0/1769247371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c0095f0 con 0x7f421c075280 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f421aa00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f420c005b40 tx=0x7f420c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c01d070 con 0x7f421c075280 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f420c00f460 con 0x7f421c075280 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c021640 con 0x7f421c075280 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c1a11d0 con 0x7f421c075280 2024-02-28T23:02:23.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.847+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c1a1720 con 0x7f421c075280 2024-02-28T23:02:23.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.849+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c10e340 con 0x7f421c075280 2024-02-28T23:02:23.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.850+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f420c00f5d0 con 0x7f421c075280 2024-02-28T23:02:23.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.850+0000 7f4213400700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 0x7f420006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:23.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.851+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f420c08c8e0 con 0x7f421c075280 2024-02-28T23:02:23.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.851+0000 7f421a000700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 0x7f420006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:23.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.851+0000 7f421a000700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 0x7f420006eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4208006fd0 tx=0x7f4208008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:23.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:23.856+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f420c0572e0 con 0x7f421c075280 2024-02-28T23:02:23.855 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:23 smithi057 ceph-mon[29096]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:23 smithi084 ceph-mon[34826]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:24.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.139+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f421c04f2a0 con 0x7f421c075280 2024-02-28T23:02:24.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.144+0000 7f4213400700 1 -- 172.21.15.57:0/1769247371 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172854 (secure 0 0 0) 0x7f420c05a900 con 0x7f421c075280 2024-02-28T23:02:24.146 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.147+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 msgr2=0x7f420006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.147+0000 7f4221c00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 0x7f420006eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4208006fd0 tx=0x7f4208008040 comp rx=0 tx=0).stop 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 msgr2=0x7f421c19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f420c005b40 tx=0x7f420c0056a0 comp rx=0 tx=0).stop 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 shutdown_connections 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f420006c680 0x7f420006eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f421c075280 0x7f421c19c5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 --2- 172.21.15.57:0/1769247371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421c075b90 0x7f421c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:24.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 >> 172.21.15.57:0/1769247371 conn(0x7f421c0fdf90 msgr2=0x7f421c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:24.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 shutdown_connections 2024-02-28T23:02:24.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:24.148+0000 7f4221c00700 1 -- 172.21.15.57:0/1769247371 wait complete. 2024-02-28T23:02:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:24 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1769247371' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-28T23:02:24.900 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"active_gid":14217,"active_name":"smithi057.zrrldj","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":3095633985},{"type":"v1","addr":"172.21.15.57:6801","nonce":3095633985}]},"active_addr":"172.21.15.57:6801/3095633985","active_change":"2024-02-28T22:57:22.445799+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14242,"name":"smithi084.fhkzis","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/","prometheus":"http://172.21.15.57:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2372541761}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3690186012}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":382831045}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":4070283299}]}]} 2024-02-28T23:02:24.904 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-28T23:02:24.905 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-28T23:02:24.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-28T23:02:25.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:24 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1769247371' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-28T23:02:25.351 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:25.785 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:25 smithi057 ceph-mon[29096]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:25.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:25 smithi084 ceph-mon[34826]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.415+0000 7f0be9a00700 1 -- 172.21.15.57:0/2073341359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 msgr2=0x7f0be4108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.415+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2073341359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4108840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009910 tx=0x7f0bd4009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.415+0000 7f0be9a00700 1 -- 172.21.15.57:0/2073341359 shutdown_connections 2024-02-28T23:02:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.416+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2073341359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 0x7f0be41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.416+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2073341359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4108840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.416+0000 7f0be9a00700 1 -- 172.21.15.57:0/2073341359 >> 172.21.15.57:0/2073341359 conn(0x7f0be40fdf90 msgr2=0x7f0be41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.416+0000 7f0be9a00700 1 -- 172.21.15.57:0/2073341359 shutdown_connections 2024-02-28T23:02:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.416+0000 7f0be9a00700 1 -- 172.21.15.57:0/2073341359 wait complete. 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 Processor -- start 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 -- start start 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 0x7f0be41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4198e10 con 0x7f0be4108470 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.417+0000 7f0be9a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be419cba0 con 0x7f0be4102470 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50406/0 (socket says 172.21.15.57:50406) 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 -- 172.21.15.57:0/2263128063 learned_addr learned my addr 172.21.15.57:0/2263128063 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 0x7f0be41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 -- 172.21.15.57:0/2263128063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 msgr2=0x7f0be41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 0x7f0be41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 -- 172.21.15.57:0/2263128063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40095f0 con 0x7f0be4108470 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.418+0000 7f0be2000700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d8d0 tx=0x7f0bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.419+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd0009940 con 0x7f0be4108470 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.419+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be419ce80 con 0x7f0be4108470 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.419+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0bd0010460 con 0x7f0be4108470 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.419+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd000f5f0 con 0x7f0be4108470 2024-02-28T23:02:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.419+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be419d3d0 con 0x7f0be4108470 2024-02-28T23:02:27.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.420+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be410a9e0 con 0x7f0be4108470 2024-02-28T23:02:27.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.421+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0bd00105d0 con 0x7f0be4108470 2024-02-28T23:02:27.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.422+0000 7f0bdb400700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 0x7f0bc806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:27.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.423+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0bd008b5e0 con 0x7f0be4108470 2024-02-28T23:02:27.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.423+0000 7f0be2a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 0x7f0bc806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:27.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.423+0000 7f0be2a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 0x7f0bc806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4006010 tx=0x7f0bd401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:27.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.427+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0bd0055f60 con 0x7f0be4108470 2024-02-28T23:02:27.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.650+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be41995f0 con 0x7f0be4108470 2024-02-28T23:02:27.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.651+0000 7f0bdb400700 1 -- 172.21.15.57:0/2263128063 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11286 (secure 0 0 0) 0x7f0bd0059580 con 0x7f0be4108470 2024-02-28T23:02:27.651 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:02:27.651 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":41,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:02:05.259124+0000","last_up_change":"2024-02-28T23:02:04.242665+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6803","nonce":119728824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6805","nonce":119728824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6809","nonce":119728824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6807","nonce":119728824}]},"public_addr":"172.21.15.57:6803/119728824","cluster_addr":"172.21.15.57:6805/119728824","heartbeat_back_addr":"172.21.15.57:6809/119728824","heartbeat_front_addr":"172.21.15.57:6807/119728824","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6811","nonce":313179796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6813","nonce":313179796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6817","nonce":313179796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6815","nonce":313179796}]},"public_addr":"172.21.15.57:6811/313179796","cluster_addr":"172.21.15.57:6813/313179796","heartbeat_back_addr":"172.21.15.57:6817/313179796","heartbeat_front_addr":"172.21.15.57:6815/313179796","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6819","nonce":2667584111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6821","nonce":2667584111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6825","nonce":2667584111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6823","nonce":2667584111}]},"public_addr":"172.21.15.57:6819/2667584111","cluster_addr":"172.21.15.57:6821/2667584111","heartbeat_back_addr":"172.21.15.57:6825/2667584111","heartbeat_front_addr":"172.21.15.57:6823/2667584111","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6801","nonce":3040593930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6803","nonce":3040593930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6807","nonce":3040593930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6805","nonce":3040593930}]},"public_addr":"172.21.15.84:6801/3040593930","cluster_addr":"172.21.15.84:6803/3040593930","heartbeat_back_addr":"172.21.15.84:6807/3040593930","heartbeat_front_addr":"172.21.15.84:6805/3040593930","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6809","nonce":3561819927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6811","nonce":3561819927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6815","nonce":3561819927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6813","nonce":3561819927}]},"public_addr":"172.21.15.84:6809/3561819927","cluster_addr":"172.21.15.84:6811/3561819927","heartbeat_back_addr":"172.21.15.84:6815/3561819927","heartbeat_front_addr":"172.21.15.84:6813/3561819927","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6817","nonce":3141452916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6819","nonce":3141452916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6823","nonce":3141452916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6821","nonce":3141452916}]},"public_addr":"172.21.15.84:6817/3141452916","cluster_addr":"172.21.15.84:6819/3141452916","heartbeat_back_addr":"172.21.15.84:6823/3141452916","heartbeat_front_addr":"172.21.15.84:6821/3141452916","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:02:27.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 msgr2=0x7f0bc806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 0x7f0bc806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4006010 tx=0x7f0bd401a040 comp rx=0 tx=0).stop 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 msgr2=0x7f0be4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d8d0 tx=0x7f0bd000dc90 comp rx=0 tx=0).stop 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 shutdown_connections 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0bc806c680 0x7f0bc806eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0be4102470 0x7f0be41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 --2- 172.21.15.57:0/2263128063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0be4108470 0x7f0be4198730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 >> 172.21.15.57:0/2263128063 conn(0x7f0be40fdf90 msgr2=0x7f0be41002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.654+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 shutdown_connections 2024-02-28T23:02:27.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:27.655+0000 7f0be9a00700 1 -- 172.21.15.57:0/2263128063 wait complete. 2024-02-28T23:02:27.949 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:27 smithi057 ceph-mon[29096]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:27 smithi084 ceph-mon[34826]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:28.330 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-28T23:02:28.330 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-28T23:02:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:28.919 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:28 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2263128063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:29.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:28 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2263128063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:29.957 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:29 smithi057 ceph-mon[29096]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:29 smithi084 ceph-mon[34826]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 -- 172.21.15.57:0/2344894018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 msgr2=0x7f99641008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 --2- 172.21.15.57:0/2344894018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f99641008b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9954009910 tx=0x7f9954009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 -- 172.21.15.57:0/2344894018 shutdown_connections 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 --2- 172.21.15.57:0/2344894018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 0x7f9964104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 --2- 172.21.15.57:0/2344894018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f99641008b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 -- 172.21.15.57:0/2344894018 >> 172.21.15.57:0/2344894018 conn(0x7f99640fbda0 msgr2=0x7f99640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.826+0000 7f9969a00700 1 -- 172.21.15.57:0/2344894018 shutdown_connections 2024-02-28T23:02:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.827+0000 7f9969a00700 1 -- 172.21.15.57:0/2344894018 wait complete. 2024-02-28T23:02:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 Processor -- start 2024-02-28T23:02:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 -- start start 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 0x7f9964073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964073ad0 con 0x7f99641004e0 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.828+0000 7f9969a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964073c10 con 0x7f9964100e80 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50420/0 (socket says 172.21.15.57:50420) 2024-02-28T23:02:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 -- 172.21.15.57:0/1015513664 learned_addr learned my addr 172.21.15.57:0/1015513664 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962000700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 0x7f9964073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 -- 172.21.15.57:0/1015513664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 msgr2=0x7f9964073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 0x7f9964073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.829+0000 7f9962a00700 1 -- 172.21.15.57:0/1015513664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540095f0 con 0x7f99641004e0 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f9962a00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f995400b5c0 tx=0x7f995400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995401d070 con 0x7f99641004e0 2024-02-28T23:02:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f9969a00700 1 -- 172.21.15.57:0/1015513664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9964073e90 con 0x7f99641004e0 2024-02-28T23:02:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9954022470 con 0x7f99641004e0 2024-02-28T23:02:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995400f670 con 0x7f99641004e0 2024-02-28T23:02:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.830+0000 7f9969a00700 1 -- 172.21.15.57:0/1015513664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99641a23f0 con 0x7f99641004e0 2024-02-28T23:02:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.832+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c0052f0 con 0x7f99641004e0 2024-02-28T23:02:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.833+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9954004c30 con 0x7f99641004e0 2024-02-28T23:02:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.833+0000 7f995b400700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.834+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f995408cae0 con 0x7f99641004e0 2024-02-28T23:02:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.834+0000 7f9962000700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.834+0000 7f9962000700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9950005fd0 tx=0x7f995000a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:30.839+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f99540575d0 con 0x7f99641004e0 2024-02-28T23:02:31.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.063+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f993c005160 con 0x7f99641004e0 2024-02-28T23:02:31.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.065+0000 7f995b400700 1 -- 172.21.15.57:0/1015513664 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11286 (secure 0 0 0) 0x7f9954027090 con 0x7f99641004e0 2024-02-28T23:02:31.064 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:02:31.064 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":41,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:02:05.259124+0000","last_up_change":"2024-02-28T23:02:04.242665+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6803","nonce":119728824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6805","nonce":119728824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6809","nonce":119728824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6807","nonce":119728824}]},"public_addr":"172.21.15.57:6803/119728824","cluster_addr":"172.21.15.57:6805/119728824","heartbeat_back_addr":"172.21.15.57:6809/119728824","heartbeat_front_addr":"172.21.15.57:6807/119728824","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6811","nonce":313179796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6813","nonce":313179796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6817","nonce":313179796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6815","nonce":313179796}]},"public_addr":"172.21.15.57:6811/313179796","cluster_addr":"172.21.15.57:6813/313179796","heartbeat_back_addr":"172.21.15.57:6817/313179796","heartbeat_front_addr":"172.21.15.57:6815/313179796","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6819","nonce":2667584111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6821","nonce":2667584111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6825","nonce":2667584111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6823","nonce":2667584111}]},"public_addr":"172.21.15.57:6819/2667584111","cluster_addr":"172.21.15.57:6821/2667584111","heartbeat_back_addr":"172.21.15.57:6825/2667584111","heartbeat_front_addr":"172.21.15.57:6823/2667584111","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6801","nonce":3040593930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6803","nonce":3040593930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6807","nonce":3040593930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6805","nonce":3040593930}]},"public_addr":"172.21.15.84:6801/3040593930","cluster_addr":"172.21.15.84:6803/3040593930","heartbeat_back_addr":"172.21.15.84:6807/3040593930","heartbeat_front_addr":"172.21.15.84:6805/3040593930","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6809","nonce":3561819927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6811","nonce":3561819927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6815","nonce":3561819927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6813","nonce":3561819927}]},"public_addr":"172.21.15.84:6809/3561819927","cluster_addr":"172.21.15.84:6811/3561819927","heartbeat_back_addr":"172.21.15.84:6815/3561819927","heartbeat_front_addr":"172.21.15.84:6813/3561819927","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6817","nonce":3141452916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6819","nonce":3141452916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6823","nonce":3141452916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6821","nonce":3141452916}]},"public_addr":"172.21.15.84:6817/3141452916","cluster_addr":"172.21.15.84:6819/3141452916","heartbeat_back_addr":"172.21.15.84:6823/3141452916","heartbeat_front_addr":"172.21.15.84:6821/3141452916","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.067+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 msgr2=0x7f994806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.067+0000 7f9958c00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9950005fd0 tx=0x7f995000a560 comp rx=0 tx=0).stop 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.067+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 msgr2=0x7f9964073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.067+0000 7f9958c00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f995400b5c0 tx=0x7f995400be00 comp rx=0 tx=0).stop 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 shutdown_connections 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99641004e0 0x7f9964073050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 --2- 172.21.15.57:0/1015513664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100e80 0x7f9964073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:31.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 >> 172.21.15.57:0/1015513664 conn(0x7f99640fbda0 msgr2=0x7f99640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:31.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 shutdown_connections 2024-02-28T23:02:31.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:31.068+0000 7f9958c00700 1 -- 172.21.15.57:0/1015513664 wait complete. 2024-02-28T23:02:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:31 smithi057 ceph-mon[29096]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:31 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1015513664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:31.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:31 smithi084 ceph-mon[34826]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:31.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:31 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1015513664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:02:31.861 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-28T23:02:31.861 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-28T23:02:31.861 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-28T23:02:31.862 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-28T23:02:31.862 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-28T23:02:31.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-28T23:02:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:32.383 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:33.795 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:33 smithi057 ceph-mon[29096]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:33.833 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:33.834 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:33.834 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:33 smithi084 ceph-mon[34826]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:34.372 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:35.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:35 smithi057 ceph-mon[29096]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:36.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:35 smithi084 ceph-mon[34826]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 -- 172.21.15.57:0/2643687333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92280687a0 msgr2=0x7f9228068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 --2- 172.21.15.57:0/2643687333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92280687a0 0x7f9228068b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f921c009910 tx=0x7f921c009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 -- 172.21.15.57:0/2643687333 shutdown_connections 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 --2- 172.21.15.57:0/2643687333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9228069140 0x7f9228104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 --2- 172.21.15.57:0/2643687333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92280687a0 0x7f9228068b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.417+0000 7f9230200700 1 -- 172.21.15.57:0/2643687333 >> 172.21.15.57:0/2643687333 conn(0x7f9228075130 msgr2=0x7f9228075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.418+0000 7f9230200700 1 -- 172.21.15.57:0/2643687333 shutdown_connections 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.418+0000 7f9230200700 1 -- 172.21.15.57:0/2643687333 wait complete. 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.419+0000 7f9230200700 1 Processor -- start 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.419+0000 7f9230200700 1 -- start start 2024-02-28T23:02:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f9230200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92280687a0 0x7f92281980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f9230200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f9230200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228198cd0 con 0x7f9228069140 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f9230200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819ca60 con 0x7f92280687a0 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f922ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f922ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54898/0 (socket says 172.21.15.57:54898) 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f922ce00700 1 -- 172.21.15.57:0/1027227064 learned_addr learned my addr 172.21.15.57:0/1027227064 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.420+0000 7f922d800700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92280687a0 0x7f92281980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f922ce00700 1 -- 172.21.15.57:0/1027227064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92280687a0 msgr2=0x7f92281980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f922ce00700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92280687a0 0x7f92281980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f922ce00700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0095f0 con 0x7f9228069140 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f922ce00700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f921400c8f0 tx=0x7f921400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92140078a0 con 0x7f9228069140 2024-02-28T23:02:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f921400f450 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.421+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9214018610 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.422+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922819cd40 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.422+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922819d240 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.423+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f922810d670 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.424+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f921400f5c0 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f9226000700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 0x7f921006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f921408ba40 con 0x7f9228069140 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f9226000700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 0x7f9210074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f922d800700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 0x7f921006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9210074e10 con 0x7f9210072350 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.425+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921408be00 con 0x7f9228069140 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.426+0000 7f922e200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 0x7f9210074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.426+0000 7f922d800700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 0x7f921006eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f921c006010 tx=0x7f921c01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.427+0000 7f922e200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 0x7f9210074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.427+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== osd.0 v2:172.21.15.57:6802/119728824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9210074e10 con 0x7f9210072350 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.446+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 --> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f922804f2a0 con 0x7f9210072350 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9226000700 1 -- 172.21.15.57:0/1027227064 <== osd.0 v2:172.21.15.57:6802/119728824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f922804f2a0 con 0x7f9210072350 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 msgr2=0x7f9210074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 0x7f9210074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 msgr2=0x7f921006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 0x7f921006eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f921c006010 tx=0x7f921c01a040 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 msgr2=0x7f92281985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.447+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f921400c8f0 tx=0x7f921400cc00 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 shutdown_connections 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6802/119728824,v1:172.21.15.57:6803/119728824] conn(0x7f9210072350 0x7f9210074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f921006c680 0x7f921006eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92280687a0 0x7f92281980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 --2- 172.21.15.57:0/1027227064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228069140 0x7f92281985f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 >> 172.21.15.57:0/1027227064 conn(0x7f9228075130 msgr2=0x7f9228101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 shutdown_connections 2024-02-28T23:02:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:37.448+0000 7f9230200700 1 -- 172.21.15.57:0/1027227064 wait complete. 2024-02-28T23:02:37.784 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:37 smithi057 ceph-mon[29096]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:37.784 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:37 smithi084 ceph-mon[34826]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:39.771 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:39 smithi057 ceph-mon[29096]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 -- 172.21.15.57:0/1279555314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 msgr2=0x7f2ce8100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 --2- 172.21.15.57:0/1279555314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8100680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009910 tx=0x7f2cd8009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 -- 172.21.15.57:0/1279555314 shutdown_connections 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 --2- 172.21.15.57:0/1279555314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 0x7f2ce8104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 --2- 172.21.15.57:0/1279555314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8100680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 -- 172.21.15.57:0/1279555314 >> 172.21.15.57:0/1279555314 conn(0x7f2ce80fbb50 msgr2=0x7f2ce80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 -- 172.21.15.57:0/1279555314 shutdown_connections 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.633+0000 7f2ce7200700 1 -- 172.21.15.57:0/1279555314 wait complete. 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 Processor -- start 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 -- start start 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 0x7f2ce81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8198ac0 con 0x7f2ce81002b0 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce819c850 con 0x7f2ce8100c50 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.635+0000 7f2ce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54910/0 (socket says 172.21.15.57:54910) 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 -- 172.21.15.57:0/25983817 learned_addr learned my addr 172.21.15.57:0/25983817 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5400700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 0x7f2ce81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 -- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 msgr2=0x7f2ce81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 0x7f2ce81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce81002b0 2024-02-28T23:02:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.636+0000 7f2ce5e00700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.637+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd801d070 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.637+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2cd8004f50 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.637+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce819cad0 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.637+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd8021640 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.637+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce819cfc0 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.638+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ce810d410 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.639+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2cd800f460 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2cdea00700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 0x7f2ccc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2cd808c740 con 0x7f2ce81002b0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2ce5400700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 0x7f2ccc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2cdea00700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 0x7f2ccc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ccc074e10 con 0x7f2ccc072350 2024-02-28T23:02:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.640+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd808ed70 con 0x7f2ce81002b0 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.641+0000 7f2ce5400700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 0x7f2ccc06eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4005950 tx=0x7f2cd40058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.641+0000 7f2ce6800700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 0x7f2ccc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.642+0000 7f2ce6800700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 0x7f2ccc074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.644+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== osd.4 v2:172.21.15.84:6808/3561819927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2ccc074e10 con 0x7f2ccc072350 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.661+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 --> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ce8062380 con 0x7f2ccc072350 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2cdea00700 1 -- 172.21.15.57:0/25983817 <== osd.4 v2:172.21.15.84:6808/3561819927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2ce8062380 con 0x7f2ccc072350 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 msgr2=0x7f2ccc074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 0x7f2ccc074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 msgr2=0x7f2ccc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 0x7f2ccc06eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4005950 tx=0x7f2cd40058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 msgr2=0x7f2ce8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.662+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd800be20 comp rx=0 tx=0).stop 2024-02-28T23:02:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 shutdown_connections 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:6808/3561819927,v1:172.21.15.84:6809/3561819927] conn(0x7f2ccc072350 0x7f2ccc074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2ccc06c680 0x7f2ccc06eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ce81002b0 0x7f2ce8197ea0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 --2- 172.21.15.57:0/25983817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ce8100c50 0x7f2ce81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 >> 172.21.15.57:0/25983817 conn(0x7f2ce80fbb50 msgr2=0x7f2ce80fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 shutdown_connections 2024-02-28T23:02:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:39.663+0000 7f2ce7200700 1 -- 172.21.15.57:0/25983817 wait complete. 2024-02-28T23:02:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:39 smithi084 ceph-mon[34826]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:40.723 INFO:teuthology.orchestra.run.smithi057.stdout:150323855377 2024-02-28T23:02:40.723 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 -- 172.21.15.57:0/952764269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8001004e0 msgr2=0x7fa8001008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 --2- 172.21.15.57:0/952764269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8001004e0 0x7fa8001008b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009910 tx=0x7fa7f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 -- 172.21.15.57:0/952764269 shutdown_connections 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 --2- 172.21.15.57:0/952764269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa800100e80 0x7fa800104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 --2- 172.21.15.57:0/952764269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8001004e0 0x7fa8001008b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 -- 172.21.15.57:0/952764269 >> 172.21.15.57:0/952764269 conn(0x7fa8000fbda0 msgr2=0x7fa8000fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 -- 172.21.15.57:0/952764269 shutdown_connections 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.549+0000 7fa807e00700 1 -- 172.21.15.57:0/952764269 wait complete. 2024-02-28T23:02:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.550+0000 7fa807e00700 1 Processor -- start 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa807e00700 1 -- start start 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa807e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 0x7fa8001980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa807e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa807e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800198d10 con 0x7fa800100e80 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa807e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80019caa0 con 0x7fa8001004e0 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa804a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa804a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54918/0 (socket says 172.21.15.57:54918) 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa804a00700 1 -- 172.21.15.57:0/1092309608 learned_addr learned my addr 172.21.15.57:0/1092309608 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.551+0000 7fa805400700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 0x7fa8001980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa804a00700 1 -- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 msgr2=0x7fa8001980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa804a00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 0x7fa8001980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa804a00700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40095f0 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa805400700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 0x7fa8001980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa804a00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d900 tx=0x7fa7ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec009900 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec010460 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec00f5f0 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80019cd80 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.552+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80019d2d0 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.553+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa80010d670 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.554+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa7ec0105d0 con 0x7fa800100e80 2024-02-28T23:02:40.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.555+0000 7fa7fe000700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 0x7fa7e806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.555+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa7ec08b2d0 con 0x7fa800100e80 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa7fe000700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 0x7fa7e8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7e8074e10 con 0x7fa7e8072350 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ec08d9a0 con 0x7fa800100e80 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa805400700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 0x7fa7e806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa805e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 0x7fa7e8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.556+0000 7fa805400700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 0x7fa7e806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.557+0000 7fa805e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 0x7fa7e8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.559+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== osd.5 v2:172.21.15.84:6816/3141452916 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa7e8074e10 con 0x7fa7e8072350 2024-02-28T23:02:40.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.576+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 --> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa80004f2a0 con 0x7fa7e8072350 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.577+0000 7fa7fe000700 1 -- 172.21.15.57:0/1092309608 <== osd.5 v2:172.21.15.84:6816/3141452916 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa80004f2a0 con 0x7fa7e8072350 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 msgr2=0x7fa7e8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 0x7fa7e8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 msgr2=0x7fa7e806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 0x7fa7e806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f400b540 comp rx=0 tx=0).stop 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 msgr2=0x7fa800198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d900 tx=0x7fa7ec00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 shutdown_connections 2024-02-28T23:02:40.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:6816/3141452916,v1:172.21.15.84:6817/3141452916] conn(0x7fa7e8072350 0x7fa7e8074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fa7e806c680 0x7fa7e806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8001004e0 0x7fa8001980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 --2- 172.21.15.57:0/1092309608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa800100e80 0x7fa800198630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 >> 172.21.15.57:0/1092309608 conn(0x7fa8000fbda0 msgr2=0x7fa8000fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 shutdown_connections 2024-02-28T23:02:40.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:40.578+0000 7fa807e00700 1 -- 172.21.15.57:0/1092309608 wait complete. 2024-02-28T23:02:41.378 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:41 smithi057 ceph-mon[29096]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1924897612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 msgr2=0x7ffb741008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1924897612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741008b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009910 tx=0x7ffb68009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1924897612 shutdown_connections 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1924897612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 0x7ffb74104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1924897612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741008b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1924897612 >> 172.21.15.57:0/1924897612 conn(0x7ffb740fbda0 msgr2=0x7ffb740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1924897612 shutdown_connections 2024-02-28T23:02:41.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.457+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1924897612 wait complete. 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.458+0000 7ffb7ba00700 1 Processor -- start 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.458+0000 7ffb7ba00700 1 -- start start 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb7ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb7ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 0x7ffb74198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb7ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74198d10 con 0x7ffb741004e0 2024-02-28T23:02:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb7ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7419caa0 con 0x7ffb74100e80 2024-02-28T23:02:41.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb79000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:41.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb79000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54930/0 (socket says 172.21.15.57:54930) 2024-02-28T23:02:41.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.459+0000 7ffb79000700 1 -- 172.21.15.57:0/1662320116 learned_addr learned my addr 172.21.15.57:0/1662320116 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb73e00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 0x7ffb74198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb79000700 1 -- 172.21.15.57:0/1662320116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 msgr2=0x7ffb74198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb79000700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 0x7ffb74198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb79000700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680095f0 con 0x7ffb741004e0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb79000700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb680092e0 tx=0x7ffb6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb6801d070 con 0x7ffb741004e0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb68004fa0 con 0x7ffb741004e0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.460+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7419cd20 con 0x7ffb741004e0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.461+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb6800f460 con 0x7ffb741004e0 2024-02-28T23:02:41.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.461+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7419d210 con 0x7ffb741004e0 2024-02-28T23:02:41.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.461+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb7410d670 con 0x7ffb741004e0 2024-02-28T23:02:41.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.463+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ffb68022810 con 0x7ffb741004e0 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb71600700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 0x7ffb5c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffb6808d2f0 con 0x7ffb741004e0 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb73e00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 0x7ffb5c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb71600700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 0x7ffb5c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb5c074e10 con 0x7ffb5c072350 2024-02-28T23:02:41.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb6808d6b0 con 0x7ffb741004e0 2024-02-28T23:02:41.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.464+0000 7ffb79a00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 0x7ffb5c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:41.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.465+0000 7ffb73e00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 0x7ffb5c06eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005950 tx=0x7ffb600058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:41.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.465+0000 7ffb79a00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 0x7ffb5c074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:41.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.466+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== osd.2 v2:172.21.15.57:6818/2667584111 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffb5c074e10 con 0x7ffb5c072350 2024-02-28T23:02:41.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.484+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 --> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb7404f2a0 con 0x7ffb5c072350 2024-02-28T23:02:41.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.484+0000 7ffb71600700 1 -- 172.21.15.57:0/1662320116 <== osd.2 v2:172.21.15.57:6818/2667584111 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb7404f2a0 con 0x7ffb5c072350 2024-02-28T23:02:41.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 msgr2=0x7ffb5c074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:41.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 0x7ffb5c074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 msgr2=0x7ffb5c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 0x7ffb5c06eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005950 tx=0x7ffb600058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 msgr2=0x7ffb741980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb680092e0 tx=0x7ffb6800be20 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 shutdown_connections 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6818/2667584111,v1:172.21.15.57:6819/2667584111] conn(0x7ffb5c072350 0x7ffb5c074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ffb5c06c680 0x7ffb5c06eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb741004e0 0x7ffb741980f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 --2- 172.21.15.57:0/1662320116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb74100e80 0x7ffb74198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 >> 172.21.15.57:0/1662320116 conn(0x7ffb740fbda0 msgr2=0x7ffb740fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 shutdown_connections 2024-02-28T23:02:41.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:41.485+0000 7ffb7ba00700 1 -- 172.21.15.57:0/1662320116 wait complete. 2024-02-28T23:02:41.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:41 smithi084 ceph-mon[34826]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:42.108 INFO:teuthology.orchestra.run.smithi057.stdout:171798691849 2024-02-28T23:02:42.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-28T23:02:43.441 INFO:teuthology.orchestra.run.smithi057.stdout:38654705708 2024-02-28T23:02:43.441 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-28T23:02:43.659 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:43 smithi057 ceph-mon[29096]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:43 smithi084 ceph-mon[34826]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:44.132 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:44.155 INFO:teuthology.orchestra.run.smithi057.stdout:81604378655 2024-02-28T23:02:44.156 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 -- 172.21.15.57:0/2952326355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b0100e80 msgr2=0x7f90b0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 --2- 172.21.15.57:0/2952326355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b0100e80 0x7f90b0104e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f909c009910 tx=0x7f909c009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 -- 172.21.15.57:0/2952326355 shutdown_connections 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 --2- 172.21.15.57:0/2952326355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b0100e80 0x7f90b0104e50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 --2- 172.21.15.57:0/2952326355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b01004e0 0x7f90b01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 -- 172.21.15.57:0/2952326355 >> 172.21.15.57:0/2952326355 conn(0x7f90b00fbda0 msgr2=0x7f90b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 -- 172.21.15.57:0/2952326355 shutdown_connections 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.747+0000 7f90b6600700 1 -- 172.21.15.57:0/2952326355 wait complete. 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.748+0000 7f90b6600700 1 Processor -- start 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.748+0000 7f90b6600700 1 -- start start 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90b6600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90b6600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 0x7f90b0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90b6600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0074e00 con 0x7f90b01004e0 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90b6600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0074f70 con 0x7f90b0100e80 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54948/0 (socket says 172.21.15.57:54948) 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 -- 172.21.15.57:0/3789915253 learned_addr learned my addr 172.21.15.57:0/3789915253 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90aea00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 0x7f90b0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 -- 172.21.15.57:0/3789915253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 msgr2=0x7f90b0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 0x7f90b0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90af400700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0095f0 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.749+0000 7f90aea00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 0x7f90b0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90af400700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c8f0 tx=0x7f90a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a0007a70 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f90a0004ce0 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b0073b90 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a0005610 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.750+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b01a6a10 con 0x7f90b01004e0 2024-02-28T23:02:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.751+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90b010d670 con 0x7f90b01004e0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.752+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f90a0004750 con 0x7f90b01004e0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.753+0000 7f90a7e00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 0x7f909406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.753+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f90a0013070 con 0x7f90b01004e0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90aea00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 0x7f909406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90a7e00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 0x7f9094074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9094074e10 con 0x7f9094072350 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90a008d8f0 con 0x7f90b01004e0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90afe00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 0x7f9094074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.754+0000 7f90aea00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 0x7f909406eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f909c006010 tx=0x7f909c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.755+0000 7f90afe00700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 0x7f9094074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.756+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== osd.1 v2:172.21.15.57:6810/313179796 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9094074e10 con 0x7f9094072350 2024-02-28T23:02:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.773+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 --> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90b004f2a0 con 0x7f9094072350 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.774+0000 7f90a7e00700 1 -- 172.21.15.57:0/3789915253 <== osd.1 v2:172.21.15.57:6810/313179796 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f90b004f2a0 con 0x7f9094072350 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.774+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 msgr2=0x7f9094074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.774+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 0x7f9094074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.774+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 msgr2=0x7f909406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.774+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 0x7f909406eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f909c006010 tx=0x7f909c00b540 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 msgr2=0x7f90b0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c8f0 tx=0x7f90a000ccb0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 shutdown_connections 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6810/313179796,v1:172.21.15.57:6811/313179796] conn(0x7f9094072350 0x7f9094074760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f909406c680 0x7f909406eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90b01004e0 0x7f90b0073050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 --2- 172.21.15.57:0/3789915253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b0100e80 0x7f90b0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 >> 172.21.15.57:0/3789915253 conn(0x7f90b00fbda0 msgr2=0x7f90b01046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 shutdown_connections 2024-02-28T23:02:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:44.775+0000 7f90b6600700 1 -- 172.21.15.57:0/3789915253 wait complete. 2024-02-28T23:02:45.962 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:45 smithi084 ceph-mon[34826]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:45.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:45 smithi057 ceph-mon[29096]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:46.564 INFO:teuthology.orchestra.run.smithi057.stdout:60129542181 2024-02-28T23:02:46.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-28T23:02:46.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 -- 172.21.15.57:0/187044596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec102470 msgr2=0x7f19ec1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 --2- 172.21.15.57:0/187044596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec102470 0x7f19ec1028e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009910 tx=0x7f19d8009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 -- 172.21.15.57:0/187044596 shutdown_connections 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 --2- 172.21.15.57:0/187044596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec102470 0x7f19ec1028e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 --2- 172.21.15.57:0/187044596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec108470 0x7f19ec108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.395+0000 7f19f2a00700 1 -- 172.21.15.57:0/187044596 >> 172.21.15.57:0/187044596 conn(0x7f19ec0fdf90 msgr2=0x7f19ec1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.396+0000 7f19f2a00700 1 -- 172.21.15.57:0/187044596 shutdown_connections 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.396+0000 7f19f2a00700 1 -- 172.21.15.57:0/187044596 wait complete. 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 Processor -- start 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 -- start start 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec102470 0x7f19ec1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec198e10 con 0x7f19ec108470 2024-02-28T23:02:46.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.397+0000 7f19f2a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19cba0 con 0x7f19ec102470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.398+0000 7f19eb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.398+0000 7f19eb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56014/0 (socket says 172.21.15.57:56014) 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.398+0000 7f19eb400700 1 -- 172.21.15.57:0/1437453844 learned_addr learned my addr 172.21.15.57:0/1437453844 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.398+0000 7f19ebe00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec102470 0x7f19ec1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19eb400700 1 -- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec102470 msgr2=0x7f19ec1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19eb400700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec102470 0x7f19ec1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19eb400700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80095f0 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19eb400700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b5c0 tx=0x7f19d800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d801d070 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec19ce20 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.399+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec19d310 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.400+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19d800f460 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.400+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d802b610 con 0x7f19ec108470 2024-02-28T23:02:46.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.401+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19ec10f860 con 0x7f19ec108470 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.402+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f19d8003b20 con 0x7f19ec108470 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19e8c00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 0x7f19d006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f19d808cab0 con 0x7f19ec108470 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19ebe00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 0x7f19d006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19e8c00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 0x7f19d0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19d0074e10 con 0x7f19d0072350 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.403+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d80912a0 con 0x7f19ec108470 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.404+0000 7f19f0a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 0x7f19d0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.404+0000 7f19ebe00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 0x7f19d006eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005950 tx=0x7f19dc00a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.405+0000 7f19f0a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 0x7f19d0074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:46.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.408+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== osd.3 v2:172.21.15.84:6800/3040593930 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f19d0074e10 con 0x7f19d0072350 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.425+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 --> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19ec04f2a0 con 0x7f19d0072350 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19e8c00700 1 -- 172.21.15.57:0/1437453844 <== osd.3 v2:172.21.15.84:6800/3040593930 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f19ec04f2a0 con 0x7f19d0072350 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 msgr2=0x7f19d0074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 0x7f19d0074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 msgr2=0x7f19d006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 0x7f19d006eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005950 tx=0x7f19dc00a300 comp rx=0 tx=0).stop 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 msgr2=0x7f19ec198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b5c0 tx=0x7f19d800bd50 comp rx=0 tx=0).stop 2024-02-28T23:02:46.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 shutdown_connections 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f19d006c680 0x7f19d006eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:6800/3040593930,v1:172.21.15.84:6801/3040593930] conn(0x7f19d0072350 0x7f19d0074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19ec102470 0x7f19ec1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 --2- 172.21.15.57:0/1437453844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19ec108470 0x7f19ec198730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 >> 172.21.15.57:0/1437453844 conn(0x7f19ec0fdf90 msgr2=0x7f19ec0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 shutdown_connections 2024-02-28T23:02:46.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:46.426+0000 7f19f2a00700 1 -- 172.21.15.57:0/1437453844 wait complete. 2024-02-28T23:02:47.698 INFO:teuthology.orchestra.run.smithi057.stdout:115964117016 2024-02-28T23:02:47.699 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-28T23:02:47.937 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:47 smithi057 ceph-mon[29096]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:47 smithi084 ceph-mon[34826]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 -- 172.21.15.57:0/1235571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 msgr2=0x7ff348104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 --2- 172.21.15.57:0/1235571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348104e30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff334009910 tx=0x7ff334009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 -- 172.21.15.57:0/1235571348 shutdown_connections 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 --2- 172.21.15.57:0/1235571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348104e30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 --2- 172.21.15.57:0/1235571348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 -- 172.21.15.57:0/1235571348 >> 172.21.15.57:0/1235571348 conn(0x7ff3480fbd80 msgr2=0x7ff3480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 -- 172.21.15.57:0/1235571348 shutdown_connections 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.628+0000 7ff350600700 1 -- 172.21.15.57:0/1235571348 wait complete. 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 Processor -- start 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 -- start start 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348073ad0 con 0x7ff348100e60 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff350600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348073c10 con 0x7ff3481004c0 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff34d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.630+0000 7ff34d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56022/0 (socket says 172.21.15.57:56022) 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34d200700 1 -- 172.21.15.57:0/4127500115 learned_addr learned my addr 172.21.15.57:0/4127500115 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34dc00700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34d200700 1 -- 172.21.15.57:0/4127500115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 msgr2=0x7ff348073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34d200700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34d200700 1 -- 172.21.15.57:0/4127500115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340095f0 con 0x7ff348100e60 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34dc00700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff34d200700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff33400bda0 tx=0x7ff33400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33401d070 con 0x7ff348100e60 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff334022470 con 0x7ff348100e60 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.631+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff348073e90 con 0x7ff348100e60 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stdout:150323855378 2024-02-28T23:02:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.632+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33400f690 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.632+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3481a2450 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.633+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff348108780 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.634+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff334022b00 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.635+0000 7ff346000700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 0x7ff33006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.635+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff33408cb70 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.635+0000 7ff34dc00700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 0x7ff33006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.636+0000 7ff34dc00700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 0x7ff33006eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff33c007900 tx=0x7ff33c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.640+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff3340575a0 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.874+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff34804f2a0 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.875+0000 7ff346000700 1 -- 172.21.15.57:0/4127500115 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff334027020 con 0x7ff348100e60 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 msgr2=0x7ff33006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 0x7ff33006eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff33c007900 tx=0x7ff33c008040 comp rx=0 tx=0).stop 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 msgr2=0x7ff348073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff33400bda0 tx=0x7ff33400be80 comp rx=0 tx=0).stop 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 shutdown_connections 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff33006c680 0x7ff33006eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3481004c0 0x7ff348073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 --2- 172.21.15.57:0/4127500115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff348100e60 0x7ff348073590 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 >> 172.21.15.57:0/4127500115 conn(0x7ff3480fbd80 msgr2=0x7ff3480fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 shutdown_connections 2024-02-28T23:02:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:48.878+0000 7ff350600700 1 -- 172.21.15.57:0/4127500115 wait complete. 2024-02-28T23:02:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:49.175 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:49.189 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:50.033 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:49 smithi057 ceph-mon[29096]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:50.033 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:49 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4127500115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-28T23:02:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:02:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:49 smithi084 ceph-mon[34826]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:49 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/4127500115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-28T23:02:50.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855377 got 150323855378 for osd.4 2024-02-28T23:02:50.084 DEBUG:teuthology.parallel:result is None 2024-02-28T23:02:51.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:51 smithi084 ceph-mon[34826]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:51 smithi057 ceph-mon[29096]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 -- 172.21.15.57:0/371170618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 msgr2=0x7fbcf81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 --2- 172.21.15.57:0/371170618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81028e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009910 tx=0x7fbce4009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 -- 172.21.15.57:0/371170618 shutdown_connections 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 --2- 172.21.15.57:0/371170618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81028e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 --2- 172.21.15.57:0/371170618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 0x7fbcf8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 -- 172.21.15.57:0/371170618 >> 172.21.15.57:0/371170618 conn(0x7fbcf80fdf90 msgr2=0x7fbcf81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 -- 172.21.15.57:0/371170618 shutdown_connections 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.169+0000 7fbcfe400700 1 -- 172.21.15.57:0/371170618 wait complete. 2024-02-28T23:02:53.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 Processor -- start 2024-02-28T23:02:53.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 -- start start 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 0x7fbcf8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8198d80 con 0x7fbcf8102470 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.170+0000 7fbcfe400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8198ec0 con 0x7fbcf8108470 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56048/0 (socket says 172.21.15.57:56048) 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 -- 172.21.15.57:0/3490075377 learned_addr learned my addr 172.21.15.57:0/3490075377 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf6a00700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 0x7fbcf8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 -- 172.21.15.57:0/3490075377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 msgr2=0x7fbcf8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:53.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 0x7fbcf8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 -- 172.21.15.57:0/3490075377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40095f0 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.171+0000 7fbcf7400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c930 tx=0x7fbce800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.172+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce8007ad0 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.172+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbce8007c30 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.172+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce8018630 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.172+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf819cd10 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.172+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf819d260 con 0x7fbcf8102470 2024-02-28T23:02:53.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.173+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf810a970 con 0x7fbcf8102470 2024-02-28T23:02:53.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.174+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbce800f450 con 0x7fbcf8102470 2024-02-28T23:02:53.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.176+0000 7fbcefe00700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 0x7fbcdc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:53.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.176+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbce808b710 con 0x7fbcf8102470 2024-02-28T23:02:53.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.176+0000 7fbcf6a00700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 0x7fbcdc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:53.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.177+0000 7fbcf6a00700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 0x7fbcdc06eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbce4000c00 tx=0x7fbce4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:53.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.181+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbce8056110 con 0x7fbcf8102470 2024-02-28T23:02:53.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.409+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbcf804f2a0 con 0x7fbcf8102470 2024-02-28T23:02:53.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.410+0000 7fbcefe00700 1 -- 172.21.15.57:0/3490075377 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbce8059730 con 0x7fbcf8102470 2024-02-28T23:02:53.408 INFO:teuthology.orchestra.run.smithi057.stdout:171798691851 2024-02-28T23:02:53.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 msgr2=0x7fbcdc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:53.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 0x7fbcdc06eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbce4000c00 tx=0x7fbce4005fd0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 msgr2=0x7fbcf81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c930 tx=0x7fbce800ccf0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 shutdown_connections 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fbcdc06c680 0x7fbcdc06eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcf8102470 0x7fbcf81981f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 --2- 172.21.15.57:0/3490075377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108470 0x7fbcf8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.412+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 >> 172.21.15.57:0/3490075377 conn(0x7fbcf80fdf90 msgr2=0x7fbcf80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.413+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 shutdown_connections 2024-02-28T23:02:53.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:53.413+0000 7fbcfe400700 1 -- 172.21.15.57:0/3490075377 wait complete. 2024-02-28T23:02:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:53 smithi057 ceph-mon[29096]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:53 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3490075377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-28T23:02:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:53 smithi084 ceph-mon[34826]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:53 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3490075377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-28T23:02:54.313 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-02-28T23:02:54.313 DEBUG:teuthology.parallel:result is None 2024-02-28T23:02:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:55 smithi057 ceph-mon[29096]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:55 smithi084 ceph-mon[34826]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:56.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.582+0000 7f54dca00700 1 -- 172.21.15.57:0/1961418736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8068540 msgr2=0x7f54d80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:56.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.582+0000 7f54dca00700 1 --2- 172.21.15.57:0/1961418736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8068540 0x7f54d80689b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009910 tx=0x7f54c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:56.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 -- 172.21.15.57:0/1961418736 shutdown_connections 2024-02-28T23:02:56.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 --2- 172.21.15.57:0/1961418736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8068540 0x7f54d80689b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 --2- 172.21.15.57:0/1961418736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8101070 0x7f54d8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 -- 172.21.15.57:0/1961418736 >> 172.21.15.57:0/1961418736 conn(0x7f54d8075130 msgr2=0x7f54d8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:56.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 -- 172.21.15.57:0/1961418736 shutdown_connections 2024-02-28T23:02:56.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.583+0000 7f54dca00700 1 -- 172.21.15.57:0/1961418736 wait complete. 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 Processor -- start 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 -- start start 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8068540 0x7f54d8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8198c90 con 0x7f54d8101070 2024-02-28T23:02:56.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.584+0000 7f54dca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d819ca20 con 0x7f54d8068540 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52230/0 (socket says 172.21.15.57:52230) 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 -- 172.21.15.57:0/3094573083 learned_addr learned my addr 172.21.15.57:0/3094573083 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5e00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8068540 0x7f54d8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 -- 172.21.15.57:0/3094573083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8068540 msgr2=0x7f54d8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8068540 0x7f54d8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 -- 172.21.15.57:0/3094573083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c40095f0 con 0x7f54d8101070 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.585+0000 7f54d5400700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54c400b5c0 tx=0x7f54c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.586+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c401d070 con 0x7f54d8101070 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.586+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54c4004fa0 con 0x7f54d8101070 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.586+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c400f460 con 0x7f54d8101070 2024-02-28T23:02:56.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.586+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d819cca0 con 0x7f54d8101070 2024-02-28T23:02:56.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.586+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d819d190 con 0x7f54d8101070 2024-02-28T23:02:56.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.587+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d804f2a0 con 0x7f54d8101070 2024-02-28T23:02:56.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.588+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f54c4022810 con 0x7f54d8101070 2024-02-28T23:02:56.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.589+0000 7f54cea00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 0x7f54bc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:56.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.589+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f54c408c970 con 0x7f54d8101070 2024-02-28T23:02:56.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.589+0000 7f54d5e00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 0x7f54bc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:56.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.590+0000 7f54d5e00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 0x7f54bc06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005950 tx=0x7f54c80058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:56.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.594+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f54c40574c0 con 0x7f54d8101070 2024-02-28T23:02:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.845+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f54d819d4a0 con 0x7f54d8101070 2024-02-28T23:02:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.846+0000 7f54cea00700 1 -- 172.21.15.57:0/3094573083 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54c4027090 con 0x7f54d8101070 2024-02-28T23:02:56.845 INFO:teuthology.orchestra.run.smithi057.stdout:81604378658 2024-02-28T23:02:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 msgr2=0x7f54bc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 0x7f54bc06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005950 tx=0x7f54c80058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 msgr2=0x7f54d81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54c400b5c0 tx=0x7f54c400be20 comp rx=0 tx=0).stop 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 shutdown_connections 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f54bc06c680 0x7f54bc06eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d8068540 0x7f54d8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 --2- 172.21.15.57:0/3094573083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54d8101070 0x7f54d81985b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 >> 172.21.15.57:0/3094573083 conn(0x7f54d8075130 msgr2=0x7f54d80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 shutdown_connections 2024-02-28T23:02:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:56.849+0000 7f54dca00700 1 -- 172.21.15.57:0/3094573083 wait complete. 2024-02-28T23:02:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:57 smithi057 ceph-mon[29096]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3094573083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-28T23:02:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1468020119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-28T23:02:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 -- 172.21.15.57:0/2386927094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0ff8f0 msgr2=0x7fe66c103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 --2- 172.21.15.57:0/2386927094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c103dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe658009910 tx=0x7fe658009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 -- 172.21.15.57:0/2386927094 shutdown_connections 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 --2- 172.21.15.57:0/2386927094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c103dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 --2- 172.21.15.57:0/2386927094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0fefe0 0x7fe66c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 -- 172.21.15.57:0/2386927094 >> 172.21.15.57:0/2386927094 conn(0x7fe66c0faa10 msgr2=0x7fe66c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.073+0000 7fe672400700 1 -- 172.21.15.57:0/2386927094 shutdown_connections 2024-02-28T23:02:58.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.074+0000 7fe672400700 1 -- 172.21.15.57:0/2386927094 wait complete. 2024-02-28T23:02:58.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 Processor -- start 2024-02-28T23:02:58.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 -- start start 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19cd80 con 0x7fe66c0fefe0 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe672400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19cec0 con 0x7fe66c0ff8f0 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.075+0000 7fe671000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52272/0 (socket says 172.21.15.57:52272) 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 -- 172.21.15.57:0/3020894720 learned_addr learned my addr 172.21.15.57:0/3020894720 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe66be00700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 -- 172.21.15.57:0/3020894720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0ff8f0 msgr2=0x7fe66c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 -- 172.21.15.57:0/3020894720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580095f0 con 0x7fe66c0fefe0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.076+0000 7fe671000700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c930 tx=0x7fe66000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.077+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe660007ad0 con 0x7fe66c0fefe0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.077+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe660007c30 con 0x7fe66c0fefe0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.077+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe660018630 con 0x7fe66c0fefe0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.077+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c1a0d10 con 0x7fe66c0fefe0 2024-02-28T23:02:58.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.077+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c1a1210 con 0x7fe66c0fefe0 2024-02-28T23:02:58.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.078+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c101460 con 0x7fe66c0fefe0 2024-02-28T23:02:58.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.079+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe66000f450 con 0x7fe66c0fefe0 2024-02-28T23:02:58.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.080+0000 7fe669600700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 0x7fe65406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.080+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe66008b670 con 0x7fe66c0fefe0 2024-02-28T23:02:58.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.081+0000 7fe66be00700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 0x7fe65406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:57 smithi084 ceph-mon[34826]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3094573083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-28T23:02:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1468020119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-28T23:02:58.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.081+0000 7fe66be00700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 0x7fe65406eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe66c19d810 tx=0x7fe65801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:58.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.085+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe660056110 con 0x7fe66c0fefe0 2024-02-28T23:02:58.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378655 got 81604378658 for osd.2 2024-02-28T23:02:58.183 DEBUG:teuthology.parallel:result is None 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.344+0000 7f77a7e00700 1 -- 172.21.15.57:0/1188513619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 msgr2=0x7f77a81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.344+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1188513619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a81020a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7794009910 tx=0x7f7794009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 -- 172.21.15.57:0/1188513619 shutdown_connections 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1188513619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a81020a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1188513619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a81007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 -- 172.21.15.57:0/1188513619 >> 172.21.15.57:0/1188513619 conn(0x7f77a80fbbb0 msgr2=0x7f77a80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 -- 172.21.15.57:0/1188513619 shutdown_connections 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.345+0000 7f77a7e00700 1 -- 172.21.15.57:0/1188513619 wait complete. 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 Processor -- start 2024-02-28T23:02:58.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 -- start start 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8198af0 con 0x7f77a8101c30 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.346+0000 7f77a7e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a819c880 con 0x7f77a81003d0 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52254/0 (socket says 172.21.15.57:52254) 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 -- 172.21.15.57:0/1468020119 learned_addr learned my addr 172.21.15.57:0/1468020119 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6a00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 -- 172.21.15.57:0/1468020119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 msgr2=0x7f77a8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6000700 1 -- 172.21.15.57:0/1468020119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940095f0 con 0x7f77a8101c30 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.347+0000 7f77a6a00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:02:58.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f77a6000700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7794006010 tx=0x7f779400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f779401d070 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7794004f50 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a819cb00 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7794021410 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.348+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a819cff0 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.349+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a8062380 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.350+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f779400f460 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.352+0000 7f779f400700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 0x7f778c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.352+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f779408c8a0 con 0x7f77a8101c30 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.352+0000 7f77a6a00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 0x7f778c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.353+0000 7f77a6a00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 0x7f778c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7798005950 tx=0x7f77980058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:58.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.356+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7794057220 con 0x7f77a8101c30 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stdout:60129542184 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.584+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77a8199230 con 0x7f77a8101c30 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.584+0000 7f779f400700 1 -- 172.21.15.57:0/1468020119 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7794026090 con 0x7f77a8101c30 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 msgr2=0x7f778c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 0x7f778c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7798005950 tx=0x7f77980058e0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 msgr2=0x7f77a8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7794006010 tx=0x7f779400be20 comp rx=0 tx=0).stop 2024-02-28T23:02:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 shutdown_connections 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f778c06c680 0x7f778c06eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77a81003d0 0x7f77a8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.587+0000 7f77a7e00700 1 --2- 172.21.15.57:0/1468020119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77a8101c30 0x7f77a8198410 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.588+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 >> 172.21.15.57:0/1468020119 conn(0x7f77a80fbbb0 msgr2=0x7f77a8072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.588+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 shutdown_connections 2024-02-28T23:02:58.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:57.588+0000 7f77a7e00700 1 -- 172.21.15.57:0/1468020119 wait complete. 2024-02-28T23:02:58.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.315+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe66c062380 con 0x7fe66c0fefe0 2024-02-28T23:02:58.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.316+0000 7fe669600700 1 -- 172.21.15.57:0/3020894720 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe660059730 con 0x7fe66c0fefe0 2024-02-28T23:02:58.314 INFO:teuthology.orchestra.run.smithi057.stdout:38654705712 2024-02-28T23:02:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 msgr2=0x7fe65406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 0x7fe65406eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe66c19d810 tx=0x7fe65801a040 comp rx=0 tx=0).stop 2024-02-28T23:02:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 msgr2=0x7fe66c19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c930 tx=0x7fe66000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 shutdown_connections 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe65406c680 0x7fe65406eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe66c0fefe0 0x7fe66c19c1f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 --2- 172.21.15.57:0/3020894720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0ff8f0 0x7fe66c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 >> 172.21.15.57:0/3020894720 conn(0x7fe66c0faa10 msgr2=0x7fe66c110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 shutdown_connections 2024-02-28T23:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:58.319+0000 7fe672400700 1 -- 172.21.15.57:0/3020894720 wait complete. 2024-02-28T23:02:58.946 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:58 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3020894720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-28T23:02:58.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542181 got 60129542184 for osd.1 2024-02-28T23:02:58.990 DEBUG:teuthology.parallel:result is None 2024-02-28T23:02:59.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:58 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3020894720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-28T23:02:59.838 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705708 got 38654705712 for osd.0 2024-02-28T23:02:59.838 DEBUG:teuthology.parallel:result is None 2024-02-28T23:02:59.860 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:02:59 smithi057 ceph-mon[29096]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 -- 172.21.15.57:0/3874326082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54280730f0 msgr2=0x7f54280734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 --2- 172.21.15.57:0/3874326082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54280730f0 0x7f54280734c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f541c009910 tx=0x7f541c009c20 comp rx=0 tx=0).stop 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 -- 172.21.15.57:0/3874326082 shutdown_connections 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 --2- 172.21.15.57:0/3874326082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5428073a00 0x7f542810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 --2- 172.21.15.57:0/3874326082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54280730f0 0x7f54280734c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 -- 172.21.15.57:0/3874326082 >> 172.21.15.57:0/3874326082 conn(0x7f54280fbda0 msgr2=0x7f54280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 -- 172.21.15.57:0/3874326082 shutdown_connections 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.601+0000 7f542f600700 1 -- 172.21.15.57:0/3874326082 wait complete. 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.602+0000 7f542f600700 1 Processor -- start 2024-02-28T23:02:59.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f542f600700 1 -- start start 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f542f600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54280730f0 0x7f5428197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f542f600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f542f600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542819c970 con 0x7f5428073a00 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f542f600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542819cae0 con 0x7f54280730f0 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f5427e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f5427e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52280/0 (socket says 172.21.15.57:52280) 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.603+0000 7f5427e00700 1 -- 172.21.15.57:0/309561688 learned_addr learned my addr 172.21.15.57:0/309561688 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f542cc00700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54280730f0 0x7f5428197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5427e00700 1 -- 172.21.15.57:0/309561688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54280730f0 msgr2=0x7f5428197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5427e00700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54280730f0 0x7f5428197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5427e00700 1 -- 172.21.15.57:0/309561688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0095f0 con 0x7f5428073a00 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5427e00700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f541400d8d0 tx=0x7f541400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:59.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5414009940 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5414010460 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541400f5f0 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.604+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542819cdc0 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.605+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542819d310 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.606+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5428109c80 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.607+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f54140105d0 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.607+0000 7f5425600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 0x7f541006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.607+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f541408b510 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.608+0000 7f542cc00700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 0x7f541006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.608+0000 7f542cc00700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 0x7f541006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f541c006010 tx=0x7f541c01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.612+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5414055f10 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.840+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5428199320 con 0x7f5428073a00 2024-02-28T23:02:59.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.841+0000 7f5425600700 1 -- 172.21.15.57:0/309561688 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f541405de60 con 0x7f5428073a00 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stdout:115964117018 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.843+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 msgr2=0x7f541006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.843+0000 7f542f600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 0x7f541006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f541c006010 tx=0x7f541c01a040 comp rx=0 tx=0).stop 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.843+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 msgr2=0x7f5428198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.843+0000 7f542f600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f541400d8d0 tx=0x7f541400dc90 comp rx=0 tx=0).stop 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 shutdown_connections 2024-02-28T23:02:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f541006c680 0x7f541006eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54280730f0 0x7f5428197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 --2- 172.21.15.57:0/309561688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5428073a00 0x7f5428198630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:02:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 >> 172.21.15.57:0/309561688 conn(0x7f54280fbda0 msgr2=0x7f5428106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:02:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 shutdown_connections 2024-02-28T23:02:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:02:59.844+0000 7f542f600700 1 -- 172.21.15.57:0/309561688 wait complete. 2024-02-28T23:03:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:02:59 smithi084 ceph-mon[34826]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:00.741 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:00 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/309561688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-28T23:03:00.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117016 got 115964117018 for osd.3 2024-02-28T23:03:00.786 DEBUG:teuthology.parallel:result is None 2024-02-28T23:03:00.786 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-28T23:03:00.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-28T23:03:01.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:00 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/309561688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-28T23:03:01.271 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:01.851 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:01 smithi057 ceph-mon[29096]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:01 smithi084 ceph-mon[34826]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 -- 172.21.15.57:0/2024788379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0440739a0 msgr2=0x7fc04410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 --2- 172.21.15.57:0/2024788379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0440739a0 0x7fc04410c5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009940 tx=0x7fc02c009c50 comp rx=0 tx=0).stop 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 -- 172.21.15.57:0/2024788379 shutdown_connections 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 --2- 172.21.15.57:0/2024788379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0440739a0 0x7fc04410c5e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 --2- 172.21.15.57:0/2024788379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044073090 0x7fc044073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.369+0000 7fc04b400700 1 -- 172.21.15.57:0/2024788379 >> 172.21.15.57:0/2024788379 conn(0x7fc0440fbda0 msgr2=0x7fc0440fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.370+0000 7fc04b400700 1 -- 172.21.15.57:0/2024788379 shutdown_connections 2024-02-28T23:03:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.370+0000 7fc04b400700 1 -- 172.21.15.57:0/2024788379 wait complete. 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 Processor -- start 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 -- start start 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0440739a0 0x7fc0441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044198dd0 con 0x7fc044073090 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.371+0000 7fc04b400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04419cb60 con 0x7fc0440739a0 2024-02-28T23:03:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52306/0 (socket says 172.21.15.57:52306) 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 -- 172.21.15.57:0/3678698427 learned_addr learned my addr 172.21.15.57:0/3678698427 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc03be00700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0440739a0 0x7fc0441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 -- 172.21.15.57:0/3678698427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0440739a0 msgr2=0x7fc0441986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0440739a0 0x7fc0441986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 -- 172.21.15.57:0/3678698427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0095f0 con 0x7fc044073090 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.372+0000 7fc048a00700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d8d0 tx=0x7fc03400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.373+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc034009940 con 0x7fc044073090 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.373+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc034010460 con 0x7fc044073090 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.373+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04419ce40 con 0x7fc044073090 2024-02-28T23:03:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.373+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03400f5f0 con 0x7fc044073090 2024-02-28T23:03:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.373+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04419d390 con 0x7fc044073090 2024-02-28T23:03:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.374+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc044109ce0 con 0x7fc044073090 2024-02-28T23:03:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.375+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc0340105d0 con 0x7fc044073090 2024-02-28T23:03:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.376+0000 7fc042000700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 0x7fc02806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.376+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc03408b5e0 con 0x7fc044073090 2024-02-28T23:03:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.376+0000 7fc03be00700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 0x7fc02806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.377+0000 7fc03be00700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 0x7fc02806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b840 tx=0x7fc02c00b790 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.381+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc034055f60 con 0x7fc044073090 2024-02-28T23:03:03.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.596+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc044199620 con 0x7fc02806c680 2024-02-28T23:03:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.599+0000 7fc042000700 1 -- 172.21.15.57:0/3678698427 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27075 (secure 0 0 0) 0x7fc044199620 con 0x7fc02806c680 2024-02-28T23:03:03.597 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 msgr2=0x7fc02806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 0x7fc02806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b840 tx=0x7fc02c00b790 comp rx=0 tx=0).stop 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 msgr2=0x7fc0441981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d8d0 tx=0x7fc03400dc90 comp rx=0 tx=0).stop 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 shutdown_connections 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc02806c680 0x7fc02806eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044073090 0x7fc0441981b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.601+0000 7fc04b400700 1 --2- 172.21.15.57:0/3678698427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0440739a0 0x7fc0441986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.602+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 >> 172.21.15.57:0/3678698427 conn(0x7fc0440fbda0 msgr2=0x7fc044106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.602+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 shutdown_connections 2024-02-28T23:03:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:03.602+0000 7fc04b400700 1 -- 172.21.15.57:0/3678698427 wait complete. 2024-02-28T23:03:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-02-28T23:03:03.968 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:03 smithi057 ceph-mon[29096]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:03 smithi084 ceph-mon[34826]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:04.372 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-02-28T23:03:02.542759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003266"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-28T23:02:04.255550+0000","last_change":"2024-02-28T23:01:29.637806+0000","last_active":"2024-02-28T23:02:04.255550+0000","last_peered":"2024-02-28T23:02:04.255550+0000","last_clean":"2024-02-28T23:02:04.255550+0000","last_became_active":"2024-02-28T23:01:29.637298+0000","last_became_peered":"2024-02-28T23:01:29.637298+0000","last_unstale":"2024-02-28T23:02:04.255550+0000","last_undegraded":"2024-02-28T23:02:04.255550+0000","last_fullsized":"2024-02-28T23:02:04.255550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-28T23:00:25.532302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-28T23:00:25.532302+0000","last_clean_scrub_stamp":"2024-02-28T23:00:25.532302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-01T00:51:40.569324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]},{"osd":4,"up_from":35,"seq":150323855381,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.93200000000000005}]},{"osd":1,"last update":"Wed Feb 28 23:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.879}]},{"osd":2,"last update":"Wed Feb 28 23:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.02}]},{"osd":4,"last update":"Wed Feb 28 23:02:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89000000000000001,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.7669999999999999,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.91900000000000004,"15min":0.90700000000000003},"min":{"1min":0.60799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.98599999999999999}]},{"osd":1,"last update":"Wed Feb 28 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.90000000000000002},"min":{"1min":0.65500000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.171,"5min":1.827,"15min":1.827},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.01,"5min":1.008,"15min":1.008},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3280000000000001,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Wed Feb 28 23:01:57 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.159},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":7.9409999999999998,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.069}]},{"osd":4,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 28 23:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.497,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2470000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.53000000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2070000000000001,"5min":1.673,"15min":1.673},"last":1}]},{"osd":2,"last update":"Wed Feb 28 23:02:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.92000000000000004,"15min":0.90600000000000003},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.876},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0029999999999999,"15min":0.99099999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.208}]},{"osd":3,"last update":"Wed Feb 28 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.615},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.1759999999999999}]},{"osd":4,"last update":"Wed Feb 28 23:02:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":1.107,"5min":1.107,"15min":1.107},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.5289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.74299999999999999,"15min":0.69799999999999995},"min":{"1min":0.55600000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.73799999999999999},"min":{"1min":0.61099999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.2030000000000001}]},{"osd":2,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.88},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.99299999999999999,"15min":0.98399999999999999},"min":{"1min":0.67300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0920000000000001}]},{"osd":3,"last update":"Wed Feb 28 23:02:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0960000000000001,"15min":1.1040000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3320000000000001,"5min":1.452,"15min":1.452},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.198,"15min":1.198},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.373},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-28T23:03:04.373 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-28T23:03:04.818 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:04.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:04 smithi057 ceph-mon[29096]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-28T23:03:05.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:04 smithi084 ceph-mon[34826]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-28T23:03:06.013 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:05 smithi057 ceph-mon[29096]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:05 smithi084 ceph-mon[34826]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 -- 172.21.15.57:0/1706623005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 msgr2=0x7f3cf4104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 --2- 172.21.15.57:0/1706623005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4104c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009910 tx=0x7f3ce4009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 -- 172.21.15.57:0/1706623005 shutdown_connections 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 --2- 172.21.15.57:0/1706623005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4104c10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 --2- 172.21.15.57:0/1706623005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf4100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 -- 172.21.15.57:0/1706623005 >> 172.21.15.57:0/1706623005 conn(0x7f3cf40fbb40 msgr2=0x7f3cf40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 -- 172.21.15.57:0/1706623005 shutdown_connections 2024-02-28T23:03:06.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.975+0000 7f3cfb200700 1 -- 172.21.15.57:0/1706623005 wait complete. 2024-02-28T23:03:06.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.976+0000 7f3cfb200700 1 Processor -- start 2024-02-28T23:03:06.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.977+0000 7f3cfb200700 1 -- start start 2024-02-28T23:03:06.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.977+0000 7f3cfb200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf41929a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.977+0000 7f3cfb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.977+0000 7f3cfb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4196aa0 con 0x7f3cf4100c40 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.977+0000 7f3cfb200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4193420 con 0x7f3cf41002a0 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53430/0 (socket says 172.21.15.57:53430) 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 -- 172.21.15.57:0/2318429687 learned_addr learned my addr 172.21.15.57:0/2318429687 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9e00700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf41929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 -- 172.21.15.57:0/2318429687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 msgr2=0x7f3cf41929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf41929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 -- 172.21.15.57:0/2318429687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40095f0 con 0x7f3cf4100c40 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9e00700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf41929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.978+0000 7f3cf9400700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400bda0 tx=0x7f3ce400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.979+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce401d070 con 0x7f3cf4100c40 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.979+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3ce4022470 con 0x7f3cf4100c40 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.979+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf41936a0 con 0x7f3cf4100c40 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.979+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce400f690 con 0x7f3cf4100c40 2024-02-28T23:03:06.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.979+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a6570 con 0x7f3cf4100c40 2024-02-28T23:03:06.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.980+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd8005320 con 0x7f3cf4100c40 2024-02-28T23:03:06.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.981+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3ce4004c50 con 0x7f3cf4100c40 2024-02-28T23:03:06.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.982+0000 7f3ceea00700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 0x7f3cdc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:06.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.982+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3ce408cad0 con 0x7f3cf4100c40 2024-02-28T23:03:06.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.983+0000 7f3cf9e00700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 0x7f3cdc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:06.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.983+0000 7f3cf9e00700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 0x7f3cdc06eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005fd0 tx=0x7f3ce8005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:06.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:06.988+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3ce4057500 con 0x7f3cf4100c40 2024-02-28T23:03:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.205+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3cd8000bf0 con 0x7f3cdc06c680 2024-02-28T23:03:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.207+0000 7f3ceea00700 1 -- 172.21.15.57:0/2318429687 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27151 (secure 0 0 0) 0x7f3cd8000bf0 con 0x7f3cdc06c680 2024-02-28T23:03:07.206 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 msgr2=0x7f3cdc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 0x7f3cdc06eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005fd0 tx=0x7f3ce8005e30 comp rx=0 tx=0).stop 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 msgr2=0x7f3cf4192ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400bda0 tx=0x7f3ce400be80 comp rx=0 tx=0).stop 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 shutdown_connections 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cdc06c680 0x7f3cdc06eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cf41002a0 0x7f3cf41929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:07.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 --2- 172.21.15.57:0/2318429687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cf4100c40 0x7f3cf4192ee0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:07.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 >> 172.21.15.57:0/2318429687 conn(0x7f3cf40fbb40 msgr2=0x7f3cf40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:07.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 shutdown_connections 2024-02-28T23:03:07.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:07.211+0000 7f3cfb200700 1 -- 172.21.15.57:0/2318429687 wait complete. 2024-02-28T23:03:07.211 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-02-28T23:03:07.930 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:07 smithi057 ceph-mon[29096]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:07.930 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:07.974 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-02-28T23:03:06.544148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003504"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-28T23:02:04.255550+0000","last_change":"2024-02-28T23:01:29.637806+0000","last_active":"2024-02-28T23:02:04.255550+0000","last_peered":"2024-02-28T23:02:04.255550+0000","last_clean":"2024-02-28T23:02:04.255550+0000","last_became_active":"2024-02-28T23:01:29.637298+0000","last_became_peered":"2024-02-28T23:01:29.637298+0000","last_unstale":"2024-02-28T23:02:04.255550+0000","last_undegraded":"2024-02-28T23:02:04.255550+0000","last_fullsized":"2024-02-28T23:02:04.255550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-28T23:00:25.532302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-28T23:00:25.532302+0000","last_clean_scrub_stamp":"2024-02-28T23:00:25.532302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-01T00:51:40.569324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]}]},{"osd":4,"up_from":35,"seq":150323855382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.2689999999999999}]},{"osd":1,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.125}]},{"osd":2,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.373}]},{"osd":3,"last update":"Wed Feb 28 23:02:31 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96799999999999997,"15min":0.96399999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.54,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.034,"5min":0.96999999999999997,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.3779999999999999}]},{"osd":1,"last update":"Wed Feb 28 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97099999999999997,"15min":0.96999999999999997},"min":{"1min":0.75,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.198},{"interface":"front","average":{"1min":1.056,"5min":1.02,"15min":1.014},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.635,"5min":2.077,"15min":2.077},"last":1.2270000000000001}]},{"osd":2,"last update":"Wed Feb 28 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.98999999999999999,"15min":0.97799999999999998},"min":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0720000000000001,"15min":1.0669999999999999},"min":{"1min":0.76200000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.593,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.3220000000000001}]},{"osd":4,"last update":"Wed Feb 28 23:02:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378660,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89000000000000001,"15min":0.88800000000000001},"min":{"1min":0.64500000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.7669999999999999,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.91900000000000004,"15min":0.90700000000000003},"min":{"1min":0.60799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed Feb 28 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.90000000000000002},"min":{"1min":0.65500000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.171,"5min":1.827,"15min":1.827},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.01,"5min":1.008,"15min":1.008},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3280000000000001,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed Feb 28 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.165,"15min":1.1499999999999999},"min":{"1min":0.89200000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.4079999999999999,"15min":1.45},"min":{"1min":0.83199999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.579,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.149}]},{"osd":4,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.194},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 28 23:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.497,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2470000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.53000000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2070000000000001,"5min":1.673,"15min":1.673},"last":1.137}]},{"osd":2,"last update":"Wed Feb 28 23:02:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.92000000000000004,"15min":0.90600000000000003},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0029999999999999,"15min":0.99099999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.403}]},{"osd":3,"last update":"Wed Feb 28 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.054,"15min":1.044},"min":{"1min":0.83499999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.1899999999999999,"15min":1.1659999999999999},"min":{"1min":0.91200000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.585,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.4670000000000001}]},{"osd":4,"last update":"Wed Feb 28 23:02:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":1.107,"5min":1.107,"15min":1.107},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.3500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 28 23:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.74299999999999999,"15min":0.69799999999999995},"min":{"1min":0.55600000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.877},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.73799999999999999},"min":{"1min":0.61099999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.1040000000000001}]},{"osd":2,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.99299999999999999,"15min":0.98399999999999999},"min":{"1min":0.67300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Feb 28 23:02:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0960000000000001,"15min":1.1040000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3320000000000001,"5min":1.452,"15min":1.452},"last":1.397},{"interface":"front","average":{"1min":1.196,"5min":1.198,"15min":1.198},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.216}]},{"osd":4,"last update":"Wed Feb 28 23:02:32 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.25}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-28T23:03:07.975 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-28T23:03:07.975 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-28T23:03:07.975 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-28T23:03:07.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph health --format=json 2024-02-28T23:03:08.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:07 smithi084 ceph-mon[34826]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:08.402 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:08.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:08 smithi057 ceph-mon[29096]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-28T23:03:08.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:09.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:08 smithi084 ceph-mon[34826]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-28T23:03:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:09.921 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:09 smithi057 ceph-mon[29096]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:09.921 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:09 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:09.922 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:09 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:09.922 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:09 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:10.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:09 smithi084 ceph-mon[34826]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:09 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:09 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:09 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 -- 172.21.15.57:0/2488765023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4102470 msgr2=0x7feba41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 --2- 172.21.15.57:0/2488765023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4102470 0x7feba41028e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb90009910 tx=0x7feb90009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 -- 172.21.15.57:0/2488765023 shutdown_connections 2024-02-28T23:03:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 --2- 172.21.15.57:0/2488765023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4102470 0x7feba41028e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 --2- 172.21.15.57:0/2488765023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4108470 0x7feba4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.461+0000 7febab200700 1 -- 172.21.15.57:0/2488765023 >> 172.21.15.57:0/2488765023 conn(0x7feba40fdf90 msgr2=0x7feba41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.462+0000 7febab200700 1 -- 172.21.15.57:0/2488765023 shutdown_connections 2024-02-28T23:03:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.462+0000 7febab200700 1 -- 172.21.15.57:0/2488765023 wait complete. 2024-02-28T23:03:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 Processor -- start 2024-02-28T23:03:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 -- start start 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4102470 0x7feba41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba4198e10 con 0x7feba4108470 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.463+0000 7febab200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba419cba0 con 0x7feba4102470 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53454/0 (socket says 172.21.15.57:53454) 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 -- 172.21.15.57:0/3302856375 learned_addr learned my addr 172.21.15.57:0/3302856375 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3e00700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4102470 0x7feba41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 -- 172.21.15.57:0/3302856375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4102470 msgr2=0x7feba41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:10.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4102470 0x7feba41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.464+0000 7feba3400700 1 -- 172.21.15.57:0/3302856375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb900095f0 con 0x7feba4108470 2024-02-28T23:03:10.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7feba3400700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb90006010 tx=0x7feb9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9001d070 con 0x7feba4108470 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb90004f50 con 0x7feba4108470 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7febab200700 1 -- 172.21.15.57:0/3302856375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba419ce20 con 0x7feba4108470 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb90021640 con 0x7feba4108470 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.465+0000 7febab200700 1 -- 172.21.15.57:0/3302856375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba419d310 con 0x7feba4108470 2024-02-28T23:03:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.466+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba404f2a0 con 0x7feba4108470 2024-02-28T23:03:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.467+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7feb9000f460 con 0x7feba4108470 2024-02-28T23:03:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.468+0000 7feba0c00700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 0x7feb8806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.468+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feb9008c710 con 0x7feba4108470 2024-02-28T23:03:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.468+0000 7feba3e00700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 0x7feb8806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.469+0000 7feba3e00700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 0x7feb8806eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb94005950 tx=0x7feb940058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.473+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7feb90057110 con 0x7feba4108470 2024-02-28T23:03:10.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.740+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feba4199550 con 0x7feba4108470 2024-02-28T23:03:10.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.741+0000 7feba0c00700 1 -- 172.21.15.57:0/3302856375 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7feb9000fe60 con 0x7feba4108470 2024-02-28T23:03:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:10.739 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-28T23:03:10.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 msgr2=0x7feb8806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:10.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 0x7feb8806eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb94005950 tx=0x7feb940058e0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 msgr2=0x7feba4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb90006010 tx=0x7feb9000be20 comp rx=0 tx=0).stop 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 shutdown_connections 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb8806c680 0x7feb8806eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba4102470 0x7feba41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 --2- 172.21.15.57:0/3302856375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feba4108470 0x7feba4198730 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.744+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 >> 172.21.15.57:0/3302856375 conn(0x7feba40fdf90 msgr2=0x7feba40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.745+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 shutdown_connections 2024-02-28T23:03:10.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:10.745+0000 7feb9a000700 1 -- 172.21.15.57:0/3302856375 wait complete. 2024-02-28T23:03:11.508 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:11 smithi084 ceph-mon[34826]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:11.508 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:11 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3302856375' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-28T23:03:11.511 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:11 smithi057 ceph-mon[29096]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:11.512 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:11 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3302856375' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-28T23:03:11.604 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-28T23:03:11.604 INFO:tasks.cephadm:Setup complete, yielding 2024-02-28T23:03:11.604 INFO:teuthology.run_tasks:Running task print... 2024-02-28T23:03:11.617 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-02-28T23:03:11.617 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:11.664 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:11.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-28T23:03:12.093 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:13.930 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:13 smithi057 ceph-mon[29096]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:13 smithi084 ceph-mon[34826]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:14.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 -- 172.21.15.57:0/16672321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8069270 msgr2=0x7f4ee81051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:14.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 --2- 172.21.15.57:0/16672321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8069270 0x7f4ee81051d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009910 tx=0x7f4ed8009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:14.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 -- 172.21.15.57:0/16672321 shutdown_connections 2024-02-28T23:03:14.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 --2- 172.21.15.57:0/16672321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8105710 0x7f4ee8107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 --2- 172.21.15.57:0/16672321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8069270 0x7f4ee81051d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 -- 172.21.15.57:0/16672321 >> 172.21.15.57:0/16672321 conn(0x7f4ee80fa780 msgr2=0x7f4ee80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:14.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 -- 172.21.15.57:0/16672321 shutdown_connections 2024-02-28T23:03:14.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.485+0000 7f4eed600700 1 -- 172.21.15.57:0/16672321 wait complete. 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 Processor -- start 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 -- start start 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8198220 0x7f4ee819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8198690 con 0x7f4ee8105710 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.486+0000 7f4eed600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8198800 con 0x7f4ee8198220 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53470/0 (socket says 172.21.15.57:53470) 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 -- 172.21.15.57:0/3656987301 learned_addr learned my addr 172.21.15.57:0/3656987301 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6000700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8198220 0x7f4ee819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:14.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 -- 172.21.15.57:0/3656987301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8198220 msgr2=0x7f4ee819d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:14.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8198220 0x7f4ee819d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.487+0000 7f4ee6a00700 1 -- 172.21.15.57:0/3656987301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80095f0 con 0x7f4ee8105710 2024-02-28T23:03:14.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4ee6a00700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800bd70 tx=0x7f4ed800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed801d070 con 0x7f4ee8105710 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ed800f460 con 0x7f4ee8105710 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee819d790 con 0x7f4ee8105710 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed8021640 con 0x7f4ee8105710 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.488+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee819dc50 con 0x7f4ee8105710 2024-02-28T23:03:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.489+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee8191dd0 con 0x7f4ee8105710 2024-02-28T23:03:14.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.490+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4ed800fab0 con 0x7f4ee8105710 2024-02-28T23:03:14.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.491+0000 7f4edf400700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 0x7f4ecc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:14.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.491+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4ed808c970 con 0x7f4ee8105710 2024-02-28T23:03:14.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.491+0000 7f4ee6000700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 0x7f4ecc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:14.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.492+0000 7f4ee6000700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 0x7f4ecc06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005fd0 tx=0x7f4ed400a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:14.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.496+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4ed8057370 con 0x7f4ee8105710 2024-02-28T23:03:14.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.713+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f4ee8069270 con 0x7f4ee8105710 2024-02-28T23:03:14.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.727+0000 7f4edf400700 1 -- 172.21.15.57:0/3656987301 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4ed80260d0 con 0x7f4ee8105710 2024-02-28T23:03:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 msgr2=0x7f4ecc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 0x7f4ecc06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005fd0 tx=0x7f4ed400a560 comp rx=0 tx=0).stop 2024-02-28T23:03:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 msgr2=0x7f4ee8197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800bd70 tx=0x7f4ed800be50 comp rx=0 tx=0).stop 2024-02-28T23:03:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 shutdown_connections 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f4ecc06c680 0x7f4ecc06eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee8105710 0x7f4ee8197ce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 --2- 172.21.15.57:0/3656987301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ee8198220 0x7f4ee819d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.730+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 >> 172.21.15.57:0/3656987301 conn(0x7f4ee80fa780 msgr2=0x7f4ee80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.731+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 shutdown_connections 2024-02-28T23:03:14.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:14.731+0000 7f4eed600700 1 -- 172.21.15.57:0/3656987301 wait complete. 2024-02-28T23:03:15.630 INFO:teuthology.run_tasks:Running task print... 2024-02-28T23:03:15.639 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-28T23:03:15.639 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:15.675 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:15.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph orch status' 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3656987301' entity='client.admin' 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:15 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3656987301' entity='client.admin' 2024-02-28T23:03:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:15 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:16.097 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:17 smithi057 ceph-mon[29096]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:17 smithi084 ceph-mon[34826]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:18.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 -- 172.21.15.57:0/3275445130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 msgr2=0x7f7258100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:18.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 --2- 172.21.15.57:0/3275445130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258100650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7248009910 tx=0x7f7248009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 -- 172.21.15.57:0/3275445130 shutdown_connections 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 --2- 172.21.15.57:0/3275445130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 0x7f7258101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 --2- 172.21.15.57:0/3275445130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258100650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 -- 172.21.15.57:0/3275445130 >> 172.21.15.57:0/3275445130 conn(0x7f72580fb7f0 msgr2=0x7f72580fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 -- 172.21.15.57:0/3275445130 shutdown_connections 2024-02-28T23:03:18.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.349+0000 7f725e000700 1 -- 172.21.15.57:0/3275445130 wait complete. 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.350+0000 7f725e000700 1 Processor -- start 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.350+0000 7f725e000700 1 -- start start 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f725e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f725e000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 0x7f72581981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f725e000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72581987f0 con 0x7f7258100240 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f725e000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7258198960 con 0x7f7258101440 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:18.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55904/0 (socket says 172.21.15.57:55904) 2024-02-28T23:03:18.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 -- 172.21.15.57:0/970067033 learned_addr learned my addr 172.21.15.57:0/970067033 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7256a00700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 0x7f72581981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 -- 172.21.15.57:0/970067033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 msgr2=0x7f72581981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 0x7f72581981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.351+0000 7f7257400700 1 -- 172.21.15.57:0/970067033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480095f0 con 0x7f7258100240 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f7257400700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f724800bd70 tx=0x7f724800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f724801d070 con 0x7f7258100240 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f724800f460 con 0x7f7258100240 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7248021640 con 0x7f7258100240 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725819d390 con 0x7f7258100240 2024-02-28T23:03:18.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.352+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725819d880 con 0x7f7258100240 2024-02-28T23:03:18.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.353+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7258105520 con 0x7f7258100240 2024-02-28T23:03:18.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.354+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f724800fa90 con 0x7f7258100240 2024-02-28T23:03:18.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.355+0000 7f724fe00700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 0x7f723c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:18.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.355+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f724808c8d0 con 0x7f7258100240 2024-02-28T23:03:18.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.355+0000 7f7256a00700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 0x7f723c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:18.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.356+0000 7f7256a00700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 0x7f723c06eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:18.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.360+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7248057250 con 0x7f7258100240 2024-02-28T23:03:18.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.598+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7258061190 con 0x7f723c06c680 2024-02-28T23:03:18.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.600+0000 7f724fe00700 1 -- 172.21.15.57:0/970067033 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7258061190 con 0x7f723c06c680 2024-02-28T23:03:18.598 INFO:teuthology.orchestra.run.smithi057.stdout:Backend: cephadm 2024-02-28T23:03:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:Available: Yes 2024-02-28T23:03:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:Paused: No 2024-02-28T23:03:18.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 msgr2=0x7f723c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:18.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 0x7f723c06eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 msgr2=0x7f7258197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:18.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f724800bd70 tx=0x7f724800be50 comp rx=0 tx=0).stop 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 shutdown_connections 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f723c06c680 0x7f723c06eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7258100240 0x7f7258197cb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 --2- 172.21.15.57:0/970067033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7258101440 0x7f72581981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 >> 172.21.15.57:0/970067033 conn(0x7f72580fb7f0 msgr2=0x7f7258102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 shutdown_connections 2024-02-28T23:03:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:18.603+0000 7f725e000700 1 -- 172.21.15.57:0/970067033 wait complete. 2024-02-28T23:03:19.441 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-28T23:03:19.878 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:20.026 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:19 smithi057 ceph-mon[29096]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:20.026 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:19 smithi057 ceph-mon[29096]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:19 smithi084 ceph-mon[34826]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:19 smithi084 ceph-mon[34826]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:21 smithi057 ceph-mon[29096]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:21 smithi084 ceph-mon[34826]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 -- 172.21.15.57:0/4154649642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 msgr2=0x7f95c4106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 --2- 172.21.15.57:0/4154649642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c4106c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009910 tx=0x7f95b8009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 -- 172.21.15.57:0/4154649642 shutdown_connections 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 --2- 172.21.15.57:0/4154649642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 0x7f95c4107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 --2- 172.21.15.57:0/4154649642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c4106c50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 -- 172.21.15.57:0/4154649642 >> 172.21.15.57:0/4154649642 conn(0x7f95c4075970 msgr2=0x7f95c4077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 -- 172.21.15.57:0/4154649642 shutdown_connections 2024-02-28T23:03:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.950+0000 7f95cbc00700 1 -- 172.21.15.57:0/4154649642 wait complete. 2024-02-28T23:03:21.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 Processor -- start 2024-02-28T23:03:21.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 -- start start 2024-02-28T23:03:21.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:21.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 0x7f95c419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:21.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c419cc60 con 0x7f95c4106840 2024-02-28T23:03:21.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95cbc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c419cda0 con 0x7f95c4107a40 2024-02-28T23:03:21.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95c9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:21.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95c9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55912/0 (socket says 172.21.15.57:55912) 2024-02-28T23:03:21.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.952+0000 7f95c9200700 1 -- 172.21.15.57:0/705380011 learned_addr learned my addr 172.21.15.57:0/705380011 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.953+0000 7f95c3e00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 0x7f95c419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.953+0000 7f95c9200700 1 -- 172.21.15.57:0/705380011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 msgr2=0x7f95c419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.953+0000 7f95c9200700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 0x7f95c419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.953+0000 7f95c9200700 1 -- 172.21.15.57:0/705380011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b80095f0 con 0x7f95c4106840 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.953+0000 7f95c9200700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f95b800bda0 tx=0x7f95b800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.954+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b801d070 con 0x7f95c4106840 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.954+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c41a17f0 con 0x7f95c4106840 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.954+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95b800f460 con 0x7f95c4106840 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.954+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b8021640 con 0x7f95c4106840 2024-02-28T23:03:21.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.954+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41a1ce0 con 0x7f95c4106840 2024-02-28T23:03:21.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.955+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c410f390 con 0x7f95c4106840 2024-02-28T23:03:21.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.956+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f95b800f5d0 con 0x7f95c4106840 2024-02-28T23:03:21.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.957+0000 7f95c1600700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 0x7f95ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:21.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.957+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f95b808c940 con 0x7f95c4106840 2024-02-28T23:03:21.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.958+0000 7f95c3e00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 0x7f95ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:21.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.958+0000 7f95c3e00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 0x7f95ac06eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005950 tx=0x7f95b00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:21.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:21.963+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f95b805ad50 con 0x7f95c4106840 2024-02-28T23:03:22.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.194+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95c41a2320 con 0x7f95ac06c680 2024-02-28T23:03:22.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.205+0000 7f95c1600700 1 -- 172.21.15.57:0/705380011 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f95c41a2320 con 0x7f95ac06c680 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (4m) 2m ago 6m 14.8M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (3m) 2m ago 7m 7050k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (5m) 79s ago 5m 6949k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (7m) 2m ago 7m 6665k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (5m) 79s ago 5m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (4m) 2m ago 6m 73.5M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (8m) 2m ago 8m 479M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (5m) 79s ago 5m 436M - 18.2.0 dc2bc1663786 4fe0352dacd9 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (8m) 2m ago 8m 44.8M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:03:22.205 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (5m) 79s ago 5m 33.4M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (7m) 2m ago 7m 7796k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (5m) 79s ago 5m 7512k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (4m) 2m ago 4m 40.0M 3949M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (3m) 2m ago 3m 39.9M 3949M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (3m) 2m ago 3m 39.9M 3949M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (2m) 79s ago 2m 40.9M 4971M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (119s) 79s ago 119s 38.3M 4971M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (85s) 79s ago 84s 11.0M 4971M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:03:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (4m) 2m ago 6m 40.0M - 2.43.0 a07b618ecd1d ffacfc83a2dc 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 msgr2=0x7f95ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 0x7f95ac06eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005950 tx=0x7f95b00058e0 comp rx=0 tx=0).stop 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 msgr2=0x7f95c419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f95b800bda0 tx=0x7f95b800be80 comp rx=0 tx=0).stop 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 shutdown_connections 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f95ac06c680 0x7f95ac06eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:22.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95c4106840 0x7f95c419c100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 --2- 172.21.15.57:0/705380011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c4107a40 0x7f95c419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 >> 172.21.15.57:0/705380011 conn(0x7f95c4075970 msgr2=0x7f95c410ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.209+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 shutdown_connections 2024-02-28T23:03:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:22.210+0000 7f95cbc00700 1 -- 172.21.15.57:0/705380011 wait complete. 2024-02-28T23:03:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:23.010 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-28T23:03:23.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:23.862 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:23 smithi057 ceph-mon[29096]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:23.863 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:23 smithi057 ceph-mon[29096]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:23 smithi084 ceph-mon[34826]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:23 smithi084 ceph-mon[34826]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 -- 172.21.15.57:0/1611661646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4103430 msgr2=0x7f88b4105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1611661646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4103430 0x7f88b4105810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f889c009910 tx=0x7f889c009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 -- 172.21.15.57:0/1611661646 shutdown_connections 2024-02-28T23:03:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1611661646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4103430 0x7f88b4105810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1611661646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4100b10 0x7f88b4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 -- 172.21.15.57:0/1611661646 >> 172.21.15.57:0/1611661646 conn(0x7f88b40fa4d0 msgr2=0x7f88b40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 -- 172.21.15.57:0/1611661646 shutdown_connections 2024-02-28T23:03:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.640+0000 7f88b3e00700 1 -- 172.21.15.57:0/1611661646 wait complete. 2024-02-28T23:03:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 Processor -- start 2024-02-28T23:03:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 -- start start 2024-02-28T23:03:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4103430 0x7f88b4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41984f0 con 0x7f88b4100b10 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.641+0000 7f88b3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4198630 con 0x7f88b4103430 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59928/0 (socket says 172.21.15.57:59928) 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 -- 172.21.15.57:0/1158787040 learned_addr learned my addr 172.21.15.57:0/1158787040 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88b2a00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4103430 0x7f88b4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 -- 172.21.15.57:0/1158787040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4103430 msgr2=0x7f88b4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4103430 0x7f88b4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.642+0000 7f88aaa00700 1 -- 172.21.15.57:0/1158787040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0095f0 con 0x7f88b4100b10 2024-02-28T23:03:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88aaa00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c930 tx=0x7f88a400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a4007ad0 con 0x7f88b4100b10 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88a4007c30 con 0x7f88b4100b10 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a4018630 con 0x7f88b4100b10 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b419d0f0 con 0x7f88b4100b10 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.643+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b419d610 con 0x7f88b4100b10 2024-02-28T23:03:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.644+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b40fc090 con 0x7f88b4100b10 2024-02-28T23:03:25.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.645+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f88a400f450 con 0x7f88b4100b10 2024-02-28T23:03:25.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.646+0000 7f88abe00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 0x7f889806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:25.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.646+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f88a408b710 con 0x7f88b4100b10 2024-02-28T23:03:25.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.646+0000 7f88b2a00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 0x7f889806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:25.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.647+0000 7f88b2a00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 0x7f889806eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f889c006010 tx=0x7f889c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:25.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.651+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f88a4059ba0 con 0x7f88b4100b10 2024-02-28T23:03:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:25 smithi057 ceph-mon[29096]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:25.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.881+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f88b4061190 con 0x7f889806c680 2024-02-28T23:03:25.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.894+0000 7f88abe00700 1 -- 172.21.15.57:0/1158787040 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7f88b4061190 con 0x7f889806c680 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter 2/2 2m ago 7m * 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:crash 2/2 2m ago 8m * 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 2/2 2m ago 8m count:2 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:mon 2/2 2m ago 6m smithi057:172.21.15.57=smithi057;smithi084:172.21.15.84=smithi084;count:2 2024-02-28T23:03:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-02-28T23:03:25.894 INFO:teuthology.orchestra.run.smithi057.stdout:osd 6 2m ago - 2024-02-28T23:03:25.894 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-02-28T23:03:25.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 msgr2=0x7f889806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:25.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 0x7f889806eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f889c006010 tx=0x7f889c00b540 comp rx=0 tx=0).stop 2024-02-28T23:03:25.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 msgr2=0x7f88b4197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c930 tx=0x7f88a400ccf0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 shutdown_connections 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f889806c680 0x7f889806eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88b4100b10 0x7f88b4197a20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 --2- 172.21.15.57:0/1158787040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b4103430 0x7f88b4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 >> 172.21.15.57:0/1158787040 conn(0x7f88b40fa4d0 msgr2=0x7f88b40ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 shutdown_connections 2024-02-28T23:03:25.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:25.897+0000 7f88b3e00700 1 -- 172.21.15.57:0/1158787040 wait complete. 2024-02-28T23:03:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:25 smithi084 ceph-mon[34826]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:26.759 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-28T23:03:27.188 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:27 smithi057 ceph-mon[29096]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:27 smithi057 ceph-mon[29096]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:27 smithi084 ceph-mon[34826]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:27 smithi084 ceph-mon[34826]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:29.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.470+0000 7fb840000700 1 -- 172.21.15.57:0/2305444867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838107a40 msgr2=0x7fb838107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:29.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.470+0000 7fb840000700 1 --2- 172.21.15.57:0/2305444867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838107a40 0x7fb838107e90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb824009910 tx=0x7fb824009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:29.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 -- 172.21.15.57:0/2305444867 shutdown_connections 2024-02-28T23:03:29.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 --2- 172.21.15.57:0/2305444867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838107a40 0x7fb838107e90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 --2- 172.21.15.57:0/2305444867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838106840 0x7fb838106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 -- 172.21.15.57:0/2305444867 >> 172.21.15.57:0/2305444867 conn(0x7fb838075970 msgr2=0x7fb838077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:29.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 -- 172.21.15.57:0/2305444867 shutdown_connections 2024-02-28T23:03:29.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.471+0000 7fb840000700 1 -- 172.21.15.57:0/2305444867 wait complete. 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.472+0000 7fb840000700 1 Processor -- start 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.472+0000 7fb840000700 1 -- start start 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.472+0000 7fb840000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb840000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838107a40 0x7fb83819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb840000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83819cbb0 con 0x7fb838106840 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb840000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83819ccf0 con 0x7fb838107a40 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59942/0 (socket says 172.21.15.57:59942) 2024-02-28T23:03:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 -- 172.21.15.57:0/570381451 learned_addr learned my addr 172.21.15.57:0/570381451 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83cc00700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838107a40 0x7fb83819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 -- 172.21.15.57:0/570381451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838107a40 msgr2=0x7fb83819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838107a40 0x7fb83819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.473+0000 7fb83d600700 1 -- 172.21.15.57:0/570381451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8240095f0 con 0x7fb838106840 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb83d600700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c910 tx=0x7fb82c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82c0041f0 con 0x7fb838106840 2024-02-28T23:03:29.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb82c007db0 con 0x7fb838106840 2024-02-28T23:03:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82c003d60 con 0x7fb838106840 2024-02-28T23:03:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8381a17b0 con 0x7fb838106840 2024-02-28T23:03:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.474+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8381a1cd0 con 0x7fb838106840 2024-02-28T23:03:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.475+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83810f390 con 0x7fb838106840 2024-02-28T23:03:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.477+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb82c007900 con 0x7fb838106840 2024-02-28T23:03:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.478+0000 7fb836000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 0x7fb82006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.478+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb82c08b220 con 0x7fb838106840 2024-02-28T23:03:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.478+0000 7fb83cc00700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 0x7fb82006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.479+0000 7fb83cc00700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 0x7fb82006eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb824006010 tx=0x7fb82400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:29.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.482+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb82c0596b0 con 0x7fb838106840 2024-02-28T23:03:29.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.710+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb83810c390 con 0x7fb82006c680 2024-02-28T23:03:29.710 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:29 smithi057 ceph-mon[29096]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:29.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.714+0000 7fb836000700 1 -- 172.21.15.57:0/570381451 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7fb83810c390 con 0x7fb82006c680 2024-02-28T23:03:29.713 INFO:teuthology.orchestra.run.smithi057.stdout:HOST ADDR LABELS STATUS 2024-02-28T23:03:29.713 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 172.21.15.57 2024-02-28T23:03:29.713 INFO:teuthology.orchestra.run.smithi057.stdout:smithi084 172.21.15.84 2024-02-28T23:03:29.713 INFO:teuthology.orchestra.run.smithi057.stdout:2 hosts in cluster 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.717+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 msgr2=0x7fb82006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.717+0000 7fb840000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 0x7fb82006eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb824006010 tx=0x7fb82400b540 comp rx=0 tx=0).stop 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.717+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 msgr2=0x7fb83819c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.717+0000 7fb840000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c910 tx=0x7fb82c00ccd0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 shutdown_connections 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb82006c680 0x7fb82006eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb838106840 0x7fb83819c0e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 --2- 172.21.15.57:0/570381451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb838107a40 0x7fb83819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 >> 172.21.15.57:0/570381451 conn(0x7fb838075970 msgr2=0x7fb83810ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:29.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 shutdown_connections 2024-02-28T23:03:29.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:29.718+0000 7fb840000700 1 -- 172.21.15.57:0/570381451 wait complete. 2024-02-28T23:03:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:29 smithi084 ceph-mon[34826]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:30.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-28T23:03:30.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:30 smithi057 ceph-mon[29096]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:31.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:30 smithi084 ceph-mon[34826]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:31 smithi057 ceph-mon[29096]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:31.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:31 smithi084 ceph-mon[34826]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 -- 172.21.15.57:0/3656237074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc100250 msgr2=0x7f50cc100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 --2- 172.21.15.57:0/3656237074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc100250 0x7f50cc100660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009910 tx=0x7f50c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 -- 172.21.15.57:0/3656237074 shutdown_connections 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 --2- 172.21.15.57:0/3656237074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc101450 0x7f50cc1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 --2- 172.21.15.57:0/3656237074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc100250 0x7f50cc100660 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 -- 172.21.15.57:0/3656237074 >> 172.21.15.57:0/3656237074 conn(0x7f50cc0fb7e0 msgr2=0x7f50cc0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 -- 172.21.15.57:0/3656237074 shutdown_connections 2024-02-28T23:03:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.180+0000 7f50d4800700 1 -- 172.21.15.57:0/3656237074 wait complete. 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.181+0000 7f50d4800700 1 Processor -- start 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.181+0000 7f50d4800700 1 -- start start 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d4800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc100250 0x7f50cc197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d4800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d4800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc198870 con 0x7f50cc101450 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d4800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19d280 con 0x7f50cc100250 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59970/0 (socket says 172.21.15.57:59970) 2024-02-28T23:03:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.182+0000 7f50d1400700 1 -- 172.21.15.57:0/4267298501 learned_addr learned my addr 172.21.15.57:0/4267298501 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d1e00700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc100250 0x7f50cc197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d1400700 1 -- 172.21.15.57:0/4267298501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc100250 msgr2=0x7f50cc197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d1400700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc100250 0x7f50cc197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d1400700 1 -- 172.21.15.57:0/4267298501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00095f0 con 0x7f50cc101450 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d1400700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d8d0 tx=0x7f50bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc009940 con 0x7f50cc101450 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f50bc010460 con 0x7f50cc101450 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc00f5f0 con 0x7f50cc101450 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.183+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc19d480 con 0x7f50cc101450 2024-02-28T23:03:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.184+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc19d9a0 con 0x7f50cc101450 2024-02-28T23:03:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.185+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50cc105530 con 0x7f50cc101450 2024-02-28T23:03:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.186+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f50bc0105d0 con 0x7f50cc101450 2024-02-28T23:03:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.187+0000 7f50c6a00700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 0x7f50b406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.187+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f50bc08b510 con 0x7f50cc101450 2024-02-28T23:03:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.187+0000 7f50d1e00700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 0x7f50b406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.188+0000 7f50d1e00700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 0x7f50b406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c000b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:33.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.192+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f50bc055f10 con 0x7f50cc101450 2024-02-28T23:03:33.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.421+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f50cc061190 con 0x7f50b406c680 2024-02-28T23:03:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.424+0000 7f50c6a00700 1 -- 172.21.15.57:0/4267298501 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f50cc061190 con 0x7f50b406c680 2024-02-28T23:03:33.423 INFO:teuthology.orchestra.run.smithi057.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-28T23:03:33.423 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE7361049H4P0KGN 3726G No 3m ago LVM detected, locked 2024-02-28T23:03:33.423 INFO:teuthology.orchestra.run.smithi057.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 86s ago LVM detected, locked 2024-02-28T23:03:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 msgr2=0x7f50b406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 0x7f50b406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c000b540 comp rx=0 tx=0).stop 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 msgr2=0x7f50cc198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d8d0 tx=0x7f50bc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 shutdown_connections 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f50b406c680 0x7f50b406eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50cc100250 0x7f50cc197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 --2- 172.21.15.57:0/4267298501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50cc101450 0x7f50cc198250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.427+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 >> 172.21.15.57:0/4267298501 conn(0x7f50cc0fb7e0 msgr2=0x7f50cc102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.428+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 shutdown_connections 2024-02-28T23:03:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:33.428+0000 7f50d4800700 1 -- 172.21.15.57:0/4267298501 wait complete. 2024-02-28T23:03:33.802 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:33 smithi057 ceph-mon[29096]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:33 smithi084 ceph-mon[34826]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:34.242 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:34.251 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:34.251 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-28T23:03:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:35.029 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:34 smithi057 ceph-mon[29096]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:34 smithi084 ceph-mon[34826]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:35.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:35 smithi057 ceph-mon[29096]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:35 smithi084 ceph-mon[34826]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.864+0000 7f30c1600700 1 -- 172.21.15.57:0/2090803068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 msgr2=0x7f30bc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.864+0000 7f30c1600700 1 --2- 172.21.15.57:0/2090803068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc101890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009910 tx=0x7f30a4009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 -- 172.21.15.57:0/2090803068 shutdown_connections 2024-02-28T23:03:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 --2- 172.21.15.57:0/2090803068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc101890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 --2- 172.21.15.57:0/2090803068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 0x7f30bc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 -- 172.21.15.57:0/2090803068 >> 172.21.15.57:0/2090803068 conn(0x7f30bc0fb7f0 msgr2=0x7f30bc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 -- 172.21.15.57:0/2090803068 shutdown_connections 2024-02-28T23:03:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.865+0000 7f30c1600700 1 -- 172.21.15.57:0/2090803068 wait complete. 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.866+0000 7f30c1600700 1 Processor -- start 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.866+0000 7f30c1600700 1 -- start start 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30c1600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 0x7f30bc197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30c1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30c1600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1987d0 con 0x7f30bc101440 2024-02-28T23:03:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30c1600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc198940 con 0x7f30bc100240 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30b3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30b3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33374/0 (socket says 172.21.15.57:33374) 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30b3e00700 1 -- 172.21.15.57:0/1188131227 learned_addr learned my addr 172.21.15.57:0/1188131227 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.867+0000 7f30baa00700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 0x7f30bc197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3e00700 1 -- 172.21.15.57:0/1188131227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 msgr2=0x7f30bc197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3e00700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 0x7f30bc197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3e00700 1 -- 172.21.15.57:0/1188131227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40095f0 con 0x7f30bc101440 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3e00700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005b40 tx=0x7f30a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a401d070 con 0x7f30bc101440 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f30a4022470 con 0x7f30bc101440 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a400f690 con 0x7f30bc101440 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc19d370 con 0x7f30bc101440 2024-02-28T23:03:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.868+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc19d860 con 0x7f30bc101440 2024-02-28T23:03:36.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.870+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc105520 con 0x7f30bc101440 2024-02-28T23:03:36.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.871+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f30a40225e0 con 0x7f30bc101440 2024-02-28T23:03:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.872+0000 7f30b3400700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 0x7f30a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.872+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f30a408cc90 con 0x7f30bc101440 2024-02-28T23:03:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.872+0000 7f30baa00700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 0x7f30a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.873+0000 7f30baa00700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 0x7f30a006eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30ac005fd0 tx=0x7f30ac005dd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:36.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:36.877+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f30a4057710 con 0x7f30bc101440 2024-02-28T23:03:37.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:37.129+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f30bc061190 con 0x7f30a006c680 2024-02-28T23:03:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:37 smithi057 ceph-mon[29096]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-28T23:03:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:37 smithi084 ceph-mon[34826]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-28T23:03:38.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.791+0000 7f30b3400700 1 -- 172.21.15.57:0/1188131227 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f30bc061190 con 0x7f30a006c680 2024-02-28T23:03:38.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 msgr2=0x7f30a006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:38.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 0x7f30a006eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30ac005fd0 tx=0x7f30ac005dd0 comp rx=0 tx=0).stop 2024-02-28T23:03:38.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 msgr2=0x7f30bc1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005b40 tx=0x7f30a400be20 comp rx=0 tx=0).stop 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 shutdown_connections 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f30a006c680 0x7f30a006eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30bc100240 0x7f30bc197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 --2- 172.21.15.57:0/1188131227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30bc101440 0x7f30bc1981d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 >> 172.21.15.57:0/1188131227 conn(0x7f30bc0fb7f0 msgr2=0x7f30bc102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 shutdown_connections 2024-02-28T23:03:38.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:38.795+0000 7f30c1600700 1 -- 172.21.15.57:0/1188131227 wait complete. 2024-02-28T23:03:39.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-mon[29096]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-28T23:03:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-mon[29096]: osdmap e42: 6 total, 6 up, 6 in 2024-02-28T23:03:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-28T23:03:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:38 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[29092]: 2024-02-28T23:03:38.757+0000 7f6c41e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:03:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:38 smithi084 ceph-mon[34826]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:03:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-28T23:03:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:38 smithi084 ceph-mon[34826]: osdmap e42: 6 total, 6 up, 6 in 2024-02-28T23:03:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:38 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-28T23:03:39.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: pgmap v218: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: osdmap e43: 6 total, 6 up, 6 in 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: osdmap e44: 6 total, 6 up, 6 in 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-28T23:03:39.852 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: fsmap cephfs:0 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: Saving service mds.cephfs spec with placement count:4 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:39.853 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: pgmap v218: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-28T23:03:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: osdmap e43: 6 total, 6 up, 6 in 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: osdmap e44: 6 total, 6 up, 6 in 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: fsmap cephfs:0 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: Saving service mds.cephfs spec with placement count:4 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:40.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:40.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:40.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:40.990 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:40 smithi057 ceph-mon[29096]: Deploying daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:03:40.990 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:40 smithi057 ceph-mon[29096]: osdmap e45: 6 total, 6 up, 6 in 2024-02-28T23:03:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:40 smithi084 ceph-mon[34826]: Deploying daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:03:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:40 smithi084 ceph-mon[34826]: osdmap e45: 6 total, 6 up, 6 in 2024-02-28T23:03:41.230 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:41 smithi057 ceph-mon[29096]: pgmap v222: 65 pgs: 13 creating+peering, 12 active+clean, 40 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:41 smithi057 ceph-mon[29096]: osdmap e46: 6 total, 6 up, 6 in 2024-02-28T23:03:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:41 smithi084 ceph-mon[34826]: pgmap v222: 65 pgs: 13 creating+peering, 12 active+clean, 40 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:41 smithi084 ceph-mon[34826]: osdmap e46: 6 total, 6 up, 6 in 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 -- 172.21.15.57:0/4074110960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 msgr2=0x7faf0c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 --2- 172.21.15.57:0/4074110960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c107af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faef8009910 tx=0x7faef8009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 -- 172.21.15.57:0/4074110960 shutdown_connections 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 --2- 172.21.15.57:0/4074110960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c107af0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 --2- 172.21.15.57:0/4074110960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c069270 0x7faf0c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 -- 172.21.15.57:0/4074110960 >> 172.21.15.57:0/4074110960 conn(0x7faf0c0fa780 msgr2=0x7faf0c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 -- 172.21.15.57:0/4074110960 shutdown_connections 2024-02-28T23:03:43.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.697+0000 7faf11400700 1 -- 172.21.15.57:0/4074110960 wait complete. 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 Processor -- start 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 -- start start 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c198230 0x7faf0c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1986a0 con 0x7faf0c105710 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.699+0000 7faf11400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c198810 con 0x7faf0c198230 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33402/0 (socket says 172.21.15.57:33402) 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 -- 172.21.15.57:0/1072519854 learned_addr learned my addr 172.21.15.57:0/1072519854 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf09e00700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c198230 0x7faf0c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 -- 172.21.15.57:0/1072519854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c198230 msgr2=0x7faf0c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c198230 0x7faf0c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.700+0000 7faf0a800700 1 -- 172.21.15.57:0/1072519854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80095f0 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf0a800700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faefc00c8f0 tx=0x7faefc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faefc007a50 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faefc007bb0 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c19d800 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faefc018630 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.701+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c19dd20 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.702+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c191df0 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.703+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7faefc007d20 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.704+0000 7faf03400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 0x7faef006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.704+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7faefc08b940 con 0x7faf0c105710 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.705+0000 7faf09e00700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 0x7faef006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.705+0000 7faf09e00700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 0x7faef006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:43.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:43.710+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faefc055f80 con 0x7faf0c105710 2024-02-28T23:03:44.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.001+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf0c02ce40 con 0x7faf0c105710 2024-02-28T23:03:44.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.002+0000 7faf03400700 1 -- 172.21.15.57:0/1072519854 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7faefc0595a0 con 0x7faf0c105710 2024-02-28T23:03:44.001 INFO:teuthology.orchestra.run.smithi057.stdout:e4 2024-02-28T23:03:44.001 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 4 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:03:43.799205+0000 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:03:44.002 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 0 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 1 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:in 0 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14466} 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:03:44.003 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:03:44.004 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 0 2024-02-28T23:03:44.004 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{0:14466} state up:creating seq 1 addr [v2:172.21.15.57:6826/1292376194,v1:172.21.15.57:6827/1292376194] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:03:44.004 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:44.004 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:03:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 msgr2=0x7faef006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 0x7faef006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef800b540 comp rx=0 tx=0).stop 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 msgr2=0x7faf0c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faefc00c8f0 tx=0x7faefc00ccb0 comp rx=0 tx=0).stop 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 shutdown_connections 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7faef006c680 0x7faef006eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf0c105710 0x7faf0c197cf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 --2- 172.21.15.57:0/1072519854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0c198230 0x7faf0c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.005+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 >> 172.21.15.57:0/1072519854 conn(0x7faf0c0fa780 msgr2=0x7faf0c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.006+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 shutdown_connections 2024-02-28T23:03:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:44.006+0000 7faf11400700 1 -- 172.21.15.57:0/1072519854 wait complete. 2024-02-28T23:03:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 4 2024-02-28T23:03:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:43 smithi057 ceph-mon[29096]: pgmap v224: 65 pgs: 13 creating+peering, 39 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:43 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:43 smithi084 ceph-mon[34826]: pgmap v224: 65 pgs: 13 creating+peering, 39 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:43 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:43 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:44.850 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:44.860 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:44.861 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6826/1292376194,v1:172.21.15.57:6827/1292376194] up:boot 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: daemon mds.cephfs.smithi057.bgmggd assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: Cluster is now healthy 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: fsmap cephfs:0 1 up:standby 2024-02-28T23:03:44.946 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:creating} 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: daemon mds.cephfs.smithi057.bgmggd is now active in filesystem cephfs as rank 0 2024-02-28T23:03:44.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:44 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1072519854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6826/1292376194,v1:172.21.15.57:6827/1292376194] up:boot 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: daemon mds.cephfs.smithi057.bgmggd assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: Cluster is now healthy 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: fsmap cephfs:0 1 up:standby 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:creating} 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:45.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: daemon mds.cephfs.smithi057.bgmggd is now active in filesystem cephfs as rank 0 2024-02-28T23:03:45.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:44 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1072519854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:03:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:45 smithi057 ceph-mon[29096]: Deploying daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:03:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:45 smithi057 ceph-mon[29096]: pgmap v225: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:45 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6826/1292376194,v1:172.21.15.57:6827/1292376194] up:active 2024-02-28T23:03:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:45 smithi057 ceph-mon[29096]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:active} 2024-02-28T23:03:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:45 smithi084 ceph-mon[34826]: Deploying daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:03:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:45 smithi084 ceph-mon[34826]: pgmap v225: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:03:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:45 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6826/1292376194,v1:172.21.15.57:6827/1292376194] up:active 2024-02-28T23:03:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:45 smithi084 ceph-mon[34826]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:active} 2024-02-28T23:03:47.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 -- 172.21.15.57:0/1078104605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 msgr2=0x7f3570105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 --2- 172.21.15.57:0/1078104605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570105a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f355c009910 tx=0x7f355c009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 -- 172.21.15.57:0/1078104605 shutdown_connections 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 --2- 172.21.15.57:0/1078104605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570105a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 --2- 172.21.15.57:0/1078104605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 0x7f3570103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 -- 172.21.15.57:0/1078104605 >> 172.21.15.57:0/1078104605 conn(0x7f35700fa780 msgr2=0x7f35700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 -- 172.21.15.57:0/1078104605 shutdown_connections 2024-02-28T23:03:47.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.490+0000 7f3577600700 1 -- 172.21.15.57:0/1078104605 wait complete. 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.491+0000 7f3577600700 1 Processor -- start 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.491+0000 7f3577600700 1 -- start start 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f3577600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 0x7f3570197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f3577600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f3577600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35701987c0 con 0x7f3570103680 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f3577600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3570198930 con 0x7f3570100d60 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f356fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:47.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f356fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40964/0 (socket says 172.21.15.57:40964) 2024-02-28T23:03:47.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f356fe00700 1 -- 172.21.15.57:0/1809253956 learned_addr learned my addr 172.21.15.57:0/1809253956 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:47.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.492+0000 7f3574c00700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 0x7f3570197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:47.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356fe00700 1 -- 172.21.15.57:0/1809253956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 msgr2=0x7f3570197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356fe00700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 0x7f3570197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356fe00700 1 -- 172.21.15.57:0/1809253956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0095f0 con 0x7f3570103680 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356fe00700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f355c00bd70 tx=0x7f355c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f355c01d070 con 0x7f3570103680 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f355c00f460 con 0x7f3570103680 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f355c021640 con 0x7f3570103680 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.493+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357019d320 con 0x7f3570103680 2024-02-28T23:03:47.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.494+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357019d7c0 con 0x7f3570103680 2024-02-28T23:03:47.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.495+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35700fc340 con 0x7f3570103680 2024-02-28T23:03:47.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.496+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f355c00fab0 con 0x7f3570103680 2024-02-28T23:03:47.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.497+0000 7f356d600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 0x7f355806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:47.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.497+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f355c08cd30 con 0x7f3570103680 2024-02-28T23:03:47.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.497+0000 7f3574c00700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 0x7f355806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:47.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.498+0000 7f3574c00700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 0x7f355806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3564005ea0 tx=0x7f3564005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:47.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.502+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f355c0573f0 con 0x7f3570103680 2024-02-28T23:03:47.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.782+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f3570061960 con 0x7f3570103680 2024-02-28T23:03:47.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.819+0000 7f356d600700 1 -- 172.21.15.57:0/1809253956 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f355c0260b0 con 0x7f3570103680 2024-02-28T23:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 msgr2=0x7f355806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 0x7f355806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3564005ea0 tx=0x7f3564005e30 comp rx=0 tx=0).stop 2024-02-28T23:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 msgr2=0x7f3570198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f355c00bd70 tx=0x7f355c00be50 comp rx=0 tx=0).stop 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 shutdown_connections 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f355806c680 0x7f355806eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3570100d60 0x7f3570197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 --2- 172.21.15.57:0/1809253956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3570103680 0x7f3570198230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.822+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 >> 172.21.15.57:0/1809253956 conn(0x7f35700fa780 msgr2=0x7f35700ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.823+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 shutdown_connections 2024-02-28T23:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:47.823+0000 7f3577600700 1 -- 172.21.15.57:0/1809253956 wait complete. 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: pgmap v226: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 788 B/s wr, 2 op/s 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:47 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1809253956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: pgmap v226: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 788 B/s wr, 2 op/s 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:47 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1809253956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-28T23:03:48.765 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:48.775 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:48.775 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: Deploying daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:boot 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1809253956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: daemon mds.cephfs.smithi084.ztfxpc assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-28T23:03:49.044 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-28T23:03:49.045 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:active} 1 up:standby 2024-02-28T23:03:49.045 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:03:49.045 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:creating} 2024-02-28T23:03:49.045 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: daemon mds.cephfs.smithi084.ztfxpc is now active in filesystem cephfs as rank 1 2024-02-28T23:03:49.045 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:48 smithi057 ceph-mon[29096]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: Deploying daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:boot 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1809253956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: daemon mds.cephfs.smithi084.ztfxpc assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-28T23:03:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: fsmap cephfs:1 {0=cephfs.smithi057.bgmggd=up:active} 1 up:standby 2024-02-28T23:03:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:03:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:creating} 2024-02-28T23:03:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: daemon mds.cephfs.smithi084.ztfxpc is now active in filesystem cephfs as rank 1 2024-02-28T23:03:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:48 smithi084 ceph-mon[34826]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-02-28T23:03:50.042 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:49 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:active 2024-02-28T23:03:50.042 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:49 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2024-02-28T23:03:50.148 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:49 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:active 2024-02-28T23:03:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:49 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2024-02-28T23:03:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:51 smithi057 ceph-mon[29096]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-02-28T23:03:51.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:51 smithi084 ceph-mon[34826]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-02-28T23:03:53.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.106+0000 7fb8e9400700 1 -- 172.21.15.57:0/768073236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4100d60 msgr2=0x7fb8e4103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.106+0000 7fb8e9400700 1 --2- 172.21.15.57:0/768073236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4100d60 0x7fb8e4103140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009910 tx=0x7fb8d4009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 -- 172.21.15.57:0/768073236 shutdown_connections 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 --2- 172.21.15.57:0/768073236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4103680 0x7fb8e4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 --2- 172.21.15.57:0/768073236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4100d60 0x7fb8e4103140 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 -- 172.21.15.57:0/768073236 >> 172.21.15.57:0/768073236 conn(0x7fb8e40fa780 msgr2=0x7fb8e40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 -- 172.21.15.57:0/768073236 shutdown_connections 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.107+0000 7fb8e9400700 1 -- 172.21.15.57:0/768073236 wait complete. 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 Processor -- start 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 -- start start 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4100d60 0x7fb8e4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:53.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41987b0 con 0x7fb8e4103680 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.108+0000 7fb8e9400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4198920 con 0x7fb8e4100d60 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41004/0 (socket says 172.21.15.57:41004) 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 -- 172.21.15.57:0/18061645 learned_addr learned my addr 172.21.15.57:0/18061645 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8e2800700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4100d60 0x7fb8e4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 -- 172.21.15.57:0/18061645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4100d60 msgr2=0x7fb8e4197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4100d60 0x7fb8e4197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.109+0000 7fb8dbe00700 1 -- 172.21.15.57:0/18061645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40095f0 con 0x7fb8e4103680 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8dbe00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c960 tx=0x7fb8cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc007a30 con 0x7fb8e4103680 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb8cc007b90 con 0x7fb8e4103680 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc018630 con 0x7fb8e4103680 2024-02-28T23:03:53.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8e9400700 1 -- 172.21.15.57:0/18061645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e419d370 con 0x7fb8e4103680 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.110+0000 7fb8e9400700 1 -- 172.21.15.57:0/18061645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e419d8c0 con 0x7fb8e4103680 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.112+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e4108ce0 con 0x7fb8e4103680 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.114+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb8cc007d00 con 0x7fb8e4103680 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.114+0000 7fb8db400700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 0x7fb8c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.114+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb8cc08b830 con 0x7fb8e4103680 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.114+0000 7fb8e2800700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 0x7fb8c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.115+0000 7fb8e2800700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 0x7fb8c806eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005090 tx=0x7fb8d400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.119+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb8cc055ef0 con 0x7fb8e4103680 2024-02-28T23:03:53.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.396+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb8e4108e70 con 0x7fb8e4103680 2024-02-28T23:03:53.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.643+0000 7fb8db400700 1 -- 172.21.15.57:0/18061645 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb8cc059510 con 0x7fb8e4103680 2024-02-28T23:03:53.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 msgr2=0x7fb8c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:53.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 0x7fb8c806eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005090 tx=0x7fb8d400b540 comp rx=0 tx=0).stop 2024-02-28T23:03:53.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 msgr2=0x7fb8e4198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c960 tx=0x7fb8cc00cc70 comp rx=0 tx=0).stop 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 shutdown_connections 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8c806c680 0x7fb8c806eb30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e4100d60 0x7fb8e4197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 --2- 172.21.15.57:0/18061645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8e4103680 0x7fb8e4198220 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:53.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.647+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 >> 172.21.15.57:0/18061645 conn(0x7fb8e40fa780 msgr2=0x7fb8e41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:53.649 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.649 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:53.649 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:53.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:53.651 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:53 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/18061645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-28T23:03:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.648+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 shutdown_connections 2024-02-28T23:03:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:53.648+0000 7fb8d8c00700 1 -- 172.21.15.57:0/18061645 wait complete. 2024-02-28T23:03:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:03:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-28T23:03:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:03:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:53 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/18061645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-28T23:03:54.506 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:54.516 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:54.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-28T23:03:54.567 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: Deploying daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:03:54.567 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: Deploying daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: Cluster is now healthy 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:boot 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/18061645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-28T23:03:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby 2024-02-28T23:03:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:03:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:54 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby-replay 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: Cluster is now healthy 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:boot 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/18061645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:03:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:54 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby-replay 2024-02-28T23:03:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:55 smithi057 ceph-mon[29096]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-28T23:03:55.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:55 smithi084 ceph-mon[34826]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-28T23:03:57.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 -- 172.21.15.57:0/2995243557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0100240 msgr2=0x7fb8f0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:57.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/2995243557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0100240 0x7fb8f0100650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009910 tx=0x7fb8e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:03:57.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 -- 172.21.15.57:0/2995243557 shutdown_connections 2024-02-28T23:03:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/2995243557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0101440 0x7fb8f0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/2995243557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0100240 0x7fb8f0100650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 -- 172.21.15.57:0/2995243557 >> 172.21.15.57:0/2995243557 conn(0x7fb8f00fb7f0 msgr2=0x7fb8f00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 -- 172.21.15.57:0/2995243557 shutdown_connections 2024-02-28T23:03:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.172+0000 7fb8f5a00700 1 -- 172.21.15.57:0/2995243557 wait complete. 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.173+0000 7fb8f5a00700 1 Processor -- start 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.173+0000 7fb8f5a00700 1 -- start start 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.173+0000 7fb8f5a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.173+0000 7fb8f5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0101440 0x7fb8f0192380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.173+0000 7fb8f5a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f018e930 con 0x7fb8f0101440 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8f5a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f018eaa0 con 0x7fb8f0100240 2024-02-28T23:03:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8ee000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0101440 0x7fb8f0192380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:37416/0 (socket says 172.21.15.57:37416) 2024-02-28T23:03:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 -- 172.21.15.57:0/1444754861 learned_addr learned my addr 172.21.15.57:0/1444754861 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:03:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 -- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0101440 msgr2=0x7fb8f0192380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0101440 0x7fb8f0192380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.174+0000 7fb8eea00700 1 -- 172.21.15.57:0/1444754861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00095f0 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8eea00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000bda0 tx=0x7fb8e000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e001d070 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb8e000f460 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e0021620 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f018ed20 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.175+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f018f210 con 0x7fb8f0100240 2024-02-28T23:03:57.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.177+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f0105520 con 0x7fb8f0100240 2024-02-28T23:03:57.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.178+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb8e000f5d0 con 0x7fb8f0100240 2024-02-28T23:03:57.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.179+0000 7fb8e7400700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 0x7fb8d406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:03:57.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.179+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb8e008cb20 con 0x7fb8f0100240 2024-02-28T23:03:57.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.179+0000 7fb8ee000700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 0x7fb8d406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:03:57.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.180+0000 7fb8ee000700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 0x7fb8d406eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005950 tx=0x7fb8dc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.186+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb8e005ac70 con 0x7fb8f0100240 2024-02-28T23:03:57.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.466+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fb8f018f480 con 0x7fb8f0100240 2024-02-28T23:03:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.588+0000 7fb8e7400700 1 -- 172.21.15.57:0/1444754861 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb8e005a800 con 0x7fb8f0100240 2024-02-28T23:03:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 msgr2=0x7fb8d406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 0x7fb8d406eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005950 tx=0x7fb8dc00b410 comp rx=0 tx=0).stop 2024-02-28T23:03:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 msgr2=0x7fb8f0191e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:03:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000bda0 tx=0x7fb8e000bdd0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 shutdown_connections 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb8d406c680 0x7fb8d406eb30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f0100240 0x7fb8f0191e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 --2- 172.21.15.57:0/1444754861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8f0101440 0x7fb8f0192380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.593+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 >> 172.21.15.57:0/1444754861 conn(0x7fb8f00fb7f0 msgr2=0x7fb8f0102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.594+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 shutdown_connections 2024-02-28T23:03:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:03:57.594+0000 7fb8f5a00700 1 -- 172.21.15.57:0/1444754861 wait complete. 2024-02-28T23:03:57.594 INFO:teuthology.orchestra.run.smithi057.stderr:inline data disabled 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1444754861' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] up:boot 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:standby-replay 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: Dropping low affinity active daemon mds.cephfs.smithi057.bgmggd in favor of higher affinity standby. 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: Replacing daemon mds.cephfs.smithi057.bgmggd as rank 0 with standby daemon mds.cephfs.smithi057.sbkaml 2024-02-28T23:03:57.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-28T23:03:57.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby-replay 1 up:standby 2024-02-28T23:03:57.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:03:57.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: osdmap e47: 6 total, 6 up, 6 in 2024-02-28T23:03:57.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:replay,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby 2024-02-28T23:03:57.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:57 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1444754861' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] up:boot 2024-02-28T23:03:57.994 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:standby-replay 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: Dropping low affinity active daemon mds.cephfs.smithi057.bgmggd in favor of higher affinity standby. 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: Replacing daemon mds.cephfs.smithi057.bgmggd as rank 0 with standby daemon mds.cephfs.smithi057.sbkaml 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby-replay 1 up:standby 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: osdmap e47: 6 total, 6 up, 6 in 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:replay,1=cephfs.smithi084.ztfxpc=up:active} 1 up:standby 2024-02-28T23:03:57.995 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:57 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:03:58.404 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:03:58.415 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:03:58.415 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-28T23:03:59.317 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:03:59.794 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:59 smithi057 ceph-mon[29096]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-28T23:03:59.794 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:59 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] up:boot 2024-02-28T23:03:59.794 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:59 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:resolve 2024-02-28T23:03:59.794 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:59 smithi057 ceph-mon[29096]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:resolve,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:03:59.794 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:03:59 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:04:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:59 smithi084 ceph-mon[34826]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-28T23:04:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:59 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] up:boot 2024-02-28T23:04:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:59 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:resolve 2024-02-28T23:04:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:59 smithi084 ceph-mon[34826]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:resolve,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:03:59 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:04:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:00 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:reconnect 2024-02-28T23:04:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:00 smithi057 ceph-mon[29096]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:reconnect,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:00 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:00 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:01.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:00 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:reconnect 2024-02-28T23:04:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:00 smithi084 ceph-mon[34826]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:reconnect,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:00 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:00 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:01.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 3 op/s 2024-02-28T23:04:01.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:rejoin 2024-02-28T23:04:01.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:active 2024-02-28T23:04:01.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] up:standby 2024-02-28T23:04:01.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:rejoin,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:01.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:01 smithi057 ceph-mon[29096]: daemon mds.cephfs.smithi057.sbkaml is now active in filesystem cephfs as rank 0 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 3 op/s 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:rejoin 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] up:active 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] up:standby 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: fsmap cephfs:2/2 {0=cephfs.smithi057.sbkaml=up:rejoin,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:01.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:01 smithi084 ceph-mon[34826]: daemon mds.cephfs.smithi057.sbkaml is now active in filesystem cephfs as rank 0 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 -- 172.21.15.57:0/25803592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075fb0 msgr2=0x7feb580769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 --2- 172.21.15.57:0/25803592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075fb0 0x7feb580769b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feb44009910 tx=0x7feb44009c20 comp rx=0 tx=0).stop 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 -- 172.21.15.57:0/25803592 shutdown_connections 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 --2- 172.21.15.57:0/25803592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075fb0 0x7feb580769b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 --2- 172.21.15.57:0/25803592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075660 0x7feb58075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 -- 172.21.15.57:0/25803592 >> 172.21.15.57:0/25803592 conn(0x7feb580fda80 msgr2=0x7feb580ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 -- 172.21.15.57:0/25803592 shutdown_connections 2024-02-28T23:04:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.036+0000 7feb60000700 1 -- 172.21.15.57:0/25803592 wait complete. 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.037+0000 7feb60000700 1 Processor -- start 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb60000700 1 -- start start 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb60000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb60000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 0x7feb5819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb60000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5819cb50 con 0x7feb58075660 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb60000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5819cc90 con 0x7feb58075fb0 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb5d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb5d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59060/0 (socket says 172.21.15.57:59060) 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.038+0000 7feb5d600700 1 -- 172.21.15.57:0/1913184507 learned_addr learned my addr 172.21.15.57:0/1913184507 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5d600700 1 -- 172.21.15.57:0/1913184507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 msgr2=0x7feb5819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5cc00700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 0x7feb5819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5d600700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 0x7feb5819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5d600700 1 -- 172.21.15.57:0/1913184507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440095f0 con 0x7feb58075660 2024-02-28T23:04:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5cc00700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 0x7feb5819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:04:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb5d600700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c910 tx=0x7feb4c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.039+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4c0041f0 con 0x7feb58075660 2024-02-28T23:04:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.040+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb4c007db0 con 0x7feb58075660 2024-02-28T23:04:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.040+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb581a1750 con 0x7feb58075660 2024-02-28T23:04:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.040+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4c003d60 con 0x7feb58075660 2024-02-28T23:04:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.040+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb581a1c70 con 0x7feb58075660 2024-02-28T23:04:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.041+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5810f2e0 con 0x7feb58075660 2024-02-28T23:04:02.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.042+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7feb4c007900 con 0x7feb58075660 2024-02-28T23:04:02.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.043+0000 7feb56000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 0x7feb4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:02.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.043+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7feb4c08b520 con 0x7feb58075660 2024-02-28T23:04:02.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.043+0000 7feb5cc00700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 0x7feb4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:02.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.044+0000 7feb5cc00700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 0x7feb4006eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b5c0 tx=0x7feb44005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:02.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.048+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7feb4c0596b0 con 0x7feb58075660 2024-02-28T23:04:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.330+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feb581a2760 con 0x7feb58075660 2024-02-28T23:04:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.331+0000 7feb56000700 1 -- 172.21.15.57:0/1913184507 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1847 (secure 0 0 0) 0x7feb4c059240 con 0x7feb58075660 2024-02-28T23:04:02.329 INFO:teuthology.orchestra.run.smithi057.stdout:e17 2024-02-28T23:04:02.329 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 17 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:04:01.694383+0000 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:04:02.330 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:04:02.331 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:04:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 msgr2=0x7feb4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 0x7feb4006eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b5c0 tx=0x7feb44005e20 comp rx=0 tx=0).stop 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 msgr2=0x7feb5819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c910 tx=0x7feb4c00ccd0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 shutdown_connections 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7feb4006c680 0x7feb4006eb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb58075660 0x7feb5819c080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 --2- 172.21.15.57:0/1913184507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb58075fb0 0x7feb5819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 >> 172.21.15.57:0/1913184507 conn(0x7feb580fda80 msgr2=0x7feb5810c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 shutdown_connections 2024-02-28T23:04:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:02.334+0000 7feb60000700 1 -- 172.21.15.57:0/1913184507 wait complete. 2024-02-28T23:04:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-02-28T23:04:03.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-28T23:04:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: Cluster is now healthy 2024-02-28T23:04:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:active 2024-02-28T23:04:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:04:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:02 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1913184507' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: Cluster is now healthy 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: mds.? [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] up:active 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:04:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:02 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1913184507' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:04:03.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-28T23:04:04.001 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:03 smithi057 ceph-mon[29096]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-28T23:04:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:03 smithi084 ceph-mon[34826]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-28T23:04:04.529 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:05 smithi057 ceph-mon[29096]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-28T23:04:06.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:05 smithi084 ceph-mon[34826]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 -- 172.21.15.57:0/744686994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c073720 msgr2=0x7f7c2c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 --2- 172.21.15.57:0/744686994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c073720 0x7f7c2c073b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009940 tx=0x7f7c18009c50 comp rx=0 tx=0).stop 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 -- 172.21.15.57:0/744686994 shutdown_connections 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 --2- 172.21.15.57:0/744686994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c073720 0x7f7c2c073b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 --2- 172.21.15.57:0/744686994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 -- 172.21.15.57:0/744686994 >> 172.21.15.57:0/744686994 conn(0x7f7c2c0fb650 msgr2=0x7f7c2c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 -- 172.21.15.57:0/744686994 shutdown_connections 2024-02-28T23:04:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.191+0000 7f7c30a00700 1 -- 172.21.15.57:0/744686994 wait complete. 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.192+0000 7f7c30a00700 1 Processor -- start 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c30a00700 1 -- start start 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c30a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 0x7f7c2c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c30a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c30a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c198560 con 0x7f7c2c074ce0 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c30a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c1986a0 con 0x7f7c2c073720 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c29e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c29e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54538/0 (socket says 172.21.15.57:54538) 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.193+0000 7f7c29e00700 1 -- 172.21.15.57:0/3225925984 learned_addr learned my addr 172.21.15.57:0/3225925984 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c2a800700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 0x7f7c2c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:07.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c29e00700 1 -- 172.21.15.57:0/3225925984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 msgr2=0x7f7c2c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:07.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c29e00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 0x7f7c2c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c29e00700 1 -- 172.21.15.57:0/3225925984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180095f0 con 0x7f7c2c074ce0 2024-02-28T23:04:07.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c2a800700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 0x7f7c2c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c29e00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c180055f0 tx=0x7f7c180056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.194+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1801d070 con 0x7f7c2c074ce0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.195+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c19d0f0 con 0x7f7c2c074ce0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.195+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c1800f460 con 0x7f7c2c074ce0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.195+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c18021620 con 0x7f7c2c074ce0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.195+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c19d660 con 0x7f7c2c074ce0 2024-02-28T23:04:07.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.196+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c10ace0 con 0x7f7c2c074ce0 2024-02-28T23:04:07.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.197+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7c1800f5d0 con 0x7f7c2c074ce0 2024-02-28T23:04:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.198+0000 7f7c23400700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 0x7f7c1006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.198+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c1808cd20 con 0x7f7c2c074ce0 2024-02-28T23:04:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.198+0000 7f7c2a800700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 0x7f7c1006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.199+0000 7f7c2a800700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 0x7f7c1006eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005fd0 tx=0x7f7c1c005f00 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:07.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.205+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7c1805ad90 con 0x7f7c2c074ce0 2024-02-28T23:04:07.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.557+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c2c10aed0 con 0x7f7c2c074ce0 2024-02-28T23:04:07.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.558+0000 7f7c23400700 1 -- 172.21.15.57:0/3225925984 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4770 (secure 0 0 0) 0x7f7c1805a920 con 0x7f7c2c074ce0 2024-02-28T23:04:07.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 msgr2=0x7f7c1006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 0x7f7c1006eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005fd0 tx=0x7f7c1c005f00 comp rx=0 tx=0).stop 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 msgr2=0x7f7c2c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c180055f0 tx=0x7f7c180056d0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 shutdown_connections 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f7c1006c680 0x7f7c1006eb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c2c073720 0x7f7c2c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.561+0000 7f7c30a00700 1 --2- 172.21.15.57:0/3225925984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c2c074ce0 0x7f7c2c197f40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.562+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 >> 172.21.15.57:0/3225925984 conn(0x7f7c2c0fb650 msgr2=0x7f7c2c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:07.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.562+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 shutdown_connections 2024-02-28T23:04:07.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:07.562+0000 7f7c30a00700 1 -- 172.21.15.57:0/3225925984 wait complete. 2024-02-28T23:04:07.561 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-02-28T23:04:07.577 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:04:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3225925984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:04:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3225925984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:04:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:08.468 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-28T23:04:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:08 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:04:09.072 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:08 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:04:09.174 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:09.935 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:09 smithi057 ceph-mon[29096]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:09.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:09 smithi084 ceph-mon[34826]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:11.425 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:11 smithi057 ceph-mon[29096]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-28T23:04:11.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:11 smithi084 ceph-mon[34826]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 -- 172.21.15.57:0/2981232405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 msgr2=0x7f9370105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 --2- 172.21.15.57:0/2981232405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f9370105a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f935c009910 tx=0x7f935c009c20 comp rx=0 tx=0).stop 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 -- 172.21.15.57:0/2981232405 shutdown_connections 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 --2- 172.21.15.57:0/2981232405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f9370105a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 --2- 172.21.15.57:0/2981232405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 0x7f9370103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 -- 172.21.15.57:0/2981232405 >> 172.21.15.57:0/2981232405 conn(0x7f93700fa780 msgr2=0x7f93700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 -- 172.21.15.57:0/2981232405 shutdown_connections 2024-02-28T23:04:12.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.095+0000 7f9375c00700 1 -- 172.21.15.57:0/2981232405 wait complete. 2024-02-28T23:04:12.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.097+0000 7f9375c00700 1 Processor -- start 2024-02-28T23:04:12.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.097+0000 7f9375c00700 1 -- start start 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f9375c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 0x7f93701006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f9375c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f9375c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700ff250 con 0x7f9370103680 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f9375c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700ff390 con 0x7f9370100d60 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f936e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f936e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54558/0 (socket says 172.21.15.57:54558) 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f936e000700 1 -- 172.21.15.57:0/3546532362 learned_addr learned my addr 172.21.15.57:0/3546532362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.098+0000 7f936ea00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 0x7f93701006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f936e000700 1 -- 172.21.15.57:0/3546532362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 msgr2=0x7f93701006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f936e000700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 0x7f93701006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f936e000700 1 -- 172.21.15.57:0/3546532362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0095f0 con 0x7f9370103680 2024-02-28T23:04:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f936e000700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f935c00bd70 tx=0x7f935c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c01d070 con 0x7f9370103680 2024-02-28T23:04:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f935c005dc0 con 0x7f9370103680 2024-02-28T23:04:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.099+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c021410 con 0x7f9370103680 2024-02-28T23:04:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.100+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93700ff610 con 0x7f9370103680 2024-02-28T23:04:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.100+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93700ffab0 con 0x7f9370103680 2024-02-28T23:04:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.101+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937018fbe0 con 0x7f9370103680 2024-02-28T23:04:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.103+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f935c00f7d0 con 0x7f9370103680 2024-02-28T23:04:12.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.104+0000 7f9367400700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 0x7f935406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.104+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f935c08cc80 con 0x7f9370103680 2024-02-28T23:04:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.105+0000 7f936ea00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 0x7f935406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.106+0000 7f936ea00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 0x7f935406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9370069000 tx=0x7f93600073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:12.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.110+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f935c05acf0 con 0x7f9370103680 2024-02-28T23:04:12.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.457+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f937002cc30 con 0x7f9370103680 2024-02-28T23:04:12.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.458+0000 7f9367400700 1 -- 172.21.15.57:0/3546532362 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7f935c00f460 con 0x7f9370103680 2024-02-28T23:04:12.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.460+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 msgr2=0x7f935406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:12.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.460+0000 7f9375c00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 0x7f935406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9370069000 tx=0x7f93600073d0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.460+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 msgr2=0x7f93700fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.460+0000 7f9375c00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f935c00bd70 tx=0x7f935c00bda0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 shutdown_connections 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f935406c680 0x7f935406eb30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9370100d60 0x7f93701006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 --2- 172.21.15.57:0/3546532362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9370103680 0x7f93700fed10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:12.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 >> 172.21.15.57:0/3546532362 conn(0x7f93700fa780 msgr2=0x7f93700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:12.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 shutdown_connections 2024-02-28T23:04:12.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:12.461+0000 7f9375c00700 1 -- 172.21.15.57:0/3546532362 wait complete. 2024-02-28T23:04:12.475 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:04:12.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:12 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:12.662 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:12.662 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:12 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:13.589 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:13 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3546532362' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-28T23:04:13.589 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:13 smithi057 ceph-mon[29096]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:13.591 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-28T23:04:13.604 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-28T23:04:13.682 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:13 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3546532362' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-28T23:04:13.682 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:13 smithi084 ceph-mon[34826]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:13.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 -- 172.21.15.57:0/875436212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 msgr2=0x7fe1900ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:13.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 --2- 172.21.15.57:0/875436212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe1900ffa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe180009880 tx=0x7fe18002bce0 comp rx=0 tx=0).stop 2024-02-28T23:04:13.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 -- 172.21.15.57:0/875436212 shutdown_connections 2024-02-28T23:04:13.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 --2- 172.21.15.57:0/875436212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:13.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 --2- 172.21.15.57:0/875436212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe1900ffa40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:13.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 -- 172.21.15.57:0/875436212 >> 172.21.15.57:0/875436212 conn(0x7fe1900fac50 msgr2=0x7fe1900fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:13.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 -- 172.21.15.57:0/875436212 shutdown_connections 2024-02-28T23:04:13.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.823+0000 7fe198400640 1 -- 172.21.15.57:0/875436212 wait complete. 2024-02-28T23:04:13.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.824+0000 7fe198400640 1 Processor -- start 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.824+0000 7fe198400640 1 -- start start 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.824+0000 7fe198400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe198400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190072650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe198400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190072b90 con 0x7fe1900ff630 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe198400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190072d00 con 0x7fe1901008c0 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54562/0 (socket says 172.21.15.57:54562) 2024-02-28T23:04:13.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 -- 172.21.15.57:0/3164115718 learned_addr learned my addr 172.21.15.57:0/3164115718 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:13.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 -- 172.21.15.57:0/3164115718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 msgr2=0x7fe190072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195000640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195a00640 1 -- 172.21.15.57:0/3164115718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe180009560 con 0x7fe1900ff630 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.825+0000 7fe195000640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe195a00640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe180005000 con 0x7fe1900ff630 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe180005610 con 0x7fe1900ff630 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190072f80 con 0x7fe1900ff630 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe180034670 con 0x7fe1900ff630 2024-02-28T23:04:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.826+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190073480 con 0x7fe1900ff630 2024-02-28T23:04:13.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.827+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1901082e0 con 0x7fe1900ff630 2024-02-28T23:04:13.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.828+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe1800039a0 con 0x7fe1900ff630 2024-02-28T23:04:13.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.829+0000 7fe18e000640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 0x7fe17406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:13.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.829+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe1800adee0 con 0x7fe1900ff630 2024-02-28T23:04:13.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.829+0000 7fe195000640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 0x7fe17406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:13.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.830+0000 7fe195000640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 0x7fe17406eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe190073d80 tx=0x7fe17c005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:13.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:13.834+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe180078540 con 0x7fe1900ff630 2024-02-28T23:04:14.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.035+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe19019d0a0 con 0x7fe1900ff630 2024-02-28T23:04:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.037+0000 7fe18e000640 1 -- 172.21.15.57:0/3164115718 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15079 (secure 0 0 0) 0x7fe18007bb60 con 0x7fe1900ff630 2024-02-28T23:04:14.035 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:04:14.035 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":47,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:03:57.588968+0000","last_up_change":"2024-02-28T23:02:04.242665+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-28T23:03:37.677738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-28T23:03:38.275939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6803","nonce":119728824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6805","nonce":119728824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6809","nonce":119728824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":119728824},{"type":"v1","addr":"172.21.15.57:6807","nonce":119728824}]},"public_addr":"172.21.15.57:6803/119728824","cluster_addr":"172.21.15.57:6805/119728824","heartbeat_back_addr":"172.21.15.57:6809/119728824","heartbeat_front_addr":"172.21.15.57:6807/119728824","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6811","nonce":313179796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6813","nonce":313179796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6817","nonce":313179796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":313179796},{"type":"v1","addr":"172.21.15.57:6815","nonce":313179796}]},"public_addr":"172.21.15.57:6811/313179796","cluster_addr":"172.21.15.57:6813/313179796","heartbeat_back_addr":"172.21.15.57:6817/313179796","heartbeat_front_addr":"172.21.15.57:6815/313179796","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6819","nonce":2667584111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6821","nonce":2667584111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6825","nonce":2667584111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2667584111},{"type":"v1","addr":"172.21.15.57:6823","nonce":2667584111}]},"public_addr":"172.21.15.57:6819/2667584111","cluster_addr":"172.21.15.57:6821/2667584111","heartbeat_back_addr":"172.21.15.57:6825/2667584111","heartbeat_front_addr":"172.21.15.57:6823/2667584111","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6801","nonce":3040593930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6803","nonce":3040593930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6807","nonce":3040593930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3040593930},{"type":"v1","addr":"172.21.15.84:6805","nonce":3040593930}]},"public_addr":"172.21.15.84:6801/3040593930","cluster_addr":"172.21.15.84:6803/3040593930","heartbeat_back_addr":"172.21.15.84:6807/3040593930","heartbeat_front_addr":"172.21.15.84:6805/3040593930","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6809","nonce":3561819927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6811","nonce":3561819927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6815","nonce":3561819927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3561819927},{"type":"v1","addr":"172.21.15.84:6813","nonce":3561819927}]},"public_addr":"172.21.15.84:6809/3561819927","cluster_addr":"172.21.15.84:6811/3561819927","heartbeat_back_addr":"172.21.15.84:6815/3561819927","heartbeat_front_addr":"172.21.15.84:6813/3561819927","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6817","nonce":3141452916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6819","nonce":3141452916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6823","nonce":3141452916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3141452916},{"type":"v1","addr":"172.21.15.84:6821","nonce":3141452916}]},"public_addr":"172.21.15.84:6817/3141452916","cluster_addr":"172.21.15.84:6819/3141452916","heartbeat_back_addr":"172.21.15.84:6823/3141452916","heartbeat_front_addr":"172.21.15.84:6821/3141452916","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6827/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6826/1292376194":"2024-02-29T23:03:57.588940+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:04:14.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.040+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 msgr2=0x7fe17406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.040+0000 7fe198400640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 0x7fe17406eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe190073d80 tx=0x7fe17c005d50 comp rx=0 tx=0).stop 2024-02-28T23:04:14.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.040+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 msgr2=0x7fe190074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.040+0000 7fe198400640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180004f00 comp rx=0 tx=0).stop 2024-02-28T23:04:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 shutdown_connections 2024-02-28T23:04:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe17406c5f0 0x7fe17406eaa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1901008c0 0x7fe190072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 --2- 172.21.15.57:0/3164115718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1900ff630 0x7fe190074010 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 >> 172.21.15.57:0/3164115718 conn(0x7fe1900fac50 msgr2=0x7fe190103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:14.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 shutdown_connections 2024-02-28T23:04:14.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.041+0000 7fe198400640 1 -- 172.21.15.57:0/3164115718 wait complete. 2024-02-28T23:04:14.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-28T23:00:25.160671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-28T23:03:37.677738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-28T23:03:38.275939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-28T23:04:14.061 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-28T23:04:14.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 -- 172.21.15.57:0/2109196711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 msgr2=0x7fb880102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 --2- 172.21.15.57:0/2109196711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb880102ed0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009880 tx=0x7fb86c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 -- 172.21.15.57:0/2109196711 shutdown_connections 2024-02-28T23:04:14.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 --2- 172.21.15.57:0/2109196711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb880102ed0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 --2- 172.21.15.57:0/2109196711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 0x7fb880101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 -- 172.21.15.57:0/2109196711 >> 172.21.15.57:0/2109196711 conn(0x7fb8800fce10 msgr2=0x7fb8800ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:14.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 -- 172.21.15.57:0/2109196711 shutdown_connections 2024-02-28T23:04:14.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.290+0000 7fb888400640 1 -- 172.21.15.57:0/2109196711 wait complete. 2024-02-28T23:04:14.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.291+0000 7fb888400640 1 Processor -- start 2024-02-28T23:04:14.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.291+0000 7fb888400640 1 -- start start 2024-02-28T23:04:14.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb888400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 0x7fb880197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb888400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb888400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880197c70 con 0x7fb880102a60 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb888400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880197de0 con 0x7fb880101870 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb885000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb885000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54576/0 (socket says 172.21.15.57:54576) 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb885000640 1 -- 172.21.15.57:0/259938771 learned_addr learned my addr 172.21.15.57:0/259938771 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.292+0000 7fb885a00640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 0x7fb880197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb885000640 1 -- 172.21.15.57:0/259938771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 msgr2=0x7fb880197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb885000640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 0x7fb880197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb885000640 1 -- 172.21.15.57:0/259938771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c009560 con 0x7fb880102a60 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb885000640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009230 tx=0x7fb86c02cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c004420 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb86c0054b0 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88019c810 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c034440 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.293+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88019cd10 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.294+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88010a5b0 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.295+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb86c0037d0 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.296+0000 7fb87e000640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 0x7fb86806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.297+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb86c0addb0 con 0x7fb880102a60 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.297+0000 7fb885a00640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 0x7fb86806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.297+0000 7fb885a00640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 0x7fb86806eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8801028c0 tx=0x7fb874005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:14.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.301+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb86c078410 con 0x7fb880102a60 2024-02-28T23:04:14.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.499+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb88019cf80 con 0x7fb880102a60 2024-02-28T23:04:14.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.499+0000 7fb87e000640 1 -- 172.21.15.57:0/259938771 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb86c04b020 con 0x7fb880102a60 2024-02-28T23:04:14.497 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 msgr2=0x7fb86806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 0x7fb86806eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8801028c0 tx=0x7fb874005f90 comp rx=0 tx=0).stop 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 msgr2=0x7fb8801976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009230 tx=0x7fb86c02cec0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 shutdown_connections 2024-02-28T23:04:14.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb880102a60 0x7fb8801976a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb86806c5f0 0x7fb86806eaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 --2- 172.21.15.57:0/259938771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb880101870 0x7fb880197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 >> 172.21.15.57:0/259938771 conn(0x7fb8800fce10 msgr2=0x7fb880105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:14.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 shutdown_connections 2024-02-28T23:04:14.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.502+0000 7fb888400640 1 -- 172.21.15.57:0/259938771 wait complete. 2024-02-28T23:04:14.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-28T23:04:14.585 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:14 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3164115718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:04:14.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 -- 172.21.15.57:0/1739086718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 msgr2=0x7fc5700ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 --2- 172.21.15.57:0/1739086718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc5700ff910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc560009880 tx=0x7fc56002bce0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 -- 172.21.15.57:0/1739086718 shutdown_connections 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 --2- 172.21.15.57:0/1739086718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 0x7fc570100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 --2- 172.21.15.57:0/1739086718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc5700ff910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 -- 172.21.15.57:0/1739086718 >> 172.21.15.57:0/1739086718 conn(0x7fc5700faa40 msgr2=0x7fc5700fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 -- 172.21.15.57:0/1739086718 shutdown_connections 2024-02-28T23:04:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.718+0000 7fc56f200640 1 -- 172.21.15.57:0/1739086718 wait complete. 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.719+0000 7fc56f200640 1 Processor -- start 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56f200640 1 -- start start 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 0x7fc570197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570197af0 con 0x7fc5700ff500 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570197c60 con 0x7fc5701006f0 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54594/0 (socket says 172.21.15.57:54594) 2024-02-28T23:04:14.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.720+0000 7fc56de00640 1 -- 172.21.15.57:0/1267564327 learned_addr learned my addr 172.21.15.57:0/1267564327 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc56de00640 1 -- 172.21.15.57:0/1267564327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 msgr2=0x7fc570197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc56d400640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 0x7fc570197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc56de00640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 0x7fc570197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc56de00640 1 -- 172.21.15.57:0/1267564327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560009560 con 0x7fc5700ff500 2024-02-28T23:04:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc56de00640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc560009230 tx=0x7fc560004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc560005000 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc560005610 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.721+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc560034440 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.722+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57019c690 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.722+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57019cb90 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.723+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570104740 con 0x7fc5700ff500 2024-02-28T23:04:14.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.724+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc560003880 con 0x7fc5700ff500 2024-02-28T23:04:14.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.725+0000 7fc566a00640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 0x7fc55406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:14.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.725+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc5600ade00 con 0x7fc5700ff500 2024-02-28T23:04:14.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.725+0000 7fc56d400640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 0x7fc55406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:14.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.726+0000 7fc56d400640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 0x7fc55406eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:14.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.729+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc560078460 con 0x7fc5700ff500 2024-02-28T23:04:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:14 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3164115718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:04:14.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.935+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc57002c4a0 con 0x7fc5700ff500 2024-02-28T23:04:14.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.936+0000 7fc566a00640 1 -- 172.21.15.57:0/1267564327 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc56004b020 con 0x7fc5700ff500 2024-02-28T23:04:14.934 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:04:14.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 msgr2=0x7fc55406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 0x7fc55406eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c005850 comp rx=0 tx=0).stop 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 msgr2=0x7fc570196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc560009230 tx=0x7fc560004e30 comp rx=0 tx=0).stop 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 shutdown_connections 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc55406c5f0 0x7fc55406eaa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5701006f0 0x7fc570197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 --2- 172.21.15.57:0/1267564327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5700ff500 0x7fc570196fe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 >> 172.21.15.57:0/1267564327 conn(0x7fc5700faa40 msgr2=0x7fc570101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 shutdown_connections 2024-02-28T23:04:14.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:14.938+0000 7fc56f200640 1 -- 172.21.15.57:0/1267564327 wait complete. 2024-02-28T23:04:14.955 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 -- 172.21.15.57:0/4120660467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 msgr2=0x7f3000102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 --2- 172.21.15.57:0/4120660467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f3000102e30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009880 tx=0x7f2fe802bcf0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 -- 172.21.15.57:0/4120660467 shutdown_connections 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 --2- 172.21.15.57:0/4120660467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f3000102e30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 --2- 172.21.15.57:0/4120660467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 0x7f3000101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 -- 172.21.15.57:0/4120660467 >> 172.21.15.57:0/4120660467 conn(0x7f30000fce10 msgr2=0x7f30000ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:15.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 -- 172.21.15.57:0/4120660467 shutdown_connections 2024-02-28T23:04:15.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.168+0000 7f3006400640 1 -- 172.21.15.57:0/4120660467 wait complete. 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 Processor -- start 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 -- start start 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 0x7f3000196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000197aa0 con 0x7f30001029c0 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.169+0000 7f3006400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000197c10 con 0x7f3000101730 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46480/0 (socket says 172.21.15.57:46480) 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 -- 172.21.15.57:0/1984107918 learned_addr learned my addr 172.21.15.57:0/1984107918 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f3005000640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 0x7f3000196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 -- 172.21.15.57:0/1984107918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 msgr2=0x7f3000196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 0x7f3000196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 -- 172.21.15.57:0/1984107918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009560 con 0x7f30001029c0 2024-02-28T23:04:15.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.170+0000 7f2ff7e00640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8036040 tx=0x7f2fe8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.171+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fe8005000 con 0x7f30001029c0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.171+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2fe8005610 con 0x7f30001029c0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.171+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fe8003ea0 con 0x7f30001029c0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.171+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300019c640 con 0x7f30001029c0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.171+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300019cba0 con 0x7f30001029c0 2024-02-28T23:04:15.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.172+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300010a3c0 con 0x7f30001029c0 2024-02-28T23:04:15.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.173+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2fe80035f0 con 0x7f30001029c0 2024-02-28T23:04:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.175+0000 7f2ffe000640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 0x7f2fe406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.175+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2fe80adeb0 con 0x7f30001029c0 2024-02-28T23:04:15.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.175+0000 7f3005000640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 0x7f2fe406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.176+0000 7f3005000640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 0x7f2fe406eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00058c0 tx=0x7f2ff000a4c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:15.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.179+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2fe8078510 con 0x7f30001029c0 2024-02-28T23:04:15.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.377+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f300010a550 con 0x7f30001029c0 2024-02-28T23:04:15.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.378+0000 7f2ffe000640 1 -- 172.21.15.57:0/1984107918 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2fe8041020 con 0x7f30001029c0 2024-02-28T23:04:15.376 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.380+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 msgr2=0x7f2fe406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 0x7f2fe406eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00058c0 tx=0x7f2ff000a4c0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 msgr2=0x7f30001974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8036040 tx=0x7f2fe8004f00 comp rx=0 tx=0).stop 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 shutdown_connections 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30001029c0 0x7f30001974d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f2fe406c5f0 0x7f2fe406eaa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 --2- 172.21.15.57:0/1984107918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3000101730 0x7f3000196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 >> 172.21.15.57:0/1984107918 conn(0x7f30000fce10 msgr2=0x7f3000105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:15.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 shutdown_connections 2024-02-28T23:04:15.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.381+0000 7f3006400640 1 -- 172.21.15.57:0/1984107918 wait complete. 2024-02-28T23:04:15.397 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:15 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/259938771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:04:15.397 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:15 smithi057 ceph-mon[29096]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:15.397 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:15 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1267564327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:04:15.400 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 -- 172.21.15.57:0/969661414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 msgr2=0x7f73a8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 --2- 172.21.15.57:0/969661414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8073090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7394009880 tx=0x7f739402bce0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 -- 172.21.15.57:0/969661414 shutdown_connections 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 --2- 172.21.15.57:0/969661414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8073090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 --2- 172.21.15.57:0/969661414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 0x7f73a8072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.606+0000 7f73ae200640 1 -- 172.21.15.57:0/969661414 >> 172.21.15.57:0/969661414 conn(0x7f73a80fa9a0 msgr2=0x7f73a80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.607+0000 7f73ae200640 1 -- 172.21.15.57:0/969661414 shutdown_connections 2024-02-28T23:04:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.607+0000 7f73ae200640 1 -- 172.21.15.57:0/969661414 wait complete. 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.607+0000 7f73ae200640 1 Processor -- start 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ae200640 1 -- start start 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ae200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ae200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 0x7f73a8197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ae200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8197a40 con 0x7f73a8072c40 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ae200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8197bb0 con 0x7f73a8074220 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ace00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ace00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46506/0 (socket says 172.21.15.57:46506) 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.608+0000 7f73ace00640 1 -- 172.21.15.57:0/3365293701 learned_addr learned my addr 172.21.15.57:0/3365293701 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73a7e00640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 0x7f73a8197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ace00640 1 -- 172.21.15.57:0/3365293701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 msgr2=0x7f73a8197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ace00640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 0x7f73a8197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ace00640 1 -- 172.21.15.57:0/3365293701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009560 con 0x7f73a8072c40 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ace00640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c810 tx=0x7f739c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739c0041f0 con 0x7f73a8072c40 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f739c007e60 con 0x7f73a8072c40 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a819c640 con 0x7f73a8072c40 2024-02-28T23:04:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739c0049d0 con 0x7f73a8072c40 2024-02-28T23:04:15.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.609+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a819cba0 con 0x7f73a8072c40 2024-02-28T23:04:15.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.611+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a810a390 con 0x7f73a8072c40 2024-02-28T23:04:15.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.612+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f739c0079b0 con 0x7f73a8072c40 2024-02-28T23:04:15.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.613+0000 7f73a5600640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 0x7f739006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:15.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.613+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f739c08b780 con 0x7f73a8072c40 2024-02-28T23:04:15.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.613+0000 7f73a7e00640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 0x7f739006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:15.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.614+0000 7f73a7e00640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 0x7f739006eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f739402c7c0 tx=0x7f7394005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:15.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.619+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f739c055d60 con 0x7f73a8072c40 2024-02-28T23:04:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:15 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/259938771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:04:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:15 smithi084 ceph-mon[34826]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-28T23:04:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:15 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1267564327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:04:15.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.867+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73a819cf10 con 0x7f73a8072c40 2024-02-28T23:04:15.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.868+0000 7f73a5600640 1 -- 172.21.15.57:0/3365293701 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4770 (secure 0 0 0) 0x7f739c059380 con 0x7f73a8072c40 2024-02-28T23:04:15.867 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:04:15.867 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:04:01.694383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24291":{"gid":24291,"name":"cephfs.smithi084.dadqfu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.84:6827/2979507340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2979507340},{"type":"v1","addr":"172.21.15.84:6827","nonce":2979507340}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24299":{"gid":24299,"name":"cephfs.smithi057.bgmggd","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6827/3659715660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3659715660},{"type":"v1","addr":"172.21.15.57:6827","nonce":3659715660}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:04:15.867 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 msgr2=0x7f739006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 0x7f739006eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f739402c7c0 tx=0x7f7394005fd0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 msgr2=0x7f73a8196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c810 tx=0x7f739c00ccd0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 shutdown_connections 2024-02-28T23:04:15.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f739006c5f0 0x7f739006eaa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a8074220 0x7f73a8197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 --2- 172.21.15.57:0/3365293701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73a8072c40 0x7f73a8196f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:15.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 >> 172.21.15.57:0/3365293701 conn(0x7f73a80fa9a0 msgr2=0x7f73a81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:15.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 shutdown_connections 2024-02-28T23:04:15.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:15.871+0000 7f73ae200640 1 -- 172.21.15.57:0/3365293701 wait complete. 2024-02-28T23:04:15.891 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-02-28T23:04:15.891 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-28T23:04:15.916 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-28T23:04:15.917 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-28T23:04:15.917 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-28T23:04:15.917 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-28T23:04:15.917 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:15.917 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-28T23:04:15.917 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-28T23:04:15.917 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:15.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:15.918 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-02-28T23:04:15.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:15.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-02-28T23:04:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2024-02-28T23:04:16.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T23:04:16.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:16.087 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-02-28T23:04:16.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:16.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2024-02-28T23:04:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2024-02-28T23:04:16.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-28T23:04:16.269 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-28T23:04:16.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-28T23:04:16.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:16 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1984107918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:04:16.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:16 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3365293701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:04:16.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:16.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:16 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:16.698 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:16 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1984107918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:04:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:16 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3365293701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:04:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:16 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:17 smithi057 ceph-mon[29096]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-28T23:04:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:04:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:04:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:17 smithi084 ceph-mon[34826]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-28T23:04:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:04:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:04:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:04:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.025+0000 7ff372600700 1 -- 172.21.15.57:0/1591551444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c0739a0 msgr2=0x7ff36c10c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.025+0000 7ff372600700 1 --2- 172.21.15.57:0/1591551444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c0739a0 0x7ff36c10c5e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff354009940 tx=0x7ff354009c50 comp rx=0 tx=0).stop 2024-02-28T23:04:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 -- 172.21.15.57:0/1591551444 shutdown_connections 2024-02-28T23:04:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 --2- 172.21.15.57:0/1591551444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c0739a0 0x7ff36c10c5e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 --2- 172.21.15.57:0/1591551444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c073090 0x7ff36c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 -- 172.21.15.57:0/1591551444 >> 172.21.15.57:0/1591551444 conn(0x7ff36c0fbda0 msgr2=0x7ff36c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:19.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 -- 172.21.15.57:0/1591551444 shutdown_connections 2024-02-28T23:04:19.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.026+0000 7ff372600700 1 -- 172.21.15.57:0/1591551444 wait complete. 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.027+0000 7ff372600700 1 Processor -- start 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.027+0000 7ff372600700 1 -- start start 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff372600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff372600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c0739a0 0x7ff36c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff372600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c198db0 con 0x7ff36c073090 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff372600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c198ef0 con 0x7ff36c0739a0 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46524/0 (socket says 172.21.15.57:46524) 2024-02-28T23:04:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 -- 172.21.15.57:0/2790677660 learned_addr learned my addr 172.21.15.57:0/2790677660 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff362a00700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c0739a0 0x7ff36c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 -- 172.21.15.57:0/2790677660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c0739a0 msgr2=0x7ff36c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c0739a0 0x7ff36c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.028+0000 7ff36b400700 1 -- 172.21.15.57:0/2790677660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540095f0 con 0x7ff36c073090 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff36b400700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00cfd0 tx=0x7ff35c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff35c00ad70 con 0x7ff36c073090 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff35c0128d0 con 0x7ff36c073090 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c19cd40 con 0x7ff36c073090 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff35c011880 con 0x7ff36c073090 2024-02-28T23:04:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.029+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c19d290 con 0x7ff36c073090 2024-02-28T23:04:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.031+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c109ce0 con 0x7ff36c073090 2024-02-28T23:04:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.031+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff35c012420 con 0x7ff36c073090 2024-02-28T23:04:19.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.032+0000 7ff368c00700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 0x7ff35006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.032+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff35c08ba20 con 0x7ff36c073090 2024-02-28T23:04:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.033+0000 7ff362a00700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 0x7ff35006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.033+0000 7ff362a00700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 0x7ff35006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff354005b40 tx=0x7ff354005a90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:19.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.037+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff35c0560a0 con 0x7ff36c073090 2024-02-28T23:04:19.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.304+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff36c199650 con 0x7ff36c073090 2024-02-28T23:04:19.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.305+0000 7ff368c00700 1 -- 172.21.15.57:0/2790677660 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff35c016070 con 0x7ff36c073090 2024-02-28T23:04:19.303 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-28T23:04:19.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 msgr2=0x7ff35006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 0x7ff35006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff354005b40 tx=0x7ff354005a90 comp rx=0 tx=0).stop 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 msgr2=0x7ff36c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00cfd0 tx=0x7ff35c00a340 comp rx=0 tx=0).stop 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 shutdown_connections 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff35006c680 0x7ff35006eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff36c073090 0x7ff36c198220 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 --2- 172.21.15.57:0/2790677660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff36c0739a0 0x7ff36c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.309+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 >> 172.21.15.57:0/2790677660 conn(0x7ff36c0fbda0 msgr2=0x7ff36c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.310+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 shutdown_connections 2024-02-28T23:04:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:19.310+0000 7ff372600700 1 -- 172.21.15.57:0/2790677660 wait complete. 2024-02-28T23:04:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:19 smithi057 ceph-mon[29096]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:04:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:19 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2790677660' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-28T23:04:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:19 smithi084 ceph-mon[34826]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:04:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:19 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2790677660' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-28T23:04:20.151 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-28T23:04:20.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-28T23:04:20.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2024-02-28T23:04:20.151 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-28T23:04:20.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:20.152 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-28T23:04:20.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-28T23:04:20.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-28T23:04:20.152 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-28T23:04:20.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:20.152 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2024-02-28T23:04:20.212 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-28T23:04:20.212 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-28T23:04:20.212 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2024-02-28T23:04:20.212 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-28T23:04:20.213 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41911sec preferred_lft 41911sec 2024-02-28T23:04:20.214 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-02-28T23:04:20.214 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:20.214 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:20.214 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:20.214 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2024-02-28T23:04:20.215 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-28T23:04:20.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:20.215 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2024-02-28T23:04:20.215 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-02-28T23:04:20.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2024-02-28T23:04:20.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2024-02-28T23:04:20.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2024-02-28T23:04:20.216 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-28T23:04:20.216 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-02-28T23:04:20.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:20.693 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-28T23:04:20.779 INFO:teuthology.orchestra.run.smithi057.stdout:1 2024-02-28T23:04:20.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:20.783 DEBUG:teuthology.orchestra.run.smithi057:> route 2024-02-28T23:04:20.841 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2024-02-28T23:04:20.842 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-28T23:04:20.842 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-28T23:04:20.842 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-28T23:04:20.842 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-28T23:04:20.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:20.842 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2024-02-28T23:04:20.842 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-02-28T23:04:20.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-28T23:04:20.843 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-28T23:04:20.843 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-28T23:04:20.843 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-02-28T23:04:21.527 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:21 smithi057 ceph-mon[29096]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:04:21.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:21.531 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-02-28T23:04:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:21 smithi084 ceph-mon[34826]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:04:21.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:21.591 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2024-02-28T23:04:21.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:21.648 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2024-02-28T23:04:21.648 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-02-28T23:04:21.648 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:21.648 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-28T23:04:21.648 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-02-28T23:04:21.799 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-28T23:04:21.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-28T23:04:21.800 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-02-28T23:04:22.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:22.108 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2024-02-28T23:04:22.108 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-02-28T23:04:22.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2024-02-28T23:04:22.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2024-02-28T23:04:22.109 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-02-28T23:04:22.285 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-28T23:04:22.285 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-28T23:04:22.285 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:22.333 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-28T23:04:22.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-28T23:04:22.333 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:22.431 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-02-28T23:04:22.537 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:22.593 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-02-28T23:04:22.593 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-02-28T23:04:22.594 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ea3970fbfd3f6c7bb44c25574e4199c98a757ef75bac8b294206547e3b8150c5/merged 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/be976540adf21812192b3acd6b806499495962eb16a1aa7ee8cede7891c8273a/merged 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/87eb50dcaedea6478785538abf666e7fb04e19a40204f856c87ef7e35109bb6a/merged 2024-02-28T23:04:22.595 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/19e7cf105e814ec4657398d4ef6da626ba6afdf0b7e20f54b7e20fecabd09ac3/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/782bd67d76354a8ba997e1d07ece4a56f0aaef449ba69d2c0e289307eb96468d/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/df1c2cdefa84f3fedcedb77dfee9e861bb62cad352f5505c34e023b82fa3e3da/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/9009f4395a1fb50b8189a8347e36b89a92654ab6878f75dc0f69299ddc72d1e1/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f7d597e05c12debd562b615291ec9136f793b9a1c65bb68a3c7760ff8617561b/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/3fd1c032252019be59564bd0ed4e8d3d27e4affd1b77a4a093f45fba7cb68b63/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6fb1b6d636b43dde27025d4cd34698a7716de88ef980c9c4e6bc5983b8c58c11/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/603bf68f35494c4ed43dbbe128232fc1fcfb36168de85709a43c2dacfa58f204/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/c87e138ca57921aca77d1dfed7e73838272e76ed43bc213d0798a75ff1bb2ea2/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/dfee5e2480c92b5afb3473daaabf27df7ecd9814c586e05fbb35723883792ddf/merged 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:22.596 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:22.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:22.597 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-02-28T23:04:22.648 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-28T23:04:22.649 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-28T23:04:22.654 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-02-28T23:04:22.728 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:22.771 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-02-28T23:04:22.771 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-02-28T23:04:22.771 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-02-28T23:04:22.772 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ea3970fbfd3f6c7bb44c25574e4199c98a757ef75bac8b294206547e3b8150c5/merged 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/be976540adf21812192b3acd6b806499495962eb16a1aa7ee8cede7891c8273a/merged 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/87eb50dcaedea6478785538abf666e7fb04e19a40204f856c87ef7e35109bb6a/merged 2024-02-28T23:04:22.773 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/19e7cf105e814ec4657398d4ef6da626ba6afdf0b7e20f54b7e20fecabd09ac3/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/782bd67d76354a8ba997e1d07ece4a56f0aaef449ba69d2c0e289307eb96468d/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/df1c2cdefa84f3fedcedb77dfee9e861bb62cad352f5505c34e023b82fa3e3da/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/9009f4395a1fb50b8189a8347e36b89a92654ab6878f75dc0f69299ddc72d1e1/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f7d597e05c12debd562b615291ec9136f793b9a1c65bb68a3c7760ff8617561b/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/3fd1c032252019be59564bd0ed4e8d3d27e4affd1b77a4a093f45fba7cb68b63/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6fb1b6d636b43dde27025d4cd34698a7716de88ef980c9c4e6bc5983b8c58c11/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/603bf68f35494c4ed43dbbe128232fc1fcfb36168de85709a43c2dacfa58f204/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/c87e138ca57921aca77d1dfed7e73838272e76ed43bc213d0798a75ff1bb2ea2/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/dfee5e2480c92b5afb3473daaabf27df7ecd9814c586e05fbb35723883792ddf/merged 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:22.774 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:22.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:22.775 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-02-28T23:04:22.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:22.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2024-02-28T23:04:22.870+0000 7fb246e46480 -1 init, newargv = 0x55d436cd0870 newargc=15 2024-02-28T23:04:22.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[67132]: starting ceph client 2024-02-28T23:04:22.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[67132]: starting fuse 2024-02-28T23:04:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:23.967 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-02-28T23:04:24.059 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:23 smithi057 ceph-mon[29096]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:04:24.077 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:23 smithi084 ceph-mon[34826]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:04:24.134 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-02-28T23:04:24.134 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-02-28T23:04:24.134 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-02-28T23:04:24.134 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-02-28T23:04:24.134 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-02-28T23:04:24.135 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-02-28T23:04:24.136 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ea3970fbfd3f6c7bb44c25574e4199c98a757ef75bac8b294206547e3b8150c5/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/be976540adf21812192b3acd6b806499495962eb16a1aa7ee8cede7891c8273a/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/87eb50dcaedea6478785538abf666e7fb04e19a40204f856c87ef7e35109bb6a/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/19e7cf105e814ec4657398d4ef6da626ba6afdf0b7e20f54b7e20fecabd09ac3/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/782bd67d76354a8ba997e1d07ece4a56f0aaef449ba69d2c0e289307eb96468d/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/df1c2cdefa84f3fedcedb77dfee9e861bb62cad352f5505c34e023b82fa3e3da/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/9009f4395a1fb50b8189a8347e36b89a92654ab6878f75dc0f69299ddc72d1e1/merged 2024-02-28T23:04:24.137 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f7d597e05c12debd562b615291ec9136f793b9a1c65bb68a3c7760ff8617561b/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/3fd1c032252019be59564bd0ed4e8d3d27e4affd1b77a4a093f45fba7cb68b63/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6fb1b6d636b43dde27025d4cd34698a7716de88ef980c9c4e6bc5983b8c58c11/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/603bf68f35494c4ed43dbbe128232fc1fcfb36168de85709a43c2dacfa58f204/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/c87e138ca57921aca77d1dfed7e73838272e76ed43bc213d0798a75ff1bb2ea2/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/dfee5e2480c92b5afb3473daaabf27df7ecd9814c586e05fbb35723883792ddf/merged 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-28T23:04:24.138 INFO:teuthology.orchestra.run.smithi057.stdout:/home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:24.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:24.139 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-02-28T23:04:24.199 INFO:teuthology.orchestra.run.smithi057.stdout:158 2024-02-28T23:04:24.199 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [158] 2024-02-28T23:04:24.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo stdin-killer -- python3 -c ' 2024-02-28T23:04:24.199 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2024-02-28T23:04:24.199 DEBUG:teuthology.orchestra.run.smithi057:> import re 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> import os 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> 2024-02-28T23:04:24.200 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> return f 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2024-02-28T23:04:24.201 DEBUG:teuthology.orchestra.run.smithi057:> ' 2024-02-28T23:04:24.393 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.67132.asok 2024-02-28T23:04:24.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-28T23:04:24.411 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67132.asok 2024-02-28T23:04:24.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:24.412 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67132.asok status 2024-02-28T23:04:24.660 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:04:24.660 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2024-02-28T23:04:24.660 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-28T23:04:24.660 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "67132", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 14534, 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 14534 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:04:24.661 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 972319470 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 972319470 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.14534 192.168.0.1:0/972319470", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/972319470", 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 17, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 47, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2024-02-28T23:04:24.662 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2024-02-28T23:04:24.663 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2024-02-28T23:04:24.663 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:04:24.677 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-28T23:04:25.126 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:25 smithi057 ceph-mon[29096]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-02-28T23:04:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:25 smithi084 ceph-mon[34826]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.402+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3347661603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 msgr2=0x7f5bc01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.402+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3347661603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01008b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009910 tx=0x7f5bb0009c20 comp rx=0 tx=0).stop 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.402+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3347661603 shutdown_connections 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.403+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3347661603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.403+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3347661603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01008b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.403+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3347661603 >> 172.21.15.57:0/3347661603 conn(0x7f5bc00fbda0 msgr2=0x7f5bc00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:27.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.403+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3347661603 shutdown_connections 2024-02-28T23:04:27.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.403+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3347661603 wait complete. 2024-02-28T23:04:27.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 Processor -- start 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 -- start start 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0198c30 con 0x7f5bc01004e0 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.404+0000 7f5bc4e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0198d70 con 0x7f5bc0100e80 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56286/0 (socket says 172.21.15.57:56286) 2024-02-28T23:04:27.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 -- 172.21.15.57:0/3438388649 learned_addr learned my addr 172.21.15.57:0/3438388649 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:27.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bb5400700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:27.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 -- 172.21.15.57:0/3438388649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 msgr2=0x7f5bc01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:27.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bbde00700 1 -- 172.21.15.57:0/3438388649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb00095f0 con 0x7f5bc01004e0 2024-02-28T23:04:27.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.405+0000 7f5bb5400700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bbde00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006010 tx=0x7f5bb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb001d070 con 0x7f5bc01004e0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5bb000f460 con 0x7f5bc01004e0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc019cb60 con 0x7f5bc01004e0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb0021640 con 0x7f5bc01004e0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.406+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc019d050 con 0x7f5bc01004e0 2024-02-28T23:04:27.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.407+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc01045d0 con 0x7f5bc01004e0 2024-02-28T23:04:27.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.408+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5bb000fab0 con 0x7f5bc01004e0 2024-02-28T23:04:27.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.409+0000 7f5bb7400700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 0x7f5ba006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:27.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.409+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5bb008cca0 con 0x7f5bc01004e0 2024-02-28T23:04:27.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.410+0000 7f5bb5400700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 0x7f5ba006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:27.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.410+0000 7f5bb5400700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 0x7f5ba006eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005fd0 tx=0x7f5ba8005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.414+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5bb0057280 con 0x7f5bc01004e0 2024-02-28T23:04:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.684+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5bc0103fb0 con 0x7f5bc01004e0 2024-02-28T23:04:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.685+0000 7f5bb7400700 1 -- 172.21.15.57:0/3438388649 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5bb0026070 con 0x7f5bc01004e0 2024-02-28T23:04:27.684 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-28T23:04:27.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 msgr2=0x7f5ba006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:27.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 0x7f5ba006eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005fd0 tx=0x7f5ba8005e20 comp rx=0 tx=0).stop 2024-02-28T23:04:27.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 msgr2=0x7f5bc01980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:27.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006010 tx=0x7f5bb000be20 comp rx=0 tx=0).stop 2024-02-28T23:04:27.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 shutdown_connections 2024-02-28T23:04:27.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5ba006c680 0x7f5ba006eb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bc01004e0 0x7f5bc01980a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 --2- 172.21.15.57:0/3438388649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bc0100e80 0x7f5bc01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:27.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.687+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 >> 172.21.15.57:0/3438388649 conn(0x7f5bc00fbda0 msgr2=0x7f5bc00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:27.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.688+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 shutdown_connections 2024-02-28T23:04:27.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:27.688+0000 7f5bc4e00700 1 -- 172.21.15.57:0/3438388649 wait complete. 2024-02-28T23:04:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:27 smithi057 ceph-mon[29096]: pgmap v247: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-28T23:04:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:27 smithi084 ceph-mon[34826]: pgmap v247: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-28T23:04:28.588 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-28T23:04:28.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:28.589 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2024-02-28T23:04:28.646 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-28T23:04:28.646 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-28T23:04:28.646 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-28T23:04:28.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2024-02-28T23:04:28.648 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-28T23:04:28.648 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41963sec preferred_lft 41963sec 2024-02-28T23:04:28.648 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2024-02-28T23:04:28.648 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-02-28T23:04:28.648 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-28T23:04:28.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:28.648 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-02-28T23:04:28.648 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-02-28T23:04:28.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2024-02-28T23:04:28.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2024-02-28T23:04:28.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2024-02-28T23:04:28.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-28T23:04:28.649 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-02-28T23:04:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:28 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3438388649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-28T23:04:28.944 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:28 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3438388649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-28T23:04:29.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:29.073 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-28T23:04:29.147 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-02-28T23:04:29.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:29.151 DEBUG:teuthology.orchestra.run.smithi084:> route 2024-02-28T23:04:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2024-02-28T23:04:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-28T23:04:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-28T23:04:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-28T23:04:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-28T23:04:29.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-28T23:04:29.206 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-02-28T23:04:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:29 smithi084 ceph-mon[34826]: pgmap v248: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-28T23:04:29.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:29.918 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-02-28T23:04:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:29.963 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2024-02-28T23:04:30.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:30.051 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-02-28T23:04:30.051 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-02-28T23:04:30.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:30.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-28T23:04:30.052 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-02-28T23:04:30.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:29 smithi057 ceph-mon[29096]: pgmap v248: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-28T23:04:30.189 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-28T23:04:30.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-28T23:04:30.190 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-02-28T23:04:30.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:30.457 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-02-28T23:04:30.457 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-02-28T23:04:30.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2024-02-28T23:04:30.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2024-02-28T23:04:30.457 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-02-28T23:04:30.602 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-28T23:04:30.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-28T23:04:30.603 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:30.648 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-28T23:04:30.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-28T23:04:30.649 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:30.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-02-28T23:04:30.821 DEBUG:teuthology.orchestra.run.smithi084:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:30.868 INFO:teuthology.orchestra.run.smithi084.stdout:/proc 2024-02-28T23:04:30.868 INFO:teuthology.orchestra.run.smithi084.stdout:/sys 2024-02-28T23:04:30.868 INFO:teuthology.orchestra.run.smithi084.stdout:/dev 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/security 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/shm 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/pts 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/run 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/cgroup 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/pstore 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/bpf 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/ 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/selinux 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/hugepages 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/mqueue 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/debug 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/tracing 2024-02-28T23:04:30.869 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/config 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/1000 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/ceph 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/0 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/88b1a7b08b2bdf7d362b48e3917211a8e9aa7e7da8e43538927cd4733a041133/merged 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/9ade016acdcc84cd5ef91446714c9f6ded7b4a061a983efdba455d5441402747/merged 2024-02-28T23:04:30.870 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/743b69bbfc85a2054035bb4995f8810b6aec35074efe186848ec4a5e727d9c21/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/994d9b1fc514330548ef391cccdf19f325fcfcb8d020e1b58b71fd0bd30f3743/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/f59a62bda5049bb60235bec22176b5c11747a962f19f14bc0153b00ecc20e89a/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/7e527cee03ee3d23fb68993341b89bbd216fc06a4a14ad287c066efafad7f657/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/49f214a034d3dd5495b7c09c238a3cd56dd1e95b7f141eaf327ac31682f642e7/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/26e09cd49e4f18c4c9fe52f3729b233fb572f73bf336a6ff785d53b53388f4fa/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/71de5012ebfeb5f9ed1a955b4c90d265328b36f1751d32459546b02d7957f61e/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/c937b1456a6e718b14792fead1566ef71a55658aeb7a84f5396e2ca991a012cf/merged 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:30.871 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:30.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:30.872 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-02-28T23:04:30.914 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-28T23:04:30.914 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-28T23:04:30.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-02-28T23:04:31.040 DEBUG:teuthology.orchestra.run.smithi084:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/proc 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/sys 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/dev 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/security 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/shm 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/pts 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/run 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/cgroup 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/pstore 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/bpf 2024-02-28T23:04:31.085 INFO:teuthology.orchestra.run.smithi084.stdout:/ 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/selinux 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/hugepages 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/mqueue 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/debug 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/tracing 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/config 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/1000 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/ceph 2024-02-28T23:04:31.086 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/0 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/88b1a7b08b2bdf7d362b48e3917211a8e9aa7e7da8e43538927cd4733a041133/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/9ade016acdcc84cd5ef91446714c9f6ded7b4a061a983efdba455d5441402747/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/743b69bbfc85a2054035bb4995f8810b6aec35074efe186848ec4a5e727d9c21/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/994d9b1fc514330548ef391cccdf19f325fcfcb8d020e1b58b71fd0bd30f3743/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/f59a62bda5049bb60235bec22176b5c11747a962f19f14bc0153b00ecc20e89a/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/7e527cee03ee3d23fb68993341b89bbd216fc06a4a14ad287c066efafad7f657/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/49f214a034d3dd5495b7c09c238a3cd56dd1e95b7f141eaf327ac31682f642e7/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/26e09cd49e4f18c4c9fe52f3729b233fb572f73bf336a6ff785d53b53388f4fa/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/71de5012ebfeb5f9ed1a955b4c90d265328b36f1751d32459546b02d7957f61e/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/c937b1456a6e718b14792fead1566ef71a55658aeb7a84f5396e2ca991a012cf/merged 2024-02-28T23:04:31.087 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns 2024-02-28T23:04:31.088 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:31.088 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:31.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:31.089 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-02-28T23:04:31.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:2024-02-28T23:04:31.189+0000 7fd1f6643480 -1 init, newargv = 0x5654dbc416b0 newargc=15 2024-02-28T23:04:31.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:ceph-fuse[54715]: starting ceph client 2024-02-28T23:04:31.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:ceph-fuse[54715]: starting fuse 2024-02-28T23:04:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:31 smithi084 ceph-mon[34826]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-28T23:04:31.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:31 smithi057 ceph-mon[29096]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-28T23:04:32.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-02-28T23:04:32.375 DEBUG:teuthology.orchestra.run.smithi084:> cat /proc/self/mounts | awk '{print $2}' 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/proc 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/sys 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/dev 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/security 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/shm 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/pts 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/run 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/cgroup 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/pstore 2024-02-28T23:04:32.430 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/bpf 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/ 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/selinux 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/hugepages 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/dev/mqueue 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/debug 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/tracing 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/fs/fuse/connections 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/sys/kernel/config 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-sysctl.service 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/nfs/rpc_pipefs 2024-02-28T23:04:32.431 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/1000 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/ceph 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/run/user/0 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/proc/sys/fs/binfmt_misc 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/88b1a7b08b2bdf7d362b48e3917211a8e9aa7e7da8e43538927cd4733a041133/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/9ade016acdcc84cd5ef91446714c9f6ded7b4a061a983efdba455d5441402747/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/743b69bbfc85a2054035bb4995f8810b6aec35074efe186848ec4a5e727d9c21/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/994d9b1fc514330548ef391cccdf19f325fcfcb8d020e1b58b71fd0bd30f3743/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/f59a62bda5049bb60235bec22176b5c11747a962f19f14bc0153b00ecc20e89a/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/7e527cee03ee3d23fb68993341b89bbd216fc06a4a14ad287c066efafad7f657/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/49f214a034d3dd5495b7c09c238a3cd56dd1e95b7f141eaf327ac31682f642e7/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/26e09cd49e4f18c4c9fe52f3729b233fb572f73bf336a6ff785d53b53388f4fa/merged 2024-02-28T23:04:32.432 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/71de5012ebfeb5f9ed1a955b4c90d265328b36f1751d32459546b02d7957f61e/merged 2024-02-28T23:04:32.433 INFO:teuthology.orchestra.run.smithi084.stdout:/var/lib/containers/storage/overlay/c937b1456a6e718b14792fead1566ef71a55658aeb7a84f5396e2ca991a012cf/merged 2024-02-28T23:04:32.433 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns 2024-02-28T23:04:32.433 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:32.433 INFO:teuthology.orchestra.run.smithi084.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-28T23:04:32.433 INFO:teuthology.orchestra.run.smithi084.stdout:/home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:32.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:32.434 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-02-28T23:04:32.486 INFO:teuthology.orchestra.run.smithi084.stdout:125 2024-02-28T23:04:32.486 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo stdin-killer -- python3 -c ' 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> import re 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> import os 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2024-02-28T23:04:32.487 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> return f 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.1")) 2024-02-28T23:04:32.488 DEBUG:teuthology.orchestra.run.smithi084:> ' 2024-02-28T23:04:32.661 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.1.54715.asok 2024-02-28T23:04:32.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-02-28T23:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-28T23:04:32.679 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.54715.asok 2024-02-28T23:04:32.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:32.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.54715.asok status 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "1", 2024-02-28T23:04:32.915 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "54715", 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24323, 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 24323 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-02-28T23:04:32.916 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3704145563 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3704145563 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.24323 192.168.0.1:0/3704145563", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3704145563", 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2024-02-28T23:04:32.917 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 17, 2024-02-28T23:04:32.918 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 47, 2024-02-28T23:04:32.918 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2024-02-28T23:04:32.918 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2024-02-28T23:04:32.918 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2024-02-28T23:04:32.918 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-02-28T23:04:32.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:32.930 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2024-02-28T23:04:32.931 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:32.993 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2024-02-28T23:04:32.994 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:32.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:32.994 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:33.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:33.103 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2024-02-28T23:04:33.103 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:33.157 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2024-02-28T23:04:33.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:33.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:04:33.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:33 smithi057 ceph-mon[29096]: pgmap v250: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-28T23:04:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:33 smithi084 ceph-mon[34826]: pgmap v250: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-28T23:04:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:35 smithi057 ceph-mon[29096]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-28T23:04:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:35 smithi084 ceph-mon[34826]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-28T23:04:37.876 INFO:teuthology.run_tasks:Running task print... 2024-02-28T23:04:37.889 INFO:teuthology.task.print:**** done client 2024-02-28T23:04:37.889 INFO:teuthology.run_tasks:Running task parallel... 2024-02-28T23:04:37.954 INFO:teuthology.task.parallel:starting parallel... 2024-02-28T23:04:37.955 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-28T23:04:37.955 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-28T23:04:37.956 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:04:37.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-28T23:04:37.957 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-28T23:04:37.957 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-28T23:04:37.960 INFO:tasks.workunit:Pulling workunits from ref 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:04:37.961 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-28T23:04:37.961 INFO:tasks.workunit:timeout=3h 2024-02-28T23:04:37.961 INFO:tasks.workunit:cleanup=True 2024-02-28T23:04:37.961 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:38.021 INFO:teuthology.orchestra.run.smithi057.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:38.021 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 9eh/158d Inode: 1 Links: 2 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-02-28 23:03:43.811847943 +0000 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-02-28 23:04:33.096855715 +0000 2024-02-28T23:04:38.022 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-02-28T23:04:38.022 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-28T23:04:38.022 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-28T23:04:38.039 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:37 smithi057 ceph-mon[29096]: pgmap v252: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:04:38.040 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:37 smithi084 ceph-mon[34826]: pgmap v252: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:04:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:38.143 DEBUG:teuthology.orchestra.run.smithi084:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-02-28 23:04:38.129214345 +0000 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-02-28 23:04:38.129214345 +0000 2024-02-28T23:04:38.204 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-02-28T23:04:38.205 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-28T23:04:38.205 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-28T23:04:38.316 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:04:38.316 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:04:38.366 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-28T23:04:38.371 INFO:tasks.workunit.client.1.smithi084.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-28T23:04:38.382 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:39 smithi057 ceph-mon[29096]: pgmap v253: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-28T23:04:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:39 smithi084 ceph-mon[34826]: pgmap v253: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 -- 172.21.15.57:0/1696952999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 msgr2=0x7f38d01030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 --2- 172.21.15.57:0/1696952999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d01030b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009910 tx=0x7f38c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 -- 172.21.15.57:0/1696952999 shutdown_connections 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 --2- 172.21.15.57:0/1696952999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 0x7f38d01059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 --2- 172.21.15.57:0/1696952999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d01030b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 -- 172.21.15.57:0/1696952999 >> 172.21.15.57:0/1696952999 conn(0x7f38d00fa6d0 msgr2=0x7f38d00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 -- 172.21.15.57:0/1696952999 shutdown_connections 2024-02-28T23:04:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.927+0000 7f38d6600700 1 -- 172.21.15.57:0/1696952999 wait complete. 2024-02-28T23:04:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 Processor -- start 2024-02-28T23:04:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 -- start start 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 0x7f38d00fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00ff1c0 con 0x7f38d0100cd0 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38d6600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00ff300 con 0x7f38d01035f0 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.929+0000 7f38cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41962/0 (socket says 172.21.15.57:41962) 2024-02-28T23:04:40.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 -- 172.21.15.57:0/1075074182 learned_addr learned my addr 172.21.15.57:0/1075074182 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cea00700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 0x7f38d00fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 -- 172.21.15.57:0/1075074182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 msgr2=0x7f38d00fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 0x7f38d00fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 -- 172.21.15.57:0/1075074182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00095f0 con 0x7f38d0100cd0 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.930+0000 7f38cf400700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38c0006010 tx=0x7f38c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:40.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.931+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c001d070 con 0x7f38d0100cd0 2024-02-28T23:04:40.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.931+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38c000f460 con 0x7f38d0100cd0 2024-02-28T23:04:40.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.931+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00ff580 con 0x7f38d0100cd0 2024-02-28T23:04:40.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.931+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c0017630 con 0x7f38d0100cd0 2024-02-28T23:04:40.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.931+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00ffa70 con 0x7f38d0100cd0 2024-02-28T23:04:40.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.932+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d018fc60 con 0x7f38d0100cd0 2024-02-28T23:04:40.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.933+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f38c0003a60 con 0x7f38d0100cd0 2024-02-28T23:04:40.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.934+0000 7f38c7e00700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 0x7f38b406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:40.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.934+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f38c008ccb0 con 0x7f38d0100cd0 2024-02-28T23:04:40.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.934+0000 7f38cea00700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 0x7f38b406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:40.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.935+0000 7f38cea00700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 0x7f38b406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005950 tx=0x7f38bc0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:40.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:40.940+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f38c005aef0 con 0x7f38d0100cd0 2024-02-28T23:04:41.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.162+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f38d002cc30 con 0x7f38d0100cd0 2024-02-28T23:04:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.163+0000 7f38c7e00700 1 -- 172.21.15.57:0/1075074182 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f38c005aa80 con 0x7f38d0100cd0 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.166+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 msgr2=0x7f38b406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.166+0000 7f38d6600700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 0x7f38b406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005950 tx=0x7f38bc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.166+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 msgr2=0x7f38d0100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.166+0000 7f38d6600700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38c0006010 tx=0x7f38c000be20 comp rx=0 tx=0).stop 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.166+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 shutdown_connections 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f38b406c680 0x7f38b406eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d0100cd0 0x7f38d0100630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 --2- 172.21.15.57:0/1075074182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d01035f0 0x7f38d00fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 >> 172.21.15.57:0/1075074182 conn(0x7f38d00fa6d0 msgr2=0x7f38d00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:41.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 shutdown_connections 2024-02-28T23:04:41.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:41.167+0000 7f38d6600700 1 -- 172.21.15.57:0/1075074182 wait complete. 2024-02-28T23:04:41.521 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:41 smithi057 ceph-mon[29096]: pgmap v254: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-02-28T23:04:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:41 smithi084 ceph-mon[34826]: pgmap v254: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-02-28T23:04:41.922 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-28T23:04:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:43.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:43 smithi057 ceph-mon[29096]: pgmap v255: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-02-28T23:04:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:43 smithi084 ceph-mon[34826]: pgmap v255: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-02-28T23:04:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 -- 172.21.15.57:0/630023400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee4102430 msgr2=0x7f6ee4102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/630023400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee4102430 0x7f6ee4102840 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400b0a0 tx=0x7f6ed400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 -- 172.21.15.57:0/630023400 shutdown_connections 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/630023400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee41036d0 0x7f6ee4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/630023400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee4102430 0x7f6ee4102840 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 -- 172.21.15.57:0/630023400 >> 172.21.15.57:0/630023400 conn(0x7f6ee40fda80 msgr2=0x7f6ee40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 -- 172.21.15.57:0/630023400 shutdown_connections 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.597+0000 7f6eeaa00700 1 -- 172.21.15.57:0/630023400 wait complete. 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.598+0000 7f6eeaa00700 1 Processor -- start 2024-02-28T23:04:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.598+0000 7f6eeaa00700 1 -- start start 2024-02-28T23:04:44.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6eeaa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee4102430 0x7f6ee4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6eeaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6eeaa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4198910 con 0x7f6ee41036d0 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6eeaa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4198a50 con 0x7f6ee4102430 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6edb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6edb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41992/0 (socket says 172.21.15.57:41992) 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6edb400700 1 -- 172.21.15.57:0/3422340318 learned_addr learned my addr 172.21.15.57:0/3422340318 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.599+0000 7f6ee3e00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee4102430 0x7f6ee4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6edb400700 1 -- 172.21.15.57:0/3422340318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee4102430 msgr2=0x7f6ee4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6edb400700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee4102430 0x7f6ee4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6edb400700 1 -- 172.21.15.57:0/3422340318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed4009d00 con 0x7f6ee41036d0 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6edb400700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b4b0 tx=0x7f6ecc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:44.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc011800 con 0x7f6ee41036d0 2024-02-28T23:04:44.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6ecc011e10 con 0x7f6ee41036d0 2024-02-28T23:04:44.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc00f2e0 con 0x7f6ee41036d0 2024-02-28T23:04:44.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.600+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee419d500 con 0x7f6ee41036d0 2024-02-28T23:04:44.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.601+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee419da50 con 0x7f6ee41036d0 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.602+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee410af90 con 0x7f6ee41036d0 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.603+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6ecc010380 con 0x7f6ee41036d0 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.604+0000 7f6ee1600700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 0x7f6ec806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.604+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6ecc08b4d0 con 0x7f6ee41036d0 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.604+0000 7f6ee3e00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 0x7f6ec806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:44.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.605+0000 7f6ee3e00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 0x7f6ec806eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4000f80 tx=0x7f6ed400b7b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:44.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.608+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6ecc059670 con 0x7f6ee41036d0 2024-02-28T23:04:44.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.831+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f6ee410b180 con 0x7f6ee41036d0 2024-02-28T23:04:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.832+0000 7f6ee1600700 1 -- 172.21.15.57:0/3422340318 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f6ecc059200 con 0x7f6ee41036d0 2024-02-28T23:04:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 msgr2=0x7f6ec806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 0x7f6ec806eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4000f80 tx=0x7f6ed400b7b0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 msgr2=0x7f6ee41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b4b0 tx=0x7f6ecc00b870 comp rx=0 tx=0).stop 2024-02-28T23:04:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 shutdown_connections 2024-02-28T23:04:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6ec806c680 0x7f6ec806eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ee4102430 0x7f6ee4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 --2- 172.21.15.57:0/3422340318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee41036d0 0x7f6ee41982f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:44.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 >> 172.21.15.57:0/3422340318 conn(0x7f6ee40fda80 msgr2=0x7f6ee4106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:44.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 shutdown_connections 2024-02-28T23:04:44.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:44.835+0000 7f6eeaa00700 1 -- 172.21.15.57:0/3422340318 wait complete. 2024-02-28T23:04:45.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-28T23:04:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:45 smithi057 ceph-mon[29096]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 4 op/s 2024-02-28T23:04:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:45 smithi084 ceph-mon[34826]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 4 op/s 2024-02-28T23:04:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:47.777 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:47 smithi057 ceph-mon[29096]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-28T23:04:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:47 smithi084 ceph-mon[34826]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-28T23:04:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.395+0000 7f996a000700 1 -- 172.21.15.57:0/2844881549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964101440 msgr2=0x7f9964101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.395+0000 7f996a000700 1 --2- 172.21.15.57:0/2844881549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964101440 0x7f9964101890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f994c00b0a0 tx=0x7f994c00b3b0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 -- 172.21.15.57:0/2844881549 shutdown_connections 2024-02-28T23:04:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 --2- 172.21.15.57:0/2844881549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964101440 0x7f9964101890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 --2- 172.21.15.57:0/2844881549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964100240 0x7f9964100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 -- 172.21.15.57:0/2844881549 >> 172.21.15.57:0/2844881549 conn(0x7f99640fb7f0 msgr2=0x7f99640fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 -- 172.21.15.57:0/2844881549 shutdown_connections 2024-02-28T23:04:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.396+0000 7f996a000700 1 -- 172.21.15.57:0/2844881549 wait complete. 2024-02-28T23:04:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.397+0000 7f996a000700 1 Processor -- start 2024-02-28T23:04:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.397+0000 7f996a000700 1 -- start start 2024-02-28T23:04:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f996a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f996a000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964101440 0x7f9964198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f996a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964198880 con 0x7f9964100240 2024-02-28T23:04:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f996a000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996419d290 con 0x7f9964101440 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f9963400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f9963400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51254/0 (socket says 172.21.15.57:51254) 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f9963400700 1 -- 172.21.15.57:0/1895792105 learned_addr learned my addr 172.21.15.57:0/1895792105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.398+0000 7f995aa00700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964101440 0x7f9964198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9963400700 1 -- 172.21.15.57:0/1895792105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964101440 msgr2=0x7f9964198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9963400700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964101440 0x7f9964198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9963400700 1 -- 172.21.15.57:0/1895792105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c009d00 con 0x7f9964100240 2024-02-28T23:04:48.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9963400700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f995400d900 tx=0x7f995400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99540041f0 con 0x7f9964100240 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9954004350 con 0x7f9964100240 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9954003da0 con 0x7f9964100240 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996419d490 con 0x7f9964100240 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.399+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996419d9b0 con 0x7f9964100240 2024-02-28T23:04:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.401+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964105520 con 0x7f9964100240 2024-02-28T23:04:48.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.402+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9954009730 con 0x7f9964100240 2024-02-28T23:04:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.403+0000 7f9960c00700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.403+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f995408ba90 con 0x7f9964100240 2024-02-28T23:04:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.403+0000 7f995aa00700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.404+0000 7f995aa00700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f994c000f80 tx=0x7f994c00b7f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:48.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.408+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f99540560f0 con 0x7f9964100240 2024-02-28T23:04:48.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.628+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f996402cc30 con 0x7f9964100240 2024-02-28T23:04:48.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.629+0000 7f9960c00700 1 -- 172.21.15.57:0/1895792105 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9954059710 con 0x7f9964100240 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 msgr2=0x7f994806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f994c000f80 tx=0x7f994c00b7f0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 msgr2=0x7f9964197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f995400d900 tx=0x7f995400dcc0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 shutdown_connections 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f994806c680 0x7f994806eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9964100240 0x7f9964197d20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 --2- 172.21.15.57:0/1895792105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9964101440 0x7f9964198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 >> 172.21.15.57:0/1895792105 conn(0x7f99640fb7f0 msgr2=0x7f9964102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 shutdown_connections 2024-02-28T23:04:48.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:48.632+0000 7f996a000700 1 -- 172.21.15.57:0/1895792105 wait complete. 2024-02-28T23:04:49.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-02-28T23:04:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:49 smithi057 ceph-mon[29096]: pgmap v258: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-28T23:04:49.942 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:04:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:49 smithi084 ceph-mon[34826]: pgmap v258: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-28T23:04:51.450 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:51 smithi057 ceph-mon[29096]: pgmap v259: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-28T23:04:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:51 smithi084 ceph-mon[34826]: pgmap v259: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-28T23:04:52.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 -- 172.21.15.57:0/972971344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c0737b0 msgr2=0x7f006c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:52.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 --2- 172.21.15.57:0/972971344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c0737b0 0x7f006c073c20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0058009910 tx=0x7f0058009c20 comp rx=0 tx=0).stop 2024-02-28T23:04:52.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 -- 172.21.15.57:0/972971344 shutdown_connections 2024-02-28T23:04:52.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 --2- 172.21.15.57:0/972971344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c0737b0 0x7f006c073c20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 --2- 172.21.15.57:0/972971344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c074d80 0x7f006c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 -- 172.21.15.57:0/972971344 >> 172.21.15.57:0/972971344 conn(0x7f006c0fb7f0 msgr2=0x7f006c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 -- 172.21.15.57:0/972971344 shutdown_connections 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.335+0000 7f0071000700 1 -- 172.21.15.57:0/972971344 wait complete. 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.336+0000 7f0071000700 1 Processor -- start 2024-02-28T23:04:52.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.336+0000 7f0071000700 1 -- start start 2024-02-28T23:04:52.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0071000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c0737b0 0x7f006c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:52.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0071000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:52.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0071000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c19cc40 con 0x7f006c074d80 2024-02-28T23:04:52.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0071000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c19cd80 con 0x7f006c0737b0 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51282/0 (socket says 172.21.15.57:51282) 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 -- 172.21.15.57:0/2917236000 learned_addr learned my addr 172.21.15.57:0/2917236000 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069e00700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c0737b0 0x7f006c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 -- 172.21.15.57:0/2917236000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c0737b0 msgr2=0x7f006c19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c0737b0 0x7f006c19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.337+0000 7f0069400700 1 -- 172.21.15.57:0/2917236000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580095f0 con 0x7f006c074d80 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0069400700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0058005b40 tx=0x7f005800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:52.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f005801d070 con 0x7f006c074d80 2024-02-28T23:04:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0058022470 con 0x7f006c074d80 2024-02-28T23:04:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c1a17d0 con 0x7f006c074d80 2024-02-28T23:04:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f005800f690 con 0x7f006c074d80 2024-02-28T23:04:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.338+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c1a1cc0 con 0x7f006c074d80 2024-02-28T23:04:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.339+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c10f3f0 con 0x7f006c074d80 2024-02-28T23:04:52.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.341+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f00580225e0 con 0x7f006c074d80 2024-02-28T23:04:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.341+0000 7f0062a00700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 0x7f005006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:04:52.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.342+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f005808d0f0 con 0x7f006c074d80 2024-02-28T23:04:52.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.342+0000 7f0069e00700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 0x7f005006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:04:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.343+0000 7f0069e00700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 0x7f005006eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:04:52.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.346+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f005805b260 con 0x7f006c074d80 2024-02-28T23:04:52.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:04:52.604+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f006c1032f0 con 0x7f005006c680 2024-02-28T23:04:53.004 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:04:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:53 smithi057 ceph-mon[29096]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-28T23:04:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:53 smithi057 ceph-mon[29096]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:04:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:53 smithi084 ceph-mon[34826]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-28T23:04:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:53 smithi084 ceph-mon[34826]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:04:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:55 smithi057 ceph-mon[29096]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-28T23:04:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:55 smithi084 ceph-mon[34826]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-28T23:04:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:57 smithi057 ceph-mon[29096]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-28T23:04:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:57 smithi084 ceph-mon[34826]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-28T23:05:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:04:59 smithi057 ceph-mon[29096]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:05:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:04:59 smithi084 ceph-mon[34826]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:05:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:01 smithi057 ceph-mon[29096]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:05:01.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:01 smithi084 ceph-mon[34826]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:05:03.933 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:03 smithi057 ceph-mon[29096]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:05:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:03 smithi084 ceph-mon[34826]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:05:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:05 smithi057 ceph-mon[29096]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:06.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:05 smithi084 ceph-mon[34826]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:07 smithi057 ceph-mon[29096]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:08.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:07 smithi084 ceph-mon[34826]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:09 smithi084 ceph-mon[34826]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:09 smithi057 ceph-mon[29096]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:11 smithi084 ceph-mon[34826]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:11 smithi057 ceph-mon[29096]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:13.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:13 smithi057 ceph-mon[29096]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:13 smithi084 ceph-mon[34826]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:15 smithi057 ceph-mon[29096]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:15 smithi084 ceph-mon[34826]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:17 smithi057 ceph-mon[29096]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:17 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:17 smithi084 ceph-mon[34826]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:17 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:05:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:05:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:18 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:05:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:05:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:18 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:19 smithi057 ceph-mon[29096]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:19 smithi084 ceph-mon[34826]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:21 smithi084 ceph-mon[34826]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:21 smithi057 ceph-mon[29096]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:23.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:23.983 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:23 smithi057 ceph-mon[29096]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:23 smithi084 ceph-mon[34826]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:25 smithi057 ceph-mon[29096]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:25 smithi084 ceph-mon[34826]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:27 smithi057 ceph-mon[29096]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:27 smithi084 ceph-mon[34826]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:29 smithi057 ceph-mon[29096]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:29 smithi084 ceph-mon[34826]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:31.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:31 smithi084 ceph-mon[34826]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:31 smithi057 ceph-mon[29096]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:33 smithi057 ceph-mon[29096]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:33 smithi084 ceph-mon[34826]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:35 smithi057 ceph-mon[29096]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:35 smithi084 ceph-mon[34826]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:37 smithi057 ceph-mon[29096]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:37 smithi084 ceph-mon[34826]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:38.947 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:05:38.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.947+0000 7f0062a00700 1 -- 172.21.15.57:0/2917236000 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f006c1032f0 con 0x7f005006c680 2024-02-28T23:05:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.950+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 msgr2=0x7f005006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.950+0000 7f0071000700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 0x7f005006eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c008040 comp rx=0 tx=0).stop 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.950+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 msgr2=0x7f006c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.950+0000 7f0071000700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0058005b40 tx=0x7f005800be20 comp rx=0 tx=0).stop 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.951+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 shutdown_connections 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.951+0000 7f0071000700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f005006c680 0x7f005006eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.951+0000 7f0071000700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006c0737b0 0x7f006c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.951+0000 7f0071000700 1 --2- 172.21.15.57:0/2917236000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f006c074d80 0x7f006c19c620 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:38.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.951+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 >> 172.21.15.57:0/2917236000 conn(0x7f006c0fb7f0 msgr2=0x7f006c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.952+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 shutdown_connections 2024-02-28T23:05:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:38.952+0000 7f0071000700 1 -- 172.21.15.57:0/2917236000 wait complete. 2024-02-28T23:05:39.673 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:39 smithi057 ceph-mon[29096]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:39.673 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:39.673 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:39.831 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-02-28T23:05:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:39 smithi084 ceph-mon[34826]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:40.360 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:05:40.792 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:40 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:05:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:05:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:05:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:40 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:42.018 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:41 smithi057 ceph-mon[29096]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:05:42.018 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:41 smithi057 ceph-mon[29096]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:41 smithi084 ceph-mon[34826]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:05:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:41 smithi084 ceph-mon[34826]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 -- 172.21.15.57:0/3895143311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0101014b0 msgr2=0x7fe010103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 --2- 172.21.15.57:0/3895143311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0101014b0 0x7fe010103930 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009910 tx=0x7fdffc009c20 comp rx=0 tx=0).stop 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 -- 172.21.15.57:0/3895143311 shutdown_connections 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 --2- 172.21.15.57:0/3895143311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0101014b0 0x7fe010103930 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 --2- 172.21.15.57:0/3895143311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0100feb50 0x7fe010100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 -- 172.21.15.57:0/3895143311 >> 172.21.15.57:0/3895143311 conn(0x7fe0100fa780 msgr2=0x7fe0100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 -- 172.21.15.57:0/3895143311 shutdown_connections 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.980+0000 7fe015400700 1 -- 172.21.15.57:0/3895143311 wait complete. 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.982+0000 7fe015400700 1 Processor -- start 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.982+0000 7fe015400700 1 -- start start 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.982+0000 7fe015400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0100feb50 0x7fe01019c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.982+0000 7fe015400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.982+0000 7fe015400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01019cc00 con 0x7fe0100feb50 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.983+0000 7fe015400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01019cd40 con 0x7fe0101014b0 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.983+0000 7fe00e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0100feb50 0x7fe01019c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.983+0000 7fe00de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.983+0000 7fe00de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:45900/0 (socket says 172.21.15.57:45900) 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.983+0000 7fe00de00700 1 -- 172.21.15.57:0/4183148460 learned_addr learned my addr 172.21.15.57:0/4183148460 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.984+0000 7fe00de00700 1 -- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0100feb50 msgr2=0x7fe01019c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.984+0000 7fe00de00700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0100feb50 0x7fe01019c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.984+0000 7fe00de00700 1 -- 172.21.15.57:0/4183148460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc0095f0 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.985+0000 7fe00de00700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0092e0 tx=0x7fdffc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.985+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc01d070 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.985+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0052a0 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.986+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0101a1790 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.986+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0101a1c80 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.986+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc017630 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.987+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0101961c0 con 0x7fe0101014b0 2024-02-28T23:05:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.989+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fdffc003680 con 0x7fe0101014b0 2024-02-28T23:05:43.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.990+0000 7fe007400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 0x7fdff406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.990+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdffc08cd60 con 0x7fe0101014b0 2024-02-28T23:05:43.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.991+0000 7fe00e800700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 0x7fdff406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.991+0000 7fe00e800700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 0x7fdff406eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe010069000 tx=0x7fe000009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:43.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:42.997+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdffc05ae50 con 0x7fe0101014b0 2024-02-28T23:05:43.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.428+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe010061190 con 0x7fdff406c680 2024-02-28T23:05:43.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.430+0000 7fe007400700 1 -- 172.21.15.57:0/4183148460 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fe010061190 con 0x7fdff406c680 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 msgr2=0x7fdff406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 0x7fdff406eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe010069000 tx=0x7fe000009260 comp rx=0 tx=0).stop 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 msgr2=0x7fe01019c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0092e0 tx=0x7fdffc00f690 comp rx=0 tx=0).stop 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 shutdown_connections 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fdff406c680 0x7fdff406eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0100feb50 0x7fe01019c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 --2- 172.21.15.57:0/4183148460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0101014b0 0x7fe01019c5e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 >> 172.21.15.57:0/4183148460 conn(0x7fe0100fa780 msgr2=0x7fe0100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:43.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.433+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 shutdown_connections 2024-02-28T23:05:43.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.434+0000 7fe015400700 1 -- 172.21.15.57:0/4183148460 wait complete. 2024-02-28T23:05:43.455 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 -- 172.21.15.57:0/3943851388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c074d20 msgr2=0x7f233c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 --2- 172.21.15.57:0/3943851388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c074d20 0x7f233c073180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2330009910 tx=0x7f2330009c20 comp rx=0 tx=0).stop 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 -- 172.21.15.57:0/3943851388 shutdown_connections 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 --2- 172.21.15.57:0/3943851388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c073750 0x7f233c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 --2- 172.21.15.57:0/3943851388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c074d20 0x7f233c073180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.669+0000 7f2344200700 1 -- 172.21.15.57:0/3943851388 >> 172.21.15.57:0/3943851388 conn(0x7f233c0fb7f0 msgr2=0x7f233c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:43.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.670+0000 7f2344200700 1 -- 172.21.15.57:0/3943851388 shutdown_connections 2024-02-28T23:05:43.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.670+0000 7f2344200700 1 -- 172.21.15.57:0/3943851388 wait complete. 2024-02-28T23:05:43.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 Processor -- start 2024-02-28T23:05:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 -- start start 2024-02-28T23:05:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c073750 0x7f233c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c1987e0 con 0x7f233c073750 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.672+0000 7f2344200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c198950 con 0x7f233c074d20 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:45912/0 (socket says 172.21.15.57:45912) 2024-02-28T23:05:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 -- 172.21.15.57:0/874907297 learned_addr learned my addr 172.21.15.57:0/874907297 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:05:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2341800700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c073750 0x7f233c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 -- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c073750 msgr2=0x7f233c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c073750 0x7f233c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.673+0000 7f2340e00700 1 -- 172.21.15.57:0/874907297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23300095f0 con 0x7f233c074d20 2024-02-28T23:05:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f2340e00700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f232800d8d0 tx=0x7f232800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2328009940 con 0x7f233c074d20 2024-02-28T23:05:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2328010460 con 0x7f233c074d20 2024-02-28T23:05:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c19d3e0 con 0x7f233c074d20 2024-02-28T23:05:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232800f5f0 con 0x7f233c074d20 2024-02-28T23:05:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.674+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c19d930 con 0x7f233c074d20 2024-02-28T23:05:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.676+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c10af80 con 0x7f233c074d20 2024-02-28T23:05:43.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.678+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f23280105d0 con 0x7f233c074d20 2024-02-28T23:05:43.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.680+0000 7f233a000700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:43.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.680+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f232808b8c0 con 0x7f233c074d20 2024-02-28T23:05:43.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.680+0000 7f2341800700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:43.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.681+0000 7f2341800700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 0x7f232406eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:43.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:43.688+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2328055f20 con 0x7f233c074d20 2024-02-28T23:05:44.061 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:43 smithi057 ceph-mon[29096]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:43 smithi084 ceph-mon[34826]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:44.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.140+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f233c1077a0 con 0x7f232406c680 2024-02-28T23:05:44.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.142+0000 7f233a000700 1 -- 172.21.15.57:0/874907297 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f233c1077a0 con 0x7f232406c680 2024-02-28T23:05:44.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 msgr2=0x7f232406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 0x7f232406eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fd0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 msgr2=0x7f233c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f232800d8d0 tx=0x7f232800dc90 comp rx=0 tx=0).stop 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 shutdown_connections 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f233c073750 0x7f233c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.146+0000 7f2344200700 1 --2- 172.21.15.57:0/874907297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233c074d20 0x7f233c1981e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.147+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 >> 172.21.15.57:0/874907297 conn(0x7f233c0fb7f0 msgr2=0x7f233c106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.147+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 shutdown_connections 2024-02-28T23:05:44.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.147+0000 7f2344200700 1 -- 172.21.15.57:0/874907297 wait complete. 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 -- 172.21.15.57:0/2623466762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8100d60 msgr2=0x7ff2f8103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 --2- 172.21.15.57:0/2623466762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8100d60 0x7ff2f8103140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009910 tx=0x7ff2e8009c20 comp rx=0 tx=0).stop 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 -- 172.21.15.57:0/2623466762 shutdown_connections 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 --2- 172.21.15.57:0/2623466762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8103680 0x7ff2f8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 --2- 172.21.15.57:0/2623466762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8100d60 0x7ff2f8103140 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 -- 172.21.15.57:0/2623466762 >> 172.21.15.57:0/2623466762 conn(0x7ff2f80fa780 msgr2=0x7ff2f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 -- 172.21.15.57:0/2623466762 shutdown_connections 2024-02-28T23:05:44.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.378+0000 7ff2fe800700 1 -- 172.21.15.57:0/2623466762 wait complete. 2024-02-28T23:05:44.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 Processor -- start 2024-02-28T23:05:44.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 -- start start 2024-02-28T23:05:44.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8103680 0x7ff2f8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8198860 con 0x7ff2f8103680 2024-02-28T23:05:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.380+0000 7ff2fe800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f819d270 con 0x7ff2f8100d60 2024-02-28T23:05:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8103680 0x7ff2f8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:45930/0 (socket says 172.21.15.57:45930) 2024-02-28T23:05:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 -- 172.21.15.57:0/3607199791 learned_addr learned my addr 172.21.15.57:0/3607199791 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 -- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8103680 msgr2=0x7ff2f8198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8103680 0x7ff2f8198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.381+0000 7ff2f7400700 1 -- 172.21.15.57:0/3607199791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e80095f0 con 0x7ff2f8100d60 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.382+0000 7ff2f7400700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bd70 tx=0x7ff2e800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.382+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e801d070 con 0x7ff2f8100d60 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.382+0000 7ff2fe800700 1 -- 172.21.15.57:0/3607199791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f819d410 con 0x7ff2f8100d60 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.382+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2e8005dc0 con 0x7ff2f8100d60 2024-02-28T23:05:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.382+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e8021410 con 0x7ff2f8100d60 2024-02-28T23:05:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.383+0000 7ff2fe800700 1 -- 172.21.15.57:0/3607199791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f819d8d0 con 0x7ff2f8100d60 2024-02-28T23:05:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.385+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d00052f0 con 0x7ff2f8100d60 2024-02-28T23:05:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.386+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff2e800f7f0 con 0x7ff2f8100d60 2024-02-28T23:05:44.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.387+0000 7ff2efe00700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 0x7ff2dc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.388+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff2e808cb10 con 0x7ff2f8100d60 2024-02-28T23:05:44.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.388+0000 7ff2f6a00700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 0x7ff2dc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.389+0000 7ff2f6a00700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 0x7ff2dc06eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005950 tx=0x7ff2e400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:44.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.395+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff2e80570f0 con 0x7ff2f8100d60 2024-02-28T23:05:44.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.645+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2d0000bc0 con 0x7ff2dc06c680 2024-02-28T23:05:44.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.658+0000 7ff2efe00700 1 -- 172.21.15.57:0/3607199791 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7ff2d0000bc0 con 0x7ff2dc06c680 2024-02-28T23:05:44.657 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:05:44.657 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (7m) 88s ago 9m 13.6M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (5m) 88s ago 9m 7578k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (8m) 93s ago 8m 7444k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (9m) 88s ago 9m 6665k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (8m) 93s ago 8m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (6m) 88s ago 8m 73.3M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (2m) 88s ago 2m 12.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (112s) 88s ago 111s 14.7M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (108s) 93s ago 108s 13.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (117s) 93s ago 117s 14.5M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (11m) 88s ago 11m 488M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (7m) 93s ago 7m 437M - 18.2.0 dc2bc1663786 4fe0352dacd9 2024-02-28T23:05:44.658 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (11m) 88s ago 11m 44.5M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (7m) 93s ago 7m 34.6M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (9m) 88s ago 9m 7768k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (7m) 93s ago 7m 8154k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 88s ago 6m 46.5M 3949M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 88s ago 6m 46.4M 3949M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (5m) 88s ago 5m 46.6M 3949M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (4m) 93s ago 4m 45.9M 4971M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (4m) 93s ago 4m 43.5M 4971M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (3m) 93s ago 3m 43.6M 4971M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:05:44.659 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (6m) 88s ago 8m 55.0M - 2.43.0 a07b618ecd1d ffacfc83a2dc 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.660+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 msgr2=0x7ff2dc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.660+0000 7ff2ed600700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 0x7ff2dc06eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005950 tx=0x7ff2e400b410 comp rx=0 tx=0).stop 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.660+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 msgr2=0x7ff2f8197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.660+0000 7ff2ed600700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bd70 tx=0x7ff2e800bda0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 shutdown_connections 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7ff2dc06c680 0x7ff2dc06eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f8100d60 0x7ff2f8197d00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 --2- 172.21.15.57:0/3607199791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2f8103680 0x7ff2f8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 >> 172.21.15.57:0/3607199791 conn(0x7ff2f80fa780 msgr2=0x7ff2f8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:44.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 shutdown_connections 2024-02-28T23:05:44.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.661+0000 7ff2ed600700 1 -- 172.21.15.57:0/3607199791 wait complete. 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 -- 172.21.15.57:0/2130622004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 msgr2=0x7f96c4100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 --2- 172.21.15.57:0/2130622004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4100840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009910 tx=0x7f96b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 -- 172.21.15.57:0/2130622004 shutdown_connections 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 --2- 172.21.15.57:0/2130622004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4100840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 --2- 172.21.15.57:0/2130622004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c40ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.811+0000 7f96c8e00700 1 -- 172.21.15.57:0/2130622004 >> 172.21.15.57:0/2130622004 conn(0x7f96c40fa780 msgr2=0x7f96c40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:44.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.812+0000 7f96c8e00700 1 -- 172.21.15.57:0/2130622004 shutdown_connections 2024-02-28T23:05:44.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.812+0000 7f96c8e00700 1 -- 172.21.15.57:0/2130622004 wait complete. 2024-02-28T23:05:44.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.813+0000 7f96c8e00700 1 Processor -- start 2024-02-28T23:05:44.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.814+0000 7f96c8e00700 1 -- start start 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.814+0000 7f96c8e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.814+0000 7f96c8e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.814+0000 7f96c8e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4071940 con 0x7f96c41003d0 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.814+0000 7f96c8e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4071ab0 con 0x7f96c40ff130 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:45954/0 (socket says 172.21.15.57:45954) 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 -- 172.21.15.57:0/637881531 learned_addr learned my addr 172.21.15.57:0/637881531 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:05:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1400700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 -- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 msgr2=0x7f96c4068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.815+0000 7f96c1e00700 1 -- 172.21.15.57:0/637881531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00095f0 con 0x7f96c40ff130 2024-02-28T23:05:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96c1e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96b400d8d0 tx=0x7f96b400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b4009940 con 0x7f96c40ff130 2024-02-28T23:05:44.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c4071d40 con 0x7f96c40ff130 2024-02-28T23:05:44.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96b4010460 con 0x7f96c40ff130 2024-02-28T23:05:44.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b400f5f0 con 0x7f96c40ff130 2024-02-28T23:05:44.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.816+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c4072290 con 0x7f96c40ff130 2024-02-28T23:05:44.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.818+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c4072570 con 0x7f96c40ff130 2024-02-28T23:05:44.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.820+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f96b40105d0 con 0x7f96c40ff130 2024-02-28T23:05:44.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.822+0000 7f96baa00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 0x7f96a806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:44.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.822+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f96b408b950 con 0x7f96c40ff130 2024-02-28T23:05:44.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.823+0000 7f96c1400700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 0x7f96a806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:44.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.824+0000 7f96c1400700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 0x7f96a806eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b000b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:44.827+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f96b40560e0 con 0x7f96c40ff130 2024-02-28T23:05:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:44 smithi057 ceph-mon[29096]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:44 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:45.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:44 smithi084 ceph-mon[34826]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:45.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:45.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:05:45.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:44 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:05:45.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.308+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f96c41a2380 con 0x7f96c40ff130 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.308+0000 7f96baa00700 1 -- 172.21.15.57:0/637881531 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f96b4059700 con 0x7f96c40ff130 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:05:45.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:05:45.309 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:05:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 msgr2=0x7f96a806eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 0x7f96a806eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b000b560 comp rx=0 tx=0).stop 2024-02-28T23:05:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 msgr2=0x7f96c4068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96b400d8d0 tx=0x7f96b400dc90 comp rx=0 tx=0).stop 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 shutdown_connections 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f96a806c6d0 0x7f96a806eb80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c40ff130 0x7f96c4068aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 --2- 172.21.15.57:0/637881531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96c41003d0 0x7f96c4068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.313+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 >> 172.21.15.57:0/637881531 conn(0x7f96c40fa780 msgr2=0x7f96c4103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.314+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 shutdown_connections 2024-02-28T23:05:45.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.314+0000 7f96c8e00700 1 -- 172.21.15.57:0/637881531 wait complete. 2024-02-28T23:05:45.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 -- 172.21.15.57:0/1236392298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e00feb50 msgr2=0x7f78e0100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 --2- 172.21.15.57:0/1236392298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e00feb50 0x7f78e0100f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d0009b80 comp rx=0 tx=0).stop 2024-02-28T23:05:45.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 -- 172.21.15.57:0/1236392298 shutdown_connections 2024-02-28T23:05:45.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 --2- 172.21.15.57:0/1236392298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e01014b0 0x7f78e0103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 --2- 172.21.15.57:0/1236392298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e00feb50 0x7f78e0100f70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 -- 172.21.15.57:0/1236392298 >> 172.21.15.57:0/1236392298 conn(0x7f78e00fa780 msgr2=0x7f78e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:45.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 -- 172.21.15.57:0/1236392298 shutdown_connections 2024-02-28T23:05:45.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.515+0000 7f78e6e00700 1 -- 172.21.15.57:0/1236392298 wait complete. 2024-02-28T23:05:45.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.517+0000 7f78e6e00700 1 Processor -- start 2024-02-28T23:05:45.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.517+0000 7f78e6e00700 1 -- start start 2024-02-28T23:05:45.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78e6e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78e6e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 0x7f78e019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78e6e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e019cc60 con 0x7f78e00feb50 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78e6e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e019cda0 con 0x7f78e01014b0 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78dfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78dfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55878/0 (socket says 172.21.15.57:55878) 2024-02-28T23:05:45.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78dfe00700 1 -- 172.21.15.57:0/2006055142 learned_addr learned my addr 172.21.15.57:0/2006055142 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.518+0000 7f78df400700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 0x7f78e019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.519+0000 7f78dfe00700 1 -- 172.21.15.57:0/2006055142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 msgr2=0x7f78e019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.519+0000 7f78dfe00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 0x7f78e019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.519+0000 7f78dfe00700 1 -- 172.21.15.57:0/2006055142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009520 con 0x7f78e00feb50 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.519+0000 7f78df400700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 0x7f78e019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:05:45.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.519+0000 7f78dfe00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009f50 tx=0x7f78d000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:45.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.520+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d001c070 con 0x7f78e00feb50 2024-02-28T23:05:45.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.520+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f78d000be80 con 0x7f78e00feb50 2024-02-28T23:05:45.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.520+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d00173f0 con 0x7f78e00feb50 2024-02-28T23:05:45.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.520+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e01a17a0 con 0x7f78e00feb50 2024-02-28T23:05:45.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.520+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e01a1c40 con 0x7f78e00feb50 2024-02-28T23:05:45.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.522+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e01961c0 con 0x7f78e00feb50 2024-02-28T23:05:45.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.524+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f78d0020070 con 0x7f78e00feb50 2024-02-28T23:05:45.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.525+0000 7f78dcc00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 0x7f78c406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:05:45.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.526+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f78d008cc00 con 0x7f78e00feb50 2024-02-28T23:05:45.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.526+0000 7f78df400700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 0x7f78c406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:05:45.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.527+0000 7f78df400700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 0x7f78c406eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005fd0 tx=0x7f78cc009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:05:45.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.535+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f78d0092020 con 0x7f78e00feb50 2024-02-28T23:05:45.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.781+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78e0061190 con 0x7f78c406c680 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.783+0000 7f78dcc00700 1 -- 172.21.15.57:0/2006055142 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f78e0061190 con 0x7f78c406c680 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/2 daemons upgraded", 2024-02-28T23:05:45.782 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc image on host smithi084", 2024-02-28T23:05:45.783 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:05:45.783 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 msgr2=0x7f78c406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 0x7f78c406eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005fd0 tx=0x7f78cc009310 comp rx=0 tx=0).stop 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 msgr2=0x7f78e019c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009f50 tx=0x7f78d000f740 comp rx=0 tx=0).stop 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 shutdown_connections 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f78c406c680 0x7f78c406eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e00feb50 0x7f78e019c100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 --2- 172.21.15.57:0/2006055142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e01014b0 0x7f78e019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.785+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 >> 172.21.15.57:0/2006055142 conn(0x7f78e00fa780 msgr2=0x7f78e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.786+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 shutdown_connections 2024-02-28T23:05:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:05:45.786+0000 7f78e6e00700 1 -- 172.21.15.57:0/2006055142 wait complete. 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: Upgrade: Target is version 19.0.0-1670-g1fb5d106 (unknown) 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc'] 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: Upgrade: Need to upgrade myself (mgr.smithi057.zrrldj) 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc on smithi084 2024-02-28T23:05:46.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:45 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/637881531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:05:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: Upgrade: Target is version 19.0.0-1670-g1fb5d106 (unknown) 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc'] 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: Upgrade: Need to upgrade myself (mgr.smithi057.zrrldj) 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc on smithi084 2024-02-28T23:05:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:45 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/637881531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:05:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:46 smithi057 ceph-mon[29096]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:47.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:46 smithi084 ceph-mon[34826]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:05:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:47 smithi057 ceph-mon[29096]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:48.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:47 smithi084 ceph-mon[34826]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:49.960 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:49 smithi057 ceph-mon[29096]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:49 smithi084 ceph-mon[34826]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:05:51.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:51 smithi084 ceph-mon[34826]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:51 smithi057 ceph-mon[29096]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:52 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:52 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:05:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:53 smithi057 ceph-mon[29096]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:53 smithi084 ceph-mon[34826]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:55 smithi057 ceph-mon[29096]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:55 smithi084 ceph-mon[34826]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:05:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:57 smithi057 ceph-mon[29096]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:05:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:57 smithi084 ceph-mon[34826]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:06:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:05:59 smithi057 ceph-mon[29096]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:06:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:05:59 smithi084 ceph-mon[34826]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:06:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:01 smithi057 ceph-mon[29096]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:01 smithi084 ceph-mon[34826]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:03 smithi057 ceph-mon[29096]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:06:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:03 smithi084 ceph-mon[34826]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:06:06.053 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 61% (7596/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-02-28T23:06:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:05 smithi057 ceph-mon[29096]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:06.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:05 smithi084 ceph-mon[34826]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:06.171 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '1fb5d10608dccee73edd6ef8a2c7b804a097c324'. 2024-02-28T23:06:06.171 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.171 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.172 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-28T23:06:06.173 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-02-28T23:06:06.173 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 1fb5d10608d Merge pull request #55565 from BBoozmen/oozmen_avoid_path_normalization 2024-02-28T23:06:06.174 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-28T23:06:06.234 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-28T23:06:06.242 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-28T23:06:06.242 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-28T23:06:06.961 INFO:tasks.workunit.client.1.smithi084.stderr:Updating files: 59% (7243/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 94% (11548/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-02-28T23:06:07.139 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-28T23:06:07.279 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-28T23:06:07.373 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-28T23:06:07.379 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-28T23:06:07.379 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-28T23:06:07.466 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-28T23:06:07.475 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T23:06:07.475 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-28T23:06:07.526 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-28T23:06:07.529 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-28T23:06:07.529 DEBUG:teuthology.orchestra.run.smithi057:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1fb5d10608dccee73edd6ef8a2c7b804a097c324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-28T23:06:07.618 INFO:tasks.workunit.client.0.smithi057.stderr:+ mkdir -p fsstress 2024-02-28T23:06:07.627 INFO:tasks.workunit.client.0.smithi057.stderr:+ pushd fsstress 2024-02-28T23:06:07.632 INFO:tasks.workunit.client.0.smithi057.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-28T23:06:07.632 INFO:tasks.workunit.client.0.smithi057.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:Note: switching to '1fb5d10608dccee73edd6ef8a2c7b804a097c324'. 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:state without impacting any branches by switching back to a branch. 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: git switch -c 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr:Or undo this operation with: 2024-02-28T23:06:07.634 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.635 INFO:tasks.workunit.client.1.smithi084.stderr: git switch - 2024-02-28T23:06:07.635 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.635 INFO:tasks.workunit.client.1.smithi084.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-28T23:06:07.635 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-02-28T23:06:07.635 INFO:tasks.workunit.client.1.smithi084.stderr:HEAD is now at 1fb5d10608d Merge pull request #55565 from BBoozmen/oozmen_avoid_path_normalization 2024-02-28T23:06:07.639 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-28T23:06:07.701 INFO:tasks.workunit.client.1.smithi084.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-28T23:06:07.707 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-28T23:06:07.708 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-28T23:06:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:07 smithi057 ceph-mon[29096]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:06:08.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:07 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:08.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:07 smithi084 ceph-mon[34826]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:06:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:07 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:08.657 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-28T23:06:08.781 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-28T23:06:08.858 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-28T23:06:08.862 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-28T23:06:08.862 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-28T23:06:08.941 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-28T23:06:08.950 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-02-28T23:06:08.950 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-28T23:06:08.993 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-28T23:06:08.994 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-28T23:06:08.995 DEBUG:teuthology.orchestra.run.smithi084:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1fb5d10608dccee73edd6ef8a2c7b804a097c324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-28T23:06:09.108 INFO:tasks.workunit.client.1.smithi084.stderr:+ mkdir -p fsstress 2024-02-28T23:06:09.114 INFO:tasks.workunit.client.1.smithi084.stderr:+ pushd fsstress 2024-02-28T23:06:09.119 INFO:tasks.workunit.client.1.smithi084.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-28T23:06:09.120 INFO:tasks.workunit.client.1.smithi084.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-28T23:06:09.661 INFO:tasks.workunit.client.0.smithi057.stderr:+ tar xzf ltp-full.tgz 2024-02-28T23:06:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:09 smithi057 ceph-mon[29096]: pgmap v298: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:06:10.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:09 smithi084 ceph-mon[34826]: pgmap v298: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:06:10.114 INFO:tasks.workunit.client.1.smithi084.stderr:+ tar xzf ltp-full.tgz 2024-02-28T23:06:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:11 smithi057 ceph-mon[29096]: pgmap v299: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:11.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:11 smithi084 ceph-mon[34826]: pgmap v299: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:06:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:13 smithi057 ceph-mon[29096]: pgmap v300: 65 pgs: 65 active+clean; 9.8 MiB data, 191 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 792 KiB/s wr, 16 op/s 2024-02-28T23:06:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:13 smithi084 ceph-mon[34826]: pgmap v300: 65 pgs: 65 active+clean; 9.8 MiB data, 191 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 792 KiB/s wr, 16 op/s 2024-02-28T23:06:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:15 smithi057 ceph-mon[29096]: pgmap v301: 65 pgs: 65 active+clean; 24 MiB data, 238 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 74 op/s 2024-02-28T23:06:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 -- 172.21.15.57:0/1938780643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c069270 msgr2=0x7f6c5c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 --2- 172.21.15.57:0/1938780643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c069270 0x7f6c5c102dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009910 tx=0x7f6c4c009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 -- 172.21.15.57:0/1938780643 shutdown_connections 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 --2- 172.21.15.57:0/1938780643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 0x7f6c5c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 --2- 172.21.15.57:0/1938780643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c069270 0x7f6c5c102dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.996+0000 7f6c61e00700 1 -- 172.21.15.57:0/1938780643 >> 172.21.15.57:0/1938780643 conn(0x7f6c5c0fa780 msgr2=0x7f6c5c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.997+0000 7f6c61e00700 1 -- 172.21.15.57:0/1938780643 shutdown_connections 2024-02-28T23:06:15.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.997+0000 7f6c61e00700 1 -- 172.21.15.57:0/1938780643 wait complete. 2024-02-28T23:06:15.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.998+0000 7f6c61e00700 1 Processor -- start 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.998+0000 7f6c61e00700 1 -- start start 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c61e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 0x7f6c5c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c61e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c61e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0ff720 con 0x7f6c5c0fed70 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c61e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0ff890 con 0x7f6c5c103310 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c5a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c5a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53308/0 (socket says 172.21.15.57:53308) 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c5a000700 1 -- 172.21.15.57:0/3953590804 learned_addr learned my addr 172.21.15.57:0/3953590804 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:15.999+0000 7f6c5aa00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 0x7f6c5c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.000+0000 7f6c5a000700 1 -- 172.21.15.57:0/3953590804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 msgr2=0x7f6c5c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.000+0000 7f6c5a000700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 0x7f6c5c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.000+0000 7f6c5a000700 1 -- 172.21.15.57:0/3953590804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c0095f0 con 0x7f6c5c0fed70 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.000+0000 7f6c5a000700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e920 tx=0x7f6c4800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.001+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c48009790 con 0x7f6c5c0fed70 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.001+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c4800cba0 con 0x7f6c5c0fed70 2024-02-28T23:06:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.001+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c48018630 con 0x7f6c5c0fed70 2024-02-28T23:06:16.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.001+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c0ffb70 con 0x7f6c5c0fed70 2024-02-28T23:06:16.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.001+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c19dd40 con 0x7f6c5c0fed70 2024-02-28T23:06:16.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.002+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c18fbd0 con 0x7f6c5c0fed70 2024-02-28T23:06:16.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.004+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6c4800cd10 con 0x7f6c5c0fed70 2024-02-28T23:06:16.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.005+0000 7f6c60a00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 0x7f6c4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:16.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.005+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6c48014070 con 0x7f6c5c0fed70 2024-02-28T23:06:16.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.005+0000 7f6c5aa00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 0x7f6c4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:16.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.006+0000 7f6c5aa00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 0x7f6c4006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006010 tx=0x7f6c4c005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:16.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.014+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6c48059b90 con 0x7f6c5c0fed70 2024-02-28T23:06:16.028 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:15 smithi084 ceph-mon[34826]: pgmap v301: 65 pgs: 65 active+clean; 24 MiB data, 238 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 74 op/s 2024-02-28T23:06:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.324+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c5c02cf80 con 0x7f6c4006c680 2024-02-28T23:06:16.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.326+0000 7f6c60a00700 1 -- 172.21.15.57:0/3953590804 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6c5c02cf80 con 0x7f6c4006c680 2024-02-28T23:06:16.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 msgr2=0x7f6c4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 0x7f6c4006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006010 tx=0x7f6c4c005940 comp rx=0 tx=0).stop 2024-02-28T23:06:16.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 msgr2=0x7f6c5c0ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e920 tx=0x7f6c4800ece0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 shutdown_connections 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f6c4006c680 0x7f6c4006eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c103310 0x7f6c5c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 --2- 172.21.15.57:0/3953590804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c5c0fed70 0x7f6c5c0ff1e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.330+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 >> 172.21.15.57:0/3953590804 conn(0x7f6c5c0fa780 msgr2=0x7f6c5c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.331+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 shutdown_connections 2024-02-28T23:06:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.331+0000 7f6c61e00700 1 -- 172.21.15.57:0/3953590804 wait complete. 2024-02-28T23:06:16.354 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:06:16.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 -- 172.21.15.57:0/2926299950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 msgr2=0x7f3140103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 --2- 172.21.15.57:0/2926299950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f3140103140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3130009910 tx=0x7f3130009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 -- 172.21.15.57:0/2926299950 shutdown_connections 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 --2- 172.21.15.57:0/2926299950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f3140105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 --2- 172.21.15.57:0/2926299950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f3140103140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.537+0000 7f3144e00700 1 -- 172.21.15.57:0/2926299950 >> 172.21.15.57:0/2926299950 conn(0x7f31400fa780 msgr2=0x7f31400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.538+0000 7f3144e00700 1 -- 172.21.15.57:0/2926299950 shutdown_connections 2024-02-28T23:06:16.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.538+0000 7f3144e00700 1 -- 172.21.15.57:0/2926299950 wait complete. 2024-02-28T23:06:16.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 Processor -- start 2024-02-28T23:06:16.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 -- start start 2024-02-28T23:06:16.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:16.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f31400fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31400ff250 con 0x7f3140100d60 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.539+0000 7f3144e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31400ff390 con 0x7f3140103680 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.540+0000 7f313de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.540+0000 7f313d400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f31400fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.540+0000 7f313de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53316/0 (socket says 172.21.15.57:53316) 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.540+0000 7f313d400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f31400fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:58878/0 (socket says 172.21.15.57:58878) 2024-02-28T23:06:16.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.540+0000 7f313de00700 1 -- 172.21.15.57:0/54605554 learned_addr learned my addr 172.21.15.57:0/54605554 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:16.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.541+0000 7f313de00700 1 -- 172.21.15.57:0/54605554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 msgr2=0x7f31400fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.541+0000 7f313de00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f31400fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.541+0000 7f313de00700 1 -- 172.21.15.57:0/54605554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31300095f0 con 0x7f3140100d60 2024-02-28T23:06:16.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.541+0000 7f313de00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3130000c00 tx=0x7f313000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:16.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.541+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f313001d070 con 0x7f3140100d60 2024-02-28T23:06:16.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.542+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3130005640 con 0x7f3140100d60 2024-02-28T23:06:16.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.542+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3130017630 con 0x7f3140100d60 2024-02-28T23:06:16.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.542+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31400ff610 con 0x7f3140100d60 2024-02-28T23:06:16.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.542+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31400ffb00 con 0x7f3140100d60 2024-02-28T23:06:16.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.544+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314018fbc0 con 0x7f3140100d60 2024-02-28T23:06:16.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.545+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f31300051d0 con 0x7f3140100d60 2024-02-28T23:06:16.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.547+0000 7f3136a00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 0x7f312406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:16.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.547+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f313008cb40 con 0x7f3140100d60 2024-02-28T23:06:16.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.547+0000 7f313d400700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 0x7f312406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:16.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.548+0000 7f313d400700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 0x7f312406ebd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f312c009c60 tx=0x7f312c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.553+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f313005ad70 con 0x7f3140100d60 2024-02-28T23:06:16.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.884+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3140061190 con 0x7f312406c720 2024-02-28T23:06:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.886+0000 7f3136a00700 1 -- 172.21.15.57:0/54605554 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3140061190 con 0x7f312406c720 2024-02-28T23:06:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.889+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 msgr2=0x7f312406ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.889+0000 7f3144e00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 0x7f312406ebd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f312c009c60 tx=0x7f312c009260 comp rx=0 tx=0).stop 2024-02-28T23:06:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.889+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 msgr2=0x7f31401006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.889+0000 7f3144e00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3130000c00 tx=0x7f313000fa00 comp rx=0 tx=0).stop 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 shutdown_connections 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f312406c720 0x7f312406ebd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3140100d60 0x7f31401006c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 --2- 172.21.15.57:0/54605554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3140103680 0x7f31400fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 >> 172.21.15.57:0/54605554 conn(0x7f31400fa780 msgr2=0x7f31400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 shutdown_connections 2024-02-28T23:06:16.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:16.890+0000 7f3144e00700 1 -- 172.21.15.57:0/54605554 wait complete. 2024-02-28T23:06:17.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 -- 172.21.15.57:0/3977260470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14074ce0 msgr2=0x7f3d140731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3977260470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14074ce0 0x7f3d140731e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009910 tx=0x7f3d08009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 -- 172.21.15.57:0/3977260470 shutdown_connections 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3977260470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14073720 0x7f3d14073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3977260470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14074ce0 0x7f3d140731e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 -- 172.21.15.57:0/3977260470 >> 172.21.15.57:0/3977260470 conn(0x7f3d140fb650 msgr2=0x7f3d140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.082+0000 7f3d1b000700 1 -- 172.21.15.57:0/3977260470 shutdown_connections 2024-02-28T23:06:17.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.083+0000 7f3d1b000700 1 -- 172.21.15.57:0/3977260470 wait complete. 2024-02-28T23:06:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.084+0000 7f3d1b000700 1 Processor -- start 2024-02-28T23:06:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.084+0000 7f3d1b000700 1 -- start start 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d1b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d1b000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14074ce0 0x7f3d1419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d1b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1419c9a0 con 0x7f3d14073720 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d1b000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1419cae0 con 0x7f3d14074ce0 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d19c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d19c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53326/0 (socket says 172.21.15.57:53326) 2024-02-28T23:06:17.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.085+0000 7f3d19c00700 1 -- 172.21.15.57:0/3384994178 learned_addr learned my addr 172.21.15.57:0/3384994178 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:17.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d19200700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14074ce0 0x7f3d1419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d19c00700 1 -- 172.21.15.57:0/3384994178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14074ce0 msgr2=0x7f3d1419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d19c00700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14074ce0 0x7f3d1419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d19c00700 1 -- 172.21.15.57:0/3384994178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080095f0 con 0x7f3d14073720 2024-02-28T23:06:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d19c00700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005b40 tx=0x7f3d0800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0801d070 con 0x7f3d14073720 2024-02-28T23:06:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.086+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d141a1530 con 0x7f3d14073720 2024-02-28T23:06:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.087+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d08005dc0 con 0x7f3d14073720 2024-02-28T23:06:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.087+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d080226a0 con 0x7f3d14073720 2024-02-28T23:06:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.087+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d141a1a20 con 0x7f3d14073720 2024-02-28T23:06:17.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.089+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1410f0e0 con 0x7f3d14073720 2024-02-28T23:06:17.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.091+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3d08003680 con 0x7f3d14073720 2024-02-28T23:06:17.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.092+0000 7f3d12000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 0x7f3cfc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.092+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3d0808cee0 con 0x7f3d14073720 2024-02-28T23:06:17.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.093+0000 7f3d19200700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 0x7f3cfc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.094+0000 7f3d19200700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 0x7f3cfc06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009bf0 tx=0x7f3d00009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:17.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.098+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3d0805aff0 con 0x7f3d14073720 2024-02-28T23:06:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.429+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d141a2120 con 0x7f3cfc06c680 2024-02-28T23:06:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.445+0000 7f3d12000700 1 -- 172.21.15.57:0/3384994178 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f3d141a2120 con 0x7f3cfc06c680 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (7m) 2m ago 9m 13.6M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (6m) 2m ago 10m 7578k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (8m) 2m ago 8m 7444k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (10m) 2m ago 10m 6665k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (8m) 2m ago 8m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (7m) 2m ago 9m 73.3M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (2m) 2m ago 2m 12.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (2m) 2m ago 2m 14.7M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:06:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (2m) 2m ago 2m 13.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (2m) 2m ago 2m 14.5M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (11m) 2m ago 11m 488M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (8m) 2m ago 8m 437M - 18.2.0 dc2bc1663786 4fe0352dacd9 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (11m) 2m ago 11m 44.5M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (8m) 2m ago 8m 34.6M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (9m) 2m ago 9m 7768k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (8m) 2m ago 8m 8154k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (7m) 2m ago 7m 46.5M 3949M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 2m ago 6m 46.4M 3949M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (6m) 2m ago 6m 46.6M 3949M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (5m) 2m ago 5m 45.9M 4971M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (4m) 2m ago 4m 43.5M 4971M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (4m) 2m ago 4m 43.6M 4971M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:06:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (7m) 2m ago 9m 55.0M - 2.43.0 a07b618ecd1d ffacfc83a2dc 2024-02-28T23:06:17.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.448+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 msgr2=0x7f3cfc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.448+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 0x7f3cfc06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009bf0 tx=0x7f3d00009260 comp rx=0 tx=0).stop 2024-02-28T23:06:17.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.448+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 msgr2=0x7f3d1419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.448+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005b40 tx=0x7f3d0800fab0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 shutdown_connections 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f3cfc06c680 0x7f3cfc06eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d14073720 0x7f3d1419be40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 --2- 172.21.15.57:0/3384994178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d14074ce0 0x7f3d1419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 >> 172.21.15.57:0/3384994178 conn(0x7f3d140fb650 msgr2=0x7f3d14101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:17.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 shutdown_connections 2024-02-28T23:06:17.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.449+0000 7f3d1b000700 1 -- 172.21.15.57:0/3384994178 wait complete. 2024-02-28T23:06:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 -- 172.21.15.57:0/457404697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 msgr2=0x7fe890101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 --2- 172.21.15.57:0/457404697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe890101890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009910 tx=0x7fe87c009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 -- 172.21.15.57:0/457404697 shutdown_connections 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 --2- 172.21.15.57:0/457404697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe890101890 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 --2- 172.21.15.57:0/457404697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 0x7fe890100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 -- 172.21.15.57:0/457404697 >> 172.21.15.57:0/457404697 conn(0x7fe8900fb7f0 msgr2=0x7fe8900fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 -- 172.21.15.57:0/457404697 shutdown_connections 2024-02-28T23:06:17.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.644+0000 7fe895200700 1 -- 172.21.15.57:0/457404697 wait complete. 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.645+0000 7fe895200700 1 Processor -- start 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.646+0000 7fe895200700 1 -- start start 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.646+0000 7fe895200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 0x7fe890197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.646+0000 7fe895200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.646+0000 7fe895200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890198740 con 0x7fe890101440 2024-02-28T23:06:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.646+0000 7fe895200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8901988b0 con 0x7fe890100240 2024-02-28T23:06:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.647+0000 7fe88d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.647+0000 7fe88de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 0x7fe890197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.647+0000 7fe88d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53350/0 (socket says 172.21.15.57:53350) 2024-02-28T23:06:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.647+0000 7fe88d400700 1 -- 172.21.15.57:0/2016478355 learned_addr learned my addr 172.21.15.57:0/2016478355 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:17.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.648+0000 7fe88d400700 1 -- 172.21.15.57:0/2016478355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 msgr2=0x7fe890197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:17.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.648+0000 7fe88d400700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 0x7fe890197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:17.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.648+0000 7fe88d400700 1 -- 172.21.15.57:0/2016478355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0095f0 con 0x7fe890101440 2024-02-28T23:06:17.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.648+0000 7fe88d400700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00b5c0 tx=0x7fe87c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:17.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.649+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c01d070 con 0x7fe890101440 2024-02-28T23:06:17.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.649+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe87c022470 con 0x7fe890101440 2024-02-28T23:06:17.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.649+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c00f690 con 0x7fe890101440 2024-02-28T23:06:17.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.649+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89019d2f0 con 0x7fe890101440 2024-02-28T23:06:17.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.649+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89019d7e0 con 0x7fe890101440 2024-02-28T23:06:17.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.652+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe890105520 con 0x7fe890101440 2024-02-28T23:06:17.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.653+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe87c0225e0 con 0x7fe890101440 2024-02-28T23:06:17.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.654+0000 7fe886a00700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 0x7fe87406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:17.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.654+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe87c08cf20 con 0x7fe890101440 2024-02-28T23:06:17.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.655+0000 7fe88de00700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 0x7fe87406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:17.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.656+0000 7fe88de00700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 0x7fe87406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe890198b70 tx=0x7fe88000a3b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:17.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:17.661+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe87c0bf720 con 0x7fe890101440 2024-02-28T23:06:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:17 smithi057 ceph-mon[29096]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:17 smithi057 ceph-mon[29096]: pgmap v302: 65 pgs: 65 active+clean; 31 MiB data, 277 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.6 MiB/s wr, 135 op/s 2024-02-28T23:06:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:17 smithi084 ceph-mon[34826]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:17 smithi084 ceph-mon[34826]: pgmap v302: 65 pgs: 65 active+clean; 31 MiB data, 277 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.6 MiB/s wr, 135 op/s 2024-02-28T23:06:18.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.112+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe89002cc30 con 0x7fe890101440 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.113+0000 7fe886a00700 1 -- 172.21.15.57:0/2016478355 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fe87c057620 con 0x7fe890101440 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:18.111 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:06:18.112 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 msgr2=0x7fe87406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 0x7fe87406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe890198b70 tx=0x7fe88000a3b0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 msgr2=0x7fe8901981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00b5c0 tx=0x7fe87c00bd70 comp rx=0 tx=0).stop 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 shutdown_connections 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe87406c680 0x7fe87406eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe890100240 0x7fe890197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 --2- 172.21.15.57:0/2016478355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe890101440 0x7fe8901981d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.116+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 >> 172.21.15.57:0/2016478355 conn(0x7fe8900fb7f0 msgr2=0x7fe890102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.117+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 shutdown_connections 2024-02-28T23:06:18.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.117+0000 7fe895200700 1 -- 172.21.15.57:0/2016478355 wait complete. 2024-02-28T23:06:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.311+0000 7fb3fd600700 1 -- 172.21.15.57:0/1254245116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8103350 msgr2=0x7fb3f8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.311+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1254245116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8103350 0x7fb3f8105790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009910 tx=0x7fb3e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 -- 172.21.15.57:0/1254245116 shutdown_connections 2024-02-28T23:06:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1254245116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8103350 0x7fb3f8105790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1254245116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8069230 0x7fb3f8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 -- 172.21.15.57:0/1254245116 >> 172.21.15.57:0/1254245116 conn(0x7fb3f80fa780 msgr2=0x7fb3f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 -- 172.21.15.57:0/1254245116 shutdown_connections 2024-02-28T23:06:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.312+0000 7fb3fd600700 1 -- 172.21.15.57:0/1254245116 wait complete. 2024-02-28T23:06:18.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 Processor -- start 2024-02-28T23:06:18.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 -- start start 2024-02-28T23:06:18.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:18.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 0x7fb3f8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8198850 con 0x7fb3f8069230 2024-02-28T23:06:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.314+0000 7fb3fd600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819d260 con 0x7fb3f8103350 2024-02-28T23:06:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53358/0 (socket says 172.21.15.57:53358) 2024-02-28T23:06:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 -- 172.21.15.57:0/1497847459 learned_addr learned my addr 172.21.15.57:0/1497847459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6000700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 0x7fb3f8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 -- 172.21.15.57:0/1497847459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 msgr2=0x7fb3f8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 0x7fb3f8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6a00700 1 -- 172.21.15.57:0/1497847459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40095f0 con 0x7fb3f8069230 2024-02-28T23:06:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.315+0000 7fb3f6000700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 0x7fb3f8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:06:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3f6a00700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d8d0 tx=0x7fb3e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e80041f0 con 0x7fb3f8069230 2024-02-28T23:06:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb3e8004350 con 0x7fb3f8069230 2024-02-28T23:06:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e8003cf0 con 0x7fb3f8069230 2024-02-28T23:06:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f819d460 con 0x7fb3f8069230 2024-02-28T23:06:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.316+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f818e620 con 0x7fb3f8069230 2024-02-28T23:06:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.318+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f80fc340 con 0x7fb3f8069230 2024-02-28T23:06:18.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.320+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb3e8009730 con 0x7fb3f8069230 2024-02-28T23:06:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.321+0000 7fb3ef400700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 0x7fb3dc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:18.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.321+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb3e808b4a0 con 0x7fb3f8069230 2024-02-28T23:06:18.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.322+0000 7fb3f6000700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 0x7fb3dc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:18.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.325+0000 7fb3f6000700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 0x7fb3dc06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006010 tx=0x7fb3e400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:18.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.330+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb3e8055b00 con 0x7fb3f8069230 2024-02-28T23:06:18.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.725+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3f8061190 con 0x7fb3dc06c680 2024-02-28T23:06:18.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.728+0000 7fb3ef400700 1 -- 172.21.15.57:0/1497847459 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb3f8061190 con 0x7fb3dc06c680 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "0/2 daemons upgraded", 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc image on host smithi084", 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:06:18.727 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:06:18.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 msgr2=0x7fb3dc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 0x7fb3dc06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006010 tx=0x7fb3e400b540 comp rx=0 tx=0).stop 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 msgr2=0x7fb3f8197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d8d0 tx=0x7fb3e800dc90 comp rx=0 tx=0).stop 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 shutdown_connections 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fb3dc06c680 0x7fb3dc06eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f8069230 0x7fb3f8197cf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 --2- 172.21.15.57:0/1497847459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8103350 0x7fb3f8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 >> 172.21.15.57:0/1497847459 conn(0x7fb3f80fa780 msgr2=0x7fb3f81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 shutdown_connections 2024-02-28T23:06:18.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:18.732+0000 7fb3fd600700 1 -- 172.21.15.57:0/1497847459 wait complete. 2024-02-28T23:06:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:18 smithi057 ceph-mon[29096]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:18 smithi057 ceph-mon[29096]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:18 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2016478355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:06:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:18 smithi084 ceph-mon[34826]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:18 smithi084 ceph-mon[34826]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:18 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2016478355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:06:20.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:19 smithi057 ceph-mon[29096]: pgmap v303: 65 pgs: 65 active+clean; 32 MiB data, 284 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 156 op/s 2024-02-28T23:06:20.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:19 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:06:20.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:19 smithi057 ceph-mon[29096]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:19 smithi084 ceph-mon[34826]: pgmap v303: 65 pgs: 65 active+clean; 32 MiB data, 284 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 156 op/s 2024-02-28T23:06:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:19 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:06:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:19 smithi084 ceph-mon[34826]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:21 smithi084 ceph-mon[34826]: pgmap v304: 65 pgs: 65 active+clean; 33 MiB data, 306 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-02-28T23:06:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:21 smithi057 ceph-mon[29096]: pgmap v304: 65 pgs: 65 active+clean; 33 MiB data, 306 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-02-28T23:06:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:22 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:23.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:22 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:23.991 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:23 smithi084 ceph-mon[34826]: pgmap v305: 65 pgs: 65 active+clean; 42 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 805 KiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-02-28T23:06:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:23 smithi057 ceph-mon[29096]: pgmap v305: 65 pgs: 65 active+clean; 42 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 805 KiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-02-28T23:06:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:25 smithi057 ceph-mon[29096]: pgmap v306: 65 pgs: 65 active+clean; 44 MiB data, 352 MiB used, 536 GiB / 536 GiB avail; 796 KiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-02-28T23:06:26.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:25 smithi084 ceph-mon[34826]: pgmap v306: 65 pgs: 65 active+clean; 44 MiB data, 352 MiB used, 536 GiB / 536 GiB avail; 796 KiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-02-28T23:06:28.020 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:27 smithi084 ceph-mon[34826]: pgmap v307: 65 pgs: 65 active+clean; 50 MiB data, 373 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-02-28T23:06:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:27 smithi057 ceph-mon[29096]: pgmap v307: 65 pgs: 65 active+clean; 50 MiB data, 373 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-02-28T23:06:29.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: Upgrade: Updating mgr.smithi084.fhkzis 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:06:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:28 smithi084 ceph-mon[34826]: pgmap v308: 65 pgs: 65 active+clean; 52 MiB data, 384 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-02-28T23:06:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: Upgrade: Updating mgr.smithi084.fhkzis 2024-02-28T23:06:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:06:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:06:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:06:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:06:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:28 smithi057 ceph-mon[29096]: pgmap v308: 65 pgs: 65 active+clean; 52 MiB data, 384 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-02-28T23:06:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:31 smithi057 ceph-mon[29096]: pgmap v309: 65 pgs: 65 active+clean; 56 MiB data, 410 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-02-28T23:06:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:31 smithi084 ceph-mon[34826]: pgmap v309: 65 pgs: 65 active+clean; 56 MiB data, 410 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-02-28T23:06:33.995 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:33 smithi084 ceph-mon[34826]: pgmap v310: 65 pgs: 65 active+clean; 61 MiB data, 433 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 257 op/s 2024-02-28T23:06:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:33 smithi057 ceph-mon[29096]: pgmap v310: 65 pgs: 65 active+clean; 61 MiB data, 433 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 257 op/s 2024-02-28T23:06:35.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:35.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:35.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:35 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:35.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:35 smithi084 ceph-mon[34826]: pgmap v311: 65 pgs: 65 active+clean; 71 MiB data, 473 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 307 op/s 2024-02-28T23:06:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:35 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:35 smithi057 ceph-mon[29096]: pgmap v311: 65 pgs: 65 active+clean; 71 MiB data, 473 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 307 op/s 2024-02-28T23:06:38.057 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:37 smithi084 ceph-mon[34826]: pgmap v312: 65 pgs: 65 active+clean; 82 MiB data, 498 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-02-28T23:06:38.057 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:37 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:37 smithi057 ceph-mon[29096]: pgmap v312: 65 pgs: 65 active+clean; 82 MiB data, 498 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-02-28T23:06:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:37 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:39 smithi057 ceph-mon[29096]: pgmap v313: 65 pgs: 65 active+clean; 86 MiB data, 517 MiB used, 536 GiB / 536 GiB avail; 799 KiB/s rd, 3.1 MiB/s wr, 299 op/s 2024-02-28T23:06:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:39 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:39 smithi084 ceph-mon[34826]: pgmap v313: 65 pgs: 65 active+clean; 86 MiB data, 517 MiB used, 536 GiB / 536 GiB avail; 799 KiB/s rd, 3.1 MiB/s wr, 299 op/s 2024-02-28T23:06:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:39 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:42.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:41 smithi084 ceph-mon[34826]: pgmap v314: 65 pgs: 65 active+clean; 92 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 821 KiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-02-28T23:06:42.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:41 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:42.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:41 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:42.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:41 smithi057 ceph-mon[29096]: pgmap v314: 65 pgs: 65 active+clean; 92 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 821 KiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-02-28T23:06:42.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:41 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:42.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:41 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:43 smithi084 ceph-mon[34826]: pgmap v315: 65 pgs: 65 active+clean; 98 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 348 op/s 2024-02-28T23:06:44.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:43 smithi057 ceph-mon[29096]: pgmap v315: 65 pgs: 65 active+clean; 98 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 348 op/s 2024-02-28T23:06:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:45 smithi084 ceph-mon[34826]: pgmap v316: 65 pgs: 65 active+clean; 107 MiB data, 548 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 379 op/s 2024-02-28T23:06:46.107 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:45 smithi057 ceph-mon[29096]: pgmap v316: 65 pgs: 65 active+clean; 107 MiB data, 548 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 379 op/s 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:06:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:46 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:06:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:06:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:06:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:06:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:06:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:06:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:46 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:06:48.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:47 smithi057 ceph-mon[29096]: pgmap v317: 65 pgs: 65 active+clean; 108 MiB data, 569 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 351 op/s 2024-02-28T23:06:48.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:47 smithi057 ceph-mon[29096]: mgrmap e20: smithi057.zrrldj(active, since 9m), standbys: smithi084.fhkzis 2024-02-28T23:06:48.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:47 smithi084 ceph-mon[34826]: pgmap v317: 65 pgs: 65 active+clean; 108 MiB data, 569 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 351 op/s 2024-02-28T23:06:48.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:47 smithi084 ceph-mon[34826]: mgrmap e20: smithi057.zrrldj(active, since 9m), standbys: smithi084.fhkzis 2024-02-28T23:06:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 -- 172.21.15.57:0/3138046323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c069270 msgr2=0x7f5b1c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 --2- 172.21.15.57:0/3138046323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c069270 0x7f5b1c102dd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009910 tx=0x7f5b0c009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 -- 172.21.15.57:0/3138046323 shutdown_connections 2024-02-28T23:06:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 --2- 172.21.15.57:0/3138046323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 --2- 172.21.15.57:0/3138046323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c069270 0x7f5b1c102dd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 -- 172.21.15.57:0/3138046323 >> 172.21.15.57:0/3138046323 conn(0x7f5b1c0fa780 msgr2=0x7f5b1c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 -- 172.21.15.57:0/3138046323 shutdown_connections 2024-02-28T23:06:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.953+0000 7f5b21e00700 1 -- 172.21.15.57:0/3138046323 wait complete. 2024-02-28T23:06:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.955+0000 7f5b21e00700 1 Processor -- start 2024-02-28T23:06:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.955+0000 7f5b21e00700 1 -- start start 2024-02-28T23:06:48.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b21e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:48.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b21e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 0x7f5b1c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:48.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b21e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0ff720 con 0x7f5b1c0fed70 2024-02-28T23:06:48.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b21e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0ff890 con 0x7f5b1c103310 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b1aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b1aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:43430/0 (socket says 172.21.15.57:43430) 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.956+0000 7f5b1aa00700 1 -- 172.21.15.57:0/4002199044 learned_addr learned my addr 172.21.15.57:0/4002199044 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.957+0000 7f5b1a000700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 0x7f5b1c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.957+0000 7f5b1aa00700 1 -- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 msgr2=0x7f5b1c0ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.957+0000 7f5b1aa00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 0x7f5b1c0ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:48.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.957+0000 7f5b1aa00700 1 -- 172.21.15.57:0/4002199044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b1c103310 2024-02-28T23:06:48.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.957+0000 7f5b1a000700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 0x7f5b1c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:06:48.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.958+0000 7f5b1aa00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00bd70 tx=0x7f5b0c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:48.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.959+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c01d070 con 0x7f5b1c103310 2024-02-28T23:06:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.959+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b0c00f460 con 0x7f5b1c103310 2024-02-28T23:06:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.959+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c021620 con 0x7f5b1c103310 2024-02-28T23:06:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.959+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c0ffb10 con 0x7f5b1c103310 2024-02-28T23:06:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.959+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c19dcf0 con 0x7f5b1c103310 2024-02-28T23:06:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.960+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c18fbd0 con 0x7f5b1c103310 2024-02-28T23:06:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.962+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) v1 ==== 90626+0+0 (secure 0 0 0) 0x7f5b0c00fa40 con 0x7f5b1c103310 2024-02-28T23:06:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.963+0000 7f5b20a00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 0x7f5b0006f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.963+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5b0c08d5b0 con 0x7f5b1c103310 2024-02-28T23:06:48.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.964+0000 7f5b1a000700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 0x7f5b0006f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:48.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.965+0000 7f5b1a000700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 0x7f5b0006f2d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005fd0 tx=0x7f5b08005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:48.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:48.973+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5b0c05b6a0 con 0x7f5b1c103310 2024-02-28T23:06:49.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.306+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1c02cee0 con 0x7f5b0006ce20 2024-02-28T23:06:49.305 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:48 smithi057 ceph-mon[29096]: pgmap v318: 65 pgs: 65 active+clean; 109 MiB data, 580 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-02-28T23:06:49.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.308+0000 7f5b20a00700 1 -- 172.21.15.57:0/4002199044 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f5b1c02cee0 con 0x7f5b0006ce20 2024-02-28T23:06:49.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.311+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 msgr2=0x7f5b0006f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.311+0000 7f5b21e00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 0x7f5b0006f2d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005fd0 tx=0x7f5b08005ee0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.311+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 msgr2=0x7f5b1c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.311+0000 7f5b21e00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00bd70 tx=0x7f5b0c00be50 comp rx=0 tx=0).stop 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.312+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 shutdown_connections 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.312+0000 7f5b21e00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f5b0006ce20 0x7f5b0006f2d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.312+0000 7f5b21e00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c103310 0x7f5b1c100720 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.312+0000 7f5b21e00700 1 --2- 172.21.15.57:0/4002199044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c0fed70 0x7f5b1c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.312+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 >> 172.21.15.57:0/4002199044 conn(0x7f5b1c0fa780 msgr2=0x7f5b1c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.313+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 shutdown_connections 2024-02-28T23:06:49.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.313+0000 7f5b21e00700 1 -- 172.21.15.57:0/4002199044 wait complete. 2024-02-28T23:06:49.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:48 smithi084 ceph-mon[34826]: pgmap v318: 65 pgs: 65 active+clean; 109 MiB data, 580 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-02-28T23:06:49.332 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:06:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 -- 172.21.15.57:0/3345751637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 msgr2=0x7fc4a4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3345751637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4101890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc490009910 tx=0x7fc490009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 -- 172.21.15.57:0/3345751637 shutdown_connections 2024-02-28T23:06:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3345751637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4101890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3345751637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 0x7fc4a4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 -- 172.21.15.57:0/3345751637 >> 172.21.15.57:0/3345751637 conn(0x7fc4a40fb7f0 msgr2=0x7fc4a40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:49.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 -- 172.21.15.57:0/3345751637 shutdown_connections 2024-02-28T23:06:49.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.508+0000 7fc4aac00700 1 -- 172.21.15.57:0/3345751637 wait complete. 2024-02-28T23:06:49.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.510+0000 7fc4aac00700 1 Processor -- start 2024-02-28T23:06:49.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.510+0000 7fc4aac00700 1 -- start start 2024-02-28T23:06:49.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4aac00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 0x7fc4a4195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:49.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4aac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:49.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4aac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4196610 con 0x7fc4a4101440 2024-02-28T23:06:49.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4aac00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4196750 con 0x7fc4a4100240 2024-02-28T23:06:49.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4a3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4a3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36438/0 (socket says 172.21.15.57:36438) 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4a3400700 1 -- 172.21.15.57:0/3468405737 learned_addr learned my addr 172.21.15.57:0/3468405737 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.511+0000 7fc4a3e00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 0x7fc4a4195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a3400700 1 -- 172.21.15.57:0/3468405737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 msgr2=0x7fc4a4195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a3400700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 0x7fc4a4195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a3400700 1 -- 172.21.15.57:0/3468405737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900095f0 con 0x7fc4a4101440 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a3400700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4900092e0 tx=0x7fc49000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49001d070 con 0x7fc4a4101440 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc490005dc0 con 0x7fc4a4101440 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc490017630 con 0x7fc4a4101440 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.512+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a419b1a0 con 0x7fc4a4101440 2024-02-28T23:06:49.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.513+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a419b690 con 0x7fc4a4101440 2024-02-28T23:06:49.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.514+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a4108d90 con 0x7fc4a4101440 2024-02-28T23:06:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.517+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 90626+0+0 (secure 0 0 0) 0x7fc490003680 con 0x7fc4a4101440 2024-02-28T23:06:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.518+0000 7fc4a0c00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 0x7fc48806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.518+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc49008d3d0 con 0x7fc4a4101440 2024-02-28T23:06:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.519+0000 7fc4a3e00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 0x7fc48806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.519+0000 7fc4a3e00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 0x7fc48806f280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4a41012a0 tx=0x7fc494009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:49.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.523+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc49005b4c0 con 0x7fc4a4101440 2024-02-28T23:06:49.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.861+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a419bd90 con 0x7fc48806cdd0 2024-02-28T23:06:49.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.863+0000 7fc4a0c00700 1 -- 172.21.15.57:0/3468405737 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fc4a419bd90 con 0x7fc48806cdd0 2024-02-28T23:06:49.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.866+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 msgr2=0x7fc48806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.866+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 0x7fc48806f280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4a41012a0 tx=0x7fc494009260 comp rx=0 tx=0).stop 2024-02-28T23:06:49.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.866+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 msgr2=0x7fc4a4195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:49.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.866+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4900092e0 tx=0x7fc49000f690 comp rx=0 tx=0).stop 2024-02-28T23:06:49.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 shutdown_connections 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fc48806cdd0 0x7fc48806f280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a4100240 0x7fc4a4195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 --2- 172.21.15.57:0/3468405737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a4101440 0x7fc4a4195ff0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 >> 172.21.15.57:0/3468405737 conn(0x7fc4a40fb7f0 msgr2=0x7fc4a4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 shutdown_connections 2024-02-28T23:06:49.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:49.867+0000 7fc4aac00700 1 -- 172.21.15.57:0/3468405737 wait complete. 2024-02-28T23:06:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 -- 172.21.15.57:0/2867198217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 msgr2=0x7fe79c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2867198217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c102df0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009910 tx=0x7fe78c009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 -- 172.21.15.57:0/2867198217 shutdown_connections 2024-02-28T23:06:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2867198217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2867198217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c102df0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 -- 172.21.15.57:0/2867198217 >> 172.21.15.57:0/2867198217 conn(0x7fe79c0fa780 msgr2=0x7fe79c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 -- 172.21.15.57:0/2867198217 shutdown_connections 2024-02-28T23:06:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.074+0000 7fe7a3200700 1 -- 172.21.15.57:0/2867198217 wait complete. 2024-02-28T23:06:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.075+0000 7fe7a3200700 1 Processor -- start 2024-02-28T23:06:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.075+0000 7fe7a3200700 1 -- start start 2024-02-28T23:06:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe7a3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c191da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe7a3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c1922e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe7a3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c18e890 con 0x7fe79c069250 2024-02-28T23:06:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe7a3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c18ea00 con 0x7fe79c103330 2024-02-28T23:06:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe79b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c1922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe79b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c1922e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:43452/0 (socket says 172.21.15.57:43452) 2024-02-28T23:06:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe79be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c191da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.076+0000 7fe79b400700 1 -- 172.21.15.57:0/2237890900 learned_addr learned my addr 172.21.15.57:0/2237890900 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.077+0000 7fe79be00700 1 -- 172.21.15.57:0/2237890900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 msgr2=0x7fe79c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.077+0000 7fe79be00700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c1922e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.077+0000 7fe79be00700 1 -- 172.21.15.57:0/2237890900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788009520 con 0x7fe79c069250 2024-02-28T23:06:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.077+0000 7fe79be00700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c191da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.078+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78c01d070 con 0x7fe79c069250 2024-02-28T23:06:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.078+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe78c005dc0 con 0x7fe79c069250 2024-02-28T23:06:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.078+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78c022470 con 0x7fe79c069250 2024-02-28T23:06:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.078+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c0095f0 con 0x7fe79c069250 2024-02-28T23:06:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.078+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c18efb0 con 0x7fe79c069250 2024-02-28T23:06:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.080+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c0fc340 con 0x7fe79c069250 2024-02-28T23:06:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.081+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 90626+0+0 (secure 0 0 0) 0x7fe78c003680 con 0x7fe79c069250 2024-02-28T23:06:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.082+0000 7fe798c00700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 0x7fe78006f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.082+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe78c08d740 con 0x7fe79c069250 2024-02-28T23:06:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.083+0000 7fe79b400700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 0x7fe78006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.084+0000 7fe79b400700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 0x7fe78006f280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe788009e40 tx=0x7fe788009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.088+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe78c05b8d0 con 0x7fe79c069250 2024-02-28T23:06:50.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:49 smithi057 ceph-mon[29096]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:50.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:49 smithi084 ceph-mon[34826]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:50.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.407+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe79c02cee0 con 0x7fe78006cdd0 2024-02-28T23:06:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.431+0000 7fe798c00700 1 -- 172.21.15.57:0/2237890900 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe79c02cee0 con 0x7fe78006cdd0 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (8m) 2m ago 10m 13.6M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (6m) 2m ago 10m 7578k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (9m) 11s ago 9m 7696k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (10m) 2m ago 10m 6665k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (9m) 11s ago 9m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:06:50.431 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (8m) 2m ago 9m 73.3M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (3m) 2m ago 3m 12.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (2m) 2m ago 2m 14.7M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (2m) 11s ago 2m 39.1M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (3m) 11s ago 3m 66.5M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (12m) 2m ago 12m 488M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (16s) 11s ago 8m 52.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b ae1c65e2de51 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (12m) 2m ago 12m 44.5M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (8m) 11s ago 8m 35.4M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (10m) 2m ago 10m 7768k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (8m) 11s ago 8m 7868k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (7m) 2m ago 7m 46.5M 3949M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (7m) 2m ago 7m 46.4M 3949M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (6m) 2m ago 6m 46.6M 3949M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:06:50.432 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (6m) 11s ago 6m 85.6M 4971M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:06:50.433 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (5m) 11s ago 5m 81.0M 4971M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:06:50.433 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (4m) 11s ago 4m 78.0M 4971M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:06:50.433 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (7m) 2m ago 9m 55.0M - 2.43.0 a07b618ecd1d ffacfc83a2dc 2024-02-28T23:06:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.434+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 msgr2=0x7fe78006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.434+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 0x7fe78006f280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe788009e40 tx=0x7fe788009260 comp rx=0 tx=0).stop 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.434+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 msgr2=0x7fe79c191da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.434+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c191da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00fab0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 shutdown_connections 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7fe78006cdd0 0x7fe78006f280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe79c069250 0x7fe79c191da0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 --2- 172.21.15.57:0/2237890900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe79c103330 0x7fe79c1922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 >> 172.21.15.57:0/2237890900 conn(0x7fe79c0fa780 msgr2=0x7fe79c104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.435+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 shutdown_connections 2024-02-28T23:06:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.436+0000 7fe7a3200700 1 -- 172.21.15.57:0/2237890900 wait complete. 2024-02-28T23:06:50.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 -- 172.21.15.57:0/3324635876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0100d60 msgr2=0x7f91d0103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 --2- 172.21.15.57:0/3324635876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0100d60 0x7f91d0103140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009910 tx=0x7f91c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 -- 172.21.15.57:0/3324635876 shutdown_connections 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 --2- 172.21.15.57:0/3324635876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0103680 0x7f91d0105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 --2- 172.21.15.57:0/3324635876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0100d60 0x7f91d0103140 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 -- 172.21.15.57:0/3324635876 >> 172.21.15.57:0/3324635876 conn(0x7f91d00fa780 msgr2=0x7f91d00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 -- 172.21.15.57:0/3324635876 shutdown_connections 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.619+0000 7f91d7200700 1 -- 172.21.15.57:0/3324635876 wait complete. 2024-02-28T23:06:50.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.620+0000 7f91d7200700 1 Processor -- start 2024-02-28T23:06:50.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91d7200700 1 -- start start 2024-02-28T23:06:50.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91d7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0100d60 0x7f91d0195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91d7200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91d7200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0196610 con 0x7f91d0103680 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91d7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0196750 con 0x7f91d0100d60 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36480/0 (socket says 172.21.15.57:36480) 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.621+0000 7f91cf400700 1 -- 172.21.15.57:0/2049747364 learned_addr learned my addr 172.21.15.57:0/2049747364 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:50.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.622+0000 7f91cf400700 1 -- 172.21.15.57:0/2049747364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0100d60 msgr2=0x7f91d0195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:50.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.622+0000 7f91cf400700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0100d60 0x7f91d0195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:50.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.622+0000 7f91cf400700 1 -- 172.21.15.57:0/2049747364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00095f0 con 0x7f91d0103680 2024-02-28T23:06:50.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91cf400700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00d8d0 tx=0x7f91bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:50.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91bc009940 con 0x7f91d0103680 2024-02-28T23:06:50.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f91bc010460 con 0x7f91d0103680 2024-02-28T23:06:50.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d019b200 con 0x7f91d0103680 2024-02-28T23:06:50.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91bc00f5f0 con 0x7f91d0103680 2024-02-28T23:06:50.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.623+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d019b750 con 0x7f91d0103680 2024-02-28T23:06:50.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.625+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d018fbc0 con 0x7f91d0103680 2024-02-28T23:06:50.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.627+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 90626+0+0 (secure 0 0 0) 0x7f91bc0105d0 con 0x7f91d0103680 2024-02-28T23:06:50.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.628+0000 7f91ccc00700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 0x7f91b406f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:50.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.628+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f91bc08c1f0 con 0x7f91d0103680 2024-02-28T23:06:50.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.628+0000 7f91cfe00700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 0x7f91b406f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:50.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.629+0000 7f91cfe00700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 0x7f91b406f2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f91c000b5c0 tx=0x7f91c0005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:50.635+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f91bc05a360 con 0x7f91d0103680 2024-02-28T23:06:51.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.080+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f91d0108dc0 con 0x7f91d0103680 2024-02-28T23:06:51.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.081+0000 7f91ccc00700 1 -- 172.21.15.57:0/2049747364 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f91bc059ef0 con 0x7f91d0103680 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:06:51.081 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:06:51.082 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:06:51.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 msgr2=0x7f91b406f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 0x7f91b406f2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f91c000b5c0 tx=0x7f91c0005fd0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 msgr2=0x7f91d0195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00d8d0 tx=0x7f91bc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 shutdown_connections 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f91b406ce20 0x7f91b406f2d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0100d60 0x7f91d0195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 --2- 172.21.15.57:0/2049747364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91d0103680 0x7f91d0195ff0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.084+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 >> 172.21.15.57:0/2049747364 conn(0x7f91d00fa780 msgr2=0x7f91d00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.085+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 shutdown_connections 2024-02-28T23:06:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.085+0000 7f91d7200700 1 -- 172.21.15.57:0/2049747364 wait complete. 2024-02-28T23:06:51.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 -- 172.21.15.57:0/3213903518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 msgr2=0x7f0a94105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 --2- 172.21.15.57:0/3213903518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94105a60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009910 tx=0x7f0a80009c20 comp rx=0 tx=0).stop 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 -- 172.21.15.57:0/3213903518 shutdown_connections 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 --2- 172.21.15.57:0/3213903518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94105a60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 --2- 172.21.15.57:0/3213903518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.283+0000 7f0a99800700 1 -- 172.21.15.57:0/3213903518 >> 172.21.15.57:0/3213903518 conn(0x7f0a940fa780 msgr2=0x7f0a940fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.284+0000 7f0a99800700 1 -- 172.21.15.57:0/3213903518 shutdown_connections 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.284+0000 7f0a99800700 1 -- 172.21.15.57:0/3213903518 wait complete. 2024-02-28T23:06:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.285+0000 7f0a99800700 1 Processor -- start 2024-02-28T23:06:51.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.286+0000 7f0a99800700 1 -- start start 2024-02-28T23:06:51.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.286+0000 7f0a99800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:51.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.286+0000 7f0a99800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:51.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.286+0000 7f0a99800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94198850 con 0x7f0a94103680 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.286+0000 7f0a99800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9419d260 con 0x7f0a94100d60 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:43494/0 (socket says 172.21.15.57:43494) 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36500/0 (socket says 172.21.15.57:36500) 2024-02-28T23:06:51.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92a00700 1 -- 172.21.15.57:0/4150878702 learned_addr learned my addr 172.21.15.57:0/4150878702 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:06:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92000700 1 -- 172.21.15.57:0/4150878702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 msgr2=0x7f0a94197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92000700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.287+0000 7f0a92000700 1 -- 172.21.15.57:0/4150878702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009520 con 0x7f0a94103680 2024-02-28T23:06:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.288+0000 7f0a92000700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005b40 tx=0x7f0a8000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.288+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a8001d070 con 0x7f0a94103680 2024-02-28T23:06:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.289+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a80005dc0 con 0x7f0a94103680 2024-02-28T23:06:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.289+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a80022470 con 0x7f0a94103680 2024-02-28T23:06:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.289+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a800095f0 con 0x7f0a94103680 2024-02-28T23:06:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.289+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9419d730 con 0x7f0a94103680 2024-02-28T23:06:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.291+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a940fc340 con 0x7f0a94103680 2024-02-28T23:06:51.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.293+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 90626+0+0 (secure 0 0 0) 0x7f0a80005160 con 0x7f0a94103680 2024-02-28T23:06:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.294+0000 7f0a8b400700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 0x7f0a7806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:06:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.294+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0a8008d720 con 0x7f0a94103680 2024-02-28T23:06:51.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.295+0000 7f0a92a00700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 0x7f0a7806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:06:51.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.298+0000 7f0a92a00700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 0x7f0a7806f280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a941991d0 tx=0x7f0a84009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:06:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.301+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a80057d80 con 0x7f0a94103680 2024-02-28T23:06:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:51 smithi057 ceph-mon[29096]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:51 smithi057 ceph-mon[29096]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:51 smithi057 ceph-mon[29096]: pgmap v319: 65 pgs: 65 active+clean; 115 MiB data, 610 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-02-28T23:06:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:51 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/2049747364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:06:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.616+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 --> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a94061190 con 0x7f0a7806cdd0 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.619+0000 7f0a8b400700 1 -- 172.21.15.57:0/4150878702 <== mgr.14217 v2:172.21.15.57:6800/3095633985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f0a94061190 con 0x7f0a7806cdd0 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:06:51.618 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "1/2 daemons upgraded", 2024-02-28T23:06:51.619 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mgr daemons", 2024-02-28T23:06:51.619 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:06:51.619 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:06:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.623+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 msgr2=0x7f0a7806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.623+0000 7f0a99800700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 0x7f0a7806f280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a941991d0 tx=0x7f0a84009260 comp rx=0 tx=0).stop 2024-02-28T23:06:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.623+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 msgr2=0x7f0a94198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:06:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.623+0000 7f0a99800700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005b40 tx=0x7f0a8000fa00 comp rx=0 tx=0).stop 2024-02-28T23:06:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 shutdown_connections 2024-02-28T23:06:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:6800/3095633985,v1:172.21.15.57:6801/3095633985] conn(0x7f0a7806cdd0 0x7f0a7806f280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a94100d60 0x7f0a94197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 --2- 172.21.15.57:0/4150878702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a94103680 0x7f0a94198230 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:06:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 >> 172.21.15.57:0/4150878702 conn(0x7f0a940fa780 msgr2=0x7f0a940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:06:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.624+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 shutdown_connections 2024-02-28T23:06:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:06:51.625+0000 7f0a99800700 1 -- 172.21.15.57:0/4150878702 wait complete. 2024-02-28T23:06:51.761 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:51 smithi084 ceph-mon[34826]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:51.761 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:51 smithi084 ceph-mon[34826]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:51.761 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:51 smithi084 ceph-mon[34826]: pgmap v319: 65 pgs: 65 active+clean; 115 MiB data, 610 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-02-28T23:06:51.762 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:51 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2049747364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:06:52.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:52 smithi084 ceph-mon[34826]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:52 smithi057 ceph-mon[29096]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: pgmap v320: 65 pgs: 65 active+clean; 117 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 292 op/s 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:06:53.370 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr fail", "who": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: osdmap e48: 6 total, 6 up, 6 in 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "mgr fail", "who": "smithi057.zrrldj"}]': finished 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: mgrmap e21: smithi084.fhkzis(active, starting, since 0.0103354s) 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:06:53.371 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: Manager daemon smithi084.fhkzis is now available 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:06:53.372 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:06:53.373 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/trash_purge_schedule"}]: dispatch 2024-02-28T23:06:53.373 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:53 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/trash_purge_schedule"}]: dispatch 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: pgmap v320: 65 pgs: 65 active+clean; 117 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 292 op/s 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:06:53.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr fail", "who": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: osdmap e48: 6 total, 6 up, 6 in 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.14217 172.21.15.57:0/583656129' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "mgr fail", "who": "smithi057.zrrldj"}]': finished 2024-02-28T23:06:53.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: mgrmap e21: smithi084.fhkzis(active, starting, since 0.0103354s) 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:06:53.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:06:53.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: Manager daemon smithi084.fhkzis is now available 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:06:53.835 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/trash_purge_schedule"}]: dispatch 2024-02-28T23:06:53.836 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:53 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi084.fhkzis/trash_purge_schedule"}]: dispatch 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: [28/Feb/2024:23:06:53] ENGINE Bus STARTING 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: [28/Feb/2024:23:06:53] ENGINE Serving on https://172.21.15.84:7150 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: [28/Feb/2024:23:06:53] ENGINE Serving on http://172.21.15.84:8765 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: [28/Feb/2024:23:06:53] ENGINE Bus STARTED 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: Deploying cephadm binary to smithi084 2024-02-28T23:06:55.400 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: Deploying cephadm binary to smithi057 2024-02-28T23:06:55.401 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: mgrmap e22: smithi084.fhkzis(active, since 1.01765s) 2024-02-28T23:06:55.401 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:55 smithi057 ceph-mon[29096]: pgmap v3: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:55.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: [28/Feb/2024:23:06:53] ENGINE Bus STARTING 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: [28/Feb/2024:23:06:53] ENGINE Serving on https://172.21.15.84:7150 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: [28/Feb/2024:23:06:53] ENGINE Serving on http://172.21.15.84:8765 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: [28/Feb/2024:23:06:53] ENGINE Bus STARTED 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: Deploying cephadm binary to smithi084 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: Deploying cephadm binary to smithi057 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: mgrmap e22: smithi084.fhkzis(active, since 1.01765s) 2024-02-28T23:06:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:55 smithi084 ceph-mon[34826]: pgmap v3: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:56.564 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:56 smithi057 ceph-mon[29096]: pgmap v4: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:56.564 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:56 smithi057 ceph-mon[29096]: mgrmap e23: smithi084.fhkzis(active, since 2s) 2024-02-28T23:06:56.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:56 smithi084 ceph-mon[34826]: pgmap v4: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:56.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:56 smithi084 ceph-mon[34826]: mgrmap e23: smithi084.fhkzis(active, since 2s) 2024-02-28T23:06:58.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:58 smithi057 ceph-mon[29096]: pgmap v5: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:58.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:06:58 smithi057 ceph-mon[29096]: mgrmap e24: smithi084.fhkzis(active, since 4s) 2024-02-28T23:06:58.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:58 smithi084 ceph-mon[34826]: pgmap v5: 65 pgs: 65 active+clean; 126 MiB data, 673 MiB used, 536 GiB / 536 GiB avail 2024-02-28T23:06:58.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:06:58 smithi084 ceph-mon[34826]: mgrmap e24: smithi084.fhkzis(active, since 4s) 2024-02-28T23:07:00.301 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:00 smithi057 ceph-mon[29096]: pgmap v6: 65 pgs: 65 active+clean; 129 MiB data, 692 MiB used, 536 GiB / 536 GiB avail; 663 KiB/s wr, 104 op/s 2024-02-28T23:07:00.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:00 smithi084 ceph-mon[34826]: pgmap v6: 65 pgs: 65 active+clean; 129 MiB data, 692 MiB used, 536 GiB / 536 GiB avail; 663 KiB/s wr, 104 op/s 2024-02-28T23:07:01.692 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:01 smithi057 ceph-mon[29096]: pgmap v7: 65 pgs: 65 active+clean; 142 MiB data, 699 MiB used, 536 GiB / 536 GiB avail; 897 KiB/s rd, 2.5 MiB/s wr, 260 op/s 2024-02-28T23:07:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:01 smithi084 ceph-mon[34826]: pgmap v7: 65 pgs: 65 active+clean; 142 MiB data, 699 MiB used, 536 GiB / 536 GiB avail; 897 KiB/s rd, 2.5 MiB/s wr, 260 op/s 2024-02-28T23:07:04.223 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:04 smithi057 ceph-mon[29096]: pgmap v8: 65 pgs: 65 active+clean; 142 MiB data, 699 MiB used, 536 GiB / 536 GiB avail; 698 KiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-28T23:07:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:04 smithi084 ceph-mon[34826]: pgmap v8: 65 pgs: 65 active+clean; 142 MiB data, 699 MiB used, 536 GiB / 536 GiB avail; 698 KiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-28T23:07:06.504 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: pgmap v9: 65 pgs: 65 active+clean; 160 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: Standby manager daemon smithi057.zrrldj started 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/crt"}]: dispatch 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/key"}]: dispatch 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:06.505 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:06 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:06.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: pgmap v9: 65 pgs: 65 active+clean; 160 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-02-28T23:07:06.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: Standby manager daemon smithi057.zrrldj started 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/crt"}]: dispatch 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/key"}]: dispatch 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/2371022709' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:06.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:06 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:07 smithi057 ceph-mon[29096]: mgrmap e25: smithi084.fhkzis(active, since 13s), standbys: smithi057.zrrldj 2024-02-28T23:07:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:07 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:07:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:07 smithi057 ceph-mon[29096]: pgmap v10: 65 pgs: 65 active+clean; 160 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-02-28T23:07:07.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:07 smithi084 ceph-mon[34826]: mgrmap e25: smithi084.fhkzis(active, since 13s), standbys: smithi057.zrrldj 2024-02-28T23:07:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:07 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:07:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:07 smithi084 ceph-mon[34826]: pgmap v10: 65 pgs: 65 active+clean; 160 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-02-28T23:07:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:09 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:09 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:09 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:09 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:09 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:09 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:09 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:09 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:09 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:09 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:10.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:10 smithi057 ceph-mon[29096]: pgmap v11: 65 pgs: 65 active+clean; 164 MiB data, 795 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-02-28T23:07:10.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:10 smithi084 ceph-mon[34826]: pgmap v11: 65 pgs: 65 active+clean; 164 MiB data, 795 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-02-28T23:07:11.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:11 smithi084 ceph-mon[34826]: pgmap v12: 65 pgs: 65 active+clean; 172 MiB data, 840 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 383 op/s 2024-02-28T23:07:11.582 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:11 smithi057 ceph-mon[29096]: pgmap v12: 65 pgs: 65 active+clean; 172 MiB data, 840 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 383 op/s 2024-02-28T23:07:12.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:12.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:12.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi057 to 1218M 2024-02-28T23:07:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:12 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi057 to 1218M 2024-02-28T23:07:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:12 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:13.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:13 smithi084 ceph-mon[34826]: pgmap v13: 65 pgs: 65 active+clean; 172 MiB data, 840 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-02-28T23:07:13.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:13 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:13 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:13 smithi057 ceph-mon[29096]: pgmap v13: 65 pgs: 65 active+clean; 172 MiB data, 840 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-02-28T23:07:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:13 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:13 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: Adjusting osd_memory_target on smithi084 to 2241M 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:07:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:14 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: Adjusting osd_memory_target on smithi084 to 2241M 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:07:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:07:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:07:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:07:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:07:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:14 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:07:15.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:15 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:07:15.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:15 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:07:15.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:15 smithi057 ceph-mon[29096]: pgmap v14: 65 pgs: 65 active+clean; 188 MiB data, 881 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-02-28T23:07:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:15 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:07:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:15 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:07:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:15 smithi084 ceph-mon[34826]: pgmap v14: 65 pgs: 65 active+clean; 188 MiB data, 881 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-02-28T23:07:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:07:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:07:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:17 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:17 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:18.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:18 smithi057 ceph-mon[29096]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T23:07:18.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:18 smithi057 ceph-mon[29096]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T23:07:18.495 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:18 smithi057 ceph-mon[29096]: pgmap v15: 65 pgs: 65 active+clean; 188 MiB data, 881 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-02-28T23:07:18.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:18 smithi084 ceph-mon[34826]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T23:07:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:18 smithi084 ceph-mon[34826]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T23:07:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:18 smithi084 ceph-mon[34826]: pgmap v15: 65 pgs: 65 active+clean; 188 MiB data, 881 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-02-28T23:07:18.828 INFO:tasks.workunit.client.1.smithi084.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-28T23:07:18.860 INFO:tasks.workunit.client.1.smithi084.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-28T23:07:18.860 INFO:tasks.workunit.client.1.smithi084.stderr:+ make 2024-02-28T23:07:19.126 INFO:tasks.workunit.client.1.smithi084.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-28T23:07:20.358 INFO:tasks.workunit.client.1.smithi084.stderr:++ readlink -f fsstress 2024-02-28T23:07:20.367 INFO:tasks.workunit.client.1.smithi084.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-28T23:07:20.367 INFO:tasks.workunit.client.1.smithi084.stderr:+ popd 2024-02-28T23:07:20.370 INFO:tasks.workunit.client.1.smithi084.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-28T23:07:20.370 INFO:tasks.workunit.client.1.smithi084.stderr:+ popd 2024-02-28T23:07:20.372 INFO:tasks.workunit.client.1.smithi084.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-28T23:07:20.374 INFO:tasks.workunit.client.1.smithi084.stderr:++ mktemp -d -p . 2024-02-28T23:07:20.381 INFO:tasks.workunit.client.1.smithi084.stderr:+ T=./tmp.Tih5bPNKzk 2024-02-28T23:07:20.381 INFO:tasks.workunit.client.1.smithi084.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Tih5bPNKzk -l 1 -n 1000 -p 10 -v 2024-02-28T23:07:20.389 INFO:tasks.workunit.client.1.smithi084.stdout:seed = 1708802504 2024-02-28T23:07:20.397 INFO:tasks.workunit.client.1.smithi084.stdout:0/0: stat - no entries 2024-02-28T23:07:20.399 INFO:tasks.workunit.client.1.smithi084.stdout:2/0: dwrite - no filename 2024-02-28T23:07:20.399 INFO:tasks.workunit.client.1.smithi084.stdout:2/1: write - no filename 2024-02-28T23:07:20.399 INFO:tasks.workunit.client.1.smithi084.stdout:2/2: dread - no filename 2024-02-28T23:07:20.400 INFO:tasks.workunit.client.1.smithi084.stdout:0/1: creat f0 x:0 0 0 2024-02-28T23:07:20.402 INFO:tasks.workunit.client.1.smithi084.stdout:0/2: mkdir d1 0 2024-02-28T23:07:20.402 INFO:tasks.workunit.client.1.smithi084.stdout:1/0: write - no filename 2024-02-28T23:07:20.403 INFO:tasks.workunit.client.1.smithi084.stdout:1/1: dwrite - no filename 2024-02-28T23:07:20.404 INFO:tasks.workunit.client.1.smithi084.stdout:2/3: creat f0 x:0 0 0 2024-02-28T23:07:20.406 INFO:tasks.workunit.client.1.smithi084.stdout:3/0: write - no filename 2024-02-28T23:07:20.406 INFO:tasks.workunit.client.1.smithi084.stdout:1/2: symlink l0 0 2024-02-28T23:07:20.406 INFO:tasks.workunit.client.1.smithi084.stdout:1/3: write - no filename 2024-02-28T23:07:20.407 INFO:tasks.workunit.client.1.smithi084.stdout:1/4: dwrite - no filename 2024-02-28T23:07:20.407 INFO:tasks.workunit.client.1.smithi084.stdout:2/4: write f0 [293817,126372] 0 2024-02-28T23:07:20.407 INFO:tasks.workunit.client.1.smithi084.stdout:0/3: mknod d1/c2 0 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/5: chown l0 345 1 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/6: write - no filename 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/7: rmdir - no directory 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/8: write - no filename 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/9: dwrite - no filename 2024-02-28T23:07:20.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/10: truncate - no filename 2024-02-28T23:07:20.410 INFO:tasks.workunit.client.1.smithi084.stdout:4/0: chown . 0 1 2024-02-28T23:07:20.410 INFO:tasks.workunit.client.1.smithi084.stdout:4/1: write - no filename 2024-02-28T23:07:20.410 INFO:tasks.workunit.client.1.smithi084.stdout:4/2: write - no filename 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/3: dwrite - no filename 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/4: link - no file 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/5: rename - no filename 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/6: dread - no filename 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/7: rename - no filename 2024-02-28T23:07:20.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/8: write - no filename 2024-02-28T23:07:20.412 INFO:tasks.workunit.client.1.smithi084.stdout:3/1: symlink l0 0 2024-02-28T23:07:20.412 INFO:tasks.workunit.client.1.smithi084.stdout:1/11: creat f1 x:0 0 0 2024-02-28T23:07:20.414 INFO:tasks.workunit.client.1.smithi084.stdout:4/9: symlink l0 0 2024-02-28T23:07:20.414 INFO:tasks.workunit.client.1.smithi084.stdout:5/0: write - no filename 2024-02-28T23:07:20.415 INFO:tasks.workunit.client.1.smithi084.stdout:5/1: truncate - no filename 2024-02-28T23:07:20.415 INFO:tasks.workunit.client.1.smithi084.stdout:4/10: dread - no filename 2024-02-28T23:07:20.415 INFO:tasks.workunit.client.1.smithi084.stdout:4/11: dwrite - no filename 2024-02-28T23:07:20.416 INFO:tasks.workunit.client.1.smithi084.stdout:3/2: creat f1 x:0 0 0 2024-02-28T23:07:20.416 INFO:tasks.workunit.client.1.smithi084.stdout:4/12: chown l0 56 1 2024-02-28T23:07:20.417 INFO:tasks.workunit.client.1.smithi084.stdout:1/12: creat f2 x:0 0 0 2024-02-28T23:07:20.419 INFO:tasks.workunit.client.1.smithi084.stdout:5/2: creat f0 x:0 0 0 2024-02-28T23:07:20.420 INFO:tasks.workunit.client.1.smithi084.stdout:4/13: mkdir d1 0 2024-02-28T23:07:20.421 INFO:tasks.workunit.client.1.smithi084.stdout:6/0: write - no filename 2024-02-28T23:07:20.423 INFO:tasks.workunit.client.1.smithi084.stdout:3/3: mknod c2 0 2024-02-28T23:07:20.425 INFO:tasks.workunit.client.1.smithi084.stdout:5/3: truncate f0 131314 0 2024-02-28T23:07:20.426 INFO:tasks.workunit.client.1.smithi084.stdout:6/1: mknod c0 0 2024-02-28T23:07:20.426 INFO:tasks.workunit.client.1.smithi084.stdout:6/2: fdatasync - no filename 2024-02-28T23:07:20.426 INFO:tasks.workunit.client.1.smithi084.stdout:6/3: dread - no filename 2024-02-28T23:07:20.426 INFO:tasks.workunit.client.1.smithi084.stdout:6/4: dread - no filename 2024-02-28T23:07:20.427 INFO:tasks.workunit.client.1.smithi084.stdout:5/4: stat f0 0 2024-02-28T23:07:20.428 INFO:tasks.workunit.client.1.smithi084.stdout:4/14: chown l0 6668231 1 2024-02-28T23:07:20.433 INFO:tasks.workunit.client.1.smithi084.stdout:6/5: chown c0 67992 1 2024-02-28T23:07:20.433 INFO:tasks.workunit.client.1.smithi084.stdout:6/6: truncate - no filename 2024-02-28T23:07:20.433 INFO:tasks.workunit.client.1.smithi084.stdout:6/7: dwrite - no filename 2024-02-28T23:07:20.433 INFO:tasks.workunit.client.1.smithi084.stdout:6/8: write - no filename 2024-02-28T23:07:20.433 INFO:tasks.workunit.client.1.smithi084.stdout:6/9: rmdir - no directory 2024-02-28T23:07:20.434 INFO:tasks.workunit.client.1.smithi084.stdout:6/10: truncate - no filename 2024-02-28T23:07:20.434 INFO:tasks.workunit.client.1.smithi084.stdout:8/0: dread - no filename 2024-02-28T23:07:20.434 INFO:tasks.workunit.client.1.smithi084.stdout:8/1: fdatasync - no filename 2024-02-28T23:07:20.434 INFO:tasks.workunit.client.1.smithi084.stdout:8/2: link - no file 2024-02-28T23:07:20.438 INFO:tasks.workunit.client.1.smithi084.stdout:5/5: write f0 [684456,65066] 0 2024-02-28T23:07:20.438 INFO:tasks.workunit.client.1.smithi084.stdout:4/15: rename d1 to d1/d2 22 2024-02-28T23:07:20.438 INFO:tasks.workunit.client.1.smithi084.stdout:4/16: fsync - no filename 2024-02-28T23:07:20.438 INFO:tasks.workunit.client.1.smithi084.stdout:4/17: fsync - no filename 2024-02-28T23:07:20.441 INFO:tasks.workunit.client.1.smithi084.stdout:7/0: mknod c0 0 2024-02-28T23:07:20.442 INFO:tasks.workunit.client.1.smithi084.stdout:7/1: rmdir - no directory 2024-02-28T23:07:20.444 INFO:tasks.workunit.client.1.smithi084.stdout:9/0: readlink - no filename 2024-02-28T23:07:20.444 INFO:tasks.workunit.client.1.smithi084.stdout:9/1: dread - no filename 2024-02-28T23:07:20.444 INFO:tasks.workunit.client.1.smithi084.stdout:9/2: read - no filename 2024-02-28T23:07:20.444 INFO:tasks.workunit.client.1.smithi084.stdout:6/11: mkdir d1 0 2024-02-28T23:07:20.447 INFO:tasks.workunit.client.1.smithi084.stdout:4/18: chown l0 603226631 1 2024-02-28T23:07:20.448 INFO:tasks.workunit.client.1.smithi084.stdout:8/3: mkdir d0 0 2024-02-28T23:07:20.452 INFO:tasks.workunit.client.1.smithi084.stdout:5/6: write f0 [673259,101460] 0 2024-02-28T23:07:20.453 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:20 smithi057 ceph-mon[29096]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 916 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-02-28T23:07:20.454 INFO:tasks.workunit.client.1.smithi084.stdout:7/2: chown c0 130 1 2024-02-28T23:07:20.454 INFO:tasks.workunit.client.1.smithi084.stdout:7/3: dread - no filename 2024-02-28T23:07:20.454 INFO:tasks.workunit.client.1.smithi084.stdout:7/4: rmdir - no directory 2024-02-28T23:07:20.455 INFO:tasks.workunit.client.1.smithi084.stdout:9/3: creat f0 x:0 0 0 2024-02-28T23:07:20.455 INFO:tasks.workunit.client.1.smithi084.stdout:8/4: chown d0 2978059 1 2024-02-28T23:07:20.461 INFO:tasks.workunit.client.1.smithi084.stdout:6/12: symlink d1/l2 0 2024-02-28T23:07:20.464 INFO:tasks.workunit.client.1.smithi084.stdout:8/5: chown d0 834 1 2024-02-28T23:07:20.464 INFO:tasks.workunit.client.1.smithi084.stdout:8/6: write - no filename 2024-02-28T23:07:20.465 INFO:tasks.workunit.client.1.smithi084.stdout:7/5: symlink l1 0 2024-02-28T23:07:20.466 INFO:tasks.workunit.client.1.smithi084.stdout:2/5: dwrite f0 [0,4194304] 0 2024-02-28T23:07:20.477 INFO:tasks.workunit.client.1.smithi084.stdout:4/19: mknod d1/c3 0 2024-02-28T23:07:20.486 INFO:tasks.workunit.client.1.smithi084.stdout:9/4: symlink l1 0 2024-02-28T23:07:20.493 INFO:tasks.workunit.client.1.smithi084.stdout:5/7: dread f0 [0,4194304] 0 2024-02-28T23:07:20.493 INFO:tasks.workunit.client.1.smithi084.stdout:1/13: dwrite f2 [0,4194304] 0 2024-02-28T23:07:20.494 INFO:tasks.workunit.client.1.smithi084.stdout:7/6: creat f2 x:0 0 0 2024-02-28T23:07:20.496 INFO:tasks.workunit.client.1.smithi084.stdout:0/4: dwrite f0 [0,4194304] 0 2024-02-28T23:07:20.496 INFO:tasks.workunit.client.1.smithi084.stdout:0/5: readlink - no filename 2024-02-28T23:07:20.502 INFO:tasks.workunit.client.1.smithi084.stdout:3/4: dwrite f1 [0,4194304] 0 2024-02-28T23:07:20.503 INFO:tasks.workunit.client.1.smithi084.stdout:9/5: stat l1 0 2024-02-28T23:07:20.505 INFO:tasks.workunit.client.1.smithi084.stdout:6/13: rename d1/l2 to d1/l3 0 2024-02-28T23:07:20.508 INFO:tasks.workunit.client.1.smithi084.stdout:4/20: creat d1/f4 x:0 0 0 2024-02-28T23:07:20.509 INFO:tasks.workunit.client.1.smithi084.stdout:9/6: chown f0 24363338 1 2024-02-28T23:07:20.510 INFO:tasks.workunit.client.1.smithi084.stdout:6/14: chown d1 2131240544 1 2024-02-28T23:07:20.514 INFO:tasks.workunit.client.1.smithi084.stdout:4/21: truncate d1/f4 204195 0 2024-02-28T23:07:20.519 INFO:tasks.workunit.client.1.smithi084.stdout:6/15: chown c0 19716566 1 2024-02-28T23:07:20.519 INFO:tasks.workunit.client.1.smithi084.stdout:8/7: symlink d0/l1 0 2024-02-28T23:07:20.524 INFO:tasks.workunit.client.1.smithi084.stdout:5/8: creat f1 x:0 0 0 2024-02-28T23:07:20.524 INFO:tasks.workunit.client.1.smithi084.stdout:8/8: rename d0 to d0/d2 22 2024-02-28T23:07:20.524 INFO:tasks.workunit.client.1.smithi084.stdout:8/9: dwrite - no filename 2024-02-28T23:07:20.528 INFO:tasks.workunit.client.1.smithi084.stdout:7/7: creat f3 x:0 0 0 2024-02-28T23:07:20.530 INFO:tasks.workunit.client.1.smithi084.stdout:2/6: dwrite f0 [0,4194304] 0 2024-02-28T23:07:20.531 INFO:tasks.workunit.client.1.smithi084.stdout:9/7: mknod c2 0 2024-02-28T23:07:20.531 INFO:tasks.workunit.client.1.smithi084.stdout:5/9: dread - f1 zero size 2024-02-28T23:07:20.531 INFO:tasks.workunit.client.1.smithi084.stdout:9/8: stat l1 0 2024-02-28T23:07:20.533 INFO:tasks.workunit.client.1.smithi084.stdout:6/16: creat d1/f4 x:0 0 0 2024-02-28T23:07:20.536 INFO:tasks.workunit.client.1.smithi084.stdout:8/10: creat d0/f3 x:0 0 0 2024-02-28T23:07:20.538 INFO:tasks.workunit.client.1.smithi084.stdout:2/7: creat f1 x:0 0 0 2024-02-28T23:07:20.539 INFO:tasks.workunit.client.1.smithi084.stdout:3/5: dread f1 [0,4194304] 0 2024-02-28T23:07:20.540 INFO:tasks.workunit.client.1.smithi084.stdout:5/10: symlink l2 0 2024-02-28T23:07:20.542 INFO:tasks.workunit.client.1.smithi084.stdout:6/17: symlink d1/l5 0 2024-02-28T23:07:20.548 INFO:tasks.workunit.client.1.smithi084.stdout:8/11: unlink d0/l1 0 2024-02-28T23:07:20.548 INFO:tasks.workunit.client.1.smithi084.stdout:1/14: dread f2 [0,4194304] 0 2024-02-28T23:07:20.550 INFO:tasks.workunit.client.1.smithi084.stdout:3/6: mkdir d3 0 2024-02-28T23:07:20.564 INFO:tasks.workunit.client.1.smithi084.stdout:8/12: mknod d0/c4 0 2024-02-28T23:07:20.571 INFO:tasks.workunit.client.1.smithi084.stdout:8/13: stat d0 0 2024-02-28T23:07:20.574 INFO:tasks.workunit.client.1.smithi084.stdout:4/22: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:20.577 INFO:tasks.workunit.client.1.smithi084.stdout:3/7: creat d3/f4 x:0 0 0 2024-02-28T23:07:20.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:20 smithi084 ceph-mon[34826]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 916 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-02-28T23:07:20.607 INFO:tasks.workunit.client.1.smithi084.stdout:8/14: symlink d0/l5 0 2024-02-28T23:07:20.608 INFO:tasks.workunit.client.1.smithi084.stdout:9/9: dwrite f0 [0,4194304] 0 2024-02-28T23:07:20.610 INFO:tasks.workunit.client.1.smithi084.stdout:8/15: chown d0 0 1 2024-02-28T23:07:20.615 INFO:tasks.workunit.client.1.smithi084.stdout:9/10: stat f0 0 2024-02-28T23:07:20.618 INFO:tasks.workunit.client.1.smithi084.stdout:0/6: fsync f0 0 2024-02-28T23:07:20.620 INFO:tasks.workunit.client.1.smithi084.stdout:1/15: dread f2 [0,4194304] 0 2024-02-28T23:07:20.625 INFO:tasks.workunit.client.1.smithi084.stdout:5/11: dwrite f1 [0,4194304] 0 2024-02-28T23:07:20.626 INFO:tasks.workunit.client.1.smithi084.stdout:7/8: fsync f2 0 2024-02-28T23:07:20.627 INFO:tasks.workunit.client.1.smithi084.stdout:9/11: chown f0 6 1 2024-02-28T23:07:20.633 INFO:tasks.workunit.client.1.smithi084.stdout:5/12: mknod c3 0 2024-02-28T23:07:20.633 INFO:tasks.workunit.client.1.smithi084.stdout:6/18: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:20.634 INFO:tasks.workunit.client.1.smithi084.stdout:2/8: dwrite f1 [0,4194304] 0 2024-02-28T23:07:20.637 INFO:tasks.workunit.client.1.smithi084.stdout:7/9: truncate f2 110413 0 2024-02-28T23:07:20.639 INFO:tasks.workunit.client.1.smithi084.stdout:6/19: chown d1/l5 9524 1 2024-02-28T23:07:20.640 INFO:tasks.workunit.client.1.smithi084.stdout:1/16: mknod c3 0 2024-02-28T23:07:20.644 INFO:tasks.workunit.client.1.smithi084.stdout:3/8: dread f1 [0,4194304] 0 2024-02-28T23:07:20.648 INFO:tasks.workunit.client.1.smithi084.stdout:4/23: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:20.648 INFO:tasks.workunit.client.1.smithi084.stdout:6/20: chown d1/f4 35233227 1 2024-02-28T23:07:20.650 INFO:tasks.workunit.client.1.smithi084.stdout:7/10: creat f4 x:0 0 0 2024-02-28T23:07:20.679 INFO:tasks.workunit.client.1.smithi084.stdout:6/21: fsync d1/f4 0 2024-02-28T23:07:20.680 INFO:tasks.workunit.client.1.smithi084.stdout:1/17: symlink l4 0 2024-02-28T23:07:20.691 INFO:tasks.workunit.client.1.smithi084.stdout:7/11: creat f5 x:0 0 0 2024-02-28T23:07:20.709 INFO:tasks.workunit.client.1.smithi084.stdout:0/7: dread f0 [0,4194304] 0 2024-02-28T23:07:20.709 INFO:tasks.workunit.client.1.smithi084.stdout:5/13: dwrite f1 [4194304,4194304] 0 2024-02-28T23:07:20.711 INFO:tasks.workunit.client.1.smithi084.stdout:7/12: dread f2 [0,4194304] 0 2024-02-28T23:07:20.719 INFO:tasks.workunit.client.1.smithi084.stdout:2/9: dread f1 [0,4194304] 0 2024-02-28T23:07:20.721 INFO:tasks.workunit.client.1.smithi084.stdout:8/16: dwrite d0/f3 [0,4194304] 0 2024-02-28T23:07:20.723 INFO:tasks.workunit.client.1.smithi084.stdout:7/13: truncate f2 193043 0 2024-02-28T23:07:20.725 INFO:tasks.workunit.client.1.smithi084.stdout:2/10: chown f1 329 1 2024-02-28T23:07:20.725 INFO:tasks.workunit.client.1.smithi084.stdout:8/17: stat d0/l5 0 2024-02-28T23:07:20.726 INFO:tasks.workunit.client.1.smithi084.stdout:7/14: write f5 [755328,42976] 0 2024-02-28T23:07:20.726 INFO:tasks.workunit.client.1.smithi084.stdout:8/18: chown d0 80779201 1 2024-02-28T23:07:20.726 INFO:tasks.workunit.client.1.smithi084.stdout:2/11: fsync f0 0 2024-02-28T23:07:20.730 INFO:tasks.workunit.client.1.smithi084.stdout:7/15: fdatasync f2 0 2024-02-28T23:07:20.733 INFO:tasks.workunit.client.1.smithi084.stdout:2/12: read f1 [594368,54239] 0 2024-02-28T23:07:20.735 INFO:tasks.workunit.client.1.smithi084.stdout:1/18: dwrite f1 [0,4194304] 0 2024-02-28T23:07:20.736 INFO:tasks.workunit.client.1.smithi084.stdout:7/16: dread - f3 zero size 2024-02-28T23:07:20.736 INFO:tasks.workunit.client.1.smithi084.stdout:4/24: dread d1/f4 [0,4194304] 0 2024-02-28T23:07:20.737 INFO:tasks.workunit.client.1.smithi084.stdout:2/13: chown f0 1805 1 2024-02-28T23:07:20.821 INFO:tasks.workunit.client.1.smithi084.stdout:5/14: symlink l4 0 2024-02-28T23:07:20.822 INFO:tasks.workunit.client.1.smithi084.stdout:0/8: unlink d1/c2 0 2024-02-28T23:07:20.823 INFO:tasks.workunit.client.1.smithi084.stdout:5/15: write f1 [8653899,42415] 0 2024-02-28T23:07:20.825 INFO:tasks.workunit.client.1.smithi084.stdout:5/16: chown l2 4296 1 2024-02-28T23:07:20.829 INFO:tasks.workunit.client.1.smithi084.stdout:8/19: unlink d0/f3 0 2024-02-28T23:07:20.829 INFO:tasks.workunit.client.1.smithi084.stdout:1/19: write f2 [966264,116481] 0 2024-02-28T23:07:20.831 INFO:tasks.workunit.client.1.smithi084.stdout:2/14: creat f2 x:0 0 0 2024-02-28T23:07:20.836 INFO:tasks.workunit.client.1.smithi084.stdout:9/12: getdents . 0 2024-02-28T23:07:20.841 INFO:tasks.workunit.client.1.smithi084.stdout:5/17: unlink f0 0 2024-02-28T23:07:20.845 INFO:tasks.workunit.client.1.smithi084.stdout:7/17: rename f2 to f6 0 2024-02-28T23:07:20.847 INFO:tasks.workunit.client.1.smithi084.stdout:7/18: chown f5 47406408 1 2024-02-28T23:07:20.847 INFO:tasks.workunit.client.1.smithi084.stdout:2/15: rename f0 to f3 0 2024-02-28T23:07:20.850 INFO:tasks.workunit.client.1.smithi084.stdout:6/22: link d1/l5 d1/l6 0 2024-02-28T23:07:20.851 INFO:tasks.workunit.client.1.smithi084.stdout:7/19: truncate f3 562601 0 2024-02-28T23:07:20.854 INFO:tasks.workunit.client.1.smithi084.stdout:1/20: dwrite f2 [0,4194304] 0 2024-02-28T23:07:20.856 INFO:tasks.workunit.client.1.smithi084.stdout:7/20: symlink l7 0 2024-02-28T23:07:20.857 INFO:tasks.workunit.client.1.smithi084.stdout:1/21: write f2 [1961636,48446] 0 2024-02-28T23:07:20.862 INFO:tasks.workunit.client.1.smithi084.stdout:7/21: write f5 [1487691,128460] 0 2024-02-28T23:07:20.862 INFO:tasks.workunit.client.1.smithi084.stdout:7/22: rmdir - no directory 2024-02-28T23:07:20.868 INFO:tasks.workunit.client.1.smithi084.stdout:7/23: unlink f3 0 2024-02-28T23:07:20.868 INFO:tasks.workunit.client.1.smithi084.stdout:7/24: chown l7 917074806 1 2024-02-28T23:07:20.869 INFO:tasks.workunit.client.1.smithi084.stdout:2/16: dread f1 [0,4194304] 0 2024-02-28T23:07:20.871 INFO:tasks.workunit.client.1.smithi084.stdout:6/23: dread d1/f4 [0,4194304] 0 2024-02-28T23:07:20.874 INFO:tasks.workunit.client.1.smithi084.stdout:7/25: unlink l7 0 2024-02-28T23:07:20.876 INFO:tasks.workunit.client.1.smithi084.stdout:2/17: creat f4 x:0 0 0 2024-02-28T23:07:20.877 INFO:tasks.workunit.client.1.smithi084.stdout:2/18: truncate f4 482226 0 2024-02-28T23:07:20.879 INFO:tasks.workunit.client.1.smithi084.stdout:1/22: getdents . 0 2024-02-28T23:07:20.882 INFO:tasks.workunit.client.1.smithi084.stdout:7/26: write f6 [474445,116521] 0 2024-02-28T23:07:20.884 INFO:tasks.workunit.client.1.smithi084.stdout:6/24: truncate d1/f4 3143090 0 2024-02-28T23:07:20.885 INFO:tasks.workunit.client.1.smithi084.stdout:7/27: truncate f5 2566777 0 2024-02-28T23:07:20.946 INFO:tasks.workunit.client.1.smithi084.stdout:5/18: getdents . 0 2024-02-28T23:07:20.946 INFO:tasks.workunit.client.1.smithi084.stdout:5/19: chown c3 358286 1 2024-02-28T23:07:20.948 INFO:tasks.workunit.client.1.smithi084.stdout:7/28: rename f6 to f8 0 2024-02-28T23:07:20.949 INFO:tasks.workunit.client.1.smithi084.stdout:7/29: write f5 [3423553,79465] 0 2024-02-28T23:07:20.950 INFO:tasks.workunit.client.1.smithi084.stdout:7/30: stat f8 0 2024-02-28T23:07:20.952 INFO:tasks.workunit.client.1.smithi084.stdout:9/13: truncate f0 1860040 0 2024-02-28T23:07:20.954 INFO:tasks.workunit.client.1.smithi084.stdout:7/31: rename f4 to f9 0 2024-02-28T23:07:20.959 INFO:tasks.workunit.client.1.smithi084.stdout:5/20: dread f1 [0,4194304] 0 2024-02-28T23:07:20.961 INFO:tasks.workunit.client.1.smithi084.stdout:5/21: symlink l5 0 2024-02-28T23:07:20.962 INFO:tasks.workunit.client.1.smithi084.stdout:6/25: rmdir d1 39 2024-02-28T23:07:20.963 INFO:tasks.workunit.client.1.smithi084.stdout:5/22: read f1 [4386735,34087] 0 2024-02-28T23:07:20.964 INFO:tasks.workunit.client.1.smithi084.stdout:1/23: truncate f2 733837 0 2024-02-28T23:07:20.967 INFO:tasks.workunit.client.1.smithi084.stdout:5/23: write f1 [3821563,86432] 0 2024-02-28T23:07:20.972 INFO:tasks.workunit.client.1.smithi084.stdout:5/24: creat f6 x:0 0 0 2024-02-28T23:07:20.974 INFO:tasks.workunit.client.1.smithi084.stdout:6/26: fdatasync d1/f4 0 2024-02-28T23:07:20.975 INFO:tasks.workunit.client.1.smithi084.stdout:9/14: dread f0 [0,4194304] 0 2024-02-28T23:07:20.976 INFO:tasks.workunit.client.1.smithi084.stdout:5/25: write f1 [3232635,114833] 0 2024-02-28T23:07:20.978 INFO:tasks.workunit.client.1.smithi084.stdout:6/27: readlink d1/l6 0 2024-02-28T23:07:20.979 INFO:tasks.workunit.client.1.smithi084.stdout:9/15: chown c2 59 1 2024-02-28T23:07:20.979 INFO:tasks.workunit.client.1.smithi084.stdout:5/26: write f6 [69680,55562] 0 2024-02-28T23:07:20.985 INFO:tasks.workunit.client.1.smithi084.stdout:9/16: read f0 [764550,83214] 0 2024-02-28T23:07:20.992 INFO:tasks.workunit.client.1.smithi084.stdout:5/27: getdents . 0 2024-02-28T23:07:20.992 INFO:tasks.workunit.client.1.smithi084.stdout:9/17: mknod c3 0 2024-02-28T23:07:20.992 INFO:tasks.workunit.client.1.smithi084.stdout:7/32: dwrite f8 [0,4194304] 0 2024-02-28T23:07:20.994 INFO:tasks.workunit.client.1.smithi084.stdout:9/18: mkdir d4 0 2024-02-28T23:07:20.995 INFO:tasks.workunit.client.1.smithi084.stdout:5/28: write f6 [956199,34640] 0 2024-02-28T23:07:20.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/19: read f0 [805660,28552] 0 2024-02-28T23:07:20.999 INFO:tasks.workunit.client.1.smithi084.stdout:1/24: dwrite f2 [0,4194304] 0 2024-02-28T23:07:21.000 INFO:tasks.workunit.client.1.smithi084.stdout:6/28: dread d1/f4 [0,4194304] 0 2024-02-28T23:07:21.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/29: chown d1/l6 4184373 1 2024-02-28T23:07:21.007 INFO:tasks.workunit.client.1.smithi084.stdout:6/30: mknod d1/c7 0 2024-02-28T23:07:21.008 INFO:tasks.workunit.client.1.smithi084.stdout:1/25: link l0 l5 0 2024-02-28T23:07:21.010 INFO:tasks.workunit.client.1.smithi084.stdout:1/26: symlink l6 0 2024-02-28T23:07:21.011 INFO:tasks.workunit.client.1.smithi084.stdout:9/20: dread f0 [0,4194304] 0 2024-02-28T23:07:21.011 INFO:tasks.workunit.client.1.smithi084.stdout:6/31: fsync d1/f4 0 2024-02-28T23:07:21.012 INFO:tasks.workunit.client.1.smithi084.stdout:7/33: dread f8 [0,4194304] 0 2024-02-28T23:07:21.016 INFO:tasks.workunit.client.1.smithi084.stdout:5/29: dread f1 [0,4194304] 0 2024-02-28T23:07:21.016 INFO:tasks.workunit.client.1.smithi084.stdout:1/27: creat f7 x:0 0 0 2024-02-28T23:07:21.017 INFO:tasks.workunit.client.1.smithi084.stdout:7/34: chown c0 0 1 2024-02-28T23:07:21.018 INFO:tasks.workunit.client.1.smithi084.stdout:7/35: dread - f9 zero size 2024-02-28T23:07:21.022 INFO:tasks.workunit.client.1.smithi084.stdout:6/32: rename c0 to d1/c8 0 2024-02-28T23:07:21.022 INFO:tasks.workunit.client.1.smithi084.stdout:9/21: dread f0 [0,4194304] 0 2024-02-28T23:07:21.022 INFO:tasks.workunit.client.1.smithi084.stdout:1/28: mknod c8 0 2024-02-28T23:07:21.023 INFO:tasks.workunit.client.1.smithi084.stdout:5/30: write f1 [1626190,65821] 0 2024-02-28T23:07:21.024 INFO:tasks.workunit.client.1.smithi084.stdout:6/33: readlink d1/l3 0 2024-02-28T23:07:21.026 INFO:tasks.workunit.client.1.smithi084.stdout:1/29: read f2 [2384458,10790] 0 2024-02-28T23:07:21.028 INFO:tasks.workunit.client.1.smithi084.stdout:1/30: write f1 [2526366,2990] 0 2024-02-28T23:07:21.029 INFO:tasks.workunit.client.1.smithi084.stdout:5/31: creat f7 x:0 0 0 2024-02-28T23:07:21.031 INFO:tasks.workunit.client.1.smithi084.stdout:6/34: mkdir d1/d9 0 2024-02-28T23:07:21.038 INFO:tasks.workunit.client.1.smithi084.stdout:6/35: fdatasync d1/f4 0 2024-02-28T23:07:21.040 INFO:tasks.workunit.client.1.smithi084.stdout:1/31: write f2 [4447808,80639] 0 2024-02-28T23:07:21.052 INFO:tasks.workunit.client.1.smithi084.stdout:1/32: write f2 [1446285,58777] 0 2024-02-28T23:07:21.054 INFO:tasks.workunit.client.1.smithi084.stdout:6/36: write d1/f4 [278645,77348] 0 2024-02-28T23:07:21.060 INFO:tasks.workunit.client.1.smithi084.stdout:7/36: dwrite f9 [0,4194304] 0 2024-02-28T23:07:21.066 INFO:tasks.workunit.client.1.smithi084.stdout:1/33: write f2 [874444,8016] 0 2024-02-28T23:07:21.066 INFO:tasks.workunit.client.1.smithi084.stdout:6/37: unlink d1/l3 0 2024-02-28T23:07:21.066 INFO:tasks.workunit.client.1.smithi084.stdout:7/37: symlink la 0 2024-02-28T23:07:21.070 INFO:tasks.workunit.client.1.smithi084.stdout:1/34: chown c8 123539515 1 2024-02-28T23:07:21.073 INFO:tasks.workunit.client.1.smithi084.stdout:5/32: dwrite f6 [0,4194304] 0 2024-02-28T23:07:21.075 INFO:tasks.workunit.client.1.smithi084.stdout:5/33: readlink l5 0 2024-02-28T23:07:21.077 INFO:tasks.workunit.client.1.smithi084.stdout:9/22: dwrite f0 [0,4194304] 0 2024-02-28T23:07:21.077 INFO:tasks.workunit.client.1.smithi084.stdout:5/34: stat l2 0 2024-02-28T23:07:21.088 INFO:tasks.workunit.client.1.smithi084.stdout:5/35: unlink f1 0 2024-02-28T23:07:21.092 INFO:tasks.workunit.client.1.smithi084.stdout:9/23: mknod d4/c5 0 2024-02-28T23:07:21.093 INFO:tasks.workunit.client.1.smithi084.stdout:6/38: truncate d1/f4 1884464 0 2024-02-28T23:07:21.094 INFO:tasks.workunit.client.1.smithi084.stdout:1/35: truncate f1 516359 0 2024-02-28T23:07:21.097 INFO:tasks.workunit.client.1.smithi084.stdout:1/36: write f7 [318041,64020] 0 2024-02-28T23:07:21.099 INFO:tasks.workunit.client.1.smithi084.stdout:9/24: mkdir d4/d6 0 2024-02-28T23:07:21.100 INFO:tasks.workunit.client.1.smithi084.stdout:5/36: rename l2 to l8 0 2024-02-28T23:07:21.101 INFO:tasks.workunit.client.1.smithi084.stdout:9/25: readlink l1 0 2024-02-28T23:07:21.102 INFO:tasks.workunit.client.1.smithi084.stdout:6/39: rename d1/l5 to d1/d9/la 0 2024-02-28T23:07:21.102 INFO:tasks.workunit.client.1.smithi084.stdout:7/38: dwrite f5 [0,4194304] 0 2024-02-28T23:07:21.103 INFO:tasks.workunit.client.1.smithi084.stdout:6/40: rename d1 to d1/db 22 2024-02-28T23:07:21.105 INFO:tasks.workunit.client.1.smithi084.stdout:5/37: mknod c9 0 2024-02-28T23:07:21.106 INFO:tasks.workunit.client.1.smithi084.stdout:1/37: creat f9 x:0 0 0 2024-02-28T23:07:21.107 INFO:tasks.workunit.client.1.smithi084.stdout:5/38: write f6 [3872450,37008] 0 2024-02-28T23:07:21.115 INFO:tasks.workunit.client.1.smithi084.stdout:9/26: creat d4/d6/f7 x:0 0 0 2024-02-28T23:07:21.115 INFO:tasks.workunit.client.1.smithi084.stdout:1/38: creat fa x:0 0 0 2024-02-28T23:07:21.118 INFO:tasks.workunit.client.1.smithi084.stdout:9/27: write d4/d6/f7 [185737,3038] 0 2024-02-28T23:07:21.126 INFO:tasks.workunit.client.1.smithi084.stdout:9/28: creat d4/d6/f8 x:0 0 0 2024-02-28T23:07:21.146 INFO:tasks.workunit.client.1.smithi084.stdout:1/39: dread f2 [0,4194304] 0 2024-02-28T23:07:21.146 INFO:tasks.workunit.client.1.smithi084.stdout:7/39: dwrite f8 [0,4194304] 0 2024-02-28T23:07:21.147 INFO:tasks.workunit.client.1.smithi084.stdout:1/40: rmdir - no directory 2024-02-28T23:07:21.148 INFO:tasks.workunit.client.1.smithi084.stdout:5/39: dwrite f7 [0,4194304] 0 2024-02-28T23:07:21.152 INFO:tasks.workunit.client.1.smithi084.stdout:7/40: mkdir db 0 2024-02-28T23:07:21.157 INFO:tasks.workunit.client.1.smithi084.stdout:5/40: chown f6 0 1 2024-02-28T23:07:21.158 INFO:tasks.workunit.client.1.smithi084.stdout:6/41: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:21.158 INFO:tasks.workunit.client.1.smithi084.stdout:5/41: write f7 [1750953,46771] 0 2024-02-28T23:07:21.160 INFO:tasks.workunit.client.1.smithi084.stdout:1/41: dread f1 [0,4194304] 0 2024-02-28T23:07:21.164 INFO:tasks.workunit.client.1.smithi084.stdout:1/42: readlink l0 0 2024-02-28T23:07:21.164 INFO:tasks.workunit.client.1.smithi084.stdout:6/42: mknod d1/cc 0 2024-02-28T23:07:21.167 INFO:tasks.workunit.client.1.smithi084.stdout:9/29: dread f0 [0,4194304] 0 2024-02-28T23:07:21.168 INFO:tasks.workunit.client.1.smithi084.stdout:1/43: symlink lb 0 2024-02-28T23:07:21.168 INFO:tasks.workunit.client.1.smithi084.stdout:6/43: read d1/f4 [2243371,126543] 0 2024-02-28T23:07:21.175 INFO:tasks.workunit.client.1.smithi084.stdout:1/44: creat fc x:0 0 0 2024-02-28T23:07:21.178 INFO:tasks.workunit.client.1.smithi084.stdout:7/41: dread f5 [0,4194304] 0 2024-02-28T23:07:21.179 INFO:tasks.workunit.client.1.smithi084.stdout:7/42: chown la 11 1 2024-02-28T23:07:21.180 INFO:tasks.workunit.client.1.smithi084.stdout:6/44: unlink d1/c8 0 2024-02-28T23:07:21.182 INFO:tasks.workunit.client.1.smithi084.stdout:1/45: creat fd x:0 0 0 2024-02-28T23:07:21.184 INFO:tasks.workunit.client.1.smithi084.stdout:7/43: unlink l1 0 2024-02-28T23:07:21.186 INFO:tasks.workunit.client.1.smithi084.stdout:6/45: mkdir d1/dd 0 2024-02-28T23:07:21.186 INFO:tasks.workunit.client.1.smithi084.stdout:7/44: truncate f8 5067434 0 2024-02-28T23:07:21.188 INFO:tasks.workunit.client.1.smithi084.stdout:1/46: write f7 [686942,52847] 0 2024-02-28T23:07:21.191 INFO:tasks.workunit.client.1.smithi084.stdout:7/45: mknod db/cc 0 2024-02-28T23:07:21.193 INFO:tasks.workunit.client.1.smithi084.stdout:7/46: rmdir db 39 2024-02-28T23:07:21.194 INFO:tasks.workunit.client.1.smithi084.stdout:6/46: fsync d1/f4 0 2024-02-28T23:07:21.194 INFO:tasks.workunit.client.1.smithi084.stdout:1/47: write f2 [3972670,53469] 0 2024-02-28T23:07:21.195 INFO:tasks.workunit.client.1.smithi084.stdout:5/42: dwrite f6 [0,4194304] 0 2024-02-28T23:07:21.195 INFO:tasks.workunit.client.1.smithi084.stdout:7/47: read f9 [3955067,58010] 0 2024-02-28T23:07:21.198 INFO:tasks.workunit.client.1.smithi084.stdout:1/48: rename l4 to le 0 2024-02-28T23:07:21.200 INFO:tasks.workunit.client.1.smithi084.stdout:6/47: stat d1/d9/la 0 2024-02-28T23:07:21.200 INFO:tasks.workunit.client.1.smithi084.stdout:1/49: chown lb 618 1 2024-02-28T23:07:21.201 INFO:tasks.workunit.client.1.smithi084.stdout:6/48: fdatasync d1/f4 0 2024-02-28T23:07:21.201 INFO:tasks.workunit.client.1.smithi084.stdout:1/50: write fd [704955,86790] 0 2024-02-28T23:07:21.234 INFO:tasks.workunit.client.1.smithi084.stdout:6/49: rmdir d1 39 2024-02-28T23:07:21.235 INFO:tasks.workunit.client.1.smithi084.stdout:1/51: chown f7 446069 1 2024-02-28T23:07:21.235 INFO:tasks.workunit.client.1.smithi084.stdout:7/48: truncate f8 4131434 0 2024-02-28T23:07:21.236 INFO:tasks.workunit.client.1.smithi084.stdout:1/52: truncate f9 84053 0 2024-02-28T23:07:21.237 INFO:tasks.workunit.client.1.smithi084.stdout:7/49: unlink f5 0 2024-02-28T23:07:21.241 INFO:tasks.workunit.client.1.smithi084.stdout:7/50: rmdir db 39 2024-02-28T23:07:21.241 INFO:tasks.workunit.client.1.smithi084.stdout:5/43: dread f7 [0,4194304] 0 2024-02-28T23:07:21.258 INFO:tasks.workunit.client.1.smithi084.stdout:6/50: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:21.260 INFO:tasks.workunit.client.1.smithi084.stdout:5/44: dread f7 [0,4194304] 0 2024-02-28T23:07:21.265 INFO:tasks.workunit.client.1.smithi084.stdout:1/53: dwrite fa [0,4194304] 0 2024-02-28T23:07:21.267 INFO:tasks.workunit.client.1.smithi084.stdout:1/54: chown fc 516504481 1 2024-02-28T23:07:21.267 INFO:tasks.workunit.client.1.smithi084.stdout:5/45: rename f6 to fa 0 2024-02-28T23:07:21.270 INFO:tasks.workunit.client.1.smithi084.stdout:6/51: rename d1/f4 to d1/fe 0 2024-02-28T23:07:21.270 INFO:tasks.workunit.client.1.smithi084.stdout:5/46: chown f7 1 1 2024-02-28T23:07:21.273 INFO:tasks.workunit.client.1.smithi084.stdout:7/51: dwrite f9 [0,4194304] 0 2024-02-28T23:07:21.274 INFO:tasks.workunit.client.1.smithi084.stdout:6/52: write d1/fe [4403971,6039] 0 2024-02-28T23:07:21.293 INFO:tasks.workunit.client.1.smithi084.stdout:5/47: rename fa to fb 0 2024-02-28T23:07:21.293 INFO:tasks.workunit.client.1.smithi084.stdout:1/55: dwrite f9 [0,4194304] 0 2024-02-28T23:07:21.294 INFO:tasks.workunit.client.1.smithi084.stdout:7/52: dread f9 [0,4194304] 0 2024-02-28T23:07:21.294 INFO:tasks.workunit.client.1.smithi084.stdout:7/53: stat c0 0 2024-02-28T23:07:21.295 INFO:tasks.workunit.client.1.smithi084.stdout:5/48: mkdir dc 0 2024-02-28T23:07:21.295 INFO:tasks.workunit.client.1.smithi084.stdout:7/54: stat c0 0 2024-02-28T23:07:21.296 INFO:tasks.workunit.client.1.smithi084.stdout:1/56: rename f7 to ff 0 2024-02-28T23:07:21.300 INFO:tasks.workunit.client.1.smithi084.stdout:1/57: mknod c10 0 2024-02-28T23:07:21.302 INFO:tasks.workunit.client.1.smithi084.stdout:7/55: creat db/fd x:0 0 0 2024-02-28T23:07:21.303 INFO:tasks.workunit.client.1.smithi084.stdout:1/58: unlink l6 0 2024-02-28T23:07:21.304 INFO:tasks.workunit.client.1.smithi084.stdout:7/56: truncate db/fd 1035356 0 2024-02-28T23:07:21.306 INFO:tasks.workunit.client.1.smithi084.stdout:1/59: creat f11 x:0 0 0 2024-02-28T23:07:21.308 INFO:tasks.workunit.client.1.smithi084.stdout:1/60: mkdir d12 0 2024-02-28T23:07:21.308 INFO:tasks.workunit.client.1.smithi084.stdout:5/49: dread f7 [0,4194304] 0 2024-02-28T23:07:21.309 INFO:tasks.workunit.client.1.smithi084.stdout:1/61: stat l5 0 2024-02-28T23:07:21.316 INFO:tasks.workunit.client.1.smithi084.stdout:7/57: dread f8 [0,4194304] 0 2024-02-28T23:07:21.327 INFO:tasks.workunit.client.1.smithi084.stdout:1/62: dwrite fd [0,4194304] 0 2024-02-28T23:07:21.333 INFO:tasks.workunit.client.1.smithi084.stdout:1/63: creat d12/f13 x:0 0 0 2024-02-28T23:07:21.336 INFO:tasks.workunit.client.1.smithi084.stdout:1/64: mknod d12/c14 0 2024-02-28T23:07:21.337 INFO:tasks.workunit.client.1.smithi084.stdout:1/65: readlink l5 0 2024-02-28T23:07:21.342 INFO:tasks.workunit.client.1.smithi084.stdout:7/58: dwrite db/fd [0,4194304] 0 2024-02-28T23:07:21.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/66: rename fa to d12/f15 0 2024-02-28T23:07:21.346 INFO:tasks.workunit.client.1.smithi084.stdout:7/59: symlink db/le 0 2024-02-28T23:07:21.350 INFO:tasks.workunit.client.1.smithi084.stdout:7/60: creat db/ff x:0 0 0 2024-02-28T23:07:21.352 INFO:tasks.workunit.client.1.smithi084.stdout:7/61: write db/ff [343837,94342] 0 2024-02-28T23:07:21.352 INFO:tasks.workunit.client.1.smithi084.stdout:1/67: fsync d12/f13 0 2024-02-28T23:07:21.354 INFO:tasks.workunit.client.1.smithi084.stdout:1/68: write f9 [4036398,54316] 0 2024-02-28T23:07:21.365 INFO:tasks.workunit.client.1.smithi084.stdout:7/62: dread db/ff [0,4194304] 0 2024-02-28T23:07:21.367 INFO:tasks.workunit.client.1.smithi084.stdout:7/63: mkdir db/d10 0 2024-02-28T23:07:21.369 INFO:tasks.workunit.client.1.smithi084.stdout:1/69: dwrite f2 [0,4194304] 0 2024-02-28T23:07:21.372 INFO:tasks.workunit.client.1.smithi084.stdout:1/70: creat d12/f16 x:0 0 0 2024-02-28T23:07:21.373 INFO:tasks.workunit.client.1.smithi084.stdout:1/71: chown c10 255239598 1 2024-02-28T23:07:21.375 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:21 smithi057 ceph-mon[29096]: pgmap v17: 65 pgs: 65 active+clean; 215 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 372 op/s 2024-02-28T23:07:21.376 INFO:tasks.workunit.client.1.smithi084.stdout:1/72: mknod d12/c17 0 2024-02-28T23:07:21.381 INFO:tasks.workunit.client.1.smithi084.stdout:7/64: dread f9 [0,4194304] 0 2024-02-28T23:07:21.381 INFO:tasks.workunit.client.1.smithi084.stdout:1/73: mkdir d12/d18 0 2024-02-28T23:07:21.383 INFO:tasks.workunit.client.1.smithi084.stdout:7/65: write db/fd [1536647,124042] 0 2024-02-28T23:07:21.384 INFO:tasks.workunit.client.1.smithi084.stdout:1/74: dread f1 [0,4194304] 0 2024-02-28T23:07:21.385 INFO:tasks.workunit.client.1.smithi084.stdout:7/66: mkdir db/d11 0 2024-02-28T23:07:21.387 INFO:tasks.workunit.client.1.smithi084.stdout:7/67: write db/fd [4883469,100983] 0 2024-02-28T23:07:21.388 INFO:tasks.workunit.client.1.smithi084.stdout:1/75: rename f2 to d12/d18/f19 0 2024-02-28T23:07:21.388 INFO:tasks.workunit.client.1.smithi084.stdout:1/76: stat f1 0 2024-02-28T23:07:21.391 INFO:tasks.workunit.client.1.smithi084.stdout:7/68: rename f8 to db/d11/f12 0 2024-02-28T23:07:21.393 INFO:tasks.workunit.client.1.smithi084.stdout:7/69: write db/fd [5585035,62208] 0 2024-02-28T23:07:21.397 INFO:tasks.workunit.client.1.smithi084.stdout:1/77: fsync d12/d18/f19 0 2024-02-28T23:07:21.397 INFO:tasks.workunit.client.1.smithi084.stdout:1/78: fdatasync f9 0 2024-02-28T23:07:21.401 INFO:tasks.workunit.client.1.smithi084.stdout:1/79: dread ff [0,4194304] 0 2024-02-28T23:07:21.415 INFO:tasks.workunit.client.1.smithi084.stdout:1/80: dwrite d12/d18/f19 [0,4194304] 0 2024-02-28T23:07:21.416 INFO:tasks.workunit.client.1.smithi084.stdout:1/81: write fd [2927758,86888] 0 2024-02-28T23:07:21.634 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:21 smithi084 ceph-mon[34826]: pgmap v17: 65 pgs: 65 active+clean; 215 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 372 op/s 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 -- 172.21.15.57:0/2507025446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28069270 msgr2=0x7f2e281051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 --2- 172.21.15.57:0/2507025446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28069270 0x7f2e281051d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009870 tx=0x7f2e18009b80 comp rx=0 tx=0).stop 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 -- 172.21.15.57:0/2507025446 shutdown_connections 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 --2- 172.21.15.57:0/2507025446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28105710 0x7f2e28107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 --2- 172.21.15.57:0/2507025446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28069270 0x7f2e281051d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.851+0000 7f2e2da00700 1 -- 172.21.15.57:0/2507025446 >> 172.21.15.57:0/2507025446 conn(0x7f2e280fa780 msgr2=0x7f2e280fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.852+0000 7f2e2da00700 1 -- 172.21.15.57:0/2507025446 shutdown_connections 2024-02-28T23:07:21.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.852+0000 7f2e2da00700 1 -- 172.21.15.57:0/2507025446 wait complete. 2024-02-28T23:07:21.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.853+0000 7f2e2da00700 1 Processor -- start 2024-02-28T23:07:21.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.853+0000 7f2e2da00700 1 -- start start 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e2da00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e2da00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28198230 0x7f2e2819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e2da00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e281986a0 con 0x7f2e28198230 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e2da00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28198810 con 0x7f2e28105710 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e26a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e26a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:49814/0 (socket says 172.21.15.57:49814) 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e26a00700 1 -- 172.21.15.57:0/1979758641 learned_addr learned my addr 172.21.15.57:0/1979758641 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.854+0000 7f2e26000700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28198230 0x7f2e2819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.855+0000 7f2e26a00700 1 -- 172.21.15.57:0/1979758641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28198230 msgr2=0x7f2e2819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.855+0000 7f2e26a00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28198230 0x7f2e2819d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.855+0000 7f2e26a00700 1 -- 172.21.15.57:0/1979758641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18009520 con 0x7f2e28105710 2024-02-28T23:07:21.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.855+0000 7f2e26a00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009f50 tx=0x7f2e1800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:21.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.856+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1801c070 con 0x7f2e28105710 2024-02-28T23:07:21.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.856+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e1800be80 con 0x7f2e28105710 2024-02-28T23:07:21.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.856+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e18022e80 con 0x7f2e28105710 2024-02-28T23:07:21.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.856+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2819d7a0 con 0x7f2e28105710 2024-02-28T23:07:21.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.856+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2819dc10 con 0x7f2e28105710 2024-02-28T23:07:21.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.858+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28191dd0 con 0x7f2e28105710 2024-02-28T23:07:21.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.859+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f2e18020070 con 0x7f2e28105710 2024-02-28T23:07:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.861+0000 7f2e1f400700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 0x7f2e0c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.861+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2e1808d830 con 0x7f2e28105710 2024-02-28T23:07:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.862+0000 7f2e26000700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 0x7f2e0c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.863+0000 7f2e26000700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 0x7f2e0c06f3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005fd0 tx=0x7f2e14009310 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:21.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:21.865+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f2e18057550 con 0x7f2e28105710 2024-02-28T23:07:22.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.292+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e2802ce80 con 0x7f2e0c06cf40 2024-02-28T23:07:22.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.300+0000 7f2e1f400700 1 -- 172.21.15.57:0/1979758641 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f2e2802ce80 con 0x7f2e0c06cf40 2024-02-28T23:07:22.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 msgr2=0x7f2e0c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 0x7f2e0c06f3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005fd0 tx=0x7f2e14009310 comp rx=0 tx=0).stop 2024-02-28T23:07:22.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 msgr2=0x7f2e28197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009f50 tx=0x7f2e1800f740 comp rx=0 tx=0).stop 2024-02-28T23:07:22.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 shutdown_connections 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e28105710 0x7f2e28197cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f2e0c06cf40 0x7f2e0c06f3f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 --2- 172.21.15.57:0/1979758641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e28198230 0x7f2e2819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 >> 172.21.15.57:0/1979758641 conn(0x7f2e280fa780 msgr2=0x7f2e280fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 shutdown_connections 2024-02-28T23:07:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.305+0000 7f2e2da00700 1 -- 172.21.15.57:0/1979758641 wait complete. 2024-02-28T23:07:22.327 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.531+0000 7f94a3c00700 1 -- 172.21.15.57:0/829595427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 msgr2=0x7f949c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.531+0000 7f94a3c00700 1 --2- 172.21.15.57:0/829595427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c0731e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9490009910 tx=0x7f9490009c20 comp rx=0 tx=0).stop 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.532+0000 7f94a3c00700 1 -- 172.21.15.57:0/829595427 shutdown_connections 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.532+0000 7f94a3c00700 1 --2- 172.21.15.57:0/829595427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f949c0737b0 0x7f949c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.532+0000 7f94a3c00700 1 --2- 172.21.15.57:0/829595427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c0731e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.532+0000 7f94a3c00700 1 -- 172.21.15.57:0/829595427 >> 172.21.15.57:0/829595427 conn(0x7f949c0fb7f0 msgr2=0x7f949c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:22.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.532+0000 7f94a3c00700 1 -- 172.21.15.57:0/829595427 shutdown_connections 2024-02-28T23:07:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.533+0000 7f94a3c00700 1 -- 172.21.15.57:0/829595427 wait complete. 2024-02-28T23:07:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.534+0000 7f94a3c00700 1 Processor -- start 2024-02-28T23:07:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.534+0000 7f94a3c00700 1 -- start start 2024-02-28T23:07:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f94a3c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f949c0737b0 0x7f949c10b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f94a3c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f94a3c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c10c0c0 con 0x7f949c0737b0 2024-02-28T23:07:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f94a3c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c10c230 con 0x7f949c074d80 2024-02-28T23:07:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f949be00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f949be00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:49838/0 (socket says 172.21.15.57:49838) 2024-02-28T23:07:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.535+0000 7f949be00700 1 -- 172.21.15.57:0/807946138 learned_addr learned my addr 172.21.15.57:0/807946138 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.536+0000 7f949be00700 1 -- 172.21.15.57:0/807946138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f949c0737b0 msgr2=0x7f949c10b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.536+0000 7f949be00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f949c0737b0 0x7f949c10b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.536+0000 7f949be00700 1 -- 172.21.15.57:0/807946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900095f0 con 0x7f949c074d80 2024-02-28T23:07:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.537+0000 7f949be00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f948800c930 tx=0x7f948800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.537+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9488007ad0 con 0x7f949c074d80 2024-02-28T23:07:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.537+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9488007c30 con 0x7f949c074d80 2024-02-28T23:07:22.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.537+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9488018630 con 0x7f949c074d80 2024-02-28T23:07:22.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.538+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c1a9f30 con 0x7f949c074d80 2024-02-28T23:07:22.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.538+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c1aa400 con 0x7f949c074d80 2024-02-28T23:07:22.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.540+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c10f370 con 0x7f949c074d80 2024-02-28T23:07:22.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.542+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f948800f450 con 0x7f949c074d80 2024-02-28T23:07:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.547+0000 7f9499600700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 0x7f948406f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.547+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f948808ccc0 con 0x7f949c074d80 2024-02-28T23:07:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.547+0000 7f94a1200700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 0x7f948406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.548+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9488056930 con 0x7f949c074d80 2024-02-28T23:07:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.550+0000 7f94a1200700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 0x7f948406f440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94900098e0 tx=0x7f94900054e0 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.921+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f949c1032f0 con 0x7f948406cf90 2024-02-28T23:07:22.921 INFO:tasks.workunit.client.0.smithi057.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-28T23:07:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.924+0000 7f9499600700 1 -- 172.21.15.57:0/807946138 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f949c1032f0 con 0x7f948406cf90 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.928+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 msgr2=0x7f948406f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.928+0000 7f94a3c00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 0x7f948406f440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94900098e0 tx=0x7f94900054e0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.928+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 msgr2=0x7f949c10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.928+0000 7f94a3c00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f948800c930 tx=0x7f948800ccf0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 shutdown_connections 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f948406cf90 0x7f948406f440 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f949c0737b0 0x7f949c10b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 --2- 172.21.15.57:0/807946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949c074d80 0x7f949c10bb30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 >> 172.21.15.57:0/807946138 conn(0x7f949c0fb7f0 msgr2=0x7f949c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 shutdown_connections 2024-02-28T23:07:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:22.929+0000 7f94a3c00700 1 -- 172.21.15.57:0/807946138 wait complete. 2024-02-28T23:07:22.939 INFO:tasks.workunit.client.0.smithi057.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-28T23:07:22.939 INFO:tasks.workunit.client.0.smithi057.stderr:+ make 2024-02-28T23:07:23.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 -- 172.21.15.57:0/3521162823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 msgr2=0x7f0624100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 --2- 172.21.15.57:0/3521162823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624100650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0614009910 tx=0x7f0614009c20 comp rx=0 tx=0).stop 2024-02-28T23:07:23.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 -- 172.21.15.57:0/3521162823 shutdown_connections 2024-02-28T23:07:23.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 --2- 172.21.15.57:0/3521162823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 0x7f0624101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 --2- 172.21.15.57:0/3521162823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624100650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 -- 172.21.15.57:0/3521162823 >> 172.21.15.57:0/3521162823 conn(0x7f06240fb7f0 msgr2=0x7f06240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:23.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 -- 172.21.15.57:0/3521162823 shutdown_connections 2024-02-28T23:07:23.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.118+0000 7f0629200700 1 -- 172.21.15.57:0/3521162823 wait complete. 2024-02-28T23:07:23.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.119+0000 7f0629200700 1 Processor -- start 2024-02-28T23:07:23.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0629200700 1 -- start start 2024-02-28T23:07:23.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0629200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0629200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 0x7f0624192380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0629200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062418e8a0 con 0x7f0624100240 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0629200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062418ea10 con 0x7f0624101440 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0621e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0621e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53076/0 (socket says 172.21.15.57:53076) 2024-02-28T23:07:23.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0621e00700 1 -- 172.21.15.57:0/1756715789 learned_addr learned my addr 172.21.15.57:0/1756715789 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:23.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.120+0000 7f0621400700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 0x7f0624192380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:23.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.121+0000 7f0621e00700 1 -- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 msgr2=0x7f0624192380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.121+0000 7f0621e00700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 0x7f0624192380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.121+0000 7f0621e00700 1 -- 172.21.15.57:0/1756715789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140095f0 con 0x7f0624100240 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.122+0000 7f0621e00700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f061400bda0 tx=0x7f061400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.122+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061401d070 con 0x7f0624100240 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.122+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0614022470 con 0x7f0624100240 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.122+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061400f650 con 0x7f0624100240 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.123+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062418ec90 con 0x7f0624100240 2024-02-28T23:07:23.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.123+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062418f180 con 0x7f0624100240 2024-02-28T23:07:23.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.124+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0604005320 con 0x7f0624100240 2024-02-28T23:07:23.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.126+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f0614004af0 con 0x7f0624100240 2024-02-28T23:07:23.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.127+0000 7f061aa00700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 0x7f060806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.127+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f061408dbf0 con 0x7f0624100240 2024-02-28T23:07:23.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.128+0000 7f0621400700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 0x7f060806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.129+0000 7f0621400700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 0x7f060806f3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f061000ac50 tx=0x7f061000a380 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:23.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.134+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f061405b3c0 con 0x7f0624100240 2024-02-28T23:07:23.206 INFO:tasks.workunit.client.0.smithi057.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-28T23:07:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.419+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0604000bf0 con 0x7f060806cf00 2024-02-28T23:07:23.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.439+0000 7f061aa00700 1 -- 172.21.15.57:0/1756715789 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0604000bf0 con 0x7f060806cf00 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (8m) 14s ago 10m 14.0M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (7m) 14s ago 11m 7822k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (9m) 17s ago 9m 7730k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (11m) 14s ago 11m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (9m) 17s ago 9m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:07:23.438 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (8m) 14s ago 10m 73.7M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (3m) 14s ago 3m 54.0M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (3m) 14s ago 3m 92.0M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (3m) 17s ago 3m 50.8M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (3m) 17s ago 3m 91.1M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (12m) 14s ago 12m 333M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (49s) 17s ago 9m 487M - 19.0.0-1670-g1fb5d106 2272b2c42f9b ae1c65e2de51 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (12m) 14s ago 12m 48.4M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (9m) 17s ago 9m 39.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (11m) 14s ago 11m 7642k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (9m) 17s ago 9m 7750k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (8m) 14s ago 8m 101M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (7m) 14s ago 7m 115M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (7m) 14s ago 7m 95.9M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (6m) 17s ago 6m 117M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:07:23.439 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (6m) 17s ago 6m 112M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:07:23.440 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (5m) 17s ago 5m 101M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:07:23.440 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (8m) 14s ago 10m 61.4M - 2.43.0 a07b618ecd1d ffacfc83a2dc 2024-02-28T23:07:23.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.441+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 msgr2=0x7f060806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.441+0000 7f0629200700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 0x7f060806f3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f061000ac50 tx=0x7f061000a380 comp rx=0 tx=0).stop 2024-02-28T23:07:23.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.441+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 msgr2=0x7f0624191e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.441+0000 7f0629200700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f061400bda0 tx=0x7f061400bdd0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 shutdown_connections 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f060806cf00 0x7f060806f3b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0624100240 0x7f0624191e40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 --2- 172.21.15.57:0/1756715789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0624101440 0x7f0624192380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 >> 172.21.15.57:0/1756715789 conn(0x7f06240fb7f0 msgr2=0x7f0624102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 shutdown_connections 2024-02-28T23:07:23.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.442+0000 7f0629200700 1 -- 172.21.15.57:0/1756715789 wait complete. 2024-02-28T23:07:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.596+0000 7fc8fd000700 1 -- 172.21.15.57:0/2129491647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 msgr2=0x7fc8f8106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.596+0000 7fc8fd000700 1 --2- 172.21.15.57:0/2129491647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f8106c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009910 tx=0x7fc8e8009c20 comp rx=0 tx=0).stop 2024-02-28T23:07:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 -- 172.21.15.57:0/2129491647 shutdown_connections 2024-02-28T23:07:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 --2- 172.21.15.57:0/2129491647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f8107a40 0x7fc8f8107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 --2- 172.21.15.57:0/2129491647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f8106c50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 -- 172.21.15.57:0/2129491647 >> 172.21.15.57:0/2129491647 conn(0x7fc8f8075970 msgr2=0x7fc8f8077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:23.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 -- 172.21.15.57:0/2129491647 shutdown_connections 2024-02-28T23:07:23.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.597+0000 7fc8fd000700 1 -- 172.21.15.57:0/2129491647 wait complete. 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 Processor -- start 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 -- start start 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f8107a40 0x7fc8f819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f819cc60 con 0x7fc8f8106840 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8fd000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f819cda0 con 0x7fc8f8107a40 2024-02-28T23:07:23.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8f5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8f5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53102/0 (socket says 172.21.15.57:53102) 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.598+0000 7fc8f5e00700 1 -- 172.21.15.57:0/3766320623 learned_addr learned my addr 172.21.15.57:0/3766320623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8f5e00700 1 -- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f8107a40 msgr2=0x7fc8f819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8f5e00700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f8107a40 0x7fc8f819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8f5e00700 1 -- 172.21.15.57:0/3766320623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80095f0 con 0x7fc8f8106840 2024-02-28T23:07:23.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8f5e00700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800bda0 tx=0x7fc8e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8e801d070 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc8e8004c50 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f81a17f0 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.599+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8e800f690 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.600+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f81a1ce0 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.601+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f810f390 con 0x7fc8f8106840 2024-02-28T23:07:23.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.602+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7fc8e80229b0 con 0x7fc8f8106840 2024-02-28T23:07:23.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.602+0000 7fc8eea00700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 0x7fc8dc06f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:23.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.602+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc8e808dd00 con 0x7fc8f8106840 2024-02-28T23:07:23.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.603+0000 7fc8efe00700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 0x7fc8dc06f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:23.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.604+0000 7fc8efe00700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 0x7fc8dc06f3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000a990 tx=0x7fc8e0005c80 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:23.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.607+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fc8e8091030 con 0x7fc8f8106840 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:6/53: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:5/50: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:8/20: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:9/30: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:7/70: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:1/82: sync 2024-02-28T23:07:23.814 INFO:tasks.workunit.client.1.smithi084.stdout:3/9: sync 2024-02-28T23:07:23.815 INFO:tasks.workunit.client.1.smithi084.stdout:2/19: sync 2024-02-28T23:07:23.815 INFO:tasks.workunit.client.1.smithi084.stdout:4/25: sync 2024-02-28T23:07:23.815 INFO:tasks.workunit.client.1.smithi084.stdout:0/9: sync 2024-02-28T23:07:23.819 INFO:tasks.workunit.client.1.smithi084.stdout:7/71: read f9 [2818319,34794] 0 2024-02-28T23:07:23.820 INFO:tasks.workunit.client.1.smithi084.stdout:2/20: write f4 [909152,34581] 0 2024-02-28T23:07:23.820 INFO:tasks.workunit.client.1.smithi084.stdout:9/31: chown d4/d6/f7 411783 1 2024-02-28T23:07:23.822 INFO:tasks.workunit.client.1.smithi084.stdout:9/32: fsync d4/d6/f7 0 2024-02-28T23:07:23.901 INFO:tasks.workunit.client.1.smithi084.stdout:1/83: mknod d12/c1a 0 2024-02-28T23:07:23.901 INFO:tasks.workunit.client.1.smithi084.stdout:8/21: mknod d0/c6 0 2024-02-28T23:07:23.901 INFO:tasks.workunit.client.1.smithi084.stdout:8/22: read - no filename 2024-02-28T23:07:23.904 INFO:tasks.workunit.client.1.smithi084.stdout:3/10: rename l0 to d3/l5 0 2024-02-28T23:07:23.904 INFO:tasks.workunit.client.1.smithi084.stdout:1/84: dread - d12/f16 zero size 2024-02-28T23:07:23.909 INFO:tasks.workunit.client.1.smithi084.stdout:4/26: creat d1/f5 x:0 0 0 2024-02-28T23:07:23.909 INFO:tasks.workunit.client.1.smithi084.stdout:9/33: creat d4/d6/f9 x:0 0 0 2024-02-28T23:07:23.909 INFO:tasks.workunit.client.1.smithi084.stdout:5/51: link f7 dc/fd 0 2024-02-28T23:07:23.911 INFO:tasks.workunit.client.1.smithi084.stdout:8/23: rename d0/c6 to d0/c7 0 2024-02-28T23:07:23.916 INFO:tasks.workunit.client.1.smithi084.stdout:3/11: creat d3/f6 x:0 0 0 2024-02-28T23:07:23.917 INFO:tasks.workunit.client.1.smithi084.stdout:7/72: fdatasync db/d11/f12 0 2024-02-28T23:07:23.917 INFO:tasks.workunit.client.1.smithi084.stdout:6/54: rmdir d1/dd 0 2024-02-28T23:07:23.921 INFO:tasks.workunit.client.1.smithi084.stdout:9/34: symlink d4/la 0 2024-02-28T23:07:23.921 INFO:tasks.workunit.client.1.smithi084.stdout:4/27: truncate d1/f4 2091418 0 2024-02-28T23:07:23.921 INFO:tasks.workunit.client.1.smithi084.stdout:6/55: fdatasync d1/fe 0 2024-02-28T23:07:23.925 INFO:tasks.workunit.client.1.smithi084.stdout:8/24: creat d0/f8 x:0 0 0 2024-02-28T23:07:23.926 INFO:tasks.workunit.client.1.smithi084.stdout:0/10: getdents d1 0 2024-02-28T23:07:23.928 INFO:tasks.workunit.client.1.smithi084.stdout:5/52: mkdir dc/de 0 2024-02-28T23:07:23.929 INFO:tasks.workunit.client.1.smithi084.stdout:9/35: chown l1 1028 1 2024-02-28T23:07:23.929 INFO:tasks.workunit.client.1.smithi084.stdout:4/28: creat d1/f6 x:0 0 0 2024-02-28T23:07:23.929 INFO:tasks.workunit.client.1.smithi084.stdout:6/56: unlink d1/l6 0 2024-02-28T23:07:23.931 INFO:tasks.workunit.client.1.smithi084.stdout:0/11: symlink d1/l3 0 2024-02-28T23:07:23.933 INFO:tasks.workunit.client.1.smithi084.stdout:6/57: truncate d1/fe 4913993 0 2024-02-28T23:07:23.934 INFO:tasks.workunit.client.1.smithi084.stdout:5/53: write f7 [4228744,112477] 0 2024-02-28T23:07:23.935 INFO:tasks.workunit.client.1.smithi084.stdout:9/36: unlink c3 0 2024-02-28T23:07:23.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.937+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8f81a23e0 con 0x7fc8f8106840 2024-02-28T23:07:23.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.938+0000 7fc8eea00700 1 -- 172.21.15.57:0/3766320623 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fc8e805b4f0 con 0x7fc8f8106840 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:07:23.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:07:23.938 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:07:23.939 INFO:tasks.workunit.client.1.smithi084.stdout:7/73: symlink db/d10/l13 0 2024-02-28T23:07:23.939 INFO:tasks.workunit.client.1.smithi084.stdout:4/29: creat d1/f7 x:0 0 0 2024-02-28T23:07:23.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 msgr2=0x7fc8dc06f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 0x7fc8dc06f3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000a990 tx=0x7fc8e0005c80 comp rx=0 tx=0).stop 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 msgr2=0x7fc8f819c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800bda0 tx=0x7fc8e800be80 comp rx=0 tx=0).stop 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 shutdown_connections 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fc8dc06cf10 0x7fc8dc06f3c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8f8106840 0x7fc8f819c100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 --2- 172.21.15.57:0/3766320623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f8107a40 0x7fc8f819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 >> 172.21.15.57:0/3766320623 conn(0x7fc8f8075970 msgr2=0x7fc8f810ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.941+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 shutdown_connections 2024-02-28T23:07:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:23.942+0000 7fc8fd000700 1 -- 172.21.15.57:0/3766320623 wait complete. 2024-02-28T23:07:23.941 INFO:tasks.workunit.client.1.smithi084.stdout:3/12: dread f1 [0,4194304] 0 2024-02-28T23:07:23.942 INFO:tasks.workunit.client.1.smithi084.stdout:3/13: write d3/f6 [828935,34803] 0 2024-02-28T23:07:23.943 INFO:tasks.workunit.client.1.smithi084.stdout:9/37: mknod d4/cb 0 2024-02-28T23:07:23.943 INFO:tasks.workunit.client.1.smithi084.stdout:5/54: write fb [834788,1572] 0 2024-02-28T23:07:23.945 INFO:tasks.workunit.client.1.smithi084.stdout:3/14: fsync d3/f4 0 2024-02-28T23:07:23.946 INFO:tasks.workunit.client.1.smithi084.stdout:5/55: chown f7 2851038 1 2024-02-28T23:07:23.947 INFO:tasks.workunit.client.1.smithi084.stdout:5/56: chown c3 5050392 1 2024-02-28T23:07:23.949 INFO:tasks.workunit.client.1.smithi084.stdout:4/30: creat d1/f8 x:0 0 0 2024-02-28T23:07:23.951 INFO:tasks.workunit.client.1.smithi084.stdout:9/38: write f0 [1925730,100310] 0 2024-02-28T23:07:23.951 INFO:tasks.workunit.client.1.smithi084.stdout:0/12: dread f0 [0,4194304] 0 2024-02-28T23:07:23.952 INFO:tasks.workunit.client.1.smithi084.stdout:5/57: mkdir dc/de/df 0 2024-02-28T23:07:23.953 INFO:tasks.workunit.client.1.smithi084.stdout:0/13: write f0 [3343832,64806] 0 2024-02-28T23:07:23.954 INFO:tasks.workunit.client.1.smithi084.stdout:5/58: write dc/fd [4281383,72843] 0 2024-02-28T23:07:23.956 INFO:tasks.workunit.client.1.smithi084.stdout:5/59: read fb [3684800,114846] 0 2024-02-28T23:07:23.961 INFO:tasks.workunit.client.1.smithi084.stdout:5/60: rename dc/de/df to dc/de/df/d10 22 2024-02-28T23:07:23.961 INFO:tasks.workunit.client.1.smithi084.stdout:4/31: link d1/f5 d1/f9 0 2024-02-28T23:07:23.962 INFO:tasks.workunit.client.1.smithi084.stdout:0/14: rename d1/l3 to d1/l4 0 2024-02-28T23:07:23.966 INFO:tasks.workunit.client.1.smithi084.stdout:5/61: chown l8 3 1 2024-02-28T23:07:23.972 INFO:tasks.workunit.client.1.smithi084.stdout:3/15: dwrite d3/f4 [0,4194304] 0 2024-02-28T23:07:23.979 INFO:tasks.workunit.client.1.smithi084.stdout:3/16: write f1 [684955,111489] 0 2024-02-28T23:07:23.979 INFO:tasks.workunit.client.1.smithi084.stdout:9/39: dwrite f0 [4194304,4194304] 0 2024-02-28T23:07:23.983 INFO:tasks.workunit.client.1.smithi084.stdout:9/40: rmdir d4 39 2024-02-28T23:07:23.994 INFO:tasks.workunit.client.1.smithi084.stdout:0/15: dwrite f0 [0,4194304] 0 2024-02-28T23:07:23.997 INFO:tasks.workunit.client.1.smithi084.stdout:0/16: mknod d1/c5 0 2024-02-28T23:07:23.999 INFO:tasks.workunit.client.1.smithi084.stdout:0/17: mkdir d1/d6 0 2024-02-28T23:07:24.015 INFO:tasks.workunit.client.1.smithi084.stdout:0/18: dwrite f0 [0,4194304] 0 2024-02-28T23:07:24.026 INFO:tasks.workunit.client.1.smithi084.stdout:2/21: sync 2024-02-28T23:07:24.029 INFO:tasks.workunit.client.1.smithi084.stdout:2/22: mkdir d5 0 2024-02-28T23:07:24.031 INFO:tasks.workunit.client.1.smithi084.stdout:2/23: chown f2 18 1 2024-02-28T23:07:24.043 INFO:tasks.workunit.client.1.smithi084.stdout:2/24: dread f1 [0,4194304] 0 2024-02-28T23:07:24.060 INFO:tasks.workunit.client.1.smithi084.stdout:2/25: dwrite f1 [0,4194304] 0 2024-02-28T23:07:24.064 INFO:tasks.workunit.client.1.smithi084.stdout:2/26: symlink d5/l6 0 2024-02-28T23:07:24.066 INFO:tasks.workunit.client.1.smithi084.stdout:2/27: mkdir d5/d7 0 2024-02-28T23:07:24.069 INFO:tasks.workunit.client.1.smithi084.stdout:2/28: rename f1 to d5/d7/f8 0 2024-02-28T23:07:24.091 INFO:tasks.workunit.client.1.smithi084.stdout:2/29: dwrite f2 [0,4194304] 0 2024-02-28T23:07:24.095 INFO:tasks.workunit.client.1.smithi084.stdout:2/30: write d5/d7/f8 [1885106,16682] 0 2024-02-28T23:07:24.097 INFO:tasks.workunit.client.1.smithi084.stdout:2/31: chown d5 21744 1 2024-02-28T23:07:24.112 INFO:tasks.workunit.client.1.smithi084.stdout:2/32: fdatasync f2 0 2024-02-28T23:07:24.112 INFO:tasks.workunit.client.1.smithi084.stdout:1/85: sync 2024-02-28T23:07:24.112 INFO:tasks.workunit.client.1.smithi084.stdout:4/32: sync 2024-02-28T23:07:24.117 INFO:tasks.workunit.client.1.smithi084.stdout:4/33: write d1/f5 [456960,19511] 0 2024-02-28T23:07:24.125 INFO:tasks.workunit.client.1.smithi084.stdout:2/33: dread f2 [0,4194304] 0 2024-02-28T23:07:24.127 INFO:tasks.workunit.client.1.smithi084.stdout:2/34: rmdir d5 39 2024-02-28T23:07:24.131 INFO:tasks.workunit.client.1.smithi084.stdout:9/41: getdents d4/d6 0 2024-02-28T23:07:24.133 INFO:tasks.workunit.client.1.smithi084.stdout:9/42: chown d4/cb 558207 1 2024-02-28T23:07:24.133 INFO:tasks.workunit.client.1.smithi084.stdout:8/25: stat d0/c7 0 2024-02-28T23:07:24.134 INFO:tasks.workunit.client.1.smithi084.stdout:9/43: write f0 [806027,79989] 0 2024-02-28T23:07:24.135 INFO:tasks.workunit.client.1.smithi084.stdout:7/74: write db/d11/f12 [1768855,122315] 0 2024-02-28T23:07:24.136 INFO:tasks.workunit.client.1.smithi084.stdout:8/26: mkdir d0/d9 0 2024-02-28T23:07:24.138 INFO:tasks.workunit.client.1.smithi084.stdout:9/44: creat d4/fc x:0 0 0 2024-02-28T23:07:24.140 INFO:tasks.workunit.client.1.smithi084.stdout:9/45: truncate d4/fc 113779 0 2024-02-28T23:07:24.141 INFO:tasks.workunit.client.1.smithi084.stdout:7/75: write db/ff [103492,112939] 0 2024-02-28T23:07:24.142 INFO:tasks.workunit.client.1.smithi084.stdout:9/46: write d4/d6/f7 [38460,6939] 0 2024-02-28T23:07:24.144 INFO:tasks.workunit.client.1.smithi084.stdout:9/47: write d4/d6/f9 [725579,4992] 0 2024-02-28T23:07:24.144 INFO:tasks.workunit.client.1.smithi084.stdout:7/76: fdatasync db/ff 0 2024-02-28T23:07:24.146 INFO:tasks.workunit.client.1.smithi084.stdout:8/27: symlink d0/la 0 2024-02-28T23:07:24.147 INFO:tasks.workunit.client.1.smithi084.stdout:4/34: truncate d1/f4 2374522 0 2024-02-28T23:07:24.148 INFO:tasks.workunit.client.1.smithi084.stdout:4/35: readlink l0 0 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 -- 172.21.15.57:0/696709848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50103680 msgr2=0x7fdf50105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 --2- 172.21.15.57:0/696709848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50103680 0x7fdf50105a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009910 tx=0x7fdf3c009c20 comp rx=0 tx=0).stop 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 -- 172.21.15.57:0/696709848 shutdown_connections 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 --2- 172.21.15.57:0/696709848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50103680 0x7fdf50105a60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 --2- 172.21.15.57:0/696709848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50100d60 0x7fdf50103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 -- 172.21.15.57:0/696709848 >> 172.21.15.57:0/696709848 conn(0x7fdf500fa780 msgr2=0x7fdf500fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.155+0000 7fdf57800700 1 -- 172.21.15.57:0/696709848 shutdown_connections 2024-02-28T23:07:24.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.156+0000 7fdf57800700 1 -- 172.21.15.57:0/696709848 wait complete. 2024-02-28T23:07:24.155 INFO:tasks.workunit.client.1.smithi084.stdout:8/28: creat d0/d9/fb x:0 0 0 2024-02-28T23:07:24.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 Processor -- start 2024-02-28T23:07:24.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 -- start start 2024-02-28T23:07:24.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:24.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50103680 0x7fdf50198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:24.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50198850 con 0x7fdf50100d60 2024-02-28T23:07:24.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf57800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5019d260 con 0x7fdf50103680 2024-02-28T23:07:24.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf54e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:24.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf54e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53114/0 (socket says 172.21.15.57:53114) 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.157+0000 7fdf54e00700 1 -- 172.21.15.57:0/803254942 learned_addr learned my addr 172.21.15.57:0/803254942 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.158+0000 7fdf4fe00700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50103680 0x7fdf50198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.158+0000 7fdf54e00700 1 -- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50103680 msgr2=0x7fdf50198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.158+0000 7fdf54e00700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50103680 0x7fdf50198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.158+0000 7fdf54e00700 1 -- 172.21.15.57:0/803254942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0095f0 con 0x7fdf50100d60 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.158+0000 7fdf54e00700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e8f0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.159+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf44009790 con 0x7fdf50100d60 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.159+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf44004ce0 con 0x7fdf50100d60 2024-02-28T23:07:24.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.159+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf44005690 con 0x7fdf50100d60 2024-02-28T23:07:24.158 INFO:tasks.workunit.client.1.smithi084.stdout:9/48: dread f0 [4194304,4194304] 0 2024-02-28T23:07:24.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.159+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5019d460 con 0x7fdf50100d60 2024-02-28T23:07:24.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.159+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5019d930 con 0x7fdf50100d60 2024-02-28T23:07:24.159 INFO:tasks.workunit.client.1.smithi084.stdout:7/77: dread db/fd [0,4194304] 0 2024-02-28T23:07:24.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.161+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf500fc340 con 0x7fdf50100d60 2024-02-28T23:07:24.160 INFO:tasks.workunit.client.1.smithi084.stdout:7/78: stat la 0 2024-02-28T23:07:24.161 INFO:tasks.workunit.client.1.smithi084.stdout:9/49: mkdir d4/dd 0 2024-02-28T23:07:24.162 INFO:tasks.workunit.client.1.smithi084.stdout:6/58: truncate d1/fe 3240644 0 2024-02-28T23:07:24.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.164+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7fdf44004880 con 0x7fdf50100d60 2024-02-28T23:07:24.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.165+0000 7fdf4d600700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 0x7fdf3806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:24.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.166+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdf44014070 con 0x7fdf50100d60 2024-02-28T23:07:24.164 INFO:tasks.workunit.client.1.smithi084.stdout:7/79: mknod db/d11/c14 0 2024-02-28T23:07:24.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.166+0000 7fdf4fe00700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 0x7fdf3806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:24.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.167+0000 7fdf4fe00700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 0x7fdf3806f3b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009fd0 tx=0x7fdf3c005fd0 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:24.165 INFO:tasks.workunit.client.1.smithi084.stdout:5/62: getdents dc/de 0 2024-02-28T23:07:24.166 INFO:tasks.workunit.client.1.smithi084.stdout:5/63: stat l8 0 2024-02-28T23:07:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.168+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fdf44056ea0 con 0x7fdf50100d60 2024-02-28T23:07:24.166 INFO:tasks.workunit.client.1.smithi084.stdout:9/50: symlink d4/d6/le 0 2024-02-28T23:07:24.168 INFO:tasks.workunit.client.1.smithi084.stdout:8/29: link d0/la d0/d9/lc 0 2024-02-28T23:07:24.169 INFO:tasks.workunit.client.1.smithi084.stdout:8/30: dread - d0/f8 zero size 2024-02-28T23:07:24.170 INFO:tasks.workunit.client.1.smithi084.stdout:7/80: creat db/d11/f15 x:0 0 0 2024-02-28T23:07:24.171 INFO:tasks.workunit.client.1.smithi084.stdout:5/64: symlink dc/de/l11 0 2024-02-28T23:07:24.173 INFO:tasks.workunit.client.1.smithi084.stdout:4/36: getdents d1 0 2024-02-28T23:07:24.174 INFO:tasks.workunit.client.1.smithi084.stdout:4/37: dread - d1/f7 zero size 2024-02-28T23:07:24.176 INFO:tasks.workunit.client.1.smithi084.stdout:6/59: unlink d1/d9/la 0 2024-02-28T23:07:24.178 INFO:tasks.workunit.client.1.smithi084.stdout:7/81: creat db/d11/f16 x:0 0 0 2024-02-28T23:07:24.179 INFO:tasks.workunit.client.1.smithi084.stdout:5/65: truncate fb 4781819 0 2024-02-28T23:07:24.180 INFO:tasks.workunit.client.1.smithi084.stdout:3/17: truncate f1 2746067 0 2024-02-28T23:07:24.184 INFO:tasks.workunit.client.1.smithi084.stdout:9/51: dwrite d4/d6/f9 [0,4194304] 0 2024-02-28T23:07:24.185 INFO:tasks.workunit.client.1.smithi084.stdout:0/19: truncate f0 2590957 0 2024-02-28T23:07:24.186 INFO:tasks.workunit.client.1.smithi084.stdout:4/38: chown d1/f4 3 1 2024-02-28T23:07:24.187 INFO:tasks.workunit.client.1.smithi084.stdout:9/52: chown d4/la 7336162 1 2024-02-28T23:07:24.187 INFO:tasks.workunit.client.1.smithi084.stdout:2/35: readlink d5/l6 0 2024-02-28T23:07:24.188 INFO:tasks.workunit.client.1.smithi084.stdout:4/39: chown l0 1227 1 2024-02-28T23:07:24.189 INFO:tasks.workunit.client.1.smithi084.stdout:5/66: creat dc/de/f12 x:0 0 0 2024-02-28T23:07:24.191 INFO:tasks.workunit.client.1.smithi084.stdout:3/18: rename d3/f4 to d3/f7 0 2024-02-28T23:07:24.191 INFO:tasks.workunit.client.1.smithi084.stdout:2/36: write f4 [1273999,3714] 0 2024-02-28T23:07:24.193 INFO:tasks.workunit.client.1.smithi084.stdout:2/37: chown f2 5918725 1 2024-02-28T23:07:24.196 INFO:tasks.workunit.client.1.smithi084.stdout:3/19: truncate d3/f7 4921817 0 2024-02-28T23:07:24.198 INFO:tasks.workunit.client.1.smithi084.stdout:2/38: write f4 [933824,96779] 0 2024-02-28T23:07:24.199 INFO:tasks.workunit.client.1.smithi084.stdout:7/82: dread f9 [0,4194304] 0 2024-02-28T23:07:24.202 INFO:tasks.workunit.client.1.smithi084.stdout:4/40: write d1/f9 [248423,104640] 0 2024-02-28T23:07:24.203 INFO:tasks.workunit.client.1.smithi084.stdout:2/39: chown d5/d7/f8 262727731 1 2024-02-28T23:07:24.209 INFO:tasks.workunit.client.1.smithi084.stdout:4/41: dread - d1/f6 zero size 2024-02-28T23:07:24.209 INFO:tasks.workunit.client.1.smithi084.stdout:5/67: rename dc/fd to dc/de/f13 0 2024-02-28T23:07:24.214 INFO:tasks.workunit.client.1.smithi084.stdout:9/53: dwrite d4/d6/f7 [0,4194304] 0 2024-02-28T23:07:24.222 INFO:tasks.workunit.client.1.smithi084.stdout:6/60: truncate d1/fe 3797115 0 2024-02-28T23:07:24.222 INFO:tasks.workunit.client.1.smithi084.stdout:8/31: fsync d0/d9/fb 0 2024-02-28T23:07:24.229 INFO:tasks.workunit.client.1.smithi084.stdout:7/83: dread db/fd [0,4194304] 0 2024-02-28T23:07:24.229 INFO:tasks.workunit.client.1.smithi084.stdout:8/32: dread - d0/f8 zero size 2024-02-28T23:07:24.230 INFO:tasks.workunit.client.1.smithi084.stdout:8/33: chown d0 108626 1 2024-02-28T23:07:24.232 INFO:tasks.workunit.client.1.smithi084.stdout:6/61: rename d1/c7 to d1/cf 0 2024-02-28T23:07:24.234 INFO:tasks.workunit.client.1.smithi084.stdout:8/34: stat d0/l5 0 2024-02-28T23:07:24.235 INFO:tasks.workunit.client.1.smithi084.stdout:1/86: dwrite f1 [0,4194304] 0 2024-02-28T23:07:24.237 INFO:tasks.workunit.client.1.smithi084.stdout:2/40: getdents d5/d7 0 2024-02-28T23:07:24.245 INFO:tasks.workunit.client.1.smithi084.stdout:6/62: symlink d1/l10 0 2024-02-28T23:07:24.246 INFO:tasks.workunit.client.1.smithi084.stdout:1/87: chown d12/f15 1036630960 1 2024-02-28T23:07:24.250 INFO:tasks.workunit.client.1.smithi084.stdout:8/35: mkdir d0/d9/dd 0 2024-02-28T23:07:24.255 INFO:tasks.workunit.client.1.smithi084.stdout:9/54: dwrite d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:24.259 INFO:tasks.workunit.client.1.smithi084.stdout:1/88: mkdir d12/d1b 0 2024-02-28T23:07:24.262 INFO:tasks.workunit.client.1.smithi084.stdout:1/89: stat f9 0 2024-02-28T23:07:24.265 INFO:tasks.workunit.client.1.smithi084.stdout:5/68: dwrite f7 [4194304,4194304] 0 2024-02-28T23:07:24.265 INFO:tasks.workunit.client.1.smithi084.stdout:3/20: dread f1 [0,4194304] 0 2024-02-28T23:07:24.266 INFO:tasks.workunit.client.1.smithi084.stdout:8/36: creat d0/fe x:0 0 0 2024-02-28T23:07:24.267 INFO:tasks.workunit.client.1.smithi084.stdout:7/84: truncate f9 3370902 0 2024-02-28T23:07:24.268 INFO:tasks.workunit.client.1.smithi084.stdout:1/90: dread - d12/f13 zero size 2024-02-28T23:07:24.271 INFO:tasks.workunit.client.1.smithi084.stdout:9/55: unlink c2 0 2024-02-28T23:07:24.273 INFO:tasks.workunit.client.1.smithi084.stdout:0/20: dwrite f0 [0,4194304] 0 2024-02-28T23:07:24.274 INFO:tasks.workunit.client.1.smithi084.stdout:2/41: truncate f2 1673118 0 2024-02-28T23:07:24.275 INFO:tasks.workunit.client.1.smithi084.stdout:3/21: mkdir d3/d8 0 2024-02-28T23:07:24.276 INFO:tasks.workunit.client.1.smithi084.stdout:7/85: mkdir db/d10/d17 0 2024-02-28T23:07:24.276 INFO:tasks.workunit.client.1.smithi084.stdout:5/69: creat dc/de/f14 x:0 0 0 2024-02-28T23:07:24.279 INFO:tasks.workunit.client.1.smithi084.stdout:8/37: truncate d0/f8 646267 0 2024-02-28T23:07:24.281 INFO:tasks.workunit.client.1.smithi084.stdout:7/86: write db/ff [487371,81859] 0 2024-02-28T23:07:24.281 INFO:tasks.workunit.client.1.smithi084.stdout:1/91: link fd d12/d18/f1c 0 2024-02-28T23:07:24.282 INFO:tasks.workunit.client.1.smithi084.stdout:9/56: creat d4/d6/ff x:0 0 0 2024-02-28T23:07:24.285 INFO:tasks.workunit.client.1.smithi084.stdout:8/38: write d0/f8 [73989,113722] 0 2024-02-28T23:07:24.285 INFO:tasks.workunit.client.1.smithi084.stdout:7/87: write db/ff [1392193,42735] 0 2024-02-28T23:07:24.286 INFO:tasks.workunit.client.1.smithi084.stdout:2/42: mkdir d5/d7/d9 0 2024-02-28T23:07:24.293 INFO:tasks.workunit.client.1.smithi084.stdout:5/70: creat dc/f15 x:0 0 0 2024-02-28T23:07:24.296 INFO:tasks.workunit.client.1.smithi084.stdout:1/92: creat d12/d18/f1d x:0 0 0 2024-02-28T23:07:24.296 INFO:tasks.workunit.client.1.smithi084.stdout:7/88: stat db/fd 0 2024-02-28T23:07:24.298 INFO:tasks.workunit.client.1.smithi084.stdout:2/43: mknod d5/d7/ca 0 2024-02-28T23:07:24.300 INFO:tasks.workunit.client.1.smithi084.stdout:5/71: chown l8 1497187478 1 2024-02-28T23:07:24.302 INFO:tasks.workunit.client.1.smithi084.stdout:8/39: readlink d0/la 0 2024-02-28T23:07:24.304 INFO:tasks.workunit.client.1.smithi084.stdout:2/44: write f4 [591118,17751] 0 2024-02-28T23:07:24.306 INFO:tasks.workunit.client.1.smithi084.stdout:5/72: dread - dc/f15 zero size 2024-02-28T23:07:24.307 INFO:tasks.workunit.client.1.smithi084.stdout:2/45: chown d5 283608 1 2024-02-28T23:07:24.310 INFO:tasks.workunit.client.1.smithi084.stdout:7/89: unlink db/cc 0 2024-02-28T23:07:24.312 INFO:tasks.workunit.client.1.smithi084.stdout:8/40: mknod d0/cf 0 2024-02-28T23:07:24.312 INFO:tasks.workunit.client.1.smithi084.stdout:3/22: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:24.313 INFO:tasks.workunit.client.1.smithi084.stdout:2/46: chown d5/l6 4009675 1 2024-02-28T23:07:24.314 INFO:tasks.workunit.client.1.smithi084.stdout:9/57: link d4/d6/le d4/l10 0 2024-02-28T23:07:24.315 INFO:tasks.workunit.client.1.smithi084.stdout:0/21: dwrite f0 [4194304,4194304] 0 2024-02-28T23:07:24.316 INFO:tasks.workunit.client.1.smithi084.stdout:7/90: write db/d11/f16 [973831,69964] 0 2024-02-28T23:07:24.319 INFO:tasks.workunit.client.1.smithi084.stdout:9/58: write d4/d6/f9 [3323060,26115] 0 2024-02-28T23:07:24.320 INFO:tasks.workunit.client.1.smithi084.stdout:2/47: read f2 [917735,17417] 0 2024-02-28T23:07:24.324 INFO:tasks.workunit.client.1.smithi084.stdout:1/93: dread f9 [0,4194304] 0 2024-02-28T23:07:24.324 INFO:tasks.workunit.client.1.smithi084.stdout:3/23: symlink d3/d8/l9 0 2024-02-28T23:07:24.325 INFO:tasks.workunit.client.1.smithi084.stdout:5/73: mkdir dc/de/df/d16 0 2024-02-28T23:07:24.325 INFO:tasks.workunit.client.1.smithi084.stdout:9/59: truncate d4/d6/f7 4670078 0 2024-02-28T23:07:24.326 INFO:tasks.workunit.client.1.smithi084.stdout:1/94: stat d12/f15 0 2024-02-28T23:07:24.329 INFO:tasks.workunit.client.1.smithi084.stdout:8/41: creat d0/d9/dd/f10 x:0 0 0 2024-02-28T23:07:24.332 INFO:tasks.workunit.client.1.smithi084.stdout:1/95: write f11 [377450,62158] 0 2024-02-28T23:07:24.335 INFO:tasks.workunit.client.1.smithi084.stdout:3/24: fdatasync d3/f7 0 2024-02-28T23:07:24.338 INFO:tasks.workunit.client.1.smithi084.stdout:8/42: chown d0/c7 1 1 2024-02-28T23:07:24.341 INFO:tasks.workunit.client.1.smithi084.stdout:5/74: mknod dc/de/df/d16/c17 0 2024-02-28T23:07:24.342 INFO:tasks.workunit.client.1.smithi084.stdout:0/22: link d1/l4 d1/l7 0 2024-02-28T23:07:24.345 INFO:tasks.workunit.client.1.smithi084.stdout:1/96: rename c3 to d12/d18/c1e 0 2024-02-28T23:07:24.351 INFO:tasks.workunit.client.1.smithi084.stdout:3/25: creat d3/d8/fa x:0 0 0 2024-02-28T23:07:24.352 INFO:tasks.workunit.client.1.smithi084.stdout:5/75: symlink dc/de/df/l18 0 2024-02-28T23:07:24.353 INFO:tasks.workunit.client.1.smithi084.stdout:8/43: write d0/d9/fb [333900,113952] 0 2024-02-28T23:07:24.356 INFO:tasks.workunit.client.1.smithi084.stdout:1/97: unlink d12/c1a 0 2024-02-28T23:07:24.357 INFO:tasks.workunit.client.1.smithi084.stdout:0/23: mkdir d1/d8 0 2024-02-28T23:07:24.358 INFO:tasks.workunit.client.1.smithi084.stdout:1/98: stat lb 0 2024-02-28T23:07:24.360 INFO:tasks.workunit.client.1.smithi084.stdout:5/76: readlink dc/de/l11 0 2024-02-28T23:07:24.361 INFO:tasks.workunit.client.1.smithi084.stdout:9/60: truncate d4/d6/f7 2276658 0 2024-02-28T23:07:24.361 INFO:tasks.workunit.client.1.smithi084.stdout:8/44: chown d0/l5 793743 1 2024-02-28T23:07:24.361 INFO:tasks.workunit.client.1.smithi084.stdout:3/26: dread - d3/d8/fa zero size 2024-02-28T23:07:24.363 INFO:tasks.workunit.client.1.smithi084.stdout:7/91: dwrite db/d11/f16 [0,4194304] 0 2024-02-28T23:07:24.364 INFO:tasks.workunit.client.1.smithi084.stdout:6/63: dwrite d1/fe [4194304,4194304] 0 2024-02-28T23:07:24.365 INFO:tasks.workunit.client.1.smithi084.stdout:1/99: write fc [188867,129963] 0 2024-02-28T23:07:24.365 INFO:tasks.workunit.client.1.smithi084.stdout:8/45: stat d0/la 0 2024-02-28T23:07:24.371 INFO:tasks.workunit.client.1.smithi084.stdout:8/46: readlink d0/la 0 2024-02-28T23:07:24.373 INFO:tasks.workunit.client.1.smithi084.stdout:9/61: write d4/d6/f9 [2300419,81078] 0 2024-02-28T23:07:24.374 INFO:tasks.workunit.client.1.smithi084.stdout:1/100: fsync f9 0 2024-02-28T23:07:24.377 INFO:tasks.workunit.client.1.smithi084.stdout:8/47: mkdir d0/d11 0 2024-02-28T23:07:24.378 INFO:tasks.workunit.client.1.smithi084.stdout:3/27: write d3/d8/fa [413963,79328] 0 2024-02-28T23:07:24.380 INFO:tasks.workunit.client.1.smithi084.stdout:0/24: link f0 d1/d8/f9 0 2024-02-28T23:07:24.380 INFO:tasks.workunit.client.1.smithi084.stdout:9/62: mknod d4/c11 0 2024-02-28T23:07:24.383 INFO:tasks.workunit.client.1.smithi084.stdout:4/42: sync 2024-02-28T23:07:24.388 INFO:tasks.workunit.client.1.smithi084.stdout:6/64: getdents d1 0 2024-02-28T23:07:24.389 INFO:tasks.workunit.client.1.smithi084.stdout:2/48: dwrite f2 [0,4194304] 0 2024-02-28T23:07:24.391 INFO:tasks.workunit.client.1.smithi084.stdout:9/63: write d4/d6/f9 [1536385,34501] 0 2024-02-28T23:07:24.392 INFO:tasks.workunit.client.1.smithi084.stdout:3/28: readlink d3/l5 0 2024-02-28T23:07:24.392 INFO:tasks.workunit.client.1.smithi084.stdout:2/49: fsync f4 0 2024-02-28T23:07:24.394 INFO:tasks.workunit.client.1.smithi084.stdout:8/48: dread d0/d9/fb [0,4194304] 0 2024-02-28T23:07:24.394 INFO:tasks.workunit.client.1.smithi084.stdout:4/43: mknod d1/ca 0 2024-02-28T23:07:24.395 INFO:tasks.workunit.client.1.smithi084.stdout:1/101: truncate fd 2658906 0 2024-02-28T23:07:24.396 INFO:tasks.workunit.client.1.smithi084.stdout:0/25: stat d1/l7 0 2024-02-28T23:07:24.396 INFO:tasks.workunit.client.1.smithi084.stdout:6/65: creat d1/f11 x:0 0 0 2024-02-28T23:07:24.399 INFO:tasks.workunit.client.1.smithi084.stdout:8/49: stat d0/la 0 2024-02-28T23:07:24.400 INFO:tasks.workunit.client.1.smithi084.stdout:4/44: chown d1/f6 3499 1 2024-02-28T23:07:24.405 INFO:tasks.workunit.client.1.smithi084.stdout:6/66: symlink d1/l12 0 2024-02-28T23:07:24.405 INFO:tasks.workunit.client.1.smithi084.stdout:4/45: truncate d1/f6 113043 0 2024-02-28T23:07:24.406 INFO:tasks.workunit.client.1.smithi084.stdout:4/46: chown d1 0 1 2024-02-28T23:07:24.406 INFO:tasks.workunit.client.1.smithi084.stdout:5/77: dread fb [0,4194304] 0 2024-02-28T23:07:24.408 INFO:tasks.workunit.client.1.smithi084.stdout:1/102: symlink d12/l1f 0 2024-02-28T23:07:24.411 INFO:tasks.workunit.client.1.smithi084.stdout:8/50: creat d0/f12 x:0 0 0 2024-02-28T23:07:24.411 INFO:tasks.workunit.client.1.smithi084.stdout:5/78: dread - dc/f15 zero size 2024-02-28T23:07:24.413 INFO:tasks.workunit.client.1.smithi084.stdout:1/103: write fc [408701,21780] 0 2024-02-28T23:07:24.416 INFO:tasks.workunit.client.1.smithi084.stdout:7/92: truncate db/d11/f12 298521 0 2024-02-28T23:07:24.418 INFO:tasks.workunit.client.1.smithi084.stdout:4/47: symlink d1/lb 0 2024-02-28T23:07:24.419 INFO:tasks.workunit.client.1.smithi084.stdout:6/67: unlink d1/f11 0 2024-02-28T23:07:24.423 INFO:tasks.workunit.client.1.smithi084.stdout:8/51: creat d0/d9/f13 x:0 0 0 2024-02-28T23:07:24.423 INFO:tasks.workunit.client.1.smithi084.stdout:6/68: fdatasync d1/fe 0 2024-02-28T23:07:24.424 INFO:tasks.workunit.client.1.smithi084.stdout:5/79: rename l5 to dc/de/df/l19 0 2024-02-28T23:07:24.425 INFO:tasks.workunit.client.1.smithi084.stdout:4/48: dread - d1/f7 zero size 2024-02-28T23:07:24.427 INFO:tasks.workunit.client.1.smithi084.stdout:7/93: mkdir db/d10/d18 0 2024-02-28T23:07:24.432 INFO:tasks.workunit.client.1.smithi084.stdout:4/49: rename d1 to d1/dc 22 2024-02-28T23:07:24.434 INFO:tasks.workunit.client.1.smithi084.stdout:0/26: dread f0 [4194304,4194304] 0 2024-02-28T23:07:24.434 INFO:tasks.workunit.client.1.smithi084.stdout:9/64: dread d4/d6/f9 [0,4194304] 0 2024-02-28T23:07:24.435 INFO:tasks.workunit.client.1.smithi084.stdout:8/52: rmdir d0/d9/dd 39 2024-02-28T23:07:24.436 INFO:tasks.workunit.client.1.smithi084.stdout:3/29: dwrite d3/f6 [0,4194304] 0 2024-02-28T23:07:24.437 INFO:tasks.workunit.client.1.smithi084.stdout:2/50: dwrite f3 [0,4194304] 0 2024-02-28T23:07:24.443 INFO:tasks.workunit.client.1.smithi084.stdout:6/69: mkdir d1/d13 0 2024-02-28T23:07:24.444 INFO:tasks.workunit.client.1.smithi084.stdout:3/30: chown d3/d8/fa 54025965 1 2024-02-28T23:07:24.450 INFO:tasks.workunit.client.1.smithi084.stdout:1/104: dread f1 [0,4194304] 0 2024-02-28T23:07:24.462 INFO:tasks.workunit.client.1.smithi084.stdout:3/31: creat d3/fb x:0 0 0 2024-02-28T23:07:24.473 INFO:tasks.workunit.client.1.smithi084.stdout:1/105: chown d12/d18/c1e 722738 1 2024-02-28T23:07:24.473 INFO:tasks.workunit.client.1.smithi084.stdout:0/27: write d1/d8/f9 [6450757,3182] 0 2024-02-28T23:07:24.474 INFO:tasks.workunit.client.1.smithi084.stdout:3/32: creat d3/fc x:0 0 0 2024-02-28T23:07:24.477 INFO:tasks.workunit.client.1.smithi084.stdout:8/53: unlink d0/d9/dd/f10 0 2024-02-28T23:07:24.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.479+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf50061190 con 0x7fdf3806cf00 2024-02-28T23:07:24.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:24 smithi057 ceph-mon[29096]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:24.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:24 smithi057 ceph-mon[29096]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:24.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:24 smithi057 ceph-mon[29096]: pgmap v18: 65 pgs: 65 active+clean; 215 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 283 op/s 2024-02-28T23:07:24.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:24 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:24.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:24 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3766320623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:07:24.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.481+0000 7fdf4d600700 1 -- 172.21.15.57:0/803254942 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdf50061190 con 0x7fdf3806cf00 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "1/2 daemons upgraded", 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:07:24.480 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:07:24.481 INFO:tasks.workunit.client.1.smithi084.stdout:2/51: dwrite f4 [0,4194304] 0 2024-02-28T23:07:24.481 INFO:tasks.workunit.client.1.smithi084.stdout:6/70: dread d1/fe [4194304,4194304] 0 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.484+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 msgr2=0x7fdf3806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.484+0000 7fdf57800700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 0x7fdf3806f3b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009fd0 tx=0x7fdf3c005fd0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.484+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 msgr2=0x7fdf50197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.484+0000 7fdf57800700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e8f0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 shutdown_connections 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fdf3806cf00 0x7fdf3806f3b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf50100d60 0x7fdf50197cf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 --2- 172.21.15.57:0/803254942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf50103680 0x7fdf50198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 >> 172.21.15.57:0/803254942 conn(0x7fdf500fa780 msgr2=0x7fdf500ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:24.484 INFO:tasks.workunit.client.1.smithi084.stdout:4/50: dwrite d1/f8 [0,4194304] 0 2024-02-28T23:07:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.485+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 shutdown_connections 2024-02-28T23:07:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:24.487+0000 7fdf57800700 1 -- 172.21.15.57:0/803254942 wait complete. 2024-02-28T23:07:24.487 INFO:tasks.workunit.client.1.smithi084.stdout:8/54: write d0/f12 [465427,118563] 0 2024-02-28T23:07:24.489 INFO:tasks.workunit.client.1.smithi084.stdout:5/80: dwrite fb [0,4194304] 0 2024-02-28T23:07:24.489 INFO:tasks.workunit.client.1.smithi084.stdout:2/52: fdatasync f2 0 2024-02-28T23:07:24.489 INFO:tasks.workunit.client.1.smithi084.stdout:3/33: creat d3/d8/fd x:0 0 0 2024-02-28T23:07:24.489 INFO:tasks.workunit.client.1.smithi084.stdout:9/65: getdents d4/d6 0 2024-02-28T23:07:24.497 INFO:tasks.workunit.client.1.smithi084.stdout:0/28: write d1/d8/f9 [6410056,34907] 0 2024-02-28T23:07:24.500 INFO:tasks.workunit.client.1.smithi084.stdout:2/53: truncate f2 4207787 0 2024-02-28T23:07:24.501 INFO:tasks.workunit.client.1.smithi084.stdout:3/34: write d3/fb [267003,112630] 0 2024-02-28T23:07:24.501 INFO:tasks.workunit.client.1.smithi084.stdout:8/55: chown d0/d9/fb 466021 1 2024-02-28T23:07:24.501 INFO:tasks.workunit.client.1.smithi084.stdout:4/51: mkdir d1/dd 0 2024-02-28T23:07:24.507 INFO:tasks.workunit.client.1.smithi084.stdout:5/81: creat dc/de/f1a x:0 0 0 2024-02-28T23:07:24.514 INFO:tasks.workunit.client.1.smithi084.stdout:9/66: mkdir d4/dd/d12 0 2024-02-28T23:07:24.516 INFO:tasks.workunit.client.1.smithi084.stdout:0/29: read d1/d8/f9 [5368097,121547] 0 2024-02-28T23:07:24.520 INFO:tasks.workunit.client.1.smithi084.stdout:1/106: dwrite f11 [0,4194304] 0 2024-02-28T23:07:24.520 INFO:tasks.workunit.client.1.smithi084.stdout:4/52: write d1/f9 [408810,73097] 0 2024-02-28T23:07:24.520 INFO:tasks.workunit.client.1.smithi084.stdout:3/35: creat d3/fe x:0 0 0 2024-02-28T23:07:24.521 INFO:tasks.workunit.client.1.smithi084.stdout:7/94: dwrite db/d11/f12 [0,4194304] 0 2024-02-28T23:07:24.522 INFO:tasks.workunit.client.1.smithi084.stdout:5/82: write dc/de/f1a [284917,4374] 0 2024-02-28T23:07:24.523 INFO:tasks.workunit.client.1.smithi084.stdout:8/56: write d0/d9/f13 [917497,116343] 0 2024-02-28T23:07:24.525 INFO:tasks.workunit.client.1.smithi084.stdout:2/54: rmdir d5/d7 39 2024-02-28T23:07:24.529 INFO:tasks.workunit.client.1.smithi084.stdout:0/30: symlink d1/la 0 2024-02-28T23:07:24.531 INFO:tasks.workunit.client.1.smithi084.stdout:5/83: dread - dc/de/f12 zero size 2024-02-28T23:07:24.533 INFO:tasks.workunit.client.1.smithi084.stdout:8/57: write d0/fe [494820,79554] 0 2024-02-28T23:07:24.534 INFO:tasks.workunit.client.1.smithi084.stdout:1/107: symlink d12/d1b/l20 0 2024-02-28T23:07:24.535 INFO:tasks.workunit.client.1.smithi084.stdout:6/71: dwrite d1/fe [0,4194304] 0 2024-02-28T23:07:24.537 INFO:tasks.workunit.client.1.smithi084.stdout:9/67: truncate d4/fc 1037680 0 2024-02-28T23:07:24.538 INFO:tasks.workunit.client.1.smithi084.stdout:4/53: creat d1/dd/fe x:0 0 0 2024-02-28T23:07:24.541 INFO:tasks.workunit.client.1.smithi084.stdout:0/31: unlink d1/d8/f9 0 2024-02-28T23:07:24.541 INFO:tasks.workunit.client.1.smithi084.stdout:8/58: chown d0/d9/lc 121960734 1 2024-02-28T23:07:24.547 INFO:tasks.workunit.client.1.smithi084.stdout:6/72: mkdir d1/d9/d14 0 2024-02-28T23:07:24.552 INFO:tasks.workunit.client.1.smithi084.stdout:6/73: stat d1/cc 0 2024-02-28T23:07:24.553 INFO:tasks.workunit.client.1.smithi084.stdout:8/59: symlink d0/d9/dd/l14 0 2024-02-28T23:07:24.559 INFO:tasks.workunit.client.1.smithi084.stdout:5/84: link fb dc/de/df/d16/f1b 0 2024-02-28T23:07:24.563 INFO:tasks.workunit.client.1.smithi084.stdout:3/36: dwrite d3/f6 [0,4194304] 0 2024-02-28T23:07:24.565 INFO:tasks.workunit.client.1.smithi084.stdout:6/74: unlink d1/l10 0 2024-02-28T23:07:24.568 INFO:tasks.workunit.client.1.smithi084.stdout:9/68: dread d4/d6/f7 [0,4194304] 0 2024-02-28T23:07:24.572 INFO:tasks.workunit.client.1.smithi084.stdout:5/85: unlink dc/de/df/l18 0 2024-02-28T23:07:24.574 INFO:tasks.workunit.client.1.smithi084.stdout:7/95: dwrite db/d11/f15 [0,4194304] 0 2024-02-28T23:07:24.576 INFO:tasks.workunit.client.1.smithi084.stdout:2/55: dread f4 [0,4194304] 0 2024-02-28T23:07:24.578 INFO:tasks.workunit.client.1.smithi084.stdout:6/75: creat d1/d9/f15 x:0 0 0 2024-02-28T23:07:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:24 smithi084 ceph-mon[34826]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:24 smithi084 ceph-mon[34826]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:24 smithi084 ceph-mon[34826]: pgmap v18: 65 pgs: 65 active+clean; 215 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 283 op/s 2024-02-28T23:07:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:24 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:24 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3766320623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:07:24.582 INFO:tasks.workunit.client.1.smithi084.stdout:3/37: write d3/d8/fa [715453,45192] 0 2024-02-28T23:07:24.583 INFO:tasks.workunit.client.1.smithi084.stdout:8/60: rename d0/f12 to d0/d9/f15 0 2024-02-28T23:07:24.590 INFO:tasks.workunit.client.1.smithi084.stdout:1/108: dread f1 [0,4194304] 0 2024-02-28T23:07:24.597 INFO:tasks.workunit.client.1.smithi084.stdout:4/54: dwrite d1/f7 [0,4194304] 0 2024-02-28T23:07:24.598 INFO:tasks.workunit.client.1.smithi084.stdout:6/76: read d1/fe [4810462,41229] 0 2024-02-28T23:07:24.605 INFO:tasks.workunit.client.1.smithi084.stdout:0/32: dread f0 [0,4194304] 0 2024-02-28T23:07:24.612 INFO:tasks.workunit.client.1.smithi084.stdout:1/109: dread fc [0,4194304] 0 2024-02-28T23:07:24.616 INFO:tasks.workunit.client.1.smithi084.stdout:4/55: mknod d1/dd/cf 0 2024-02-28T23:07:24.617 INFO:tasks.workunit.client.1.smithi084.stdout:6/77: rename d1/cf to d1/c16 0 2024-02-28T23:07:24.620 INFO:tasks.workunit.client.1.smithi084.stdout:2/56: creat d5/fb x:0 0 0 2024-02-28T23:07:24.621 INFO:tasks.workunit.client.1.smithi084.stdout:8/61: dread d0/f8 [0,4194304] 0 2024-02-28T23:07:24.622 INFO:tasks.workunit.client.1.smithi084.stdout:5/86: dwrite dc/de/f13 [0,4194304] 0 2024-02-28T23:07:24.627 INFO:tasks.workunit.client.1.smithi084.stdout:2/57: chown f4 407239 1 2024-02-28T23:07:24.628 INFO:tasks.workunit.client.1.smithi084.stdout:0/33: mkdir d1/d6/db 0 2024-02-28T23:07:24.628 INFO:tasks.workunit.client.1.smithi084.stdout:9/69: dwrite d4/fc [0,4194304] 0 2024-02-28T23:07:24.628 INFO:tasks.workunit.client.1.smithi084.stdout:4/56: write d1/f5 [793324,72885] 0 2024-02-28T23:07:24.629 INFO:tasks.workunit.client.1.smithi084.stdout:1/110: write d12/f15 [4875510,91784] 0 2024-02-28T23:07:24.635 INFO:tasks.workunit.client.1.smithi084.stdout:6/78: symlink d1/d9/d14/l17 0 2024-02-28T23:07:24.636 INFO:tasks.workunit.client.1.smithi084.stdout:8/62: mknod d0/d9/c16 0 2024-02-28T23:07:24.637 INFO:tasks.workunit.client.1.smithi084.stdout:2/58: truncate d5/fb 334082 0 2024-02-28T23:07:24.638 INFO:tasks.workunit.client.1.smithi084.stdout:3/38: dwrite d3/fb [0,4194304] 0 2024-02-28T23:07:24.638 INFO:tasks.workunit.client.1.smithi084.stdout:4/57: fdatasync d1/dd/fe 0 2024-02-28T23:07:24.640 INFO:tasks.workunit.client.1.smithi084.stdout:0/34: rename f0 to d1/fc 0 2024-02-28T23:07:24.643 INFO:tasks.workunit.client.1.smithi084.stdout:8/63: chown d0/c4 17740859 1 2024-02-28T23:07:24.643 INFO:tasks.workunit.client.1.smithi084.stdout:6/79: dread - d1/d9/f15 zero size 2024-02-28T23:07:24.643 INFO:tasks.workunit.client.1.smithi084.stdout:9/70: fsync d4/d6/f7 0 2024-02-28T23:07:24.647 INFO:tasks.workunit.client.1.smithi084.stdout:0/35: rename d1/d6 to d1/d6/dd 22 2024-02-28T23:07:24.647 INFO:tasks.workunit.client.1.smithi084.stdout:1/111: mknod d12/d18/c21 0 2024-02-28T23:07:24.648 INFO:tasks.workunit.client.1.smithi084.stdout:8/64: chown d0/c4 0 1 2024-02-28T23:07:24.650 INFO:tasks.workunit.client.1.smithi084.stdout:6/80: chown d1/d9/d14/l17 1017856 1 2024-02-28T23:07:24.651 INFO:tasks.workunit.client.1.smithi084.stdout:7/96: dwrite f9 [0,4194304] 0 2024-02-28T23:07:24.653 INFO:tasks.workunit.client.1.smithi084.stdout:5/87: creat dc/de/df/d16/f1c x:0 0 0 2024-02-28T23:07:24.654 INFO:tasks.workunit.client.1.smithi084.stdout:3/39: chown d3/d8/fa 1863 1 2024-02-28T23:07:24.655 INFO:tasks.workunit.client.1.smithi084.stdout:4/58: dread d1/f5 [0,4194304] 0 2024-02-28T23:07:24.657 INFO:tasks.workunit.client.1.smithi084.stdout:9/71: creat d4/f13 x:0 0 0 2024-02-28T23:07:24.661 INFO:tasks.workunit.client.1.smithi084.stdout:0/36: mkdir d1/de 0 2024-02-28T23:07:24.663 INFO:tasks.workunit.client.1.smithi084.stdout:8/65: rename d0/d9/dd/l14 to d0/d9/dd/l17 0 2024-02-28T23:07:24.668 INFO:tasks.workunit.client.1.smithi084.stdout:6/81: fsync d1/d9/f15 0 2024-02-28T23:07:24.670 INFO:tasks.workunit.client.1.smithi084.stdout:9/72: rename d4/c11 to d4/d6/c14 0 2024-02-28T23:07:24.671 INFO:tasks.workunit.client.1.smithi084.stdout:7/97: dread db/ff [0,4194304] 0 2024-02-28T23:07:24.673 INFO:tasks.workunit.client.1.smithi084.stdout:1/112: link fd d12/f22 0 2024-02-28T23:07:24.677 INFO:tasks.workunit.client.1.smithi084.stdout:8/66: symlink d0/d11/l18 0 2024-02-28T23:07:24.678 INFO:tasks.workunit.client.1.smithi084.stdout:6/82: fsync d1/d9/f15 0 2024-02-28T23:07:24.679 INFO:tasks.workunit.client.1.smithi084.stdout:0/37: rmdir d1/de 0 2024-02-28T23:07:24.680 INFO:tasks.workunit.client.1.smithi084.stdout:2/59: dwrite f2 [0,4194304] 0 2024-02-28T23:07:24.682 INFO:tasks.workunit.client.1.smithi084.stdout:7/98: write db/d11/f15 [4429392,124099] 0 2024-02-28T23:07:24.683 INFO:tasks.workunit.client.1.smithi084.stdout:1/113: creat d12/f23 x:0 0 0 2024-02-28T23:07:24.689 INFO:tasks.workunit.client.1.smithi084.stdout:9/73: rename d4/dd to d4/d6/d15 0 2024-02-28T23:07:24.690 INFO:tasks.workunit.client.1.smithi084.stdout:8/67: mknod d0/c19 0 2024-02-28T23:07:24.690 INFO:tasks.workunit.client.1.smithi084.stdout:6/83: chown d1/fe 982535 1 2024-02-28T23:07:24.694 INFO:tasks.workunit.client.1.smithi084.stdout:0/38: unlink d1/l7 0 2024-02-28T23:07:24.700 INFO:tasks.workunit.client.1.smithi084.stdout:0/39: chown d1/la 17991684 1 2024-02-28T23:07:24.700 INFO:tasks.workunit.client.1.smithi084.stdout:1/114: mknod d12/d1b/c24 0 2024-02-28T23:07:24.703 INFO:tasks.workunit.client.1.smithi084.stdout:7/99: rmdir db/d10 39 2024-02-28T23:07:24.706 INFO:tasks.workunit.client.1.smithi084.stdout:9/74: rmdir d4 39 2024-02-28T23:07:24.706 INFO:tasks.workunit.client.1.smithi084.stdout:8/68: creat d0/f1a x:0 0 0 2024-02-28T23:07:24.707 INFO:tasks.workunit.client.1.smithi084.stdout:3/40: dwrite d3/fc [0,4194304] 0 2024-02-28T23:07:24.709 INFO:tasks.workunit.client.1.smithi084.stdout:6/84: creat d1/d9/f18 x:0 0 0 2024-02-28T23:07:24.710 INFO:tasks.workunit.client.1.smithi084.stdout:4/59: dwrite d1/f9 [0,4194304] 0 2024-02-28T23:07:24.710 INFO:tasks.workunit.client.1.smithi084.stdout:8/69: stat d0/d9/lc 0 2024-02-28T23:07:24.716 INFO:tasks.workunit.client.1.smithi084.stdout:1/115: mkdir d12/d1b/d25 0 2024-02-28T23:07:24.719 INFO:tasks.workunit.client.1.smithi084.stdout:7/100: rename db to db/d10/d17/d19 22 2024-02-28T23:07:24.719 INFO:tasks.workunit.client.1.smithi084.stdout:2/60: mknod d5/d7/d9/cc 0 2024-02-28T23:07:24.722 INFO:tasks.workunit.client.1.smithi084.stdout:7/101: chown db/le 90420 1 2024-02-28T23:07:24.722 INFO:tasks.workunit.client.1.smithi084.stdout:1/116: write d12/f13 [1024901,63395] 0 2024-02-28T23:07:24.722 INFO:tasks.workunit.client.1.smithi084.stdout:6/85: mknod d1/d9/c19 0 2024-02-28T23:07:24.726 INFO:tasks.workunit.client.1.smithi084.stdout:7/102: chown db 14 1 2024-02-28T23:07:24.734 INFO:tasks.workunit.client.1.smithi084.stdout:4/60: mknod d1/dd/c10 0 2024-02-28T23:07:24.734 INFO:tasks.workunit.client.1.smithi084.stdout:5/88: dwrite dc/de/df/d16/f1b [4194304,4194304] 0 2024-02-28T23:07:24.734 INFO:tasks.workunit.client.1.smithi084.stdout:1/117: creat d12/d18/f26 x:0 0 0 2024-02-28T23:07:24.737 INFO:tasks.workunit.client.1.smithi084.stdout:9/75: link d4/l10 d4/l16 0 2024-02-28T23:07:24.739 INFO:tasks.workunit.client.1.smithi084.stdout:8/70: rename d0/d9/dd to d0/d9/d1b 0 2024-02-28T23:07:24.739 INFO:tasks.workunit.client.1.smithi084.stdout:9/76: chown d4/d6/f7 78309 1 2024-02-28T23:07:24.745 INFO:tasks.workunit.client.1.smithi084.stdout:7/103: mknod db/d11/c1a 0 2024-02-28T23:07:24.751 INFO:tasks.workunit.client.1.smithi084.stdout:5/89: unlink fb 0 2024-02-28T23:07:24.751 INFO:tasks.workunit.client.1.smithi084.stdout:4/61: creat d1/dd/f11 x:0 0 0 2024-02-28T23:07:24.752 INFO:tasks.workunit.client.1.smithi084.stdout:3/41: dread d3/fc [0,4194304] 0 2024-02-28T23:07:24.752 INFO:tasks.workunit.client.1.smithi084.stdout:8/71: mknod d0/d9/d1b/c1c 0 2024-02-28T23:07:24.754 INFO:tasks.workunit.client.1.smithi084.stdout:5/90: rmdir dc 39 2024-02-28T23:07:24.756 INFO:tasks.workunit.client.1.smithi084.stdout:4/62: creat d1/f12 x:0 0 0 2024-02-28T23:07:24.757 INFO:tasks.workunit.client.1.smithi084.stdout:0/40: dwrite d1/fc [0,4194304] 0 2024-02-28T23:07:24.758 INFO:tasks.workunit.client.1.smithi084.stdout:7/104: creat db/d10/d18/f1b x:0 0 0 2024-02-28T23:07:24.758 INFO:tasks.workunit.client.1.smithi084.stdout:3/42: mkdir d3/d8/df 0 2024-02-28T23:07:24.767 INFO:tasks.workunit.client.1.smithi084.stdout:4/63: mknod d1/c13 0 2024-02-28T23:07:24.768 INFO:tasks.workunit.client.1.smithi084.stdout:2/61: truncate d5/d7/f8 4155598 0 2024-02-28T23:07:24.770 INFO:tasks.workunit.client.1.smithi084.stdout:9/77: write d4/d6/f7 [839492,18804] 0 2024-02-28T23:07:24.782 INFO:tasks.workunit.client.1.smithi084.stdout:3/43: creat d3/d8/f10 x:0 0 0 2024-02-28T23:07:24.783 INFO:tasks.workunit.client.1.smithi084.stdout:8/72: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:24.786 INFO:tasks.workunit.client.1.smithi084.stdout:0/41: symlink d1/d8/lf 0 2024-02-28T23:07:24.790 INFO:tasks.workunit.client.1.smithi084.stdout:3/44: creat d3/f11 x:0 0 0 2024-02-28T23:07:24.798 INFO:tasks.workunit.client.1.smithi084.stdout:3/45: stat d3/d8/f10 0 2024-02-28T23:07:24.799 INFO:tasks.workunit.client.1.smithi084.stdout:8/73: write d0/fe [1460319,42164] 0 2024-02-28T23:07:24.810 INFO:tasks.workunit.client.1.smithi084.stdout:0/42: rename d1/c5 to d1/d6/db/c10 0 2024-02-28T23:07:24.812 INFO:tasks.workunit.client.1.smithi084.stdout:7/105: dwrite db/d10/d18/f1b [0,4194304] 0 2024-02-28T23:07:24.813 INFO:tasks.workunit.client.1.smithi084.stdout:2/62: dwrite f3 [4194304,4194304] 0 2024-02-28T23:07:24.816 INFO:tasks.workunit.client.1.smithi084.stdout:3/46: creat d3/d8/df/f12 x:0 0 0 2024-02-28T23:07:24.822 INFO:tasks.workunit.client.1.smithi084.stdout:8/74: rename d0/d9/f13 to d0/d9/d1b/f1d 0 2024-02-28T23:07:24.823 INFO:tasks.workunit.client.1.smithi084.stdout:5/91: creat dc/de/df/d16/f1d x:0 0 0 2024-02-28T23:07:24.824 INFO:tasks.workunit.client.1.smithi084.stdout:7/106: symlink db/d10/l1c 0 2024-02-28T23:07:24.826 INFO:tasks.workunit.client.1.smithi084.stdout:9/78: dwrite d4/d6/f9 [0,4194304] 0 2024-02-28T23:07:24.827 INFO:tasks.workunit.client.1.smithi084.stdout:3/47: symlink d3/d8/l13 0 2024-02-28T23:07:24.827 INFO:tasks.workunit.client.1.smithi084.stdout:2/63: mkdir d5/dd 0 2024-02-28T23:07:24.828 INFO:tasks.workunit.client.1.smithi084.stdout:8/75: chown d0/d9/c16 6166 1 2024-02-28T23:07:24.830 INFO:tasks.workunit.client.1.smithi084.stdout:3/48: write d3/f7 [3088577,124198] 0 2024-02-28T23:07:24.834 INFO:tasks.workunit.client.1.smithi084.stdout:9/79: creat d4/d6/f17 x:0 0 0 2024-02-28T23:07:24.836 INFO:tasks.workunit.client.1.smithi084.stdout:3/49: mkdir d3/d8/d14 0 2024-02-28T23:07:24.837 INFO:tasks.workunit.client.1.smithi084.stdout:0/43: dread d1/fc [4194304,4194304] 0 2024-02-28T23:07:24.839 INFO:tasks.workunit.client.1.smithi084.stdout:2/64: creat d5/d7/fe x:0 0 0 2024-02-28T23:07:24.839 INFO:tasks.workunit.client.1.smithi084.stdout:7/107: link db/d11/c1a db/d10/d18/c1d 0 2024-02-28T23:07:24.839 INFO:tasks.workunit.client.1.smithi084.stdout:7/108: chown db/fd 1012186143 1 2024-02-28T23:07:24.840 INFO:tasks.workunit.client.1.smithi084.stdout:9/80: creat d4/d6/d15/f18 x:0 0 0 2024-02-28T23:07:24.841 INFO:tasks.workunit.client.1.smithi084.stdout:7/109: chown db/d10/d18 298112 1 2024-02-28T23:07:24.846 INFO:tasks.workunit.client.1.smithi084.stdout:9/81: write d4/d6/f8 [3099607,123967] 0 2024-02-28T23:07:24.846 INFO:tasks.workunit.client.1.smithi084.stdout:3/50: getdents d3/d8/d14 0 2024-02-28T23:07:24.846 INFO:tasks.workunit.client.1.smithi084.stdout:2/65: creat d5/dd/ff x:0 0 0 2024-02-28T23:07:24.852 INFO:tasks.workunit.client.1.smithi084.stdout:9/82: creat d4/d6/d15/f19 x:0 0 0 2024-02-28T23:07:24.855 INFO:tasks.workunit.client.1.smithi084.stdout:5/92: getdents dc/de/df/d16 0 2024-02-28T23:07:24.856 INFO:tasks.workunit.client.1.smithi084.stdout:9/83: write d4/d6/ff [1043926,127798] 0 2024-02-28T23:07:24.858 INFO:tasks.workunit.client.1.smithi084.stdout:2/66: creat d5/f10 x:0 0 0 2024-02-28T23:07:24.858 INFO:tasks.workunit.client.1.smithi084.stdout:3/51: rmdir d3 39 2024-02-28T23:07:24.859 INFO:tasks.workunit.client.1.smithi084.stdout:2/67: readlink d5/l6 0 2024-02-28T23:07:24.859 INFO:tasks.workunit.client.1.smithi084.stdout:0/44: dread d1/fc [4194304,4194304] 0 2024-02-28T23:07:24.863 INFO:tasks.workunit.client.1.smithi084.stdout:3/52: symlink d3/d8/df/l15 0 2024-02-28T23:07:24.864 INFO:tasks.workunit.client.1.smithi084.stdout:3/53: readlink d3/d8/l9 0 2024-02-28T23:07:24.864 INFO:tasks.workunit.client.1.smithi084.stdout:7/110: dread f9 [0,4194304] 0 2024-02-28T23:07:24.868 INFO:tasks.workunit.client.1.smithi084.stdout:7/111: creat db/f1e x:0 0 0 2024-02-28T23:07:24.868 INFO:tasks.workunit.client.1.smithi084.stdout:3/54: creat d3/d8/d14/f16 x:0 0 0 2024-02-28T23:07:24.872 INFO:tasks.workunit.client.1.smithi084.stdout:9/84: truncate d4/d6/f9 2103673 0 2024-02-28T23:07:24.872 INFO:tasks.workunit.client.1.smithi084.stdout:3/55: write d3/f11 [824973,107199] 0 2024-02-28T23:07:24.873 INFO:tasks.workunit.client.1.smithi084.stdout:3/56: stat f1 0 2024-02-28T23:07:24.877 INFO:tasks.workunit.client.1.smithi084.stdout:3/57: creat d3/f17 x:0 0 0 2024-02-28T23:07:24.881 INFO:tasks.workunit.client.1.smithi084.stdout:3/58: unlink d3/f17 0 2024-02-28T23:07:24.883 INFO:tasks.workunit.client.1.smithi084.stdout:0/45: dwrite d1/fc [4194304,4194304] 0 2024-02-28T23:07:24.884 INFO:tasks.workunit.client.1.smithi084.stdout:2/68: dwrite d5/f10 [0,4194304] 0 2024-02-28T23:07:24.885 INFO:tasks.workunit.client.1.smithi084.stdout:3/59: dread - d3/d8/d14/f16 zero size 2024-02-28T23:07:24.885 INFO:tasks.workunit.client.1.smithi084.stdout:9/85: write d4/d6/f9 [1499988,91020] 0 2024-02-28T23:07:24.886 INFO:tasks.workunit.client.1.smithi084.stdout:0/46: symlink d1/d6/l11 0 2024-02-28T23:07:24.889 INFO:tasks.workunit.client.1.smithi084.stdout:2/69: write d5/dd/ff [989825,38396] 0 2024-02-28T23:07:24.890 INFO:tasks.workunit.client.1.smithi084.stdout:3/60: mkdir d3/d18 0 2024-02-28T23:07:24.894 INFO:tasks.workunit.client.1.smithi084.stdout:2/70: fsync d5/fb 0 2024-02-28T23:07:24.896 INFO:tasks.workunit.client.1.smithi084.stdout:2/71: write f4 [1067940,37980] 0 2024-02-28T23:07:24.903 INFO:tasks.workunit.client.1.smithi084.stdout:2/72: creat d5/d7/d9/f11 x:0 0 0 2024-02-28T23:07:24.910 INFO:tasks.workunit.client.1.smithi084.stdout:2/73: mkdir d5/d7/d12 0 2024-02-28T23:07:24.914 INFO:tasks.workunit.client.1.smithi084.stdout:2/74: read f3 [4402299,94658] 0 2024-02-28T23:07:24.915 INFO:tasks.workunit.client.1.smithi084.stdout:9/86: dwrite d4/fc [4194304,4194304] 0 2024-02-28T23:07:24.917 INFO:tasks.workunit.client.1.smithi084.stdout:0/47: fsync d1/fc 0 2024-02-28T23:07:24.923 INFO:tasks.workunit.client.1.smithi084.stdout:9/87: creat d4/f1a x:0 0 0 2024-02-28T23:07:24.926 INFO:tasks.workunit.client.1.smithi084.stdout:2/75: creat d5/d7/d12/f13 x:0 0 0 2024-02-28T23:07:24.930 INFO:tasks.workunit.client.1.smithi084.stdout:3/61: dwrite d3/d8/d14/f16 [0,4194304] 0 2024-02-28T23:07:24.933 INFO:tasks.workunit.client.1.smithi084.stdout:3/62: stat d3 0 2024-02-28T23:07:24.933 INFO:tasks.workunit.client.1.smithi084.stdout:2/76: unlink d5/fb 0 2024-02-28T23:07:24.938 INFO:tasks.workunit.client.1.smithi084.stdout:3/63: creat d3/d8/df/f19 x:0 0 0 2024-02-28T23:07:24.939 INFO:tasks.workunit.client.1.smithi084.stdout:0/48: dread d1/fc [4194304,4194304] 0 2024-02-28T23:07:24.943 INFO:tasks.workunit.client.1.smithi084.stdout:3/64: dread - d3/d8/fd zero size 2024-02-28T23:07:24.946 INFO:tasks.workunit.client.1.smithi084.stdout:3/65: chown d3/d8/fd 14373 1 2024-02-28T23:07:24.946 INFO:tasks.workunit.client.1.smithi084.stdout:9/88: dwrite d4/fc [4194304,4194304] 0 2024-02-28T23:07:24.948 INFO:tasks.workunit.client.0.smithi057.stderr:++ readlink -f fsstress 2024-02-28T23:07:24.951 INFO:tasks.workunit.client.1.smithi084.stdout:3/66: write d3/f11 [681644,113170] 0 2024-02-28T23:07:24.954 INFO:tasks.workunit.client.0.smithi057.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-28T23:07:24.954 INFO:tasks.workunit.client.0.smithi057.stderr:+ popd 2024-02-28T23:07:24.958 INFO:tasks.workunit.client.0.smithi057.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-28T23:07:24.958 INFO:tasks.workunit.client.0.smithi057.stderr:+ popd 2024-02-28T23:07:24.959 INFO:tasks.workunit.client.1.smithi084.stdout:0/49: dread d1/fc [0,4194304] 0 2024-02-28T23:07:24.959 INFO:tasks.workunit.client.1.smithi084.stdout:3/67: mkdir d3/d8/d1a 0 2024-02-28T23:07:24.961 INFO:tasks.workunit.client.0.smithi057.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-28T23:07:24.961 INFO:tasks.workunit.client.1.smithi084.stdout:2/77: dwrite d5/d7/d12/f13 [0,4194304] 0 2024-02-28T23:07:24.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ mktemp -d -p . 2024-02-28T23:07:24.966 INFO:tasks.workunit.client.1.smithi084.stdout:3/68: symlink d3/d8/d14/l1b 0 2024-02-28T23:07:24.966 INFO:tasks.workunit.client.1.smithi084.stdout:2/78: mknod d5/d7/d9/c14 0 2024-02-28T23:07:24.968 INFO:tasks.workunit.client.1.smithi084.stdout:3/69: readlink d3/d8/l9 0 2024-02-28T23:07:24.970 INFO:tasks.workunit.client.0.smithi057.stderr:+ T=./tmp.6RtNsWrv6j 2024-02-28T23:07:24.970 INFO:tasks.workunit.client.0.smithi057.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.6RtNsWrv6j -l 1 -n 1000 -p 10 -v 2024-02-28T23:07:24.972 INFO:tasks.workunit.client.1.smithi084.stdout:9/89: truncate d4/d6/f7 1016150 0 2024-02-28T23:07:24.979 INFO:tasks.workunit.client.0.smithi057.stdout:seed = 1708214123 2024-02-28T23:07:24.981 INFO:tasks.workunit.client.1.smithi084.stdout:3/70: dread f1 [0,4194304] 0 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.1.smithi084.stdout:2/79: dread d5/d7/f8 [0,4194304] 0 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/0: dwrite - no filename 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/1: dwrite - no filename 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/2: dread - no filename 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/3: dwrite - no filename 2024-02-28T23:07:24.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/4: dread - no filename 2024-02-28T23:07:24.986 INFO:tasks.workunit.client.0.smithi057.stdout:0/5: read - no filename 2024-02-28T23:07:24.986 INFO:tasks.workunit.client.0.smithi057.stdout:0/6: fdatasync - no filename 2024-02-28T23:07:24.986 INFO:tasks.workunit.client.0.smithi057.stdout:0/7: dwrite - no filename 2024-02-28T23:07:24.986 INFO:tasks.workunit.client.0.smithi057.stdout:0/8: write - no filename 2024-02-28T23:07:24.986 INFO:tasks.workunit.client.0.smithi057.stdout:0/9: dwrite - no filename 2024-02-28T23:07:24.987 INFO:tasks.workunit.client.1.smithi084.stdout:3/71: link d3/f6 d3/d8/d14/f1c 0 2024-02-28T23:07:24.987 INFO:tasks.workunit.client.1.smithi084.stdout:0/50: dwrite d1/fc [0,4194304] 0 2024-02-28T23:07:24.988 INFO:tasks.workunit.client.0.smithi057.stdout:1/0: dread - no filename 2024-02-28T23:07:24.988 INFO:tasks.workunit.client.0.smithi057.stdout:0/10: mknod c0 0 2024-02-28T23:07:24.988 INFO:tasks.workunit.client.0.smithi057.stdout:1/1: chown . 7 1 2024-02-28T23:07:24.988 INFO:tasks.workunit.client.0.smithi057.stdout:1/2: dwrite - no filename 2024-02-28T23:07:24.988 INFO:tasks.workunit.client.1.smithi084.stdout:2/80: fdatasync d5/d7/d9/f11 0 2024-02-28T23:07:24.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/72: mknod d3/d8/df/c1d 0 2024-02-28T23:07:24.992 INFO:tasks.workunit.client.0.smithi057.stdout:1/3: creat f0 x:0 0 0 2024-02-28T23:07:24.995 INFO:tasks.workunit.client.1.smithi084.stdout:9/90: dwrite d4/d6/d15/f18 [0,4194304] 0 2024-02-28T23:07:24.995 INFO:tasks.workunit.client.0.smithi057.stdout:2/0: creat f0 x:0 0 0 2024-02-28T23:07:24.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/91: chown d4/d6/d15/f18 398150 1 2024-02-28T23:07:24.998 INFO:tasks.workunit.client.0.smithi057.stdout:1/4: mkdir d1 0 2024-02-28T23:07:24.999 INFO:tasks.workunit.client.1.smithi084.stdout:3/73: rename d3/d8/d14/l1b to d3/d8/d14/l1e 0 2024-02-28T23:07:25.001 INFO:tasks.workunit.client.1.smithi084.stdout:3/74: readlink d3/d8/l13 0 2024-02-28T23:07:25.002 INFO:tasks.workunit.client.1.smithi084.stdout:9/92: chown d4/l10 240340950 1 2024-02-28T23:07:25.002 INFO:tasks.workunit.client.0.smithi057.stdout:1/5: write f0 [330552,41324] 0 2024-02-28T23:07:25.003 INFO:tasks.workunit.client.0.smithi057.stdout:2/1: creat f1 x:0 0 0 2024-02-28T23:07:25.003 INFO:tasks.workunit.client.0.smithi057.stdout:4/0: dread - no filename 2024-02-28T23:07:25.005 INFO:tasks.workunit.client.0.smithi057.stdout:3/0: symlink l0 0 2024-02-28T23:07:25.005 INFO:tasks.workunit.client.0.smithi057.stdout:3/1: dread - no filename 2024-02-28T23:07:25.005 INFO:tasks.workunit.client.1.smithi084.stdout:0/51: dread d1/fc [4194304,4194304] 0 2024-02-28T23:07:25.006 INFO:tasks.workunit.client.1.smithi084.stdout:9/93: symlink d4/d6/l1b 0 2024-02-28T23:07:25.007 INFO:tasks.workunit.client.0.smithi057.stdout:5/0: chown . 227586147 1 2024-02-28T23:07:25.007 INFO:tasks.workunit.client.0.smithi057.stdout:3/2: readlink l0 0 2024-02-28T23:07:25.009 INFO:tasks.workunit.client.1.smithi084.stdout:0/52: chown d1/d6/db/c10 782 1 2024-02-28T23:07:25.010 INFO:tasks.workunit.client.1.smithi084.stdout:2/81: dread f4 [0,4194304] 0 2024-02-28T23:07:25.011 INFO:tasks.workunit.client.1.smithi084.stdout:3/75: write d3/d8/d14/f1c [4571875,74180] 0 2024-02-28T23:07:25.012 INFO:tasks.workunit.client.1.smithi084.stdout:0/53: rename d1/d6/l11 to d1/l12 0 2024-02-28T23:07:25.014 INFO:tasks.workunit.client.1.smithi084.stdout:0/54: write d1/fc [2972311,113888] 0 2024-02-28T23:07:25.022 INFO:tasks.workunit.client.0.smithi057.stdout:3/3: stat l0 0 2024-02-28T23:07:25.027 INFO:tasks.workunit.client.0.smithi057.stdout:4/1: creat f0 x:0 0 0 2024-02-28T23:07:25.028 INFO:tasks.workunit.client.0.smithi057.stdout:6/0: write - no filename 2024-02-28T23:07:25.028 INFO:tasks.workunit.client.0.smithi057.stdout:6/1: dwrite - no filename 2024-02-28T23:07:25.028 INFO:tasks.workunit.client.0.smithi057.stdout:6/2: dread - no filename 2024-02-28T23:07:25.028 INFO:tasks.workunit.client.0.smithi057.stdout:6/3: dread - no filename 2024-02-28T23:07:25.028 INFO:tasks.workunit.client.0.smithi057.stdout:6/4: write - no filename 2024-02-28T23:07:25.056 INFO:tasks.workunit.client.1.smithi084.stdout:9/94: read d4/d6/ff [1016384,65033] 0 2024-02-28T23:07:25.058 INFO:tasks.workunit.client.1.smithi084.stdout:0/55: truncate d1/fc 4147746 0 2024-02-28T23:07:25.059 INFO:tasks.workunit.client.1.smithi084.stdout:9/95: mkdir d4/d6/d15/d1c 0 2024-02-28T23:07:25.064 INFO:tasks.workunit.client.1.smithi084.stdout:9/96: rename d4/d6/d15/d1c to d4/d1d 0 2024-02-28T23:07:25.070 INFO:tasks.workunit.client.1.smithi084.stdout:9/97: rmdir d4 39 2024-02-28T23:07:25.073 INFO:tasks.workunit.client.1.smithi084.stdout:9/98: creat d4/d6/d15/f1e x:0 0 0 2024-02-28T23:07:25.076 INFO:tasks.workunit.client.1.smithi084.stdout:0/56: dread d1/fc [0,4194304] 0 2024-02-28T23:07:25.077 INFO:tasks.workunit.client.1.smithi084.stdout:0/57: stat d1/d6/db/c10 0 2024-02-28T23:07:25.079 INFO:tasks.workunit.client.1.smithi084.stdout:0/58: write d1/fc [2109936,61568] 0 2024-02-28T23:07:25.081 INFO:tasks.workunit.client.1.smithi084.stdout:0/59: write d1/fc [3088820,62834] 0 2024-02-28T23:07:25.084 INFO:tasks.workunit.client.1.smithi084.stdout:9/99: dread d4/fc [4194304,4194304] 0 2024-02-28T23:07:25.088 INFO:tasks.workunit.client.1.smithi084.stdout:0/60: truncate d1/fc 3161983 0 2024-02-28T23:07:25.090 INFO:tasks.workunit.client.1.smithi084.stdout:0/61: creat d1/d6/f13 x:0 0 0 2024-02-28T23:07:25.090 INFO:tasks.workunit.client.1.smithi084.stdout:0/62: chown d1/d6 27500 1 2024-02-28T23:07:25.092 INFO:tasks.workunit.client.1.smithi084.stdout:0/63: chown d1/l12 3660 1 2024-02-28T23:07:25.094 INFO:tasks.workunit.client.1.smithi084.stdout:0/64: creat d1/d6/f14 x:0 0 0 2024-02-28T23:07:25.094 INFO:tasks.workunit.client.1.smithi084.stdout:9/100: dread d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:25.096 INFO:tasks.workunit.client.1.smithi084.stdout:0/65: symlink d1/l15 0 2024-02-28T23:07:25.099 INFO:tasks.workunit.client.1.smithi084.stdout:9/101: getdents d4 0 2024-02-28T23:07:25.101 INFO:tasks.workunit.client.1.smithi084.stdout:9/102: unlink f0 0 2024-02-28T23:07:25.104 INFO:tasks.workunit.client.1.smithi084.stdout:9/103: truncate d4/d6/d15/f1e 963380 0 2024-02-28T23:07:25.107 INFO:tasks.workunit.client.1.smithi084.stdout:9/104: write d4/d6/f8 [1366389,93189] 0 2024-02-28T23:07:25.107 INFO:tasks.workunit.client.1.smithi084.stdout:0/66: dread d1/fc [0,4194304] 0 2024-02-28T23:07:25.111 INFO:tasks.workunit.client.1.smithi084.stdout:9/105: mkdir d4/d1d/d1f 0 2024-02-28T23:07:25.111 INFO:tasks.workunit.client.1.smithi084.stdout:0/67: creat d1/d6/f16 x:0 0 0 2024-02-28T23:07:25.112 INFO:tasks.workunit.client.1.smithi084.stdout:9/106: symlink d4/d1d/l20 0 2024-02-28T23:07:25.112 INFO:tasks.workunit.client.1.smithi084.stdout:0/68: write d1/d6/f13 [934339,114087] 0 2024-02-28T23:07:25.114 INFO:tasks.workunit.client.1.smithi084.stdout:9/107: mknod d4/d1d/c21 0 2024-02-28T23:07:25.115 INFO:tasks.workunit.client.1.smithi084.stdout:6/86: sync 2024-02-28T23:07:25.116 INFO:tasks.workunit.client.1.smithi084.stdout:9/108: read d4/d6/d15/f1e [415030,17213] 0 2024-02-28T23:07:25.117 INFO:tasks.workunit.client.1.smithi084.stdout:6/87: read - d1/d9/f18 zero size 2024-02-28T23:07:25.117 INFO:tasks.workunit.client.1.smithi084.stdout:9/109: readlink d4/d1d/l20 0 2024-02-28T23:07:25.120 INFO:tasks.workunit.client.1.smithi084.stdout:0/69: dread d1/fc [0,4194304] 0 2024-02-28T23:07:25.121 INFO:tasks.workunit.client.1.smithi084.stdout:9/110: link d4/d6/d15/f1e d4/f22 0 2024-02-28T23:07:25.123 INFO:tasks.workunit.client.1.smithi084.stdout:0/70: mknod d1/d6/db/c17 0 2024-02-28T23:07:25.124 INFO:tasks.workunit.client.1.smithi084.stdout:6/88: getdents d1/d9 0 2024-02-28T23:07:25.128 INFO:tasks.workunit.client.1.smithi084.stdout:9/111: symlink d4/d6/d15/d12/l23 0 2024-02-28T23:07:25.134 INFO:tasks.workunit.client.1.smithi084.stdout:0/71: dread d1/fc [0,4194304] 0 2024-02-28T23:07:25.135 INFO:tasks.workunit.client.1.smithi084.stdout:0/72: mkdir d1/d8/d18 0 2024-02-28T23:07:25.139 INFO:tasks.workunit.client.1.smithi084.stdout:0/73: chown d1/l12 3165 1 2024-02-28T23:07:25.139 INFO:tasks.workunit.client.0.smithi057.stdout:3/4: creat f1 x:0 0 0 2024-02-28T23:07:25.140 INFO:tasks.workunit.client.0.smithi057.stdout:3/5: fdatasync f1 0 2024-02-28T23:07:25.141 INFO:tasks.workunit.client.0.smithi057.stdout:7/0: dwrite - no filename 2024-02-28T23:07:25.141 INFO:tasks.workunit.client.0.smithi057.stdout:7/1: dread - no filename 2024-02-28T23:07:25.141 INFO:tasks.workunit.client.1.smithi084.stdout:0/74: fdatasync d1/d6/f16 0 2024-02-28T23:07:25.142 INFO:tasks.workunit.client.1.smithi084.stdout:6/89: dwrite d1/d9/f18 [0,4194304] 0 2024-02-28T23:07:25.142 INFO:tasks.workunit.client.0.smithi057.stdout:4/2: creat f1 x:0 0 0 2024-02-28T23:07:25.143 INFO:tasks.workunit.client.0.smithi057.stdout:5/1: creat f0 x:0 0 0 2024-02-28T23:07:25.143 INFO:tasks.workunit.client.0.smithi057.stdout:5/2: readlink - no filename 2024-02-28T23:07:25.144 INFO:tasks.workunit.client.0.smithi057.stdout:4/3: chown f1 245377 1 2024-02-28T23:07:25.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/5: getdents . 0 2024-02-28T23:07:25.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/6: fsync - no filename 2024-02-28T23:07:25.148 INFO:tasks.workunit.client.1.smithi084.stdout:6/90: truncate d1/d9/f15 940480 0 2024-02-28T23:07:25.149 INFO:tasks.workunit.client.1.smithi084.stdout:0/75: link d1/d6/f16 d1/d6/f19 0 2024-02-28T23:07:25.150 INFO:tasks.workunit.client.0.smithi057.stdout:8/0: write - no filename 2024-02-28T23:07:25.150 INFO:tasks.workunit.client.0.smithi057.stdout:8/1: fdatasync - no filename 2024-02-28T23:07:25.150 INFO:tasks.workunit.client.0.smithi057.stdout:4/4: mknod c2 0 2024-02-28T23:07:25.151 INFO:tasks.workunit.client.1.smithi084.stdout:9/112: dwrite d4/d6/d15/f1e [0,4194304] 0 2024-02-28T23:07:25.152 INFO:tasks.workunit.client.1.smithi084.stdout:6/91: read d1/fe [7200373,21730] 0 2024-02-28T23:07:25.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/2: mkdir d0 0 2024-02-28T23:07:25.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/3: dread - no filename 2024-02-28T23:07:25.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/4: dread - no filename 2024-02-28T23:07:25.155 INFO:tasks.workunit.client.0.smithi057.stdout:6/7: mkdir d0 0 2024-02-28T23:07:25.155 INFO:tasks.workunit.client.1.smithi084.stdout:6/92: truncate d1/fe 6418292 0 2024-02-28T23:07:25.158 INFO:tasks.workunit.client.0.smithi057.stdout:4/5: creat f3 x:0 0 0 2024-02-28T23:07:25.158 INFO:tasks.workunit.client.0.smithi057.stdout:8/2: symlink l0 0 2024-02-28T23:07:25.160 INFO:tasks.workunit.client.0.smithi057.stdout:4/6: dread - f1 zero size 2024-02-28T23:07:25.163 INFO:tasks.workunit.client.0.smithi057.stdout:7/5: mknod d0/c1 0 2024-02-28T23:07:25.163 INFO:tasks.workunit.client.0.smithi057.stdout:7/6: write - no filename 2024-02-28T23:07:25.164 INFO:tasks.workunit.client.0.smithi057.stdout:4/7: write f3 [711471,38756] 0 2024-02-28T23:07:25.164 INFO:tasks.workunit.client.0.smithi057.stdout:4/8: rmdir - no directory 2024-02-28T23:07:25.165 INFO:tasks.workunit.client.0.smithi057.stdout:9/0: mknod c0 0 2024-02-28T23:07:25.166 INFO:tasks.workunit.client.0.smithi057.stdout:8/3: mknod c1 0 2024-02-28T23:07:25.166 INFO:tasks.workunit.client.0.smithi057.stdout:8/4: write - no filename 2024-02-28T23:07:25.167 INFO:tasks.workunit.client.1.smithi084.stdout:6/93: dread d1/d9/f18 [0,4194304] 0 2024-02-28T23:07:25.169 INFO:tasks.workunit.client.0.smithi057.stdout:7/7: chown d0/c1 103504440 1 2024-02-28T23:07:25.169 INFO:tasks.workunit.client.0.smithi057.stdout:7/8: dread - no filename 2024-02-28T23:07:25.170 INFO:tasks.workunit.client.0.smithi057.stdout:6/8: creat d0/f1 x:0 0 0 2024-02-28T23:07:25.171 INFO:tasks.workunit.client.1.smithi084.stdout:6/94: mknod d1/d13/c1a 0 2024-02-28T23:07:25.173 INFO:tasks.workunit.client.1.smithi084.stdout:6/95: creat d1/f1b x:0 0 0 2024-02-28T23:07:25.175 INFO:tasks.workunit.client.0.smithi057.stdout:6/9: write d0/f1 [187745,28142] 0 2024-02-28T23:07:25.175 INFO:tasks.workunit.client.0.smithi057.stdout:4/9: mknod c4 0 2024-02-28T23:07:25.177 INFO:tasks.workunit.client.0.smithi057.stdout:9/1: symlink l1 0 2024-02-28T23:07:25.177 INFO:tasks.workunit.client.0.smithi057.stdout:9/2: write - no filename 2024-02-28T23:07:25.177 INFO:tasks.workunit.client.0.smithi057.stdout:9/3: write - no filename 2024-02-28T23:07:25.177 INFO:tasks.workunit.client.0.smithi057.stdout:8/5: symlink l2 0 2024-02-28T23:07:25.178 INFO:tasks.workunit.client.1.smithi084.stdout:6/96: dread d1/d9/f15 [0,4194304] 0 2024-02-28T23:07:25.180 INFO:tasks.workunit.client.1.smithi084.stdout:6/97: mkdir d1/d9/d1c 0 2024-02-28T23:07:25.181 INFO:tasks.workunit.client.0.smithi057.stdout:7/9: rename d0/c1 to d0/c2 0 2024-02-28T23:07:25.183 INFO:tasks.workunit.client.1.smithi084.stdout:6/98: mkdir d1/d9/d1d 0 2024-02-28T23:07:25.183 INFO:tasks.workunit.client.0.smithi057.stdout:4/10: fdatasync f1 0 2024-02-28T23:07:25.183 INFO:tasks.workunit.client.0.smithi057.stdout:6/10: rename d0 to d0/d2 22 2024-02-28T23:07:25.185 INFO:tasks.workunit.client.0.smithi057.stdout:8/6: mknod c3 0 2024-02-28T23:07:25.185 INFO:tasks.workunit.client.0.smithi057.stdout:8/7: read - no filename 2024-02-28T23:07:25.185 INFO:tasks.workunit.client.0.smithi057.stdout:8/8: dread - no filename 2024-02-28T23:07:25.185 INFO:tasks.workunit.client.0.smithi057.stdout:3/6: dwrite f1 [0,4194304] 0 2024-02-28T23:07:25.186 INFO:tasks.workunit.client.0.smithi057.stdout:9/4: chown c0 6755485 1 2024-02-28T23:07:25.186 INFO:tasks.workunit.client.0.smithi057.stdout:9/5: write - no filename 2024-02-28T23:07:25.186 INFO:tasks.workunit.client.0.smithi057.stdout:9/6: write - no filename 2024-02-28T23:07:25.186 INFO:tasks.workunit.client.0.smithi057.stdout:6/11: stat d0/f1 0 2024-02-28T23:07:25.188 INFO:tasks.workunit.client.0.smithi057.stdout:4/11: mknod c5 0 2024-02-28T23:07:25.192 INFO:tasks.workunit.client.0.smithi057.stdout:4/12: write f0 [32237,89479] 0 2024-02-28T23:07:25.193 INFO:tasks.workunit.client.0.smithi057.stdout:8/9: mkdir d4 0 2024-02-28T23:07:25.196 INFO:tasks.workunit.client.0.smithi057.stdout:9/7: mknod c2 0 2024-02-28T23:07:25.197 INFO:tasks.workunit.client.0.smithi057.stdout:8/10: chown l0 42 1 2024-02-28T23:07:25.198 INFO:tasks.workunit.client.0.smithi057.stdout:8/11: write - no filename 2024-02-28T23:07:25.198 INFO:tasks.workunit.client.0.smithi057.stdout:8/12: dread - no filename 2024-02-28T23:07:25.200 INFO:tasks.workunit.client.0.smithi057.stdout:3/7: creat f2 x:0 0 0 2024-02-28T23:07:25.201 INFO:tasks.workunit.client.0.smithi057.stdout:4/13: dread f3 [0,4194304] 0 2024-02-28T23:07:25.201 INFO:tasks.workunit.client.0.smithi057.stdout:4/14: readlink - no filename 2024-02-28T23:07:25.201 INFO:tasks.workunit.client.1.smithi084.stdout:6/99: dwrite d1/d9/f15 [0,4194304] 0 2024-02-28T23:07:25.204 INFO:tasks.workunit.client.0.smithi057.stdout:9/8: mknod c3 0 2024-02-28T23:07:25.206 INFO:tasks.workunit.client.1.smithi084.stdout:6/100: rename d1/d9/d1c to d1/d9/d1d/d1e 0 2024-02-28T23:07:25.206 INFO:tasks.workunit.client.1.smithi084.stdout:6/101: stat d1/cc 0 2024-02-28T23:07:25.207 INFO:tasks.workunit.client.1.smithi084.stdout:6/102: fdatasync d1/f1b 0 2024-02-28T23:07:25.209 INFO:tasks.workunit.client.1.smithi084.stdout:6/103: chown d1/c16 20897913 1 2024-02-28T23:07:25.210 INFO:tasks.workunit.client.1.smithi084.stdout:6/104: rename d1/d9/d1d to d1/d9/d1d/d1e/d1f 22 2024-02-28T23:07:25.211 INFO:tasks.workunit.client.0.smithi057.stdout:9/9: symlink l4 0 2024-02-28T23:07:25.212 INFO:tasks.workunit.client.0.smithi057.stdout:9/10: stat l1 0 2024-02-28T23:07:25.212 INFO:tasks.workunit.client.0.smithi057.stdout:9/11: dread - no filename 2024-02-28T23:07:25.220 INFO:tasks.workunit.client.0.smithi057.stdout:9/12: rename c3 to c5 0 2024-02-28T23:07:25.220 INFO:tasks.workunit.client.0.smithi057.stdout:9/13: write - no filename 2024-02-28T23:07:25.220 INFO:tasks.workunit.client.0.smithi057.stdout:9/14: truncate - no filename 2024-02-28T23:07:25.220 INFO:tasks.workunit.client.0.smithi057.stdout:9/15: dwrite - no filename 2024-02-28T23:07:25.220 INFO:tasks.workunit.client.1.smithi084.stdout:6/105: dread d1/d9/f18 [0,4194304] 0 2024-02-28T23:07:25.222 INFO:tasks.workunit.client.0.smithi057.stdout:0/11: getdents . 0 2024-02-28T23:07:25.222 INFO:tasks.workunit.client.1.smithi084.stdout:6/106: fsync d1/fe 0 2024-02-28T23:07:25.224 INFO:tasks.workunit.client.0.smithi057.stdout:3/8: dread f1 [0,4194304] 0 2024-02-28T23:07:25.230 INFO:tasks.workunit.client.0.smithi057.stdout:2/2: getdents . 0 2024-02-28T23:07:25.232 INFO:tasks.workunit.client.0.smithi057.stdout:2/3: stat f0 0 2024-02-28T23:07:25.232 INFO:tasks.workunit.client.0.smithi057.stdout:3/9: mkdir d3 0 2024-02-28T23:07:25.233 INFO:tasks.workunit.client.0.smithi057.stdout:3/10: chown d3 25 1 2024-02-28T23:07:25.235 INFO:tasks.workunit.client.0.smithi057.stdout:0/12: rename c0 to c1 0 2024-02-28T23:07:25.236 INFO:tasks.workunit.client.0.smithi057.stdout:2/4: mkdir d2 0 2024-02-28T23:07:25.237 INFO:tasks.workunit.client.0.smithi057.stdout:2/5: stat d2 0 2024-02-28T23:07:25.241 INFO:tasks.workunit.client.0.smithi057.stdout:3/11: mkdir d3/d4 0 2024-02-28T23:07:25.241 INFO:tasks.workunit.client.0.smithi057.stdout:4/15: dwrite f3 [0,4194304] 0 2024-02-28T23:07:25.241 INFO:tasks.workunit.client.0.smithi057.stdout:2/6: write f1 [79632,98843] 0 2024-02-28T23:07:25.243 INFO:tasks.workunit.client.0.smithi057.stdout:2/7: chown d2 394948 1 2024-02-28T23:07:25.245 INFO:tasks.workunit.client.0.smithi057.stdout:2/8: chown f1 217053 1 2024-02-28T23:07:25.245 INFO:tasks.workunit.client.0.smithi057.stdout:2/9: fsync f1 0 2024-02-28T23:07:25.246 INFO:tasks.workunit.client.0.smithi057.stdout:3/12: mkdir d3/d5 0 2024-02-28T23:07:25.248 INFO:tasks.workunit.client.0.smithi057.stdout:4/16: rename c2 to c6 0 2024-02-28T23:07:25.252 INFO:tasks.workunit.client.0.smithi057.stdout:3/13: creat d3/d4/f6 x:0 0 0 2024-02-28T23:07:25.254 INFO:tasks.workunit.client.0.smithi057.stdout:4/17: read f3 [2466641,64811] 0 2024-02-28T23:07:25.261 INFO:tasks.workunit.client.0.smithi057.stdout:4/18: write f1 [552721,48805] 0 2024-02-28T23:07:25.266 INFO:tasks.workunit.client.1.smithi084.stdout:1/118: sync 2024-02-28T23:07:25.266 INFO:tasks.workunit.client.1.smithi084.stdout:4/64: sync 2024-02-28T23:07:25.269 INFO:tasks.workunit.client.0.smithi057.stdout:3/14: rename d3/d5 to d3/d4/d7 0 2024-02-28T23:07:25.272 INFO:tasks.workunit.client.1.smithi084.stdout:6/107: rename d1 to d1/d13/d20 22 2024-02-28T23:07:25.273 INFO:tasks.workunit.client.0.smithi057.stdout:3/15: mkdir d3/d8 0 2024-02-28T23:07:25.274 INFO:tasks.workunit.client.1.smithi084.stdout:4/65: chown d1/ca 3 1 2024-02-28T23:07:25.277 INFO:tasks.workunit.client.1.smithi084.stdout:4/66: stat d1/dd/f11 0 2024-02-28T23:07:25.278 INFO:tasks.workunit.client.0.smithi057.stdout:2/10: dwrite f1 [0,4194304] 0 2024-02-28T23:07:25.278 INFO:tasks.workunit.client.1.smithi084.stdout:6/108: creat d1/d9/f21 x:0 0 0 2024-02-28T23:07:25.281 INFO:tasks.workunit.client.1.smithi084.stdout:6/109: chown d1/d9/d1d/d1e 729 1 2024-02-28T23:07:25.283 INFO:tasks.workunit.client.1.smithi084.stdout:4/67: mkdir d1/dd/d14 0 2024-02-28T23:07:25.287 INFO:tasks.workunit.client.0.smithi057.stdout:4/19: dread f3 [0,4194304] 0 2024-02-28T23:07:25.288 INFO:tasks.workunit.client.1.smithi084.stdout:4/68: read - d1/f12 zero size 2024-02-28T23:07:25.291 INFO:tasks.workunit.client.0.smithi057.stdout:4/20: symlink l7 0 2024-02-28T23:07:25.295 INFO:tasks.workunit.client.0.smithi057.stdout:4/21: creat f8 x:0 0 0 2024-02-28T23:07:25.295 INFO:tasks.workunit.client.0.smithi057.stdout:3/16: dread f1 [0,4194304] 0 2024-02-28T23:07:25.299 INFO:tasks.workunit.client.0.smithi057.stdout:4/22: mknod c9 0 2024-02-28T23:07:25.302 INFO:tasks.workunit.client.0.smithi057.stdout:6/12: fsync d0/f1 0 2024-02-28T23:07:25.304 INFO:tasks.workunit.client.1.smithi084.stdout:1/119: dwrite fc [0,4194304] 0 2024-02-28T23:07:25.308 INFO:tasks.workunit.client.0.smithi057.stdout:7/10: rename d0/c2 to d0/c3 0 2024-02-28T23:07:25.312 INFO:tasks.workunit.client.0.smithi057.stdout:2/11: fdatasync f1 0 2024-02-28T23:07:25.313 INFO:tasks.workunit.client.1.smithi084.stdout:1/120: symlink d12/d1b/l27 0 2024-02-28T23:07:25.313 INFO:tasks.workunit.client.1.smithi084.stdout:6/110: dwrite d1/d9/f18 [0,4194304] 0 2024-02-28T23:07:25.313 INFO:tasks.workunit.client.0.smithi057.stdout:7/11: mknod d0/c4 0 2024-02-28T23:07:25.313 INFO:tasks.workunit.client.0.smithi057.stdout:7/12: write - no filename 2024-02-28T23:07:25.314 INFO:tasks.workunit.client.1.smithi084.stdout:4/69: dwrite d1/f5 [0,4194304] 0 2024-02-28T23:07:25.315 INFO:tasks.workunit.client.0.smithi057.stdout:2/12: write f0 [542773,80073] 0 2024-02-28T23:07:25.315 INFO:tasks.workunit.client.0.smithi057.stdout:9/16: chown c5 1459625 1 2024-02-28T23:07:25.318 INFO:tasks.workunit.client.0.smithi057.stdout:7/13: chown d0/c4 1038521 1 2024-02-28T23:07:25.319 INFO:tasks.workunit.client.1.smithi084.stdout:4/70: write d1/dd/fe [677082,25441] 0 2024-02-28T23:07:25.319 INFO:tasks.workunit.client.1.smithi084.stdout:6/111: mkdir d1/d13/d22 0 2024-02-28T23:07:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:25 smithi057 ceph-mon[29096]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:25.320 INFO:tasks.workunit.client.0.smithi057.stdout:4/23: dread f3 [0,4194304] 0 2024-02-28T23:07:25.323 INFO:tasks.workunit.client.1.smithi084.stdout:1/121: link f11 d12/f28 0 2024-02-28T23:07:25.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/14: mkdir d0/d5 0 2024-02-28T23:07:25.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/15: dwrite - no filename 2024-02-28T23:07:25.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/16: dwrite - no filename 2024-02-28T23:07:25.326 INFO:tasks.workunit.client.1.smithi084.stdout:4/71: mkdir d1/dd/d15 0 2024-02-28T23:07:25.327 INFO:tasks.workunit.client.1.smithi084.stdout:1/122: symlink d12/d1b/l29 0 2024-02-28T23:07:25.330 INFO:tasks.workunit.client.0.smithi057.stdout:2/13: rmdir d2 0 2024-02-28T23:07:25.331 INFO:tasks.workunit.client.1.smithi084.stdout:1/123: creat d12/d1b/d25/f2a x:0 0 0 2024-02-28T23:07:25.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/17: symlink d0/d5/l6 0 2024-02-28T23:07:25.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/18: fsync - no filename 2024-02-28T23:07:25.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/19: dwrite - no filename 2024-02-28T23:07:25.334 INFO:tasks.workunit.client.1.smithi084.stdout:1/124: mknod d12/d18/c2b 0 2024-02-28T23:07:25.335 INFO:tasks.workunit.client.1.smithi084.stdout:1/125: truncate fc 4934134 0 2024-02-28T23:07:25.336 INFO:tasks.workunit.client.0.smithi057.stdout:3/17: dwrite f1 [0,4194304] 0 2024-02-28T23:07:25.336 INFO:tasks.workunit.client.0.smithi057.stdout:2/14: symlink l3 0 2024-02-28T23:07:25.338 INFO:tasks.workunit.client.0.smithi057.stdout:7/20: creat d0/d5/f7 x:0 0 0 2024-02-28T23:07:25.341 INFO:tasks.workunit.client.0.smithi057.stdout:4/24: dread f3 [0,4194304] 0 2024-02-28T23:07:25.341 INFO:tasks.workunit.client.1.smithi084.stdout:6/112: write d1/fe [6504555,45990] 0 2024-02-28T23:07:25.342 INFO:tasks.workunit.client.0.smithi057.stdout:2/15: rename l3 to l4 0 2024-02-28T23:07:25.343 INFO:tasks.workunit.client.1.smithi084.stdout:7/112: sync 2024-02-28T23:07:25.343 INFO:tasks.workunit.client.1.smithi084.stdout:8/76: sync 2024-02-28T23:07:25.343 INFO:tasks.workunit.client.1.smithi084.stdout:5/93: sync 2024-02-28T23:07:25.343 INFO:tasks.workunit.client.1.smithi084.stdout:4/72: dwrite d1/f7 [0,4194304] 0 2024-02-28T23:07:25.343 INFO:tasks.workunit.client.0.smithi057.stdout:7/21: mkdir d0/d8 0 2024-02-28T23:07:25.345 INFO:tasks.workunit.client.0.smithi057.stdout:2/16: creat f5 x:0 0 0 2024-02-28T23:07:25.347 INFO:tasks.workunit.client.0.smithi057.stdout:7/22: write d0/d5/f7 [971016,23842] 0 2024-02-28T23:07:25.349 INFO:tasks.workunit.client.1.smithi084.stdout:7/113: rename db/d10/l13 to db/d10/d17/l1f 0 2024-02-28T23:07:25.350 INFO:tasks.workunit.client.1.smithi084.stdout:8/77: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:25.350 INFO:tasks.workunit.client.0.smithi057.stdout:2/17: rename f1 to f6 0 2024-02-28T23:07:25.351 INFO:tasks.workunit.client.1.smithi084.stdout:6/113: rmdir d1/d13/d22 0 2024-02-28T23:07:25.352 INFO:tasks.workunit.client.0.smithi057.stdout:7/23: truncate d0/d5/f7 1934775 0 2024-02-28T23:07:25.353 INFO:tasks.workunit.client.1.smithi084.stdout:6/114: chown d1/d9/d14 5377 1 2024-02-28T23:07:25.356 INFO:tasks.workunit.client.1.smithi084.stdout:8/78: mknod d0/c1e 0 2024-02-28T23:07:25.360 INFO:tasks.workunit.client.1.smithi084.stdout:7/114: write db/fd [5029879,34443] 0 2024-02-28T23:07:25.360 INFO:tasks.workunit.client.1.smithi084.stdout:6/115: unlink d1/d9/f18 0 2024-02-28T23:07:25.362 INFO:tasks.workunit.client.0.smithi057.stdout:2/18: link f0 f7 0 2024-02-28T23:07:25.363 INFO:tasks.workunit.client.0.smithi057.stdout:7/24: creat d0/d8/f9 x:0 0 0 2024-02-28T23:07:25.364 INFO:tasks.workunit.client.1.smithi084.stdout:1/126: getdents d12 0 2024-02-28T23:07:25.365 INFO:tasks.workunit.client.1.smithi084.stdout:7/115: chown f9 1512992 1 2024-02-28T23:07:25.365 INFO:tasks.workunit.client.1.smithi084.stdout:6/116: dread - d1/d9/f21 zero size 2024-02-28T23:07:25.367 INFO:tasks.workunit.client.0.smithi057.stdout:4/25: fsync f8 0 2024-02-28T23:07:25.367 INFO:tasks.workunit.client.1.smithi084.stdout:6/117: chown d1/d13 30040112 1 2024-02-28T23:07:25.368 INFO:tasks.workunit.client.0.smithi057.stdout:4/26: stat f1 0 2024-02-28T23:07:25.369 INFO:tasks.workunit.client.0.smithi057.stdout:4/27: fsync f0 0 2024-02-28T23:07:25.370 INFO:tasks.workunit.client.1.smithi084.stdout:1/127: write d12/f16 [170527,104432] 0 2024-02-28T23:07:25.373 INFO:tasks.workunit.client.0.smithi057.stdout:7/25: symlink d0/d8/la 0 2024-02-28T23:07:25.374 INFO:tasks.workunit.client.1.smithi084.stdout:7/116: link db/d11/c14 db/d10/c20 0 2024-02-28T23:07:25.375 INFO:tasks.workunit.client.0.smithi057.stdout:7/26: write d0/d5/f7 [290083,65551] 0 2024-02-28T23:07:25.379 INFO:tasks.workunit.client.1.smithi084.stdout:5/94: dwrite dc/de/df/d16/f1b [0,4194304] 0 2024-02-28T23:07:25.380 INFO:tasks.workunit.client.1.smithi084.stdout:1/128: rename d12/f13 to d12/d1b/f2c 0 2024-02-28T23:07:25.384 INFO:tasks.workunit.client.1.smithi084.stdout:5/95: symlink dc/l1e 0 2024-02-28T23:07:25.386 INFO:tasks.workunit.client.0.smithi057.stdout:2/19: truncate f7 191072 0 2024-02-28T23:07:25.387 INFO:tasks.workunit.client.1.smithi084.stdout:7/117: write db/d11/f15 [1657309,119257] 0 2024-02-28T23:07:25.389 INFO:tasks.workunit.client.0.smithi057.stdout:2/20: stat l4 0 2024-02-28T23:07:25.390 INFO:tasks.workunit.client.0.smithi057.stdout:4/28: truncate f3 2878148 0 2024-02-28T23:07:25.391 INFO:tasks.workunit.client.1.smithi084.stdout:4/73: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:25.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/21: mkdir d8 0 2024-02-28T23:07:25.397 INFO:tasks.workunit.client.1.smithi084.stdout:8/79: dwrite d0/fe [0,4194304] 0 2024-02-28T23:07:25.400 INFO:tasks.workunit.client.0.smithi057.stdout:7/27: dread d0/d5/f7 [0,4194304] 0 2024-02-28T23:07:25.408 INFO:tasks.workunit.client.1.smithi084.stdout:4/74: creat d1/dd/d15/f16 x:0 0 0 2024-02-28T23:07:25.409 INFO:tasks.workunit.client.1.smithi084.stdout:6/118: dwrite d1/d9/f21 [0,4194304] 0 2024-02-28T23:07:25.409 INFO:tasks.workunit.client.1.smithi084.stdout:5/96: rename dc/de/df/d16/f1d to dc/de/df/d16/f1f 0 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:6/119: write d1/d9/f15 [3108843,115378] 0 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:4/75: write d1/dd/d15/f16 [408713,111099] 0 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:2/82: sync 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:0/76: sync 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:3/76: sync 2024-02-28T23:07:25.414 INFO:tasks.workunit.client.1.smithi084.stdout:9/113: sync 2024-02-28T23:07:25.416 INFO:tasks.workunit.client.0.smithi057.stdout:4/29: dread f3 [0,4194304] 0 2024-02-28T23:07:25.416 INFO:tasks.workunit.client.1.smithi084.stdout:0/77: stat d1 0 2024-02-28T23:07:25.419 INFO:tasks.workunit.client.1.smithi084.stdout:2/83: write f2 [3649765,96557] 0 2024-02-28T23:07:25.419 INFO:tasks.workunit.client.1.smithi084.stdout:9/114: fdatasync d4/f1a 0 2024-02-28T23:07:25.420 INFO:tasks.workunit.client.0.smithi057.stdout:4/30: creat fa x:0 0 0 2024-02-28T23:07:25.422 INFO:tasks.workunit.client.1.smithi084.stdout:7/118: truncate db/d11/f15 1860993 0 2024-02-28T23:07:25.422 INFO:tasks.workunit.client.0.smithi057.stdout:2/22: write f7 [320146,109718] 0 2024-02-28T23:07:25.423 INFO:tasks.workunit.client.1.smithi084.stdout:5/97: truncate dc/de/df/d16/f1f 360527 0 2024-02-28T23:07:25.424 INFO:tasks.workunit.client.1.smithi084.stdout:0/78: chown d1/l15 77867 1 2024-02-28T23:07:25.425 INFO:tasks.workunit.client.0.smithi057.stdout:4/31: symlink lb 0 2024-02-28T23:07:25.425 INFO:tasks.workunit.client.1.smithi084.stdout:5/98: fdatasync f7 0 2024-02-28T23:07:25.425 INFO:tasks.workunit.client.1.smithi084.stdout:7/119: chown c0 244131 1 2024-02-28T23:07:25.428 INFO:tasks.workunit.client.1.smithi084.stdout:3/77: unlink d3/d8/fd 0 2024-02-28T23:07:25.429 INFO:tasks.workunit.client.1.smithi084.stdout:9/115: write d4/d6/d15/f1e [4850909,108003] 0 2024-02-28T23:07:25.430 INFO:tasks.workunit.client.1.smithi084.stdout:1/129: dwrite f11 [0,4194304] 0 2024-02-28T23:07:25.430 INFO:tasks.workunit.client.0.smithi057.stdout:2/23: unlink f6 0 2024-02-28T23:07:25.431 INFO:tasks.workunit.client.1.smithi084.stdout:6/120: rename d1/d9/d14/l17 to d1/d9/l23 0 2024-02-28T23:07:25.432 INFO:tasks.workunit.client.1.smithi084.stdout:8/80: dread d0/fe [0,4194304] 0 2024-02-28T23:07:25.432 INFO:tasks.workunit.client.1.smithi084.stdout:9/116: chown d4/f22 80621 1 2024-02-28T23:07:25.435 INFO:tasks.workunit.client.1.smithi084.stdout:0/79: fsync d1/d6/f19 0 2024-02-28T23:07:25.435 INFO:tasks.workunit.client.1.smithi084.stdout:9/117: dread - d4/f13 zero size 2024-02-28T23:07:25.437 INFO:tasks.workunit.client.0.smithi057.stdout:7/28: dwrite d0/d5/f7 [0,4194304] 0 2024-02-28T23:07:25.439 INFO:tasks.workunit.client.0.smithi057.stdout:2/24: mknod d8/c9 0 2024-02-28T23:07:25.440 INFO:tasks.workunit.client.0.smithi057.stdout:4/32: dread f1 [0,4194304] 0 2024-02-28T23:07:25.440 INFO:tasks.workunit.client.1.smithi084.stdout:6/121: unlink d1/d13/c1a 0 2024-02-28T23:07:25.446 INFO:tasks.workunit.client.1.smithi084.stdout:2/84: creat d5/f15 x:0 0 0 2024-02-28T23:07:25.447 INFO:tasks.workunit.client.0.smithi057.stdout:2/25: dread f7 [0,4194304] 0 2024-02-28T23:07:25.452 INFO:tasks.workunit.client.1.smithi084.stdout:3/78: creat d3/d8/d1a/f1f x:0 0 0 2024-02-28T23:07:25.452 INFO:tasks.workunit.client.1.smithi084.stdout:9/118: fdatasync d4/d6/ff 0 2024-02-28T23:07:25.454 INFO:tasks.workunit.client.0.smithi057.stdout:2/26: unlink f5 0 2024-02-28T23:07:25.454 INFO:tasks.workunit.client.0.smithi057.stdout:4/33: creat fc x:0 0 0 2024-02-28T23:07:25.454 INFO:tasks.workunit.client.1.smithi084.stdout:1/130: truncate d12/d18/f1c 3098113 0 2024-02-28T23:07:25.459 INFO:tasks.workunit.client.0.smithi057.stdout:4/34: creat fd x:0 0 0 2024-02-28T23:07:25.460 INFO:tasks.workunit.client.1.smithi084.stdout:1/131: chown d12/d1b/f2c 294 1 2024-02-28T23:07:25.460 INFO:tasks.workunit.client.1.smithi084.stdout:5/99: dwrite dc/de/df/d16/f1b [8388608,4194304] 0 2024-02-28T23:07:25.461 INFO:tasks.workunit.client.0.smithi057.stdout:2/27: unlink d8/c9 0 2024-02-28T23:07:25.461 INFO:tasks.workunit.client.1.smithi084.stdout:2/85: rename f4 to d5/dd/f16 0 2024-02-28T23:07:25.462 INFO:tasks.workunit.client.1.smithi084.stdout:7/120: read db/d11/f15 [1554557,120806] 0 2024-02-28T23:07:25.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/35: chown c6 561466 1 2024-02-28T23:07:25.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/36: stat f3 0 2024-02-28T23:07:25.465 INFO:tasks.workunit.client.0.smithi057.stdout:4/37: rmdir - no directory 2024-02-28T23:07:25.465 INFO:tasks.workunit.client.0.smithi057.stdout:2/28: dread f0 [0,4194304] 0 2024-02-28T23:07:25.466 INFO:tasks.workunit.client.1.smithi084.stdout:6/122: link d1/d9/f21 d1/f24 0 2024-02-28T23:07:25.466 INFO:tasks.workunit.client.1.smithi084.stdout:0/80: link d1/d6/db/c17 d1/c1a 0 2024-02-28T23:07:25.468 INFO:tasks.workunit.client.1.smithi084.stdout:1/132: symlink d12/d18/l2d 0 2024-02-28T23:07:25.471 INFO:tasks.workunit.client.1.smithi084.stdout:1/133: stat d12/d1b/c24 0 2024-02-28T23:07:25.472 INFO:tasks.workunit.client.1.smithi084.stdout:0/81: chown d1/d6/f13 15800525 1 2024-02-28T23:07:25.473 INFO:tasks.workunit.client.0.smithi057.stdout:4/38: rename fc to fe 0 2024-02-28T23:07:25.474 INFO:tasks.workunit.client.1.smithi084.stdout:5/100: creat dc/de/df/f20 x:0 0 0 2024-02-28T23:07:25.474 INFO:tasks.workunit.client.1.smithi084.stdout:7/121: write f9 [1169837,77512] 0 2024-02-28T23:07:25.475 INFO:tasks.workunit.client.0.smithi057.stdout:4/39: fsync f0 0 2024-02-28T23:07:25.476 INFO:tasks.workunit.client.1.smithi084.stdout:8/81: dwrite d0/fe [4194304,4194304] 0 2024-02-28T23:07:25.476 INFO:tasks.workunit.client.1.smithi084.stdout:5/101: chown dc/de/f12 13 1 2024-02-28T23:07:25.481 INFO:tasks.workunit.client.1.smithi084.stdout:1/134: mkdir d12/d1b/d2e 0 2024-02-28T23:07:25.482 INFO:tasks.workunit.client.0.smithi057.stdout:4/40: unlink f8 0 2024-02-28T23:07:25.482 INFO:tasks.workunit.client.0.smithi057.stdout:7/29: fdatasync d0/d5/f7 0 2024-02-28T23:07:25.483 INFO:tasks.workunit.client.1.smithi084.stdout:6/123: link d1/d9/l23 d1/l25 0 2024-02-28T23:07:25.486 INFO:tasks.workunit.client.1.smithi084.stdout:5/102: creat dc/f21 x:0 0 0 2024-02-28T23:07:25.487 INFO:tasks.workunit.client.1.smithi084.stdout:7/122: fdatasync f9 0 2024-02-28T23:07:25.487 INFO:tasks.workunit.client.0.smithi057.stdout:7/30: mknod d0/d5/cb 0 2024-02-28T23:07:25.488 INFO:tasks.workunit.client.1.smithi084.stdout:1/135: stat le 0 2024-02-28T23:07:25.489 INFO:tasks.workunit.client.1.smithi084.stdout:0/82: mkdir d1/d6/d1b 0 2024-02-28T23:07:25.489 INFO:tasks.workunit.client.0.smithi057.stdout:4/41: dread f0 [0,4194304] 0 2024-02-28T23:07:25.490 INFO:tasks.workunit.client.1.smithi084.stdout:7/123: chown db/d11 6486 1 2024-02-28T23:07:25.492 INFO:tasks.workunit.client.1.smithi084.stdout:6/124: write d1/fe [6854250,113028] 0 2024-02-28T23:07:25.493 INFO:tasks.workunit.client.1.smithi084.stdout:9/119: dwrite d4/d6/d15/f19 [0,4194304] 0 2024-02-28T23:07:25.493 INFO:tasks.workunit.client.1.smithi084.stdout:7/124: fdatasync f9 0 2024-02-28T23:07:25.494 INFO:tasks.workunit.client.1.smithi084.stdout:8/82: creat d0/d9/f1f x:0 0 0 2024-02-28T23:07:25.494 INFO:tasks.workunit.client.0.smithi057.stdout:4/42: write fd [826020,98133] 0 2024-02-28T23:07:25.496 INFO:tasks.workunit.client.1.smithi084.stdout:5/103: rmdir dc/de/df/d16 39 2024-02-28T23:07:25.498 INFO:tasks.workunit.client.0.smithi057.stdout:2/29: dwrite f7 [0,4194304] 0 2024-02-28T23:07:25.498 INFO:tasks.workunit.client.1.smithi084.stdout:5/104: fdatasync dc/de/df/f20 0 2024-02-28T23:07:25.500 INFO:tasks.workunit.client.0.smithi057.stdout:2/30: chown l4 39 1 2024-02-28T23:07:25.505 INFO:tasks.workunit.client.1.smithi084.stdout:2/86: rmdir d5/dd 39 2024-02-28T23:07:25.507 INFO:tasks.workunit.client.0.smithi057.stdout:4/43: rename c5 to cf 0 2024-02-28T23:07:25.507 INFO:tasks.workunit.client.1.smithi084.stdout:0/83: creat d1/d8/f1c x:0 0 0 2024-02-28T23:07:25.511 INFO:tasks.workunit.client.1.smithi084.stdout:8/83: mkdir d0/d9/d1b/d20 0 2024-02-28T23:07:25.512 INFO:tasks.workunit.client.0.smithi057.stdout:4/44: dread f0 [0,4194304] 0 2024-02-28T23:07:25.514 INFO:tasks.workunit.client.1.smithi084.stdout:7/125: dread db/d11/f12 [0,4194304] 0 2024-02-28T23:07:25.515 INFO:tasks.workunit.client.1.smithi084.stdout:1/136: link l5 d12/d1b/d25/l2f 0 2024-02-28T23:07:25.519 INFO:tasks.workunit.client.1.smithi084.stdout:6/125: unlink d1/d9/l23 0 2024-02-28T23:07:25.520 INFO:tasks.workunit.client.0.smithi057.stdout:4/45: rename c4 to c10 0 2024-02-28T23:07:25.520 INFO:tasks.workunit.client.0.smithi057.stdout:4/46: stat fa 0 2024-02-28T23:07:25.520 INFO:tasks.workunit.client.1.smithi084.stdout:1/137: chown d12/l1f 6341 1 2024-02-28T23:07:25.521 INFO:tasks.workunit.client.1.smithi084.stdout:9/120: link d4/l16 d4/d1d/d1f/l24 0 2024-02-28T23:07:25.523 INFO:tasks.workunit.client.1.smithi084.stdout:5/105: dwrite dc/de/f13 [4194304,4194304] 0 2024-02-28T23:07:25.523 INFO:tasks.workunit.client.0.smithi057.stdout:4/47: rename f3 to f11 0 2024-02-28T23:07:25.526 INFO:tasks.workunit.client.0.smithi057.stdout:7/31: dwrite d0/d8/f9 [0,4194304] 0 2024-02-28T23:07:25.526 INFO:tasks.workunit.client.1.smithi084.stdout:0/84: dread d1/fc [0,4194304] 0 2024-02-28T23:07:25.527 INFO:tasks.workunit.client.1.smithi084.stdout:2/87: rename d5/d7/f8 to d5/f17 0 2024-02-28T23:07:25.528 INFO:tasks.workunit.client.1.smithi084.stdout:5/106: write dc/de/f12 [870734,88186] 0 2024-02-28T23:07:25.530 INFO:tasks.workunit.client.0.smithi057.stdout:4/48: rename f1 to f12 0 2024-02-28T23:07:25.531 INFO:tasks.workunit.client.1.smithi084.stdout:9/121: mknod d4/d6/c25 0 2024-02-28T23:07:25.531 INFO:tasks.workunit.client.1.smithi084.stdout:8/84: rename d0/d9/f1f to d0/d9/f21 0 2024-02-28T23:07:25.532 INFO:tasks.workunit.client.1.smithi084.stdout:7/126: rename c0 to db/d10/d17/c21 0 2024-02-28T23:07:25.534 INFO:tasks.workunit.client.0.smithi057.stdout:4/49: mknod c13 0 2024-02-28T23:07:25.535 INFO:tasks.workunit.client.1.smithi084.stdout:0/85: creat d1/d8/f1d x:0 0 0 2024-02-28T23:07:25.538 INFO:tasks.workunit.client.1.smithi084.stdout:5/107: symlink dc/de/l22 0 2024-02-28T23:07:25.538 INFO:tasks.workunit.client.1.smithi084.stdout:0/86: readlink d1/l15 0 2024-02-28T23:07:25.540 INFO:tasks.workunit.client.1.smithi084.stdout:2/88: write d5/dd/ff [281733,77269] 0 2024-02-28T23:07:25.541 INFO:tasks.workunit.client.1.smithi084.stdout:0/87: chown d1/l15 0 1 2024-02-28T23:07:25.544 INFO:tasks.workunit.client.1.smithi084.stdout:2/89: fdatasync d5/d7/fe 0 2024-02-28T23:07:25.545 INFO:tasks.workunit.client.1.smithi084.stdout:1/138: dread d12/f22 [0,4194304] 0 2024-02-28T23:07:25.545 INFO:tasks.workunit.client.0.smithi057.stdout:7/32: dread d0/d5/f7 [0,4194304] 0 2024-02-28T23:07:25.549 INFO:tasks.workunit.client.1.smithi084.stdout:3/79: truncate d3/d8/d14/f16 1780983 0 2024-02-28T23:07:25.550 INFO:tasks.workunit.client.1.smithi084.stdout:1/139: chown d12/f28 33116 1 2024-02-28T23:07:25.554 INFO:tasks.workunit.client.1.smithi084.stdout:2/90: write d5/f15 [594629,97094] 0 2024-02-28T23:07:25.554 INFO:tasks.workunit.client.1.smithi084.stdout:6/126: dwrite d1/f24 [0,4194304] 0 2024-02-28T23:07:25.563 INFO:tasks.workunit.client.1.smithi084.stdout:0/88: rename d1/d6/db/c10 to d1/d6/c1e 0 2024-02-28T23:07:25.565 INFO:tasks.workunit.client.1.smithi084.stdout:3/80: rmdir d3/d8/df 39 2024-02-28T23:07:25.568 INFO:tasks.workunit.client.1.smithi084.stdout:2/91: mknod d5/d7/d12/c18 0 2024-02-28T23:07:25.570 INFO:tasks.workunit.client.0.smithi057.stdout:2/31: fdatasync f0 0 2024-02-28T23:07:25.570 INFO:tasks.workunit.client.1.smithi084.stdout:8/85: dread d0/fe [0,4194304] 0 2024-02-28T23:07:25.573 INFO:tasks.workunit.client.1.smithi084.stdout:3/81: dread - d3/d8/df/f12 zero size 2024-02-28T23:07:25.574 INFO:tasks.workunit.client.1.smithi084.stdout:2/92: fdatasync d5/d7/d12/f13 0 2024-02-28T23:07:25.574 INFO:tasks.workunit.client.1.smithi084.stdout:0/89: creat d1/d8/f1f x:0 0 0 2024-02-28T23:07:25.575 INFO:tasks.workunit.client.0.smithi057.stdout:2/32: creat d8/fa x:0 0 0 2024-02-28T23:07:25.576 INFO:tasks.workunit.client.0.smithi057.stdout:7/33: dwrite d0/d8/f9 [0,4194304] 0 2024-02-28T23:07:25.577 INFO:tasks.workunit.client.1.smithi084.stdout:7/127: dwrite f9 [0,4194304] 0 2024-02-28T23:07:25.578 INFO:tasks.workunit.client.0.smithi057.stdout:2/33: creat d8/fb x:0 0 0 2024-02-28T23:07:25.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:25 smithi084 ceph-mon[34826]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:25.580 INFO:tasks.workunit.client.0.smithi057.stdout:2/34: chown d8/fb 572 1 2024-02-28T23:07:25.581 INFO:tasks.workunit.client.1.smithi084.stdout:5/108: dwrite dc/de/f13 [4194304,4194304] 0 2024-02-28T23:07:25.585 INFO:tasks.workunit.client.0.smithi057.stdout:2/35: symlink d8/lc 0 2024-02-28T23:07:25.587 INFO:tasks.workunit.client.1.smithi084.stdout:3/82: write d3/fb [3892757,19885] 0 2024-02-28T23:07:25.587 INFO:tasks.workunit.client.1.smithi084.stdout:7/128: write f9 [1946077,72407] 0 2024-02-28T23:07:25.588 INFO:tasks.workunit.client.1.smithi084.stdout:7/129: readlink db/le 0 2024-02-28T23:07:25.591 INFO:tasks.workunit.client.0.smithi057.stdout:7/34: getdents d0/d8 0 2024-02-28T23:07:25.592 INFO:tasks.workunit.client.1.smithi084.stdout:1/140: dread d12/f15 [0,4194304] 0 2024-02-28T23:07:25.592 INFO:tasks.workunit.client.1.smithi084.stdout:6/127: dwrite d1/d9/f21 [4194304,4194304] 0 2024-02-28T23:07:25.593 INFO:tasks.workunit.client.1.smithi084.stdout:8/86: write d0/d9/f21 [749137,65588] 0 2024-02-28T23:07:25.601 INFO:tasks.workunit.client.1.smithi084.stdout:2/93: dread d5/f10 [0,4194304] 0 2024-02-28T23:07:25.602 INFO:tasks.workunit.client.1.smithi084.stdout:2/94: chown d5/l6 583 1 2024-02-28T23:07:25.603 INFO:tasks.workunit.client.1.smithi084.stdout:2/95: stat d5/dd/f16 0 2024-02-28T23:07:25.604 INFO:tasks.workunit.client.1.smithi084.stdout:3/83: mknod d3/d8/df/c20 0 2024-02-28T23:07:25.607 INFO:tasks.workunit.client.1.smithi084.stdout:0/90: chown d1/d6/db/c17 13924170 1 2024-02-28T23:07:25.608 INFO:tasks.workunit.client.0.smithi057.stdout:2/36: dwrite f7 [4194304,4194304] 0 2024-02-28T23:07:25.609 INFO:tasks.workunit.client.1.smithi084.stdout:3/84: read d3/f7 [554876,15464] 0 2024-02-28T23:07:25.610 INFO:tasks.workunit.client.0.smithi057.stdout:7/35: dread d0/d8/f9 [0,4194304] 0 2024-02-28T23:07:25.610 INFO:tasks.workunit.client.1.smithi084.stdout:8/87: mknod d0/d9/d1b/c22 0 2024-02-28T23:07:25.612 INFO:tasks.workunit.client.0.smithi057.stdout:2/37: write d8/fa [278233,109739] 0 2024-02-28T23:07:25.612 INFO:tasks.workunit.client.1.smithi084.stdout:3/85: fsync d3/d8/d1a/f1f 0 2024-02-28T23:07:25.614 INFO:tasks.workunit.client.0.smithi057.stdout:7/36: symlink d0/d8/lc 0 2024-02-28T23:07:25.614 INFO:tasks.workunit.client.1.smithi084.stdout:5/109: mknod dc/de/df/d16/c23 0 2024-02-28T23:07:25.615 INFO:tasks.workunit.client.0.smithi057.stdout:2/38: creat d8/fd x:0 0 0 2024-02-28T23:07:25.619 INFO:tasks.workunit.client.0.smithi057.stdout:2/39: read d8/fa [323268,117996] 0 2024-02-28T23:07:25.622 INFO:tasks.workunit.client.1.smithi084.stdout:2/96: unlink d5/d7/d12/f13 0 2024-02-28T23:07:25.626 INFO:tasks.workunit.client.0.smithi057.stdout:7/37: rename d0/d5/f7 to d0/d8/fd 0 2024-02-28T23:07:25.627 INFO:tasks.workunit.client.1.smithi084.stdout:2/97: write d5/d7/d9/f11 [763049,69154] 0 2024-02-28T23:07:25.637 INFO:tasks.workunit.client.1.smithi084.stdout:8/88: creat d0/d11/f23 x:0 0 0 2024-02-28T23:07:25.638 INFO:tasks.workunit.client.1.smithi084.stdout:5/110: mknod dc/de/c24 0 2024-02-28T23:07:25.639 INFO:tasks.workunit.client.1.smithi084.stdout:2/98: fsync f3 0 2024-02-28T23:07:25.640 INFO:tasks.workunit.client.1.smithi084.stdout:6/128: read - d1/f1b zero size 2024-02-28T23:07:25.644 INFO:tasks.workunit.client.1.smithi084.stdout:7/130: creat db/f22 x:0 0 0 2024-02-28T23:07:25.646 INFO:tasks.workunit.client.1.smithi084.stdout:4/76: sync 2024-02-28T23:07:25.647 INFO:tasks.workunit.client.0.smithi057.stdout:7/38: dwrite d0/d8/f9 [0,4194304] 0 2024-02-28T23:07:25.649 INFO:tasks.workunit.client.1.smithi084.stdout:2/99: dread d5/dd/f16 [0,4194304] 0 2024-02-28T23:07:25.650 INFO:tasks.workunit.client.0.smithi057.stdout:2/40: rmdir d8 39 2024-02-28T23:07:25.651 INFO:tasks.workunit.client.1.smithi084.stdout:2/100: fdatasync d5/f15 0 2024-02-28T23:07:25.651 INFO:tasks.workunit.client.0.smithi057.stdout:7/39: write d0/d8/f9 [810309,20353] 0 2024-02-28T23:07:25.654 INFO:tasks.workunit.client.1.smithi084.stdout:5/111: fdatasync f7 0 2024-02-28T23:07:25.655 INFO:tasks.workunit.client.1.smithi084.stdout:6/129: read - d1/f1b zero size 2024-02-28T23:07:25.660 INFO:tasks.workunit.client.0.smithi057.stdout:7/40: write d0/d8/fd [2954448,120625] 0 2024-02-28T23:07:25.661 INFO:tasks.workunit.client.0.smithi057.stdout:2/41: creat d8/fe x:0 0 0 2024-02-28T23:07:25.667 INFO:tasks.workunit.client.1.smithi084.stdout:1/141: dwrite d12/d18/f1c [0,4194304] 0 2024-02-28T23:07:25.669 INFO:tasks.workunit.client.1.smithi084.stdout:4/77: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:25.670 INFO:tasks.workunit.client.0.smithi057.stdout:2/42: rmdir d8 39 2024-02-28T23:07:25.671 INFO:tasks.workunit.client.0.smithi057.stdout:7/41: truncate d0/d8/f9 425795 0 2024-02-28T23:07:25.673 INFO:tasks.workunit.client.0.smithi057.stdout:2/43: dread - d8/fe zero size 2024-02-28T23:07:25.679 INFO:tasks.workunit.client.1.smithi084.stdout:4/78: mkdir d1/dd/d14/d17 0 2024-02-28T23:07:25.683 INFO:tasks.workunit.client.1.smithi084.stdout:6/130: rename d1/fe to d1/d9/d1d/d1e/f26 0 2024-02-28T23:07:25.684 INFO:tasks.workunit.client.1.smithi084.stdout:6/131: write d1/d9/f15 [1688207,32268] 0 2024-02-28T23:07:25.686 INFO:tasks.workunit.client.1.smithi084.stdout:1/142: rename d12/d18 to d12/d1b/d2e/d30 0 2024-02-28T23:07:25.687 INFO:tasks.workunit.client.1.smithi084.stdout:6/132: readlink d1/l12 0 2024-02-28T23:07:25.690 INFO:tasks.workunit.client.1.smithi084.stdout:4/79: rename d1/dd/d14/d17 to d1/d18 0 2024-02-28T23:07:25.691 INFO:tasks.workunit.client.1.smithi084.stdout:6/133: chown d1/cc 10 1 2024-02-28T23:07:25.692 INFO:tasks.workunit.client.1.smithi084.stdout:2/101: dwrite d5/f10 [0,4194304] 0 2024-02-28T23:07:25.692 INFO:tasks.workunit.client.1.smithi084.stdout:4/80: dread d1/f6 [0,4194304] 0 2024-02-28T23:07:25.694 INFO:tasks.workunit.client.1.smithi084.stdout:2/102: read d5/dd/f16 [2755407,108372] 0 2024-02-28T23:07:25.694 INFO:tasks.workunit.client.1.smithi084.stdout:4/81: write d1/f12 [425862,53207] 0 2024-02-28T23:07:25.696 INFO:tasks.workunit.client.1.smithi084.stdout:2/103: stat d5/d7 0 2024-02-28T23:07:25.696 INFO:tasks.workunit.client.1.smithi084.stdout:6/134: creat d1/d9/d14/f27 x:0 0 0 2024-02-28T23:07:25.696 INFO:tasks.workunit.client.1.smithi084.stdout:1/143: fdatasync d12/f22 0 2024-02-28T23:07:25.698 INFO:tasks.workunit.client.1.smithi084.stdout:4/82: dread - d1/dd/f11 zero size 2024-02-28T23:07:25.700 INFO:tasks.workunit.client.1.smithi084.stdout:8/89: getdents d0/d11 0 2024-02-28T23:07:25.700 INFO:tasks.workunit.client.1.smithi084.stdout:4/83: dread - d1/dd/f11 zero size 2024-02-28T23:07:25.701 INFO:tasks.workunit.client.1.smithi084.stdout:2/104: rmdir d5 39 2024-02-28T23:07:25.702 INFO:tasks.workunit.client.0.smithi057.stdout:7/42: dwrite d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.702 INFO:tasks.workunit.client.0.smithi057.stdout:2/44: dwrite d8/fa [0,4194304] 0 2024-02-28T23:07:25.703 INFO:tasks.workunit.client.1.smithi084.stdout:1/144: creat d12/d1b/f31 x:0 0 0 2024-02-28T23:07:25.705 INFO:tasks.workunit.client.1.smithi084.stdout:6/135: rename d1/d9/d14/f27 to d1/d9/f28 0 2024-02-28T23:07:25.707 INFO:tasks.workunit.client.0.smithi057.stdout:2/45: creat d8/ff x:0 0 0 2024-02-28T23:07:25.707 INFO:tasks.workunit.client.1.smithi084.stdout:8/90: fsync d0/fe 0 2024-02-28T23:07:25.709 INFO:tasks.workunit.client.1.smithi084.stdout:4/84: symlink d1/dd/l19 0 2024-02-28T23:07:25.712 INFO:tasks.workunit.client.1.smithi084.stdout:7/131: truncate db/fd 3267830 0 2024-02-28T23:07:25.713 INFO:tasks.workunit.client.1.smithi084.stdout:1/145: unlink d12/l1f 0 2024-02-28T23:07:25.717 INFO:tasks.workunit.client.1.smithi084.stdout:5/112: truncate dc/de/f12 806247 0 2024-02-28T23:07:25.719 INFO:tasks.workunit.client.1.smithi084.stdout:2/105: creat d5/d7/f19 x:0 0 0 2024-02-28T23:07:25.725 INFO:tasks.workunit.client.1.smithi084.stdout:8/91: mkdir d0/d24 0 2024-02-28T23:07:25.725 INFO:tasks.workunit.client.1.smithi084.stdout:2/106: chown d5/d7/d9/c14 1 1 2024-02-28T23:07:25.726 INFO:tasks.workunit.client.1.smithi084.stdout:1/146: mkdir d12/d1b/d2e/d32 0 2024-02-28T23:07:25.726 INFO:tasks.workunit.client.1.smithi084.stdout:2/107: chown d5/f15 63486 1 2024-02-28T23:07:25.726 INFO:tasks.workunit.client.1.smithi084.stdout:8/92: chown d0/d9/d1b/c1c 467 1 2024-02-28T23:07:25.726 INFO:tasks.workunit.client.0.smithi057.stdout:7/43: dread d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.729 INFO:tasks.workunit.client.0.smithi057.stdout:7/44: unlink d0/d5/cb 0 2024-02-28T23:07:25.729 INFO:tasks.workunit.client.1.smithi084.stdout:4/85: creat d1/dd/d15/f1a x:0 0 0 2024-02-28T23:07:25.730 INFO:tasks.workunit.client.0.smithi057.stdout:2/46: fsync d8/ff 0 2024-02-28T23:07:25.730 INFO:tasks.workunit.client.1.smithi084.stdout:6/136: dwrite d1/d9/f15 [0,4194304] 0 2024-02-28T23:07:25.731 INFO:tasks.workunit.client.1.smithi084.stdout:8/93: chown d0/d9/d1b/c1c 1156938359 1 2024-02-28T23:07:25.732 INFO:tasks.workunit.client.0.smithi057.stdout:7/45: write d0/d8/fd [835387,92464] 0 2024-02-28T23:07:25.733 INFO:tasks.workunit.client.0.smithi057.stdout:2/47: creat d8/f10 x:0 0 0 2024-02-28T23:07:25.734 INFO:tasks.workunit.client.0.smithi057.stdout:2/48: fdatasync f0 0 2024-02-28T23:07:25.734 INFO:tasks.workunit.client.1.smithi084.stdout:8/94: fdatasync d0/fe 0 2024-02-28T23:07:25.735 INFO:tasks.workunit.client.1.smithi084.stdout:4/86: symlink d1/dd/d15/l1b 0 2024-02-28T23:07:25.737 INFO:tasks.workunit.client.1.smithi084.stdout:6/137: unlink d1/d9/f15 0 2024-02-28T23:07:25.741 INFO:tasks.workunit.client.0.smithi057.stdout:2/49: unlink d8/f10 0 2024-02-28T23:07:25.741 INFO:tasks.workunit.client.1.smithi084.stdout:6/138: chown d1/f1b 291714 1 2024-02-28T23:07:25.742 INFO:tasks.workunit.client.1.smithi084.stdout:2/108: link d5/d7/d9/cc d5/dd/c1a 0 2024-02-28T23:07:25.743 INFO:tasks.workunit.client.1.smithi084.stdout:1/147: dread f9 [0,4194304] 0 2024-02-28T23:07:25.744 INFO:tasks.workunit.client.0.smithi057.stdout:2/50: mkdir d8/d11 0 2024-02-28T23:07:25.745 INFO:tasks.workunit.client.1.smithi084.stdout:5/113: dwrite dc/de/df/f20 [0,4194304] 0 2024-02-28T23:07:25.746 INFO:tasks.workunit.client.1.smithi084.stdout:1/148: dread - d12/f23 zero size 2024-02-28T23:07:25.748 INFO:tasks.workunit.client.1.smithi084.stdout:5/114: creat dc/f25 x:0 0 0 2024-02-28T23:07:25.749 INFO:tasks.workunit.client.0.smithi057.stdout:7/46: dread d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.750 INFO:tasks.workunit.client.0.smithi057.stdout:2/51: creat d8/d11/f12 x:0 0 0 2024-02-28T23:07:25.751 INFO:tasks.workunit.client.1.smithi084.stdout:6/139: getdents d1 0 2024-02-28T23:07:25.753 INFO:tasks.workunit.client.0.smithi057.stdout:2/52: symlink d8/l13 0 2024-02-28T23:07:25.755 INFO:tasks.workunit.client.0.smithi057.stdout:7/47: mknod d0/d5/ce 0 2024-02-28T23:07:25.757 INFO:tasks.workunit.client.1.smithi084.stdout:6/140: symlink d1/d13/l29 0 2024-02-28T23:07:25.758 INFO:tasks.workunit.client.0.smithi057.stdout:7/48: chown d0/d8/fd 365690 1 2024-02-28T23:07:25.760 INFO:tasks.workunit.client.0.smithi057.stdout:2/53: rename l4 to d8/l14 0 2024-02-28T23:07:25.760 INFO:tasks.workunit.client.1.smithi084.stdout:1/149: symlink d12/d1b/d2e/d32/l33 0 2024-02-28T23:07:25.761 INFO:tasks.workunit.client.1.smithi084.stdout:5/115: dread f7 [4194304,4194304] 0 2024-02-28T23:07:25.762 INFO:tasks.workunit.client.1.smithi084.stdout:1/150: write f9 [784459,95028] 0 2024-02-28T23:07:25.764 INFO:tasks.workunit.client.0.smithi057.stdout:7/49: rename d0/d8/f9 to d0/d5/ff 0 2024-02-28T23:07:25.765 INFO:tasks.workunit.client.1.smithi084.stdout:1/151: creat d12/f34 x:0 0 0 2024-02-28T23:07:25.765 INFO:tasks.workunit.client.0.smithi057.stdout:2/54: mkdir d8/d11/d15 0 2024-02-28T23:07:25.766 INFO:tasks.workunit.client.1.smithi084.stdout:5/116: stat dc/de/df/l19 0 2024-02-28T23:07:25.767 INFO:tasks.workunit.client.0.smithi057.stdout:7/50: stat d0/d8/fd 0 2024-02-28T23:07:25.770 INFO:tasks.workunit.client.0.smithi057.stdout:2/55: creat d8/f16 x:0 0 0 2024-02-28T23:07:25.770 INFO:tasks.workunit.client.1.smithi084.stdout:4/87: truncate d1/dd/fe 221079 0 2024-02-28T23:07:25.770 INFO:tasks.workunit.client.1.smithi084.stdout:4/88: chown d1 154768925 1 2024-02-28T23:07:25.771 INFO:tasks.workunit.client.1.smithi084.stdout:5/117: write f7 [7879271,83578] 0 2024-02-28T23:07:25.773 INFO:tasks.workunit.client.0.smithi057.stdout:2/56: creat d8/d11/f17 x:0 0 0 2024-02-28T23:07:25.774 INFO:tasks.workunit.client.1.smithi084.stdout:5/118: chown dc/de/df/d16/f1f 841512 1 2024-02-28T23:07:25.774 INFO:tasks.workunit.client.1.smithi084.stdout:4/89: truncate d1/dd/d15/f16 650878 0 2024-02-28T23:07:25.775 INFO:tasks.workunit.client.0.smithi057.stdout:2/57: write f0 [3190400,104716] 0 2024-02-28T23:07:25.777 INFO:tasks.workunit.client.0.smithi057.stdout:2/58: readlink d8/lc 0 2024-02-28T23:07:25.778 INFO:tasks.workunit.client.1.smithi084.stdout:5/119: truncate dc/f21 886725 0 2024-02-28T23:07:25.778 INFO:tasks.workunit.client.1.smithi084.stdout:4/90: write d1/f5 [2173235,64282] 0 2024-02-28T23:07:25.780 INFO:tasks.workunit.client.0.smithi057.stdout:2/59: creat d8/d11/f18 x:0 0 0 2024-02-28T23:07:25.782 INFO:tasks.workunit.client.0.smithi057.stdout:2/60: write d8/ff [146847,95131] 0 2024-02-28T23:07:25.782 INFO:tasks.workunit.client.1.smithi084.stdout:5/120: creat dc/de/df/d16/f26 x:0 0 0 2024-02-28T23:07:25.783 INFO:tasks.workunit.client.1.smithi084.stdout:6/141: dwrite d1/d9/f28 [0,4194304] 0 2024-02-28T23:07:25.784 INFO:tasks.workunit.client.0.smithi057.stdout:2/61: readlink d8/l13 0 2024-02-28T23:07:25.784 INFO:tasks.workunit.client.0.smithi057.stdout:7/51: dread d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.787 INFO:tasks.workunit.client.0.smithi057.stdout:7/52: readlink d0/d5/l6 0 2024-02-28T23:07:25.787 INFO:tasks.workunit.client.1.smithi084.stdout:6/142: symlink d1/d9/d14/l2a 0 2024-02-28T23:07:25.789 INFO:tasks.workunit.client.1.smithi084.stdout:1/152: dwrite d12/d1b/d2e/d30/f1d [0,4194304] 0 2024-02-28T23:07:25.790 INFO:tasks.workunit.client.1.smithi084.stdout:1/153: chown le 61723550 1 2024-02-28T23:07:25.790 INFO:tasks.workunit.client.1.smithi084.stdout:6/143: mkdir d1/d9/d1d/d1e/d2b 0 2024-02-28T23:07:25.792 INFO:tasks.workunit.client.0.smithi057.stdout:7/53: creat d0/f10 x:0 0 0 2024-02-28T23:07:25.794 INFO:tasks.workunit.client.1.smithi084.stdout:6/144: mkdir d1/d9/d1d/d1e/d2b/d2c 0 2024-02-28T23:07:25.794 INFO:tasks.workunit.client.1.smithi084.stdout:1/154: getdents d12/d1b/d2e 0 2024-02-28T23:07:25.795 INFO:tasks.workunit.client.1.smithi084.stdout:6/145: write d1/f1b [390588,90998] 0 2024-02-28T23:07:25.796 INFO:tasks.workunit.client.1.smithi084.stdout:1/155: symlink d12/d1b/d2e/d30/l35 0 2024-02-28T23:07:25.797 INFO:tasks.workunit.client.1.smithi084.stdout:6/146: truncate d1/f1b 899294 0 2024-02-28T23:07:25.798 INFO:tasks.workunit.client.1.smithi084.stdout:1/156: symlink d12/d1b/l36 0 2024-02-28T23:07:25.799 INFO:tasks.workunit.client.1.smithi084.stdout:6/147: fsync d1/d9/d1d/d1e/f26 0 2024-02-28T23:07:25.800 INFO:tasks.workunit.client.1.smithi084.stdout:9/122: sync 2024-02-28T23:07:25.800 INFO:tasks.workunit.client.0.smithi057.stdout:7/54: stat d0/c3 0 2024-02-28T23:07:25.800 INFO:tasks.workunit.client.1.smithi084.stdout:6/148: write d1/f1b [91250,26537] 0 2024-02-28T23:07:25.801 INFO:tasks.workunit.client.1.smithi084.stdout:9/123: readlink d4/d6/l1b 0 2024-02-28T23:07:25.804 INFO:tasks.workunit.client.1.smithi084.stdout:9/124: symlink d4/d1d/d1f/l26 0 2024-02-28T23:07:25.807 INFO:tasks.workunit.client.0.smithi057.stdout:7/55: symlink d0/d5/l11 0 2024-02-28T23:07:25.810 INFO:tasks.workunit.client.0.smithi057.stdout:7/56: creat d0/f12 x:0 0 0 2024-02-28T23:07:25.814 INFO:tasks.workunit.client.1.smithi084.stdout:1/157: dread d12/d1b/d2e/d30/f1c [0,4194304] 0 2024-02-28T23:07:25.816 INFO:tasks.workunit.client.1.smithi084.stdout:9/125: dread d4/f22 [0,4194304] 0 2024-02-28T23:07:25.817 INFO:tasks.workunit.client.0.smithi057.stdout:7/57: fsync d0/d5/ff 0 2024-02-28T23:07:25.818 INFO:tasks.workunit.client.1.smithi084.stdout:2/109: truncate d5/d7/d9/f11 19805 0 2024-02-28T23:07:25.818 INFO:tasks.workunit.client.1.smithi084.stdout:1/158: mknod d12/d1b/d2e/d32/c37 0 2024-02-28T23:07:25.819 INFO:tasks.workunit.client.0.smithi057.stdout:7/58: rename d0 to d0/d5/d13 22 2024-02-28T23:07:25.819 INFO:tasks.workunit.client.1.smithi084.stdout:2/110: chown d5/d7/fe 1884843 1 2024-02-28T23:07:25.820 INFO:tasks.workunit.client.0.smithi057.stdout:2/62: getdents d8 0 2024-02-28T23:07:25.821 INFO:tasks.workunit.client.1.smithi084.stdout:1/159: dread - d12/d1b/d2e/d30/f26 zero size 2024-02-28T23:07:25.825 INFO:tasks.workunit.client.0.smithi057.stdout:2/63: write d8/fa [203777,14425] 0 2024-02-28T23:07:25.825 INFO:tasks.workunit.client.1.smithi084.stdout:2/111: mkdir d5/d1b 0 2024-02-28T23:07:25.825 INFO:tasks.workunit.client.1.smithi084.stdout:9/126: dread d4/d6/f7 [0,4194304] 0 2024-02-28T23:07:25.827 INFO:tasks.workunit.client.1.smithi084.stdout:4/91: truncate d1/f4 3455062 0 2024-02-28T23:07:25.827 INFO:tasks.workunit.client.0.smithi057.stdout:2/64: write d8/d11/f12 [192469,62792] 0 2024-02-28T23:07:25.828 INFO:tasks.workunit.client.1.smithi084.stdout:6/149: dread d1/f1b [0,4194304] 0 2024-02-28T23:07:25.831 INFO:tasks.workunit.client.1.smithi084.stdout:2/112: rename d5/dd/ff to d5/dd/f1c 0 2024-02-28T23:07:25.832 INFO:tasks.workunit.client.1.smithi084.stdout:1/160: dread ff [0,4194304] 0 2024-02-28T23:07:25.832 INFO:tasks.workunit.client.1.smithi084.stdout:4/92: mknod d1/dd/d14/c1c 0 2024-02-28T23:07:25.833 INFO:tasks.workunit.client.1.smithi084.stdout:6/150: write d1/f1b [1182035,111774] 0 2024-02-28T23:07:25.835 INFO:tasks.workunit.client.1.smithi084.stdout:4/93: creat d1/f1d x:0 0 0 2024-02-28T23:07:25.836 INFO:tasks.workunit.client.1.smithi084.stdout:1/161: rmdir d12/d1b/d25 39 2024-02-28T23:07:25.840 INFO:tasks.workunit.client.1.smithi084.stdout:4/94: mknod d1/c1e 0 2024-02-28T23:07:25.842 INFO:tasks.workunit.client.1.smithi084.stdout:6/151: mknod d1/d9/d1d/d1e/d2b/d2c/c2d 0 2024-02-28T23:07:25.842 INFO:tasks.workunit.client.0.smithi057.stdout:7/59: dwrite d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.846 INFO:tasks.workunit.client.1.smithi084.stdout:9/127: dwrite d4/f1a [0,4194304] 0 2024-02-28T23:07:25.846 INFO:tasks.workunit.client.1.smithi084.stdout:6/152: mkdir d1/d13/d2e 0 2024-02-28T23:07:25.848 INFO:tasks.workunit.client.0.smithi057.stdout:2/65: dwrite f0 [4194304,4194304] 0 2024-02-28T23:07:25.849 INFO:tasks.workunit.client.1.smithi084.stdout:9/128: write d4/d6/d15/f19 [3207271,119121] 0 2024-02-28T23:07:25.852 INFO:tasks.workunit.client.0.smithi057.stdout:7/60: symlink d0/d5/l14 0 2024-02-28T23:07:25.853 INFO:tasks.workunit.client.0.smithi057.stdout:7/61: mkdir d0/d8/d15 0 2024-02-28T23:07:25.858 INFO:tasks.workunit.client.1.smithi084.stdout:1/162: dread d12/d1b/d2e/d30/f1d [0,4194304] 0 2024-02-28T23:07:25.862 INFO:tasks.workunit.client.1.smithi084.stdout:9/129: rmdir d4/d6/d15 39 2024-02-28T23:07:25.866 INFO:tasks.workunit.client.1.smithi084.stdout:2/113: truncate d5/dd/f16 1419829 0 2024-02-28T23:07:25.867 INFO:tasks.workunit.client.1.smithi084.stdout:1/163: chown d12/d1b/d2e/d30/c21 106255608 1 2024-02-28T23:07:25.867 INFO:tasks.workunit.client.1.smithi084.stdout:4/95: dwrite d1/dd/d15/f16 [0,4194304] 0 2024-02-28T23:07:25.867 INFO:tasks.workunit.client.1.smithi084.stdout:6/153: getdents d1/d9/d14 0 2024-02-28T23:07:25.869 INFO:tasks.workunit.client.1.smithi084.stdout:6/154: fsync d1/d9/f21 0 2024-02-28T23:07:25.869 INFO:tasks.workunit.client.1.smithi084.stdout:1/164: stat d12/d1b/d2e/d30/f1d 0 2024-02-28T23:07:25.871 INFO:tasks.workunit.client.1.smithi084.stdout:6/155: write d1/d9/f28 [872953,117348] 0 2024-02-28T23:07:25.873 INFO:tasks.workunit.client.0.smithi057.stdout:7/62: dwrite d0/f12 [0,4194304] 0 2024-02-28T23:07:25.877 INFO:tasks.workunit.client.1.smithi084.stdout:4/96: symlink d1/dd/l1f 0 2024-02-28T23:07:25.880 INFO:tasks.workunit.client.1.smithi084.stdout:1/165: write d12/d1b/d2e/d30/f1d [4554200,113691] 0 2024-02-28T23:07:25.880 INFO:tasks.workunit.client.0.smithi057.stdout:7/63: creat d0/d8/f16 x:0 0 0 2024-02-28T23:07:25.882 INFO:tasks.workunit.client.1.smithi084.stdout:9/130: link d4/d1d/c21 d4/d6/c27 0 2024-02-28T23:07:25.887 INFO:tasks.workunit.client.1.smithi084.stdout:9/131: rmdir d4/d6/d15/d12 39 2024-02-28T23:07:25.891 INFO:tasks.workunit.client.1.smithi084.stdout:9/132: mknod d4/d6/d15/d12/c28 0 2024-02-28T23:07:25.893 INFO:tasks.workunit.client.0.smithi057.stdout:7/64: dread d0/f12 [0,4194304] 0 2024-02-28T23:07:25.893 INFO:tasks.workunit.client.1.smithi084.stdout:1/166: dread d12/d1b/d2e/d30/f1d [0,4194304] 0 2024-02-28T23:07:25.895 INFO:tasks.workunit.client.0.smithi057.stdout:7/65: truncate d0/f10 436471 0 2024-02-28T23:07:25.895 INFO:tasks.workunit.client.0.smithi057.stdout:2/66: fsync f0 0 2024-02-28T23:07:25.897 INFO:tasks.workunit.client.1.smithi084.stdout:9/133: rename d4/d6/f9 to d4/d6/f29 0 2024-02-28T23:07:25.898 INFO:tasks.workunit.client.0.smithi057.stdout:7/66: creat d0/d8/f17 x:0 0 0 2024-02-28T23:07:25.899 INFO:tasks.workunit.client.1.smithi084.stdout:1/167: fsync d12/d1b/d25/f2a 0 2024-02-28T23:07:25.899 INFO:tasks.workunit.client.1.smithi084.stdout:2/114: dread d5/d7/d9/f11 [0,4194304] 0 2024-02-28T23:07:25.900 INFO:tasks.workunit.client.0.smithi057.stdout:2/67: link d8/fd d8/f19 0 2024-02-28T23:07:25.901 INFO:tasks.workunit.client.0.smithi057.stdout:7/67: truncate d0/d8/f17 625200 0 2024-02-28T23:07:25.901 INFO:tasks.workunit.client.1.smithi084.stdout:2/115: chown d5/f15 37413734 1 2024-02-28T23:07:25.902 INFO:tasks.workunit.client.0.smithi057.stdout:7/68: readlink d0/d8/la 0 2024-02-28T23:07:25.903 INFO:tasks.workunit.client.1.smithi084.stdout:9/134: unlink d4/d1d/c21 0 2024-02-28T23:07:25.905 INFO:tasks.workunit.client.1.smithi084.stdout:1/168: rename fd to d12/d1b/d25/f38 0 2024-02-28T23:07:25.907 INFO:tasks.workunit.client.1.smithi084.stdout:1/169: chown d12/d1b/d2e/d30/c1e 354200 1 2024-02-28T23:07:25.907 INFO:tasks.workunit.client.0.smithi057.stdout:7/69: rename d0/d8/f17 to d0/d8/f18 0 2024-02-28T23:07:25.910 INFO:tasks.workunit.client.0.smithi057.stdout:7/70: creat d0/d8/f19 x:0 0 0 2024-02-28T23:07:25.911 INFO:tasks.workunit.client.1.smithi084.stdout:1/170: symlink d12/d1b/d2e/d30/l39 0 2024-02-28T23:07:25.912 INFO:tasks.workunit.client.1.smithi084.stdout:1/171: read d12/d1b/d25/f38 [1048410,27462] 0 2024-02-28T23:07:25.914 INFO:tasks.workunit.client.0.smithi057.stdout:7/71: chown d0/d8/la 2681 1 2024-02-28T23:07:25.916 INFO:tasks.workunit.client.1.smithi084.stdout:9/135: creat d4/d6/d15/f2a x:0 0 0 2024-02-28T23:07:25.922 INFO:tasks.workunit.client.0.smithi057.stdout:2/68: dwrite d8/d11/f18 [0,4194304] 0 2024-02-28T23:07:25.923 INFO:tasks.workunit.client.1.smithi084.stdout:1/172: rename f9 to d12/d1b/d25/f3a 0 2024-02-28T23:07:25.925 INFO:tasks.workunit.client.0.smithi057.stdout:2/69: rmdir d8/d11 39 2024-02-28T23:07:25.928 INFO:tasks.workunit.client.1.smithi084.stdout:1/173: dread d12/f16 [0,4194304] 0 2024-02-28T23:07:25.931 INFO:tasks.workunit.client.1.smithi084.stdout:2/116: write d5/dd/f16 [1711962,97608] 0 2024-02-28T23:07:25.932 INFO:tasks.workunit.client.0.smithi057.stdout:2/70: getdents d8 0 2024-02-28T23:07:25.932 INFO:tasks.workunit.client.0.smithi057.stdout:2/71: stat d8/l13 0 2024-02-28T23:07:25.932 INFO:tasks.workunit.client.0.smithi057.stdout:7/72: dread d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.933 INFO:tasks.workunit.client.0.smithi057.stdout:2/72: rmdir d8/d11 39 2024-02-28T23:07:25.933 INFO:tasks.workunit.client.0.smithi057.stdout:7/73: readlink d0/d8/lc 0 2024-02-28T23:07:25.934 INFO:tasks.workunit.client.1.smithi084.stdout:1/174: rmdir d12/d1b/d2e 39 2024-02-28T23:07:25.937 INFO:tasks.workunit.client.1.smithi084.stdout:9/136: symlink d4/d6/d15/d12/l2b 0 2024-02-28T23:07:25.939 INFO:tasks.workunit.client.1.smithi084.stdout:9/137: chown d4/d6/d15/f2a 13742073 1 2024-02-28T23:07:25.940 INFO:tasks.workunit.client.1.smithi084.stdout:1/175: stat d12/d1b/d2e/d30/l2d 0 2024-02-28T23:07:25.943 INFO:tasks.workunit.client.1.smithi084.stdout:1/176: mkdir d12/d1b/d3b 0 2024-02-28T23:07:25.944 INFO:tasks.workunit.client.1.smithi084.stdout:9/138: creat d4/d6/d15/f2c x:0 0 0 2024-02-28T23:07:25.946 INFO:tasks.workunit.client.1.smithi084.stdout:1/177: stat d12/d1b/d25/f2a 0 2024-02-28T23:07:25.947 INFO:tasks.workunit.client.1.smithi084.stdout:9/139: mkdir d4/d1d/d1f/d2d 0 2024-02-28T23:07:25.949 INFO:tasks.workunit.client.1.smithi084.stdout:2/117: dwrite d5/d7/f19 [0,4194304] 0 2024-02-28T23:07:25.950 INFO:tasks.workunit.client.1.smithi084.stdout:1/178: creat d12/d1b/d25/f3c x:0 0 0 2024-02-28T23:07:25.950 INFO:tasks.workunit.client.1.smithi084.stdout:2/118: readlink d5/l6 0 2024-02-28T23:07:25.951 INFO:tasks.workunit.client.0.smithi057.stdout:7/74: dread d0/d8/fd [0,4194304] 0 2024-02-28T23:07:25.951 INFO:tasks.workunit.client.1.smithi084.stdout:9/140: mknod d4/c2e 0 2024-02-28T23:07:25.955 INFO:tasks.workunit.client.0.smithi057.stdout:2/73: dwrite d8/f19 [0,4194304] 0 2024-02-28T23:07:25.956 INFO:tasks.workunit.client.1.smithi084.stdout:2/119: mkdir d5/d7/d1d 0 2024-02-28T23:07:25.959 INFO:tasks.workunit.client.0.smithi057.stdout:2/74: chown d8/d11/f18 1721 1 2024-02-28T23:07:25.963 INFO:tasks.workunit.client.0.smithi057.stdout:2/75: dread - d8/fb zero size 2024-02-28T23:07:25.964 INFO:tasks.workunit.client.1.smithi084.stdout:1/179: link d12/d1b/l29 d12/d1b/d2e/d32/l3d 0 2024-02-28T23:07:25.967 INFO:tasks.workunit.client.1.smithi084.stdout:2/120: truncate d5/f15 339901 0 2024-02-28T23:07:25.969 INFO:tasks.workunit.client.1.smithi084.stdout:2/121: chown d5/d7/f19 80759228 1 2024-02-28T23:07:25.972 INFO:tasks.workunit.client.1.smithi084.stdout:2/122: mknod d5/d7/d9/c1e 0 2024-02-28T23:07:25.975 INFO:tasks.workunit.client.1.smithi084.stdout:0/91: sync 2024-02-28T23:07:25.976 INFO:tasks.workunit.client.0.smithi057.stdout:7/75: dwrite d0/f10 [0,4194304] 0 2024-02-28T23:07:25.981 INFO:tasks.workunit.client.1.smithi084.stdout:9/141: dwrite d4/d6/f7 [0,4194304] 0 2024-02-28T23:07:25.990 INFO:tasks.workunit.client.0.smithi057.stdout:2/76: dwrite d8/fa [4194304,4194304] 0 2024-02-28T23:07:25.992 INFO:tasks.workunit.client.1.smithi084.stdout:2/123: dread d5/f10 [0,4194304] 0 2024-02-28T23:07:25.997 INFO:tasks.workunit.client.1.smithi084.stdout:1/180: dwrite d12/d1b/f2c [0,4194304] 0 2024-02-28T23:07:25.997 INFO:tasks.workunit.client.0.smithi057.stdout:2/77: link d8/fa d8/f1a 0 2024-02-28T23:07:25.998 INFO:tasks.workunit.client.1.smithi084.stdout:2/124: mkdir d5/d1f 0 2024-02-28T23:07:25.998 INFO:tasks.workunit.client.0.smithi057.stdout:2/78: chown d8/fb 0 1 2024-02-28T23:07:26.000 INFO:tasks.workunit.client.0.smithi057.stdout:2/79: creat d8/d11/f1b x:0 0 0 2024-02-28T23:07:26.002 INFO:tasks.workunit.client.0.smithi057.stdout:2/80: chown d8/fd 84303 1 2024-02-28T23:07:26.008 INFO:tasks.workunit.client.1.smithi084.stdout:2/125: fdatasync d5/f17 0 2024-02-28T23:07:26.011 INFO:tasks.workunit.client.1.smithi084.stdout:0/92: dwrite d1/d8/f1f [0,4194304] 0 2024-02-28T23:07:26.012 INFO:tasks.workunit.client.1.smithi084.stdout:1/181: dread ff [0,4194304] 0 2024-02-28T23:07:26.018 INFO:tasks.workunit.client.1.smithi084.stdout:0/93: write d1/d6/f13 [1616935,76935] 0 2024-02-28T23:07:26.018 INFO:tasks.workunit.client.1.smithi084.stdout:1/182: dread - d12/d1b/d2e/d30/f26 zero size 2024-02-28T23:07:26.021 INFO:tasks.workunit.client.1.smithi084.stdout:2/126: symlink d5/d1f/l20 0 2024-02-28T23:07:26.021 INFO:tasks.workunit.client.1.smithi084.stdout:0/94: dread - d1/d6/f16 zero size 2024-02-28T23:07:26.023 INFO:tasks.workunit.client.1.smithi084.stdout:9/142: dwrite d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:26.026 INFO:tasks.workunit.client.1.smithi084.stdout:0/95: write d1/d8/f1f [2916143,123151] 0 2024-02-28T23:07:26.028 INFO:tasks.workunit.client.1.smithi084.stdout:1/183: creat d12/d1b/d2e/d30/f3e x:0 0 0 2024-02-28T23:07:26.028 INFO:tasks.workunit.client.1.smithi084.stdout:2/127: symlink d5/d7/d12/l21 0 2024-02-28T23:07:26.031 INFO:tasks.workunit.client.1.smithi084.stdout:0/96: creat d1/d8/f20 x:0 0 0 2024-02-28T23:07:26.032 INFO:tasks.workunit.client.1.smithi084.stdout:9/143: symlink d4/d6/d15/l2f 0 2024-02-28T23:07:26.032 INFO:tasks.workunit.client.1.smithi084.stdout:1/184: read d12/d1b/d25/f3a [649017,120939] 0 2024-02-28T23:07:26.034 INFO:tasks.workunit.client.1.smithi084.stdout:2/128: mknod d5/d7/d12/c22 0 2024-02-28T23:07:26.036 INFO:tasks.workunit.client.0.smithi057.stdout:2/81: dwrite d8/f16 [0,4194304] 0 2024-02-28T23:07:26.042 INFO:tasks.workunit.client.1.smithi084.stdout:2/129: truncate d5/d7/d9/f11 780908 0 2024-02-28T23:07:26.045 INFO:tasks.workunit.client.1.smithi084.stdout:1/185: link d12/d1b/d2e/d30/f19 d12/d1b/d25/f3f 0 2024-02-28T23:07:26.046 INFO:tasks.workunit.client.1.smithi084.stdout:2/130: symlink d5/d7/d1d/l23 0 2024-02-28T23:07:26.049 INFO:tasks.workunit.client.1.smithi084.stdout:9/144: dread d4/d6/d15/f18 [0,4194304] 0 2024-02-28T23:07:26.051 INFO:tasks.workunit.client.1.smithi084.stdout:1/186: write d12/f23 [981931,26543] 0 2024-02-28T23:07:26.052 INFO:tasks.workunit.client.1.smithi084.stdout:2/131: chown d5/d7/d9/cc 48 1 2024-02-28T23:07:26.052 INFO:tasks.workunit.client.1.smithi084.stdout:9/145: mknod d4/c30 0 2024-02-28T23:07:26.056 INFO:tasks.workunit.client.1.smithi084.stdout:2/132: mkdir d5/d7/d12/d24 0 2024-02-28T23:07:26.057 INFO:tasks.workunit.client.1.smithi084.stdout:2/133: chown d5/d7/d9 170920185 1 2024-02-28T23:07:26.057 INFO:tasks.workunit.client.1.smithi084.stdout:0/97: dwrite d1/d8/f1f [0,4194304] 0 2024-02-28T23:07:26.060 INFO:tasks.workunit.client.1.smithi084.stdout:1/187: rename d12/d1b/d25/f38 to d12/f40 0 2024-02-28T23:07:26.062 INFO:tasks.workunit.client.1.smithi084.stdout:2/134: symlink d5/d1f/l25 0 2024-02-28T23:07:26.067 INFO:tasks.workunit.client.1.smithi084.stdout:9/146: rename d4/d6/d15/f1e to d4/d1d/f31 0 2024-02-28T23:07:26.067 INFO:tasks.workunit.client.1.smithi084.stdout:2/135: symlink d5/d7/d12/l26 0 2024-02-28T23:07:26.068 INFO:tasks.workunit.client.1.smithi084.stdout:1/188: mknod d12/d1b/d3b/c41 0 2024-02-28T23:07:26.070 INFO:tasks.workunit.client.1.smithi084.stdout:9/147: readlink d4/d1d/d1f/l24 0 2024-02-28T23:07:26.070 INFO:tasks.workunit.client.1.smithi084.stdout:2/136: dread - d5/d7/fe zero size 2024-02-28T23:07:26.075 INFO:tasks.workunit.client.1.smithi084.stdout:0/98: dwrite d1/d8/f1f [0,4194304] 0 2024-02-28T23:07:26.079 INFO:tasks.workunit.client.1.smithi084.stdout:0/99: creat d1/d6/f21 x:0 0 0 2024-02-28T23:07:26.083 INFO:tasks.workunit.client.1.smithi084.stdout:0/100: unlink d1/fc 0 2024-02-28T23:07:26.086 INFO:tasks.workunit.client.1.smithi084.stdout:0/101: dread - d1/d6/f14 zero size 2024-02-28T23:07:26.088 INFO:tasks.workunit.client.1.smithi084.stdout:1/189: getdents d12/d1b/d25 0 2024-02-28T23:07:26.089 INFO:tasks.workunit.client.1.smithi084.stdout:1/190: fsync d12/f28 0 2024-02-28T23:07:26.090 INFO:tasks.workunit.client.1.smithi084.stdout:0/102: unlink d1/c1a 0 2024-02-28T23:07:26.091 INFO:tasks.workunit.client.1.smithi084.stdout:2/137: dwrite d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:26.093 INFO:tasks.workunit.client.1.smithi084.stdout:9/148: rename d4/d1d to d4/d32 0 2024-02-28T23:07:26.096 INFO:tasks.workunit.client.1.smithi084.stdout:1/191: creat d12/d1b/d3b/f42 x:0 0 0 2024-02-28T23:07:26.096 INFO:tasks.workunit.client.1.smithi084.stdout:9/149: dread - d4/d6/d15/f2c zero size 2024-02-28T23:07:26.097 INFO:tasks.workunit.client.1.smithi084.stdout:9/150: stat d4/cb 0 2024-02-28T23:07:26.098 INFO:tasks.workunit.client.1.smithi084.stdout:2/138: symlink d5/dd/l27 0 2024-02-28T23:07:26.099 INFO:tasks.workunit.client.1.smithi084.stdout:0/103: truncate d1/d6/f19 533124 0 2024-02-28T23:07:26.100 INFO:tasks.workunit.client.1.smithi084.stdout:1/192: write d12/d1b/d3b/f42 [1003994,68452] 0 2024-02-28T23:07:26.101 INFO:tasks.workunit.client.1.smithi084.stdout:0/104: dread - d1/d8/f1d zero size 2024-02-28T23:07:26.104 INFO:tasks.workunit.client.1.smithi084.stdout:1/193: creat d12/d1b/d25/f43 x:0 0 0 2024-02-28T23:07:26.107 INFO:tasks.workunit.client.1.smithi084.stdout:2/139: creat d5/f28 x:0 0 0 2024-02-28T23:07:26.108 INFO:tasks.workunit.client.1.smithi084.stdout:0/105: link d1/d8/f1c d1/d6/d1b/f22 0 2024-02-28T23:07:26.109 INFO:tasks.workunit.client.1.smithi084.stdout:1/194: truncate d12/d1b/d25/f3c 1040975 0 2024-02-28T23:07:26.112 INFO:tasks.workunit.client.1.smithi084.stdout:9/151: dread d4/d6/f7 [0,4194304] 0 2024-02-28T23:07:26.114 INFO:tasks.workunit.client.1.smithi084.stdout:1/195: creat d12/d1b/d3b/f44 x:0 0 0 2024-02-28T23:07:26.119 INFO:tasks.workunit.client.1.smithi084.stdout:9/152: creat d4/d6/d15/f33 x:0 0 0 2024-02-28T23:07:26.121 INFO:tasks.workunit.client.1.smithi084.stdout:0/106: rmdir d1 39 2024-02-28T23:07:26.123 INFO:tasks.workunit.client.1.smithi084.stdout:9/153: write d4/d6/f17 [601495,24496] 0 2024-02-28T23:07:26.123 INFO:tasks.workunit.client.1.smithi084.stdout:2/140: dwrite d5/d7/f19 [0,4194304] 0 2024-02-28T23:07:26.124 INFO:tasks.workunit.client.1.smithi084.stdout:0/107: read - d1/d6/f14 zero size 2024-02-28T23:07:26.124 INFO:tasks.workunit.client.1.smithi084.stdout:1/196: rmdir d12/d1b 39 2024-02-28T23:07:26.126 INFO:tasks.workunit.client.1.smithi084.stdout:0/108: readlink d1/d8/lf 0 2024-02-28T23:07:26.128 INFO:tasks.workunit.client.1.smithi084.stdout:1/197: rmdir d12/d1b 39 2024-02-28T23:07:26.129 INFO:tasks.workunit.client.1.smithi084.stdout:2/141: creat d5/f29 x:0 0 0 2024-02-28T23:07:26.135 INFO:tasks.workunit.client.1.smithi084.stdout:0/109: fdatasync d1/d6/d1b/f22 0 2024-02-28T23:07:26.137 INFO:tasks.workunit.client.1.smithi084.stdout:1/198: creat d12/f45 x:0 0 0 2024-02-28T23:07:26.138 INFO:tasks.workunit.client.1.smithi084.stdout:9/154: dread d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:26.138 INFO:tasks.workunit.client.1.smithi084.stdout:1/199: stat f11 0 2024-02-28T23:07:26.139 INFO:tasks.workunit.client.1.smithi084.stdout:7/132: sync 2024-02-28T23:07:26.139 INFO:tasks.workunit.client.1.smithi084.stdout:3/86: sync 2024-02-28T23:07:26.139 INFO:tasks.workunit.client.1.smithi084.stdout:8/95: sync 2024-02-28T23:07:26.139 INFO:tasks.workunit.client.1.smithi084.stdout:1/200: chown d12/f23 34 1 2024-02-28T23:07:26.140 INFO:tasks.workunit.client.1.smithi084.stdout:0/110: creat d1/d8/f23 x:0 0 0 2024-02-28T23:07:26.140 INFO:tasks.workunit.client.1.smithi084.stdout:2/142: truncate d5/d7/d9/f11 57925 0 2024-02-28T23:07:26.141 INFO:tasks.workunit.client.1.smithi084.stdout:8/96: stat d0/d24 0 2024-02-28T23:07:26.142 INFO:tasks.workunit.client.1.smithi084.stdout:0/111: chown d1/l15 89632619 1 2024-02-28T23:07:26.143 INFO:tasks.workunit.client.1.smithi084.stdout:0/112: readlink d1/l12 0 2024-02-28T23:07:26.146 INFO:tasks.workunit.client.1.smithi084.stdout:7/133: creat db/d10/f23 x:0 0 0 2024-02-28T23:07:26.151 INFO:tasks.workunit.client.1.smithi084.stdout:8/97: truncate d0/d9/fb 1378499 0 2024-02-28T23:07:26.151 INFO:tasks.workunit.client.1.smithi084.stdout:0/113: mkdir d1/d6/d24 0 2024-02-28T23:07:26.151 INFO:tasks.workunit.client.1.smithi084.stdout:8/98: write d0/f8 [4770761,7392] 0 2024-02-28T23:07:26.152 INFO:tasks.workunit.client.1.smithi084.stdout:9/155: link d4/d32/f31 d4/d32/d1f/d2d/f34 0 2024-02-28T23:07:26.152 INFO:tasks.workunit.client.1.smithi084.stdout:8/99: fsync d0/fe 0 2024-02-28T23:07:26.154 INFO:tasks.workunit.client.1.smithi084.stdout:3/87: dread d3/d8/d14/f16 [0,4194304] 0 2024-02-28T23:07:26.155 INFO:tasks.workunit.client.1.smithi084.stdout:9/156: chown d4/d6 1 1 2024-02-28T23:07:26.157 INFO:tasks.workunit.client.1.smithi084.stdout:3/88: chown c2 0 1 2024-02-28T23:07:26.158 INFO:tasks.workunit.client.1.smithi084.stdout:0/114: unlink d1/d8/lf 0 2024-02-28T23:07:26.159 INFO:tasks.workunit.client.1.smithi084.stdout:9/157: mknod d4/d32/d1f/c35 0 2024-02-28T23:07:26.161 INFO:tasks.workunit.client.1.smithi084.stdout:3/89: write d3/fe [944999,75287] 0 2024-02-28T23:07:26.162 INFO:tasks.workunit.client.1.smithi084.stdout:0/115: readlink d1/la 0 2024-02-28T23:07:26.167 INFO:tasks.workunit.client.1.smithi084.stdout:2/143: dwrite d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:26.167 INFO:tasks.workunit.client.1.smithi084.stdout:1/201: dwrite d12/f40 [0,4194304] 0 2024-02-28T23:07:26.167 INFO:tasks.workunit.client.1.smithi084.stdout:9/158: read d4/d6/f29 [176614,70755] 0 2024-02-28T23:07:26.170 INFO:tasks.workunit.client.1.smithi084.stdout:0/116: creat d1/d6/f25 x:0 0 0 2024-02-28T23:07:26.172 INFO:tasks.workunit.client.1.smithi084.stdout:3/90: mkdir d3/d8/d1a/d21 0 2024-02-28T23:07:26.174 INFO:tasks.workunit.client.1.smithi084.stdout:9/159: rmdir d4 39 2024-02-28T23:07:26.186 INFO:tasks.workunit.client.1.smithi084.stdout:3/91: write d3/fc [2388676,127064] 0 2024-02-28T23:07:26.187 INFO:tasks.workunit.client.1.smithi084.stdout:2/144: mkdir d5/d7/d12/d24/d2a 0 2024-02-28T23:07:26.188 INFO:tasks.workunit.client.1.smithi084.stdout:2/145: stat f2 0 2024-02-28T23:07:26.190 INFO:tasks.workunit.client.1.smithi084.stdout:3/92: chown d3/d8/df/c1d 3007348 1 2024-02-28T23:07:26.193 INFO:tasks.workunit.client.1.smithi084.stdout:9/160: fsync d4/d6/d15/f18 0 2024-02-28T23:07:26.193 INFO:tasks.workunit.client.1.smithi084.stdout:1/202: creat d12/d1b/d2e/d30/f46 x:0 0 0 2024-02-28T23:07:26.193 INFO:tasks.workunit.client.1.smithi084.stdout:0/117: dwrite d1/d8/f20 [0,4194304] 0 2024-02-28T23:07:26.193 INFO:tasks.workunit.client.1.smithi084.stdout:2/146: chown d5/d7/d9/cc 147849 1 2024-02-28T23:07:26.193 INFO:tasks.workunit.client.1.smithi084.stdout:9/161: fsync d4/d6/d15/f33 0 2024-02-28T23:07:26.194 INFO:tasks.workunit.client.1.smithi084.stdout:7/134: dread db/fd [0,4194304] 0 2024-02-28T23:07:26.199 INFO:tasks.workunit.client.1.smithi084.stdout:1/203: fdatasync d12/f16 0 2024-02-28T23:07:26.201 INFO:tasks.workunit.client.1.smithi084.stdout:2/147: creat d5/d1f/f2b x:0 0 0 2024-02-28T23:07:26.201 INFO:tasks.workunit.client.1.smithi084.stdout:3/93: creat d3/d18/f22 x:0 0 0 2024-02-28T23:07:26.204 INFO:tasks.workunit.client.1.smithi084.stdout:0/118: rmdir d1/d6/d24 0 2024-02-28T23:07:26.205 INFO:tasks.workunit.client.1.smithi084.stdout:1/204: creat d12/d1b/d2e/d32/f47 x:0 0 0 2024-02-28T23:07:26.207 INFO:tasks.workunit.client.1.smithi084.stdout:7/135: rename la to db/d10/d17/l24 0 2024-02-28T23:07:26.208 INFO:tasks.workunit.client.1.smithi084.stdout:9/162: creat d4/d6/f36 x:0 0 0 2024-02-28T23:07:26.212 INFO:tasks.workunit.client.1.smithi084.stdout:9/163: truncate d4/d6/d15/f2a 995093 0 2024-02-28T23:07:26.219 INFO:tasks.workunit.client.1.smithi084.stdout:3/94: dwrite d3/f6 [0,4194304] 0 2024-02-28T23:07:26.220 INFO:tasks.workunit.client.1.smithi084.stdout:2/148: dread d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:26.225 INFO:tasks.workunit.client.1.smithi084.stdout:2/149: creat d5/d7/d9/f2c x:0 0 0 2024-02-28T23:07:26.226 INFO:tasks.workunit.client.1.smithi084.stdout:2/150: stat f2 0 2024-02-28T23:07:26.229 INFO:tasks.workunit.client.1.smithi084.stdout:1/205: dwrite d12/d1b/d2e/d32/f47 [0,4194304] 0 2024-02-28T23:07:26.229 INFO:tasks.workunit.client.1.smithi084.stdout:3/95: link d3/d8/fa d3/d8/d1a/d21/f23 0 2024-02-28T23:07:26.230 INFO:tasks.workunit.client.1.smithi084.stdout:2/151: write d5/d7/f19 [86030,87653] 0 2024-02-28T23:07:26.232 INFO:tasks.workunit.client.1.smithi084.stdout:2/152: fdatasync d5/d7/fe 0 2024-02-28T23:07:26.234 INFO:tasks.workunit.client.1.smithi084.stdout:2/153: stat d5 0 2024-02-28T23:07:26.234 INFO:tasks.workunit.client.1.smithi084.stdout:1/206: write d12/d1b/d3b/f44 [456535,66911] 0 2024-02-28T23:07:26.236 INFO:tasks.workunit.client.1.smithi084.stdout:2/154: readlink d5/d7/d12/l26 0 2024-02-28T23:07:26.238 INFO:tasks.workunit.client.1.smithi084.stdout:3/96: creat d3/d8/f24 x:0 0 0 2024-02-28T23:07:26.239 INFO:tasks.workunit.client.1.smithi084.stdout:9/164: dwrite d4/f22 [0,4194304] 0 2024-02-28T23:07:26.242 INFO:tasks.workunit.client.1.smithi084.stdout:1/207: creat d12/d1b/f48 x:0 0 0 2024-02-28T23:07:26.242 INFO:tasks.workunit.client.1.smithi084.stdout:9/165: fsync d4/d6/f8 0 2024-02-28T23:07:26.244 INFO:tasks.workunit.client.1.smithi084.stdout:1/208: symlink d12/d1b/d3b/l49 0 2024-02-28T23:07:26.245 INFO:tasks.workunit.client.1.smithi084.stdout:9/166: creat d4/f37 x:0 0 0 2024-02-28T23:07:26.251 INFO:tasks.workunit.client.1.smithi084.stdout:9/167: write d4/d6/f17 [1390335,14127] 0 2024-02-28T23:07:26.257 INFO:tasks.workunit.client.1.smithi084.stdout:9/168: dread - d4/d6/d15/f2c zero size 2024-02-28T23:07:26.261 INFO:tasks.workunit.client.1.smithi084.stdout:2/155: dwrite d5/d7/fe [0,4194304] 0 2024-02-28T23:07:26.262 INFO:tasks.workunit.client.1.smithi084.stdout:3/97: dwrite d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:26.269 INFO:tasks.workunit.client.1.smithi084.stdout:3/98: write d3/d8/fa [1118360,79162] 0 2024-02-28T23:07:26.274 INFO:tasks.workunit.client.1.smithi084.stdout:3/99: mkdir d3/d18/d25 0 2024-02-28T23:07:26.274 INFO:tasks.workunit.client.1.smithi084.stdout:4/97: sync 2024-02-28T23:07:26.274 INFO:tasks.workunit.client.1.smithi084.stdout:5/121: sync 2024-02-28T23:07:26.275 INFO:tasks.workunit.client.1.smithi084.stdout:6/156: sync 2024-02-28T23:07:26.277 INFO:tasks.workunit.client.1.smithi084.stdout:3/100: chown d3/d18/d25 674 1 2024-02-28T23:07:26.281 INFO:tasks.workunit.client.1.smithi084.stdout:4/98: readlink d1/dd/d15/l1b 0 2024-02-28T23:07:26.282 INFO:tasks.workunit.client.1.smithi084.stdout:1/209: dread d12/d1b/d25/f3f [0,4194304] 0 2024-02-28T23:07:26.287 INFO:tasks.workunit.client.1.smithi084.stdout:1/210: fdatasync d12/d1b/d2e/d30/f26 0 2024-02-28T23:07:26.289 INFO:tasks.workunit.client.1.smithi084.stdout:4/99: chown d1/dd/c10 13 1 2024-02-28T23:07:26.291 INFO:tasks.workunit.client.1.smithi084.stdout:9/169: dwrite d4/d32/d1f/d2d/f34 [0,4194304] 0 2024-02-28T23:07:26.297 INFO:tasks.workunit.client.1.smithi084.stdout:4/100: readlink d1/dd/l1f 0 2024-02-28T23:07:26.302 INFO:tasks.workunit.client.1.smithi084.stdout:5/122: dwrite dc/f21 [0,4194304] 0 2024-02-28T23:07:26.314 INFO:tasks.workunit.client.1.smithi084.stdout:6/157: dread d1/d9/f21 [4194304,4194304] 0 2024-02-28T23:07:26.314 INFO:tasks.workunit.client.1.smithi084.stdout:3/101: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:26.314 INFO:tasks.workunit.client.1.smithi084.stdout:9/170: mknod d4/d32/d1f/d2d/c38 0 2024-02-28T23:07:26.314 INFO:tasks.workunit.client.1.smithi084.stdout:6/158: write d1/f24 [2151741,40993] 0 2024-02-28T23:07:26.315 INFO:tasks.workunit.client.1.smithi084.stdout:6/159: stat d1/l12 0 2024-02-28T23:07:26.316 INFO:tasks.workunit.client.1.smithi084.stdout:2/156: truncate f2 275618 0 2024-02-28T23:07:26.316 INFO:tasks.workunit.client.1.smithi084.stdout:1/211: dread d12/d1b/f2c [0,4194304] 0 2024-02-28T23:07:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:26 smithi057 ceph-mon[29096]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:26 smithi057 ceph-mon[29096]: pgmap v19: 65 pgs: 65 active+clean; 304 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 359 op/s 2024-02-28T23:07:26.319 INFO:tasks.workunit.client.1.smithi084.stdout:2/157: read - d5/d1f/f2b zero size 2024-02-28T23:07:26.319 INFO:tasks.workunit.client.1.smithi084.stdout:3/102: write d3/d8/d1a/f1f [852261,129175] 0 2024-02-28T23:07:26.319 INFO:tasks.workunit.client.1.smithi084.stdout:9/171: rmdir d4/d32/d1f/d2d 39 2024-02-28T23:07:26.322 INFO:tasks.workunit.client.1.smithi084.stdout:6/160: write d1/d9/d1d/d1e/f26 [2170796,96281] 0 2024-02-28T23:07:26.323 INFO:tasks.workunit.client.1.smithi084.stdout:5/123: dread dc/de/f1a [0,4194304] 0 2024-02-28T23:07:26.324 INFO:tasks.workunit.client.1.smithi084.stdout:1/212: truncate d12/d1b/d3b/f44 1505538 0 2024-02-28T23:07:26.328 INFO:tasks.workunit.client.1.smithi084.stdout:1/213: write d12/d1b/d2e/d30/f26 [337694,37816] 0 2024-02-28T23:07:26.329 INFO:tasks.workunit.client.1.smithi084.stdout:4/101: getdents d1/dd/d14 0 2024-02-28T23:07:26.330 INFO:tasks.workunit.client.0.smithi057.stdout:2/82: readlink d8/l13 0 2024-02-28T23:07:26.331 INFO:tasks.workunit.client.1.smithi084.stdout:5/124: creat dc/f27 x:0 0 0 2024-02-28T23:07:26.331 INFO:tasks.workunit.client.1.smithi084.stdout:4/102: chown l0 14107156 1 2024-02-28T23:07:26.332 INFO:tasks.workunit.client.1.smithi084.stdout:6/161: creat d1/d9/d1d/d1e/d2b/f2f x:0 0 0 2024-02-28T23:07:26.334 INFO:tasks.workunit.client.1.smithi084.stdout:3/103: symlink d3/d18/d25/l26 0 2024-02-28T23:07:26.336 INFO:tasks.workunit.client.0.smithi057.stdout:2/83: rename f0 to d8/d11/f1c 0 2024-02-28T23:07:26.337 INFO:tasks.workunit.client.0.smithi057.stdout:2/84: stat d8/d11/f1b 0 2024-02-28T23:07:26.338 INFO:tasks.workunit.client.0.smithi057.stdout:2/85: stat d8/d11/f18 0 2024-02-28T23:07:26.338 INFO:tasks.workunit.client.1.smithi084.stdout:4/103: rename d1/dd to d1/d18/d20 0 2024-02-28T23:07:26.338 INFO:tasks.workunit.client.1.smithi084.stdout:5/125: fdatasync dc/de/f1a 0 2024-02-28T23:07:26.339 INFO:tasks.workunit.client.1.smithi084.stdout:9/172: link d4/c2e d4/c39 0 2024-02-28T23:07:26.343 INFO:tasks.workunit.client.0.smithi057.stdout:2/86: getdents d8 0 2024-02-28T23:07:26.345 INFO:tasks.workunit.client.1.smithi084.stdout:9/173: read d4/fc [291565,91192] 0 2024-02-28T23:07:26.345 INFO:tasks.workunit.client.1.smithi084.stdout:3/104: rename d3/d18/d25 to d3/d27 0 2024-02-28T23:07:26.346 INFO:tasks.workunit.client.0.smithi057.stdout:2/87: rename d8/d11/f12 to d8/d11/f1d 0 2024-02-28T23:07:26.349 INFO:tasks.workunit.client.1.smithi084.stdout:9/174: chown d4/f13 54352 1 2024-02-28T23:07:26.350 INFO:tasks.workunit.client.1.smithi084.stdout:4/104: symlink d1/d18/d20/d15/l21 0 2024-02-28T23:07:26.351 INFO:tasks.workunit.client.1.smithi084.stdout:1/214: dwrite d12/d1b/d25/f3c [0,4194304] 0 2024-02-28T23:07:26.354 INFO:tasks.workunit.client.1.smithi084.stdout:1/215: write d12/d1b/f31 [602041,86832] 0 2024-02-28T23:07:26.355 INFO:tasks.workunit.client.1.smithi084.stdout:5/126: rmdir dc/de/df/d16 39 2024-02-28T23:07:26.356 INFO:tasks.workunit.client.1.smithi084.stdout:1/216: readlink d12/d1b/l20 0 2024-02-28T23:07:26.356 INFO:tasks.workunit.client.1.smithi084.stdout:9/175: write d4/f1a [2589782,4622] 0 2024-02-28T23:07:26.358 INFO:tasks.workunit.client.1.smithi084.stdout:6/162: truncate d1/d9/f21 8262140 0 2024-02-28T23:07:26.359 INFO:tasks.workunit.client.0.smithi057.stdout:2/88: dread d8/f16 [0,4194304] 0 2024-02-28T23:07:26.360 INFO:tasks.workunit.client.1.smithi084.stdout:4/105: symlink d1/d18/d20/d14/l22 0 2024-02-28T23:07:26.360 INFO:tasks.workunit.client.1.smithi084.stdout:3/105: creat d3/f28 x:0 0 0 2024-02-28T23:07:26.360 INFO:tasks.workunit.client.1.smithi084.stdout:1/217: write fc [5339730,8483] 0 2024-02-28T23:07:26.362 INFO:tasks.workunit.client.1.smithi084.stdout:4/106: chown d1/d18/d20/d14 70460317 1 2024-02-28T23:07:26.362 INFO:tasks.workunit.client.1.smithi084.stdout:2/158: dread f3 [0,4194304] 0 2024-02-28T23:07:26.362 INFO:tasks.workunit.client.1.smithi084.stdout:9/176: chown d4/d6/c14 0 1 2024-02-28T23:07:26.363 INFO:tasks.workunit.client.1.smithi084.stdout:5/127: creat dc/f28 x:0 0 0 2024-02-28T23:07:26.372 INFO:tasks.workunit.client.1.smithi084.stdout:1/218: creat d12/d1b/f4a x:0 0 0 2024-02-28T23:07:26.376 INFO:tasks.workunit.client.1.smithi084.stdout:5/128: creat dc/de/df/d16/f29 x:0 0 0 2024-02-28T23:07:26.377 INFO:tasks.workunit.client.1.smithi084.stdout:5/129: readlink dc/de/l11 0 2024-02-28T23:07:26.379 INFO:tasks.workunit.client.0.smithi057.stdout:2/89: dwrite d8/fb [0,4194304] 0 2024-02-28T23:07:26.383 INFO:tasks.workunit.client.0.smithi057.stdout:2/90: mknod d8/d11/d15/c1e 0 2024-02-28T23:07:26.385 INFO:tasks.workunit.client.0.smithi057.stdout:2/91: rename d8/d11 to d8/d11/d15/d1f 22 2024-02-28T23:07:26.386 INFO:tasks.workunit.client.1.smithi084.stdout:9/177: link d4/d32/d1f/d2d/c38 d4/c3a 0 2024-02-28T23:07:26.386 INFO:tasks.workunit.client.1.smithi084.stdout:3/106: dwrite f1 [0,4194304] 0 2024-02-28T23:07:26.386 INFO:tasks.workunit.client.1.smithi084.stdout:1/219: link d12/f15 d12/d1b/d2e/d32/f4b 0 2024-02-28T23:07:26.389 INFO:tasks.workunit.client.1.smithi084.stdout:1/220: chown d12/d1b/d2e/d30/c2b 484467 1 2024-02-28T23:07:26.389 INFO:tasks.workunit.client.0.smithi057.stdout:2/92: write d8/f1a [7529121,116432] 0 2024-02-28T23:07:26.390 INFO:tasks.workunit.client.1.smithi084.stdout:4/107: getdents d1/d18/d20/d14 0 2024-02-28T23:07:26.391 INFO:tasks.workunit.client.1.smithi084.stdout:1/221: fsync d12/d1b/d25/f43 0 2024-02-28T23:07:26.392 INFO:tasks.workunit.client.1.smithi084.stdout:9/178: unlink d4/la 0 2024-02-28T23:07:26.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/93: creat d8/d11/d15/f20 x:0 0 0 2024-02-28T23:07:26.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/94: readlink d8/l14 0 2024-02-28T23:07:26.395 INFO:tasks.workunit.client.0.smithi057.stdout:2/95: mkdir d8/d11/d21 0 2024-02-28T23:07:26.397 INFO:tasks.workunit.client.1.smithi084.stdout:1/222: write d12/d1b/d2e/d30/f26 [416343,11390] 0 2024-02-28T23:07:26.397 INFO:tasks.workunit.client.1.smithi084.stdout:4/108: mknod d1/d18/d20/c23 0 2024-02-28T23:07:26.397 INFO:tasks.workunit.client.0.smithi057.stdout:2/96: chown d8/d11/f1c 102 1 2024-02-28T23:07:26.401 INFO:tasks.workunit.client.1.smithi084.stdout:4/109: chown d1/c1e 1362118 1 2024-02-28T23:07:26.401 INFO:tasks.workunit.client.1.smithi084.stdout:1/223: dread - d12/d1b/d25/f43 zero size 2024-02-28T23:07:26.401 INFO:tasks.workunit.client.1.smithi084.stdout:5/130: dwrite dc/de/f14 [0,4194304] 0 2024-02-28T23:07:26.402 INFO:tasks.workunit.client.0.smithi057.stdout:2/97: symlink d8/d11/d21/l22 0 2024-02-28T23:07:26.406 INFO:tasks.workunit.client.1.smithi084.stdout:4/110: mkdir d1/d18/d20/d24 0 2024-02-28T23:07:26.407 INFO:tasks.workunit.client.0.smithi057.stdout:2/98: creat d8/f23 x:0 0 0 2024-02-28T23:07:26.412 INFO:tasks.workunit.client.1.smithi084.stdout:5/131: link dc/de/df/d16/f29 dc/f2a 0 2024-02-28T23:07:26.412 INFO:tasks.workunit.client.1.smithi084.stdout:4/111: creat d1/d18/d20/d14/f25 x:0 0 0 2024-02-28T23:07:26.413 INFO:tasks.workunit.client.1.smithi084.stdout:3/107: dread d3/d8/d1a/d21/f23 [0,4194304] 0 2024-02-28T23:07:26.414 INFO:tasks.workunit.client.1.smithi084.stdout:5/132: mknod dc/c2b 0 2024-02-28T23:07:26.415 INFO:tasks.workunit.client.1.smithi084.stdout:3/108: dread - d3/d18/f22 zero size 2024-02-28T23:07:26.420 INFO:tasks.workunit.client.1.smithi084.stdout:3/109: mkdir d3/d8/d1a/d29 0 2024-02-28T23:07:26.421 INFO:tasks.workunit.client.1.smithi084.stdout:4/112: symlink d1/d18/d20/d24/l26 0 2024-02-28T23:07:26.422 INFO:tasks.workunit.client.1.smithi084.stdout:4/113: stat d1/d18/d20/f11 0 2024-02-28T23:07:26.423 INFO:tasks.workunit.client.1.smithi084.stdout:5/133: creat dc/de/df/f2c x:0 0 0 2024-02-28T23:07:26.425 INFO:tasks.workunit.client.1.smithi084.stdout:4/114: chown d1/f4 15710 1 2024-02-28T23:07:26.428 INFO:tasks.workunit.client.1.smithi084.stdout:5/134: getdents dc/de 0 2024-02-28T23:07:26.430 INFO:tasks.workunit.client.1.smithi084.stdout:4/115: truncate d1/d18/d20/d15/f1a 241113 0 2024-02-28T23:07:26.431 INFO:tasks.workunit.client.1.smithi084.stdout:1/224: dwrite d12/d1b/d25/f3f [4194304,4194304] 0 2024-02-28T23:07:26.432 INFO:tasks.workunit.client.1.smithi084.stdout:5/135: write dc/de/df/f20 [3079072,118572] 0 2024-02-28T23:07:26.433 INFO:tasks.workunit.client.0.smithi057.stdout:2/99: creat d8/f24 x:0 0 0 2024-02-28T23:07:26.438 INFO:tasks.workunit.client.1.smithi084.stdout:5/136: creat dc/de/f2d x:0 0 0 2024-02-28T23:07:26.439 INFO:tasks.workunit.client.1.smithi084.stdout:1/225: mknod d12/d1b/d3b/c4c 0 2024-02-28T23:07:26.439 INFO:tasks.workunit.client.0.smithi057.stdout:2/100: rename d8/d11/d15/c1e to d8/c25 0 2024-02-28T23:07:26.439 INFO:tasks.workunit.client.1.smithi084.stdout:5/137: fdatasync dc/f25 0 2024-02-28T23:07:26.444 INFO:tasks.workunit.client.0.smithi057.stdout:2/101: chown d8/f24 2326343 1 2024-02-28T23:07:26.444 INFO:tasks.workunit.client.1.smithi084.stdout:4/116: write d1/d18/d20/d15/f16 [1354704,108726] 0 2024-02-28T23:07:26.444 INFO:tasks.workunit.client.1.smithi084.stdout:3/110: dwrite d3/d18/f22 [0,4194304] 0 2024-02-28T23:07:26.445 INFO:tasks.workunit.client.1.smithi084.stdout:5/138: truncate dc/f25 502362 0 2024-02-28T23:07:26.448 INFO:tasks.workunit.client.1.smithi084.stdout:3/111: mknod d3/d8/d1a/d21/c2a 0 2024-02-28T23:07:26.452 INFO:tasks.workunit.client.1.smithi084.stdout:3/112: rmdir d3/d8/df 39 2024-02-28T23:07:26.460 INFO:tasks.workunit.client.1.smithi084.stdout:3/113: creat d3/d18/f2b x:0 0 0 2024-02-28T23:07:26.464 INFO:tasks.workunit.client.1.smithi084.stdout:1/226: dwrite d12/d1b/f48 [0,4194304] 0 2024-02-28T23:07:26.468 INFO:tasks.workunit.client.1.smithi084.stdout:3/114: rename d3/d8/d1a/d21/c2a to d3/d18/c2c 0 2024-02-28T23:07:26.470 INFO:tasks.workunit.client.1.smithi084.stdout:1/227: write d12/d1b/d25/f3c [52832,114979] 0 2024-02-28T23:07:26.472 INFO:tasks.workunit.client.1.smithi084.stdout:3/115: stat d3/f11 0 2024-02-28T23:07:26.473 INFO:tasks.workunit.client.1.smithi084.stdout:5/139: dwrite dc/de/df/f2c [0,4194304] 0 2024-02-28T23:07:26.473 INFO:tasks.workunit.client.1.smithi084.stdout:1/228: unlink ff 0 2024-02-28T23:07:26.474 INFO:tasks.workunit.client.1.smithi084.stdout:3/116: creat d3/d27/f2d x:0 0 0 2024-02-28T23:07:26.476 INFO:tasks.workunit.client.1.smithi084.stdout:5/140: write f7 [4919899,24960] 0 2024-02-28T23:07:26.479 INFO:tasks.workunit.client.1.smithi084.stdout:4/117: truncate d1/d18/d20/d15/f16 2090724 0 2024-02-28T23:07:26.485 INFO:tasks.workunit.client.1.smithi084.stdout:4/118: dread d1/d18/d20/d15/f1a [0,4194304] 0 2024-02-28T23:07:26.485 INFO:tasks.workunit.client.1.smithi084.stdout:5/141: unlink dc/de/f13 0 2024-02-28T23:07:26.485 INFO:tasks.workunit.client.1.smithi084.stdout:3/117: unlink d3/d8/d1a/d21/f23 0 2024-02-28T23:07:26.489 INFO:tasks.workunit.client.1.smithi084.stdout:3/118: creat d3/d8/d14/f2e x:0 0 0 2024-02-28T23:07:26.490 INFO:tasks.workunit.client.1.smithi084.stdout:4/119: read - d1/f1d zero size 2024-02-28T23:07:26.490 INFO:tasks.workunit.client.1.smithi084.stdout:5/142: mknod dc/de/df/c2e 0 2024-02-28T23:07:26.492 INFO:tasks.workunit.client.1.smithi084.stdout:3/119: dread - d3/d8/df/f12 zero size 2024-02-28T23:07:26.493 INFO:tasks.workunit.client.1.smithi084.stdout:4/120: mkdir d1/d18/d27 0 2024-02-28T23:07:26.495 INFO:tasks.workunit.client.1.smithi084.stdout:3/120: write d3/f11 [1658296,78404] 0 2024-02-28T23:07:26.497 INFO:tasks.workunit.client.1.smithi084.stdout:4/121: creat d1/d18/d20/d15/f28 x:0 0 0 2024-02-28T23:07:26.498 INFO:tasks.workunit.client.1.smithi084.stdout:4/122: readlink d1/d18/d20/l19 0 2024-02-28T23:07:26.501 INFO:tasks.workunit.client.1.smithi084.stdout:4/123: mknod d1/d18/d20/c29 0 2024-02-28T23:07:26.501 INFO:tasks.workunit.client.1.smithi084.stdout:1/229: fdatasync d12/d1b/d25/f3c 0 2024-02-28T23:07:26.506 INFO:tasks.workunit.client.1.smithi084.stdout:1/230: creat d12/d1b/d2e/d32/f4d x:0 0 0 2024-02-28T23:07:26.511 INFO:tasks.workunit.client.1.smithi084.stdout:1/231: creat d12/d1b/d2e/f4e x:0 0 0 2024-02-28T23:07:26.513 INFO:tasks.workunit.client.1.smithi084.stdout:3/121: dwrite d3/f11 [0,4194304] 0 2024-02-28T23:07:26.515 INFO:tasks.workunit.client.1.smithi084.stdout:5/143: write dc/de/df/d16/f29 [169656,37251] 0 2024-02-28T23:07:26.516 INFO:tasks.workunit.client.1.smithi084.stdout:3/122: truncate d3/d18/f22 4864884 0 2024-02-28T23:07:26.518 INFO:tasks.workunit.client.1.smithi084.stdout:5/144: read dc/de/df/f2c [219558,71040] 0 2024-02-28T23:07:26.528 INFO:tasks.workunit.client.1.smithi084.stdout:0/119: sync 2024-02-28T23:07:26.529 INFO:tasks.workunit.client.1.smithi084.stdout:7/136: sync 2024-02-28T23:07:26.529 INFO:tasks.workunit.client.1.smithi084.stdout:8/100: sync 2024-02-28T23:07:26.529 INFO:tasks.workunit.client.1.smithi084.stdout:1/232: dread d12/d1b/f2c [0,4194304] 0 2024-02-28T23:07:26.529 INFO:tasks.workunit.client.1.smithi084.stdout:0/120: dread - d1/d8/f23 zero size 2024-02-28T23:07:26.532 INFO:tasks.workunit.client.1.smithi084.stdout:7/137: creat db/d10/d18/f25 x:0 0 0 2024-02-28T23:07:26.533 INFO:tasks.workunit.client.1.smithi084.stdout:4/124: dread d1/f7 [0,4194304] 0 2024-02-28T23:07:26.534 INFO:tasks.workunit.client.1.smithi084.stdout:1/233: readlink d12/d1b/d3b/l49 0 2024-02-28T23:07:26.536 INFO:tasks.workunit.client.1.smithi084.stdout:8/101: getdents d0/d9/d1b 0 2024-02-28T23:07:26.544 INFO:tasks.workunit.client.1.smithi084.stdout:1/234: write d12/d1b/d25/f2a [747387,16184] 0 2024-02-28T23:07:26.545 INFO:tasks.workunit.client.1.smithi084.stdout:3/123: dwrite d3/d27/f2d [0,4194304] 0 2024-02-28T23:07:26.547 INFO:tasks.workunit.client.1.smithi084.stdout:5/145: dwrite dc/de/f2d [0,4194304] 0 2024-02-28T23:07:26.552 INFO:tasks.workunit.client.1.smithi084.stdout:3/124: stat d3/d8/df/l15 0 2024-02-28T23:07:26.554 INFO:tasks.workunit.client.1.smithi084.stdout:8/102: creat d0/d9/d1b/d20/f25 x:0 0 0 2024-02-28T23:07:26.556 INFO:tasks.workunit.client.1.smithi084.stdout:5/146: write dc/de/df/f20 [1838661,76893] 0 2024-02-28T23:07:26.557 INFO:tasks.workunit.client.1.smithi084.stdout:3/125: mkdir d3/d8/df/d2f 0 2024-02-28T23:07:26.562 INFO:tasks.workunit.client.1.smithi084.stdout:4/125: dread d1/f7 [0,4194304] 0 2024-02-28T23:07:26.563 INFO:tasks.workunit.client.1.smithi084.stdout:5/147: symlink dc/de/df/d16/l2f 0 2024-02-28T23:07:26.565 INFO:tasks.workunit.client.1.smithi084.stdout:3/126: rmdir d3/d8/df 39 2024-02-28T23:07:26.569 INFO:tasks.workunit.client.1.smithi084.stdout:4/126: write d1/d18/d20/d15/f28 [596869,84941] 0 2024-02-28T23:07:26.570 INFO:tasks.workunit.client.1.smithi084.stdout:1/235: dread d12/f40 [0,4194304] 0 2024-02-28T23:07:26.574 INFO:tasks.workunit.client.1.smithi084.stdout:5/148: dread - dc/de/df/d16/f26 zero size 2024-02-28T23:07:26.577 INFO:tasks.workunit.client.1.smithi084.stdout:0/121: dwrite d1/d6/f19 [0,4194304] 0 2024-02-28T23:07:26.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:26 smithi084 ceph-mon[34826]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:26.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:26 smithi084 ceph-mon[34826]: pgmap v19: 65 pgs: 65 active+clean; 304 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 359 op/s 2024-02-28T23:07:26.580 INFO:tasks.workunit.client.1.smithi084.stdout:1/236: dread - d12/d1b/d2e/d32/f4d zero size 2024-02-28T23:07:26.581 INFO:tasks.workunit.client.1.smithi084.stdout:5/149: read - dc/f27 zero size 2024-02-28T23:07:26.582 INFO:tasks.workunit.client.1.smithi084.stdout:7/138: dwrite db/d11/f12 [0,4194304] 0 2024-02-28T23:07:26.583 INFO:tasks.workunit.client.1.smithi084.stdout:0/122: write d1/d6/f16 [3040430,83971] 0 2024-02-28T23:07:26.586 INFO:tasks.workunit.client.1.smithi084.stdout:7/139: rename db to db/d26 22 2024-02-28T23:07:26.587 INFO:tasks.workunit.client.1.smithi084.stdout:5/150: mknod dc/de/c30 0 2024-02-28T23:07:26.589 INFO:tasks.workunit.client.1.smithi084.stdout:0/123: write d1/d8/f23 [443815,76555] 0 2024-02-28T23:07:26.591 INFO:tasks.workunit.client.1.smithi084.stdout:4/127: dread d1/d18/d20/d15/f28 [0,4194304] 0 2024-02-28T23:07:26.591 INFO:tasks.workunit.client.1.smithi084.stdout:8/103: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:26.593 INFO:tasks.workunit.client.1.smithi084.stdout:4/128: read d1/f4 [2777655,67059] 0 2024-02-28T23:07:26.593 INFO:tasks.workunit.client.1.smithi084.stdout:3/127: dread d3/f11 [0,4194304] 0 2024-02-28T23:07:26.601 INFO:tasks.workunit.client.1.smithi084.stdout:7/140: mkdir db/d11/d27 0 2024-02-28T23:07:26.607 INFO:tasks.workunit.client.1.smithi084.stdout:5/151: mkdir dc/de/df/d31 0 2024-02-28T23:07:26.616 INFO:tasks.workunit.client.1.smithi084.stdout:1/237: dwrite d12/f40 [0,4194304] 0 2024-02-28T23:07:26.618 INFO:tasks.workunit.client.1.smithi084.stdout:3/128: dread d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:26.622 INFO:tasks.workunit.client.1.smithi084.stdout:8/104: dwrite d0/fe [4194304,4194304] 0 2024-02-28T23:07:26.625 INFO:tasks.workunit.client.1.smithi084.stdout:7/141: dwrite db/d11/f16 [0,4194304] 0 2024-02-28T23:07:26.626 INFO:tasks.workunit.client.1.smithi084.stdout:1/238: mknod d12/c4f 0 2024-02-28T23:07:26.634 INFO:tasks.workunit.client.1.smithi084.stdout:7/142: mknod db/d10/d18/c28 0 2024-02-28T23:07:26.636 INFO:tasks.workunit.client.1.smithi084.stdout:1/239: mkdir d12/d1b/d2e/d30/d50 0 2024-02-28T23:07:26.638 INFO:tasks.workunit.client.1.smithi084.stdout:8/105: getdents d0/d11 0 2024-02-28T23:07:26.643 INFO:tasks.workunit.client.1.smithi084.stdout:7/143: creat db/d11/f29 x:0 0 0 2024-02-28T23:07:26.643 INFO:tasks.workunit.client.1.smithi084.stdout:5/152: dwrite dc/f15 [0,4194304] 0 2024-02-28T23:07:26.650 INFO:tasks.workunit.client.1.smithi084.stdout:1/240: creat d12/d1b/d2e/d32/f51 x:0 0 0 2024-02-28T23:07:26.653 INFO:tasks.workunit.client.1.smithi084.stdout:8/106: symlink d0/d11/l26 0 2024-02-28T23:07:26.655 INFO:tasks.workunit.client.1.smithi084.stdout:1/241: mkdir d12/d1b/d2e/d52 0 2024-02-28T23:07:26.657 INFO:tasks.workunit.client.1.smithi084.stdout:4/129: dread d1/f5 [0,4194304] 0 2024-02-28T23:07:26.659 INFO:tasks.workunit.client.1.smithi084.stdout:5/153: rename dc/l1e to dc/de/df/l32 0 2024-02-28T23:07:26.660 INFO:tasks.workunit.client.1.smithi084.stdout:7/144: getdents db/d10 0 2024-02-28T23:07:26.660 INFO:tasks.workunit.client.1.smithi084.stdout:1/242: symlink d12/d1b/d2e/d30/l53 0 2024-02-28T23:07:26.662 INFO:tasks.workunit.client.1.smithi084.stdout:4/130: creat d1/d18/d20/d24/f2a x:0 0 0 2024-02-28T23:07:26.663 INFO:tasks.workunit.client.1.smithi084.stdout:8/107: mknod d0/d9/c27 0 2024-02-28T23:07:26.664 INFO:tasks.workunit.client.1.smithi084.stdout:5/154: rmdir dc/de/df/d16 39 2024-02-28T23:07:26.665 INFO:tasks.workunit.client.1.smithi084.stdout:8/108: read - d0/d11/f23 zero size 2024-02-28T23:07:26.668 INFO:tasks.workunit.client.1.smithi084.stdout:8/109: write d0/d9/d1b/d20/f25 [592941,116164] 0 2024-02-28T23:07:26.671 INFO:tasks.workunit.client.1.smithi084.stdout:8/110: chown d0/d9/f21 519 1 2024-02-28T23:07:26.671 INFO:tasks.workunit.client.1.smithi084.stdout:3/129: truncate d3/d8/d1a/f1f 475633 0 2024-02-28T23:07:26.717 INFO:tasks.workunit.client.1.smithi084.stdout:6/163: sync 2024-02-28T23:07:26.727 INFO:tasks.workunit.client.1.smithi084.stdout:8/111: creat d0/d9/d1b/f28 x:0 0 0 2024-02-28T23:07:26.728 INFO:tasks.workunit.client.1.smithi084.stdout:4/131: truncate d1/f7 4077886 0 2024-02-28T23:07:26.729 INFO:tasks.workunit.client.1.smithi084.stdout:3/130: creat d3/d8/d1a/f30 x:0 0 0 2024-02-28T23:07:26.732 INFO:tasks.workunit.client.1.smithi084.stdout:6/164: mknod d1/d9/d1d/c30 0 2024-02-28T23:07:26.734 INFO:tasks.workunit.client.1.smithi084.stdout:8/112: creat d0/d11/f29 x:0 0 0 2024-02-28T23:07:26.734 INFO:tasks.workunit.client.1.smithi084.stdout:3/131: write d3/d8/d1a/f30 [254097,37602] 0 2024-02-28T23:07:26.735 INFO:tasks.workunit.client.1.smithi084.stdout:4/132: unlink d1/d18/d20/d14/c1c 0 2024-02-28T23:07:26.736 INFO:tasks.workunit.client.1.smithi084.stdout:5/155: dwrite dc/f2a [0,4194304] 0 2024-02-28T23:07:26.738 INFO:tasks.workunit.client.1.smithi084.stdout:6/165: creat d1/f31 x:0 0 0 2024-02-28T23:07:26.739 INFO:tasks.workunit.client.1.smithi084.stdout:8/113: write d0/d9/d1b/f28 [962294,31602] 0 2024-02-28T23:07:26.739 INFO:tasks.workunit.client.1.smithi084.stdout:3/132: write d3/fe [1228695,175] 0 2024-02-28T23:07:26.741 INFO:tasks.workunit.client.1.smithi084.stdout:4/133: truncate d1/d18/d20/d24/f2a 723979 0 2024-02-28T23:07:26.748 INFO:tasks.workunit.client.1.smithi084.stdout:8/114: rename d0/d11/f29 to d0/d11/f2a 0 2024-02-28T23:07:26.748 INFO:tasks.workunit.client.1.smithi084.stdout:5/156: read dc/de/df/d16/f1f [25448,47722] 0 2024-02-28T23:07:26.751 INFO:tasks.workunit.client.1.smithi084.stdout:8/115: stat d0/l5 0 2024-02-28T23:07:26.751 INFO:tasks.workunit.client.1.smithi084.stdout:5/157: fsync dc/f25 0 2024-02-28T23:07:26.755 INFO:tasks.workunit.client.1.smithi084.stdout:3/133: getdents d3/d8/d1a 0 2024-02-28T23:07:26.756 INFO:tasks.workunit.client.1.smithi084.stdout:5/158: fdatasync f7 0 2024-02-28T23:07:26.756 INFO:tasks.workunit.client.1.smithi084.stdout:3/134: chown f1 523563 1 2024-02-28T23:07:26.761 INFO:tasks.workunit.client.1.smithi084.stdout:6/166: chown d1/d9/c19 826157 1 2024-02-28T23:07:26.761 INFO:tasks.workunit.client.1.smithi084.stdout:4/134: dwrite d1/f1d [0,4194304] 0 2024-02-28T23:07:26.762 INFO:tasks.workunit.client.1.smithi084.stdout:8/116: rename d0/c1e to d0/d9/c2b 0 2024-02-28T23:07:26.763 INFO:tasks.workunit.client.1.smithi084.stdout:6/167: truncate d1/f31 324933 0 2024-02-28T23:07:26.766 INFO:tasks.workunit.client.1.smithi084.stdout:3/135: mkdir d3/d8/d14/d31 0 2024-02-28T23:07:26.767 INFO:tasks.workunit.client.1.smithi084.stdout:3/136: write d3/d18/f22 [1569201,57938] 0 2024-02-28T23:07:26.768 INFO:tasks.workunit.client.1.smithi084.stdout:8/117: mknod d0/d24/c2c 0 2024-02-28T23:07:26.771 INFO:tasks.workunit.client.1.smithi084.stdout:3/137: readlink d3/d8/l9 0 2024-02-28T23:07:26.772 INFO:tasks.workunit.client.1.smithi084.stdout:6/168: link d1/d9/d1d/d1e/f26 d1/d9/d1d/f32 0 2024-02-28T23:07:26.774 INFO:tasks.workunit.client.1.smithi084.stdout:8/118: creat d0/d11/f2d x:0 0 0 2024-02-28T23:07:26.775 INFO:tasks.workunit.client.1.smithi084.stdout:5/159: link dc/de/df/l32 dc/de/df/l33 0 2024-02-28T23:07:26.776 INFO:tasks.workunit.client.1.smithi084.stdout:8/119: dread - d0/f1a zero size 2024-02-28T23:07:26.779 INFO:tasks.workunit.client.1.smithi084.stdout:3/138: rename d3/d18/f22 to d3/d8/d1a/d21/f32 0 2024-02-28T23:07:26.780 INFO:tasks.workunit.client.1.smithi084.stdout:8/120: mkdir d0/d9/d1b/d20/d2e 0 2024-02-28T23:07:26.780 INFO:tasks.workunit.client.1.smithi084.stdout:5/160: chown dc/de/df/d16 0 1 2024-02-28T23:07:26.781 INFO:tasks.workunit.client.1.smithi084.stdout:3/139: symlink d3/d27/l33 0 2024-02-28T23:07:26.784 INFO:tasks.workunit.client.1.smithi084.stdout:8/121: write d0/d11/f2a [803340,26014] 0 2024-02-28T23:07:26.793 INFO:tasks.workunit.client.1.smithi084.stdout:8/122: dread d0/d9/d1b/f28 [0,4194304] 0 2024-02-28T23:07:26.796 INFO:tasks.workunit.client.1.smithi084.stdout:8/123: creat d0/d9/d1b/d20/f2f x:0 0 0 2024-02-28T23:07:26.797 INFO:tasks.workunit.client.1.smithi084.stdout:6/169: dwrite d1/d9/d1d/d1e/f26 [0,4194304] 0 2024-02-28T23:07:26.800 INFO:tasks.workunit.client.1.smithi084.stdout:5/161: dwrite dc/de/df/d16/f1b [4194304,4194304] 0 2024-02-28T23:07:26.801 INFO:tasks.workunit.client.1.smithi084.stdout:3/140: rename d3/d18 to d3/d8/d14/d31/d34 0 2024-02-28T23:07:26.803 INFO:tasks.workunit.client.1.smithi084.stdout:6/170: mkdir d1/d9/d1d/d1e/d33 0 2024-02-28T23:07:26.805 INFO:tasks.workunit.client.1.smithi084.stdout:5/162: readlink dc/de/df/l33 0 2024-02-28T23:07:26.819 INFO:tasks.workunit.client.1.smithi084.stdout:8/124: mkdir d0/d9/d30 0 2024-02-28T23:07:26.819 INFO:tasks.workunit.client.1.smithi084.stdout:8/125: dread - d0/d11/f23 zero size 2024-02-28T23:07:26.822 INFO:tasks.workunit.client.1.smithi084.stdout:3/141: fsync d3/d8/d14/f16 0 2024-02-28T23:07:26.822 INFO:tasks.workunit.client.1.smithi084.stdout:5/163: mkdir dc/de/df/d16/d34 0 2024-02-28T23:07:26.822 INFO:tasks.workunit.client.1.smithi084.stdout:3/142: stat d3/d8/d1a/d21/f32 0 2024-02-28T23:07:26.825 INFO:tasks.workunit.client.1.smithi084.stdout:6/171: link d1/d9/d14/l2a d1/d9/d1d/d1e/d2b/l34 0 2024-02-28T23:07:26.827 INFO:tasks.workunit.client.1.smithi084.stdout:3/143: creat d3/d8/d1a/d29/f35 x:0 0 0 2024-02-28T23:07:26.827 INFO:tasks.workunit.client.1.smithi084.stdout:6/172: chown d1/d9/d1d/d1e 495575 1 2024-02-28T23:07:26.830 INFO:tasks.workunit.client.1.smithi084.stdout:6/173: rename d1/d9/d1d to d1/d9/d1d/d1e/d33/d35 22 2024-02-28T23:07:26.830 INFO:tasks.workunit.client.1.smithi084.stdout:3/144: write d3/d8/f24 [1954182,85844] 0 2024-02-28T23:07:26.833 INFO:tasks.workunit.client.1.smithi084.stdout:8/126: getdents d0/d9 0 2024-02-28T23:07:26.835 INFO:tasks.workunit.client.1.smithi084.stdout:8/127: readlink d0/d9/lc 0 2024-02-28T23:07:26.835 INFO:tasks.workunit.client.1.smithi084.stdout:6/174: mknod d1/d9/d14/c36 0 2024-02-28T23:07:26.837 INFO:tasks.workunit.client.1.smithi084.stdout:3/145: rename d3/l5 to d3/d8/d1a/d21/l36 0 2024-02-28T23:07:26.841 INFO:tasks.workunit.client.1.smithi084.stdout:5/164: dwrite dc/f21 [4194304,4194304] 0 2024-02-28T23:07:26.845 INFO:tasks.workunit.client.1.smithi084.stdout:8/128: dread d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:26.849 INFO:tasks.workunit.client.1.smithi084.stdout:8/129: write d0/f1a [654445,106586] 0 2024-02-28T23:07:26.850 INFO:tasks.workunit.client.1.smithi084.stdout:6/175: write d1/f24 [7986304,63341] 0 2024-02-28T23:07:26.853 INFO:tasks.workunit.client.1.smithi084.stdout:6/176: fdatasync d1/f31 0 2024-02-28T23:07:26.857 INFO:tasks.workunit.client.1.smithi084.stdout:5/165: truncate dc/f15 3663469 0 2024-02-28T23:07:26.858 INFO:tasks.workunit.client.1.smithi084.stdout:8/130: link d0/l5 d0/d9/d30/l31 0 2024-02-28T23:07:26.860 INFO:tasks.workunit.client.1.smithi084.stdout:6/177: rename d1/d9/d1d/d1e/f26 to d1/d9/d1d/d1e/d2b/f37 0 2024-02-28T23:07:26.860 INFO:tasks.workunit.client.1.smithi084.stdout:3/146: dwrite d3/d8/d1a/f30 [0,4194304] 0 2024-02-28T23:07:26.863 INFO:tasks.workunit.client.1.smithi084.stdout:5/166: symlink dc/de/df/d16/l35 0 2024-02-28T23:07:26.867 INFO:tasks.workunit.client.1.smithi084.stdout:5/167: mknod dc/c36 0 2024-02-28T23:07:26.867 INFO:tasks.workunit.client.1.smithi084.stdout:8/131: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:26.867 INFO:tasks.workunit.client.1.smithi084.stdout:5/168: stat c9 0 2024-02-28T23:07:26.868 INFO:tasks.workunit.client.1.smithi084.stdout:6/178: mkdir d1/d9/d1d/d1e/d33/d38 0 2024-02-28T23:07:26.872 INFO:tasks.workunit.client.1.smithi084.stdout:6/179: creat d1/d9/d1d/d1e/d33/f39 x:0 0 0 2024-02-28T23:07:26.876 INFO:tasks.workunit.client.1.smithi084.stdout:6/180: truncate d1/f1b 1492397 0 2024-02-28T23:07:26.876 INFO:tasks.workunit.client.1.smithi084.stdout:5/169: rename dc/de/df/l19 to dc/de/df/d16/l37 0 2024-02-28T23:07:26.877 INFO:tasks.workunit.client.1.smithi084.stdout:8/132: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:26.880 INFO:tasks.workunit.client.1.smithi084.stdout:6/181: write d1/d9/d1d/d1e/d2b/f37 [7191491,126315] 0 2024-02-28T23:07:26.881 INFO:tasks.workunit.client.1.smithi084.stdout:8/133: mknod d0/d11/c32 0 2024-02-28T23:07:26.881 INFO:tasks.workunit.client.1.smithi084.stdout:5/170: symlink dc/de/df/d31/l38 0 2024-02-28T23:07:26.884 INFO:tasks.workunit.client.1.smithi084.stdout:8/134: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:26.885 INFO:tasks.workunit.client.1.smithi084.stdout:5/171: truncate f7 1335857 0 2024-02-28T23:07:26.886 INFO:tasks.workunit.client.1.smithi084.stdout:8/135: write d0/fe [7115715,85529] 0 2024-02-28T23:07:26.890 INFO:tasks.workunit.client.1.smithi084.stdout:0/124: sync 2024-02-28T23:07:26.890 INFO:tasks.workunit.client.1.smithi084.stdout:2/159: sync 2024-02-28T23:07:26.890 INFO:tasks.workunit.client.1.smithi084.stdout:9/179: sync 2024-02-28T23:07:26.892 INFO:tasks.workunit.client.1.smithi084.stdout:8/136: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:26.892 INFO:tasks.workunit.client.1.smithi084.stdout:9/180: dread - d4/d6/f36 zero size 2024-02-28T23:07:26.894 INFO:tasks.workunit.client.1.smithi084.stdout:0/125: symlink d1/d8/l26 0 2024-02-28T23:07:26.895 INFO:tasks.workunit.client.1.smithi084.stdout:2/160: chown d5/d7/d9/f11 7370 1 2024-02-28T23:07:26.896 INFO:tasks.workunit.client.1.smithi084.stdout:0/126: chown d1/d8/d18 1009025633 1 2024-02-28T23:07:26.899 INFO:tasks.workunit.client.1.smithi084.stdout:9/181: write d4/d6/f7 [4315236,70164] 0 2024-02-28T23:07:26.903 INFO:tasks.workunit.client.1.smithi084.stdout:0/127: fdatasync d1/d8/f1c 0 2024-02-28T23:07:26.903 INFO:tasks.workunit.client.1.smithi084.stdout:6/182: dwrite d1/d9/d1d/f32 [0,4194304] 0 2024-02-28T23:07:26.910 INFO:tasks.workunit.client.1.smithi084.stdout:0/128: truncate d1/d6/f14 665515 0 2024-02-28T23:07:26.922 INFO:tasks.workunit.client.1.smithi084.stdout:3/147: dread d3/fc [0,4194304] 0 2024-02-28T23:07:26.922 INFO:tasks.workunit.client.1.smithi084.stdout:0/129: write d1/d8/f1d [772390,62065] 0 2024-02-28T23:07:26.924 INFO:tasks.workunit.client.1.smithi084.stdout:2/161: truncate d5/d7/f19 2517848 0 2024-02-28T23:07:26.932 INFO:tasks.workunit.client.1.smithi084.stdout:8/137: dwrite d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:26.935 INFO:tasks.workunit.client.1.smithi084.stdout:0/130: write d1/d6/f13 [1089581,58136] 0 2024-02-28T23:07:26.936 INFO:tasks.workunit.client.1.smithi084.stdout:9/182: dwrite d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:26.936 INFO:tasks.workunit.client.1.smithi084.stdout:3/148: symlink d3/d8/l37 0 2024-02-28T23:07:26.936 INFO:tasks.workunit.client.1.smithi084.stdout:5/172: dwrite dc/de/df/f2c [0,4194304] 0 2024-02-28T23:07:26.942 INFO:tasks.workunit.client.1.smithi084.stdout:6/183: dwrite d1/f1b [0,4194304] 0 2024-02-28T23:07:26.946 INFO:tasks.workunit.client.1.smithi084.stdout:2/162: rmdir d5/d7/d12 39 2024-02-28T23:07:26.953 INFO:tasks.workunit.client.1.smithi084.stdout:6/184: write d1/d9/f28 [3202605,96773] 0 2024-02-28T23:07:26.955 INFO:tasks.workunit.client.1.smithi084.stdout:0/131: creat d1/d8/f27 x:0 0 0 2024-02-28T23:07:26.957 INFO:tasks.workunit.client.1.smithi084.stdout:0/132: read - d1/d6/f21 zero size 2024-02-28T23:07:26.958 INFO:tasks.workunit.client.1.smithi084.stdout:9/183: dread d4/d6/d15/f2a [0,4194304] 0 2024-02-28T23:07:26.958 INFO:tasks.workunit.client.1.smithi084.stdout:6/185: fdatasync d1/d9/d1d/d1e/d33/f39 0 2024-02-28T23:07:26.961 INFO:tasks.workunit.client.1.smithi084.stdout:6/186: stat d1/d9/d1d/c30 0 2024-02-28T23:07:26.962 INFO:tasks.workunit.client.1.smithi084.stdout:7/145: sync 2024-02-28T23:07:26.965 INFO:tasks.workunit.client.1.smithi084.stdout:6/187: chown d1/f1b 4566 1 2024-02-28T23:07:26.967 INFO:tasks.workunit.client.1.smithi084.stdout:5/173: dread dc/f21 [4194304,4194304] 0 2024-02-28T23:07:26.967 INFO:tasks.workunit.client.1.smithi084.stdout:9/184: creat d4/f3b x:0 0 0 2024-02-28T23:07:26.967 INFO:tasks.workunit.client.1.smithi084.stdout:0/133: rmdir d1/d6/d1b 39 2024-02-28T23:07:26.969 INFO:tasks.workunit.client.1.smithi084.stdout:7/146: mknod db/d10/d18/c2a 0 2024-02-28T23:07:26.971 INFO:tasks.workunit.client.1.smithi084.stdout:2/163: rename d5/d7/d12/c22 to d5/d7/c2d 0 2024-02-28T23:07:26.972 INFO:tasks.workunit.client.1.smithi084.stdout:5/174: write dc/de/df/f2c [156740,33833] 0 2024-02-28T23:07:26.977 INFO:tasks.workunit.client.1.smithi084.stdout:8/138: dwrite d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:26.978 INFO:tasks.workunit.client.1.smithi084.stdout:9/185: read d4/d6/d15/f19 [1166773,59673] 0 2024-02-28T23:07:26.981 INFO:tasks.workunit.client.1.smithi084.stdout:5/175: dread dc/f25 [0,4194304] 0 2024-02-28T23:07:26.982 INFO:tasks.workunit.client.1.smithi084.stdout:2/164: rmdir d5/d1b 0 2024-02-28T23:07:26.983 INFO:tasks.workunit.client.1.smithi084.stdout:5/176: chown dc/de/df/d16/d34 3591 1 2024-02-28T23:07:26.985 INFO:tasks.workunit.client.1.smithi084.stdout:8/139: rmdir d0/d11 39 2024-02-28T23:07:26.986 INFO:tasks.workunit.client.1.smithi084.stdout:9/186: rmdir d4/d6 39 2024-02-28T23:07:26.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/149: write d3/d8/d1a/f1f [95887,126965] 0 2024-02-28T23:07:26.994 INFO:tasks.workunit.client.1.smithi084.stdout:8/140: chown d0/d11/l26 109142 1 2024-02-28T23:07:26.998 INFO:tasks.workunit.client.1.smithi084.stdout:7/147: dwrite db/f1e [0,4194304] 0 2024-02-28T23:07:27.001 INFO:tasks.workunit.client.1.smithi084.stdout:5/177: mknod dc/de/df/d16/c39 0 2024-02-28T23:07:27.002 INFO:tasks.workunit.client.1.smithi084.stdout:1/243: sync 2024-02-28T23:07:27.002 INFO:tasks.workunit.client.1.smithi084.stdout:4/135: sync 2024-02-28T23:07:27.004 INFO:tasks.workunit.client.1.smithi084.stdout:0/134: truncate d1/d6/f19 4073 0 2024-02-28T23:07:27.005 INFO:tasks.workunit.client.1.smithi084.stdout:9/187: read d4/d6/d15/f2a [72441,25936] 0 2024-02-28T23:07:27.005 INFO:tasks.workunit.client.1.smithi084.stdout:8/141: unlink d0/d11/f2d 0 2024-02-28T23:07:27.006 INFO:tasks.workunit.client.1.smithi084.stdout:1/244: chown lb 8826 1 2024-02-28T23:07:27.008 INFO:tasks.workunit.client.1.smithi084.stdout:6/188: link d1/d9/d1d/d1e/d2b/l34 d1/d13/d2e/l3a 0 2024-02-28T23:07:27.009 INFO:tasks.workunit.client.1.smithi084.stdout:4/136: fdatasync d1/d18/d20/d24/f2a 0 2024-02-28T23:07:27.010 INFO:tasks.workunit.client.1.smithi084.stdout:7/148: creat db/d11/f2b x:0 0 0 2024-02-28T23:07:27.015 INFO:tasks.workunit.client.1.smithi084.stdout:2/165: dwrite d5/d7/d9/f2c [0,4194304] 0 2024-02-28T23:07:27.015 INFO:tasks.workunit.client.1.smithi084.stdout:5/178: link dc/de/l22 dc/de/l3a 0 2024-02-28T23:07:27.018 INFO:tasks.workunit.client.1.smithi084.stdout:6/189: creat d1/d9/f3b x:0 0 0 2024-02-28T23:07:27.020 INFO:tasks.workunit.client.1.smithi084.stdout:2/166: dread - d5/f29 zero size 2024-02-28T23:07:27.023 INFO:tasks.workunit.client.1.smithi084.stdout:5/179: chown dc/de/df/d16/d34 10445286 1 2024-02-28T23:07:27.025 INFO:tasks.workunit.client.1.smithi084.stdout:0/135: fsync d1/d6/f16 0 2024-02-28T23:07:27.027 INFO:tasks.workunit.client.1.smithi084.stdout:2/167: truncate d5/dd/f16 2588213 0 2024-02-28T23:07:27.030 INFO:tasks.workunit.client.1.smithi084.stdout:4/137: rmdir d1/d18/d27 0 2024-02-28T23:07:27.031 INFO:tasks.workunit.client.1.smithi084.stdout:3/150: dwrite d3/f6 [0,4194304] 0 2024-02-28T23:07:27.033 INFO:tasks.workunit.client.1.smithi084.stdout:9/188: mkdir d4/d6/d15/d3c 0 2024-02-28T23:07:27.036 INFO:tasks.workunit.client.1.smithi084.stdout:5/180: mknod dc/de/df/d31/c3b 0 2024-02-28T23:07:27.036 INFO:tasks.workunit.client.1.smithi084.stdout:8/142: dread d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:27.039 INFO:tasks.workunit.client.1.smithi084.stdout:5/181: write dc/de/f2d [3115563,38846] 0 2024-02-28T23:07:27.039 INFO:tasks.workunit.client.1.smithi084.stdout:8/143: write d0/d9/d1b/d20/f2f [42572,22353] 0 2024-02-28T23:07:27.040 INFO:tasks.workunit.client.1.smithi084.stdout:6/190: rename d1/cc to d1/d13/c3c 0 2024-02-28T23:07:27.041 INFO:tasks.workunit.client.1.smithi084.stdout:5/182: write dc/f25 [322907,27720] 0 2024-02-28T23:07:27.045 INFO:tasks.workunit.client.1.smithi084.stdout:3/151: dread - d3/d8/df/f19 zero size 2024-02-28T23:07:27.047 INFO:tasks.workunit.client.1.smithi084.stdout:7/149: link db/d11/f16 db/d10/f2c 0 2024-02-28T23:07:27.050 INFO:tasks.workunit.client.1.smithi084.stdout:2/168: mkdir d5/d7/d2e 0 2024-02-28T23:07:27.051 INFO:tasks.workunit.client.1.smithi084.stdout:2/169: chown d5/d1f 2813954 1 2024-02-28T23:07:27.054 INFO:tasks.workunit.client.1.smithi084.stdout:1/245: dread d12/d1b/d25/f3a [0,4194304] 0 2024-02-28T23:07:27.056 INFO:tasks.workunit.client.1.smithi084.stdout:0/136: mkdir d1/d8/d18/d28 0 2024-02-28T23:07:27.056 INFO:tasks.workunit.client.1.smithi084.stdout:8/144: mkdir d0/d9/d30/d33 0 2024-02-28T23:07:27.056 INFO:tasks.workunit.client.1.smithi084.stdout:3/152: creat d3/d8/d1a/d29/f38 x:0 0 0 2024-02-28T23:07:27.057 INFO:tasks.workunit.client.1.smithi084.stdout:5/183: dread dc/f21 [4194304,4194304] 0 2024-02-28T23:07:27.059 INFO:tasks.workunit.client.1.smithi084.stdout:4/138: getdents d1/d18 0 2024-02-28T23:07:27.061 INFO:tasks.workunit.client.1.smithi084.stdout:0/137: write d1/d6/f25 [1046244,52652] 0 2024-02-28T23:07:27.061 INFO:tasks.workunit.client.1.smithi084.stdout:4/139: fdatasync d1/f12 0 2024-02-28T23:07:27.062 INFO:tasks.workunit.client.1.smithi084.stdout:1/246: write d12/d1b/d25/f3f [3021389,35215] 0 2024-02-28T23:07:27.063 INFO:tasks.workunit.client.1.smithi084.stdout:3/153: mknod d3/d8/d1a/c39 0 2024-02-28T23:07:27.064 INFO:tasks.workunit.client.1.smithi084.stdout:2/170: getdents d5/d7/d9 0 2024-02-28T23:07:27.066 INFO:tasks.workunit.client.1.smithi084.stdout:5/184: mkdir dc/de/df/d16/d34/d3c 0 2024-02-28T23:07:27.070 INFO:tasks.workunit.client.1.smithi084.stdout:2/171: fsync d5/d7/d9/f2c 0 2024-02-28T23:07:27.072 INFO:tasks.workunit.client.1.smithi084.stdout:9/189: dwrite d4/d32/f31 [0,4194304] 0 2024-02-28T23:07:27.074 INFO:tasks.workunit.client.1.smithi084.stdout:5/185: write dc/f2a [3283292,77355] 0 2024-02-28T23:07:27.075 INFO:tasks.workunit.client.1.smithi084.stdout:4/140: mknod d1/d18/d20/c2b 0 2024-02-28T23:07:27.077 INFO:tasks.workunit.client.0.smithi057.stdout:1/6: sync 2024-02-28T23:07:27.078 INFO:tasks.workunit.client.0.smithi057.stdout:5/3: sync 2024-02-28T23:07:27.078 INFO:tasks.workunit.client.0.smithi057.stdout:8/13: sync 2024-02-28T23:07:27.079 INFO:tasks.workunit.client.1.smithi084.stdout:5/186: chown dc/c36 27 1 2024-02-28T23:07:27.079 INFO:tasks.workunit.client.0.smithi057.stdout:5/4: truncate f0 583094 0 2024-02-28T23:07:27.084 INFO:tasks.workunit.client.0.smithi057.stdout:5/5: dread f0 [0,4194304] 0 2024-02-28T23:07:27.085 INFO:tasks.workunit.client.1.smithi084.stdout:8/145: rename d0/d9/d1b/d20/f2f to d0/f34 0 2024-02-28T23:07:27.088 INFO:tasks.workunit.client.1.smithi084.stdout:7/150: dwrite db/d10/d18/f25 [0,4194304] 0 2024-02-28T23:07:27.093 INFO:tasks.workunit.client.1.smithi084.stdout:9/190: creat d4/d6/d15/d12/f3d x:0 0 0 2024-02-28T23:07:27.093 INFO:tasks.workunit.client.1.smithi084.stdout:4/141: rmdir d1/d18/d20 39 2024-02-28T23:07:27.096 INFO:tasks.workunit.client.1.smithi084.stdout:4/142: read d1/f12 [411472,1995] 0 2024-02-28T23:07:27.101 INFO:tasks.workunit.client.1.smithi084.stdout:9/191: write d4/f3b [871987,52697] 0 2024-02-28T23:07:27.109 INFO:tasks.workunit.client.1.smithi084.stdout:8/146: write d0/d9/fb [2049256,52034] 0 2024-02-28T23:07:27.109 INFO:tasks.workunit.client.1.smithi084.stdout:7/151: rename db/d10/l1c to db/d11/l2d 0 2024-02-28T23:07:27.109 INFO:tasks.workunit.client.1.smithi084.stdout:1/247: dwrite d12/d1b/f31 [0,4194304] 0 2024-02-28T23:07:27.111 INFO:tasks.workunit.client.1.smithi084.stdout:9/192: read - d4/d6/f36 zero size 2024-02-28T23:07:27.116 INFO:tasks.workunit.client.0.smithi057.stdout:9/17: sync 2024-02-28T23:07:27.116 INFO:tasks.workunit.client.0.smithi057.stdout:3/18: sync 2024-02-28T23:07:27.116 INFO:tasks.workunit.client.0.smithi057.stdout:4/50: sync 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:2/102: sync 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:0/13: sync 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:6/13: sync 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:0/14: dread - no filename 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:0/15: dwrite - no filename 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.0.smithi057.stdout:7/76: sync 2024-02-28T23:07:27.117 INFO:tasks.workunit.client.1.smithi084.stdout:0/138: dwrite d1/d8/f1f [0,4194304] 0 2024-02-28T23:07:27.119 INFO:tasks.workunit.client.1.smithi084.stdout:5/187: creat dc/de/df/f3d x:0 0 0 2024-02-28T23:07:27.119 INFO:tasks.workunit.client.0.smithi057.stdout:4/51: dread - fe zero size 2024-02-28T23:07:27.120 INFO:tasks.workunit.client.0.smithi057.stdout:2/103: readlink d8/d11/d21/l22 0 2024-02-28T23:07:27.121 INFO:tasks.workunit.client.1.smithi084.stdout:7/152: creat db/d10/f2e x:0 0 0 2024-02-28T23:07:27.121 INFO:tasks.workunit.client.0.smithi057.stdout:4/52: truncate f12 997376 0 2024-02-28T23:07:27.122 INFO:tasks.workunit.client.0.smithi057.stdout:7/77: write d0/d8/f19 [315126,100250] 0 2024-02-28T23:07:27.122 INFO:tasks.workunit.client.0.smithi057.stdout:2/104: fsync d8/f23 0 2024-02-28T23:07:27.122 INFO:tasks.workunit.client.1.smithi084.stdout:6/191: rename d1/d9/d1d/d1e/d2b/l34 to d1/d9/d1d/d1e/d33/d38/l3d 0 2024-02-28T23:07:27.126 INFO:tasks.workunit.client.0.smithi057.stdout:7/78: write d0/f10 [4665286,17967] 0 2024-02-28T23:07:27.126 INFO:tasks.workunit.client.1.smithi084.stdout:1/248: unlink d12/d1b/l36 0 2024-02-28T23:07:27.126 INFO:tasks.workunit.client.0.smithi057.stdout:2/105: write d8/d11/d15/f20 [484895,46281] 0 2024-02-28T23:07:27.128 INFO:tasks.workunit.client.1.smithi084.stdout:8/147: creat d0/d9/f35 x:0 0 0 2024-02-28T23:07:27.129 INFO:tasks.workunit.client.1.smithi084.stdout:0/139: write d1/d6/f13 [2338157,35355] 0 2024-02-28T23:07:27.129 INFO:tasks.workunit.client.1.smithi084.stdout:3/154: dwrite d3/d8/d14/f16 [0,4194304] 0 2024-02-28T23:07:27.130 INFO:tasks.workunit.client.1.smithi084.stdout:4/143: symlink d1/d18/l2c 0 2024-02-28T23:07:27.133 INFO:tasks.workunit.client.1.smithi084.stdout:9/193: mknod d4/d32/d1f/d2d/c3e 0 2024-02-28T23:07:27.135 INFO:tasks.workunit.client.1.smithi084.stdout:7/153: creat db/d10/f2f x:0 0 0 2024-02-28T23:07:27.135 INFO:tasks.workunit.client.1.smithi084.stdout:0/140: readlink d1/l15 0 2024-02-28T23:07:27.136 INFO:tasks.workunit.client.1.smithi084.stdout:5/188: symlink dc/de/df/d16/l3e 0 2024-02-28T23:07:27.136 INFO:tasks.workunit.client.1.smithi084.stdout:2/172: dwrite d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:27.141 INFO:tasks.workunit.client.0.smithi057.stdout:8/14: unlink l2 0 2024-02-28T23:07:27.141 INFO:tasks.workunit.client.0.smithi057.stdout:8/15: truncate - no filename 2024-02-28T23:07:27.141 INFO:tasks.workunit.client.0.smithi057.stdout:8/16: write - no filename 2024-02-28T23:07:27.142 INFO:tasks.workunit.client.1.smithi084.stdout:0/141: write d1/d6/f25 [1265170,75452] 0 2024-02-28T23:07:27.142 INFO:tasks.workunit.client.1.smithi084.stdout:9/194: mkdir d4/d32/d3f 0 2024-02-28T23:07:27.143 INFO:tasks.workunit.client.0.smithi057.stdout:8/17: readlink l0 0 2024-02-28T23:07:27.145 INFO:tasks.workunit.client.0.smithi057.stdout:8/18: stat c1 0 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:8/19: dread - no filename 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:8/20: write - no filename 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:8/21: fsync - no filename 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:9/18: mknod c6 0 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:9/19: fdatasync - no filename 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:9/20: read - no filename 2024-02-28T23:07:27.146 INFO:tasks.workunit.client.0.smithi057.stdout:9/21: write - no filename 2024-02-28T23:07:27.147 INFO:tasks.workunit.client.0.smithi057.stdout:9/22: stat c6 0 2024-02-28T23:07:27.147 INFO:tasks.workunit.client.0.smithi057.stdout:9/23: write - no filename 2024-02-28T23:07:27.148 INFO:tasks.workunit.client.0.smithi057.stdout:9/24: dread - no filename 2024-02-28T23:07:27.148 INFO:tasks.workunit.client.0.smithi057.stdout:6/14: creat d0/f3 x:0 0 0 2024-02-28T23:07:27.148 INFO:tasks.workunit.client.0.smithi057.stdout:0/16: mknod c2 0 2024-02-28T23:07:27.149 INFO:tasks.workunit.client.0.smithi057.stdout:4/53: rename fd to f14 0 2024-02-28T23:07:27.153 INFO:tasks.workunit.client.0.smithi057.stdout:2/106: dread d8/d11/d15/f20 [0,4194304] 0 2024-02-28T23:07:27.153 INFO:tasks.workunit.client.1.smithi084.stdout:2/173: read f2 [57510,13406] 0 2024-02-28T23:07:27.156 INFO:tasks.workunit.client.0.smithi057.stdout:5/6: rename f0 to f1 0 2024-02-28T23:07:27.160 INFO:tasks.workunit.client.0.smithi057.stdout:3/19: creat d3/d8/f9 x:0 0 0 2024-02-28T23:07:27.160 INFO:tasks.workunit.client.1.smithi084.stdout:3/155: mkdir d3/d8/df/d2f/d3a 0 2024-02-28T23:07:27.160 INFO:tasks.workunit.client.1.smithi084.stdout:4/144: read d1/d18/d20/d15/f1a [34039,28770] 0 2024-02-28T23:07:27.160 INFO:tasks.workunit.client.1.smithi084.stdout:0/142: creat d1/d8/f29 x:0 0 0 2024-02-28T23:07:27.160 INFO:tasks.workunit.client.0.smithi057.stdout:7/79: dwrite d0/d5/ff [0,4194304] 0 2024-02-28T23:07:27.163 INFO:tasks.workunit.client.0.smithi057.stdout:3/20: stat d3/d4/f6 0 2024-02-28T23:07:27.164 INFO:tasks.workunit.client.0.smithi057.stdout:1/7: rmdir d1 0 2024-02-28T23:07:27.165 INFO:tasks.workunit.client.1.smithi084.stdout:9/195: mknod d4/d32/d1f/d2d/c40 0 2024-02-28T23:07:27.165 INFO:tasks.workunit.client.1.smithi084.stdout:7/154: rename db/d11/l2d to db/d10/d17/l30 0 2024-02-28T23:07:27.165 INFO:tasks.workunit.client.1.smithi084.stdout:1/249: creat d12/d1b/f54 x:0 0 0 2024-02-28T23:07:27.165 INFO:tasks.workunit.client.1.smithi084.stdout:6/192: dwrite d1/d9/d1d/d1e/d33/f39 [0,4194304] 0 2024-02-28T23:07:27.167 INFO:tasks.workunit.client.0.smithi057.stdout:9/25: symlink l7 0 2024-02-28T23:07:27.167 INFO:tasks.workunit.client.0.smithi057.stdout:2/107: chown d8/f1a 95747 1 2024-02-28T23:07:27.169 INFO:tasks.workunit.client.0.smithi057.stdout:8/22: creat d4/f5 x:0 0 0 2024-02-28T23:07:27.169 INFO:tasks.workunit.client.1.smithi084.stdout:8/148: symlink d0/d9/d1b/d20/d2e/l36 0 2024-02-28T23:07:27.169 INFO:tasks.workunit.client.1.smithi084.stdout:2/174: write d5/f10 [1280471,28734] 0 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.1.smithi084.stdout:7/155: truncate db/d11/f29 184823 0 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.1.smithi084.stdout:1/250: truncate d12/d1b/f4a 914537 0 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.0.smithi057.stdout:0/17: rename c1 to c3 0 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.0.smithi057.stdout:0/18: dwrite - no filename 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.0.smithi057.stdout:9/26: symlink l8 0 2024-02-28T23:07:27.170 INFO:tasks.workunit.client.0.smithi057.stdout:9/27: fdatasync - no filename 2024-02-28T23:07:27.171 INFO:tasks.workunit.client.0.smithi057.stdout:2/108: dread - d8/d11/f17 zero size 2024-02-28T23:07:27.172 INFO:tasks.workunit.client.0.smithi057.stdout:4/54: dread f14 [0,4194304] 0 2024-02-28T23:07:27.173 INFO:tasks.workunit.client.1.smithi084.stdout:1/251: chown d12/f15 0 1 2024-02-28T23:07:27.175 INFO:tasks.workunit.client.1.smithi084.stdout:7/156: chown db/d10/d18/c28 1081150552 1 2024-02-28T23:07:27.175 INFO:tasks.workunit.client.1.smithi084.stdout:4/145: mknod d1/d18/d20/d14/c2d 0 2024-02-28T23:07:27.176 INFO:tasks.workunit.client.1.smithi084.stdout:1/252: readlink d12/d1b/d2e/d30/l35 0 2024-02-28T23:07:27.176 INFO:tasks.workunit.client.0.smithi057.stdout:1/8: write f0 [556802,71543] 0 2024-02-28T23:07:27.176 INFO:tasks.workunit.client.0.smithi057.stdout:7/80: rename d0/d8/fd to d0/d8/d15/f1a 0 2024-02-28T23:07:27.178 INFO:tasks.workunit.client.1.smithi084.stdout:9/196: mkdir d4/d6/d15/d12/d41 0 2024-02-28T23:07:27.178 INFO:tasks.workunit.client.0.smithi057.stdout:3/21: truncate d3/d4/f6 521651 0 2024-02-28T23:07:27.180 INFO:tasks.workunit.client.0.smithi057.stdout:8/23: rename d4 to d4/d6 22 2024-02-28T23:07:27.180 INFO:tasks.workunit.client.1.smithi084.stdout:9/197: fsync d4/f1a 0 2024-02-28T23:07:27.182 INFO:tasks.workunit.client.0.smithi057.stdout:9/28: chown c6 9724 1 2024-02-28T23:07:27.182 INFO:tasks.workunit.client.0.smithi057.stdout:9/29: dread - no filename 2024-02-28T23:07:27.182 INFO:tasks.workunit.client.0.smithi057.stdout:1/9: write f0 [529685,69317] 0 2024-02-28T23:07:27.184 INFO:tasks.workunit.client.1.smithi084.stdout:9/198: dread - d4/d6/d15/f2c zero size 2024-02-28T23:07:27.186 INFO:tasks.workunit.client.0.smithi057.stdout:3/22: write d3/d4/f6 [883025,128780] 0 2024-02-28T23:07:27.188 INFO:tasks.workunit.client.0.smithi057.stdout:6/15: dwrite d0/f1 [0,4194304] 0 2024-02-28T23:07:27.188 INFO:tasks.workunit.client.0.smithi057.stdout:9/30: chown c0 771564 1 2024-02-28T23:07:27.188 INFO:tasks.workunit.client.0.smithi057.stdout:9/31: fsync - no filename 2024-02-28T23:07:27.189 INFO:tasks.workunit.client.0.smithi057.stdout:0/19: creat f4 x:0 0 0 2024-02-28T23:07:27.191 INFO:tasks.workunit.client.0.smithi057.stdout:6/16: dread - d0/f3 zero size 2024-02-28T23:07:27.191 INFO:tasks.workunit.client.1.smithi084.stdout:6/193: dwrite d1/f31 [0,4194304] 0 2024-02-28T23:07:27.191 INFO:tasks.workunit.client.0.smithi057.stdout:4/55: creat f15 x:0 0 0 2024-02-28T23:07:27.192 INFO:tasks.workunit.client.0.smithi057.stdout:3/23: dread - d3/d8/f9 zero size 2024-02-28T23:07:27.192 INFO:tasks.workunit.client.0.smithi057.stdout:0/20: dread - f4 zero size 2024-02-28T23:07:27.195 INFO:tasks.workunit.client.0.smithi057.stdout:0/21: dread - f4 zero size 2024-02-28T23:07:27.196 INFO:tasks.workunit.client.0.smithi057.stdout:2/109: symlink d8/d11/d21/l26 0 2024-02-28T23:07:27.196 INFO:tasks.workunit.client.1.smithi084.stdout:3/156: dwrite d3/fc [0,4194304] 0 2024-02-28T23:07:27.197 INFO:tasks.workunit.client.0.smithi057.stdout:7/81: mknod d0/d8/c1b 0 2024-02-28T23:07:27.198 INFO:tasks.workunit.client.0.smithi057.stdout:2/110: stat d8/d11/d15/f20 0 2024-02-28T23:07:27.200 INFO:tasks.workunit.client.0.smithi057.stdout:3/24: write d3/d4/f6 [2012007,73534] 0 2024-02-28T23:07:27.202 INFO:tasks.workunit.client.1.smithi084.stdout:3/157: write d3/f6 [5535965,78966] 0 2024-02-28T23:07:27.203 INFO:tasks.workunit.client.1.smithi084.stdout:8/149: symlink d0/d24/l37 0 2024-02-28T23:07:27.204 INFO:tasks.workunit.client.0.smithi057.stdout:9/32: mknod c9 0 2024-02-28T23:07:27.205 INFO:tasks.workunit.client.0.smithi057.stdout:4/56: creat f16 x:0 0 0 2024-02-28T23:07:27.207 INFO:tasks.workunit.client.0.smithi057.stdout:1/10: dread f0 [0,4194304] 0 2024-02-28T23:07:27.212 INFO:tasks.workunit.client.0.smithi057.stdout:0/22: creat f5 x:0 0 0 2024-02-28T23:07:27.216 INFO:tasks.workunit.client.0.smithi057.stdout:0/23: truncate f5 977003 0 2024-02-28T23:07:27.217 INFO:tasks.workunit.client.0.smithi057.stdout:3/25: creat d3/d8/fa x:0 0 0 2024-02-28T23:07:27.220 INFO:tasks.workunit.client.1.smithi084.stdout:6/194: dwrite d1/d9/f3b [0,4194304] 0 2024-02-28T23:07:27.223 INFO:tasks.workunit.client.0.smithi057.stdout:7/82: dwrite d0/d5/ff [0,4194304] 0 2024-02-28T23:07:27.225 INFO:tasks.workunit.client.0.smithi057.stdout:1/11: mkdir d2 0 2024-02-28T23:07:27.229 INFO:tasks.workunit.client.1.smithi084.stdout:6/195: write d1/d9/d1d/d1e/d33/f39 [732769,66078] 0 2024-02-28T23:07:27.232 INFO:tasks.workunit.client.1.smithi084.stdout:3/158: dwrite d3/fc [0,4194304] 0 2024-02-28T23:07:27.237 INFO:tasks.workunit.client.0.smithi057.stdout:7/83: write d0/f12 [988578,92856] 0 2024-02-28T23:07:27.237 INFO:tasks.workunit.client.0.smithi057.stdout:3/26: dread f1 [0,4194304] 0 2024-02-28T23:07:27.239 INFO:tasks.workunit.client.1.smithi084.stdout:5/189: link dc/de/df/d16/c23 dc/c3f 0 2024-02-28T23:07:27.239 INFO:tasks.workunit.client.0.smithi057.stdout:3/27: stat d3/d4 0 2024-02-28T23:07:27.242 INFO:tasks.workunit.client.0.smithi057.stdout:9/33: link c2 ca 0 2024-02-28T23:07:27.242 INFO:tasks.workunit.client.0.smithi057.stdout:9/34: dread - no filename 2024-02-28T23:07:27.242 INFO:tasks.workunit.client.0.smithi057.stdout:9/35: dwrite - no filename 2024-02-28T23:07:27.243 INFO:tasks.workunit.client.1.smithi084.stdout:5/190: read dc/de/f14 [2368894,51923] 0 2024-02-28T23:07:27.245 INFO:tasks.workunit.client.0.smithi057.stdout:7/84: mkdir d0/d1c 0 2024-02-28T23:07:27.248 INFO:tasks.workunit.client.1.smithi084.stdout:7/157: unlink db/d10/d18/f25 0 2024-02-28T23:07:27.248 INFO:tasks.workunit.client.1.smithi084.stdout:8/150: dwrite d0/fe [4194304,4194304] 0 2024-02-28T23:07:27.252 INFO:tasks.workunit.client.0.smithi057.stdout:4/57: dwrite f0 [0,4194304] 0 2024-02-28T23:07:27.257 INFO:tasks.workunit.client.1.smithi084.stdout:9/199: write d4/d6/d15/f19 [1025539,68468] 0 2024-02-28T23:07:27.259 INFO:tasks.workunit.client.0.smithi057.stdout:7/85: creat d0/d8/d15/f1d x:0 0 0 2024-02-28T23:07:27.267 INFO:tasks.workunit.client.0.smithi057.stdout:9/36: link c9 cb 0 2024-02-28T23:07:27.268 INFO:tasks.workunit.client.1.smithi084.stdout:2/175: truncate d5/d7/f19 2452811 0 2024-02-28T23:07:27.273 INFO:tasks.workunit.client.1.smithi084.stdout:3/159: mknod d3/d8/d1a/c3b 0 2024-02-28T23:07:27.274 INFO:tasks.workunit.client.0.smithi057.stdout:9/37: chown l8 216046651 1 2024-02-28T23:07:27.276 INFO:tasks.workunit.client.1.smithi084.stdout:3/160: write d3/d27/f2d [1612894,4031] 0 2024-02-28T23:07:27.278 INFO:tasks.workunit.client.0.smithi057.stdout:1/12: dwrite f0 [0,4194304] 0 2024-02-28T23:07:27.278 INFO:tasks.workunit.client.1.smithi084.stdout:5/191: unlink dc/de/df/d16/f1c 0 2024-02-28T23:07:27.281 INFO:tasks.workunit.client.0.smithi057.stdout:3/28: dwrite f1 [0,4194304] 0 2024-02-28T23:07:27.281 INFO:tasks.workunit.client.1.smithi084.stdout:5/192: write dc/de/df/f2c [774456,34861] 0 2024-02-28T23:07:27.285 INFO:tasks.workunit.client.0.smithi057.stdout:9/38: mknod cc 0 2024-02-28T23:07:27.285 INFO:tasks.workunit.client.0.smithi057.stdout:9/39: dwrite - no filename 2024-02-28T23:07:27.285 INFO:tasks.workunit.client.0.smithi057.stdout:9/40: read - no filename 2024-02-28T23:07:27.285 INFO:tasks.workunit.client.0.smithi057.stdout:9/41: read - no filename 2024-02-28T23:07:27.286 INFO:tasks.workunit.client.0.smithi057.stdout:9/42: dwrite - no filename 2024-02-28T23:07:27.286 INFO:tasks.workunit.client.0.smithi057.stdout:4/58: dread f0 [0,4194304] 0 2024-02-28T23:07:27.287 INFO:tasks.workunit.client.0.smithi057.stdout:7/86: rmdir d0/d1c 0 2024-02-28T23:07:27.295 INFO:tasks.workunit.client.0.smithi057.stdout:7/87: truncate d0/d8/d15/f1d 246466 0 2024-02-28T23:07:27.297 INFO:tasks.workunit.client.0.smithi057.stdout:9/43: mkdir dd 0 2024-02-28T23:07:27.298 INFO:tasks.workunit.client.1.smithi084.stdout:7/158: dread db/ff [0,4194304] 0 2024-02-28T23:07:27.298 INFO:tasks.workunit.client.0.smithi057.stdout:7/88: fdatasync d0/d5/ff 0 2024-02-28T23:07:27.298 INFO:tasks.workunit.client.1.smithi084.stdout:3/161: rename d3/d8/d1a/d29/f35 to d3/d8/df/d2f/d3a/f3c 0 2024-02-28T23:07:27.302 INFO:tasks.workunit.client.1.smithi084.stdout:8/151: write d0/d9/f21 [1726924,97974] 0 2024-02-28T23:07:27.304 INFO:tasks.workunit.client.1.smithi084.stdout:3/162: read - d3/d8/df/f19 zero size 2024-02-28T23:07:27.304 INFO:tasks.workunit.client.1.smithi084.stdout:9/200: creat d4/d32/d3f/f42 x:0 0 0 2024-02-28T23:07:27.307 INFO:tasks.workunit.client.1.smithi084.stdout:7/159: readlink db/d10/d17/l30 0 2024-02-28T23:07:27.308 INFO:tasks.workunit.client.1.smithi084.stdout:5/193: dwrite dc/f25 [0,4194304] 0 2024-02-28T23:07:27.309 INFO:tasks.workunit.client.0.smithi057.stdout:4/59: link fe f17 0 2024-02-28T23:07:27.312 INFO:tasks.workunit.client.0.smithi057.stdout:3/29: dread d3/d4/f6 [0,4194304] 0 2024-02-28T23:07:27.313 INFO:tasks.workunit.client.1.smithi084.stdout:3/163: readlink d3/d8/l13 0 2024-02-28T23:07:27.313 INFO:tasks.workunit.client.1.smithi084.stdout:7/160: dread - db/d11/f2b zero size 2024-02-28T23:07:27.314 INFO:tasks.workunit.client.0.smithi057.stdout:9/44: stat cb 0 2024-02-28T23:07:27.314 INFO:tasks.workunit.client.0.smithi057.stdout:7/89: unlink d0/d5/l11 0 2024-02-28T23:07:27.315 INFO:tasks.workunit.client.0.smithi057.stdout:9/45: chown dd 8562663 1 2024-02-28T23:07:27.316 INFO:tasks.workunit.client.1.smithi084.stdout:7/161: readlink db/d10/d17/l30 0 2024-02-28T23:07:27.322 INFO:tasks.workunit.client.1.smithi084.stdout:8/152: rename d0/d9/d1b/d20/d2e to d0/d9/d1b/d38 0 2024-02-28T23:07:27.323 INFO:tasks.workunit.client.0.smithi057.stdout:7/90: creat d0/d5/f1e x:0 0 0 2024-02-28T23:07:27.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/91: readlink d0/d8/la 0 2024-02-28T23:07:27.328 INFO:tasks.workunit.client.0.smithi057.stdout:4/60: rename f11 to f18 0 2024-02-28T23:07:27.329 INFO:tasks.workunit.client.0.smithi057.stdout:3/30: link d3/d8/fa d3/d8/fb 0 2024-02-28T23:07:27.329 INFO:tasks.workunit.client.1.smithi084.stdout:1/253: symlink d12/l55 0 2024-02-28T23:07:27.331 INFO:tasks.workunit.client.0.smithi057.stdout:9/46: mkdir dd/de 0 2024-02-28T23:07:27.331 INFO:tasks.workunit.client.0.smithi057.stdout:9/47: fsync - no filename 2024-02-28T23:07:27.331 INFO:tasks.workunit.client.0.smithi057.stdout:9/48: write - no filename 2024-02-28T23:07:27.331 INFO:tasks.workunit.client.0.smithi057.stdout:9/49: write - no filename 2024-02-28T23:07:27.334 INFO:tasks.workunit.client.0.smithi057.stdout:4/61: mkdir d19 0 2024-02-28T23:07:27.334 INFO:tasks.workunit.client.0.smithi057.stdout:5/7: sync 2024-02-28T23:07:27.334 INFO:tasks.workunit.client.1.smithi084.stdout:7/162: creat db/d11/f31 x:0 0 0 2024-02-28T23:07:27.334 INFO:tasks.workunit.client.1.smithi084.stdout:9/201: dwrite d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:27.337 INFO:tasks.workunit.client.0.smithi057.stdout:9/50: creat dd/ff x:0 0 0 2024-02-28T23:07:27.338 INFO:tasks.workunit.client.0.smithi057.stdout:1/13: fsync f0 0 2024-02-28T23:07:27.339 INFO:tasks.workunit.client.1.smithi084.stdout:8/153: creat d0/d9/d1b/f39 x:0 0 0 2024-02-28T23:07:27.339 INFO:tasks.workunit.client.0.smithi057.stdout:1/14: stat d2 0 2024-02-28T23:07:27.342 INFO:tasks.workunit.client.1.smithi084.stdout:5/194: dwrite dc/de/df/d16/f1b [8388608,4194304] 0 2024-02-28T23:07:27.342 INFO:tasks.workunit.client.0.smithi057.stdout:5/8: write f1 [58550,118427] 0 2024-02-28T23:07:27.343 INFO:tasks.workunit.client.1.smithi084.stdout:1/254: creat d12/d1b/d25/f56 x:0 0 0 2024-02-28T23:07:27.345 INFO:tasks.workunit.client.1.smithi084.stdout:4/146: truncate d1/f9 849293 0 2024-02-28T23:07:27.347 INFO:tasks.workunit.client.0.smithi057.stdout:4/62: rmdir d19 0 2024-02-28T23:07:27.348 INFO:tasks.workunit.client.1.smithi084.stdout:6/196: truncate d1/d9/d1d/d1e/d33/f39 3520319 0 2024-02-28T23:07:27.348 INFO:tasks.workunit.client.1.smithi084.stdout:7/163: creat db/d11/f32 x:0 0 0 2024-02-28T23:07:27.349 INFO:tasks.workunit.client.1.smithi084.stdout:9/202: symlink d4/d32/d3f/l43 0 2024-02-28T23:07:27.349 INFO:tasks.workunit.client.1.smithi084.stdout:4/147: chown d1/d18/d20/d15/f16 107 1 2024-02-28T23:07:27.350 INFO:tasks.workunit.client.1.smithi084.stdout:8/154: fdatasync d0/d9/d1b/f1d 0 2024-02-28T23:07:27.350 INFO:tasks.workunit.client.0.smithi057.stdout:5/9: dread f1 [0,4194304] 0 2024-02-28T23:07:27.351 INFO:tasks.workunit.client.1.smithi084.stdout:7/164: dread - db/d11/f2b zero size 2024-02-28T23:07:27.352 INFO:tasks.workunit.client.1.smithi084.stdout:5/195: mknod dc/de/c40 0 2024-02-28T23:07:27.354 INFO:tasks.workunit.client.0.smithi057.stdout:9/51: link c6 dd/de/c10 0 2024-02-28T23:07:27.356 INFO:tasks.workunit.client.1.smithi084.stdout:4/148: write d1/f1d [3934412,41159] 0 2024-02-28T23:07:27.358 INFO:tasks.workunit.client.1.smithi084.stdout:7/165: fdatasync db/d11/f12 0 2024-02-28T23:07:27.358 INFO:tasks.workunit.client.0.smithi057.stdout:9/52: chown l8 39262157 1 2024-02-28T23:07:27.359 INFO:tasks.workunit.client.1.smithi084.stdout:3/164: rename d3/d8/df/d2f/d3a to d3/d8/d1a/d21/d3d 0 2024-02-28T23:07:27.359 INFO:tasks.workunit.client.1.smithi084.stdout:8/155: chown d0/cf 2977850 1 2024-02-28T23:07:27.362 INFO:tasks.workunit.client.0.smithi057.stdout:5/10: dread f1 [0,4194304] 0 2024-02-28T23:07:27.363 INFO:tasks.workunit.client.0.smithi057.stdout:5/11: chown f1 3 1 2024-02-28T23:07:27.364 INFO:tasks.workunit.client.1.smithi084.stdout:4/149: rmdir d1/d18/d20/d14 39 2024-02-28T23:07:27.366 INFO:tasks.workunit.client.1.smithi084.stdout:8/156: write d0/d9/f21 [4633766,75681] 0 2024-02-28T23:07:27.367 INFO:tasks.workunit.client.0.smithi057.stdout:7/92: dwrite d0/d8/f18 [0,4194304] 0 2024-02-28T23:07:27.369 INFO:tasks.workunit.client.0.smithi057.stdout:9/53: chown cb 30221720 1 2024-02-28T23:07:27.371 INFO:tasks.workunit.client.0.smithi057.stdout:5/12: creat f2 x:0 0 0 2024-02-28T23:07:27.378 INFO:tasks.workunit.client.1.smithi084.stdout:5/196: creat dc/de/df/d31/f41 x:0 0 0 2024-02-28T23:07:27.379 INFO:tasks.workunit.client.1.smithi084.stdout:7/166: creat db/d11/d27/f33 x:0 0 0 2024-02-28T23:07:27.379 INFO:tasks.workunit.client.0.smithi057.stdout:4/63: dwrite f0 [0,4194304] 0 2024-02-28T23:07:27.381 INFO:tasks.workunit.client.1.smithi084.stdout:8/157: chown d0/f34 181 1 2024-02-28T23:07:27.383 INFO:tasks.workunit.client.1.smithi084.stdout:1/255: dwrite d12/d1b/f48 [0,4194304] 0 2024-02-28T23:07:27.390 INFO:tasks.workunit.client.1.smithi084.stdout:8/158: dread - d0/d9/d1b/f39 zero size 2024-02-28T23:07:27.391 INFO:tasks.workunit.client.0.smithi057.stdout:4/64: unlink cf 0 2024-02-28T23:07:27.398 INFO:tasks.workunit.client.1.smithi084.stdout:4/150: stat d1/d18/d20/d14/f25 0 2024-02-28T23:07:27.400 INFO:tasks.workunit.client.0.smithi057.stdout:4/65: mkdir d1a 0 2024-02-28T23:07:27.405 INFO:tasks.workunit.client.1.smithi084.stdout:2/176: dwrite d5/f15 [0,4194304] 0 2024-02-28T23:07:27.406 INFO:tasks.workunit.client.1.smithi084.stdout:1/256: creat d12/d1b/d2e/f57 x:0 0 0 2024-02-28T23:07:27.407 INFO:tasks.workunit.client.0.smithi057.stdout:9/54: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:27.408 INFO:tasks.workunit.client.1.smithi084.stdout:6/197: dread d1/d9/d1d/f32 [4194304,4194304] 0 2024-02-28T23:07:27.409 INFO:tasks.workunit.client.1.smithi084.stdout:5/197: truncate f7 864677 0 2024-02-28T23:07:27.409 INFO:tasks.workunit.client.1.smithi084.stdout:9/203: dwrite d4/d6/d15/f2c [0,4194304] 0 2024-02-28T23:07:27.410 INFO:tasks.workunit.client.0.smithi057.stdout:8/24: getdents d4 0 2024-02-28T23:07:27.410 INFO:tasks.workunit.client.1.smithi084.stdout:1/257: write d12/f23 [1686929,56188] 0 2024-02-28T23:07:27.411 INFO:tasks.workunit.client.0.smithi057.stdout:8/25: read - d4/f5 zero size 2024-02-28T23:07:27.412 INFO:tasks.workunit.client.0.smithi057.stdout:0/24: chown c3 2232 1 2024-02-28T23:07:27.414 INFO:tasks.workunit.client.1.smithi084.stdout:6/198: chown d1/d9/d1d 0 1 2024-02-28T23:07:27.414 INFO:tasks.workunit.client.1.smithi084.stdout:3/165: link d3/d8/d1a/d21/f32 d3/d8/d1a/f3e 0 2024-02-28T23:07:27.418 INFO:tasks.workunit.client.0.smithi057.stdout:4/66: mkdir d1a/d1b 0 2024-02-28T23:07:27.418 INFO:tasks.workunit.client.0.smithi057.stdout:0/25: mknod c6 0 2024-02-28T23:07:27.419 INFO:tasks.workunit.client.0.smithi057.stdout:0/26: dread - f4 zero size 2024-02-28T23:07:27.422 INFO:tasks.workunit.client.1.smithi084.stdout:4/151: creat d1/f2e x:0 0 0 2024-02-28T23:07:27.423 INFO:tasks.workunit.client.0.smithi057.stdout:8/26: creat d4/f7 x:0 0 0 2024-02-28T23:07:27.423 INFO:tasks.workunit.client.0.smithi057.stdout:4/67: rmdir d1a 39 2024-02-28T23:07:27.424 INFO:tasks.workunit.client.1.smithi084.stdout:4/152: read d1/d18/d20/d15/f1a [1864,5906] 0 2024-02-28T23:07:27.425 INFO:tasks.workunit.client.0.smithi057.stdout:8/27: readlink l0 0 2024-02-28T23:07:27.427 INFO:tasks.workunit.client.0.smithi057.stdout:2/111: rmdir d8/d11 39 2024-02-28T23:07:27.428 INFO:tasks.workunit.client.0.smithi057.stdout:8/28: dread - d4/f7 zero size 2024-02-28T23:07:27.430 INFO:tasks.workunit.client.1.smithi084.stdout:0/143: sync 2024-02-28T23:07:27.434 INFO:tasks.workunit.client.0.smithi057.stdout:6/17: truncate d0/f1 2733959 0 2024-02-28T23:07:27.435 INFO:tasks.workunit.client.1.smithi084.stdout:1/258: creat d12/d1b/d2e/d30/f58 x:0 0 0 2024-02-28T23:07:27.435 INFO:tasks.workunit.client.1.smithi084.stdout:6/199: chown d1/d13 10 1 2024-02-28T23:07:27.437 INFO:tasks.workunit.client.0.smithi057.stdout:9/55: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:27.441 INFO:tasks.workunit.client.1.smithi084.stdout:3/166: dread d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:27.445 INFO:tasks.workunit.client.1.smithi084.stdout:8/159: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:27.445 INFO:tasks.workunit.client.1.smithi084.stdout:4/153: write d1/d18/d20/d15/f28 [42819,126848] 0 2024-02-28T23:07:27.447 INFO:tasks.workunit.client.0.smithi057.stdout:9/56: rename dd to dd/d11 22 2024-02-28T23:07:27.447 INFO:tasks.workunit.client.1.smithi084.stdout:0/144: symlink d1/d8/d18/l2a 0 2024-02-28T23:07:27.448 INFO:tasks.workunit.client.1.smithi084.stdout:3/167: readlink d3/d8/l37 0 2024-02-28T23:07:27.449 INFO:tasks.workunit.client.0.smithi057.stdout:2/112: getdents d8 0 2024-02-28T23:07:27.455 INFO:tasks.workunit.client.1.smithi084.stdout:7/167: link db/d11/c14 db/d11/c34 0 2024-02-28T23:07:27.455 INFO:tasks.workunit.client.1.smithi084.stdout:6/200: rename d1/d9/d14 to d1/d13/d2e/d3e 0 2024-02-28T23:07:27.458 INFO:tasks.workunit.client.0.smithi057.stdout:9/57: creat dd/f12 x:0 0 0 2024-02-28T23:07:27.459 INFO:tasks.workunit.client.1.smithi084.stdout:2/177: write d5/f17 [3459742,2653] 0 2024-02-28T23:07:27.462 INFO:tasks.workunit.client.1.smithi084.stdout:6/201: write d1/d9/f3b [4545885,109574] 0 2024-02-28T23:07:27.462 INFO:tasks.workunit.client.1.smithi084.stdout:5/198: dwrite dc/f21 [0,4194304] 0 2024-02-28T23:07:27.463 INFO:tasks.workunit.client.0.smithi057.stdout:0/27: dwrite f5 [0,4194304] 0 2024-02-28T23:07:27.465 INFO:tasks.workunit.client.1.smithi084.stdout:2/178: write d5/d7/fe [2096230,113934] 0 2024-02-28T23:07:27.465 INFO:tasks.workunit.client.1.smithi084.stdout:8/160: unlink d0/d11/l18 0 2024-02-28T23:07:27.467 INFO:tasks.workunit.client.1.smithi084.stdout:8/161: fsync d0/fe 0 2024-02-28T23:07:27.471 INFO:tasks.workunit.client.0.smithi057.stdout:7/93: truncate d0/f10 3660459 0 2024-02-28T23:07:27.477 INFO:tasks.workunit.client.0.smithi057.stdout:5/13: dwrite f1 [0,4194304] 0 2024-02-28T23:07:27.477 INFO:tasks.workunit.client.0.smithi057.stdout:4/68: dwrite f15 [0,4194304] 0 2024-02-28T23:07:27.478 INFO:tasks.workunit.client.1.smithi084.stdout:1/259: link d12/d1b/d2e/f57 d12/d1b/d3b/f59 0 2024-02-28T23:07:27.485 INFO:tasks.workunit.client.0.smithi057.stdout:2/113: link d8/fb d8/d11/f27 0 2024-02-28T23:07:27.488 INFO:tasks.workunit.client.0.smithi057.stdout:5/14: symlink l3 0 2024-02-28T23:07:27.493 INFO:tasks.workunit.client.1.smithi084.stdout:6/202: fsync d1/f24 0 2024-02-28T23:07:27.494 INFO:tasks.workunit.client.0.smithi057.stdout:8/29: dwrite d4/f7 [0,4194304] 0 2024-02-28T23:07:27.495 INFO:tasks.workunit.client.1.smithi084.stdout:0/145: dwrite d1/d6/f13 [0,4194304] 0 2024-02-28T23:07:27.501 INFO:tasks.workunit.client.0.smithi057.stdout:8/30: creat d4/f8 x:0 0 0 2024-02-28T23:07:27.501 INFO:tasks.workunit.client.0.smithi057.stdout:6/18: dwrite d0/f3 [0,4194304] 0 2024-02-28T23:07:27.502 INFO:tasks.workunit.client.1.smithi084.stdout:7/168: dread db/d10/f2c [0,4194304] 0 2024-02-28T23:07:27.502 INFO:tasks.workunit.client.1.smithi084.stdout:0/146: chown d1/d8/f1f 8 1 2024-02-28T23:07:27.502 INFO:tasks.workunit.client.1.smithi084.stdout:3/168: dwrite d3/fe [0,4194304] 0 2024-02-28T23:07:27.504 INFO:tasks.workunit.client.1.smithi084.stdout:1/260: truncate d12/f23 1937466 0 2024-02-28T23:07:27.506 INFO:tasks.workunit.client.1.smithi084.stdout:9/204: dread d4/fc [4194304,4194304] 0 2024-02-28T23:07:27.506 INFO:tasks.workunit.client.1.smithi084.stdout:0/147: readlink d1/l4 0 2024-02-28T23:07:27.507 INFO:tasks.workunit.client.0.smithi057.stdout:2/114: mkdir d8/d11/d21/d28 0 2024-02-28T23:07:27.510 INFO:tasks.workunit.client.1.smithi084.stdout:6/203: mknod d1/d9/d1d/d1e/d2b/d2c/c3f 0 2024-02-28T23:07:27.516 INFO:tasks.workunit.client.1.smithi084.stdout:1/261: fdatasync d12/d1b/d2e/f4e 0 2024-02-28T23:07:27.517 INFO:tasks.workunit.client.1.smithi084.stdout:7/169: creat db/d11/d27/f35 x:0 0 0 2024-02-28T23:07:27.522 INFO:tasks.workunit.client.1.smithi084.stdout:9/205: chown d4/d6/d15/d12/c28 225 1 2024-02-28T23:07:27.524 INFO:tasks.workunit.client.0.smithi057.stdout:8/31: link d4/f8 d4/f9 0 2024-02-28T23:07:27.527 INFO:tasks.workunit.client.0.smithi057.stdout:9/58: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:27.528 INFO:tasks.workunit.client.0.smithi057.stdout:8/32: stat l0 0 2024-02-28T23:07:27.528 INFO:tasks.workunit.client.0.smithi057.stdout:2/115: rmdir d8/d11/d21 39 2024-02-28T23:07:27.531 INFO:tasks.workunit.client.0.smithi057.stdout:4/69: write fe [903361,7131] 0 2024-02-28T23:07:27.531 INFO:tasks.workunit.client.1.smithi084.stdout:0/148: unlink d1/d6/f25 0 2024-02-28T23:07:27.533 INFO:tasks.workunit.client.0.smithi057.stdout:0/28: dwrite f5 [0,4194304] 0 2024-02-28T23:07:27.534 INFO:tasks.workunit.client.1.smithi084.stdout:5/199: dwrite dc/de/df/f3d [0,4194304] 0 2024-02-28T23:07:27.540 INFO:tasks.workunit.client.0.smithi057.stdout:8/33: chown d4/f8 179914 1 2024-02-28T23:07:27.540 INFO:tasks.workunit.client.0.smithi057.stdout:0/29: write f5 [4010939,7349] 0 2024-02-28T23:07:27.541 INFO:tasks.workunit.client.0.smithi057.stdout:4/70: write f14 [1019123,15083] 0 2024-02-28T23:07:27.542 INFO:tasks.workunit.client.0.smithi057.stdout:9/59: write dd/f12 [284401,52789] 0 2024-02-28T23:07:27.542 INFO:tasks.workunit.client.1.smithi084.stdout:3/169: link d3/d8/d1a/d29/f38 d3/d8/df/d2f/f3f 0 2024-02-28T23:07:27.543 INFO:tasks.workunit.client.1.smithi084.stdout:8/162: dwrite d0/d11/f23 [0,4194304] 0 2024-02-28T23:07:27.544 INFO:tasks.workunit.client.0.smithi057.stdout:7/94: dwrite d0/d8/f18 [0,4194304] 0 2024-02-28T23:07:27.545 INFO:tasks.workunit.client.0.smithi057.stdout:6/19: dread d0/f3 [0,4194304] 0 2024-02-28T23:07:27.548 INFO:tasks.workunit.client.1.smithi084.stdout:2/179: write d5/d7/d9/f11 [267673,56338] 0 2024-02-28T23:07:27.553 INFO:tasks.workunit.client.0.smithi057.stdout:5/15: dwrite f1 [0,4194304] 0 2024-02-28T23:07:27.554 INFO:tasks.workunit.client.1.smithi084.stdout:5/200: mknod dc/de/df/d31/c42 0 2024-02-28T23:07:27.554 INFO:tasks.workunit.client.0.smithi057.stdout:6/20: unlink d0/f3 0 2024-02-28T23:07:27.556 INFO:tasks.workunit.client.0.smithi057.stdout:8/34: creat d4/fa x:0 0 0 2024-02-28T23:07:27.556 INFO:tasks.workunit.client.0.smithi057.stdout:9/60: write dd/ff [2087231,84485] 0 2024-02-28T23:07:27.557 INFO:tasks.workunit.client.1.smithi084.stdout:3/170: mknod d3/d8/d1a/d29/c40 0 2024-02-28T23:07:27.557 INFO:tasks.workunit.client.1.smithi084.stdout:2/180: unlink d5/d1f/l20 0 2024-02-28T23:07:27.561 INFO:tasks.workunit.client.0.smithi057.stdout:2/116: rename d8/d11/f18 to d8/d11/f29 0 2024-02-28T23:07:27.565 INFO:tasks.workunit.client.0.smithi057.stdout:4/71: rename fa to d1a/d1b/f1c 0 2024-02-28T23:07:27.565 INFO:tasks.workunit.client.0.smithi057.stdout:7/95: dread d0/d8/d15/f1d [0,4194304] 0 2024-02-28T23:07:27.565 INFO:tasks.workunit.client.1.smithi084.stdout:6/204: dread d1/f1b [0,4194304] 0 2024-02-28T23:07:27.565 INFO:tasks.workunit.client.1.smithi084.stdout:8/163: creat d0/d9/d1b/d38/f3a x:0 0 0 2024-02-28T23:07:27.566 INFO:tasks.workunit.client.0.smithi057.stdout:8/35: rename d4 to d4/db 22 2024-02-28T23:07:27.567 INFO:tasks.workunit.client.1.smithi084.stdout:3/171: write d3/f6 [1142569,5769] 0 2024-02-28T23:07:27.567 INFO:tasks.workunit.client.1.smithi084.stdout:2/181: readlink d5/l6 0 2024-02-28T23:07:27.571 INFO:tasks.workunit.client.0.smithi057.stdout:6/21: unlink d0/f1 0 2024-02-28T23:07:27.571 INFO:tasks.workunit.client.0.smithi057.stdout:6/22: chown d0 9996529 1 2024-02-28T23:07:27.571 INFO:tasks.workunit.client.0.smithi057.stdout:6/23: dwrite - no filename 2024-02-28T23:07:27.572 INFO:tasks.workunit.client.0.smithi057.stdout:9/61: chown c9 154849 1 2024-02-28T23:07:27.572 INFO:tasks.workunit.client.0.smithi057.stdout:9/62: write dd/ff [3913299,87294] 0 2024-02-28T23:07:27.572 INFO:tasks.workunit.client.1.smithi084.stdout:5/201: write dc/f25 [2995275,23412] 0 2024-02-28T23:07:27.574 INFO:tasks.workunit.client.0.smithi057.stdout:7/96: fsync d0/d8/d15/f1d 0 2024-02-28T23:07:27.576 INFO:tasks.workunit.client.1.smithi084.stdout:9/206: dwrite d4/d6/d15/d12/f3d [0,4194304] 0 2024-02-28T23:07:27.577 INFO:tasks.workunit.client.0.smithi057.stdout:4/72: write d1a/d1b/f1c [550283,15921] 0 2024-02-28T23:07:27.582 INFO:tasks.workunit.client.1.smithi084.stdout:8/164: creat d0/d9/d30/f3b x:0 0 0 2024-02-28T23:07:27.582 INFO:tasks.workunit.client.1.smithi084.stdout:1/262: dwrite fc [0,4194304] 0 2024-02-28T23:07:27.583 INFO:tasks.workunit.client.1.smithi084.stdout:6/205: rename d1/d9/f21 to d1/d9/f40 0 2024-02-28T23:07:27.586 INFO:tasks.workunit.client.1.smithi084.stdout:5/202: creat dc/de/df/d31/f43 x:0 0 0 2024-02-28T23:07:27.586 INFO:tasks.workunit.client.1.smithi084.stdout:3/172: truncate d3/d8/d1a/d21/d3d/f3c 250507 0 2024-02-28T23:07:27.587 INFO:tasks.workunit.client.0.smithi057.stdout:4/73: mknod d1a/c1d 0 2024-02-28T23:07:27.587 INFO:tasks.workunit.client.0.smithi057.stdout:2/117: mknod d8/d11/d21/d28/c2a 0 2024-02-28T23:07:27.587 INFO:tasks.workunit.client.1.smithi084.stdout:9/207: symlink d4/d32/d1f/d2d/l44 0 2024-02-28T23:07:27.589 INFO:tasks.workunit.client.1.smithi084.stdout:6/206: fsync d1/d9/d1d/d1e/d2b/f2f 0 2024-02-28T23:07:27.590 INFO:tasks.workunit.client.0.smithi057.stdout:5/16: dwrite f1 [4194304,4194304] 0 2024-02-28T23:07:27.590 INFO:tasks.workunit.client.1.smithi084.stdout:1/263: creat d12/d1b/d2e/d52/f5a x:0 0 0 2024-02-28T23:07:27.592 INFO:tasks.workunit.client.0.smithi057.stdout:5/17: chown l3 216273 1 2024-02-28T23:07:27.594 INFO:tasks.workunit.client.0.smithi057.stdout:9/63: link c6 dd/de/c13 0 2024-02-28T23:07:27.594 INFO:tasks.workunit.client.0.smithi057.stdout:7/97: getdents d0/d5 0 2024-02-28T23:07:27.594 INFO:tasks.workunit.client.0.smithi057.stdout:5/18: dread - f2 zero size 2024-02-28T23:07:27.595 INFO:tasks.workunit.client.1.smithi084.stdout:8/165: dread d0/f1a [0,4194304] 0 2024-02-28T23:07:27.595 INFO:tasks.workunit.client.1.smithi084.stdout:5/203: symlink dc/de/l44 0 2024-02-28T23:07:27.595 INFO:tasks.workunit.client.0.smithi057.stdout:5/19: dread - f2 zero size 2024-02-28T23:07:27.596 INFO:tasks.workunit.client.1.smithi084.stdout:9/208: unlink d4/l16 0 2024-02-28T23:07:27.597 INFO:tasks.workunit.client.0.smithi057.stdout:2/118: write d8/d11/f29 [4032172,80131] 0 2024-02-28T23:07:27.598 INFO:tasks.workunit.client.0.smithi057.stdout:7/98: unlink d0/d8/f18 0 2024-02-28T23:07:27.601 INFO:tasks.workunit.client.1.smithi084.stdout:5/204: creat dc/de/df/d16/f45 x:0 0 0 2024-02-28T23:07:27.602 INFO:tasks.workunit.client.0.smithi057.stdout:5/20: mkdir d4 0 2024-02-28T23:07:27.602 INFO:tasks.workunit.client.1.smithi084.stdout:1/264: mkdir d12/d1b/d2e/d52/d5b 0 2024-02-28T23:07:27.606 INFO:tasks.workunit.client.1.smithi084.stdout:5/205: creat dc/de/f46 x:0 0 0 2024-02-28T23:07:27.607 INFO:tasks.workunit.client.0.smithi057.stdout:7/99: mknod d0/d8/c1f 0 2024-02-28T23:07:27.608 INFO:tasks.workunit.client.0.smithi057.stdout:9/64: mkdir dd/de/d14 0 2024-02-28T23:07:27.609 INFO:tasks.workunit.client.1.smithi084.stdout:1/265: mknod d12/d1b/d25/c5c 0 2024-02-28T23:07:27.613 INFO:tasks.workunit.client.0.smithi057.stdout:7/100: getdents d0/d5 0 2024-02-28T23:07:27.615 INFO:tasks.workunit.client.0.smithi057.stdout:9/65: mknod dd/de/d14/c15 0 2024-02-28T23:07:27.617 INFO:tasks.workunit.client.1.smithi084.stdout:9/209: rename d4/d32/d1f/d2d/c3e to d4/d32/d1f/c45 0 2024-02-28T23:07:27.618 INFO:tasks.workunit.client.1.smithi084.stdout:8/166: link d0/d11/f2a d0/d11/f3c 0 2024-02-28T23:07:27.618 INFO:tasks.workunit.client.1.smithi084.stdout:1/266: readlink d12/d1b/l20 0 2024-02-28T23:07:27.618 INFO:tasks.workunit.client.1.smithi084.stdout:5/206: mknod dc/de/df/d31/c47 0 2024-02-28T23:07:27.623 INFO:tasks.workunit.client.0.smithi057.stdout:5/21: dwrite f2 [0,4194304] 0 2024-02-28T23:07:27.625 INFO:tasks.workunit.client.0.smithi057.stdout:6/24: getdents d0 0 2024-02-28T23:07:27.625 INFO:tasks.workunit.client.0.smithi057.stdout:6/25: fsync - no filename 2024-02-28T23:07:27.625 INFO:tasks.workunit.client.0.smithi057.stdout:8/36: truncate d4/f7 3800347 0 2024-02-28T23:07:27.626 INFO:tasks.workunit.client.1.smithi084.stdout:6/207: dwrite d1/f31 [0,4194304] 0 2024-02-28T23:07:27.627 INFO:tasks.workunit.client.0.smithi057.stdout:8/37: fsync d4/f5 0 2024-02-28T23:07:27.630 INFO:tasks.workunit.client.0.smithi057.stdout:6/26: mkdir d0/d4 0 2024-02-28T23:07:27.630 INFO:tasks.workunit.client.0.smithi057.stdout:6/27: write - no filename 2024-02-28T23:07:27.630 INFO:tasks.workunit.client.0.smithi057.stdout:6/28: fsync - no filename 2024-02-28T23:07:27.631 INFO:tasks.workunit.client.0.smithi057.stdout:8/38: truncate d4/f8 371350 0 2024-02-28T23:07:27.631 INFO:tasks.workunit.client.1.smithi084.stdout:3/173: dread d3/d27/f2d [0,4194304] 0 2024-02-28T23:07:27.632 INFO:tasks.workunit.client.0.smithi057.stdout:6/29: chown d0/d4 49207689 1 2024-02-28T23:07:27.633 INFO:tasks.workunit.client.0.smithi057.stdout:8/39: stat c3 0 2024-02-28T23:07:27.633 INFO:tasks.workunit.client.0.smithi057.stdout:7/101: dread d0/f12 [0,4194304] 0 2024-02-28T23:07:27.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/30: chown d0/d4 5240 1 2024-02-28T23:07:27.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/31: fsync - no filename 2024-02-28T23:07:27.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/32: fsync - no filename 2024-02-28T23:07:27.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/33: dread - no filename 2024-02-28T23:07:27.637 INFO:tasks.workunit.client.0.smithi057.stdout:6/34: mknod d0/c5 0 2024-02-28T23:07:27.637 INFO:tasks.workunit.client.0.smithi057.stdout:6/35: dwrite - no filename 2024-02-28T23:07:27.638 INFO:tasks.workunit.client.0.smithi057.stdout:6/36: chown d0 17 1 2024-02-28T23:07:27.638 INFO:tasks.workunit.client.0.smithi057.stdout:6/37: dread - no filename 2024-02-28T23:07:27.643 INFO:tasks.workunit.client.0.smithi057.stdout:4/74: fsync fe 0 2024-02-28T23:07:27.644 INFO:tasks.workunit.client.1.smithi084.stdout:3/174: read d3/d27/f2d [2954405,119453] 0 2024-02-28T23:07:27.644 INFO:tasks.workunit.client.1.smithi084.stdout:6/208: write d1/f1b [1516892,11524] 0 2024-02-28T23:07:27.644 INFO:tasks.workunit.client.0.smithi057.stdout:7/102: fsync d0/d8/d15/f1a 0 2024-02-28T23:07:27.646 INFO:tasks.workunit.client.0.smithi057.stdout:9/66: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:27.647 INFO:tasks.workunit.client.0.smithi057.stdout:6/38: rename d0/c5 to d0/d4/c6 0 2024-02-28T23:07:27.647 INFO:tasks.workunit.client.0.smithi057.stdout:6/39: read - no filename 2024-02-28T23:07:27.647 INFO:tasks.workunit.client.0.smithi057.stdout:6/40: read - no filename 2024-02-28T23:07:27.647 INFO:tasks.workunit.client.0.smithi057.stdout:6/41: truncate - no filename 2024-02-28T23:07:27.649 INFO:tasks.workunit.client.0.smithi057.stdout:2/119: truncate d8/d11/f1c 6462112 0 2024-02-28T23:07:27.652 INFO:tasks.workunit.client.1.smithi084.stdout:9/210: dwrite d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:27.652 INFO:tasks.workunit.client.0.smithi057.stdout:2/120: chown d8/d11 31307439 1 2024-02-28T23:07:27.652 INFO:tasks.workunit.client.0.smithi057.stdout:7/103: mkdir d0/d20 0 2024-02-28T23:07:27.656 INFO:tasks.workunit.client.0.smithi057.stdout:6/42: creat d0/d4/f7 x:0 0 0 2024-02-28T23:07:27.656 INFO:tasks.workunit.client.0.smithi057.stdout:2/121: symlink d8/l2b 0 2024-02-28T23:07:27.661 INFO:tasks.workunit.client.1.smithi084.stdout:6/209: fsync d1/d9/d1d/d1e/d2b/f2f 0 2024-02-28T23:07:27.663 INFO:tasks.workunit.client.1.smithi084.stdout:2/182: dread d5/d7/f19 [0,4194304] 0 2024-02-28T23:07:27.664 INFO:tasks.workunit.client.0.smithi057.stdout:8/40: fsync d4/f7 0 2024-02-28T23:07:27.664 INFO:tasks.workunit.client.0.smithi057.stdout:9/67: dread dd/f12 [0,4194304] 0 2024-02-28T23:07:27.664 INFO:tasks.workunit.client.0.smithi057.stdout:4/75: dread fe [0,4194304] 0 2024-02-28T23:07:27.665 INFO:tasks.workunit.client.0.smithi057.stdout:6/43: mknod d0/d4/c8 0 2024-02-28T23:07:27.668 INFO:tasks.workunit.client.1.smithi084.stdout:8/167: dread d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:27.669 INFO:tasks.workunit.client.1.smithi084.stdout:4/154: dwrite d1/f9 [0,4194304] 0 2024-02-28T23:07:27.669 INFO:tasks.workunit.client.0.smithi057.stdout:4/76: mkdir d1a/d1e 0 2024-02-28T23:07:27.670 INFO:tasks.workunit.client.0.smithi057.stdout:8/41: creat d4/fc x:0 0 0 2024-02-28T23:07:27.670 INFO:tasks.workunit.client.1.smithi084.stdout:1/267: dwrite d12/d1b/d2e/d30/f26 [0,4194304] 0 2024-02-28T23:07:27.670 INFO:tasks.workunit.client.0.smithi057.stdout:9/68: write dd/ff [3706432,90804] 0 2024-02-28T23:07:27.672 INFO:tasks.workunit.client.0.smithi057.stdout:6/44: creat d0/f9 x:0 0 0 2024-02-28T23:07:27.672 INFO:tasks.workunit.client.1.smithi084.stdout:2/183: dread - d5/f28 zero size 2024-02-28T23:07:27.674 INFO:tasks.workunit.client.1.smithi084.stdout:8/168: write d0/fe [6952363,6370] 0 2024-02-28T23:07:27.674 INFO:tasks.workunit.client.0.smithi057.stdout:0/30: fdatasync f5 0 2024-02-28T23:07:27.675 INFO:tasks.workunit.client.1.smithi084.stdout:8/169: chown d0/d9 2 1 2024-02-28T23:07:27.676 INFO:tasks.workunit.client.0.smithi057.stdout:6/45: truncate d0/f9 280159 0 2024-02-28T23:07:27.676 INFO:tasks.workunit.client.0.smithi057.stdout:6/46: readlink - no filename 2024-02-28T23:07:27.676 INFO:tasks.workunit.client.1.smithi084.stdout:5/207: dwrite dc/de/df/f3d [0,4194304] 0 2024-02-28T23:07:27.677 INFO:tasks.workunit.client.1.smithi084.stdout:3/175: mkdir d3/d8/d1a/d29/d41 0 2024-02-28T23:07:27.677 INFO:tasks.workunit.client.0.smithi057.stdout:0/31: dread - f4 zero size 2024-02-28T23:07:27.680 INFO:tasks.workunit.client.0.smithi057.stdout:7/104: dwrite d0/d8/f16 [0,4194304] 0 2024-02-28T23:07:27.680 INFO:tasks.workunit.client.0.smithi057.stdout:8/42: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:27.681 INFO:tasks.workunit.client.0.smithi057.stdout:9/69: fsync dd/f12 0 2024-02-28T23:07:27.682 INFO:tasks.workunit.client.0.smithi057.stdout:0/32: write f5 [1881441,106889] 0 2024-02-28T23:07:27.683 INFO:tasks.workunit.client.1.smithi084.stdout:1/268: fdatasync f1 0 2024-02-28T23:07:27.689 INFO:tasks.workunit.client.1.smithi084.stdout:6/210: rename d1/d9/d1d/d1e/d2b to d1/d13/d2e/d41 0 2024-02-28T23:07:27.693 INFO:tasks.workunit.client.0.smithi057.stdout:8/43: fsync d4/f9 0 2024-02-28T23:07:27.694 INFO:tasks.workunit.client.0.smithi057.stdout:9/70: rmdir dd 39 2024-02-28T23:07:27.694 INFO:tasks.workunit.client.0.smithi057.stdout:7/105: dread d0/d8/f19 [0,4194304] 0 2024-02-28T23:07:27.695 INFO:tasks.workunit.client.0.smithi057.stdout:0/33: symlink l7 0 2024-02-28T23:07:27.696 INFO:tasks.workunit.client.1.smithi084.stdout:5/208: creat dc/f48 x:0 0 0 2024-02-28T23:07:27.697 INFO:tasks.workunit.client.1.smithi084.stdout:3/176: mknod d3/d8/d14/c42 0 2024-02-28T23:07:27.699 INFO:tasks.workunit.client.1.smithi084.stdout:9/211: creat d4/d6/d15/d12/d41/f46 x:0 0 0 2024-02-28T23:07:27.700 INFO:tasks.workunit.client.0.smithi057.stdout:7/106: write d0/d8/d15/f1a [3158981,3806] 0 2024-02-28T23:07:27.702 INFO:tasks.workunit.client.1.smithi084.stdout:5/209: read dc/f21 [5099705,129658] 0 2024-02-28T23:07:27.704 INFO:tasks.workunit.client.0.smithi057.stdout:9/71: rename l4 to dd/de/l16 0 2024-02-28T23:07:27.705 INFO:tasks.workunit.client.0.smithi057.stdout:4/77: dwrite f12 [0,4194304] 0 2024-02-28T23:07:27.705 INFO:tasks.workunit.client.0.smithi057.stdout:7/107: dread - d0/d5/f1e zero size 2024-02-28T23:07:27.705 INFO:tasks.workunit.client.1.smithi084.stdout:5/210: write dc/f28 [820091,100791] 0 2024-02-28T23:07:27.706 INFO:tasks.workunit.client.1.smithi084.stdout:6/211: readlink d1/d13/d2e/l3a 0 2024-02-28T23:07:27.713 INFO:tasks.workunit.client.1.smithi084.stdout:5/211: write dc/de/df/d16/f29 [4475013,93154] 0 2024-02-28T23:07:27.714 INFO:tasks.workunit.client.1.smithi084.stdout:2/184: creat d5/d7/d2e/f2f x:0 0 0 2024-02-28T23:07:27.714 INFO:tasks.workunit.client.0.smithi057.stdout:7/108: symlink d0/d5/l21 0 2024-02-28T23:07:27.716 INFO:tasks.workunit.client.0.smithi057.stdout:4/78: unlink lb 0 2024-02-28T23:07:27.718 INFO:tasks.workunit.client.0.smithi057.stdout:9/72: creat dd/de/d14/f17 x:0 0 0 2024-02-28T23:07:27.721 INFO:tasks.workunit.client.1.smithi084.stdout:3/177: mkdir d3/d8/d1a/d21/d3d/d43 0 2024-02-28T23:07:27.722 INFO:tasks.workunit.client.1.smithi084.stdout:8/170: dwrite d0/f34 [0,4194304] 0 2024-02-28T23:07:27.722 INFO:tasks.workunit.client.0.smithi057.stdout:7/109: write d0/d8/d15/f1d [77636,35598] 0 2024-02-28T23:07:27.723 INFO:tasks.workunit.client.0.smithi057.stdout:9/73: stat dd/de/c13 0 2024-02-28T23:07:27.724 INFO:tasks.workunit.client.0.smithi057.stdout:4/79: write d1a/d1b/f1c [1227401,34194] 0 2024-02-28T23:07:27.730 INFO:tasks.workunit.client.1.smithi084.stdout:1/269: link d12/d1b/d3b/f44 d12/d1b/d3b/f5d 0 2024-02-28T23:07:27.731 INFO:tasks.workunit.client.0.smithi057.stdout:8/44: dwrite d4/fa [0,4194304] 0 2024-02-28T23:07:27.732 INFO:tasks.workunit.client.1.smithi084.stdout:3/178: write d3/f28 [792541,100960] 0 2024-02-28T23:07:27.732 INFO:tasks.workunit.client.0.smithi057.stdout:4/80: creat d1a/d1b/f1f x:0 0 0 2024-02-28T23:07:27.734 INFO:tasks.workunit.client.0.smithi057.stdout:9/74: creat dd/de/d14/f18 x:0 0 0 2024-02-28T23:07:27.737 INFO:tasks.workunit.client.0.smithi057.stdout:0/34: dwrite f4 [0,4194304] 0 2024-02-28T23:07:27.739 INFO:tasks.workunit.client.0.smithi057.stdout:4/81: symlink d1a/d1b/l20 0 2024-02-28T23:07:27.739 INFO:tasks.workunit.client.0.smithi057.stdout:9/75: read - dd/de/d14/f17 zero size 2024-02-28T23:07:27.741 INFO:tasks.workunit.client.1.smithi084.stdout:3/179: symlink d3/l44 0 2024-02-28T23:07:27.743 INFO:tasks.workunit.client.1.smithi084.stdout:1/270: fdatasync d12/d1b/d3b/f44 0 2024-02-28T23:07:27.745 INFO:tasks.workunit.client.1.smithi084.stdout:2/185: dwrite f3 [4194304,4194304] 0 2024-02-28T23:07:27.748 INFO:tasks.workunit.client.1.smithi084.stdout:8/171: symlink d0/d9/l3d 0 2024-02-28T23:07:27.749 INFO:tasks.workunit.client.1.smithi084.stdout:3/180: mkdir d3/d8/d1a/d45 0 2024-02-28T23:07:27.751 INFO:tasks.workunit.client.1.smithi084.stdout:5/212: dwrite dc/de/f2d [0,4194304] 0 2024-02-28T23:07:27.759 INFO:tasks.workunit.client.0.smithi057.stdout:7/110: dread d0/f12 [0,4194304] 0 2024-02-28T23:07:27.760 INFO:tasks.workunit.client.0.smithi057.stdout:4/82: link d1a/d1b/f1c d1a/d1b/f21 0 2024-02-28T23:07:27.761 INFO:tasks.workunit.client.0.smithi057.stdout:3/31: sync 2024-02-28T23:07:27.762 INFO:tasks.workunit.client.0.smithi057.stdout:1/15: sync 2024-02-28T23:07:27.765 INFO:tasks.workunit.client.1.smithi084.stdout:8/172: rename d0/fe to d0/d9/d1b/d38/f3e 0 2024-02-28T23:07:27.765 INFO:tasks.workunit.client.0.smithi057.stdout:3/32: rename d3 to d3/dc 22 2024-02-28T23:07:27.768 INFO:tasks.workunit.client.1.smithi084.stdout:5/213: mknod dc/de/df/d16/d34/c49 0 2024-02-28T23:07:27.768 INFO:tasks.workunit.client.1.smithi084.stdout:4/155: write d1/d18/d20/d15/f1a [854088,39586] 0 2024-02-28T23:07:27.769 INFO:tasks.workunit.client.1.smithi084.stdout:8/173: mkdir d0/d9/d30/d3f 0 2024-02-28T23:07:27.772 INFO:tasks.workunit.client.0.smithi057.stdout:8/45: rmdir d4 39 2024-02-28T23:07:27.774 INFO:tasks.workunit.client.0.smithi057.stdout:6/47: fsync d0/f9 0 2024-02-28T23:07:27.776 INFO:tasks.workunit.client.1.smithi084.stdout:4/156: unlink d1/d18/l2c 0 2024-02-28T23:07:27.776 INFO:tasks.workunit.client.0.smithi057.stdout:7/111: readlink d0/d8/la 0 2024-02-28T23:07:27.780 INFO:tasks.workunit.client.0.smithi057.stdout:4/83: symlink d1a/d1e/l22 0 2024-02-28T23:07:27.782 INFO:tasks.workunit.client.0.smithi057.stdout:6/48: dread - d0/d4/f7 zero size 2024-02-28T23:07:27.788 INFO:tasks.workunit.client.0.smithi057.stdout:0/35: dwrite f4 [0,4194304] 0 2024-02-28T23:07:27.788 INFO:tasks.workunit.client.1.smithi084.stdout:5/214: getdents dc/de/df/d16 0 2024-02-28T23:07:27.791 INFO:tasks.workunit.client.0.smithi057.stdout:7/112: creat d0/d8/d15/f22 x:0 0 0 2024-02-28T23:07:27.791 INFO:tasks.workunit.client.0.smithi057.stdout:0/36: readlink l7 0 2024-02-28T23:07:27.794 INFO:tasks.workunit.client.1.smithi084.stdout:1/271: dwrite d12/d1b/d25/f43 [0,4194304] 0 2024-02-28T23:07:27.795 INFO:tasks.workunit.client.0.smithi057.stdout:9/76: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:27.797 INFO:tasks.workunit.client.0.smithi057.stdout:8/46: unlink d4/fc 0 2024-02-28T23:07:27.801 INFO:tasks.workunit.client.1.smithi084.stdout:3/181: dwrite d3/d8/d14/d31/d34/f2b [0,4194304] 0 2024-02-28T23:07:27.802 INFO:tasks.workunit.client.1.smithi084.stdout:9/212: dwrite d4/d6/d15/f2a [0,4194304] 0 2024-02-28T23:07:27.805 INFO:tasks.workunit.client.1.smithi084.stdout:5/215: creat dc/de/df/d16/f4a x:0 0 0 2024-02-28T23:07:27.808 INFO:tasks.workunit.client.1.smithi084.stdout:8/174: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:27.809 INFO:tasks.workunit.client.1.smithi084.stdout:9/213: read d4/d6/d15/f2a [2361301,96174] 0 2024-02-28T23:07:27.810 INFO:tasks.workunit.client.0.smithi057.stdout:0/37: creat f8 x:0 0 0 2024-02-28T23:07:27.813 INFO:tasks.workunit.client.1.smithi084.stdout:4/157: dread d1/f9 [0,4194304] 0 2024-02-28T23:07:27.818 INFO:tasks.workunit.client.0.smithi057.stdout:8/47: mknod d4/cd 0 2024-02-28T23:07:27.819 INFO:tasks.workunit.client.0.smithi057.stdout:9/77: symlink dd/l19 0 2024-02-28T23:07:27.820 INFO:tasks.workunit.client.0.smithi057.stdout:1/16: dread f0 [0,4194304] 0 2024-02-28T23:07:27.823 INFO:tasks.workunit.client.1.smithi084.stdout:1/272: rename d12/d1b/d25/f3c to d12/d1b/d2e/d32/f5e 0 2024-02-28T23:07:27.824 INFO:tasks.workunit.client.0.smithi057.stdout:1/17: chown f0 207 1 2024-02-28T23:07:27.824 INFO:tasks.workunit.client.0.smithi057.stdout:4/84: link f17 d1a/d1b/f23 0 2024-02-28T23:07:27.827 INFO:tasks.workunit.client.0.smithi057.stdout:1/18: truncate f0 4225184 0 2024-02-28T23:07:27.829 INFO:tasks.workunit.client.0.smithi057.stdout:6/49: dwrite d0/f9 [0,4194304] 0 2024-02-28T23:07:27.831 INFO:tasks.workunit.client.0.smithi057.stdout:0/38: creat f9 x:0 0 0 2024-02-28T23:07:27.831 INFO:tasks.workunit.client.1.smithi084.stdout:7/170: sync 2024-02-28T23:07:27.831 INFO:tasks.workunit.client.1.smithi084.stdout:0/149: sync 2024-02-28T23:07:27.832 INFO:tasks.workunit.client.1.smithi084.stdout:7/171: stat db/d11/f2b 0 2024-02-28T23:07:27.833 INFO:tasks.workunit.client.1.smithi084.stdout:5/216: symlink dc/de/df/d31/l4b 0 2024-02-28T23:07:27.834 INFO:tasks.workunit.client.0.smithi057.stdout:0/39: write f4 [4857244,87418] 0 2024-02-28T23:07:27.834 INFO:tasks.workunit.client.1.smithi084.stdout:4/158: mkdir d1/d18/d20/d24/d2f 0 2024-02-28T23:07:27.835 INFO:tasks.workunit.client.0.smithi057.stdout:0/40: dread - f8 zero size 2024-02-28T23:07:27.836 INFO:tasks.workunit.client.0.smithi057.stdout:0/41: chown f4 15657 1 2024-02-28T23:07:27.838 INFO:tasks.workunit.client.0.smithi057.stdout:7/113: link d0/d8/d15/f1d d0/f23 0 2024-02-28T23:07:27.839 INFO:tasks.workunit.client.1.smithi084.stdout:8/175: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:27.840 INFO:tasks.workunit.client.0.smithi057.stdout:7/114: chown d0/d8/f19 14465021 1 2024-02-28T23:07:27.841 INFO:tasks.workunit.client.0.smithi057.stdout:2/122: getdents d8/d11/d21/d28 0 2024-02-28T23:07:27.841 INFO:tasks.workunit.client.1.smithi084.stdout:9/214: creat d4/d6/d15/d3c/f47 x:0 0 0 2024-02-28T23:07:27.844 INFO:tasks.workunit.client.1.smithi084.stdout:0/150: symlink d1/d6/db/l2b 0 2024-02-28T23:07:27.845 INFO:tasks.workunit.client.1.smithi084.stdout:7/172: mkdir db/d11/d27/d36 0 2024-02-28T23:07:27.846 INFO:tasks.workunit.client.1.smithi084.stdout:9/215: creat d4/d6/d15/d12/f48 x:0 0 0 2024-02-28T23:07:27.849 INFO:tasks.workunit.client.0.smithi057.stdout:0/42: symlink la 0 2024-02-28T23:07:27.850 INFO:tasks.workunit.client.0.smithi057.stdout:1/19: mkdir d2/d3 0 2024-02-28T23:07:27.851 INFO:tasks.workunit.client.0.smithi057.stdout:8/48: link l0 d4/le 0 2024-02-28T23:07:27.851 INFO:tasks.workunit.client.1.smithi084.stdout:9/216: symlink d4/l49 0 2024-02-28T23:07:27.851 INFO:tasks.workunit.client.1.smithi084.stdout:8/176: symlink d0/d9/d30/d33/l40 0 2024-02-28T23:07:27.853 INFO:tasks.workunit.client.1.smithi084.stdout:7/173: rename db/le to db/d10/d17/l37 0 2024-02-28T23:07:27.855 INFO:tasks.workunit.client.1.smithi084.stdout:9/217: write d4/d32/d3f/f42 [583712,42587] 0 2024-02-28T23:07:27.855 INFO:tasks.workunit.client.0.smithi057.stdout:0/43: rename c3 to cb 0 2024-02-28T23:07:27.856 INFO:tasks.workunit.client.1.smithi084.stdout:0/151: symlink d1/d6/db/l2c 0 2024-02-28T23:07:27.858 INFO:tasks.workunit.client.0.smithi057.stdout:6/50: getdents d0/d4 0 2024-02-28T23:07:27.860 INFO:tasks.workunit.client.1.smithi084.stdout:3/182: dwrite d3/d8/d1a/d21/d3d/f3c [0,4194304] 0 2024-02-28T23:07:27.861 INFO:tasks.workunit.client.0.smithi057.stdout:0/44: mknod cc 0 2024-02-28T23:07:27.864 INFO:tasks.workunit.client.1.smithi084.stdout:3/183: dread - d3/d8/df/f12 zero size 2024-02-28T23:07:27.867 INFO:tasks.workunit.client.0.smithi057.stdout:5/22: sync 2024-02-28T23:07:27.867 INFO:tasks.workunit.client.0.smithi057.stdout:5/23: chown d4 22496169 1 2024-02-28T23:07:27.869 INFO:tasks.workunit.client.1.smithi084.stdout:9/218: link l1 d4/d32/d3f/l4a 0 2024-02-28T23:07:27.869 INFO:tasks.workunit.client.1.smithi084.stdout:5/217: write f7 [1870783,70823] 0 2024-02-28T23:07:27.874 INFO:tasks.workunit.client.0.smithi057.stdout:5/24: readlink l3 0 2024-02-28T23:07:27.893 INFO:tasks.workunit.client.1.smithi084.stdout:1/273: dwrite d12/d1b/d3b/f59 [0,4194304] 0 2024-02-28T23:07:27.895 INFO:tasks.workunit.client.1.smithi084.stdout:5/218: write dc/de/f12 [963057,4178] 0 2024-02-28T23:07:27.896 INFO:tasks.workunit.client.1.smithi084.stdout:0/152: dwrite d1/d8/f1d [0,4194304] 0 2024-02-28T23:07:27.898 INFO:tasks.workunit.client.1.smithi084.stdout:7/174: dwrite db/d11/f2b [0,4194304] 0 2024-02-28T23:07:27.899 INFO:tasks.workunit.client.0.smithi057.stdout:4/85: dwrite fe [0,4194304] 0 2024-02-28T23:07:27.901 INFO:tasks.workunit.client.1.smithi084.stdout:8/177: dwrite d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:27.910 INFO:tasks.workunit.client.0.smithi057.stdout:7/115: dwrite d0/d8/d15/f1d [0,4194304] 0 2024-02-28T23:07:27.915 INFO:tasks.workunit.client.1.smithi084.stdout:1/274: creat d12/d1b/d2e/d30/f5f x:0 0 0 2024-02-28T23:07:27.920 INFO:tasks.workunit.client.1.smithi084.stdout:8/178: fdatasync d0/d9/d1b/d38/f3a 0 2024-02-28T23:07:27.922 INFO:tasks.workunit.client.1.smithi084.stdout:9/219: dwrite d4/d6/d15/d3c/f47 [0,4194304] 0 2024-02-28T23:07:27.924 INFO:tasks.workunit.client.0.smithi057.stdout:8/49: dwrite d4/f7 [4194304,4194304] 0 2024-02-28T23:07:27.926 INFO:tasks.workunit.client.1.smithi084.stdout:1/275: creat d12/d1b/d2e/d52/f60 x:0 0 0 2024-02-28T23:07:27.927 INFO:tasks.workunit.client.1.smithi084.stdout:0/153: getdents d1/d6/d1b 0 2024-02-28T23:07:27.930 INFO:tasks.workunit.client.1.smithi084.stdout:1/276: chown d12/d1b/d2e/d30/f58 2029534 1 2024-02-28T23:07:27.931 INFO:tasks.workunit.client.0.smithi057.stdout:6/51: dwrite d0/d4/f7 [0,4194304] 0 2024-02-28T23:07:27.934 INFO:tasks.workunit.client.0.smithi057.stdout:7/116: mknod d0/d8/d15/c24 0 2024-02-28T23:07:27.936 INFO:tasks.workunit.client.1.smithi084.stdout:9/220: mknod d4/d6/d15/d3c/c4b 0 2024-02-28T23:07:27.936 INFO:tasks.workunit.client.1.smithi084.stdout:1/277: mkdir d12/d1b/d2e/d52/d61 0 2024-02-28T23:07:27.939 INFO:tasks.workunit.client.0.smithi057.stdout:0/45: dwrite f5 [0,4194304] 0 2024-02-28T23:07:27.939 INFO:tasks.workunit.client.1.smithi084.stdout:0/154: mkdir d1/d6/d2d 0 2024-02-28T23:07:27.940 INFO:tasks.workunit.client.0.smithi057.stdout:8/50: creat d4/ff x:0 0 0 2024-02-28T23:07:27.946 INFO:tasks.workunit.client.1.smithi084.stdout:9/221: mkdir d4/d6/d15/d4c 0 2024-02-28T23:07:27.950 INFO:tasks.workunit.client.0.smithi057.stdout:4/86: dwrite f12 [0,4194304] 0 2024-02-28T23:07:27.952 INFO:tasks.workunit.client.1.smithi084.stdout:0/155: mknod d1/d6/db/c2e 0 2024-02-28T23:07:27.959 INFO:tasks.workunit.client.1.smithi084.stdout:7/175: dwrite db/d11/f12 [0,4194304] 0 2024-02-28T23:07:27.960 INFO:tasks.workunit.client.1.smithi084.stdout:0/156: chown d1/l15 14783898 1 2024-02-28T23:07:27.962 INFO:tasks.workunit.client.1.smithi084.stdout:5/219: dwrite dc/de/df/d31/f41 [0,4194304] 0 2024-02-28T23:07:27.965 INFO:tasks.workunit.client.1.smithi084.stdout:7/176: fsync db/d10/f23 0 2024-02-28T23:07:27.966 INFO:tasks.workunit.client.1.smithi084.stdout:9/222: getdents d4 0 2024-02-28T23:07:27.969 INFO:tasks.workunit.client.0.smithi057.stdout:8/51: dread d4/f7 [0,4194304] 0 2024-02-28T23:07:27.971 INFO:tasks.workunit.client.1.smithi084.stdout:0/157: symlink d1/d8/d18/l2f 0 2024-02-28T23:07:27.975 INFO:tasks.workunit.client.1.smithi084.stdout:8/179: dwrite d0/d9/d1b/f39 [0,4194304] 0 2024-02-28T23:07:27.976 INFO:tasks.workunit.client.1.smithi084.stdout:9/223: rmdir d4/d6/d15 39 2024-02-28T23:07:27.976 INFO:tasks.workunit.client.0.smithi057.stdout:6/52: dwrite d0/f9 [0,4194304] 0 2024-02-28T23:07:27.977 INFO:tasks.workunit.client.0.smithi057.stdout:4/87: fdatasync f17 0 2024-02-28T23:07:27.977 INFO:tasks.workunit.client.1.smithi084.stdout:5/220: mkdir dc/de/df/d31/d4c 0 2024-02-28T23:07:27.984 INFO:tasks.workunit.client.0.smithi057.stdout:0/46: dwrite f5 [0,4194304] 0 2024-02-28T23:07:27.984 INFO:tasks.workunit.client.0.smithi057.stdout:6/53: chown d0/d4 28 1 2024-02-28T23:07:27.984 INFO:tasks.workunit.client.0.smithi057.stdout:4/88: write f0 [582316,30650] 0 2024-02-28T23:07:27.987 INFO:tasks.workunit.client.1.smithi084.stdout:1/278: rmdir d12/d1b/d2e/d30 39 2024-02-28T23:07:27.988 INFO:tasks.workunit.client.0.smithi057.stdout:6/54: creat d0/d4/fa x:0 0 0 2024-02-28T23:07:27.990 INFO:tasks.workunit.client.1.smithi084.stdout:1/279: stat d12/d1b/d3b/l49 0 2024-02-28T23:07:27.992 INFO:tasks.workunit.client.0.smithi057.stdout:6/55: creat d0/d4/fb x:0 0 0 2024-02-28T23:07:27.996 INFO:tasks.workunit.client.1.smithi084.stdout:5/221: creat dc/de/df/d16/d34/f4d x:0 0 0 2024-02-28T23:07:27.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/224: write d4/d6/d15/f19 [4474583,40947] 0 2024-02-28T23:07:28.000 INFO:tasks.workunit.client.1.smithi084.stdout:5/222: read - dc/f27 zero size 2024-02-28T23:07:28.004 INFO:tasks.workunit.client.0.smithi057.stdout:8/52: getdents d4 0 2024-02-28T23:07:28.005 INFO:tasks.workunit.client.0.smithi057.stdout:7/117: truncate d0/f12 1582312 0 2024-02-28T23:07:28.008 INFO:tasks.workunit.client.1.smithi084.stdout:5/223: creat dc/de/df/d31/f4e x:0 0 0 2024-02-28T23:07:28.008 INFO:tasks.workunit.client.1.smithi084.stdout:0/158: dwrite d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:28.010 INFO:tasks.workunit.client.0.smithi057.stdout:8/53: rename c3 to d4/c10 0 2024-02-28T23:07:28.011 INFO:tasks.workunit.client.1.smithi084.stdout:0/159: fsync d1/d6/f16 0 2024-02-28T23:07:28.011 INFO:tasks.workunit.client.0.smithi057.stdout:4/89: dwrite f12 [0,4194304] 0 2024-02-28T23:07:28.012 INFO:tasks.workunit.client.1.smithi084.stdout:8/180: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:28.015 INFO:tasks.workunit.client.1.smithi084.stdout:0/160: chown d1/d6/f13 54 1 2024-02-28T23:07:28.018 INFO:tasks.workunit.client.0.smithi057.stdout:8/54: rename d4/ff to d4/f11 0 2024-02-28T23:07:28.019 INFO:tasks.workunit.client.1.smithi084.stdout:9/225: rename d4/d32/l20 to d4/d6/d15/d4c/l4d 0 2024-02-28T23:07:28.020 INFO:tasks.workunit.client.0.smithi057.stdout:4/90: fdatasync f18 0 2024-02-28T23:07:28.021 INFO:tasks.workunit.client.0.smithi057.stdout:8/55: symlink d4/l12 0 2024-02-28T23:07:28.024 INFO:tasks.workunit.client.1.smithi084.stdout:9/226: fdatasync d4/f3b 0 2024-02-28T23:07:28.024 INFO:tasks.workunit.client.1.smithi084.stdout:5/224: symlink dc/de/df/l4f 0 2024-02-28T23:07:28.025 INFO:tasks.workunit.client.0.smithi057.stdout:8/56: chown d4/le 920307 1 2024-02-28T23:07:28.026 INFO:tasks.workunit.client.1.smithi084.stdout:9/227: stat d4/d32 0 2024-02-28T23:07:28.029 INFO:tasks.workunit.client.0.smithi057.stdout:4/91: mkdir d1a/d1b/d24 0 2024-02-28T23:07:28.029 INFO:tasks.workunit.client.0.smithi057.stdout:6/56: rmdir d0/d4 39 2024-02-28T23:07:28.029 INFO:tasks.workunit.client.1.smithi084.stdout:1/280: getdents d12/d1b/d2e/d30 0 2024-02-28T23:07:28.030 INFO:tasks.workunit.client.0.smithi057.stdout:4/92: chown f14 3 1 2024-02-28T23:07:28.031 INFO:tasks.workunit.client.0.smithi057.stdout:8/57: mkdir d4/d13 0 2024-02-28T23:07:28.032 INFO:tasks.workunit.client.0.smithi057.stdout:8/58: dread - d4/f5 zero size 2024-02-28T23:07:28.035 INFO:tasks.workunit.client.0.smithi057.stdout:6/57: creat d0/fc x:0 0 0 2024-02-28T23:07:28.035 INFO:tasks.workunit.client.0.smithi057.stdout:6/58: chown d0 12563 1 2024-02-28T23:07:28.036 INFO:tasks.workunit.client.1.smithi084.stdout:5/225: write dc/de/df/d16/f1f [315653,90552] 0 2024-02-28T23:07:28.036 INFO:tasks.workunit.client.1.smithi084.stdout:1/281: write d12/d1b/f31 [3304067,88909] 0 2024-02-28T23:07:28.040 INFO:tasks.workunit.client.0.smithi057.stdout:8/59: rename d4/fa to d4/f14 0 2024-02-28T23:07:28.041 INFO:tasks.workunit.client.0.smithi057.stdout:8/60: chown d4 50006 1 2024-02-28T23:07:28.047 INFO:tasks.workunit.client.1.smithi084.stdout:1/282: rename d12/d1b/d2e/d30/f5f to d12/d1b/d2e/d52/d5b/f62 0 2024-02-28T23:07:28.049 INFO:tasks.workunit.client.1.smithi084.stdout:1/283: stat d12/d1b/d2e/d32/c37 0 2024-02-28T23:07:28.057 INFO:tasks.workunit.client.0.smithi057.stdout:0/47: fdatasync f5 0 2024-02-28T23:07:28.058 INFO:tasks.workunit.client.0.smithi057.stdout:3/33: sync 2024-02-28T23:07:28.060 INFO:tasks.workunit.client.1.smithi084.stdout:5/226: dread dc/de/df/d16/f1f [0,4194304] 0 2024-02-28T23:07:28.061 INFO:tasks.workunit.client.1.smithi084.stdout:9/228: dwrite d4/d6/d15/f2c [0,4194304] 0 2024-02-28T23:07:28.061 INFO:tasks.workunit.client.0.smithi057.stdout:4/93: dwrite d1a/d1b/f1f [0,4194304] 0 2024-02-28T23:07:28.062 INFO:tasks.workunit.client.0.smithi057.stdout:3/34: write f1 [813734,66290] 0 2024-02-28T23:07:28.065 INFO:tasks.workunit.client.1.smithi084.stdout:5/227: write dc/de/f1a [593886,111159] 0 2024-02-28T23:07:28.067 INFO:tasks.workunit.client.1.smithi084.stdout:9/229: truncate d4/d6/d15/f33 81739 0 2024-02-28T23:07:28.067 INFO:tasks.workunit.client.0.smithi057.stdout:4/94: readlink d1a/d1e/l22 0 2024-02-28T23:07:28.071 INFO:tasks.workunit.client.1.smithi084.stdout:9/230: fdatasync d4/fc 0 2024-02-28T23:07:28.073 INFO:tasks.workunit.client.1.smithi084.stdout:5/228: mkdir dc/de/df/d50 0 2024-02-28T23:07:28.073 INFO:tasks.workunit.client.1.smithi084.stdout:1/284: dread d12/d1b/d3b/f42 [0,4194304] 0 2024-02-28T23:07:28.073 INFO:tasks.workunit.client.0.smithi057.stdout:7/118: dwrite d0/f10 [0,4194304] 0 2024-02-28T23:07:28.074 INFO:tasks.workunit.client.1.smithi084.stdout:7/177: dread f9 [0,4194304] 0 2024-02-28T23:07:28.076 INFO:tasks.workunit.client.1.smithi084.stdout:9/231: write d4/d32/f31 [3604409,124073] 0 2024-02-28T23:07:28.077 INFO:tasks.workunit.client.0.smithi057.stdout:7/119: fsync d0/d8/d15/f22 0 2024-02-28T23:07:28.081 INFO:tasks.workunit.client.1.smithi084.stdout:7/178: symlink db/d10/d17/l38 0 2024-02-28T23:07:28.081 INFO:tasks.workunit.client.0.smithi057.stdout:0/48: dwrite f4 [4194304,4194304] 0 2024-02-28T23:07:28.083 INFO:tasks.workunit.client.1.smithi084.stdout:9/232: fsync d4/d6/ff 0 2024-02-28T23:07:28.086 INFO:tasks.workunit.client.1.smithi084.stdout:7/179: read - db/d11/d27/f35 zero size 2024-02-28T23:07:28.087 INFO:tasks.workunit.client.1.smithi084.stdout:1/285: rename d12/d1b/d25 to d12/d1b/d63 0 2024-02-28T23:07:28.090 INFO:tasks.workunit.client.1.smithi084.stdout:9/233: rmdir d4/d6/d15/d12/d41 39 2024-02-28T23:07:28.091 INFO:tasks.workunit.client.1.smithi084.stdout:0/161: dread d1/d8/f20 [0,4194304] 0 2024-02-28T23:07:28.097 INFO:tasks.workunit.client.1.smithi084.stdout:7/180: symlink db/d10/d17/l39 0 2024-02-28T23:07:28.099 INFO:tasks.workunit.client.1.smithi084.stdout:9/234: creat d4/f4e x:0 0 0 2024-02-28T23:07:28.100 INFO:tasks.workunit.client.1.smithi084.stdout:1/286: rename d12/l55 to d12/d1b/d3b/l64 0 2024-02-28T23:07:28.102 INFO:tasks.workunit.client.1.smithi084.stdout:1/287: write d12/f45 [324241,99888] 0 2024-02-28T23:07:28.110 INFO:tasks.workunit.client.1.smithi084.stdout:5/229: dwrite dc/de/f1a [0,4194304] 0 2024-02-28T23:07:28.112 INFO:tasks.workunit.client.0.smithi057.stdout:0/49: fdatasync f4 0 2024-02-28T23:07:28.119 INFO:tasks.workunit.client.1.smithi084.stdout:7/181: rmdir db/d10 39 2024-02-28T23:07:28.126 INFO:tasks.workunit.client.1.smithi084.stdout:7/182: rename db/d11/f2b to db/d11/d27/f3a 0 2024-02-28T23:07:28.134 INFO:tasks.workunit.client.1.smithi084.stdout:0/162: dwrite d1/d6/d1b/f22 [0,4194304] 0 2024-02-28T23:07:28.137 INFO:tasks.workunit.client.1.smithi084.stdout:1/288: dwrite d12/f16 [0,4194304] 0 2024-02-28T23:07:28.139 INFO:tasks.workunit.client.1.smithi084.stdout:5/230: dwrite dc/de/f12 [0,4194304] 0 2024-02-28T23:07:28.140 INFO:tasks.workunit.client.0.smithi057.stdout:6/59: creat d0/fd x:0 0 0 2024-02-28T23:07:28.140 INFO:tasks.workunit.client.0.smithi057.stdout:8/61: mknod d4/d13/c15 0 2024-02-28T23:07:28.141 INFO:tasks.workunit.client.1.smithi084.stdout:0/163: creat d1/d6/d2d/f30 x:0 0 0 2024-02-28T23:07:28.141 INFO:tasks.workunit.client.1.smithi084.stdout:1/289: rmdir d12/d1b/d63 39 2024-02-28T23:07:28.143 INFO:tasks.workunit.client.1.smithi084.stdout:5/231: mkdir dc/d51 0 2024-02-28T23:07:28.145 INFO:tasks.workunit.client.0.smithi057.stdout:3/35: creat d3/fd x:0 0 0 2024-02-28T23:07:28.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/60: write d0/f9 [155670,94424] 0 2024-02-28T23:07:28.146 INFO:tasks.workunit.client.0.smithi057.stdout:4/95: creat d1a/d1e/f25 x:0 0 0 2024-02-28T23:07:28.147 INFO:tasks.workunit.client.1.smithi084.stdout:2/186: sync 2024-02-28T23:07:28.147 INFO:tasks.workunit.client.0.smithi057.stdout:0/50: mknod cd 0 2024-02-28T23:07:28.149 INFO:tasks.workunit.client.1.smithi084.stdout:1/290: symlink d12/d1b/d2e/d32/l65 0 2024-02-28T23:07:28.149 INFO:tasks.workunit.client.1.smithi084.stdout:4/159: sync 2024-02-28T23:07:28.149 INFO:tasks.workunit.client.1.smithi084.stdout:6/212: sync 2024-02-28T23:07:28.152 INFO:tasks.workunit.client.0.smithi057.stdout:4/96: mknod d1a/d1b/c26 0 2024-02-28T23:07:28.154 INFO:tasks.workunit.client.0.smithi057.stdout:8/62: chown d4/c10 2824884 1 2024-02-28T23:07:28.154 INFO:tasks.workunit.client.1.smithi084.stdout:7/183: dread db/d11/f16 [0,4194304] 0 2024-02-28T23:07:28.155 INFO:tasks.workunit.client.0.smithi057.stdout:7/120: rmdir d0/d20 0 2024-02-28T23:07:28.158 INFO:tasks.workunit.client.0.smithi057.stdout:7/121: chown d0/f23 771253 1 2024-02-28T23:07:28.162 INFO:tasks.workunit.client.1.smithi084.stdout:4/160: mkdir d1/d18/d20/d14/d30 0 2024-02-28T23:07:28.163 INFO:tasks.workunit.client.0.smithi057.stdout:6/61: rename d0/f9 to d0/d4/fe 0 2024-02-28T23:07:28.163 INFO:tasks.workunit.client.0.smithi057.stdout:8/63: symlink d4/l16 0 2024-02-28T23:07:28.164 INFO:tasks.workunit.client.1.smithi084.stdout:4/161: stat d1/d18/d20/l1f 0 2024-02-28T23:07:28.165 INFO:tasks.workunit.client.1.smithi084.stdout:1/291: creat d12/d1b/d63/f66 x:0 0 0 2024-02-28T23:07:28.166 INFO:tasks.workunit.client.0.smithi057.stdout:6/62: chown d0/fc 0 1 2024-02-28T23:07:28.166 INFO:tasks.workunit.client.1.smithi084.stdout:7/184: chown db/d10/d17/l39 15 1 2024-02-28T23:07:28.168 INFO:tasks.workunit.client.1.smithi084.stdout:4/162: chown d1/d18/d20/c23 0 1 2024-02-28T23:07:28.168 INFO:tasks.workunit.client.1.smithi084.stdout:0/164: dwrite d1/d8/f1c [0,4194304] 0 2024-02-28T23:07:28.169 INFO:tasks.workunit.client.0.smithi057.stdout:6/63: dread - d0/fc zero size 2024-02-28T23:07:28.169 INFO:tasks.workunit.client.1.smithi084.stdout:6/213: rename d1/d13/d2e/d41/d2c/c3f to d1/c42 0 2024-02-28T23:07:28.170 INFO:tasks.workunit.client.1.smithi084.stdout:1/292: creat d12/d1b/d2e/d52/f67 x:0 0 0 2024-02-28T23:07:28.172 INFO:tasks.workunit.client.1.smithi084.stdout:4/163: fdatasync d1/f4 0 2024-02-28T23:07:28.174 INFO:tasks.workunit.client.0.smithi057.stdout:6/64: creat d0/d4/ff x:0 0 0 2024-02-28T23:07:28.174 INFO:tasks.workunit.client.0.smithi057.stdout:0/51: dwrite f5 [4194304,4194304] 0 2024-02-28T23:07:28.176 INFO:tasks.workunit.client.0.smithi057.stdout:4/97: dread d1a/d1b/f1f [0,4194304] 0 2024-02-28T23:07:28.179 INFO:tasks.workunit.client.0.smithi057.stdout:0/52: creat fe x:0 0 0 2024-02-28T23:07:28.180 INFO:tasks.workunit.client.0.smithi057.stdout:6/65: unlink d0/d4/c8 0 2024-02-28T23:07:28.182 INFO:tasks.workunit.client.0.smithi057.stdout:6/66: chown d0/fd 3450 1 2024-02-28T23:07:28.182 INFO:tasks.workunit.client.0.smithi057.stdout:0/53: mknod cf 0 2024-02-28T23:07:28.184 INFO:tasks.workunit.client.0.smithi057.stdout:4/98: rename d1a/d1b/l20 to d1a/l27 0 2024-02-28T23:07:28.187 INFO:tasks.workunit.client.0.smithi057.stdout:0/54: read f5 [7333916,9778] 0 2024-02-28T23:07:28.188 INFO:tasks.workunit.client.0.smithi057.stdout:4/99: truncate f0 4549934 0 2024-02-28T23:07:28.188 INFO:tasks.workunit.client.0.smithi057.stdout:0/55: truncate f9 437658 0 2024-02-28T23:07:28.188 INFO:tasks.workunit.client.0.smithi057.stdout:0/56: dread - fe zero size 2024-02-28T23:07:28.190 INFO:tasks.workunit.client.0.smithi057.stdout:6/67: symlink d0/l10 0 2024-02-28T23:07:28.191 INFO:tasks.workunit.client.1.smithi084.stdout:6/214: symlink d1/d9/d1d/d1e/l43 0 2024-02-28T23:07:28.195 INFO:tasks.workunit.client.1.smithi084.stdout:9/235: dread d4/f22 [0,4194304] 0 2024-02-28T23:07:28.195 INFO:tasks.workunit.client.0.smithi057.stdout:6/68: mkdir d0/d4/d11 0 2024-02-28T23:07:28.196 INFO:tasks.workunit.client.1.smithi084.stdout:7/185: getdents db/d11 0 2024-02-28T23:07:28.197 INFO:tasks.workunit.client.0.smithi057.stdout:4/100: link l7 d1a/d1b/d24/l28 0 2024-02-28T23:07:28.198 INFO:tasks.workunit.client.1.smithi084.stdout:7/186: stat db/d11/f29 0 2024-02-28T23:07:28.200 INFO:tasks.workunit.client.1.smithi084.stdout:4/164: creat d1/d18/d20/d14/d30/f31 x:0 0 0 2024-02-28T23:07:28.203 INFO:tasks.workunit.client.1.smithi084.stdout:9/236: chown d4/d6/d15/f2a 226 1 2024-02-28T23:07:28.204 INFO:tasks.workunit.client.0.smithi057.stdout:0/57: dread f5 [0,4194304] 0 2024-02-28T23:07:28.204 INFO:tasks.workunit.client.0.smithi057.stdout:6/69: rename d0/d4/f7 to d0/d4/d11/f12 0 2024-02-28T23:07:28.204 INFO:tasks.workunit.client.1.smithi084.stdout:4/165: dread - d1/d18/d20/d14/f25 zero size 2024-02-28T23:07:28.205 INFO:tasks.workunit.client.1.smithi084.stdout:7/187: mkdir db/d10/d17/d3b 0 2024-02-28T23:07:28.207 INFO:tasks.workunit.client.1.smithi084.stdout:1/293: getdents d12/d1b/d2e/d52 0 2024-02-28T23:07:28.209 INFO:tasks.workunit.client.0.smithi057.stdout:0/58: creat f10 x:0 0 0 2024-02-28T23:07:28.210 INFO:tasks.workunit.client.0.smithi057.stdout:6/70: creat d0/d4/d11/f13 x:0 0 0 2024-02-28T23:07:28.211 INFO:tasks.workunit.client.1.smithi084.stdout:0/165: truncate d1/d6/f14 375561 0 2024-02-28T23:07:28.211 INFO:tasks.workunit.client.1.smithi084.stdout:4/166: write d1/d18/d20/d15/f16 [2899920,109042] 0 2024-02-28T23:07:28.212 INFO:tasks.workunit.client.0.smithi057.stdout:0/59: truncate f9 1012755 0 2024-02-28T23:07:28.213 INFO:tasks.workunit.client.1.smithi084.stdout:9/237: write d4/d32/d3f/f42 [496657,36124] 0 2024-02-28T23:07:28.214 INFO:tasks.workunit.client.0.smithi057.stdout:6/71: creat d0/d4/d11/f14 x:0 0 0 2024-02-28T23:07:28.216 INFO:tasks.workunit.client.0.smithi057.stdout:6/72: write d0/d4/fb [63064,11125] 0 2024-02-28T23:07:28.218 INFO:tasks.workunit.client.1.smithi084.stdout:7/188: getdents db/d11 0 2024-02-28T23:07:28.219 INFO:tasks.workunit.client.1.smithi084.stdout:0/166: creat d1/d8/d18/f31 x:0 0 0 2024-02-28T23:07:28.220 INFO:tasks.workunit.client.0.smithi057.stdout:6/73: symlink d0/d4/l15 0 2024-02-28T23:07:28.222 INFO:tasks.workunit.client.0.smithi057.stdout:6/74: fdatasync d0/d4/fe 0 2024-02-28T23:07:28.222 INFO:tasks.workunit.client.1.smithi084.stdout:4/167: chown d1/f7 138216615 1 2024-02-28T23:07:28.222 INFO:tasks.workunit.client.1.smithi084.stdout:6/215: dwrite d1/d13/d2e/d41/f2f [0,4194304] 0 2024-02-28T23:07:28.224 INFO:tasks.workunit.client.1.smithi084.stdout:4/168: readlink d1/d18/d20/l19 0 2024-02-28T23:07:28.225 INFO:tasks.workunit.client.0.smithi057.stdout:6/75: mknod d0/c16 0 2024-02-28T23:07:28.226 INFO:tasks.workunit.client.1.smithi084.stdout:7/189: creat db/d10/d17/d3b/f3c x:0 0 0 2024-02-28T23:07:28.227 INFO:tasks.workunit.client.1.smithi084.stdout:0/167: symlink d1/d6/d2d/l32 0 2024-02-28T23:07:28.228 INFO:tasks.workunit.client.1.smithi084.stdout:1/294: dread d12/f16 [0,4194304] 0 2024-02-28T23:07:28.230 INFO:tasks.workunit.client.1.smithi084.stdout:7/190: write db/d10/d17/d3b/f3c [123173,106170] 0 2024-02-28T23:07:28.230 INFO:tasks.workunit.client.1.smithi084.stdout:6/216: creat d1/d13/d2e/f44 x:0 0 0 2024-02-28T23:07:28.230 INFO:tasks.workunit.client.1.smithi084.stdout:9/238: link d4/d6/d15/d12/l2b d4/d32/d1f/l4f 0 2024-02-28T23:07:28.231 INFO:tasks.workunit.client.0.smithi057.stdout:6/76: mkdir d0/d17 0 2024-02-28T23:07:28.231 INFO:tasks.workunit.client.1.smithi084.stdout:0/168: mknod d1/d6/d2d/c33 0 2024-02-28T23:07:28.233 INFO:tasks.workunit.client.1.smithi084.stdout:4/169: creat d1/d18/f32 x:0 0 0 2024-02-28T23:07:28.235 INFO:tasks.workunit.client.0.smithi057.stdout:6/77: mknod d0/d17/c18 0 2024-02-28T23:07:28.238 INFO:tasks.workunit.client.0.smithi057.stdout:6/78: creat d0/d4/f19 x:0 0 0 2024-02-28T23:07:28.239 INFO:tasks.workunit.client.1.smithi084.stdout:4/170: unlink d1/d18/d20/d14/f25 0 2024-02-28T23:07:28.242 INFO:tasks.workunit.client.0.smithi057.stdout:6/79: write d0/d4/fe [372749,114563] 0 2024-02-28T23:07:28.244 INFO:tasks.workunit.client.1.smithi084.stdout:9/239: symlink d4/d6/d15/d12/d41/l50 0 2024-02-28T23:07:28.247 INFO:tasks.workunit.client.1.smithi084.stdout:6/217: link d1/d9/d1d/d1e/l43 d1/d13/d2e/d3e/l45 0 2024-02-28T23:07:28.250 INFO:tasks.workunit.client.1.smithi084.stdout:4/171: truncate d1/d18/d20/fe 158757 0 2024-02-28T23:07:28.250 INFO:tasks.workunit.client.1.smithi084.stdout:6/218: write d1/f1b [2968557,13697] 0 2024-02-28T23:07:28.252 INFO:tasks.workunit.client.1.smithi084.stdout:0/169: dwrite d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:28.253 INFO:tasks.workunit.client.1.smithi084.stdout:7/191: write db/fd [4233037,11014] 0 2024-02-28T23:07:28.254 INFO:tasks.workunit.client.1.smithi084.stdout:9/240: rename d4/f22 to d4/f51 0 2024-02-28T23:07:28.256 INFO:tasks.workunit.client.1.smithi084.stdout:7/192: chown db/d10/d18/f1b 12196431 1 2024-02-28T23:07:28.259 INFO:tasks.workunit.client.1.smithi084.stdout:4/172: fsync d1/f9 0 2024-02-28T23:07:28.260 INFO:tasks.workunit.client.1.smithi084.stdout:0/170: mknod d1/d8/d18/d28/c34 0 2024-02-28T23:07:28.263 INFO:tasks.workunit.client.1.smithi084.stdout:7/193: link db/d11/d27/f35 db/d10/d17/f3d 0 2024-02-28T23:07:28.264 INFO:tasks.workunit.client.1.smithi084.stdout:7/194: dread - db/d10/f2f zero size 2024-02-28T23:07:28.266 INFO:tasks.workunit.client.0.smithi057.stdout:7/122: truncate d0/f12 2505726 0 2024-02-28T23:07:28.267 INFO:tasks.workunit.client.0.smithi057.stdout:8/64: write d4/f9 [401892,1659] 0 2024-02-28T23:07:28.272 INFO:tasks.workunit.client.0.smithi057.stdout:7/123: unlink d0/d8/d15/f1d 0 2024-02-28T23:07:28.273 INFO:tasks.workunit.client.0.smithi057.stdout:8/65: write d4/f7 [7531895,50453] 0 2024-02-28T23:07:28.276 INFO:tasks.workunit.client.0.smithi057.stdout:7/124: chown d0/f12 472 1 2024-02-28T23:07:28.278 INFO:tasks.workunit.client.0.smithi057.stdout:8/66: rename d4/f8 to d4/f17 0 2024-02-28T23:07:28.279 INFO:tasks.workunit.client.1.smithi084.stdout:6/219: truncate d1/d9/f3b 1526660 0 2024-02-28T23:07:28.279 INFO:tasks.workunit.client.1.smithi084.stdout:4/173: dwrite d1/f12 [0,4194304] 0 2024-02-28T23:07:28.280 INFO:tasks.workunit.client.1.smithi084.stdout:0/171: rmdir d1/d8 39 2024-02-28T23:07:28.280 INFO:tasks.workunit.client.0.smithi057.stdout:7/125: mkdir d0/d5/d25 0 2024-02-28T23:07:28.281 INFO:tasks.workunit.client.1.smithi084.stdout:3/184: sync 2024-02-28T23:07:28.281 INFO:tasks.workunit.client.1.smithi084.stdout:0/172: chown d1/d6 3100 1 2024-02-28T23:07:28.282 INFO:tasks.workunit.client.1.smithi084.stdout:8/181: sync 2024-02-28T23:07:28.282 INFO:tasks.workunit.client.1.smithi084.stdout:4/174: stat d1/f12 0 2024-02-28T23:07:28.284 INFO:tasks.workunit.client.0.smithi057.stdout:0/60: getdents . 0 2024-02-28T23:07:28.284 INFO:tasks.workunit.client.0.smithi057.stdout:6/80: fsync d0/d4/d11/f13 0 2024-02-28T23:07:28.285 INFO:tasks.workunit.client.0.smithi057.stdout:8/67: write d4/f14 [5223092,35244] 0 2024-02-28T23:07:28.285 INFO:tasks.workunit.client.0.smithi057.stdout:7/126: write d0/d5/f1e [131978,12424] 0 2024-02-28T23:07:28.286 INFO:tasks.workunit.client.1.smithi084.stdout:7/195: dwrite db/f1e [4194304,4194304] 0 2024-02-28T23:07:28.287 INFO:tasks.workunit.client.0.smithi057.stdout:8/68: fsync d4/f5 0 2024-02-28T23:07:28.288 INFO:tasks.workunit.client.0.smithi057.stdout:0/61: chown f8 412 1 2024-02-28T23:07:28.290 INFO:tasks.workunit.client.1.smithi084.stdout:0/173: mkdir d1/d6/db/d35 0 2024-02-28T23:07:28.290 INFO:tasks.workunit.client.0.smithi057.stdout:0/62: fdatasync f9 0 2024-02-28T23:07:28.290 INFO:tasks.workunit.client.1.smithi084.stdout:4/175: truncate d1/f6 102798 0 2024-02-28T23:07:28.292 INFO:tasks.workunit.client.1.smithi084.stdout:3/185: link d3/d27/f2d d3/d8/df/f46 0 2024-02-28T23:07:28.292 INFO:tasks.workunit.client.1.smithi084.stdout:7/196: rmdir db/d10/d17 39 2024-02-28T23:07:28.293 INFO:tasks.workunit.client.1.smithi084.stdout:3/186: chown d3/d8/d14 1755049 1 2024-02-28T23:07:28.296 INFO:tasks.workunit.client.0.smithi057.stdout:8/69: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:28.296 INFO:tasks.workunit.client.0.smithi057.stdout:6/81: truncate d0/d4/d11/f12 4844878 0 2024-02-28T23:07:28.296 INFO:tasks.workunit.client.1.smithi084.stdout:9/241: rename d4/d6/d15/d12/l23 to d4/d6/d15/l52 0 2024-02-28T23:07:28.298 INFO:tasks.workunit.client.0.smithi057.stdout:7/127: link d0/d5/ce d0/c26 0 2024-02-28T23:07:28.299 INFO:tasks.workunit.client.0.smithi057.stdout:6/82: chown d0/d4/d11 28950 1 2024-02-28T23:07:28.299 INFO:tasks.workunit.client.0.smithi057.stdout:4/101: dwrite d1a/d1b/f1c [0,4194304] 0 2024-02-28T23:07:28.300 INFO:tasks.workunit.client.1.smithi084.stdout:4/176: symlink d1/d18/d20/d14/d30/l33 0 2024-02-28T23:07:28.301 INFO:tasks.workunit.client.0.smithi057.stdout:8/70: mkdir d4/d13/d18 0 2024-02-28T23:07:28.303 INFO:tasks.workunit.client.1.smithi084.stdout:3/187: chown d3/d8/d14/l1e 229085292 1 2024-02-28T23:07:28.303 INFO:tasks.workunit.client.1.smithi084.stdout:6/220: dwrite d1/f1b [0,4194304] 0 2024-02-28T23:07:28.304 INFO:tasks.workunit.client.1.smithi084.stdout:4/177: readlink d1/d18/d20/d15/l21 0 2024-02-28T23:07:28.304 INFO:tasks.workunit.client.0.smithi057.stdout:8/71: write d4/f7 [3756307,66730] 0 2024-02-28T23:07:28.306 INFO:tasks.workunit.client.0.smithi057.stdout:4/102: creat d1a/d1e/f29 x:0 0 0 2024-02-28T23:07:28.308 INFO:tasks.workunit.client.1.smithi084.stdout:7/197: rmdir db/d10/d17 39 2024-02-28T23:07:28.309 INFO:tasks.workunit.client.0.smithi057.stdout:6/83: write d0/d4/d11/f12 [1469910,73914] 0 2024-02-28T23:07:28.313 INFO:tasks.workunit.client.0.smithi057.stdout:4/103: rmdir d1a 39 2024-02-28T23:07:28.313 INFO:tasks.workunit.client.1.smithi084.stdout:3/188: unlink d3/d8/d14/f16 0 2024-02-28T23:07:28.315 INFO:tasks.workunit.client.1.smithi084.stdout:8/182: rename d0/d9/lc to d0/d9/d1b/d20/l41 0 2024-02-28T23:07:28.316 INFO:tasks.workunit.client.0.smithi057.stdout:0/63: dread f4 [4194304,4194304] 0 2024-02-28T23:07:28.316 INFO:tasks.workunit.client.0.smithi057.stdout:6/84: unlink d0/d4/fe 0 2024-02-28T23:07:28.317 INFO:tasks.workunit.client.1.smithi084.stdout:6/221: fdatasync d1/d9/d1d/d1e/d33/f39 0 2024-02-28T23:07:28.317 INFO:tasks.workunit.client.1.smithi084.stdout:8/183: read d0/d11/f23 [3120279,117873] 0 2024-02-28T23:07:28.319 INFO:tasks.workunit.client.1.smithi084.stdout:3/189: creat d3/d8/d1a/f47 x:0 0 0 2024-02-28T23:07:28.319 INFO:tasks.workunit.client.0.smithi057.stdout:0/64: dread - f10 zero size 2024-02-28T23:07:28.319 INFO:tasks.workunit.client.0.smithi057.stdout:6/85: write d0/d4/ff [153162,74788] 0 2024-02-28T23:07:28.321 INFO:tasks.workunit.client.0.smithi057.stdout:6/86: write d0/fc [354314,64975] 0 2024-02-28T23:07:28.325 INFO:tasks.workunit.client.1.smithi084.stdout:8/184: fdatasync d0/d9/d1b/f28 0 2024-02-28T23:07:28.327 INFO:tasks.workunit.client.0.smithi057.stdout:4/104: symlink d1a/d1b/l2a 0 2024-02-28T23:07:28.327 INFO:tasks.workunit.client.0.smithi057.stdout:0/65: creat f11 x:0 0 0 2024-02-28T23:07:28.328 INFO:tasks.workunit.client.0.smithi057.stdout:6/87: unlink d0/c16 0 2024-02-28T23:07:28.329 INFO:tasks.workunit.client.0.smithi057.stdout:0/66: write f10 [28143,124380] 0 2024-02-28T23:07:28.330 INFO:tasks.workunit.client.0.smithi057.stdout:4/105: unlink f12 0 2024-02-28T23:07:28.330 INFO:tasks.workunit.client.0.smithi057.stdout:6/88: symlink d0/d17/l1a 0 2024-02-28T23:07:28.331 INFO:tasks.workunit.client.0.smithi057.stdout:0/67: symlink l12 0 2024-02-28T23:07:28.332 INFO:tasks.workunit.client.0.smithi057.stdout:4/106: mknod d1a/c2b 0 2024-02-28T23:07:28.334 INFO:tasks.workunit.client.1.smithi084.stdout:0/174: truncate d1/d8/f1c 2370410 0 2024-02-28T23:07:28.334 INFO:tasks.workunit.client.1.smithi084.stdout:4/178: write d1/f4 [953121,109968] 0 2024-02-28T23:07:28.336 INFO:tasks.workunit.client.1.smithi084.stdout:9/242: rename d4/d6/d15/d4c to d4/d6/d53 0 2024-02-28T23:07:28.336 INFO:tasks.workunit.client.0.smithi057.stdout:8/72: truncate d4/f14 622508 0 2024-02-28T23:07:28.336 INFO:tasks.workunit.client.1.smithi084.stdout:4/179: fdatasync d1/f8 0 2024-02-28T23:07:28.337 INFO:tasks.workunit.client.0.smithi057.stdout:0/68: write f5 [9022584,111908] 0 2024-02-28T23:07:28.339 INFO:tasks.workunit.client.1.smithi084.stdout:4/180: fdatasync d1/d18/d20/d24/f2a 0 2024-02-28T23:07:28.340 INFO:tasks.workunit.client.0.smithi057.stdout:6/89: mkdir d0/d1b 0 2024-02-28T23:07:28.340 INFO:tasks.workunit.client.0.smithi057.stdout:4/107: creat d1a/d1b/f2c x:0 0 0 2024-02-28T23:07:28.340 INFO:tasks.workunit.client.0.smithi057.stdout:7/128: getdents d0 0 2024-02-28T23:07:28.341 INFO:tasks.workunit.client.1.smithi084.stdout:0/175: creat d1/d8/f36 x:0 0 0 2024-02-28T23:07:28.342 INFO:tasks.workunit.client.0.smithi057.stdout:6/90: chown d0/d4/fb 7 1 2024-02-28T23:07:28.344 INFO:tasks.workunit.client.0.smithi057.stdout:8/73: mknod d4/d13/d18/c19 0 2024-02-28T23:07:28.346 INFO:tasks.workunit.client.1.smithi084.stdout:0/176: fdatasync d1/d6/f13 0 2024-02-28T23:07:28.346 INFO:tasks.workunit.client.1.smithi084.stdout:3/190: dwrite d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:28.347 INFO:tasks.workunit.client.1.smithi084.stdout:0/177: write d1/d6/d2d/f30 [24875,124605] 0 2024-02-28T23:07:28.348 INFO:tasks.workunit.client.0.smithi057.stdout:8/74: write d4/f7 [1614150,107798] 0 2024-02-28T23:07:28.350 INFO:tasks.workunit.client.0.smithi057.stdout:7/129: truncate d0/d8/d15/f22 496625 0 2024-02-28T23:07:28.350 INFO:tasks.workunit.client.0.smithi057.stdout:8/75: read - d4/f11 zero size 2024-02-28T23:07:28.351 INFO:tasks.workunit.client.0.smithi057.stdout:2/123: sync 2024-02-28T23:07:28.351 INFO:tasks.workunit.client.0.smithi057.stdout:9/78: sync 2024-02-28T23:07:28.357 INFO:tasks.workunit.client.0.smithi057.stdout:7/130: creat d0/d5/f27 x:0 0 0 2024-02-28T23:07:28.358 INFO:tasks.workunit.client.1.smithi084.stdout:7/198: truncate db/fd 1297159 0 2024-02-28T23:07:28.359 INFO:tasks.workunit.client.1.smithi084.stdout:8/185: getdents d0/d9/d1b 0 2024-02-28T23:07:28.359 INFO:tasks.workunit.client.1.smithi084.stdout:4/181: creat d1/d18/d20/d24/f34 x:0 0 0 2024-02-28T23:07:28.360 INFO:tasks.workunit.client.0.smithi057.stdout:2/124: mknod d8/c2c 0 2024-02-28T23:07:28.362 INFO:tasks.workunit.client.0.smithi057.stdout:0/69: dread f4 [4194304,4194304] 0 2024-02-28T23:07:28.363 INFO:tasks.workunit.client.1.smithi084.stdout:8/186: fdatasync d0/d9/d1b/d38/f3a 0 2024-02-28T23:07:28.366 INFO:tasks.workunit.client.1.smithi084.stdout:8/187: stat d0/d9/d30/f3b 0 2024-02-28T23:07:28.367 INFO:tasks.workunit.client.1.smithi084.stdout:9/243: mknod d4/d32/d1f/c54 0 2024-02-28T23:07:28.368 INFO:tasks.workunit.client.1.smithi084.stdout:0/178: dwrite d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:28.368 INFO:tasks.workunit.client.1.smithi084.stdout:3/191: creat d3/d8/d1a/d21/d3d/d43/f48 x:0 0 0 2024-02-28T23:07:28.369 INFO:tasks.workunit.client.0.smithi057.stdout:8/76: getdents d4/d13 0 2024-02-28T23:07:28.370 INFO:tasks.workunit.client.1.smithi084.stdout:4/182: mkdir d1/d18/d20/d24/d2f/d35 0 2024-02-28T23:07:28.375 INFO:tasks.workunit.client.1.smithi084.stdout:8/188: creat d0/d9/d30/f42 x:0 0 0 2024-02-28T23:07:28.378 INFO:tasks.workunit.client.1.smithi084.stdout:6/222: dwrite d1/d9/f3b [0,4194304] 0 2024-02-28T23:07:28.379 INFO:tasks.workunit.client.0.smithi057.stdout:0/70: rename c6 to c13 0 2024-02-28T23:07:28.380 INFO:tasks.workunit.client.1.smithi084.stdout:9/244: write d4/d32/d1f/d2d/f34 [5073408,8231] 0 2024-02-28T23:07:28.380 INFO:tasks.workunit.client.1.smithi084.stdout:5/232: sync 2024-02-28T23:07:28.382 INFO:tasks.workunit.client.0.smithi057.stdout:2/125: creat d8/d11/d15/f2d x:0 0 0 2024-02-28T23:07:28.388 INFO:tasks.workunit.client.0.smithi057.stdout:4/108: dwrite d1a/d1b/f1f [4194304,4194304] 0 2024-02-28T23:07:28.388 INFO:tasks.workunit.client.0.smithi057.stdout:6/91: dwrite d0/d4/fb [0,4194304] 0 2024-02-28T23:07:28.389 INFO:tasks.workunit.client.0.smithi057.stdout:7/131: rename d0/f12 to d0/d5/d25/f28 0 2024-02-28T23:07:28.393 INFO:tasks.workunit.client.0.smithi057.stdout:9/79: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:28.393 INFO:tasks.workunit.client.1.smithi084.stdout:6/223: unlink d1/c16 0 2024-02-28T23:07:28.394 INFO:tasks.workunit.client.0.smithi057.stdout:1/20: sync 2024-02-28T23:07:28.394 INFO:tasks.workunit.client.0.smithi057.stdout:3/36: sync 2024-02-28T23:07:28.394 INFO:tasks.workunit.client.0.smithi057.stdout:5/25: sync 2024-02-28T23:07:28.394 INFO:tasks.workunit.client.0.smithi057.stdout:8/77: mkdir d4/d13/d1a 0 2024-02-28T23:07:28.395 INFO:tasks.workunit.client.1.smithi084.stdout:7/199: dwrite db/d10/f23 [0,4194304] 0 2024-02-28T23:07:28.398 INFO:tasks.workunit.client.0.smithi057.stdout:1/21: read f0 [2339724,42058] 0 2024-02-28T23:07:28.398 INFO:tasks.workunit.client.0.smithi057.stdout:0/71: creat f14 x:0 0 0 2024-02-28T23:07:28.400 INFO:tasks.workunit.client.0.smithi057.stdout:9/80: write dd/de/d14/f18 [1013028,58607] 0 2024-02-28T23:07:28.405 INFO:tasks.workunit.client.1.smithi084.stdout:5/233: rename dc/de/df/f3d to dc/de/df/d50/f52 0 2024-02-28T23:07:28.408 INFO:tasks.workunit.client.0.smithi057.stdout:0/72: read - f14 zero size 2024-02-28T23:07:28.408 INFO:tasks.workunit.client.1.smithi084.stdout:8/189: getdents d0/d9/d30/d33 0 2024-02-28T23:07:28.411 INFO:tasks.workunit.client.1.smithi084.stdout:6/224: chown d1/d9/d1d/f32 17243 1 2024-02-28T23:07:28.413 INFO:tasks.workunit.client.1.smithi084.stdout:3/192: write d3/d8/df/d2f/f3f [663734,27734] 0 2024-02-28T23:07:28.416 INFO:tasks.workunit.client.0.smithi057.stdout:7/132: mkdir d0/d8/d15/d29 0 2024-02-28T23:07:28.420 INFO:tasks.workunit.client.0.smithi057.stdout:3/37: unlink d3/d8/fa 0 2024-02-28T23:07:28.420 INFO:tasks.workunit.client.0.smithi057.stdout:6/92: write d0/d4/ff [630052,125030] 0 2024-02-28T23:07:28.421 INFO:tasks.workunit.client.1.smithi084.stdout:4/183: dread d1/f9 [0,4194304] 0 2024-02-28T23:07:28.421 INFO:tasks.workunit.client.1.smithi084.stdout:0/179: dwrite d1/d8/f23 [0,4194304] 0 2024-02-28T23:07:28.421 INFO:tasks.workunit.client.1.smithi084.stdout:6/225: dread - d1/d13/d2e/f44 zero size 2024-02-28T23:07:28.421 INFO:tasks.workunit.client.1.smithi084.stdout:5/234: creat dc/de/df/d16/d34/f53 x:0 0 0 2024-02-28T23:07:28.422 INFO:tasks.workunit.client.0.smithi057.stdout:2/126: rename d8/c25 to d8/d11/d15/c2e 0 2024-02-28T23:07:28.424 INFO:tasks.workunit.client.0.smithi057.stdout:8/78: creat d4/f1b x:0 0 0 2024-02-28T23:07:28.427 INFO:tasks.workunit.client.0.smithi057.stdout:3/38: write f1 [2415205,67698] 0 2024-02-28T23:07:28.427 INFO:tasks.workunit.client.0.smithi057.stdout:0/73: creat f15 x:0 0 0 2024-02-28T23:07:28.429 INFO:tasks.workunit.client.1.smithi084.stdout:1/295: sync 2024-02-28T23:07:28.430 INFO:tasks.workunit.client.1.smithi084.stdout:2/187: sync 2024-02-28T23:07:28.431 INFO:tasks.workunit.client.1.smithi084.stdout:5/235: write dc/de/df/d16/d34/f4d [317149,57410] 0 2024-02-28T23:07:28.436 INFO:tasks.workunit.client.1.smithi084.stdout:9/245: dwrite d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:28.437 INFO:tasks.workunit.client.1.smithi084.stdout:0/180: mkdir d1/d6/db/d37 0 2024-02-28T23:07:28.442 INFO:tasks.workunit.client.1.smithi084.stdout:0/181: dread - d1/d8/f36 zero size 2024-02-28T23:07:28.444 INFO:tasks.workunit.client.1.smithi084.stdout:7/200: dread db/d10/f2c [0,4194304] 0 2024-02-28T23:07:28.445 INFO:tasks.workunit.client.1.smithi084.stdout:2/188: unlink d5/d7/d12/c18 0 2024-02-28T23:07:28.445 INFO:tasks.workunit.client.1.smithi084.stdout:2/189: stat d5/d7/d9 0 2024-02-28T23:07:28.449 INFO:tasks.workunit.client.1.smithi084.stdout:5/236: mkdir dc/de/df/d16/d34/d54 0 2024-02-28T23:07:28.449 INFO:tasks.workunit.client.1.smithi084.stdout:2/190: fdatasync d5/dd/f1c 0 2024-02-28T23:07:28.451 INFO:tasks.workunit.client.1.smithi084.stdout:9/246: unlink d4/d6/d15/f33 0 2024-02-28T23:07:28.451 INFO:tasks.workunit.client.1.smithi084.stdout:8/190: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:28.454 INFO:tasks.workunit.client.1.smithi084.stdout:2/191: stat d5/d7/d2e/f2f 0 2024-02-28T23:07:28.455 INFO:tasks.workunit.client.1.smithi084.stdout:4/184: truncate d1/f1d 4027378 0 2024-02-28T23:07:28.457 INFO:tasks.workunit.client.1.smithi084.stdout:0/182: symlink d1/d8/d18/l38 0 2024-02-28T23:07:28.461 INFO:tasks.workunit.client.1.smithi084.stdout:9/247: read - d4/f13 zero size 2024-02-28T23:07:28.462 INFO:tasks.workunit.client.1.smithi084.stdout:0/183: readlink d1/d6/db/l2c 0 2024-02-28T23:07:28.464 INFO:tasks.workunit.client.1.smithi084.stdout:5/237: truncate dc/de/df/d16/f1f 574346 0 2024-02-28T23:07:28.464 INFO:tasks.workunit.client.1.smithi084.stdout:3/193: dwrite d3/d8/df/f12 [0,4194304] 0 2024-02-28T23:07:28.467 INFO:tasks.workunit.client.1.smithi084.stdout:1/296: dwrite fc [0,4194304] 0 2024-02-28T23:07:28.467 INFO:tasks.workunit.client.1.smithi084.stdout:7/201: creat db/d10/f3e x:0 0 0 2024-02-28T23:07:28.472 INFO:tasks.workunit.client.1.smithi084.stdout:9/248: chown d4/d6/f8 71128552 1 2024-02-28T23:07:28.474 INFO:tasks.workunit.client.1.smithi084.stdout:2/192: mknod d5/d7/d12/d24/d2a/c30 0 2024-02-28T23:07:28.485 INFO:tasks.workunit.client.1.smithi084.stdout:8/191: dwrite d0/d9/d30/f42 [0,4194304] 0 2024-02-28T23:07:28.486 INFO:tasks.workunit.client.1.smithi084.stdout:5/238: rmdir dc/de/df 39 2024-02-28T23:07:28.492 INFO:tasks.workunit.client.1.smithi084.stdout:1/297: creat d12/d1b/d2e/d52/d5b/f68 x:0 0 0 2024-02-28T23:07:28.494 INFO:tasks.workunit.client.1.smithi084.stdout:0/184: symlink d1/d6/db/d37/l39 0 2024-02-28T23:07:28.497 INFO:tasks.workunit.client.1.smithi084.stdout:8/192: rmdir d0/d24 39 2024-02-28T23:07:28.499 INFO:tasks.workunit.client.1.smithi084.stdout:7/202: creat db/d10/d17/d3b/f3f x:0 0 0 2024-02-28T23:07:28.500 INFO:tasks.workunit.client.1.smithi084.stdout:2/193: dwrite d5/d7/fe [0,4194304] 0 2024-02-28T23:07:28.501 INFO:tasks.workunit.client.1.smithi084.stdout:4/185: dread d1/f8 [0,4194304] 0 2024-02-28T23:07:28.502 INFO:tasks.workunit.client.1.smithi084.stdout:7/203: chown db/d10/d17/d3b 17 1 2024-02-28T23:07:28.502 INFO:tasks.workunit.client.1.smithi084.stdout:1/298: truncate d12/d1b/d2e/d30/f19 8541613 0 2024-02-28T23:07:28.503 INFO:tasks.workunit.client.1.smithi084.stdout:3/194: truncate d3/d8/d1a/d21/f32 3267566 0 2024-02-28T23:07:28.503 INFO:tasks.workunit.client.1.smithi084.stdout:0/185: unlink d1/d8/d18/f31 0 2024-02-28T23:07:28.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/186: write d1/d6/f19 [893568,49335] 0 2024-02-28T23:07:28.510 INFO:tasks.workunit.client.0.smithi057.stdout:6/93: truncate d0/d4/f19 702193 0 2024-02-28T23:07:28.512 INFO:tasks.workunit.client.0.smithi057.stdout:0/74: symlink l16 0 2024-02-28T23:07:28.512 INFO:tasks.workunit.client.0.smithi057.stdout:8/79: mknod d4/c1c 0 2024-02-28T23:07:28.513 INFO:tasks.workunit.client.1.smithi084.stdout:9/249: getdents d4/d6/d15/d12/d41 0 2024-02-28T23:07:28.513 INFO:tasks.workunit.client.1.smithi084.stdout:7/204: truncate db/d11/d27/f33 759947 0 2024-02-28T23:07:28.514 INFO:tasks.workunit.client.0.smithi057.stdout:6/94: chown d0/d17 759 1 2024-02-28T23:07:28.514 INFO:tasks.workunit.client.0.smithi057.stdout:2/127: creat d8/d11/d21/d28/f2f x:0 0 0 2024-02-28T23:07:28.516 INFO:tasks.workunit.client.1.smithi084.stdout:7/205: readlink db/d10/d17/l38 0 2024-02-28T23:07:28.517 INFO:tasks.workunit.client.0.smithi057.stdout:4/109: write f18 [2175321,127216] 0 2024-02-28T23:07:28.517 INFO:tasks.workunit.client.0.smithi057.stdout:0/75: mkdir d17 0 2024-02-28T23:07:28.517 INFO:tasks.workunit.client.1.smithi084.stdout:0/187: dread d1/d6/d2d/f30 [0,4194304] 0 2024-02-28T23:07:28.518 INFO:tasks.workunit.client.1.smithi084.stdout:1/299: symlink d12/d1b/d2e/d52/d5b/l69 0 2024-02-28T23:07:28.518 INFO:tasks.workunit.client.0.smithi057.stdout:2/128: readlink d8/lc 0 2024-02-28T23:07:28.518 INFO:tasks.workunit.client.0.smithi057.stdout:1/22: dwrite f0 [0,4194304] 0 2024-02-28T23:07:28.519 INFO:tasks.workunit.client.0.smithi057.stdout:1/23: readlink - no filename 2024-02-28T23:07:28.520 INFO:tasks.workunit.client.0.smithi057.stdout:6/95: chown d0/d4/ff 11 1 2024-02-28T23:07:28.521 INFO:tasks.workunit.client.0.smithi057.stdout:6/96: chown d0/fc 1 1 2024-02-28T23:07:28.521 INFO:tasks.workunit.client.0.smithi057.stdout:8/80: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:28.521 INFO:tasks.workunit.client.0.smithi057.stdout:9/81: dread dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:28.523 INFO:tasks.workunit.client.1.smithi084.stdout:7/206: creat db/d10/d18/f40 x:0 0 0 2024-02-28T23:07:28.524 INFO:tasks.workunit.client.1.smithi084.stdout:2/194: rename d5/f28 to d5/f31 0 2024-02-28T23:07:28.524 INFO:tasks.workunit.client.1.smithi084.stdout:0/188: chown d1/d6/d2d/c33 89641 1 2024-02-28T23:07:28.526 INFO:tasks.workunit.client.1.smithi084.stdout:9/250: write d4/fc [8445149,128462] 0 2024-02-28T23:07:28.526 INFO:tasks.workunit.client.0.smithi057.stdout:5/26: dread f1 [0,4194304] 0 2024-02-28T23:07:28.527 INFO:tasks.workunit.client.0.smithi057.stdout:2/129: read f7 [3791765,88517] 0 2024-02-28T23:07:28.527 INFO:tasks.workunit.client.0.smithi057.stdout:4/110: mkdir d1a/d1b/d2d 0 2024-02-28T23:07:28.528 INFO:tasks.workunit.client.0.smithi057.stdout:8/81: unlink d4/f1b 0 2024-02-28T23:07:28.530 INFO:tasks.workunit.client.1.smithi084.stdout:8/193: dwrite d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:28.532 INFO:tasks.workunit.client.0.smithi057.stdout:9/82: rmdir dd/de 39 2024-02-28T23:07:28.533 INFO:tasks.workunit.client.0.smithi057.stdout:2/130: mknod d8/d11/d21/c30 0 2024-02-28T23:07:28.537 INFO:tasks.workunit.client.1.smithi084.stdout:5/239: dread dc/de/df/d31/f41 [0,4194304] 0 2024-02-28T23:07:28.537 INFO:tasks.workunit.client.1.smithi084.stdout:0/189: write d1/d8/f29 [572753,63230] 0 2024-02-28T23:07:28.537 INFO:tasks.workunit.client.1.smithi084.stdout:7/207: chown db/d10/d17/d3b/f3f 95 1 2024-02-28T23:07:28.537 INFO:tasks.workunit.client.1.smithi084.stdout:2/195: write d5/f17 [4943363,105511] 0 2024-02-28T23:07:28.538 INFO:tasks.workunit.client.0.smithi057.stdout:4/111: mkdir d1a/d1b/d24/d2e 0 2024-02-28T23:07:28.538 INFO:tasks.workunit.client.0.smithi057.stdout:5/27: mknod d4/c5 0 2024-02-28T23:07:28.542 INFO:tasks.workunit.client.0.smithi057.stdout:8/82: mknod d4/d13/d1a/c1d 0 2024-02-28T23:07:28.542 INFO:tasks.workunit.client.0.smithi057.stdout:3/39: dread f1 [0,4194304] 0 2024-02-28T23:07:28.544 INFO:tasks.workunit.client.0.smithi057.stdout:2/131: creat d8/d11/d21/f31 x:0 0 0 2024-02-28T23:07:28.545 INFO:tasks.workunit.client.1.smithi084.stdout:9/251: chown d4/d6/d15/d3c/c4b 8483462 1 2024-02-28T23:07:28.548 INFO:tasks.workunit.client.1.smithi084.stdout:4/186: dwrite d1/d18/d20/d15/f28 [0,4194304] 0 2024-02-28T23:07:28.549 INFO:tasks.workunit.client.0.smithi057.stdout:9/83: write dd/de/d14/f17 [1597989,843] 0 2024-02-28T23:07:28.550 INFO:tasks.workunit.client.1.smithi084.stdout:2/196: chown d5/d7/c2d 2279883 1 2024-02-28T23:07:28.555 INFO:tasks.workunit.client.0.smithi057.stdout:8/83: mkdir d4/d1e 0 2024-02-28T23:07:28.555 INFO:tasks.workunit.client.0.smithi057.stdout:0/76: dwrite f9 [0,4194304] 0 2024-02-28T23:07:28.558 INFO:tasks.workunit.client.1.smithi084.stdout:0/190: dread d1/d6/d1b/f22 [0,4194304] 0 2024-02-28T23:07:28.558 INFO:tasks.workunit.client.1.smithi084.stdout:1/300: dread d12/d1b/d3b/f59 [0,4194304] 0 2024-02-28T23:07:28.559 INFO:tasks.workunit.client.0.smithi057.stdout:9/84: write dd/f12 [996849,7529] 0 2024-02-28T23:07:28.561 INFO:tasks.workunit.client.1.smithi084.stdout:0/191: write d1/d8/f29 [477899,12655] 0 2024-02-28T23:07:28.561 INFO:tasks.workunit.client.0.smithi057.stdout:8/84: dread - d4/f5 zero size 2024-02-28T23:07:28.563 INFO:tasks.workunit.client.0.smithi057.stdout:0/77: read f4 [1489203,71234] 0 2024-02-28T23:07:28.563 INFO:tasks.workunit.client.0.smithi057.stdout:9/85: chown dd/de/d14 1 1 2024-02-28T23:07:28.564 INFO:tasks.workunit.client.1.smithi084.stdout:6/226: sync 2024-02-28T23:07:28.565 INFO:tasks.workunit.client.1.smithi084.stdout:8/194: fdatasync d0/d9/f15 0 2024-02-28T23:07:28.565 INFO:tasks.workunit.client.0.smithi057.stdout:0/78: write f4 [7833235,35125] 0 2024-02-28T23:07:28.566 INFO:tasks.workunit.client.1.smithi084.stdout:7/208: mknod db/c41 0 2024-02-28T23:07:28.567 INFO:tasks.workunit.client.1.smithi084.stdout:9/252: mkdir d4/d6/d15/d12/d41/d55 0 2024-02-28T23:07:28.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:28 smithi057 ceph-mon[29096]: pgmap v20: 65 pgs: 65 active+clean; 304 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 227 op/s 2024-02-28T23:07:28.570 INFO:tasks.workunit.client.0.smithi057.stdout:2/132: creat d8/d11/f32 x:0 0 0 2024-02-28T23:07:28.571 INFO:tasks.workunit.client.1.smithi084.stdout:5/240: symlink dc/d51/l55 0 2024-02-28T23:07:28.573 INFO:tasks.workunit.client.0.smithi057.stdout:5/28: dread f2 [0,4194304] 0 2024-02-28T23:07:28.573 INFO:tasks.workunit.client.0.smithi057.stdout:8/85: write d4/f9 [1177780,120769] 0 2024-02-28T23:07:28.574 INFO:tasks.workunit.client.0.smithi057.stdout:9/86: rename dd/l19 to dd/de/l1a 0 2024-02-28T23:07:28.574 INFO:tasks.workunit.client.1.smithi084.stdout:3/195: dread d3/fe [0,4194304] 0 2024-02-28T23:07:28.577 INFO:tasks.workunit.client.0.smithi057.stdout:1/24: truncate f0 184112 0 2024-02-28T23:07:28.578 INFO:tasks.workunit.client.1.smithi084.stdout:0/192: mknod d1/d8/d18/d28/c3a 0 2024-02-28T23:07:28.578 INFO:tasks.workunit.client.1.smithi084.stdout:5/241: chown dc/de/df/d16/c17 19 1 2024-02-28T23:07:28.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:28 smithi084 ceph-mon[34826]: pgmap v20: 65 pgs: 65 active+clean; 304 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 227 op/s 2024-02-28T23:07:28.579 INFO:tasks.workunit.client.0.smithi057.stdout:9/87: chown l7 472174 1 2024-02-28T23:07:28.581 INFO:tasks.workunit.client.0.smithi057.stdout:8/86: write d4/f9 [2331692,16400] 0 2024-02-28T23:07:28.581 INFO:tasks.workunit.client.0.smithi057.stdout:4/112: dwrite f14 [0,4194304] 0 2024-02-28T23:07:28.581 INFO:tasks.workunit.client.0.smithi057.stdout:5/29: creat d4/f6 x:0 0 0 2024-02-28T23:07:28.583 INFO:tasks.workunit.client.1.smithi084.stdout:4/187: dwrite d1/d18/d20/d24/f2a [0,4194304] 0 2024-02-28T23:07:28.585 INFO:tasks.workunit.client.1.smithi084.stdout:8/195: mkdir d0/d9/d1b/d38/d43 0 2024-02-28T23:07:28.585 INFO:tasks.workunit.client.0.smithi057.stdout:2/133: rename d8/d11/f27 to d8/d11/d21/f33 0 2024-02-28T23:07:28.586 INFO:tasks.workunit.client.0.smithi057.stdout:0/79: link f15 d17/f18 0 2024-02-28T23:07:28.587 INFO:tasks.workunit.client.0.smithi057.stdout:9/88: write dd/de/d14/f17 [3442338,59554] 0 2024-02-28T23:07:28.587 INFO:tasks.workunit.client.0.smithi057.stdout:4/113: mkdir d1a/d2f 0 2024-02-28T23:07:28.587 INFO:tasks.workunit.client.1.smithi084.stdout:9/253: fdatasync d4/f51 0 2024-02-28T23:07:28.590 INFO:tasks.workunit.client.1.smithi084.stdout:9/254: stat d4/d32/d1f/d2d/f34 0 2024-02-28T23:07:28.597 INFO:tasks.workunit.client.1.smithi084.stdout:9/255: write d4/fc [3642305,54966] 0 2024-02-28T23:07:28.597 INFO:tasks.workunit.client.1.smithi084.stdout:7/209: rename db/d11/f31 to db/d10/d17/f42 0 2024-02-28T23:07:28.598 INFO:tasks.workunit.client.0.smithi057.stdout:0/80: write f4 [5236309,89425] 0 2024-02-28T23:07:28.598 INFO:tasks.workunit.client.0.smithi057.stdout:3/40: write d3/d4/f6 [1307522,18740] 0 2024-02-28T23:07:28.598 INFO:tasks.workunit.client.0.smithi057.stdout:9/89: unlink cc 0 2024-02-28T23:07:28.598 INFO:tasks.workunit.client.0.smithi057.stdout:5/30: write f1 [2068829,62405] 0 2024-02-28T23:07:28.598 INFO:tasks.workunit.client.0.smithi057.stdout:2/134: mknod d8/c34 0 2024-02-28T23:07:28.600 INFO:tasks.workunit.client.1.smithi084.stdout:6/227: symlink d1/l46 0 2024-02-28T23:07:28.602 INFO:tasks.workunit.client.1.smithi084.stdout:6/228: fdatasync d1/d13/d2e/d41/f2f 0 2024-02-28T23:07:28.603 INFO:tasks.workunit.client.0.smithi057.stdout:4/114: symlink d1a/d1b/d24/d2e/l30 0 2024-02-28T23:07:28.607 INFO:tasks.workunit.client.0.smithi057.stdout:0/81: mkdir d17/d19 0 2024-02-28T23:07:28.607 INFO:tasks.workunit.client.0.smithi057.stdout:2/135: creat d8/d11/d15/f35 x:0 0 0 2024-02-28T23:07:28.608 INFO:tasks.workunit.client.0.smithi057.stdout:8/87: dread d4/f7 [4194304,4194304] 0 2024-02-28T23:07:28.608 INFO:tasks.workunit.client.0.smithi057.stdout:9/90: dread dd/f12 [0,4194304] 0 2024-02-28T23:07:28.609 INFO:tasks.workunit.client.1.smithi084.stdout:1/301: mknod d12/d1b/c6a 0 2024-02-28T23:07:28.612 INFO:tasks.workunit.client.1.smithi084.stdout:5/242: rename dc/de/df/l33 to dc/de/df/d16/d34/l56 0 2024-02-28T23:07:28.614 INFO:tasks.workunit.client.0.smithi057.stdout:0/82: rename d17/d19 to d17/d19/d1a 22 2024-02-28T23:07:28.617 INFO:tasks.workunit.client.0.smithi057.stdout:4/115: creat d1a/d2f/f31 x:0 0 0 2024-02-28T23:07:28.618 INFO:tasks.workunit.client.1.smithi084.stdout:7/210: dread db/d10/f23 [0,4194304] 0 2024-02-28T23:07:28.620 INFO:tasks.workunit.client.1.smithi084.stdout:6/229: write d1/d9/f40 [2973229,9014] 0 2024-02-28T23:07:28.620 INFO:tasks.workunit.client.0.smithi057.stdout:0/83: write f10 [743827,4291] 0 2024-02-28T23:07:28.621 INFO:tasks.workunit.client.0.smithi057.stdout:2/136: creat d8/d11/d21/f36 x:0 0 0 2024-02-28T23:07:28.622 INFO:tasks.workunit.client.1.smithi084.stdout:1/302: creat d12/d1b/d2e/d52/f6b x:0 0 0 2024-02-28T23:07:28.622 INFO:tasks.workunit.client.1.smithi084.stdout:4/188: fsync d1/f6 0 2024-02-28T23:07:28.625 INFO:tasks.workunit.client.0.smithi057.stdout:9/91: rmdir dd/de/d14 39 2024-02-28T23:07:28.627 INFO:tasks.workunit.client.1.smithi084.stdout:2/197: dread d5/d7/d9/f2c [0,4194304] 0 2024-02-28T23:07:28.628 INFO:tasks.workunit.client.1.smithi084.stdout:7/211: creat db/d11/d27/f43 x:0 0 0 2024-02-28T23:07:28.630 INFO:tasks.workunit.client.0.smithi057.stdout:3/41: chown d3/d8/f9 57724 1 2024-02-28T23:07:28.631 INFO:tasks.workunit.client.1.smithi084.stdout:6/230: mkdir d1/d13/d47 0 2024-02-28T23:07:28.631 INFO:tasks.workunit.client.1.smithi084.stdout:5/243: getdents dc/de/df/d16/d34/d54 0 2024-02-28T23:07:28.631 INFO:tasks.workunit.client.1.smithi084.stdout:0/193: getdents d1/d6/db/d37 0 2024-02-28T23:07:28.632 INFO:tasks.workunit.client.0.smithi057.stdout:5/31: dwrite f1 [8388608,4194304] 0 2024-02-28T23:07:28.632 INFO:tasks.workunit.client.0.smithi057.stdout:3/42: chown d3 697 1 2024-02-28T23:07:28.634 INFO:tasks.workunit.client.1.smithi084.stdout:5/244: chown dc/de/df/d16/l35 56977 1 2024-02-28T23:07:28.634 INFO:tasks.workunit.client.1.smithi084.stdout:0/194: write d1/d8/f1d [3295145,10016] 0 2024-02-28T23:07:28.635 INFO:tasks.workunit.client.0.smithi057.stdout:8/88: dread d4/f17 [0,4194304] 0 2024-02-28T23:07:28.636 INFO:tasks.workunit.client.1.smithi084.stdout:4/189: creat d1/d18/d20/d14/f36 x:0 0 0 2024-02-28T23:07:28.637 INFO:tasks.workunit.client.0.smithi057.stdout:5/32: truncate d4/f6 873562 0 2024-02-28T23:07:28.637 INFO:tasks.workunit.client.0.smithi057.stdout:0/84: mknod d17/d19/c1b 0 2024-02-28T23:07:28.638 INFO:tasks.workunit.client.0.smithi057.stdout:5/33: stat d4 0 2024-02-28T23:07:28.638 INFO:tasks.workunit.client.1.smithi084.stdout:0/195: write d1/d8/f29 [1177043,68631] 0 2024-02-28T23:07:28.639 INFO:tasks.workunit.client.1.smithi084.stdout:4/190: chown d1/d18/d20/d14/d30 8650472 1 2024-02-28T23:07:28.639 INFO:tasks.workunit.client.0.smithi057.stdout:1/25: truncate f0 838249 0 2024-02-28T23:07:28.640 INFO:tasks.workunit.client.1.smithi084.stdout:8/196: rename d0/d9/d30/d3f to d0/d44 0 2024-02-28T23:07:28.643 INFO:tasks.workunit.client.0.smithi057.stdout:3/43: symlink d3/d4/d7/le 0 2024-02-28T23:07:28.643 INFO:tasks.workunit.client.0.smithi057.stdout:1/26: mkdir d2/d4 0 2024-02-28T23:07:28.643 INFO:tasks.workunit.client.0.smithi057.stdout:1/27: readlink - no filename 2024-02-28T23:07:28.643 INFO:tasks.workunit.client.1.smithi084.stdout:4/191: truncate d1/f2e 727979 0 2024-02-28T23:07:28.643 INFO:tasks.workunit.client.0.smithi057.stdout:5/34: chown f1 1481716 1 2024-02-28T23:07:28.646 INFO:tasks.workunit.client.1.smithi084.stdout:8/197: stat d0/d9/d1b/d20/f25 0 2024-02-28T23:07:28.646 INFO:tasks.workunit.client.1.smithi084.stdout:6/231: creat d1/d9/f48 x:0 0 0 2024-02-28T23:07:28.649 INFO:tasks.workunit.client.1.smithi084.stdout:1/303: dwrite d12/d1b/f54 [0,4194304] 0 2024-02-28T23:07:28.651 INFO:tasks.workunit.client.1.smithi084.stdout:8/198: fdatasync d0/d9/d30/f3b 0 2024-02-28T23:07:28.653 INFO:tasks.workunit.client.0.smithi057.stdout:8/89: link d4/c1c d4/c1f 0 2024-02-28T23:07:28.654 INFO:tasks.workunit.client.1.smithi084.stdout:5/245: creat dc/de/df/d16/f57 x:0 0 0 2024-02-28T23:07:28.654 INFO:tasks.workunit.client.1.smithi084.stdout:7/212: dwrite db/d11/f12 [4194304,4194304] 0 2024-02-28T23:07:28.654 INFO:tasks.workunit.client.1.smithi084.stdout:1/304: stat d12/d1b/f4a 0 2024-02-28T23:07:28.657 INFO:tasks.workunit.client.0.smithi057.stdout:2/137: dread d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:28.658 INFO:tasks.workunit.client.0.smithi057.stdout:4/116: dwrite d1a/d1b/f1f [4194304,4194304] 0 2024-02-28T23:07:28.661 INFO:tasks.workunit.client.0.smithi057.stdout:5/35: dread d4/f6 [0,4194304] 0 2024-02-28T23:07:28.664 INFO:tasks.workunit.client.0.smithi057.stdout:4/117: rmdir d1a 39 2024-02-28T23:07:28.666 INFO:tasks.workunit.client.1.smithi084.stdout:8/199: dread d0/f34 [0,4194304] 0 2024-02-28T23:07:28.667 INFO:tasks.workunit.client.0.smithi057.stdout:4/118: fsync f16 0 2024-02-28T23:07:28.667 INFO:tasks.workunit.client.0.smithi057.stdout:2/138: mknod d8/d11/d21/c37 0 2024-02-28T23:07:28.667 INFO:tasks.workunit.client.1.smithi084.stdout:0/196: symlink d1/d8/d18/l3b 0 2024-02-28T23:07:28.668 INFO:tasks.workunit.client.1.smithi084.stdout:3/196: truncate d3/fc 361706 0 2024-02-28T23:07:28.670 INFO:tasks.workunit.client.0.smithi057.stdout:8/90: creat d4/d13/d1a/f20 x:0 0 0 2024-02-28T23:07:28.681 INFO:tasks.workunit.client.1.smithi084.stdout:1/305: unlink d12/d1b/d2e/d32/f4d 0 2024-02-28T23:07:28.681 INFO:tasks.workunit.client.1.smithi084.stdout:7/213: creat db/d11/f44 x:0 0 0 2024-02-28T23:07:28.681 INFO:tasks.workunit.client.1.smithi084.stdout:9/256: truncate d4/d6/d15/f2c 3277363 0 2024-02-28T23:07:28.681 INFO:tasks.workunit.client.0.smithi057.stdout:9/92: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:28.682 INFO:tasks.workunit.client.0.smithi057.stdout:2/139: mknod d8/d11/d15/c38 0 2024-02-28T23:07:28.682 INFO:tasks.workunit.client.0.smithi057.stdout:4/119: write d1a/d2f/f31 [295167,80877] 0 2024-02-28T23:07:28.682 INFO:tasks.workunit.client.0.smithi057.stdout:0/85: dwrite f5 [8388608,4194304] 0 2024-02-28T23:07:28.682 INFO:tasks.workunit.client.0.smithi057.stdout:3/44: write d3/d8/fb [45014,104382] 0 2024-02-28T23:07:28.683 INFO:tasks.workunit.client.1.smithi084.stdout:7/214: chown db/d10/d17/l39 7 1 2024-02-28T23:07:28.684 INFO:tasks.workunit.client.1.smithi084.stdout:4/192: mkdir d1/d18/d20/d24/d2f/d35/d37 0 2024-02-28T23:07:28.687 INFO:tasks.workunit.client.1.smithi084.stdout:6/232: symlink d1/d13/d47/l49 0 2024-02-28T23:07:28.688 INFO:tasks.workunit.client.0.smithi057.stdout:2/140: dread - d8/fe zero size 2024-02-28T23:07:28.689 INFO:tasks.workunit.client.1.smithi084.stdout:7/215: write db/d11/f29 [1158783,101128] 0 2024-02-28T23:07:28.691 INFO:tasks.workunit.client.0.smithi057.stdout:3/45: unlink d3/d4/f6 0 2024-02-28T23:07:28.691 INFO:tasks.workunit.client.0.smithi057.stdout:5/36: dwrite f1 [8388608,4194304] 0 2024-02-28T23:07:28.693 INFO:tasks.workunit.client.0.smithi057.stdout:0/86: mknod d17/c1c 0 2024-02-28T23:07:28.696 INFO:tasks.workunit.client.1.smithi084.stdout:5/246: symlink dc/de/df/d16/d34/d3c/l58 0 2024-02-28T23:07:28.696 INFO:tasks.workunit.client.1.smithi084.stdout:8/200: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:28.698 INFO:tasks.workunit.client.1.smithi084.stdout:5/247: write dc/f2a [4524850,87121] 0 2024-02-28T23:07:28.699 INFO:tasks.workunit.client.1.smithi084.stdout:2/198: truncate d5/f10 1021106 0 2024-02-28T23:07:28.701 INFO:tasks.workunit.client.0.smithi057.stdout:9/93: mkdir dd/d1b 0 2024-02-28T23:07:28.761 INFO:tasks.workunit.client.0.smithi057.stdout:3/46: mknod d3/d4/d7/cf 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:2/141: mkdir d8/d11/d39 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:0/87: rename l12 to d17/d19/l1d 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:8/91: dwrite d4/f9 [0,4194304] 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:3/47: rmdir d3/d8 39 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:4/120: dwrite f17 [4194304,4194304] 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:0/88: creat d17/f1e x:0 0 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:2/142: mknod d8/d11/d21/c3a 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:2/143: chown d8/f23 49111614 1 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:9/94: symlink dd/de/l1c 0 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:6/97: sync 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:7/133: sync 2024-02-28T23:07:28.762 INFO:tasks.workunit.client.0.smithi057.stdout:4/121: rmdir d1a/d1e 39 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.0.smithi057.stdout:1/28: dwrite f0 [0,4194304] 0 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.0.smithi057.stdout:3/48: write f1 [4719678,130039] 0 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.0.smithi057.stdout:0/89: mkdir d17/d19/d1f 0 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.0.smithi057.stdout:8/92: link c1 d4/d13/d1a/c21 0 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.1.smithi084.stdout:2/199: write f3 [3567971,128532] 0 2024-02-28T23:07:28.763 INFO:tasks.workunit.client.1.smithi084.stdout:5/248: write dc/de/f12 [2782940,68388] 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:7/216: symlink db/d10/d17/d3b/l45 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:6/233: rename d1/d13/d47/l49 to d1/d13/d2e/d41/l4a 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:6/234: read - d1/d9/f48 zero size 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:4/193: fdatasync d1/f1d 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:4/194: stat d1/d18/d20/d15/f1a 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:9/257: dwrite d4/fc [4194304,4194304] 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:2/200: creat d5/d7/d9/f32 x:0 0 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:1/306: creat d12/d1b/d2e/f6c x:0 0 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:5/249: rename dc/de/c30 to dc/de/df/d16/d34/d54/c59 0 2024-02-28T23:07:28.764 INFO:tasks.workunit.client.1.smithi084.stdout:0/197: getdents d1/d8 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:7/217: dwrite db/d10/d18/f1b [0,4194304] 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:3/197: dread d3/f11 [0,4194304] 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:4/195: creat d1/d18/d20/d14/d30/f38 x:0 0 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:1/307: creat d12/d1b/d2e/d52/f6d x:0 0 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:0/198: creat d1/d8/f3c x:0 0 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:4/196: chown d1/d18/d20/d15/l21 634 1 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:7/218: creat db/d10/d18/f46 x:0 0 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:7/219: dread - db/d10/d17/f3d zero size 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:8/201: dwrite d0/f34 [4194304,4194304] 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:9/258: creat d4/f56 x:0 0 0 2024-02-28T23:07:28.765 INFO:tasks.workunit.client.1.smithi084.stdout:0/199: symlink d1/d8/d18/l3d 0 2024-02-28T23:07:28.766 INFO:tasks.workunit.client.1.smithi084.stdout:0/200: fsync d1/d8/f29 0 2024-02-28T23:07:28.769 INFO:tasks.workunit.client.1.smithi084.stdout:4/197: rename d1/f5 to d1/d18/d20/d14/f39 0 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:4/122: stat d1a/d1e/l22 0 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:5/37: dwrite d4/f6 [0,4194304] 0 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/90: chown f14 705 1 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:6/98: rmdir d0 39 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:3/49: write d3/fd [253692,88710] 0 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/91: write f4 [9039540,65705] 0 2024-02-28T23:07:28.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/92: dread - fe zero size 2024-02-28T23:07:28.771 INFO:tasks.workunit.client.1.smithi084.stdout:4/198: dread - d1/d18/f32 zero size 2024-02-28T23:07:28.777 INFO:tasks.workunit.client.0.smithi057.stdout:5/38: dread f2 [0,4194304] 0 2024-02-28T23:07:28.777 INFO:tasks.workunit.client.0.smithi057.stdout:8/93: symlink d4/d13/d1a/l22 0 2024-02-28T23:07:28.778 INFO:tasks.workunit.client.0.smithi057.stdout:7/134: read d0/d5/d25/f28 [1076290,96506] 0 2024-02-28T23:07:28.779 INFO:tasks.workunit.client.0.smithi057.stdout:4/123: creat d1a/d1b/f32 x:0 0 0 2024-02-28T23:07:28.785 INFO:tasks.workunit.client.1.smithi084.stdout:3/198: link f1 d3/d8/d14/f49 0 2024-02-28T23:07:28.785 INFO:tasks.workunit.client.0.smithi057.stdout:8/94: fdatasync d4/f9 0 2024-02-28T23:07:28.786 INFO:tasks.workunit.client.0.smithi057.stdout:5/39: write f1 [5334826,67026] 0 2024-02-28T23:07:28.786 INFO:tasks.workunit.client.0.smithi057.stdout:0/93: rename d17/d19/d1f to d17/d20 0 2024-02-28T23:07:28.787 INFO:tasks.workunit.client.0.smithi057.stdout:2/144: dread d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:28.787 INFO:tasks.workunit.client.1.smithi084.stdout:3/199: fsync d3/d8/d14/d31/d34/f2b 0 2024-02-28T23:07:28.788 INFO:tasks.workunit.client.1.smithi084.stdout:7/220: creat db/d10/d18/f47 x:0 0 0 2024-02-28T23:07:28.789 INFO:tasks.workunit.client.0.smithi057.stdout:9/95: creat dd/de/f1d x:0 0 0 2024-02-28T23:07:28.789 INFO:tasks.workunit.client.0.smithi057.stdout:7/135: write d0/d5/f27 [111037,123693] 0 2024-02-28T23:07:28.789 INFO:tasks.workunit.client.0.smithi057.stdout:6/99: stat d0/d4/fa 0 2024-02-28T23:07:28.790 INFO:tasks.workunit.client.0.smithi057.stdout:2/145: stat d8/fe 0 2024-02-28T23:07:28.792 INFO:tasks.workunit.client.1.smithi084.stdout:7/221: chown db/d10/d17/f42 51633 1 2024-02-28T23:07:28.792 INFO:tasks.workunit.client.0.smithi057.stdout:8/95: stat d4/d13/d1a/f20 0 2024-02-28T23:07:28.796 INFO:tasks.workunit.client.0.smithi057.stdout:9/96: dread - dd/de/f1d zero size 2024-02-28T23:07:28.796 INFO:tasks.workunit.client.0.smithi057.stdout:5/40: fsync f1 0 2024-02-28T23:07:28.800 INFO:tasks.workunit.client.0.smithi057.stdout:4/124: unlink f17 0 2024-02-28T23:07:28.800 INFO:tasks.workunit.client.0.smithi057.stdout:1/29: dread f0 [0,4194304] 0 2024-02-28T23:07:28.801 INFO:tasks.workunit.client.1.smithi084.stdout:0/201: creat d1/d8/f3e x:0 0 0 2024-02-28T23:07:28.801 INFO:tasks.workunit.client.1.smithi084.stdout:2/201: getdents d5/d7 0 2024-02-28T23:07:28.801 INFO:tasks.workunit.client.1.smithi084.stdout:4/199: dread d1/d18/d20/d15/f1a [0,4194304] 0 2024-02-28T23:07:28.804 INFO:tasks.workunit.client.1.smithi084.stdout:2/202: write d5/dd/f1c [1922644,78029] 0 2024-02-28T23:07:28.804 INFO:tasks.workunit.client.1.smithi084.stdout:5/250: rmdir dc/de 39 2024-02-28T23:07:28.805 INFO:tasks.workunit.client.0.smithi057.stdout:0/94: mknod d17/c21 0 2024-02-28T23:07:28.806 INFO:tasks.workunit.client.1.smithi084.stdout:8/202: mkdir d0/d9/d1b/d38/d43/d45 0 2024-02-28T23:07:28.806 INFO:tasks.workunit.client.1.smithi084.stdout:1/308: link d12/d1b/l27 d12/d1b/d63/l6e 0 2024-02-28T23:07:28.807 INFO:tasks.workunit.client.0.smithi057.stdout:0/95: write f4 [5031547,31001] 0 2024-02-28T23:07:28.808 INFO:tasks.workunit.client.0.smithi057.stdout:2/146: mknod d8/d11/d21/d28/c3b 0 2024-02-28T23:07:28.809 INFO:tasks.workunit.client.1.smithi084.stdout:1/309: fsync d12/d1b/d2e/d30/f58 0 2024-02-28T23:07:28.809 INFO:tasks.workunit.client.1.smithi084.stdout:8/203: dread - d0/d9/f35 zero size 2024-02-28T23:07:28.812 INFO:tasks.workunit.client.1.smithi084.stdout:0/202: creat d1/d8/f3f x:0 0 0 2024-02-28T23:07:28.812 INFO:tasks.workunit.client.1.smithi084.stdout:4/200: truncate d1/f6 269388 0 2024-02-28T23:07:28.813 INFO:tasks.workunit.client.0.smithi057.stdout:5/41: mkdir d4/d7 0 2024-02-28T23:07:28.814 INFO:tasks.workunit.client.1.smithi084.stdout:9/259: dwrite d4/d6/d15/d3c/f47 [4194304,4194304] 0 2024-02-28T23:07:28.816 INFO:tasks.workunit.client.0.smithi057.stdout:9/97: creat dd/de/d14/f1e x:0 0 0 2024-02-28T23:07:28.818 INFO:tasks.workunit.client.1.smithi084.stdout:4/201: write d1/d18/d20/d15/f28 [3164212,57841] 0 2024-02-28T23:07:28.819 INFO:tasks.workunit.client.0.smithi057.stdout:1/30: fdatasync f0 0 2024-02-28T23:07:28.819 INFO:tasks.workunit.client.0.smithi057.stdout:7/136: creat d0/d8/d15/d29/f2a x:0 0 0 2024-02-28T23:07:28.819 INFO:tasks.workunit.client.0.smithi057.stdout:0/96: mknod d17/c22 0 2024-02-28T23:07:28.822 INFO:tasks.workunit.client.1.smithi084.stdout:1/310: rename d12/d1b/d63/f43 to d12/d1b/d2e/d52/d5b/f6f 0 2024-02-28T23:07:28.823 INFO:tasks.workunit.client.0.smithi057.stdout:6/100: dwrite d0/d4/fb [0,4194304] 0 2024-02-28T23:07:28.823 INFO:tasks.workunit.client.1.smithi084.stdout:4/202: write d1/d18/d20/d24/f2a [2462396,118472] 0 2024-02-28T23:07:28.824 INFO:tasks.workunit.client.1.smithi084.stdout:5/251: fdatasync dc/de/df/d50/f52 0 2024-02-28T23:07:28.824 INFO:tasks.workunit.client.1.smithi084.stdout:9/260: write d4/d6/f29 [2362048,26913] 0 2024-02-28T23:07:28.829 INFO:tasks.workunit.client.0.smithi057.stdout:4/125: mknod d1a/c33 0 2024-02-28T23:07:28.831 INFO:tasks.workunit.client.0.smithi057.stdout:8/96: truncate d4/f9 3929009 0 2024-02-28T23:07:28.838 INFO:tasks.workunit.client.0.smithi057.stdout:2/147: creat d8/d11/d39/f3c x:0 0 0 2024-02-28T23:07:28.839 INFO:tasks.workunit.client.0.smithi057.stdout:4/126: symlink d1a/d1b/d24/l34 0 2024-02-28T23:07:28.842 INFO:tasks.workunit.client.0.smithi057.stdout:5/42: dread f1 [0,4194304] 0 2024-02-28T23:07:28.845 INFO:tasks.workunit.client.0.smithi057.stdout:8/97: creat d4/d13/d18/f23 x:0 0 0 2024-02-28T23:07:28.850 INFO:tasks.workunit.client.0.smithi057.stdout:2/148: chown d8/l13 10 1 2024-02-28T23:07:28.855 INFO:tasks.workunit.client.0.smithi057.stdout:2/149: write d8/d11/f1b [254939,1481] 0 2024-02-28T23:07:28.859 INFO:tasks.workunit.client.0.smithi057.stdout:7/137: dwrite d0/d5/f1e [0,4194304] 0 2024-02-28T23:07:28.859 INFO:tasks.workunit.client.0.smithi057.stdout:8/98: write d4/f11 [825028,67415] 0 2024-02-28T23:07:28.860 INFO:tasks.workunit.client.0.smithi057.stdout:4/127: creat d1a/d1b/d2d/f35 x:0 0 0 2024-02-28T23:07:28.863 INFO:tasks.workunit.client.0.smithi057.stdout:0/97: dwrite f8 [0,4194304] 0 2024-02-28T23:07:28.863 INFO:tasks.workunit.client.0.smithi057.stdout:4/128: fdatasync f16 0 2024-02-28T23:07:28.865 INFO:tasks.workunit.client.0.smithi057.stdout:1/31: dwrite f0 [0,4194304] 0 2024-02-28T23:07:28.865 INFO:tasks.workunit.client.0.smithi057.stdout:5/43: mkdir d4/d7/d8 0 2024-02-28T23:07:28.866 INFO:tasks.workunit.client.0.smithi057.stdout:0/98: stat cb 0 2024-02-28T23:07:28.870 INFO:tasks.workunit.client.0.smithi057.stdout:7/138: rmdir d0/d8/d15/d29 39 2024-02-28T23:07:28.876 INFO:tasks.workunit.client.0.smithi057.stdout:5/44: mkdir d4/d9 0 2024-02-28T23:07:28.882 INFO:tasks.workunit.client.0.smithi057.stdout:1/32: mknod d2/d4/c5 0 2024-02-28T23:07:28.885 INFO:tasks.workunit.client.0.smithi057.stdout:0/99: dread f8 [0,4194304] 0 2024-02-28T23:07:28.887 INFO:tasks.workunit.client.1.smithi084.stdout:7/222: dread db/fd [0,4194304] 0 2024-02-28T23:07:28.889 INFO:tasks.workunit.client.0.smithi057.stdout:4/129: dread f15 [0,4194304] 0 2024-02-28T23:07:28.890 INFO:tasks.workunit.client.0.smithi057.stdout:0/100: read f9 [830320,4297] 0 2024-02-28T23:07:28.890 INFO:tasks.workunit.client.0.smithi057.stdout:5/45: getdents d4/d7/d8 0 2024-02-28T23:07:28.890 INFO:tasks.workunit.client.0.smithi057.stdout:4/130: write f18 [3632038,110191] 0 2024-02-28T23:07:28.890 INFO:tasks.workunit.client.1.smithi084.stdout:2/203: unlink d5/f10 0 2024-02-28T23:07:28.896 INFO:tasks.workunit.client.1.smithi084.stdout:7/223: unlink db/d10/d18/f40 0 2024-02-28T23:07:28.898 INFO:tasks.workunit.client.1.smithi084.stdout:8/204: dwrite d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:28.899 INFO:tasks.workunit.client.1.smithi084.stdout:9/261: mknod d4/d6/d15/c57 0 2024-02-28T23:07:28.902 INFO:tasks.workunit.client.0.smithi057.stdout:1/33: link d2/d4/c5 d2/d4/c6 0 2024-02-28T23:07:28.905 INFO:tasks.workunit.client.0.smithi057.stdout:4/131: creat d1a/f36 x:0 0 0 2024-02-28T23:07:28.908 INFO:tasks.workunit.client.1.smithi084.stdout:8/205: write d0/f1a [279202,10441] 0 2024-02-28T23:07:28.908 INFO:tasks.workunit.client.1.smithi084.stdout:5/252: dread dc/de/df/d31/f41 [0,4194304] 0 2024-02-28T23:07:28.909 INFO:tasks.workunit.client.1.smithi084.stdout:9/262: fdatasync d4/d6/d15/f2a 0 2024-02-28T23:07:28.910 INFO:tasks.workunit.client.0.smithi057.stdout:7/139: truncate d0/d5/f27 107728 0 2024-02-28T23:07:28.910 INFO:tasks.workunit.client.0.smithi057.stdout:1/34: mkdir d2/d7 0 2024-02-28T23:07:28.911 INFO:tasks.workunit.client.0.smithi057.stdout:4/132: write d1a/d1b/f2c [782277,98284] 0 2024-02-28T23:07:28.911 INFO:tasks.workunit.client.0.smithi057.stdout:5/46: getdents d4/d7 0 2024-02-28T23:07:28.912 INFO:tasks.workunit.client.1.smithi084.stdout:1/311: dwrite d12/d1b/d2e/d30/f19 [0,4194304] 0 2024-02-28T23:07:28.912 INFO:tasks.workunit.client.0.smithi057.stdout:8/99: write d4/f9 [29370,75838] 0 2024-02-28T23:07:28.915 INFO:tasks.workunit.client.0.smithi057.stdout:5/47: fdatasync f2 0 2024-02-28T23:07:28.917 INFO:tasks.workunit.client.1.smithi084.stdout:1/312: write d12/d1b/d63/f2a [1230749,20935] 0 2024-02-28T23:07:28.921 INFO:tasks.workunit.client.1.smithi084.stdout:5/253: creat dc/de/df/d16/d34/d3c/f5a x:0 0 0 2024-02-28T23:07:28.921 INFO:tasks.workunit.client.1.smithi084.stdout:2/204: truncate d5/dd/f1c 1291893 0 2024-02-28T23:07:28.924 INFO:tasks.workunit.client.0.smithi057.stdout:7/140: creat d0/d8/d15/f2b x:0 0 0 2024-02-28T23:07:28.924 INFO:tasks.workunit.client.0.smithi057.stdout:0/101: truncate f8 512014 0 2024-02-28T23:07:28.925 INFO:tasks.workunit.client.1.smithi084.stdout:7/224: dread db/d11/f12 [0,4194304] 0 2024-02-28T23:07:28.926 INFO:tasks.workunit.client.0.smithi057.stdout:8/100: getdents d4/d13/d18 0 2024-02-28T23:07:28.926 INFO:tasks.workunit.client.1.smithi084.stdout:8/206: write d0/d11/f23 [2626136,117583] 0 2024-02-28T23:07:28.927 INFO:tasks.workunit.client.0.smithi057.stdout:0/102: truncate f10 1354653 0 2024-02-28T23:07:28.928 INFO:tasks.workunit.client.1.smithi084.stdout:5/254: read dc/de/df/d16/f29 [4036831,61198] 0 2024-02-28T23:07:28.928 INFO:tasks.workunit.client.0.smithi057.stdout:7/141: dread - d0/d8/d15/d29/f2a zero size 2024-02-28T23:07:28.929 INFO:tasks.workunit.client.1.smithi084.stdout:7/225: dread - db/d10/d17/f42 zero size 2024-02-28T23:07:28.930 INFO:tasks.workunit.client.0.smithi057.stdout:4/133: dread d1a/d1b/f1f [4194304,4194304] 0 2024-02-28T23:07:28.931 INFO:tasks.workunit.client.0.smithi057.stdout:8/101: rmdir d4 39 2024-02-28T23:07:28.932 INFO:tasks.workunit.client.0.smithi057.stdout:7/142: write d0/d8/d15/f2b [186290,51135] 0 2024-02-28T23:07:28.933 INFO:tasks.workunit.client.0.smithi057.stdout:4/134: truncate f15 4676285 0 2024-02-28T23:07:28.935 INFO:tasks.workunit.client.0.smithi057.stdout:8/102: chown d4/d13/d1a/c1d 3 1 2024-02-28T23:07:28.935 INFO:tasks.workunit.client.1.smithi084.stdout:9/263: dread d4/d6/d15/f2c [0,4194304] 0 2024-02-28T23:07:28.935 INFO:tasks.workunit.client.1.smithi084.stdout:1/313: dread d12/d1b/d3b/f44 [0,4194304] 0 2024-02-28T23:07:28.936 INFO:tasks.workunit.client.0.smithi057.stdout:5/48: dread f2 [0,4194304] 0 2024-02-28T23:07:28.938 INFO:tasks.workunit.client.0.smithi057.stdout:8/103: rename l0 to d4/d13/d1a/l24 0 2024-02-28T23:07:28.938 INFO:tasks.workunit.client.1.smithi084.stdout:7/226: rename db/d10/d18/f46 to db/d10/d17/d3b/f48 0 2024-02-28T23:07:28.939 INFO:tasks.workunit.client.0.smithi057.stdout:4/135: creat d1a/f37 x:0 0 0 2024-02-28T23:07:28.939 INFO:tasks.workunit.client.1.smithi084.stdout:7/227: dread - db/f22 zero size 2024-02-28T23:07:28.939 INFO:tasks.workunit.client.1.smithi084.stdout:2/205: link d5/d7/d12/l26 d5/d7/d1d/l33 0 2024-02-28T23:07:28.942 INFO:tasks.workunit.client.0.smithi057.stdout:5/49: symlink d4/la 0 2024-02-28T23:07:28.944 INFO:tasks.workunit.client.1.smithi084.stdout:6/235: sync 2024-02-28T23:07:28.944 INFO:tasks.workunit.client.1.smithi084.stdout:3/200: sync 2024-02-28T23:07:28.944 INFO:tasks.workunit.client.0.smithi057.stdout:5/50: chown d4/la 99909501 1 2024-02-28T23:07:28.945 INFO:tasks.workunit.client.0.smithi057.stdout:8/104: symlink d4/l25 0 2024-02-28T23:07:28.946 INFO:tasks.workunit.client.0.smithi057.stdout:1/35: truncate f0 3484176 0 2024-02-28T23:07:28.946 INFO:tasks.workunit.client.1.smithi084.stdout:6/236: chown d1/f31 3248547 1 2024-02-28T23:07:28.946 INFO:tasks.workunit.client.1.smithi084.stdout:7/228: mknod db/d11/c49 0 2024-02-28T23:07:28.950 INFO:tasks.workunit.client.1.smithi084.stdout:5/255: truncate f7 1337802 0 2024-02-28T23:07:28.951 INFO:tasks.workunit.client.1.smithi084.stdout:7/229: truncate db/d10/d18/f47 860213 0 2024-02-28T23:07:28.951 INFO:tasks.workunit.client.0.smithi057.stdout:8/105: creat d4/d13/d18/f26 x:0 0 0 2024-02-28T23:07:28.951 INFO:tasks.workunit.client.1.smithi084.stdout:6/237: creat d1/d13/d47/f4b x:0 0 0 2024-02-28T23:07:28.958 INFO:tasks.workunit.client.0.smithi057.stdout:1/36: rmdir d2/d3 0 2024-02-28T23:07:28.960 INFO:tasks.workunit.client.1.smithi084.stdout:9/264: dwrite d4/d6/d15/f2a [0,4194304] 0 2024-02-28T23:07:28.960 INFO:tasks.workunit.client.1.smithi084.stdout:1/314: truncate d12/f40 2696065 0 2024-02-28T23:07:28.960 INFO:tasks.workunit.client.1.smithi084.stdout:7/230: unlink db/f22 0 2024-02-28T23:07:28.960 INFO:tasks.workunit.client.0.smithi057.stdout:0/103: write f9 [905982,18538] 0 2024-02-28T23:07:28.960 INFO:tasks.workunit.client.0.smithi057.stdout:5/51: dread f1 [0,4194304] 0 2024-02-28T23:07:28.961 INFO:tasks.workunit.client.1.smithi084.stdout:5/256: symlink dc/de/df/d50/l5b 0 2024-02-28T23:07:28.961 INFO:tasks.workunit.client.1.smithi084.stdout:6/238: write d1/d9/f40 [6623081,21940] 0 2024-02-28T23:07:28.961 INFO:tasks.workunit.client.0.smithi057.stdout:4/136: dwrite d1a/f36 [0,4194304] 0 2024-02-28T23:07:28.962 INFO:tasks.workunit.client.0.smithi057.stdout:0/104: write f10 [1615480,57071] 0 2024-02-28T23:07:28.962 INFO:tasks.workunit.client.1.smithi084.stdout:5/257: stat dc/de/df/d31/l4b 0 2024-02-28T23:07:28.963 INFO:tasks.workunit.client.0.smithi057.stdout:1/37: creat d2/f8 x:0 0 0 2024-02-28T23:07:28.967 INFO:tasks.workunit.client.0.smithi057.stdout:0/105: write f4 [6680577,73581] 0 2024-02-28T23:07:28.968 INFO:tasks.workunit.client.1.smithi084.stdout:7/231: write f9 [4508521,39703] 0 2024-02-28T23:07:28.971 INFO:tasks.workunit.client.1.smithi084.stdout:1/315: rmdir d12/d1b/d2e 39 2024-02-28T23:07:28.973 INFO:tasks.workunit.client.1.smithi084.stdout:5/258: unlink dc/de/df/d16/f1f 0 2024-02-28T23:07:28.974 INFO:tasks.workunit.client.0.smithi057.stdout:7/143: truncate d0/d8/d15/f1a 1136446 0 2024-02-28T23:07:28.975 INFO:tasks.workunit.client.1.smithi084.stdout:3/201: getdents d3/d8/d1a/d21/d3d/d43 0 2024-02-28T23:07:28.975 INFO:tasks.workunit.client.1.smithi084.stdout:6/239: write d1/f24 [2865396,35222] 0 2024-02-28T23:07:28.981 INFO:tasks.workunit.client.0.smithi057.stdout:4/137: dread d1a/d1b/f21 [0,4194304] 0 2024-02-28T23:07:28.982 INFO:tasks.workunit.client.1.smithi084.stdout:9/265: creat d4/d32/d1f/f58 x:0 0 0 2024-02-28T23:07:28.982 INFO:tasks.workunit.client.1.smithi084.stdout:1/316: fsync d12/d1b/d2e/f57 0 2024-02-28T23:07:28.984 INFO:tasks.workunit.client.1.smithi084.stdout:1/317: read d12/d1b/d2e/f57 [2396124,2629] 0 2024-02-28T23:07:28.986 INFO:tasks.workunit.client.1.smithi084.stdout:6/240: creat d1/d9/d1d/d1e/d33/f4c x:0 0 0 2024-02-28T23:07:28.988 INFO:tasks.workunit.client.1.smithi084.stdout:6/241: readlink d1/d13/d2e/l3a 0 2024-02-28T23:07:28.989 INFO:tasks.workunit.client.1.smithi084.stdout:3/202: symlink d3/d8/l4a 0 2024-02-28T23:07:28.995 INFO:tasks.workunit.client.1.smithi084.stdout:3/203: mkdir d3/d27/d4b 0 2024-02-28T23:07:28.998 INFO:tasks.workunit.client.1.smithi084.stdout:3/204: chown d3/f11 256 1 2024-02-28T23:07:28.998 INFO:tasks.workunit.client.1.smithi084.stdout:6/242: mkdir d1/d13/d2e/d41/d4d 0 2024-02-28T23:07:29.000 INFO:tasks.workunit.client.1.smithi084.stdout:1/318: unlink d12/d1b/d2e/d32/l3d 0 2024-02-28T23:07:29.004 INFO:tasks.workunit.client.1.smithi084.stdout:7/232: dwrite db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:29.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/243: write d1/f24 [227983,112806] 0 2024-02-28T23:07:29.004 INFO:tasks.workunit.client.1.smithi084.stdout:3/205: write d3/d8/d1a/f1f [325041,116552] 0 2024-02-28T23:07:29.005 INFO:tasks.workunit.client.1.smithi084.stdout:7/233: fsync db/d10/f23 0 2024-02-28T23:07:29.007 INFO:tasks.workunit.client.1.smithi084.stdout:3/206: fsync d3/d8/d1a/d29/f38 0 2024-02-28T23:07:29.007 INFO:tasks.workunit.client.1.smithi084.stdout:7/234: readlink db/d10/d17/d3b/l45 0 2024-02-28T23:07:29.012 INFO:tasks.workunit.client.1.smithi084.stdout:6/244: creat d1/d9/d1d/d1e/d33/d38/f4e x:0 0 0 2024-02-28T23:07:29.013 INFO:tasks.workunit.client.1.smithi084.stdout:5/259: truncate dc/de/df/d16/f29 1182481 0 2024-02-28T23:07:29.014 INFO:tasks.workunit.client.1.smithi084.stdout:6/245: read - d1/d13/d2e/f44 zero size 2024-02-28T23:07:29.016 INFO:tasks.workunit.client.1.smithi084.stdout:1/319: creat d12/f70 x:0 0 0 2024-02-28T23:07:29.017 INFO:tasks.workunit.client.1.smithi084.stdout:3/207: mknod d3/d8/c4c 0 2024-02-28T23:07:29.020 INFO:tasks.workunit.client.1.smithi084.stdout:1/320: fsync d12/d1b/d63/f2a 0 2024-02-28T23:07:29.021 INFO:tasks.workunit.client.1.smithi084.stdout:3/208: dread - d3/d8/d14/f2e zero size 2024-02-28T23:07:29.022 INFO:tasks.workunit.client.1.smithi084.stdout:6/246: creat d1/d9/f4f x:0 0 0 2024-02-28T23:07:29.023 INFO:tasks.workunit.client.1.smithi084.stdout:9/266: dwrite d4/d6/d15/f2c [4194304,4194304] 0 2024-02-28T23:07:29.025 INFO:tasks.workunit.client.1.smithi084.stdout:1/321: symlink d12/d1b/d3b/l71 0 2024-02-28T23:07:29.026 INFO:tasks.workunit.client.1.smithi084.stdout:3/209: write d3/f28 [1280729,98433] 0 2024-02-28T23:07:29.030 INFO:tasks.workunit.client.1.smithi084.stdout:7/235: dwrite db/d10/d18/f47 [0,4194304] 0 2024-02-28T23:07:29.030 INFO:tasks.workunit.client.1.smithi084.stdout:6/247: mkdir d1/d13/d50 0 2024-02-28T23:07:29.031 INFO:tasks.workunit.client.1.smithi084.stdout:9/267: unlink d4/c30 0 2024-02-28T23:07:29.033 INFO:tasks.workunit.client.1.smithi084.stdout:7/236: chown db/d11/d27/f33 49887 1 2024-02-28T23:07:29.035 INFO:tasks.workunit.client.1.smithi084.stdout:6/248: mknod d1/d9/d1d/c51 0 2024-02-28T23:07:29.036 INFO:tasks.workunit.client.1.smithi084.stdout:1/322: creat d12/d1b/d2e/d52/d61/f72 x:0 0 0 2024-02-28T23:07:29.040 INFO:tasks.workunit.client.1.smithi084.stdout:3/210: rename d3/d8/d1a/f30 to d3/d27/d4b/f4d 0 2024-02-28T23:07:29.042 INFO:tasks.workunit.client.1.smithi084.stdout:6/249: rmdir d1/d9/d1d 39 2024-02-28T23:07:29.050 INFO:tasks.workunit.client.1.smithi084.stdout:3/211: creat d3/d8/d1a/d21/d3d/f4e x:0 0 0 2024-02-28T23:07:29.051 INFO:tasks.workunit.client.1.smithi084.stdout:6/250: mknod d1/d9/c52 0 2024-02-28T23:07:29.052 INFO:tasks.workunit.client.1.smithi084.stdout:5/260: write f7 [2286116,61288] 0 2024-02-28T23:07:29.054 INFO:tasks.workunit.client.1.smithi084.stdout:5/261: fsync dc/f48 0 2024-02-28T23:07:29.055 INFO:tasks.workunit.client.1.smithi084.stdout:7/237: rename db/d10/d17/l24 to db/l4a 0 2024-02-28T23:07:29.060 INFO:tasks.workunit.client.1.smithi084.stdout:1/323: dwrite d12/d1b/f31 [4194304,4194304] 0 2024-02-28T23:07:29.062 INFO:tasks.workunit.client.1.smithi084.stdout:9/268: rename d4/d32/d1f/f58 to d4/d6/d15/d12/d41/f59 0 2024-02-28T23:07:29.064 INFO:tasks.workunit.client.1.smithi084.stdout:1/324: read - d12/d1b/d2e/d30/f3e zero size 2024-02-28T23:07:29.066 INFO:tasks.workunit.client.1.smithi084.stdout:6/251: mkdir d1/d9/d1d/d1e/d33/d38/d53 0 2024-02-28T23:07:29.068 INFO:tasks.workunit.client.1.smithi084.stdout:6/252: fdatasync d1/d9/f40 0 2024-02-28T23:07:29.074 INFO:tasks.workunit.client.0.smithi057.stdout:8/106: write d4/f14 [53631,33335] 0 2024-02-28T23:07:29.074 INFO:tasks.workunit.client.1.smithi084.stdout:1/325: creat d12/d1b/d2e/d52/f73 x:0 0 0 2024-02-28T23:07:29.075 INFO:tasks.workunit.client.1.smithi084.stdout:0/203: sync 2024-02-28T23:07:29.075 INFO:tasks.workunit.client.0.smithi057.stdout:0/106: mkdir d17/d20/d23 0 2024-02-28T23:07:29.078 INFO:tasks.workunit.client.0.smithi057.stdout:5/52: truncate f1 8406575 0 2024-02-28T23:07:29.078 INFO:tasks.workunit.client.1.smithi084.stdout:0/204: write d1/d8/f3c [141068,35007] 0 2024-02-28T23:07:29.080 INFO:tasks.workunit.client.1.smithi084.stdout:7/238: getdents db/d11/d27 0 2024-02-28T23:07:29.080 INFO:tasks.workunit.client.0.smithi057.stdout:7/144: mknod d0/d8/d15/d29/c2c 0 2024-02-28T23:07:29.081 INFO:tasks.workunit.client.1.smithi084.stdout:1/326: mkdir d12/d1b/d2e/d52/d61/d74 0 2024-02-28T23:07:29.082 INFO:tasks.workunit.client.1.smithi084.stdout:9/269: dwrite d4/d6/d15/d12/f3d [0,4194304] 0 2024-02-28T23:07:29.083 INFO:tasks.workunit.client.1.smithi084.stdout:3/212: rename d3/d8/d1a/f3e to d3/d8/f4f 0 2024-02-28T23:07:29.085 INFO:tasks.workunit.client.0.smithi057.stdout:0/107: rename d17/c22 to d17/d20/d23/c24 0 2024-02-28T23:07:29.086 INFO:tasks.workunit.client.0.smithi057.stdout:5/53: mkdir d4/db 0 2024-02-28T23:07:29.087 INFO:tasks.workunit.client.0.smithi057.stdout:7/145: rmdir d0 39 2024-02-28T23:07:29.090 INFO:tasks.workunit.client.1.smithi084.stdout:5/262: write dc/de/f14 [48710,126496] 0 2024-02-28T23:07:29.090 INFO:tasks.workunit.client.1.smithi084.stdout:8/207: sync 2024-02-28T23:07:29.090 INFO:tasks.workunit.client.1.smithi084.stdout:4/203: sync 2024-02-28T23:07:29.090 INFO:tasks.workunit.client.1.smithi084.stdout:2/206: sync 2024-02-28T23:07:29.090 INFO:tasks.workunit.client.1.smithi084.stdout:3/213: read d3/d8/d14/f1c [2442468,70179] 0 2024-02-28T23:07:29.097 INFO:tasks.workunit.client.1.smithi084.stdout:4/204: chown d1/d18/d20/d15/f28 1 1 2024-02-28T23:07:29.098 INFO:tasks.workunit.client.1.smithi084.stdout:5/263: fsync dc/de/df/d16/f57 0 2024-02-28T23:07:29.098 INFO:tasks.workunit.client.1.smithi084.stdout:1/327: mknod d12/d1b/d2e/d52/d61/c75 0 2024-02-28T23:07:29.099 INFO:tasks.workunit.client.0.smithi057.stdout:0/108: dread f4 [0,4194304] 0 2024-02-28T23:07:29.100 INFO:tasks.workunit.client.0.smithi057.stdout:4/138: getdents d1a/d2f 0 2024-02-28T23:07:29.100 INFO:tasks.workunit.client.0.smithi057.stdout:8/107: getdents d4/d13/d18 0 2024-02-28T23:07:29.101 INFO:tasks.workunit.client.1.smithi084.stdout:9/270: write d4/f51 [4409784,42248] 0 2024-02-28T23:07:29.101 INFO:tasks.workunit.client.1.smithi084.stdout:3/214: stat d3/d8/d14/d31/d34/f2b 0 2024-02-28T23:07:29.102 INFO:tasks.workunit.client.0.smithi057.stdout:4/139: chown d1a/d2f 1679 1 2024-02-28T23:07:29.102 INFO:tasks.workunit.client.1.smithi084.stdout:5/264: chown dc/f28 104 1 2024-02-28T23:07:29.103 INFO:tasks.workunit.client.0.smithi057.stdout:5/54: dwrite f2 [0,4194304] 0 2024-02-28T23:07:29.105 INFO:tasks.workunit.client.1.smithi084.stdout:6/253: truncate d1/d13/d2e/d41/f37 7120353 0 2024-02-28T23:07:29.106 INFO:tasks.workunit.client.1.smithi084.stdout:4/205: mknod d1/d18/d20/d15/c3a 0 2024-02-28T23:07:29.106 INFO:tasks.workunit.client.1.smithi084.stdout:0/205: write d1/d6/d2d/f30 [190257,32196] 0 2024-02-28T23:07:29.111 INFO:tasks.workunit.client.1.smithi084.stdout:4/206: chown d1/f8 240959 1 2024-02-28T23:07:29.111 INFO:tasks.workunit.client.1.smithi084.stdout:5/265: chown dc/de/df/d16/d34/c49 403928 1 2024-02-28T23:07:29.111 INFO:tasks.workunit.client.1.smithi084.stdout:6/254: write d1/d9/f40 [7273799,76479] 0 2024-02-28T23:07:29.112 INFO:tasks.workunit.client.1.smithi084.stdout:1/328: unlink d12/f16 0 2024-02-28T23:07:29.113 INFO:tasks.workunit.client.1.smithi084.stdout:3/215: mkdir d3/d50 0 2024-02-28T23:07:29.114 INFO:tasks.workunit.client.0.smithi057.stdout:4/140: creat d1a/d1b/d24/f38 x:0 0 0 2024-02-28T23:07:29.117 INFO:tasks.workunit.client.1.smithi084.stdout:5/266: write dc/f25 [264644,88626] 0 2024-02-28T23:07:29.120 INFO:tasks.workunit.client.1.smithi084.stdout:9/271: rmdir d4/d6/d15/d3c 39 2024-02-28T23:07:29.122 INFO:tasks.workunit.client.1.smithi084.stdout:0/206: creat d1/d8/f40 x:0 0 0 2024-02-28T23:07:29.122 INFO:tasks.workunit.client.0.smithi057.stdout:5/55: creat d4/d9/fc x:0 0 0 2024-02-28T23:07:29.123 INFO:tasks.workunit.client.1.smithi084.stdout:7/239: link db/d11/c14 db/d11/d27/d36/c4b 0 2024-02-28T23:07:29.124 INFO:tasks.workunit.client.1.smithi084.stdout:1/329: dread - d12/d1b/d2e/d30/f58 zero size 2024-02-28T23:07:29.125 INFO:tasks.workunit.client.0.smithi057.stdout:3/50: sync 2024-02-28T23:07:29.126 INFO:tasks.workunit.client.0.smithi057.stdout:8/108: dwrite d4/f14 [0,4194304] 0 2024-02-28T23:07:29.127 INFO:tasks.workunit.client.1.smithi084.stdout:6/255: creat d1/d9/d1d/d1e/d33/d38/f54 x:0 0 0 2024-02-28T23:07:29.128 INFO:tasks.workunit.client.1.smithi084.stdout:4/207: truncate d1/d18/d20/d14/d30/f38 911436 0 2024-02-28T23:07:29.129 INFO:tasks.workunit.client.0.smithi057.stdout:7/146: truncate d0/f23 1539309 0 2024-02-28T23:07:29.130 INFO:tasks.workunit.client.1.smithi084.stdout:3/216: write d3/d8/df/f19 [961516,12460] 0 2024-02-28T23:07:29.131 INFO:tasks.workunit.client.1.smithi084.stdout:5/267: write dc/de/df/d16/f4a [297229,29131] 0 2024-02-28T23:07:29.133 INFO:tasks.workunit.client.0.smithi057.stdout:8/109: readlink d4/l12 0 2024-02-28T23:07:29.135 INFO:tasks.workunit.client.0.smithi057.stdout:7/147: rename d0/d8 to d0/d8/d15/d2d 22 2024-02-28T23:07:29.137 INFO:tasks.workunit.client.0.smithi057.stdout:3/51: creat d3/d4/d7/f10 x:0 0 0 2024-02-28T23:07:29.138 INFO:tasks.workunit.client.1.smithi084.stdout:2/207: dwrite d5/f29 [0,4194304] 0 2024-02-28T23:07:29.140 INFO:tasks.workunit.client.1.smithi084.stdout:9/272: write d4/d6/d15/d12/d41/f59 [451946,111065] 0 2024-02-28T23:07:29.140 INFO:tasks.workunit.client.1.smithi084.stdout:4/208: readlink d1/d18/d20/d14/l22 0 2024-02-28T23:07:29.140 INFO:tasks.workunit.client.1.smithi084.stdout:0/207: unlink d1/d6/d2d/c33 0 2024-02-28T23:07:29.142 INFO:tasks.workunit.client.0.smithi057.stdout:8/110: creat d4/d13/d18/f27 x:0 0 0 2024-02-28T23:07:29.143 INFO:tasks.workunit.client.0.smithi057.stdout:7/148: creat d0/d5/f2e x:0 0 0 2024-02-28T23:07:29.144 INFO:tasks.workunit.client.1.smithi084.stdout:0/208: dread - d1/d8/f36 zero size 2024-02-28T23:07:29.145 INFO:tasks.workunit.client.0.smithi057.stdout:0/109: dwrite f8 [0,4194304] 0 2024-02-28T23:07:29.146 INFO:tasks.workunit.client.0.smithi057.stdout:4/141: dwrite d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:29.148 INFO:tasks.workunit.client.0.smithi057.stdout:8/111: fsync d4/f17 0 2024-02-28T23:07:29.149 INFO:tasks.workunit.client.1.smithi084.stdout:9/273: dread - d4/d6/d15/d12/d41/f46 zero size 2024-02-28T23:07:29.150 INFO:tasks.workunit.client.1.smithi084.stdout:7/240: creat db/d11/d27/d36/f4c x:0 0 0 2024-02-28T23:07:29.152 INFO:tasks.workunit.client.0.smithi057.stdout:8/112: readlink d4/l16 0 2024-02-28T23:07:29.155 INFO:tasks.workunit.client.1.smithi084.stdout:3/217: rmdir d3/d8/d14 39 2024-02-28T23:07:29.155 INFO:tasks.workunit.client.0.smithi057.stdout:3/52: link d3/d4/d7/cf d3/d4/c11 0 2024-02-28T23:07:29.156 INFO:tasks.workunit.client.0.smithi057.stdout:0/110: symlink d17/l25 0 2024-02-28T23:07:29.156 INFO:tasks.workunit.client.1.smithi084.stdout:8/208: dwrite d0/d9/fb [0,4194304] 0 2024-02-28T23:07:29.156 INFO:tasks.workunit.client.1.smithi084.stdout:9/274: write d4/d32/f31 [2589033,11077] 0 2024-02-28T23:07:29.161 INFO:tasks.workunit.client.1.smithi084.stdout:5/268: rename dc/c36 to dc/de/df/d16/d34/d54/c5c 0 2024-02-28T23:07:29.161 INFO:tasks.workunit.client.0.smithi057.stdout:4/142: creat d1a/d1b/d2d/f39 x:0 0 0 2024-02-28T23:07:29.162 INFO:tasks.workunit.client.0.smithi057.stdout:7/149: creat d0/d8/d15/f2f x:0 0 0 2024-02-28T23:07:29.163 INFO:tasks.workunit.client.0.smithi057.stdout:5/56: dwrite f1 [0,4194304] 0 2024-02-28T23:07:29.167 INFO:tasks.workunit.client.1.smithi084.stdout:8/209: chown d0/c7 3611023 1 2024-02-28T23:07:29.169 INFO:tasks.workunit.client.1.smithi084.stdout:5/269: dread - dc/de/df/d16/d34/d3c/f5a zero size 2024-02-28T23:07:29.172 INFO:tasks.workunit.client.0.smithi057.stdout:4/143: mknod d1a/d1e/c3a 0 2024-02-28T23:07:29.173 INFO:tasks.workunit.client.1.smithi084.stdout:0/209: fdatasync d1/d8/f20 0 2024-02-28T23:07:29.175 INFO:tasks.workunit.client.1.smithi084.stdout:7/241: creat db/d10/d17/d3b/f4d x:0 0 0 2024-02-28T23:07:29.175 INFO:tasks.workunit.client.1.smithi084.stdout:1/330: dwrite d12/d1b/d3b/f42 [0,4194304] 0 2024-02-28T23:07:29.176 INFO:tasks.workunit.client.0.smithi057.stdout:3/53: dread f1 [0,4194304] 0 2024-02-28T23:07:29.177 INFO:tasks.workunit.client.1.smithi084.stdout:3/218: mknod d3/d8/d1a/d21/d3d/d43/c51 0 2024-02-28T23:07:29.178 INFO:tasks.workunit.client.1.smithi084.stdout:2/208: dwrite d5/f17 [4194304,4194304] 0 2024-02-28T23:07:29.179 INFO:tasks.workunit.client.1.smithi084.stdout:0/210: chown d1/d6/db/l2b 117 1 2024-02-28T23:07:29.182 INFO:tasks.workunit.client.0.smithi057.stdout:5/57: dread d4/f6 [0,4194304] 0 2024-02-28T23:07:29.186 INFO:tasks.workunit.client.0.smithi057.stdout:7/150: creat d0/d5/f30 x:0 0 0 2024-02-28T23:07:29.187 INFO:tasks.workunit.client.0.smithi057.stdout:4/144: symlink d1a/d1b/d24/d2e/l3b 0 2024-02-28T23:07:29.188 INFO:tasks.workunit.client.1.smithi084.stdout:4/209: fdatasync d1/d18/d20/fe 0 2024-02-28T23:07:29.188 INFO:tasks.workunit.client.1.smithi084.stdout:8/210: creat d0/d9/d1b/f46 x:0 0 0 2024-02-28T23:07:29.190 INFO:tasks.workunit.client.0.smithi057.stdout:3/54: creat d3/d4/f12 x:0 0 0 2024-02-28T23:07:29.191 INFO:tasks.workunit.client.0.smithi057.stdout:3/55: fdatasync d3/fd 0 2024-02-28T23:07:29.194 INFO:tasks.workunit.client.0.smithi057.stdout:3/56: dread - f2 zero size 2024-02-28T23:07:29.195 INFO:tasks.workunit.client.0.smithi057.stdout:5/58: creat d4/fd x:0 0 0 2024-02-28T23:07:29.197 INFO:tasks.workunit.client.1.smithi084.stdout:7/242: symlink db/l4e 0 2024-02-28T23:07:29.197 INFO:tasks.workunit.client.0.smithi057.stdout:8/113: getdents d4/d13/d18 0 2024-02-28T23:07:29.201 INFO:tasks.workunit.client.0.smithi057.stdout:8/114: dread - d4/f5 zero size 2024-02-28T23:07:29.201 INFO:tasks.workunit.client.0.smithi057.stdout:5/59: chown d4/la 20951 1 2024-02-28T23:07:29.203 INFO:tasks.workunit.client.1.smithi084.stdout:1/331: read d12/d1b/d2e/d52/d5b/f6f [2540679,78717] 0 2024-02-28T23:07:29.208 INFO:tasks.workunit.client.0.smithi057.stdout:5/60: dread - d4/d9/fc zero size 2024-02-28T23:07:29.208 INFO:tasks.workunit.client.1.smithi084.stdout:2/209: read d5/dd/f1c [15991,91865] 0 2024-02-28T23:07:29.216 INFO:tasks.workunit.client.1.smithi084.stdout:5/270: dwrite dc/de/df/d16/d34/f4d [0,4194304] 0 2024-02-28T23:07:29.218 INFO:tasks.workunit.client.0.smithi057.stdout:7/151: dwrite d0/d5/f2e [0,4194304] 0 2024-02-28T23:07:29.220 INFO:tasks.workunit.client.0.smithi057.stdout:4/145: dwrite d1a/f36 [0,4194304] 0 2024-02-28T23:07:29.220 INFO:tasks.workunit.client.1.smithi084.stdout:2/210: fdatasync d5/f31 0 2024-02-28T23:07:29.222 INFO:tasks.workunit.client.1.smithi084.stdout:9/275: dwrite d4/d6/d15/d3c/f47 [4194304,4194304] 0 2024-02-28T23:07:29.224 INFO:tasks.workunit.client.1.smithi084.stdout:1/332: chown d12/d1b/d63/f3a 0 1 2024-02-28T23:07:29.225 INFO:tasks.workunit.client.1.smithi084.stdout:8/211: rename d0/d24/c2c to d0/d9/c47 0 2024-02-28T23:07:29.228 INFO:tasks.workunit.client.1.smithi084.stdout:6/256: truncate d1/d13/d2e/d41/f37 6046564 0 2024-02-28T23:07:29.229 INFO:tasks.workunit.client.1.smithi084.stdout:0/211: dwrite d1/d8/f29 [0,4194304] 0 2024-02-28T23:07:29.232 INFO:tasks.workunit.client.0.smithi057.stdout:4/146: creat d1a/d1b/d2d/f3c x:0 0 0 2024-02-28T23:07:29.233 INFO:tasks.workunit.client.1.smithi084.stdout:5/271: fsync dc/f15 0 2024-02-28T23:07:29.233 INFO:tasks.workunit.client.1.smithi084.stdout:1/333: fsync d12/d1b/d3b/f59 0 2024-02-28T23:07:29.236 INFO:tasks.workunit.client.1.smithi084.stdout:8/212: stat d0/d9/d30/l31 0 2024-02-28T23:07:29.237 INFO:tasks.workunit.client.0.smithi057.stdout:8/115: link d4/d13/d1a/c1d d4/d13/d18/c28 0 2024-02-28T23:07:29.240 INFO:tasks.workunit.client.0.smithi057.stdout:7/152: truncate d0/d8/d15/f2f 787668 0 2024-02-28T23:07:29.242 INFO:tasks.workunit.client.0.smithi057.stdout:3/57: dwrite d3/fd [0,4194304] 0 2024-02-28T23:07:29.242 INFO:tasks.workunit.client.0.smithi057.stdout:8/116: fsync d4/d13/d1a/f20 0 2024-02-28T23:07:29.244 INFO:tasks.workunit.client.0.smithi057.stdout:4/147: creat d1a/d1b/d24/f3d x:0 0 0 2024-02-28T23:07:29.247 INFO:tasks.workunit.client.0.smithi057.stdout:8/117: readlink d4/l16 0 2024-02-28T23:07:29.247 INFO:tasks.workunit.client.1.smithi084.stdout:4/210: dwrite d1/d18/d20/d14/d30/f38 [0,4194304] 0 2024-02-28T23:07:29.247 INFO:tasks.workunit.client.0.smithi057.stdout:7/153: fsync d0/d5/d25/f28 0 2024-02-28T23:07:29.247 INFO:tasks.workunit.client.1.smithi084.stdout:0/212: mknod d1/d8/d18/c41 0 2024-02-28T23:07:29.250 INFO:tasks.workunit.client.0.smithi057.stdout:5/61: dwrite f2 [0,4194304] 0 2024-02-28T23:07:29.251 INFO:tasks.workunit.client.1.smithi084.stdout:1/334: write d12/d1b/d2e/d30/f46 [597421,66384] 0 2024-02-28T23:07:29.253 INFO:tasks.workunit.client.0.smithi057.stdout:3/58: symlink d3/d4/l13 0 2024-02-28T23:07:29.253 INFO:tasks.workunit.client.0.smithi057.stdout:4/148: creat d1a/d1b/f3e x:0 0 0 2024-02-28T23:07:29.257 INFO:tasks.workunit.client.1.smithi084.stdout:2/211: rename d5/d1f/f2b to d5/dd/f34 0 2024-02-28T23:07:29.259 INFO:tasks.workunit.client.0.smithi057.stdout:4/149: creat d1a/d1b/d24/d2e/f3f x:0 0 0 2024-02-28T23:07:29.262 INFO:tasks.workunit.client.0.smithi057.stdout:4/150: truncate d1a/d1b/d2d/f3c 352982 0 2024-02-28T23:07:29.263 INFO:tasks.workunit.client.1.smithi084.stdout:5/272: creat dc/de/df/d16/d34/d54/f5d x:0 0 0 2024-02-28T23:07:29.264 INFO:tasks.workunit.client.0.smithi057.stdout:3/59: getdents d3 0 2024-02-28T23:07:29.265 INFO:tasks.workunit.client.1.smithi084.stdout:7/243: dwrite db/d10/d17/d3b/f4d [0,4194304] 0 2024-02-28T23:07:29.268 INFO:tasks.workunit.client.0.smithi057.stdout:8/118: dread d4/f7 [0,4194304] 0 2024-02-28T23:07:29.269 INFO:tasks.workunit.client.0.smithi057.stdout:3/60: getdents d3 0 2024-02-28T23:07:29.270 INFO:tasks.workunit.client.0.smithi057.stdout:3/61: chown d3 1933 1 2024-02-28T23:07:29.270 INFO:tasks.workunit.client.0.smithi057.stdout:8/119: read - d4/d13/d18/f27 zero size 2024-02-28T23:07:29.273 INFO:tasks.workunit.client.0.smithi057.stdout:8/120: truncate d4/f5 219925 0 2024-02-28T23:07:29.276 INFO:tasks.workunit.client.1.smithi084.stdout:7/244: write db/d10/f3e [961899,45787] 0 2024-02-28T23:07:29.276 INFO:tasks.workunit.client.0.smithi057.stdout:8/121: write d4/d13/d1a/f20 [717930,87677] 0 2024-02-28T23:07:29.278 INFO:tasks.workunit.client.1.smithi084.stdout:1/335: creat d12/d1b/d63/f76 x:0 0 0 2024-02-28T23:07:29.279 INFO:tasks.workunit.client.0.smithi057.stdout:4/151: dread f14 [0,4194304] 0 2024-02-28T23:07:29.279 INFO:tasks.workunit.client.0.smithi057.stdout:8/122: mknod d4/d1e/c29 0 2024-02-28T23:07:29.281 INFO:tasks.workunit.client.1.smithi084.stdout:0/213: rename d1/d8/d18/c41 to d1/d6/db/c42 0 2024-02-28T23:07:29.282 INFO:tasks.workunit.client.0.smithi057.stdout:3/62: dread d3/d8/fb [0,4194304] 0 2024-02-28T23:07:29.284 INFO:tasks.workunit.client.0.smithi057.stdout:4/152: mkdir d1a/d2f/d40 0 2024-02-28T23:07:29.284 INFO:tasks.workunit.client.1.smithi084.stdout:2/212: mkdir d5/d7/d12/d24/d2a/d35 0 2024-02-28T23:07:29.285 INFO:tasks.workunit.client.0.smithi057.stdout:3/63: readlink d3/d4/l13 0 2024-02-28T23:07:29.286 INFO:tasks.workunit.client.1.smithi084.stdout:4/211: dwrite d1/d18/d20/d15/f16 [0,4194304] 0 2024-02-28T23:07:29.286 INFO:tasks.workunit.client.1.smithi084.stdout:3/219: dread d3/d8/d14/f1c [0,4194304] 0 2024-02-28T23:07:29.286 INFO:tasks.workunit.client.1.smithi084.stdout:5/273: symlink dc/de/df/d16/l5e 0 2024-02-28T23:07:29.289 INFO:tasks.workunit.client.1.smithi084.stdout:4/212: stat d1/d18/d20/fe 0 2024-02-28T23:07:29.290 INFO:tasks.workunit.client.0.smithi057.stdout:3/64: dread - d3/d4/d7/f10 zero size 2024-02-28T23:07:29.291 INFO:tasks.workunit.client.0.smithi057.stdout:8/123: getdents d4/d13/d1a 0 2024-02-28T23:07:29.292 INFO:tasks.workunit.client.1.smithi084.stdout:0/214: read - d1/d8/f3e zero size 2024-02-28T23:07:29.294 INFO:tasks.workunit.client.1.smithi084.stdout:3/220: stat d3/d8/d1a/d29/d41 0 2024-02-28T23:07:29.294 INFO:tasks.workunit.client.1.smithi084.stdout:0/215: fdatasync d1/d6/f19 0 2024-02-28T23:07:29.295 INFO:tasks.workunit.client.1.smithi084.stdout:1/336: mkdir d12/d1b/d2e/d30/d77 0 2024-02-28T23:07:29.296 INFO:tasks.workunit.client.0.smithi057.stdout:8/124: mknod d4/d13/d1a/c2a 0 2024-02-28T23:07:29.297 INFO:tasks.workunit.client.0.smithi057.stdout:4/153: dread d1a/d1b/f2c [0,4194304] 0 2024-02-28T23:07:29.298 INFO:tasks.workunit.client.1.smithi084.stdout:9/276: dwrite d4/f51 [4194304,4194304] 0 2024-02-28T23:07:29.300 INFO:tasks.workunit.client.1.smithi084.stdout:2/213: fdatasync d5/dd/f1c 0 2024-02-28T23:07:29.300 INFO:tasks.workunit.client.0.smithi057.stdout:8/125: rename d4/f14 to d4/d13/f2b 0 2024-02-28T23:07:29.303 INFO:tasks.workunit.client.1.smithi084.stdout:1/337: write d12/f34 [557820,39272] 0 2024-02-28T23:07:29.305 INFO:tasks.workunit.client.0.smithi057.stdout:8/126: mkdir d4/d13/d2c 0 2024-02-28T23:07:29.307 INFO:tasks.workunit.client.1.smithi084.stdout:1/338: write f11 [3839951,127730] 0 2024-02-28T23:07:29.312 INFO:tasks.workunit.client.1.smithi084.stdout:5/274: mkdir dc/de/df/d16/d34/d54/d5f 0 2024-02-28T23:07:29.314 INFO:tasks.workunit.client.0.smithi057.stdout:8/127: stat d4/d13/d18/f23 0 2024-02-28T23:07:29.315 INFO:tasks.workunit.client.1.smithi084.stdout:7/245: dread db/d11/f12 [0,4194304] 0 2024-02-28T23:07:29.318 INFO:tasks.workunit.client.0.smithi057.stdout:9/98: sync 2024-02-28T23:07:29.319 INFO:tasks.workunit.client.0.smithi057.stdout:8/128: write d4/f17 [4907406,52266] 0 2024-02-28T23:07:29.320 INFO:tasks.workunit.client.1.smithi084.stdout:4/213: dread d1/d18/d20/d14/d30/f38 [0,4194304] 0 2024-02-28T23:07:29.320 INFO:tasks.workunit.client.0.smithi057.stdout:3/65: dwrite d3/d4/d7/f10 [0,4194304] 0 2024-02-28T23:07:29.320 INFO:tasks.workunit.client.0.smithi057.stdout:7/154: write d0/d5/f27 [167849,85161] 0 2024-02-28T23:07:29.322 INFO:tasks.workunit.client.1.smithi084.stdout:2/214: symlink d5/d1f/l36 0 2024-02-28T23:07:29.324 INFO:tasks.workunit.client.1.smithi084.stdout:3/221: dwrite d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:29.325 INFO:tasks.workunit.client.0.smithi057.stdout:9/99: creat dd/de/d14/f1f x:0 0 0 2024-02-28T23:07:29.326 INFO:tasks.workunit.client.0.smithi057.stdout:3/66: dread - d3/d4/f12 zero size 2024-02-28T23:07:29.332 INFO:tasks.workunit.client.0.smithi057.stdout:5/62: fdatasync f2 0 2024-02-28T23:07:29.334 INFO:tasks.workunit.client.0.smithi057.stdout:4/154: dwrite d1a/d1e/f29 [0,4194304] 0 2024-02-28T23:07:29.334 INFO:tasks.workunit.client.0.smithi057.stdout:9/100: write dd/de/d14/f1f [836329,92429] 0 2024-02-28T23:07:29.337 INFO:tasks.workunit.client.0.smithi057.stdout:9/101: chown l8 38008954 1 2024-02-28T23:07:29.338 INFO:tasks.workunit.client.0.smithi057.stdout:5/63: creat d4/d7/fe x:0 0 0 2024-02-28T23:07:29.338 INFO:tasks.workunit.client.1.smithi084.stdout:4/214: rename d1/d18/d20/fe to d1/f3b 0 2024-02-28T23:07:29.345 INFO:tasks.workunit.client.0.smithi057.stdout:5/64: stat d4/la 0 2024-02-28T23:07:29.345 INFO:tasks.workunit.client.1.smithi084.stdout:4/215: mknod d1/d18/d20/d14/d30/c3c 0 2024-02-28T23:07:29.345 INFO:tasks.workunit.client.1.smithi084.stdout:9/277: truncate d4/d6/ff 616244 0 2024-02-28T23:07:29.349 INFO:tasks.workunit.client.1.smithi084.stdout:4/216: creat d1/d18/d20/d14/d30/f3d x:0 0 0 2024-02-28T23:07:29.351 INFO:tasks.workunit.client.0.smithi057.stdout:7/155: dread d0/d5/f1e [0,4194304] 0 2024-02-28T23:07:29.353 INFO:tasks.workunit.client.0.smithi057.stdout:1/38: sync 2024-02-28T23:07:29.353 INFO:tasks.workunit.client.0.smithi057.stdout:2/150: sync 2024-02-28T23:07:29.353 INFO:tasks.workunit.client.0.smithi057.stdout:6/101: sync 2024-02-28T23:07:29.353 INFO:tasks.workunit.client.0.smithi057.stdout:0/111: sync 2024-02-28T23:07:29.357 INFO:tasks.workunit.client.0.smithi057.stdout:3/67: dread f1 [0,4194304] 0 2024-02-28T23:07:29.358 INFO:tasks.workunit.client.0.smithi057.stdout:4/155: getdents d1a/d1b/d2d 0 2024-02-28T23:07:29.359 INFO:tasks.workunit.client.1.smithi084.stdout:1/339: dwrite d12/d1b/d2e/d30/f1d [0,4194304] 0 2024-02-28T23:07:29.361 INFO:tasks.workunit.client.0.smithi057.stdout:5/65: creat d4/d7/d8/ff x:0 0 0 2024-02-28T23:07:29.362 INFO:tasks.workunit.client.0.smithi057.stdout:4/156: chown c6 107 1 2024-02-28T23:07:29.362 INFO:tasks.workunit.client.1.smithi084.stdout:9/278: dread d4/f51 [4194304,4194304] 0 2024-02-28T23:07:29.363 INFO:tasks.workunit.client.1.smithi084.stdout:0/216: truncate d1/d8/f23 2096312 0 2024-02-28T23:07:29.364 INFO:tasks.workunit.client.0.smithi057.stdout:0/112: mkdir d17/d20/d26 0 2024-02-28T23:07:29.365 INFO:tasks.workunit.client.0.smithi057.stdout:7/156: rmdir d0/d5 39 2024-02-28T23:07:29.366 INFO:tasks.workunit.client.1.smithi084.stdout:6/257: dread d1/d9/f40 [4194304,4194304] 0 2024-02-28T23:07:29.367 INFO:tasks.workunit.client.1.smithi084.stdout:3/222: write d3/f7 [1970928,86001] 0 2024-02-28T23:07:29.369 INFO:tasks.workunit.client.0.smithi057.stdout:9/102: dwrite dd/de/f1d [0,4194304] 0 2024-02-28T23:07:29.370 INFO:tasks.workunit.client.1.smithi084.stdout:0/217: fsync d1/d8/f3c 0 2024-02-28T23:07:29.370 INFO:tasks.workunit.client.0.smithi057.stdout:3/68: rename d3/d8 to d3/d8/d14 22 2024-02-28T23:07:29.371 INFO:tasks.workunit.client.0.smithi057.stdout:8/129: truncate d4/f17 4663492 0 2024-02-28T23:07:29.373 INFO:tasks.workunit.client.0.smithi057.stdout:4/157: unlink d1a/c1d 0 2024-02-28T23:07:29.373 INFO:tasks.workunit.client.1.smithi084.stdout:1/340: rmdir d12/d1b/d2e/d30 39 2024-02-28T23:07:29.376 INFO:tasks.workunit.client.1.smithi084.stdout:6/258: truncate d1/d13/d2e/f44 88554 0 2024-02-28T23:07:29.380 INFO:tasks.workunit.client.0.smithi057.stdout:8/130: write d4/d13/d18/f27 [936102,55914] 0 2024-02-28T23:07:29.380 INFO:tasks.workunit.client.0.smithi057.stdout:3/69: truncate d3/d8/fb 431853 0 2024-02-28T23:07:29.381 INFO:tasks.workunit.client.0.smithi057.stdout:6/102: link d0/fc d0/f1c 0 2024-02-28T23:07:29.383 INFO:tasks.workunit.client.1.smithi084.stdout:9/279: rename d4/d32/d1f to d4/d32/d3f/d5a 0 2024-02-28T23:07:29.383 INFO:tasks.workunit.client.1.smithi084.stdout:6/259: fdatasync d1/d9/f40 0 2024-02-28T23:07:29.384 INFO:tasks.workunit.client.0.smithi057.stdout:5/66: link d4/d7/fe d4/db/f10 0 2024-02-28T23:07:29.384 INFO:tasks.workunit.client.0.smithi057.stdout:7/157: dread d0/d8/d15/f2f [0,4194304] 0 2024-02-28T23:07:29.385 INFO:tasks.workunit.client.1.smithi084.stdout:4/217: dwrite d1/f8 [4194304,4194304] 0 2024-02-28T23:07:29.386 INFO:tasks.workunit.client.0.smithi057.stdout:9/103: symlink dd/d1b/l20 0 2024-02-28T23:07:29.388 INFO:tasks.workunit.client.1.smithi084.stdout:2/215: dread f3 [4194304,4194304] 0 2024-02-28T23:07:29.388 INFO:tasks.workunit.client.1.smithi084.stdout:6/260: write d1/d13/d2e/f44 [436587,15416] 0 2024-02-28T23:07:29.388 INFO:tasks.workunit.client.0.smithi057.stdout:1/39: dwrite d2/f8 [0,4194304] 0 2024-02-28T23:07:29.389 INFO:tasks.workunit.client.0.smithi057.stdout:9/104: write dd/ff [229790,26082] 0 2024-02-28T23:07:29.390 INFO:tasks.workunit.client.1.smithi084.stdout:9/280: unlink d4/d32/d3f/d5a/l24 0 2024-02-28T23:07:29.390 INFO:tasks.workunit.client.1.smithi084.stdout:4/218: write d1/f8 [3471960,118542] 0 2024-02-28T23:07:29.391 INFO:tasks.workunit.client.0.smithi057.stdout:8/131: creat d4/d13/f2d x:0 0 0 2024-02-28T23:07:29.391 INFO:tasks.workunit.client.0.smithi057.stdout:6/103: creat d0/d17/f1d x:0 0 0 2024-02-28T23:07:29.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/151: dread d8/f19 [0,4194304] 0 2024-02-28T23:07:29.394 INFO:tasks.workunit.client.0.smithi057.stdout:5/67: symlink d4/d7/l11 0 2024-02-28T23:07:29.395 INFO:tasks.workunit.client.0.smithi057.stdout:1/40: write d2/f8 [624376,85343] 0 2024-02-28T23:07:29.395 INFO:tasks.workunit.client.0.smithi057.stdout:1/41: readlink - no filename 2024-02-28T23:07:29.396 INFO:tasks.workunit.client.1.smithi084.stdout:2/216: rmdir d5/d7/d9 39 2024-02-28T23:07:29.396 INFO:tasks.workunit.client.0.smithi057.stdout:7/158: rmdir d0/d5/d25 39 2024-02-28T23:07:29.397 INFO:tasks.workunit.client.0.smithi057.stdout:5/68: dread - d4/d7/fe zero size 2024-02-28T23:07:29.397 INFO:tasks.workunit.client.0.smithi057.stdout:4/158: rmdir d1a/d2f/d40 0 2024-02-28T23:07:29.399 INFO:tasks.workunit.client.0.smithi057.stdout:7/159: dread - d0/d5/f30 zero size 2024-02-28T23:07:29.399 INFO:tasks.workunit.client.1.smithi084.stdout:6/261: creat d1/d13/d50/f55 x:0 0 0 2024-02-28T23:07:29.401 INFO:tasks.workunit.client.0.smithi057.stdout:8/132: creat d4/d1e/f2e x:0 0 0 2024-02-28T23:07:29.401 INFO:tasks.workunit.client.0.smithi057.stdout:6/104: rmdir d0/d4/d11 39 2024-02-28T23:07:29.401 INFO:tasks.workunit.client.0.smithi057.stdout:5/69: fdatasync d4/d9/fc 0 2024-02-28T23:07:29.401 INFO:tasks.workunit.client.1.smithi084.stdout:2/217: mknod d5/c37 0 2024-02-28T23:07:29.404 INFO:tasks.workunit.client.0.smithi057.stdout:2/152: creat d8/d11/d21/d28/f3d x:0 0 0 2024-02-28T23:07:29.405 INFO:tasks.workunit.client.0.smithi057.stdout:4/159: mknod d1a/d1b/d24/d2e/c41 0 2024-02-28T23:07:29.405 INFO:tasks.workunit.client.0.smithi057.stdout:6/105: dread - d0/d17/f1d zero size 2024-02-28T23:07:29.406 INFO:tasks.workunit.client.0.smithi057.stdout:3/70: getdents d3 0 2024-02-28T23:07:29.409 INFO:tasks.workunit.client.1.smithi084.stdout:1/341: dwrite d12/d1b/d63/f76 [0,4194304] 0 2024-02-28T23:07:29.409 INFO:tasks.workunit.client.0.smithi057.stdout:5/70: mkdir d4/db/d12 0 2024-02-28T23:07:29.410 INFO:tasks.workunit.client.0.smithi057.stdout:3/71: stat d3/d4/d7/f10 0 2024-02-28T23:07:29.414 INFO:tasks.workunit.client.1.smithi084.stdout:6/262: creat d1/d13/d47/f56 x:0 0 0 2024-02-28T23:07:29.419 INFO:tasks.workunit.client.1.smithi084.stdout:9/281: dwrite d4/d6/f36 [0,4194304] 0 2024-02-28T23:07:29.419 INFO:tasks.workunit.client.0.smithi057.stdout:8/133: getdents d4/d13/d2c 0 2024-02-28T23:07:29.422 INFO:tasks.workunit.client.0.smithi057.stdout:0/113: write f15 [904403,125862] 0 2024-02-28T23:07:29.423 INFO:tasks.workunit.client.1.smithi084.stdout:3/223: write d3/f6 [2764499,5004] 0 2024-02-28T23:07:29.424 INFO:tasks.workunit.client.0.smithi057.stdout:8/134: fdatasync d4/d13/d18/f27 0 2024-02-28T23:07:29.424 INFO:tasks.workunit.client.0.smithi057.stdout:3/72: mknod d3/d8/c15 0 2024-02-28T23:07:29.424 INFO:tasks.workunit.client.0.smithi057.stdout:1/42: dread f0 [0,4194304] 0 2024-02-28T23:07:29.426 INFO:tasks.workunit.client.0.smithi057.stdout:9/105: dwrite dd/de/d14/f1e [0,4194304] 0 2024-02-28T23:07:29.433 INFO:tasks.workunit.client.1.smithi084.stdout:9/282: chown d4/d32/d3f/d5a/d2d/f34 4055166 1 2024-02-28T23:07:29.434 INFO:tasks.workunit.client.0.smithi057.stdout:1/43: fsync f0 0 2024-02-28T23:07:29.434 INFO:tasks.workunit.client.0.smithi057.stdout:2/153: dread d8/f19 [0,4194304] 0 2024-02-28T23:07:29.434 INFO:tasks.workunit.client.1.smithi084.stdout:2/218: creat d5/d7/d12/d24/f38 x:0 0 0 2024-02-28T23:07:29.435 INFO:tasks.workunit.client.0.smithi057.stdout:9/106: write dd/ff [4453832,85651] 0 2024-02-28T23:07:29.435 INFO:tasks.workunit.client.1.smithi084.stdout:6/263: rmdir d1/d9/d1d/d1e/d33/d38/d53 0 2024-02-28T23:07:29.435 INFO:tasks.workunit.client.0.smithi057.stdout:2/154: chown d8 230026 1 2024-02-28T23:07:29.436 INFO:tasks.workunit.client.0.smithi057.stdout:5/71: link d4/d7/d8/ff d4/d7/d8/f13 0 2024-02-28T23:07:29.436 INFO:tasks.workunit.client.1.smithi084.stdout:2/219: chown d5/dd/f16 1879553 1 2024-02-28T23:07:29.439 INFO:tasks.workunit.client.0.smithi057.stdout:0/114: link f15 d17/d20/f27 0 2024-02-28T23:07:29.439 INFO:tasks.workunit.client.1.smithi084.stdout:2/220: truncate d5/f31 502540 0 2024-02-28T23:07:29.439 INFO:tasks.workunit.client.1.smithi084.stdout:4/219: dread d1/f7 [0,4194304] 0 2024-02-28T23:07:29.443 INFO:tasks.workunit.client.0.smithi057.stdout:6/106: getdents d0 0 2024-02-28T23:07:29.444 INFO:tasks.workunit.client.1.smithi084.stdout:6/264: symlink d1/d9/d1d/d1e/d33/d38/l57 0 2024-02-28T23:07:29.445 INFO:tasks.workunit.client.0.smithi057.stdout:3/73: rename f2 to d3/d8/f16 0 2024-02-28T23:07:29.446 INFO:tasks.workunit.client.0.smithi057.stdout:8/135: symlink d4/d13/d2c/l2f 0 2024-02-28T23:07:29.447 INFO:tasks.workunit.client.0.smithi057.stdout:4/160: dwrite f15 [4194304,4194304] 0 2024-02-28T23:07:29.451 INFO:tasks.workunit.client.0.smithi057.stdout:1/44: rmdir d2/d4 39 2024-02-28T23:07:29.452 INFO:tasks.workunit.client.0.smithi057.stdout:3/74: truncate d3/d8/f9 364054 0 2024-02-28T23:07:29.453 INFO:tasks.workunit.client.0.smithi057.stdout:8/136: truncate d4/d13/f2d 953286 0 2024-02-28T23:07:29.454 INFO:tasks.workunit.client.0.smithi057.stdout:3/75: chown d3 25925380 1 2024-02-28T23:07:29.455 INFO:tasks.workunit.client.0.smithi057.stdout:4/161: fdatasync d1a/d1b/d2d/f3c 0 2024-02-28T23:07:29.455 INFO:tasks.workunit.client.1.smithi084.stdout:4/220: mkdir d1/d18/d20/d24/d2f/d3e 0 2024-02-28T23:07:29.455 INFO:tasks.workunit.client.0.smithi057.stdout:0/115: write f4 [6589060,70200] 0 2024-02-28T23:07:29.457 INFO:tasks.workunit.client.1.smithi084.stdout:9/283: creat d4/d6/d15/d12/d41/d55/f5b x:0 0 0 2024-02-28T23:07:29.459 INFO:tasks.workunit.client.0.smithi057.stdout:3/76: stat d3/d4/d7/f10 0 2024-02-28T23:07:29.461 INFO:tasks.workunit.client.0.smithi057.stdout:4/162: dread - d1a/d1b/d2d/f39 zero size 2024-02-28T23:07:29.461 INFO:tasks.workunit.client.1.smithi084.stdout:6/265: mknod d1/d13/d2e/d41/c58 0 2024-02-28T23:07:29.462 INFO:tasks.workunit.client.1.smithi084.stdout:9/284: read d4/d6/f17 [3415020,49107] 0 2024-02-28T23:07:29.462 INFO:tasks.workunit.client.1.smithi084.stdout:2/221: dwrite d5/d7/d12/d24/f38 [0,4194304] 0 2024-02-28T23:07:29.463 INFO:tasks.workunit.client.0.smithi057.stdout:9/107: unlink dd/de/l16 0 2024-02-28T23:07:29.463 INFO:tasks.workunit.client.0.smithi057.stdout:3/77: dread - d3/d8/f16 zero size 2024-02-28T23:07:29.466 INFO:tasks.workunit.client.0.smithi057.stdout:5/72: write d4/db/f10 [1043357,107897] 0 2024-02-28T23:07:29.468 INFO:tasks.workunit.client.0.smithi057.stdout:8/137: mknod d4/d13/c30 0 2024-02-28T23:07:29.468 INFO:tasks.workunit.client.0.smithi057.stdout:4/163: creat d1a/d1e/f42 x:0 0 0 2024-02-28T23:07:29.469 INFO:tasks.workunit.client.1.smithi084.stdout:3/224: dwrite d3/d8/fa [0,4194304] 0 2024-02-28T23:07:29.470 INFO:tasks.workunit.client.1.smithi084.stdout:4/221: mknod d1/d18/d20/d24/d2f/c3f 0 2024-02-28T23:07:29.470 INFO:tasks.workunit.client.0.smithi057.stdout:1/45: mknod d2/d4/c9 0 2024-02-28T23:07:29.472 INFO:tasks.workunit.client.0.smithi057.stdout:0/116: write f15 [1024343,74956] 0 2024-02-28T23:07:29.473 INFO:tasks.workunit.client.0.smithi057.stdout:9/108: creat dd/de/d14/f21 x:0 0 0 2024-02-28T23:07:29.474 INFO:tasks.workunit.client.1.smithi084.stdout:4/222: rename d1/d18/d20 to d1/d18/d20/d24/d2f/d35/d37/d40 22 2024-02-28T23:07:29.476 INFO:tasks.workunit.client.0.smithi057.stdout:3/78: creat d3/d4/f17 x:0 0 0 2024-02-28T23:07:29.480 INFO:tasks.workunit.client.0.smithi057.stdout:8/138: creat d4/d13/d18/f31 x:0 0 0 2024-02-28T23:07:29.481 INFO:tasks.workunit.client.1.smithi084.stdout:2/222: dread - d5/d7/d9/f32 zero size 2024-02-28T23:07:29.483 INFO:tasks.workunit.client.0.smithi057.stdout:6/107: link d0/d4/l15 d0/d4/d11/l1e 0 2024-02-28T23:07:29.483 INFO:tasks.workunit.client.1.smithi084.stdout:3/225: unlink d3/d27/d4b/f4d 0 2024-02-28T23:07:29.483 INFO:tasks.workunit.client.0.smithi057.stdout:9/109: truncate dd/de/d14/f18 1631632 0 2024-02-28T23:07:29.486 INFO:tasks.workunit.client.1.smithi084.stdout:4/223: symlink d1/d18/d20/d15/l41 0 2024-02-28T23:07:29.487 INFO:tasks.workunit.client.0.smithi057.stdout:3/79: fdatasync f1 0 2024-02-28T23:07:29.488 INFO:tasks.workunit.client.0.smithi057.stdout:1/46: creat d2/d7/fa x:0 0 0 2024-02-28T23:07:29.488 INFO:tasks.workunit.client.1.smithi084.stdout:6/266: mknod d1/d13/d2e/d41/d4d/c59 0 2024-02-28T23:07:29.489 INFO:tasks.workunit.client.0.smithi057.stdout:5/73: truncate f2 3274327 0 2024-02-28T23:07:29.490 INFO:tasks.workunit.client.1.smithi084.stdout:6/267: truncate d1/d9/f4f 925004 0 2024-02-28T23:07:29.492 INFO:tasks.workunit.client.0.smithi057.stdout:7/160: dwrite d0/d5/d25/f28 [0,4194304] 0 2024-02-28T23:07:29.495 INFO:tasks.workunit.client.0.smithi057.stdout:8/139: rmdir d4/d1e 39 2024-02-28T23:07:29.495 INFO:tasks.workunit.client.1.smithi084.stdout:3/226: creat d3/d8/d1a/d21/d3d/f52 x:0 0 0 2024-02-28T23:07:29.496 INFO:tasks.workunit.client.0.smithi057.stdout:6/108: mkdir d0/d4/d11/d1f 0 2024-02-28T23:07:29.497 INFO:tasks.workunit.client.0.smithi057.stdout:4/164: dread d1a/d1b/f23 [4194304,4194304] 0 2024-02-28T23:07:29.497 INFO:tasks.workunit.client.1.smithi084.stdout:3/227: write d3/f7 [4502967,5572] 0 2024-02-28T23:07:29.498 INFO:tasks.workunit.client.1.smithi084.stdout:6/268: chown d1/d13/d2e/d41/d2c/c2d 1975671975 1 2024-02-28T23:07:29.501 INFO:tasks.workunit.client.0.smithi057.stdout:3/80: mknod d3/c18 0 2024-02-28T23:07:29.502 INFO:tasks.workunit.client.0.smithi057.stdout:0/117: link d17/d20/f27 d17/d19/f28 0 2024-02-28T23:07:29.503 INFO:tasks.workunit.client.0.smithi057.stdout:8/140: fsync d4/d13/d18/f26 0 2024-02-28T23:07:29.504 INFO:tasks.workunit.client.1.smithi084.stdout:0/218: truncate d1/d8/f23 137381 0 2024-02-28T23:07:29.506 INFO:tasks.workunit.client.0.smithi057.stdout:5/74: mkdir d4/d9/d14 0 2024-02-28T23:07:29.506 INFO:tasks.workunit.client.1.smithi084.stdout:3/228: creat d3/d8/d1a/d29/f53 x:0 0 0 2024-02-28T23:07:29.507 INFO:tasks.workunit.client.0.smithi057.stdout:2/155: write d8/fb [3389584,83147] 0 2024-02-28T23:07:29.510 INFO:tasks.workunit.client.1.smithi084.stdout:6/269: unlink d1/l12 0 2024-02-28T23:07:29.510 INFO:tasks.workunit.client.0.smithi057.stdout:3/81: rename d3/d8/c15 to d3/c19 0 2024-02-28T23:07:29.510 INFO:tasks.workunit.client.0.smithi057.stdout:2/156: chown d8/f19 14 1 2024-02-28T23:07:29.512 INFO:tasks.workunit.client.1.smithi084.stdout:4/224: creat d1/d18/f42 x:0 0 0 2024-02-28T23:07:29.512 INFO:tasks.workunit.client.1.smithi084.stdout:3/229: write d3/d8/d1a/d29/f53 [232505,81667] 0 2024-02-28T23:07:29.513 INFO:tasks.workunit.client.0.smithi057.stdout:8/141: creat d4/d13/d1a/f32 x:0 0 0 2024-02-28T23:07:29.513 INFO:tasks.workunit.client.0.smithi057.stdout:7/161: mknod d0/c31 0 2024-02-28T23:07:29.515 INFO:tasks.workunit.client.1.smithi084.stdout:2/223: getdents d5/d7/d12/d24 0 2024-02-28T23:07:29.515 INFO:tasks.workunit.client.1.smithi084.stdout:6/270: truncate d1/d9/f28 4280381 0 2024-02-28T23:07:29.516 INFO:tasks.workunit.client.0.smithi057.stdout:0/118: symlink d17/d20/d23/l29 0 2024-02-28T23:07:29.518 INFO:tasks.workunit.client.0.smithi057.stdout:8/142: write d4/f5 [320643,20133] 0 2024-02-28T23:07:29.519 INFO:tasks.workunit.client.1.smithi084.stdout:4/225: write d1/d18/d20/d15/f16 [2352340,124258] 0 2024-02-28T23:07:29.519 INFO:tasks.workunit.client.0.smithi057.stdout:2/157: chown d8/d11/d21/c37 15 1 2024-02-28T23:07:29.520 INFO:tasks.workunit.client.0.smithi057.stdout:7/162: chown d0/d8/c1f 166 1 2024-02-28T23:07:29.524 INFO:tasks.workunit.client.1.smithi084.stdout:2/224: symlink d5/d7/d12/l39 0 2024-02-28T23:07:29.524 INFO:tasks.workunit.client.0.smithi057.stdout:3/82: dread d3/d8/fb [0,4194304] 0 2024-02-28T23:07:29.524 INFO:tasks.workunit.client.1.smithi084.stdout:8/213: sync 2024-02-28T23:07:29.525 INFO:tasks.workunit.client.0.smithi057.stdout:8/143: write d4/f5 [508693,115575] 0 2024-02-28T23:07:29.525 INFO:tasks.workunit.client.0.smithi057.stdout:4/165: dwrite d1a/d1b/d2d/f3c [0,4194304] 0 2024-02-28T23:07:29.529 INFO:tasks.workunit.client.1.smithi084.stdout:2/225: write d5/d7/d12/d24/f38 [218252,83428] 0 2024-02-28T23:07:29.532 INFO:tasks.workunit.client.1.smithi084.stdout:2/226: dread - d5/d7/d9/f32 zero size 2024-02-28T23:07:29.532 INFO:tasks.workunit.client.0.smithi057.stdout:0/119: mknod d17/d20/c2a 0 2024-02-28T23:07:29.533 INFO:tasks.workunit.client.1.smithi084.stdout:9/285: dwrite d4/d6/ff [0,4194304] 0 2024-02-28T23:07:29.533 INFO:tasks.workunit.client.0.smithi057.stdout:9/110: dwrite dd/f12 [0,4194304] 0 2024-02-28T23:07:29.533 INFO:tasks.workunit.client.0.smithi057.stdout:1/47: write f0 [2401149,75792] 0 2024-02-28T23:07:29.534 INFO:tasks.workunit.client.0.smithi057.stdout:1/48: chown d2/d7 77045266 1 2024-02-28T23:07:29.535 INFO:tasks.workunit.client.0.smithi057.stdout:9/111: chown c6 1977182958 1 2024-02-28T23:07:29.536 INFO:tasks.workunit.client.1.smithi084.stdout:0/219: dread d1/d8/f23 [0,4194304] 0 2024-02-28T23:07:29.540 INFO:tasks.workunit.client.1.smithi084.stdout:4/226: rename d1/c1e to d1/c43 0 2024-02-28T23:07:29.542 INFO:tasks.workunit.client.0.smithi057.stdout:4/166: mkdir d1a/d1b/d24/d2e/d43 0 2024-02-28T23:07:29.543 INFO:tasks.workunit.client.0.smithi057.stdout:9/112: chown dd/de/f1d 29 1 2024-02-28T23:07:29.543 INFO:tasks.workunit.client.0.smithi057.stdout:6/109: getdents d0/d4/d11 0 2024-02-28T23:07:29.544 INFO:tasks.workunit.client.1.smithi084.stdout:8/214: dread d0/d9/fb [0,4194304] 0 2024-02-28T23:07:29.545 INFO:tasks.workunit.client.0.smithi057.stdout:9/113: write dd/de/d14/f21 [456659,38340] 0 2024-02-28T23:07:29.546 INFO:tasks.workunit.client.1.smithi084.stdout:3/230: getdents d3/d8/d14/d31 0 2024-02-28T23:07:29.546 INFO:tasks.workunit.client.0.smithi057.stdout:0/120: creat d17/d20/f2b x:0 0 0 2024-02-28T23:07:29.546 INFO:tasks.workunit.client.1.smithi084.stdout:9/286: creat d4/d32/f5c x:0 0 0 2024-02-28T23:07:29.548 INFO:tasks.workunit.client.0.smithi057.stdout:4/167: truncate d1a/d1b/d24/d2e/f3f 662122 0 2024-02-28T23:07:29.548 INFO:tasks.workunit.client.0.smithi057.stdout:2/158: read d8/fa [329800,69687] 0 2024-02-28T23:07:29.549 INFO:tasks.workunit.client.0.smithi057.stdout:6/110: read - d0/d4/d11/f13 zero size 2024-02-28T23:07:29.550 INFO:tasks.workunit.client.1.smithi084.stdout:2/227: mkdir d5/d7/d12/d24/d3a 0 2024-02-28T23:07:29.551 INFO:tasks.workunit.client.1.smithi084.stdout:6/271: dwrite d1/d9/d1d/d1e/d33/f39 [4194304,4194304] 0 2024-02-28T23:07:29.553 INFO:tasks.workunit.client.0.smithi057.stdout:4/168: mknod d1a/d1b/d24/c44 0 2024-02-28T23:07:29.553 INFO:tasks.workunit.client.0.smithi057.stdout:3/83: link d3/d4/d7/cf d3/d4/c1a 0 2024-02-28T23:07:29.554 INFO:tasks.workunit.client.0.smithi057.stdout:2/159: dread - d8/d11/d21/f31 zero size 2024-02-28T23:07:29.559 INFO:tasks.workunit.client.1.smithi084.stdout:8/215: mknod d0/d9/d30/d33/c48 0 2024-02-28T23:07:29.564 INFO:tasks.workunit.client.1.smithi084.stdout:4/227: creat d1/d18/d20/d24/d2f/d35/d37/f44 x:0 0 0 2024-02-28T23:07:29.567 INFO:tasks.workunit.client.0.smithi057.stdout:6/111: mkdir d0/d17/d20 0 2024-02-28T23:07:29.570 INFO:tasks.workunit.client.0.smithi057.stdout:1/49: dwrite f0 [0,4194304] 0 2024-02-28T23:07:29.574 INFO:tasks.workunit.client.1.smithi084.stdout:6/272: mknod d1/c5a 0 2024-02-28T23:07:29.575 INFO:tasks.workunit.client.1.smithi084.stdout:8/216: mknod d0/d9/d30/d33/c49 0 2024-02-28T23:07:29.579 INFO:tasks.workunit.client.1.smithi084.stdout:2/228: dwrite d5/f29 [0,4194304] 0 2024-02-28T23:07:29.580 INFO:tasks.workunit.client.1.smithi084.stdout:3/231: link d3/d8/df/l15 d3/d50/l54 0 2024-02-28T23:07:29.581 INFO:tasks.workunit.client.0.smithi057.stdout:5/75: write f2 [1849157,25122] 0 2024-02-28T23:07:29.584 INFO:tasks.workunit.client.1.smithi084.stdout:0/220: dwrite d1/d8/f23 [0,4194304] 0 2024-02-28T23:07:29.584 INFO:tasks.workunit.client.1.smithi084.stdout:8/217: write d0/f8 [4637315,29319] 0 2024-02-28T23:07:29.585 INFO:tasks.workunit.client.0.smithi057.stdout:7/163: write d0/f23 [1980,121492] 0 2024-02-28T23:07:29.587 INFO:tasks.workunit.client.1.smithi084.stdout:6/273: truncate d1/d13/d2e/f44 944147 0 2024-02-28T23:07:29.588 INFO:tasks.workunit.client.0.smithi057.stdout:1/50: truncate d2/d7/fa 188582 0 2024-02-28T23:07:29.589 INFO:tasks.workunit.client.1.smithi084.stdout:9/287: getdents d4/d6/d15 0 2024-02-28T23:07:29.590 INFO:tasks.workunit.client.1.smithi084.stdout:3/232: creat d3/d27/d4b/f55 x:0 0 0 2024-02-28T23:07:29.590 INFO:tasks.workunit.client.1.smithi084.stdout:8/218: fdatasync d0/d9/d1b/d38/f3a 0 2024-02-28T23:07:29.591 INFO:tasks.workunit.client.0.smithi057.stdout:8/144: dwrite d4/f7 [8388608,4194304] 0 2024-02-28T23:07:29.592 INFO:tasks.workunit.client.1.smithi084.stdout:8/219: stat d0/c19 0 2024-02-28T23:07:29.596 INFO:tasks.workunit.client.0.smithi057.stdout:6/112: link d0/d4/fb d0/d17/d20/f21 0 2024-02-28T23:07:29.597 INFO:tasks.workunit.client.1.smithi084.stdout:8/220: creat d0/d44/f4a x:0 0 0 2024-02-28T23:07:29.597 INFO:tasks.workunit.client.1.smithi084.stdout:9/288: mknod d4/d32/d3f/d5a/c5d 0 2024-02-28T23:07:29.600 INFO:tasks.workunit.client.1.smithi084.stdout:9/289: fsync d4/f3b 0 2024-02-28T23:07:29.601 INFO:tasks.workunit.client.0.smithi057.stdout:1/51: mkdir d2/d4/db 0 2024-02-28T23:07:29.601 INFO:tasks.workunit.client.1.smithi084.stdout:0/221: dread d1/d8/f1c [0,4194304] 0 2024-02-28T23:07:29.602 INFO:tasks.workunit.client.0.smithi057.stdout:9/114: dwrite dd/f12 [0,4194304] 0 2024-02-28T23:07:29.604 INFO:tasks.workunit.client.1.smithi084.stdout:9/290: stat d4/d6/d15/d3c/f47 0 2024-02-28T23:07:29.605 INFO:tasks.workunit.client.0.smithi057.stdout:7/164: creat d0/d8/d15/d29/f32 x:0 0 0 2024-02-28T23:07:29.606 INFO:tasks.workunit.client.1.smithi084.stdout:3/233: link d3/l44 d3/d27/d4b/l56 0 2024-02-28T23:07:29.606 INFO:tasks.workunit.client.1.smithi084.stdout:9/291: chown d4/d32 428420 1 2024-02-28T23:07:29.609 INFO:tasks.workunit.client.1.smithi084.stdout:0/222: mknod d1/d6/db/c43 0 2024-02-28T23:07:29.611 INFO:tasks.workunit.client.1.smithi084.stdout:2/229: dread d5/d7/f19 [0,4194304] 0 2024-02-28T23:07:29.611 INFO:tasks.workunit.client.1.smithi084.stdout:3/234: mknod d3/d50/c57 0 2024-02-28T23:07:29.611 INFO:tasks.workunit.client.0.smithi057.stdout:3/84: dwrite d3/d4/f12 [0,4194304] 0 2024-02-28T23:07:29.612 INFO:tasks.workunit.client.0.smithi057.stdout:6/113: mkdir d0/d1b/d22 0 2024-02-28T23:07:29.612 INFO:tasks.workunit.client.1.smithi084.stdout:2/230: stat d5 0 2024-02-28T23:07:29.613 INFO:tasks.workunit.client.0.smithi057.stdout:8/145: truncate d4/d1e/f2e 1030616 0 2024-02-28T23:07:29.613 INFO:tasks.workunit.client.1.smithi084.stdout:9/292: unlink d4/d32/d3f/d5a/d2d/l44 0 2024-02-28T23:07:29.614 INFO:tasks.workunit.client.0.smithi057.stdout:8/146: stat d4/d13 0 2024-02-28T23:07:29.615 INFO:tasks.workunit.client.1.smithi084.stdout:3/235: creat d3/d8/d14/f58 x:0 0 0 2024-02-28T23:07:29.619 INFO:tasks.workunit.client.1.smithi084.stdout:3/236: truncate d3/d8/f10 750703 0 2024-02-28T23:07:29.619 INFO:tasks.workunit.client.0.smithi057.stdout:1/52: link d2/d4/c5 d2/d4/cc 0 2024-02-28T23:07:29.619 INFO:tasks.workunit.client.0.smithi057.stdout:1/53: readlink - no filename 2024-02-28T23:07:29.620 INFO:tasks.workunit.client.0.smithi057.stdout:0/121: dwrite d17/d19/f28 [0,4194304] 0 2024-02-28T23:07:29.621 INFO:tasks.workunit.client.0.smithi057.stdout:3/85: write d3/d8/f16 [892375,34723] 0 2024-02-28T23:07:29.621 INFO:tasks.workunit.client.0.smithi057.stdout:8/147: write d4/d13/d18/f27 [1488233,56801] 0 2024-02-28T23:07:29.622 INFO:tasks.workunit.client.1.smithi084.stdout:0/223: mkdir d1/d6/db/d35/d44 0 2024-02-28T23:07:29.623 INFO:tasks.workunit.client.1.smithi084.stdout:3/237: chown d3/d27/f2d 246575652 1 2024-02-28T23:07:29.625 INFO:tasks.workunit.client.1.smithi084.stdout:0/224: stat d1/d6/db/d35 0 2024-02-28T23:07:29.625 INFO:tasks.workunit.client.1.smithi084.stdout:8/221: dwrite d0/f1a [0,4194304] 0 2024-02-28T23:07:29.625 INFO:tasks.workunit.client.1.smithi084.stdout:2/231: getdents d5/d7/d1d 0 2024-02-28T23:07:29.627 INFO:tasks.workunit.client.0.smithi057.stdout:3/86: dread - d3/d4/f17 zero size 2024-02-28T23:07:29.627 INFO:tasks.workunit.client.0.smithi057.stdout:6/114: symlink d0/d4/d11/d1f/l23 0 2024-02-28T23:07:29.628 INFO:tasks.workunit.client.1.smithi084.stdout:0/225: chown d1/d6/db/d35 2321 1 2024-02-28T23:07:29.630 INFO:tasks.workunit.client.0.smithi057.stdout:7/165: link d0/d5/l14 d0/l33 0 2024-02-28T23:07:29.631 INFO:tasks.workunit.client.1.smithi084.stdout:3/238: symlink d3/d8/d1a/l59 0 2024-02-28T23:07:29.631 INFO:tasks.workunit.client.1.smithi084.stdout:8/222: chown d0/d44/f4a 55688 1 2024-02-28T23:07:29.631 INFO:tasks.workunit.client.0.smithi057.stdout:5/76: dread f1 [0,4194304] 0 2024-02-28T23:07:29.634 INFO:tasks.workunit.client.0.smithi057.stdout:7/166: read - d0/d8/d15/d29/f32 zero size 2024-02-28T23:07:29.635 INFO:tasks.workunit.client.1.smithi084.stdout:0/226: stat d1/d6/db/l2b 0 2024-02-28T23:07:29.635 INFO:tasks.workunit.client.1.smithi084.stdout:0/227: fdatasync d1/d6/f19 0 2024-02-28T23:07:29.637 INFO:tasks.workunit.client.0.smithi057.stdout:6/115: unlink d0/d17/c18 0 2024-02-28T23:07:29.638 INFO:tasks.workunit.client.1.smithi084.stdout:3/239: write d3/f11 [3929996,24579] 0 2024-02-28T23:07:29.639 INFO:tasks.workunit.client.0.smithi057.stdout:5/77: creat d4/f15 x:0 0 0 2024-02-28T23:07:29.642 INFO:tasks.workunit.client.1.smithi084.stdout:0/228: symlink d1/d6/db/l45 0 2024-02-28T23:07:29.643 INFO:tasks.workunit.client.0.smithi057.stdout:1/54: rename d2/d4/c6 to d2/d7/cd 0 2024-02-28T23:07:29.643 INFO:tasks.workunit.client.0.smithi057.stdout:9/115: dwrite dd/de/d14/f1e [0,4194304] 0 2024-02-28T23:07:29.646 INFO:tasks.workunit.client.0.smithi057.stdout:7/167: truncate d0/d5/f30 660470 0 2024-02-28T23:07:29.650 INFO:tasks.workunit.client.1.smithi084.stdout:0/229: chown d1/d6/f14 6258 1 2024-02-28T23:07:29.651 INFO:tasks.workunit.client.0.smithi057.stdout:9/116: fdatasync dd/f12 0 2024-02-28T23:07:29.652 INFO:tasks.workunit.client.1.smithi084.stdout:8/223: getdents d0/d9 0 2024-02-28T23:07:29.653 INFO:tasks.workunit.client.1.smithi084.stdout:0/230: chown d1/d8/f40 72 1 2024-02-28T23:07:29.655 INFO:tasks.workunit.client.0.smithi057.stdout:1/55: write d2/d7/fa [198247,51369] 0 2024-02-28T23:07:29.655 INFO:tasks.workunit.client.1.smithi084.stdout:0/231: chown d1/d6/db/d37 16 1 2024-02-28T23:07:29.655 INFO:tasks.workunit.client.0.smithi057.stdout:0/122: rename f11 to d17/d20/d23/f2c 0 2024-02-28T23:07:29.657 INFO:tasks.workunit.client.0.smithi057.stdout:2/160: truncate d8/fa 6834263 0 2024-02-28T23:07:29.660 INFO:tasks.workunit.client.0.smithi057.stdout:0/123: truncate f10 2215012 0 2024-02-28T23:07:29.662 INFO:tasks.workunit.client.0.smithi057.stdout:3/87: dread f1 [0,4194304] 0 2024-02-28T23:07:29.665 INFO:tasks.workunit.client.1.smithi084.stdout:0/232: symlink d1/d8/d18/l46 0 2024-02-28T23:07:29.668 INFO:tasks.workunit.client.1.smithi084.stdout:0/233: creat d1/d8/f47 x:0 0 0 2024-02-28T23:07:29.668 INFO:tasks.workunit.client.0.smithi057.stdout:8/148: dwrite d4/d13/d1a/f32 [0,4194304] 0 2024-02-28T23:07:29.668 INFO:tasks.workunit.client.0.smithi057.stdout:0/124: dread - d17/d20/d23/f2c zero size 2024-02-28T23:07:29.671 INFO:tasks.workunit.client.0.smithi057.stdout:8/149: dread - d4/d13/d18/f23 zero size 2024-02-28T23:07:29.672 INFO:tasks.workunit.client.1.smithi084.stdout:0/234: creat d1/d8/d18/f48 x:0 0 0 2024-02-28T23:07:29.673 INFO:tasks.workunit.client.1.smithi084.stdout:8/224: dwrite d0/d9/d30/f42 [0,4194304] 0 2024-02-28T23:07:29.674 INFO:tasks.workunit.client.0.smithi057.stdout:6/116: dread d0/d4/d11/f12 [0,4194304] 0 2024-02-28T23:07:29.675 INFO:tasks.workunit.client.0.smithi057.stdout:5/78: rename d4/db to d4/d9/d16 0 2024-02-28T23:07:29.678 INFO:tasks.workunit.client.1.smithi084.stdout:8/225: mknod d0/d11/c4b 0 2024-02-28T23:07:29.678 INFO:tasks.workunit.client.0.smithi057.stdout:2/161: symlink d8/d11/d21/l3e 0 2024-02-28T23:07:29.678 INFO:tasks.workunit.client.0.smithi057.stdout:6/117: stat d0/d4/fa 0 2024-02-28T23:07:29.679 INFO:tasks.workunit.client.0.smithi057.stdout:8/150: mknod d4/d13/d2c/c33 0 2024-02-28T23:07:29.679 INFO:tasks.workunit.client.0.smithi057.stdout:3/88: mkdir d3/d8/d1b 0 2024-02-28T23:07:29.680 INFO:tasks.workunit.client.0.smithi057.stdout:5/79: write d4/f15 [214420,2032] 0 2024-02-28T23:07:29.680 INFO:tasks.workunit.client.1.smithi084.stdout:1/342: sync 2024-02-28T23:07:29.681 INFO:tasks.workunit.client.1.smithi084.stdout:5/275: sync 2024-02-28T23:07:29.681 INFO:tasks.workunit.client.1.smithi084.stdout:7/246: sync 2024-02-28T23:07:29.682 INFO:tasks.workunit.client.0.smithi057.stdout:1/56: chown d2/d4/cc 28855 1 2024-02-28T23:07:29.684 INFO:tasks.workunit.client.1.smithi084.stdout:8/226: creat d0/d9/f4c x:0 0 0 2024-02-28T23:07:29.686 INFO:tasks.workunit.client.1.smithi084.stdout:7/247: fsync db/d11/f29 0 2024-02-28T23:07:29.688 INFO:tasks.workunit.client.1.smithi084.stdout:8/227: write d0/d44/f4a [814193,49092] 0 2024-02-28T23:07:29.690 INFO:tasks.workunit.client.1.smithi084.stdout:7/248: truncate db/ff 2210213 0 2024-02-28T23:07:29.690 INFO:tasks.workunit.client.0.smithi057.stdout:2/162: symlink d8/d11/d21/d28/l3f 0 2024-02-28T23:07:29.690 INFO:tasks.workunit.client.0.smithi057.stdout:3/89: readlink d3/d4/d7/le 0 2024-02-28T23:07:29.691 INFO:tasks.workunit.client.0.smithi057.stdout:6/118: truncate d0/f1c 1395066 0 2024-02-28T23:07:29.692 INFO:tasks.workunit.client.1.smithi084.stdout:0/235: dread d1/d6/f19 [0,4194304] 0 2024-02-28T23:07:29.693 INFO:tasks.workunit.client.0.smithi057.stdout:8/151: write d4/d13/d18/f27 [1433805,26208] 0 2024-02-28T23:07:29.693 INFO:tasks.workunit.client.0.smithi057.stdout:1/57: read d2/d7/fa [11567,77371] 0 2024-02-28T23:07:29.694 INFO:tasks.workunit.client.1.smithi084.stdout:5/276: symlink dc/de/df/d16/d34/l60 0 2024-02-28T23:07:29.694 INFO:tasks.workunit.client.0.smithi057.stdout:6/119: stat d0/d17/d20 0 2024-02-28T23:07:29.697 INFO:tasks.workunit.client.1.smithi084.stdout:1/343: symlink d12/d1b/l78 0 2024-02-28T23:07:29.697 INFO:tasks.workunit.client.0.smithi057.stdout:5/80: creat d4/d9/d14/f17 x:0 0 0 2024-02-28T23:07:29.698 INFO:tasks.workunit.client.0.smithi057.stdout:9/117: dwrite dd/de/d14/f18 [0,4194304] 0 2024-02-28T23:07:29.700 INFO:tasks.workunit.client.1.smithi084.stdout:7/249: creat db/d11/f4f x:0 0 0 2024-02-28T23:07:29.700 INFO:tasks.workunit.client.0.smithi057.stdout:5/81: write d4/f15 [1082493,50086] 0 2024-02-28T23:07:29.703 INFO:tasks.workunit.client.1.smithi084.stdout:0/236: mknod d1/d6/db/c49 0 2024-02-28T23:07:29.704 INFO:tasks.workunit.client.0.smithi057.stdout:5/82: dread - d4/d9/fc zero size 2024-02-28T23:07:29.706 INFO:tasks.workunit.client.1.smithi084.stdout:5/277: chown dc/de/df/d16/d34/l56 1079504 1 2024-02-28T23:07:29.707 INFO:tasks.workunit.client.0.smithi057.stdout:2/163: fsync d8/fd 0 2024-02-28T23:07:29.708 INFO:tasks.workunit.client.0.smithi057.stdout:6/120: mknod d0/d4/d11/d1f/c24 0 2024-02-28T23:07:29.709 INFO:tasks.workunit.client.0.smithi057.stdout:5/83: truncate d4/d9/fc 786194 0 2024-02-28T23:07:29.711 INFO:tasks.workunit.client.0.smithi057.stdout:6/121: dread - d0/d17/f1d zero size 2024-02-28T23:07:29.712 INFO:tasks.workunit.client.1.smithi084.stdout:9/293: truncate d4/fc 2908806 0 2024-02-28T23:07:29.714 INFO:tasks.workunit.client.1.smithi084.stdout:9/294: chown d4/d6/d15/d3c 1423462164 1 2024-02-28T23:07:29.715 INFO:tasks.workunit.client.1.smithi084.stdout:2/232: truncate d5/f17 755231 0 2024-02-28T23:07:29.716 INFO:tasks.workunit.client.1.smithi084.stdout:0/237: truncate d1/d8/f20 1734177 0 2024-02-28T23:07:29.717 INFO:tasks.workunit.client.1.smithi084.stdout:2/233: readlink d5/d7/d1d/l23 0 2024-02-28T23:07:29.719 INFO:tasks.workunit.client.1.smithi084.stdout:9/295: symlink d4/d6/d15/d12/l5e 0 2024-02-28T23:07:29.719 INFO:tasks.workunit.client.1.smithi084.stdout:7/250: dwrite db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:29.721 INFO:tasks.workunit.client.0.smithi057.stdout:1/58: dwrite f0 [4194304,4194304] 0 2024-02-28T23:07:29.721 INFO:tasks.workunit.client.0.smithi057.stdout:6/122: truncate d0/d4/d11/f13 627769 0 2024-02-28T23:07:29.723 INFO:tasks.workunit.client.1.smithi084.stdout:9/296: readlink d4/d6/le 0 2024-02-28T23:07:29.723 INFO:tasks.workunit.client.1.smithi084.stdout:7/251: stat db/d10/d17 0 2024-02-28T23:07:29.726 INFO:tasks.workunit.client.1.smithi084.stdout:2/234: creat d5/f3b x:0 0 0 2024-02-28T23:07:29.727 INFO:tasks.workunit.client.0.smithi057.stdout:9/118: dread dd/de/d14/f18 [0,4194304] 0 2024-02-28T23:07:29.727 INFO:tasks.workunit.client.0.smithi057.stdout:2/164: creat d8/f40 x:0 0 0 2024-02-28T23:07:29.729 INFO:tasks.workunit.client.1.smithi084.stdout:2/235: write d5/f31 [877539,115389] 0 2024-02-28T23:07:29.732 INFO:tasks.workunit.client.0.smithi057.stdout:9/119: chown l8 2849231 1 2024-02-28T23:07:29.735 INFO:tasks.workunit.client.0.smithi057.stdout:6/123: mkdir d0/d4/d25 0 2024-02-28T23:07:29.739 INFO:tasks.workunit.client.1.smithi084.stdout:3/240: dread d3/d27/f2d [0,4194304] 0 2024-02-28T23:07:29.741 INFO:tasks.workunit.client.0.smithi057.stdout:2/165: mkdir d8/d11/d21/d28/d41 0 2024-02-28T23:07:29.741 INFO:tasks.workunit.client.1.smithi084.stdout:1/344: truncate d12/d1b/d63/f3a 872857 0 2024-02-28T23:07:29.745 INFO:tasks.workunit.client.1.smithi084.stdout:3/241: write d3/f28 [468133,29925] 0 2024-02-28T23:07:29.745 INFO:tasks.workunit.client.0.smithi057.stdout:7/168: dwrite d0/d8/d15/f2f [0,4194304] 0 2024-02-28T23:07:29.747 INFO:tasks.workunit.client.0.smithi057.stdout:2/166: write d8/d11/f32 [637091,130334] 0 2024-02-28T23:07:29.748 INFO:tasks.workunit.client.0.smithi057.stdout:5/84: dwrite f2 [0,4194304] 0 2024-02-28T23:07:29.748 INFO:tasks.workunit.client.1.smithi084.stdout:9/297: dwrite d4/d6/f36 [0,4194304] 0 2024-02-28T23:07:29.748 INFO:tasks.workunit.client.1.smithi084.stdout:8/228: rename d0/d9/d1b/d38/d43/d45 to d0/d24/d4d 0 2024-02-28T23:07:29.749 INFO:tasks.workunit.client.0.smithi057.stdout:2/167: stat d8/d11/f1d 0 2024-02-28T23:07:29.750 INFO:tasks.workunit.client.1.smithi084.stdout:3/242: chown d3/d8/f24 156 1 2024-02-28T23:07:29.751 INFO:tasks.workunit.client.0.smithi057.stdout:9/120: link dd/de/c13 dd/de/d14/c22 0 2024-02-28T23:07:29.751 INFO:tasks.workunit.client.0.smithi057.stdout:7/169: symlink d0/l34 0 2024-02-28T23:07:29.752 INFO:tasks.workunit.client.1.smithi084.stdout:1/345: creat d12/d1b/d2e/f79 x:0 0 0 2024-02-28T23:07:29.754 INFO:tasks.workunit.client.0.smithi057.stdout:9/121: read dd/de/d14/f17 [1735489,29935] 0 2024-02-28T23:07:29.756 INFO:tasks.workunit.client.0.smithi057.stdout:9/122: fsync dd/de/f1d 0 2024-02-28T23:07:29.757 INFO:tasks.workunit.client.1.smithi084.stdout:9/298: creat d4/d32/f5f x:0 0 0 2024-02-28T23:07:29.757 INFO:tasks.workunit.client.1.smithi084.stdout:8/229: symlink d0/d44/l4e 0 2024-02-28T23:07:29.763 INFO:tasks.workunit.client.1.smithi084.stdout:8/230: symlink d0/d9/d1b/d38/d43/l4f 0 2024-02-28T23:07:29.765 INFO:tasks.workunit.client.0.smithi057.stdout:7/170: creat d0/f35 x:0 0 0 2024-02-28T23:07:29.765 INFO:tasks.workunit.client.0.smithi057.stdout:2/168: creat d8/f42 x:0 0 0 2024-02-28T23:07:29.765 INFO:tasks.workunit.client.1.smithi084.stdout:8/231: write d0/f34 [3387406,26170] 0 2024-02-28T23:07:29.767 INFO:tasks.workunit.client.0.smithi057.stdout:7/171: read - d0/d8/d15/d29/f32 zero size 2024-02-28T23:07:29.768 INFO:tasks.workunit.client.1.smithi084.stdout:8/232: chown d0/d9/d1b/d38/d43/l4f 19 1 2024-02-28T23:07:29.769 INFO:tasks.workunit.client.1.smithi084.stdout:3/243: getdents d3/d8/d1a/d21 0 2024-02-28T23:07:29.769 INFO:tasks.workunit.client.1.smithi084.stdout:7/252: write db/d10/f2c [199485,56556] 0 2024-02-28T23:07:29.771 INFO:tasks.workunit.client.0.smithi057.stdout:9/123: mknod dd/de/c23 0 2024-02-28T23:07:29.773 INFO:tasks.workunit.client.1.smithi084.stdout:7/253: dread - db/d11/d27/d36/f4c zero size 2024-02-28T23:07:29.773 INFO:tasks.workunit.client.1.smithi084.stdout:8/233: read d0/d9/d1b/f1d [1972133,29923] 0 2024-02-28T23:07:29.774 INFO:tasks.workunit.client.0.smithi057.stdout:9/124: mkdir dd/de/d24 0 2024-02-28T23:07:29.775 INFO:tasks.workunit.client.1.smithi084.stdout:9/299: rename d4/d6/d15/d12/d41/f59 to d4/f60 0 2024-02-28T23:07:29.777 INFO:tasks.workunit.client.0.smithi057.stdout:9/125: truncate dd/de/d14/f1e 4522687 0 2024-02-28T23:07:29.778 INFO:tasks.workunit.client.0.smithi057.stdout:9/126: fdatasync dd/ff 0 2024-02-28T23:07:29.780 INFO:tasks.workunit.client.0.smithi057.stdout:2/169: dread d8/ff [0,4194304] 0 2024-02-28T23:07:29.781 INFO:tasks.workunit.client.0.smithi057.stdout:7/172: dread d0/d8/f19 [0,4194304] 0 2024-02-28T23:07:29.783 INFO:tasks.workunit.client.0.smithi057.stdout:9/127: fdatasync dd/de/d14/f1f 0 2024-02-28T23:07:29.786 INFO:tasks.workunit.client.1.smithi084.stdout:3/244: mkdir d3/d50/d5a 0 2024-02-28T23:07:29.787 INFO:tasks.workunit.client.0.smithi057.stdout:1/59: truncate f0 1028255 0 2024-02-28T23:07:29.788 INFO:tasks.workunit.client.1.smithi084.stdout:9/300: fdatasync d4/f51 0 2024-02-28T23:07:29.788 INFO:tasks.workunit.client.0.smithi057.stdout:2/170: truncate d8/f23 574913 0 2024-02-28T23:07:29.789 INFO:tasks.workunit.client.1.smithi084.stdout:3/245: fdatasync d3/d8/df/f19 0 2024-02-28T23:07:29.790 INFO:tasks.workunit.client.1.smithi084.stdout:7/254: mknod db/d10/c50 0 2024-02-28T23:07:29.790 INFO:tasks.workunit.client.0.smithi057.stdout:6/124: write d0/fc [383194,32267] 0 2024-02-28T23:07:29.791 INFO:tasks.workunit.client.1.smithi084.stdout:8/234: truncate d0/d11/f2a 338597 0 2024-02-28T23:07:29.792 INFO:tasks.workunit.client.0.smithi057.stdout:7/173: unlink d0/d8/f16 0 2024-02-28T23:07:29.792 INFO:tasks.workunit.client.1.smithi084.stdout:0/238: dread d1/d8/f20 [0,4194304] 0 2024-02-28T23:07:29.793 INFO:tasks.workunit.client.1.smithi084.stdout:7/255: symlink db/d11/l51 0 2024-02-28T23:07:29.794 INFO:tasks.workunit.client.1.smithi084.stdout:9/301: creat d4/d6/d53/f61 x:0 0 0 2024-02-28T23:07:29.795 INFO:tasks.workunit.client.1.smithi084.stdout:3/246: rmdir d3/d8/df 39 2024-02-28T23:07:29.795 INFO:tasks.workunit.client.0.smithi057.stdout:2/171: symlink d8/d11/d21/d28/l43 0 2024-02-28T23:07:29.796 INFO:tasks.workunit.client.1.smithi084.stdout:8/235: fdatasync d0/d11/f3c 0 2024-02-28T23:07:29.797 INFO:tasks.workunit.client.0.smithi057.stdout:7/174: mkdir d0/d5/d36 0 2024-02-28T23:07:29.798 INFO:tasks.workunit.client.0.smithi057.stdout:1/60: read d2/d7/fa [128738,91983] 0 2024-02-28T23:07:29.798 INFO:tasks.workunit.client.0.smithi057.stdout:6/125: creat d0/d4/d25/f26 x:0 0 0 2024-02-28T23:07:29.800 INFO:tasks.workunit.client.1.smithi084.stdout:9/302: rmdir d4/d32/d3f 39 2024-02-28T23:07:29.803 INFO:tasks.workunit.client.1.smithi084.stdout:9/303: truncate d4/d32/f5c 324976 0 2024-02-28T23:07:29.805 INFO:tasks.workunit.client.1.smithi084.stdout:8/236: mknod d0/d9/d30/c50 0 2024-02-28T23:07:29.806 INFO:tasks.workunit.client.0.smithi057.stdout:6/126: unlink d0/d17/d20/f21 0 2024-02-28T23:07:29.807 INFO:tasks.workunit.client.1.smithi084.stdout:3/247: mkdir d3/d8/d1a/d29/d41/d5b 0 2024-02-28T23:07:29.808 INFO:tasks.workunit.client.1.smithi084.stdout:9/304: dread - d4/d6/d15/d12/d41/f46 zero size 2024-02-28T23:07:29.810 INFO:tasks.workunit.client.1.smithi084.stdout:3/248: readlink d3/d8/l4a 0 2024-02-28T23:07:29.811 INFO:tasks.workunit.client.0.smithi057.stdout:9/128: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:29.812 INFO:tasks.workunit.client.0.smithi057.stdout:6/127: creat d0/d1b/f27 x:0 0 0 2024-02-28T23:07:29.815 INFO:tasks.workunit.client.1.smithi084.stdout:8/237: mkdir d0/d9/d1b/d38/d43/d51 0 2024-02-28T23:07:29.816 INFO:tasks.workunit.client.1.smithi084.stdout:1/346: dwrite d12/d1b/d2e/d30/f1c [0,4194304] 0 2024-02-28T23:07:29.819 INFO:tasks.workunit.client.1.smithi084.stdout:9/305: truncate d4/d32/d3f/f42 768876 0 2024-02-28T23:07:29.820 INFO:tasks.workunit.client.0.smithi057.stdout:7/175: dread d0/d5/f2e [0,4194304] 0 2024-02-28T23:07:29.820 INFO:tasks.workunit.client.1.smithi084.stdout:8/238: write d0/d9/f4c [29897,76002] 0 2024-02-28T23:07:29.822 INFO:tasks.workunit.client.1.smithi084.stdout:3/249: creat d3/d8/d14/f5c x:0 0 0 2024-02-28T23:07:29.824 INFO:tasks.workunit.client.0.smithi057.stdout:1/61: dread d2/f8 [0,4194304] 0 2024-02-28T23:07:29.825 INFO:tasks.workunit.client.0.smithi057.stdout:5/85: dwrite d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:29.826 INFO:tasks.workunit.client.1.smithi084.stdout:8/239: mknod d0/d11/c52 0 2024-02-28T23:07:29.826 INFO:tasks.workunit.client.0.smithi057.stdout:7/176: fsync d0/d5/d25/f28 0 2024-02-28T23:07:29.829 INFO:tasks.workunit.client.0.smithi057.stdout:7/177: read d0/d5/f1e [3993857,114499] 0 2024-02-28T23:07:29.830 INFO:tasks.workunit.client.0.smithi057.stdout:5/86: write d4/f6 [3553683,70002] 0 2024-02-28T23:07:29.831 INFO:tasks.workunit.client.0.smithi057.stdout:6/128: link d0/d4/d11/d1f/c24 d0/d4/d11/c28 0 2024-02-28T23:07:29.832 INFO:tasks.workunit.client.1.smithi084.stdout:9/306: creat d4/d6/d15/f62 x:0 0 0 2024-02-28T23:07:29.832 INFO:tasks.workunit.client.0.smithi057.stdout:1/62: creat d2/d4/db/fe x:0 0 0 2024-02-28T23:07:29.833 INFO:tasks.workunit.client.1.smithi084.stdout:6/274: sync 2024-02-28T23:07:29.833 INFO:tasks.workunit.client.1.smithi084.stdout:4/228: sync 2024-02-28T23:07:29.835 INFO:tasks.workunit.client.0.smithi057.stdout:2/172: write d8/fa [2873405,707] 0 2024-02-28T23:07:29.836 INFO:tasks.workunit.client.1.smithi084.stdout:0/239: dread d1/d8/f1f [0,4194304] 0 2024-02-28T23:07:29.836 INFO:tasks.workunit.client.0.smithi057.stdout:6/129: symlink d0/d4/d25/l29 0 2024-02-28T23:07:29.838 INFO:tasks.workunit.client.0.smithi057.stdout:2/173: stat d8/f16 0 2024-02-28T23:07:29.839 INFO:tasks.workunit.client.1.smithi084.stdout:6/275: write d1/d13/d2e/d41/f2f [750014,25820] 0 2024-02-28T23:07:29.839 INFO:tasks.workunit.client.1.smithi084.stdout:3/250: creat d3/d8/d14/d31/f5d x:0 0 0 2024-02-28T23:07:29.842 INFO:tasks.workunit.client.0.smithi057.stdout:2/174: stat d8/d11/d21/c3a 0 2024-02-28T23:07:29.842 INFO:tasks.workunit.client.0.smithi057.stdout:6/130: write d0/d4/ff [887189,69183] 0 2024-02-28T23:07:29.845 INFO:tasks.workunit.client.1.smithi084.stdout:0/240: mkdir d1/d6/db/d35/d4a 0 2024-02-28T23:07:29.849 INFO:tasks.workunit.client.1.smithi084.stdout:0/241: write d1/d8/f36 [365705,99748] 0 2024-02-28T23:07:29.849 INFO:tasks.workunit.client.1.smithi084.stdout:6/276: creat d1/d9/d1d/d1e/f5b x:0 0 0 2024-02-28T23:07:29.850 INFO:tasks.workunit.client.0.smithi057.stdout:2/175: truncate d8/f24 611382 0 2024-02-28T23:07:29.853 INFO:tasks.workunit.client.1.smithi084.stdout:6/277: write d1/d9/f4f [1685111,91160] 0 2024-02-28T23:07:29.854 INFO:tasks.workunit.client.1.smithi084.stdout:8/240: rename d0/d9/d30/f3b to d0/f53 0 2024-02-28T23:07:29.856 INFO:tasks.workunit.client.0.smithi057.stdout:9/129: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:29.856 INFO:tasks.workunit.client.0.smithi057.stdout:6/131: dread d0/d4/f19 [0,4194304] 0 2024-02-28T23:07:29.856 INFO:tasks.workunit.client.1.smithi084.stdout:6/278: chown d1/d9/f3b 4393953 1 2024-02-28T23:07:29.859 INFO:tasks.workunit.client.1.smithi084.stdout:9/307: dread d4/d32/d3f/d5a/d2d/f34 [4194304,4194304] 0 2024-02-28T23:07:29.859 INFO:tasks.workunit.client.1.smithi084.stdout:8/241: write d0/d9/d1b/f46 [1041814,6952] 0 2024-02-28T23:07:29.863 INFO:tasks.workunit.client.1.smithi084.stdout:6/279: rename d1/d13/d2e/d3e to d1/d9/d1d/d1e/d33/d5c 0 2024-02-28T23:07:29.864 INFO:tasks.workunit.client.0.smithi057.stdout:2/176: write d8/d11/d21/d28/f2f [331902,8301] 0 2024-02-28T23:07:29.866 INFO:tasks.workunit.client.0.smithi057.stdout:9/130: fsync dd/de/d14/f18 0 2024-02-28T23:07:29.867 INFO:tasks.workunit.client.0.smithi057.stdout:6/132: fsync d0/d4/d11/f12 0 2024-02-28T23:07:29.870 INFO:tasks.workunit.client.1.smithi084.stdout:0/242: dwrite d1/d8/f23 [0,4194304] 0 2024-02-28T23:07:29.871 INFO:tasks.workunit.client.0.smithi057.stdout:6/133: chown d0/d4/d11 0 1 2024-02-28T23:07:29.872 INFO:tasks.workunit.client.1.smithi084.stdout:9/308: unlink d4/d32/d3f/d5a/c45 0 2024-02-28T23:07:29.873 INFO:tasks.workunit.client.1.smithi084.stdout:9/309: fdatasync d4/f56 0 2024-02-28T23:07:29.874 INFO:tasks.workunit.client.0.smithi057.stdout:9/131: rmdir dd 39 2024-02-28T23:07:29.874 INFO:tasks.workunit.client.0.smithi057.stdout:7/178: dwrite d0/f35 [0,4194304] 0 2024-02-28T23:07:29.874 INFO:tasks.workunit.client.0.smithi057.stdout:2/177: mknod d8/d11/c44 0 2024-02-28T23:07:29.877 INFO:tasks.workunit.client.0.smithi057.stdout:7/179: stat d0/d5/l14 0 2024-02-28T23:07:29.880 INFO:tasks.workunit.client.1.smithi084.stdout:0/243: link d1/d6/f21 d1/d6/db/d35/d44/f4b 0 2024-02-28T23:07:29.880 INFO:tasks.workunit.client.1.smithi084.stdout:1/347: truncate d12/d1b/f2c 3386824 0 2024-02-28T23:07:29.881 INFO:tasks.workunit.client.0.smithi057.stdout:7/180: mknod d0/d8/c37 0 2024-02-28T23:07:29.882 INFO:tasks.workunit.client.0.smithi057.stdout:1/63: dwrite d2/d7/fa [0,4194304] 0 2024-02-28T23:07:29.883 INFO:tasks.workunit.client.1.smithi084.stdout:0/244: chown d1/d6/db/l45 37602 1 2024-02-28T23:07:29.884 INFO:tasks.workunit.client.1.smithi084.stdout:4/229: truncate d1/d18/d20/d14/d30/f38 630495 0 2024-02-28T23:07:29.887 INFO:tasks.workunit.client.1.smithi084.stdout:6/280: link d1/d9/d1d/d1e/l43 d1/d9/l5d 0 2024-02-28T23:07:29.888 INFO:tasks.workunit.client.1.smithi084.stdout:3/251: truncate d3/d8/fa 856320 0 2024-02-28T23:07:29.890 INFO:tasks.workunit.client.1.smithi084.stdout:1/348: rename d12/d1b/d2e/d52/d61/c75 to d12/d1b/d2e/d52/c7a 0 2024-02-28T23:07:29.892 INFO:tasks.workunit.client.0.smithi057.stdout:1/64: write d2/f8 [521455,35000] 0 2024-02-28T23:07:29.895 INFO:tasks.workunit.client.0.smithi057.stdout:1/65: chown d2/d7/fa 1843401 1 2024-02-28T23:07:29.895 INFO:tasks.workunit.client.1.smithi084.stdout:4/230: mkdir d1/d18/d20/d24/d2f/d35/d45 0 2024-02-28T23:07:29.896 INFO:tasks.workunit.client.1.smithi084.stdout:0/245: dread d1/d8/f3c [0,4194304] 0 2024-02-28T23:07:29.896 INFO:tasks.workunit.client.0.smithi057.stdout:1/66: write d2/f8 [2084547,44190] 0 2024-02-28T23:07:29.897 INFO:tasks.workunit.client.0.smithi057.stdout:1/67: readlink - no filename 2024-02-28T23:07:29.898 INFO:tasks.workunit.client.1.smithi084.stdout:6/281: truncate d1/d9/d1d/d1e/f5b 817263 0 2024-02-28T23:07:29.898 INFO:tasks.workunit.client.1.smithi084.stdout:9/310: dwrite d4/f4e [0,4194304] 0 2024-02-28T23:07:29.898 INFO:tasks.workunit.client.0.smithi057.stdout:9/132: dread dd/ff [0,4194304] 0 2024-02-28T23:07:29.902 INFO:tasks.workunit.client.0.smithi057.stdout:1/68: rename d2/d4 to d2/d7/df 0 2024-02-28T23:07:29.902 INFO:tasks.workunit.client.1.smithi084.stdout:1/349: mknod d12/c7b 0 2024-02-28T23:07:29.903 INFO:tasks.workunit.client.0.smithi057.stdout:9/133: symlink dd/de/d14/l25 0 2024-02-28T23:07:29.909 INFO:tasks.workunit.client.0.smithi057.stdout:1/69: truncate d2/d7/df/db/fe 978609 0 2024-02-28T23:07:29.910 INFO:tasks.workunit.client.1.smithi084.stdout:4/231: link d1/d18/d20/d24/l26 d1/d18/d20/d24/d2f/d35/d37/l46 0 2024-02-28T23:07:29.913 INFO:tasks.workunit.client.1.smithi084.stdout:3/252: dwrite d3/f7 [0,4194304] 0 2024-02-28T23:07:29.914 INFO:tasks.workunit.client.1.smithi084.stdout:1/350: rename d12/d1b/d2e/f6c to d12/d1b/d2e/d52/f7c 0 2024-02-28T23:07:29.915 INFO:tasks.workunit.client.0.smithi057.stdout:9/134: dread dd/de/d14/f21 [0,4194304] 0 2024-02-28T23:07:29.916 INFO:tasks.workunit.client.0.smithi057.stdout:1/70: rename d2/d7/fa to d2/f10 0 2024-02-28T23:07:29.916 INFO:tasks.workunit.client.1.smithi084.stdout:4/232: chown d1/d18/d20/d15/f28 890 1 2024-02-28T23:07:29.921 INFO:tasks.workunit.client.1.smithi084.stdout:1/351: mkdir d12/d1b/d63/d7d 0 2024-02-28T23:07:29.921 INFO:tasks.workunit.client.0.smithi057.stdout:9/135: link dd/de/l1a dd/l26 0 2024-02-28T23:07:29.923 INFO:tasks.workunit.client.0.smithi057.stdout:7/181: truncate d0/d5/f30 48563 0 2024-02-28T23:07:29.925 INFO:tasks.workunit.client.1.smithi084.stdout:0/246: dwrite d1/d8/d18/f48 [0,4194304] 0 2024-02-28T23:07:29.930 INFO:tasks.workunit.client.1.smithi084.stdout:4/233: mkdir d1/d18/d20/d14/d30/d47 0 2024-02-28T23:07:29.932 INFO:tasks.workunit.client.0.smithi057.stdout:1/71: rmdir d2 39 2024-02-28T23:07:29.935 INFO:tasks.workunit.client.1.smithi084.stdout:3/253: mkdir d3/d8/d1a/d45/d5e 0 2024-02-28T23:07:29.942 INFO:tasks.workunit.client.1.smithi084.stdout:1/352: stat d12/d1b/d63/l6e 0 2024-02-28T23:07:29.945 INFO:tasks.workunit.client.1.smithi084.stdout:4/234: mkdir d1/d18/d20/d24/d2f/d35/d48 0 2024-02-28T23:07:29.947 INFO:tasks.workunit.client.1.smithi084.stdout:1/353: write d12/d1b/d2e/d52/f6b [472236,30276] 0 2024-02-28T23:07:29.947 INFO:tasks.workunit.client.0.smithi057.stdout:7/182: dwrite d0/d5/ff [0,4194304] 0 2024-02-28T23:07:29.950 INFO:tasks.workunit.client.0.smithi057.stdout:9/136: truncate dd/ff 331204 0 2024-02-28T23:07:29.950 INFO:tasks.workunit.client.1.smithi084.stdout:1/354: chown d12/d1b/d2e/d30/l35 989 1 2024-02-28T23:07:29.953 INFO:tasks.workunit.client.0.smithi057.stdout:9/137: readlink dd/de/l1c 0 2024-02-28T23:07:29.956 INFO:tasks.workunit.client.1.smithi084.stdout:6/282: dread d1/f31 [0,4194304] 0 2024-02-28T23:07:29.958 INFO:tasks.workunit.client.1.smithi084.stdout:3/254: dwrite d3/f6 [0,4194304] 0 2024-02-28T23:07:29.958 INFO:tasks.workunit.client.1.smithi084.stdout:6/283: stat d1/d13 0 2024-02-28T23:07:29.960 INFO:tasks.workunit.client.0.smithi057.stdout:1/72: dread d2/f8 [0,4194304] 0 2024-02-28T23:07:29.961 INFO:tasks.workunit.client.1.smithi084.stdout:6/284: stat d1/d13/d2e/d41/f2f 0 2024-02-28T23:07:29.964 INFO:tasks.workunit.client.1.smithi084.stdout:4/235: rename d1/d18/d20/d15/f28 to d1/d18/d20/d14/d30/d47/f49 0 2024-02-28T23:07:29.965 INFO:tasks.workunit.client.1.smithi084.stdout:9/311: truncate d4/d6/d15/d12/f3d 1207746 0 2024-02-28T23:07:29.966 INFO:tasks.workunit.client.1.smithi084.stdout:1/355: unlink d12/d1b/d3b/c41 0 2024-02-28T23:07:29.967 INFO:tasks.workunit.client.0.smithi057.stdout:9/138: link l1 dd/de/d24/l27 0 2024-02-28T23:07:29.967 INFO:tasks.workunit.client.0.smithi057.stdout:1/73: write f0 [1677472,31647] 0 2024-02-28T23:07:29.967 INFO:tasks.workunit.client.1.smithi084.stdout:6/285: write d1/d9/f48 [149645,63188] 0 2024-02-28T23:07:29.970 INFO:tasks.workunit.client.0.smithi057.stdout:9/139: chown dd/de 1156000671 1 2024-02-28T23:07:29.972 INFO:tasks.workunit.client.1.smithi084.stdout:9/312: symlink d4/d32/d3f/l63 0 2024-02-28T23:07:29.976 INFO:tasks.workunit.client.1.smithi084.stdout:6/286: creat d1/d9/d1d/d1e/d33/f5e x:0 0 0 2024-02-28T23:07:29.977 INFO:tasks.workunit.client.1.smithi084.stdout:1/356: mknod d12/d1b/d2e/d52/d61/d74/c7e 0 2024-02-28T23:07:29.977 INFO:tasks.workunit.client.1.smithi084.stdout:6/287: chown d1/f1b 1775 1 2024-02-28T23:07:29.981 INFO:tasks.workunit.client.1.smithi084.stdout:3/255: dread d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:29.984 INFO:tasks.workunit.client.1.smithi084.stdout:3/256: fsync d3/d8/d14/f2e 0 2024-02-28T23:07:29.986 INFO:tasks.workunit.client.0.smithi057.stdout:9/140: truncate dd/de/d14/f17 4071863 0 2024-02-28T23:07:29.986 INFO:tasks.workunit.client.1.smithi084.stdout:6/288: rename d1/d13/d2e/d41/d2c/c2d to d1/d9/d1d/d1e/d33/d5c/c5f 0 2024-02-28T23:07:29.987 INFO:tasks.workunit.client.0.smithi057.stdout:7/183: dwrite d0/f10 [0,4194304] 0 2024-02-28T23:07:29.992 INFO:tasks.workunit.client.1.smithi084.stdout:6/289: symlink d1/d13/d47/l60 0 2024-02-28T23:07:29.994 INFO:tasks.workunit.client.0.smithi057.stdout:1/74: dwrite f0 [0,4194304] 0 2024-02-28T23:07:29.997 INFO:tasks.workunit.client.1.smithi084.stdout:5/278: sync 2024-02-28T23:07:29.998 INFO:tasks.workunit.client.0.smithi057.stdout:7/184: fdatasync d0/d5/f30 0 2024-02-28T23:07:29.998 INFO:tasks.workunit.client.1.smithi084.stdout:1/357: dwrite fc [4194304,4194304] 0 2024-02-28T23:07:30.001 INFO:tasks.workunit.client.0.smithi057.stdout:1/75: stat d2/d7/df/c5 0 2024-02-28T23:07:30.003 INFO:tasks.workunit.client.1.smithi084.stdout:3/257: link d3/d8/d14/d31/d34/c2c d3/d8/d1a/d29/c5f 0 2024-02-28T23:07:30.003 INFO:tasks.workunit.client.1.smithi084.stdout:6/290: creat d1/d9/d1d/f61 x:0 0 0 2024-02-28T23:07:30.004 INFO:tasks.workunit.client.1.smithi084.stdout:5/279: rmdir dc/de/df/d16/d34 39 2024-02-28T23:07:30.008 INFO:tasks.workunit.client.1.smithi084.stdout:3/258: chown d3/d8/d14/c42 1 1 2024-02-28T23:07:30.010 INFO:tasks.workunit.client.1.smithi084.stdout:1/358: creat d12/d1b/d3b/f7f x:0 0 0 2024-02-28T23:07:30.011 INFO:tasks.workunit.client.0.smithi057.stdout:1/76: rename d2/d7/df/c5 to d2/d7/df/c11 0 2024-02-28T23:07:30.014 INFO:tasks.workunit.client.1.smithi084.stdout:5/280: creat dc/de/df/d16/d34/d54/f61 x:0 0 0 2024-02-28T23:07:30.015 INFO:tasks.workunit.client.1.smithi084.stdout:1/359: mknod d12/d1b/d2e/d52/c80 0 2024-02-28T23:07:30.016 INFO:tasks.workunit.client.1.smithi084.stdout:6/291: link d1/d13/d2e/d41/d4d/c59 d1/d13/d47/c62 0 2024-02-28T23:07:30.017 INFO:tasks.workunit.client.0.smithi057.stdout:1/77: mkdir d2/d7/df/db/d12 0 2024-02-28T23:07:30.023 INFO:tasks.workunit.client.0.smithi057.stdout:9/141: dwrite dd/de/d14/f21 [0,4194304] 0 2024-02-28T23:07:30.027 INFO:tasks.workunit.client.1.smithi084.stdout:6/292: rmdir d1/d9/d1d/d1e/d33/d38 39 2024-02-28T23:07:30.027 INFO:tasks.workunit.client.0.smithi057.stdout:1/78: mknod d2/d7/df/db/c13 0 2024-02-28T23:07:30.028 INFO:tasks.workunit.client.1.smithi084.stdout:5/281: getdents dc/de/df/d16 0 2024-02-28T23:07:30.028 INFO:tasks.workunit.client.0.smithi057.stdout:9/142: write dd/de/d14/f1e [1050737,57379] 0 2024-02-28T23:07:30.030 INFO:tasks.workunit.client.0.smithi057.stdout:7/185: write d0/d5/f30 [271422,15248] 0 2024-02-28T23:07:30.032 INFO:tasks.workunit.client.1.smithi084.stdout:6/293: unlink d1/d13/d2e/d41/l4a 0 2024-02-28T23:07:30.032 INFO:tasks.workunit.client.1.smithi084.stdout:5/282: fsync dc/de/df/d16/d34/f4d 0 2024-02-28T23:07:30.032 INFO:tasks.workunit.client.1.smithi084.stdout:9/313: dread d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:30.038 INFO:tasks.workunit.client.1.smithi084.stdout:6/294: creat d1/d13/d2e/d41/d4d/f63 x:0 0 0 2024-02-28T23:07:30.040 INFO:tasks.workunit.client.1.smithi084.stdout:9/314: creat d4/d6/d15/d12/d41/f64 x:0 0 0 2024-02-28T23:07:30.043 INFO:tasks.workunit.client.1.smithi084.stdout:3/259: chown d3/d8/fa 0 1 2024-02-28T23:07:30.045 INFO:tasks.workunit.client.1.smithi084.stdout:6/295: readlink d1/d9/d1d/d1e/d33/d38/l3d 0 2024-02-28T23:07:30.047 INFO:tasks.workunit.client.1.smithi084.stdout:9/315: creat d4/d6/d15/d12/d41/f65 x:0 0 0 2024-02-28T23:07:30.048 INFO:tasks.workunit.client.1.smithi084.stdout:5/283: getdents dc/de/df/d16/d34/d3c 0 2024-02-28T23:07:30.050 INFO:tasks.workunit.client.1.smithi084.stdout:6/296: truncate d1/d9/d1d/d1e/d33/f5e 710597 0 2024-02-28T23:07:30.051 INFO:tasks.workunit.client.0.smithi057.stdout:7/186: dread d0/f23 [0,4194304] 0 2024-02-28T23:07:30.053 INFO:tasks.workunit.client.1.smithi084.stdout:6/297: write d1/d13/d2e/d41/d4d/f63 [939251,86299] 0 2024-02-28T23:07:30.054 INFO:tasks.workunit.client.0.smithi057.stdout:1/79: dwrite d2/f10 [4194304,4194304] 0 2024-02-28T23:07:30.055 INFO:tasks.workunit.client.1.smithi084.stdout:5/284: unlink dc/f21 0 2024-02-28T23:07:30.055 INFO:tasks.workunit.client.1.smithi084.stdout:6/298: readlink d1/d9/d1d/d1e/d33/d5c/l2a 0 2024-02-28T23:07:30.056 INFO:tasks.workunit.client.1.smithi084.stdout:5/285: chown dc 0 1 2024-02-28T23:07:30.060 INFO:tasks.workunit.client.0.smithi057.stdout:9/143: dread dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:30.061 INFO:tasks.workunit.client.0.smithi057.stdout:1/80: creat d2/d7/df/db/f14 x:0 0 0 2024-02-28T23:07:30.063 INFO:tasks.workunit.client.0.smithi057.stdout:7/187: truncate d0/f35 880810 0 2024-02-28T23:07:30.065 INFO:tasks.workunit.client.1.smithi084.stdout:3/260: getdents d3/d8/d1a/d29/d41 0 2024-02-28T23:07:30.066 INFO:tasks.workunit.client.0.smithi057.stdout:7/188: symlink d0/d8/l38 0 2024-02-28T23:07:30.067 INFO:tasks.workunit.client.1.smithi084.stdout:5/286: mknod dc/de/df/d50/c62 0 2024-02-28T23:07:30.068 INFO:tasks.workunit.client.0.smithi057.stdout:4/169: sync 2024-02-28T23:07:30.070 INFO:tasks.workunit.client.1.smithi084.stdout:9/316: link d4/d6/d15/d12/d41/l50 d4/d6/d15/d12/d41/l66 0 2024-02-28T23:07:30.071 INFO:tasks.workunit.client.0.smithi057.stdout:7/189: creat d0/d5/f39 x:0 0 0 2024-02-28T23:07:30.073 INFO:tasks.workunit.client.0.smithi057.stdout:1/81: truncate d2/d7/df/db/fe 720451 0 2024-02-28T23:07:30.075 INFO:tasks.workunit.client.1.smithi084.stdout:6/299: rename d1/d9/f40 to d1/d9/d1d/d1e/d33/f64 0 2024-02-28T23:07:30.076 INFO:tasks.workunit.client.1.smithi084.stdout:1/360: write d12/f15 [4089617,49047] 0 2024-02-28T23:07:30.078 INFO:tasks.workunit.client.0.smithi057.stdout:1/82: chown d2/d7/df/db/c13 4985531 1 2024-02-28T23:07:30.078 INFO:tasks.workunit.client.1.smithi084.stdout:1/361: readlink lb 0 2024-02-28T23:07:30.081 INFO:tasks.workunit.client.1.smithi084.stdout:6/300: mkdir d1/d13/d50/d65 0 2024-02-28T23:07:30.081 INFO:tasks.workunit.client.0.smithi057.stdout:1/83: write d2/f10 [8383764,3224] 0 2024-02-28T23:07:30.081 INFO:tasks.workunit.client.0.smithi057.stdout:4/170: getdents d1a 0 2024-02-28T23:07:30.081 INFO:tasks.workunit.client.0.smithi057.stdout:9/144: dwrite dd/f12 [0,4194304] 0 2024-02-28T23:07:30.083 INFO:tasks.workunit.client.1.smithi084.stdout:3/261: dwrite d3/f28 [0,4194304] 0 2024-02-28T23:07:30.084 INFO:tasks.workunit.client.0.smithi057.stdout:4/171: chown f15 41628 1 2024-02-28T23:07:30.085 INFO:tasks.workunit.client.0.smithi057.stdout:7/190: rename d0/d5/ff to d0/d8/d15/f3a 0 2024-02-28T23:07:30.088 INFO:tasks.workunit.client.1.smithi084.stdout:9/317: rename d4/d6/d15/f19 to d4/d32/f67 0 2024-02-28T23:07:30.090 INFO:tasks.workunit.client.1.smithi084.stdout:5/287: write dc/f15 [879118,83033] 0 2024-02-28T23:07:30.094 INFO:tasks.workunit.client.0.smithi057.stdout:9/145: creat dd/de/d24/f28 x:0 0 0 2024-02-28T23:07:30.096 INFO:tasks.workunit.client.1.smithi084.stdout:5/288: chown dc/de/df/l4f 18091378 1 2024-02-28T23:07:30.096 INFO:tasks.workunit.client.1.smithi084.stdout:9/318: mknod d4/d32/c68 0 2024-02-28T23:07:30.099 INFO:tasks.workunit.client.0.smithi057.stdout:1/84: creat d2/f15 x:0 0 0 2024-02-28T23:07:30.104 INFO:tasks.workunit.client.0.smithi057.stdout:9/146: stat dd/de/c10 0 2024-02-28T23:07:30.104 INFO:tasks.workunit.client.1.smithi084.stdout:6/301: dwrite d1/d13/d47/f56 [0,4194304] 0 2024-02-28T23:07:30.106 INFO:tasks.workunit.client.0.smithi057.stdout:7/191: dwrite d0/d8/d15/f2f [0,4194304] 0 2024-02-28T23:07:30.108 INFO:tasks.workunit.client.0.smithi057.stdout:9/147: chown dd/d1b/l20 251218 1 2024-02-28T23:07:30.109 INFO:tasks.workunit.client.1.smithi084.stdout:5/289: rmdir dc/de/df/d31/d4c 0 2024-02-28T23:07:30.110 INFO:tasks.workunit.client.1.smithi084.stdout:3/262: dwrite d3/d8/f24 [0,4194304] 0 2024-02-28T23:07:30.111 INFO:tasks.workunit.client.0.smithi057.stdout:1/85: getdents d2/d7/df/db/d12 0 2024-02-28T23:07:30.115 INFO:tasks.workunit.client.0.smithi057.stdout:7/192: unlink d0/d8/d15/f2f 0 2024-02-28T23:07:30.117 INFO:tasks.workunit.client.0.smithi057.stdout:9/148: creat dd/de/d14/f29 x:0 0 0 2024-02-28T23:07:30.117 INFO:tasks.workunit.client.0.smithi057.stdout:1/86: write d2/d7/df/db/f14 [860881,10567] 0 2024-02-28T23:07:30.118 INFO:tasks.workunit.client.0.smithi057.stdout:4/172: dwrite d1a/d1b/f32 [0,4194304] 0 2024-02-28T23:07:30.119 INFO:tasks.workunit.client.0.smithi057.stdout:7/193: dread - d0/d8/d15/d29/f32 zero size 2024-02-28T23:07:30.120 INFO:tasks.workunit.client.1.smithi084.stdout:5/290: creat dc/de/df/d31/f63 x:0 0 0 2024-02-28T23:07:30.121 INFO:tasks.workunit.client.1.smithi084.stdout:9/319: fsync d4/d6/d15/f18 0 2024-02-28T23:07:30.123 INFO:tasks.workunit.client.0.smithi057.stdout:7/194: chown d0/d8 0 1 2024-02-28T23:07:30.126 INFO:tasks.workunit.client.0.smithi057.stdout:9/149: dread dd/ff [0,4194304] 0 2024-02-28T23:07:30.128 INFO:tasks.workunit.client.1.smithi084.stdout:6/302: dread d1/f24 [4194304,4194304] 0 2024-02-28T23:07:30.129 INFO:tasks.workunit.client.1.smithi084.stdout:9/320: unlink d4/d32/d3f/d5a/l26 0 2024-02-28T23:07:30.130 INFO:tasks.workunit.client.0.smithi057.stdout:4/173: truncate d1a/d1b/d24/d2e/f3f 1035530 0 2024-02-28T23:07:30.132 INFO:tasks.workunit.client.1.smithi084.stdout:6/303: chown d1/d13/c3c 114969 1 2024-02-28T23:07:30.133 INFO:tasks.workunit.client.0.smithi057.stdout:7/195: getdents d0/d8/d15/d29 0 2024-02-28T23:07:30.134 INFO:tasks.workunit.client.1.smithi084.stdout:3/263: dwrite d3/d8/d1a/f1f [0,4194304] 0 2024-02-28T23:07:30.135 INFO:tasks.workunit.client.0.smithi057.stdout:1/87: rmdir d2 39 2024-02-28T23:07:30.135 INFO:tasks.workunit.client.0.smithi057.stdout:1/88: readlink - no filename 2024-02-28T23:07:30.138 INFO:tasks.workunit.client.1.smithi084.stdout:3/264: mkdir d3/d8/d1a/d29/d41/d60 0 2024-02-28T23:07:30.141 INFO:tasks.workunit.client.0.smithi057.stdout:4/174: creat d1a/d2f/f45 x:0 0 0 2024-02-28T23:07:30.141 INFO:tasks.workunit.client.1.smithi084.stdout:3/265: mkdir d3/d8/d1a/d21/d3d/d43/d61 0 2024-02-28T23:07:30.142 INFO:tasks.workunit.client.1.smithi084.stdout:3/266: stat d3/f7 0 2024-02-28T23:07:30.143 INFO:tasks.workunit.client.0.smithi057.stdout:7/196: mkdir d0/d8/d3b 0 2024-02-28T23:07:30.144 INFO:tasks.workunit.client.0.smithi057.stdout:9/150: link dd/de/d14/c22 dd/c2a 0 2024-02-28T23:07:30.146 INFO:tasks.workunit.client.0.smithi057.stdout:4/175: creat d1a/d1b/f46 x:0 0 0 2024-02-28T23:07:30.148 INFO:tasks.workunit.client.1.smithi084.stdout:9/321: dwrite d4/f60 [0,4194304] 0 2024-02-28T23:07:30.149 INFO:tasks.workunit.client.0.smithi057.stdout:7/197: creat d0/d5/d36/f3c x:0 0 0 2024-02-28T23:07:30.151 INFO:tasks.workunit.client.1.smithi084.stdout:3/267: link f1 d3/d8/d1a/d45/d5e/f62 0 2024-02-28T23:07:30.153 INFO:tasks.workunit.client.1.smithi084.stdout:9/322: readlink d4/l49 0 2024-02-28T23:07:30.156 INFO:tasks.workunit.client.0.smithi057.stdout:7/198: mkdir d0/d8/d15/d29/d3d 0 2024-02-28T23:07:30.158 INFO:tasks.workunit.client.0.smithi057.stdout:4/176: creat d1a/d1b/d24/d2e/d43/f47 x:0 0 0 2024-02-28T23:07:30.162 INFO:tasks.workunit.client.0.smithi057.stdout:7/199: write d0/d8/d15/d29/f32 [796885,67750] 0 2024-02-28T23:07:30.167 INFO:tasks.workunit.client.0.smithi057.stdout:4/177: mkdir d1a/d48 0 2024-02-28T23:07:30.169 INFO:tasks.workunit.client.1.smithi084.stdout:9/323: dwrite d4/d32/d3f/f42 [0,4194304] 0 2024-02-28T23:07:30.170 INFO:tasks.workunit.client.1.smithi084.stdout:3/268: dwrite d3/d8/df/f46 [4194304,4194304] 0 2024-02-28T23:07:30.176 INFO:tasks.workunit.client.1.smithi084.stdout:9/324: mkdir d4/d69 0 2024-02-28T23:07:30.180 INFO:tasks.workunit.client.0.smithi057.stdout:4/178: readlink d1a/d1e/l22 0 2024-02-28T23:07:30.191 INFO:tasks.workunit.client.0.smithi057.stdout:7/200: dwrite d0/d8/d15/d29/f2a [0,4194304] 0 2024-02-28T23:07:30.191 INFO:tasks.workunit.client.0.smithi057.stdout:4/179: write d1a/d1b/f23 [1555207,49786] 0 2024-02-28T23:07:30.192 INFO:tasks.workunit.client.1.smithi084.stdout:5/291: dread dc/de/f2d [0,4194304] 0 2024-02-28T23:07:30.193 INFO:tasks.workunit.client.0.smithi057.stdout:1/89: dwrite d2/d7/df/db/fe [0,4194304] 0 2024-02-28T23:07:30.198 INFO:tasks.workunit.client.1.smithi084.stdout:9/325: truncate d4/f4e 1769963 0 2024-02-28T23:07:30.200 INFO:tasks.workunit.client.0.smithi057.stdout:9/151: dwrite dd/de/d14/f18 [0,4194304] 0 2024-02-28T23:07:30.203 INFO:tasks.workunit.client.1.smithi084.stdout:5/292: mknod dc/de/df/d50/c64 0 2024-02-28T23:07:30.206 INFO:tasks.workunit.client.1.smithi084.stdout:9/326: fdatasync d4/d6/d15/d12/f48 0 2024-02-28T23:07:30.206 INFO:tasks.workunit.client.0.smithi057.stdout:9/152: write dd/de/d14/f1f [936539,27701] 0 2024-02-28T23:07:30.207 INFO:tasks.workunit.client.0.smithi057.stdout:4/180: dread - d1a/f37 zero size 2024-02-28T23:07:30.207 INFO:tasks.workunit.client.1.smithi084.stdout:5/293: fdatasync dc/de/df/d31/f41 0 2024-02-28T23:07:30.209 INFO:tasks.workunit.client.0.smithi057.stdout:7/201: write d0/d5/f1e [5165913,46371] 0 2024-02-28T23:07:30.212 INFO:tasks.workunit.client.1.smithi084.stdout:9/327: rename d4/l49 to d4/d6/d53/l6a 0 2024-02-28T23:07:30.215 INFO:tasks.workunit.client.0.smithi057.stdout:9/153: fdatasync dd/de/d14/f17 0 2024-02-28T23:07:30.216 INFO:tasks.workunit.client.1.smithi084.stdout:5/294: mknod dc/de/df/d16/d34/d3c/c65 0 2024-02-28T23:07:30.217 INFO:tasks.workunit.client.0.smithi057.stdout:1/90: unlink d2/d7/df/db/fe 0 2024-02-28T23:07:30.219 INFO:tasks.workunit.client.1.smithi084.stdout:5/295: write dc/de/df/d16/d34/d54/f5d [745763,84830] 0 2024-02-28T23:07:30.224 INFO:tasks.workunit.client.1.smithi084.stdout:5/296: mkdir dc/de/df/d16/d34/d54/d5f/d66 0 2024-02-28T23:07:30.224 INFO:tasks.workunit.client.0.smithi057.stdout:7/202: dread d0/f23 [0,4194304] 0 2024-02-28T23:07:30.226 INFO:tasks.workunit.client.1.smithi084.stdout:2/236: sync 2024-02-28T23:07:30.227 INFO:tasks.workunit.client.0.smithi057.stdout:9/154: creat dd/de/d14/f2b x:0 0 0 2024-02-28T23:07:30.228 INFO:tasks.workunit.client.1.smithi084.stdout:9/328: dread d4/d32/d3f/f42 [0,4194304] 0 2024-02-28T23:07:30.229 INFO:tasks.workunit.client.1.smithi084.stdout:2/237: unlink f3 0 2024-02-28T23:07:30.232 INFO:tasks.workunit.client.1.smithi084.stdout:2/238: rename d5/d7/f19 to d5/d7/d9/f3c 0 2024-02-28T23:07:30.232 INFO:tasks.workunit.client.1.smithi084.stdout:2/239: stat d5/d7/d12/d24 0 2024-02-28T23:07:30.233 INFO:tasks.workunit.client.1.smithi084.stdout:2/240: write d5/d7/d9/f11 [1179763,5691] 0 2024-02-28T23:07:30.235 INFO:tasks.workunit.client.1.smithi084.stdout:9/329: creat d4/d6/f6b x:0 0 0 2024-02-28T23:07:30.236 INFO:tasks.workunit.client.0.smithi057.stdout:9/155: mknod dd/c2c 0 2024-02-28T23:07:30.237 INFO:tasks.workunit.client.1.smithi084.stdout:9/330: dread - d4/f56 zero size 2024-02-28T23:07:30.237 INFO:tasks.workunit.client.0.smithi057.stdout:0/125: sync 2024-02-28T23:07:30.239 INFO:tasks.workunit.client.1.smithi084.stdout:9/331: chown d4/d32/d3f 622 1 2024-02-28T23:07:30.239 INFO:tasks.workunit.client.0.smithi057.stdout:9/156: fdatasync dd/de/f1d 0 2024-02-28T23:07:30.241 INFO:tasks.workunit.client.0.smithi057.stdout:4/181: dwrite d1a/f36 [4194304,4194304] 0 2024-02-28T23:07:30.243 INFO:tasks.workunit.client.1.smithi084.stdout:2/241: rename f2 to d5/d7/d2e/f3d 0 2024-02-28T23:07:30.244 INFO:tasks.workunit.client.1.smithi084.stdout:9/332: readlink d4/d6/d15/d12/l2b 0 2024-02-28T23:07:30.244 INFO:tasks.workunit.client.1.smithi084.stdout:5/297: rmdir dc/de/df/d16/d34/d54 39 2024-02-28T23:07:30.245 INFO:tasks.workunit.client.0.smithi057.stdout:1/91: dread f0 [0,4194304] 0 2024-02-28T23:07:30.249 INFO:tasks.workunit.client.0.smithi057.stdout:7/203: dwrite d0/d5/d36/f3c [0,4194304] 0 2024-02-28T23:07:30.252 INFO:tasks.workunit.client.0.smithi057.stdout:4/182: mkdir d1a/d1b/d2d/d49 0 2024-02-28T23:07:30.252 INFO:tasks.workunit.client.0.smithi057.stdout:7/204: fsync d0/d5/f30 0 2024-02-28T23:07:30.253 INFO:tasks.workunit.client.0.smithi057.stdout:7/205: readlink d0/l33 0 2024-02-28T23:07:30.254 INFO:tasks.workunit.client.1.smithi084.stdout:2/242: rename d5/d7/d9/f11 to d5/d7/f3e 0 2024-02-28T23:07:30.254 INFO:tasks.workunit.client.1.smithi084.stdout:9/333: truncate d4/d6/d15/d12/f3d 863181 0 2024-02-28T23:07:30.254 INFO:tasks.workunit.client.0.smithi057.stdout:1/92: fdatasync d2/f8 0 2024-02-28T23:07:30.255 INFO:tasks.workunit.client.0.smithi057.stdout:6/134: sync 2024-02-28T23:07:30.256 INFO:tasks.workunit.client.0.smithi057.stdout:8/152: sync 2024-02-28T23:07:30.257 INFO:tasks.workunit.client.0.smithi057.stdout:2/178: sync 2024-02-28T23:07:30.258 INFO:tasks.workunit.client.0.smithi057.stdout:3/90: sync 2024-02-28T23:07:30.258 INFO:tasks.workunit.client.0.smithi057.stdout:5/87: sync 2024-02-28T23:07:30.258 INFO:tasks.workunit.client.1.smithi084.stdout:2/243: write d5/d7/d2e/f2f [463123,90085] 0 2024-02-28T23:07:30.259 INFO:tasks.workunit.client.1.smithi084.stdout:9/334: creat d4/d32/f6c x:0 0 0 2024-02-28T23:07:30.261 INFO:tasks.workunit.client.0.smithi057.stdout:9/157: creat dd/f2d x:0 0 0 2024-02-28T23:07:30.261 INFO:tasks.workunit.client.0.smithi057.stdout:4/183: creat d1a/d1b/d24/d2e/f4a x:0 0 0 2024-02-28T23:07:30.262 INFO:tasks.workunit.client.0.smithi057.stdout:2/179: dread - d8/d11/d21/d28/f3d zero size 2024-02-28T23:07:30.265 INFO:tasks.workunit.client.0.smithi057.stdout:1/93: rmdir d2/d7 39 2024-02-28T23:07:30.266 INFO:tasks.workunit.client.0.smithi057.stdout:6/135: rmdir d0/d1b 39 2024-02-28T23:07:30.266 INFO:tasks.workunit.client.1.smithi084.stdout:9/335: mkdir d4/d6/d15/d12/d6d 0 2024-02-28T23:07:30.267 INFO:tasks.workunit.client.1.smithi084.stdout:2/244: link d5/dd/f34 d5/d7/f3f 0 2024-02-28T23:07:30.268 INFO:tasks.workunit.client.0.smithi057.stdout:0/126: dwrite d17/f18 [0,4194304] 0 2024-02-28T23:07:30.268 INFO:tasks.workunit.client.0.smithi057.stdout:9/158: write dd/de/d14/f1f [988828,57451] 0 2024-02-28T23:07:30.268 INFO:tasks.workunit.client.1.smithi084.stdout:5/298: dwrite dc/de/df/d16/f57 [0,4194304] 0 2024-02-28T23:07:30.269 INFO:tasks.workunit.client.1.smithi084.stdout:2/245: chown d5/d7/d1d 40825 1 2024-02-28T23:07:30.269 INFO:tasks.workunit.client.0.smithi057.stdout:8/153: write d4/f9 [3176872,60765] 0 2024-02-28T23:07:30.270 INFO:tasks.workunit.client.1.smithi084.stdout:5/299: fdatasync dc/de/f14 0 2024-02-28T23:07:30.270 INFO:tasks.workunit.client.0.smithi057.stdout:0/127: chown d17/d20 1546 1 2024-02-28T23:07:30.271 INFO:tasks.workunit.client.0.smithi057.stdout:4/184: mkdir d1a/d1e/d4b 0 2024-02-28T23:07:30.271 INFO:tasks.workunit.client.0.smithi057.stdout:0/128: dread - d17/d20/f2b zero size 2024-02-28T23:07:30.273 INFO:tasks.workunit.client.1.smithi084.stdout:9/336: mknod d4/d32/c6e 0 2024-02-28T23:07:30.276 INFO:tasks.workunit.client.1.smithi084.stdout:5/300: rename dc/f15 to dc/de/df/d50/f67 0 2024-02-28T23:07:30.277 INFO:tasks.workunit.client.0.smithi057.stdout:8/154: rmdir d4/d13/d1a 39 2024-02-28T23:07:30.282 INFO:tasks.workunit.client.0.smithi057.stdout:9/159: mknod dd/de/d24/c2e 0 2024-02-28T23:07:30.283 INFO:tasks.workunit.client.1.smithi084.stdout:2/246: rename d5/d7/d12/d24/f38 to d5/d7/d12/d24/d2a/d35/f40 0 2024-02-28T23:07:30.288 INFO:tasks.workunit.client.1.smithi084.stdout:5/301: symlink dc/de/df/d16/d34/d54/d5f/d66/l68 0 2024-02-28T23:07:30.289 INFO:tasks.workunit.client.1.smithi084.stdout:5/302: fsync dc/de/df/d31/f43 0 2024-02-28T23:07:30.290 INFO:tasks.workunit.client.0.smithi057.stdout:0/129: symlink d17/l2d 0 2024-02-28T23:07:30.292 INFO:tasks.workunit.client.1.smithi084.stdout:5/303: rmdir dc/de/df/d16/d34 39 2024-02-28T23:07:30.294 INFO:tasks.workunit.client.0.smithi057.stdout:8/155: fdatasync d4/d13/f2b 0 2024-02-28T23:07:30.298 INFO:tasks.workunit.client.0.smithi057.stdout:0/130: mknod d17/c2e 0 2024-02-28T23:07:30.301 INFO:tasks.workunit.client.0.smithi057.stdout:3/91: truncate d3/d4/d7/f10 568470 0 2024-02-28T23:07:30.302 INFO:tasks.workunit.client.1.smithi084.stdout:2/247: dwrite d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:30.306 INFO:tasks.workunit.client.0.smithi057.stdout:8/156: write d4/f7 [10178706,114126] 0 2024-02-28T23:07:30.307 INFO:tasks.workunit.client.0.smithi057.stdout:5/88: dwrite d4/d7/fe [0,4194304] 0 2024-02-28T23:07:30.311 INFO:tasks.workunit.client.0.smithi057.stdout:4/185: truncate d1a/d1e/f29 2576009 0 2024-02-28T23:07:30.316 INFO:tasks.workunit.client.0.smithi057.stdout:5/89: fsync d4/d7/d8/ff 0 2024-02-28T23:07:30.317 INFO:tasks.workunit.client.0.smithi057.stdout:0/131: creat d17/d19/f2f x:0 0 0 2024-02-28T23:07:30.317 INFO:tasks.workunit.client.0.smithi057.stdout:4/186: chown d1a/d1b/d24/d2e 26 1 2024-02-28T23:07:30.318 INFO:tasks.workunit.client.0.smithi057.stdout:7/206: dwrite d0/f35 [0,4194304] 0 2024-02-28T23:07:30.321 INFO:tasks.workunit.client.1.smithi084.stdout:9/337: dwrite d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:30.324 INFO:tasks.workunit.client.0.smithi057.stdout:9/160: getdents dd/d1b 0 2024-02-28T23:07:30.328 INFO:tasks.workunit.client.1.smithi084.stdout:9/338: symlink d4/d6/d15/l6f 0 2024-02-28T23:07:30.329 INFO:tasks.workunit.client.0.smithi057.stdout:1/94: link d2/d7/cd d2/d7/df/db/c16 0 2024-02-28T23:07:30.330 INFO:tasks.workunit.client.1.smithi084.stdout:9/339: read d4/d32/d3f/d5a/d2d/f34 [4492292,76999] 0 2024-02-28T23:07:30.332 INFO:tasks.workunit.client.1.smithi084.stdout:2/248: dwrite d5/d7/f3f [0,4194304] 0 2024-02-28T23:07:30.332 INFO:tasks.workunit.client.1.smithi084.stdout:9/340: fdatasync d4/f60 0 2024-02-28T23:07:30.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/207: fdatasync d0/f10 0 2024-02-28T23:07:30.332 INFO:tasks.workunit.client.0.smithi057.stdout:8/157: creat d4/d13/d1a/f34 x:0 0 0 2024-02-28T23:07:30.338 INFO:tasks.workunit.client.0.smithi057.stdout:0/132: mkdir d17/d20/d26/d30 0 2024-02-28T23:07:30.341 INFO:tasks.workunit.client.1.smithi084.stdout:2/249: rmdir d5 39 2024-02-28T23:07:30.342 INFO:tasks.workunit.client.0.smithi057.stdout:3/92: dwrite d3/fd [0,4194304] 0 2024-02-28T23:07:30.344 INFO:tasks.workunit.client.0.smithi057.stdout:4/187: write d1a/d1b/f21 [2207816,31941] 0 2024-02-28T23:07:30.344 INFO:tasks.workunit.client.0.smithi057.stdout:1/95: link d2/f15 d2/d7/df/db/f17 0 2024-02-28T23:07:30.345 INFO:tasks.workunit.client.1.smithi084.stdout:5/304: dread dc/de/df/f20 [0,4194304] 0 2024-02-28T23:07:30.346 INFO:tasks.workunit.client.0.smithi057.stdout:7/208: chown d0/d8/d3b 84427 1 2024-02-28T23:07:30.346 INFO:tasks.workunit.client.0.smithi057.stdout:9/161: truncate dd/de/d14/f18 4510088 0 2024-02-28T23:07:30.350 INFO:tasks.workunit.client.1.smithi084.stdout:9/341: dread d4/f60 [0,4194304] 0 2024-02-28T23:07:30.352 INFO:tasks.workunit.client.0.smithi057.stdout:6/136: dwrite d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:30.352 INFO:tasks.workunit.client.0.smithi057.stdout:7/209: chown d0/d5/f30 509540895 1 2024-02-28T23:07:30.356 INFO:tasks.workunit.client.1.smithi084.stdout:5/305: creat dc/de/df/d16/d34/d54/f69 x:0 0 0 2024-02-28T23:07:30.357 INFO:tasks.workunit.client.1.smithi084.stdout:9/342: dread - d4/d6/d15/d12/d41/f65 zero size 2024-02-28T23:07:30.358 INFO:tasks.workunit.client.1.smithi084.stdout:5/306: chown f7 3992371 1 2024-02-28T23:07:30.359 INFO:tasks.workunit.client.0.smithi057.stdout:3/93: symlink d3/d4/d7/l1c 0 2024-02-28T23:07:30.361 INFO:tasks.workunit.client.0.smithi057.stdout:6/137: write d0/d4/d11/f13 [308859,10812] 0 2024-02-28T23:07:30.363 INFO:tasks.workunit.client.0.smithi057.stdout:8/158: mknod d4/d13/d2c/c35 0 2024-02-28T23:07:30.366 INFO:tasks.workunit.client.1.smithi084.stdout:9/343: creat d4/d6/d15/d3c/f70 x:0 0 0 2024-02-28T23:07:30.370 INFO:tasks.workunit.client.1.smithi084.stdout:2/250: dwrite d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:30.371 INFO:tasks.workunit.client.0.smithi057.stdout:1/96: fdatasync d2/d7/df/db/f17 0 2024-02-28T23:07:30.371 INFO:tasks.workunit.client.0.smithi057.stdout:0/133: creat d17/d20/d26/d30/f31 x:0 0 0 2024-02-28T23:07:30.371 INFO:tasks.workunit.client.0.smithi057.stdout:4/188: symlink d1a/d1b/d2d/d49/l4c 0 2024-02-28T23:07:30.372 INFO:tasks.workunit.client.0.smithi057.stdout:5/90: truncate d4/d7/fe 3175881 0 2024-02-28T23:07:30.373 INFO:tasks.workunit.client.1.smithi084.stdout:9/344: creat d4/d6/d15/f71 x:0 0 0 2024-02-28T23:07:30.378 INFO:tasks.workunit.client.1.smithi084.stdout:9/345: creat d4/d6/f72 x:0 0 0 2024-02-28T23:07:30.382 INFO:tasks.workunit.client.1.smithi084.stdout:5/307: dwrite dc/de/df/d16/d34/d54/f61 [0,4194304] 0 2024-02-28T23:07:30.383 INFO:tasks.workunit.client.1.smithi084.stdout:9/346: rmdir d4/d6/d15/d12 39 2024-02-28T23:07:30.389 INFO:tasks.workunit.client.1.smithi084.stdout:7/256: sync 2024-02-28T23:07:30.390 INFO:tasks.workunit.client.1.smithi084.stdout:9/347: mknod d4/d32/d3f/c73 0 2024-02-28T23:07:30.390 INFO:tasks.workunit.client.1.smithi084.stdout:2/251: dread d5/dd/f34 [0,4194304] 0 2024-02-28T23:07:30.391 INFO:tasks.workunit.client.1.smithi084.stdout:7/257: stat db/d11/d27/f3a 0 2024-02-28T23:07:30.394 INFO:tasks.workunit.client.1.smithi084.stdout:9/348: chown d4/d6/d15/d12/l5e 1159197 1 2024-02-28T23:07:30.396 INFO:tasks.workunit.client.1.smithi084.stdout:7/258: symlink db/d10/d18/l52 0 2024-02-28T23:07:30.396 INFO:tasks.workunit.client.1.smithi084.stdout:9/349: write d4/f56 [862972,124981] 0 2024-02-28T23:07:30.398 INFO:tasks.workunit.client.1.smithi084.stdout:7/259: stat db/d10/d18/c2a 0 2024-02-28T23:07:30.403 INFO:tasks.workunit.client.0.smithi057.stdout:3/94: chown d3/c19 47100 1 2024-02-28T23:07:30.404 INFO:tasks.workunit.client.0.smithi057.stdout:6/138: creat d0/d4/d25/f2a x:0 0 0 2024-02-28T23:07:30.404 INFO:tasks.workunit.client.0.smithi057.stdout:8/159: creat d4/d1e/f36 x:0 0 0 2024-02-28T23:07:30.405 INFO:tasks.workunit.client.1.smithi084.stdout:5/308: dwrite dc/de/df/d16/d34/d54/f69 [0,4194304] 0 2024-02-28T23:07:30.406 INFO:tasks.workunit.client.0.smithi057.stdout:9/162: truncate dd/f12 79700 0 2024-02-28T23:07:30.408 INFO:tasks.workunit.client.0.smithi057.stdout:5/91: creat d4/d9/d16/f18 x:0 0 0 2024-02-28T23:07:30.408 INFO:tasks.workunit.client.1.smithi084.stdout:7/260: rmdir db 39 2024-02-28T23:07:30.409 INFO:tasks.workunit.client.0.smithi057.stdout:8/160: chown d4/d13/d18/f31 27 1 2024-02-28T23:07:30.409 INFO:tasks.workunit.client.1.smithi084.stdout:2/252: write d5/d7/d2e/f3d [842452,119675] 0 2024-02-28T23:07:30.410 INFO:tasks.workunit.client.0.smithi057.stdout:5/92: write d4/f15 [1777241,114741] 0 2024-02-28T23:07:30.412 INFO:tasks.workunit.client.0.smithi057.stdout:1/97: creat d2/d7/df/db/f18 x:0 0 0 2024-02-28T23:07:30.412 INFO:tasks.workunit.client.1.smithi084.stdout:5/309: creat dc/de/df/d16/f6a x:0 0 0 2024-02-28T23:07:30.413 INFO:tasks.workunit.client.0.smithi057.stdout:3/95: mkdir d3/d4/d1d 0 2024-02-28T23:07:30.414 INFO:tasks.workunit.client.1.smithi084.stdout:2/253: creat d5/d1f/f41 x:0 0 0 2024-02-28T23:07:30.415 INFO:tasks.workunit.client.1.smithi084.stdout:2/254: readlink d5/d1f/l36 0 2024-02-28T23:07:30.415 INFO:tasks.workunit.client.0.smithi057.stdout:6/139: creat d0/d17/f2b x:0 0 0 2024-02-28T23:07:30.416 INFO:tasks.workunit.client.1.smithi084.stdout:2/255: stat d5/d7/f3f 0 2024-02-28T23:07:30.419 INFO:tasks.workunit.client.0.smithi057.stdout:9/163: chown dd/de/c10 284988 1 2024-02-28T23:07:30.419 INFO:tasks.workunit.client.1.smithi084.stdout:2/256: mkdir d5/d1f/d42 0 2024-02-28T23:07:30.422 INFO:tasks.workunit.client.0.smithi057.stdout:6/140: dread - d0/d4/d25/f2a zero size 2024-02-28T23:07:30.422 INFO:tasks.workunit.client.1.smithi084.stdout:2/257: unlink d5/d1f/l25 0 2024-02-28T23:07:30.423 INFO:tasks.workunit.client.0.smithi057.stdout:5/93: rename d4/d9/d14/f17 to d4/d9/d16/f19 0 2024-02-28T23:07:30.423 INFO:tasks.workunit.client.1.smithi084.stdout:7/261: truncate db/d10/d18/f1b 492148 0 2024-02-28T23:07:30.425 INFO:tasks.workunit.client.0.smithi057.stdout:0/134: getdents d17/d20/d23 0 2024-02-28T23:07:30.425 INFO:tasks.workunit.client.1.smithi084.stdout:7/262: read - db/d10/d17/d3b/f3f zero size 2024-02-28T23:07:30.426 INFO:tasks.workunit.client.1.smithi084.stdout:2/258: write d5/d7/f3f [4798319,58435] 0 2024-02-28T23:07:30.426 INFO:tasks.workunit.client.0.smithi057.stdout:0/135: dread - f14 zero size 2024-02-28T23:07:30.428 INFO:tasks.workunit.client.0.smithi057.stdout:6/141: creat d0/d17/f2c x:0 0 0 2024-02-28T23:07:30.430 INFO:tasks.workunit.client.0.smithi057.stdout:5/94: creat d4/d9/d16/f1a x:0 0 0 2024-02-28T23:07:30.430 INFO:tasks.workunit.client.1.smithi084.stdout:5/310: dwrite dc/de/df/d31/f4e [0,4194304] 0 2024-02-28T23:07:30.430 INFO:tasks.workunit.client.1.smithi084.stdout:2/259: read d5/d7/d2e/f3d [909457,104992] 0 2024-02-28T23:07:30.431 INFO:tasks.workunit.client.0.smithi057.stdout:8/161: dread d4/d13/d1a/f32 [0,4194304] 0 2024-02-28T23:07:30.431 INFO:tasks.workunit.client.0.smithi057.stdout:3/96: creat d3/d4/d1d/f1e x:0 0 0 2024-02-28T23:07:30.432 INFO:tasks.workunit.client.1.smithi084.stdout:7/263: link db/d10/f2f db/d11/d27/f53 0 2024-02-28T23:07:30.433 INFO:tasks.workunit.client.0.smithi057.stdout:9/164: creat dd/f2f x:0 0 0 2024-02-28T23:07:30.435 INFO:tasks.workunit.client.0.smithi057.stdout:6/142: creat d0/d4/d11/f2d x:0 0 0 2024-02-28T23:07:30.441 INFO:tasks.workunit.client.0.smithi057.stdout:8/162: mkdir d4/d1e/d37 0 2024-02-28T23:07:30.442 INFO:tasks.workunit.client.0.smithi057.stdout:6/143: mknod d0/d17/d20/c2e 0 2024-02-28T23:07:30.442 INFO:tasks.workunit.client.1.smithi084.stdout:2/260: creat d5/d7/d12/d24/d3a/f43 x:0 0 0 2024-02-28T23:07:30.443 INFO:tasks.workunit.client.1.smithi084.stdout:5/311: rename dc/de/df/f2c to dc/f6b 0 2024-02-28T23:07:30.443 INFO:tasks.workunit.client.1.smithi084.stdout:7/264: dread db/d10/d17/d3b/f3c [0,4194304] 0 2024-02-28T23:07:30.444 INFO:tasks.workunit.client.0.smithi057.stdout:3/97: symlink d3/d8/l1f 0 2024-02-28T23:07:30.445 INFO:tasks.workunit.client.1.smithi084.stdout:5/312: write dc/de/f1a [3272520,107586] 0 2024-02-28T23:07:30.445 INFO:tasks.workunit.client.1.smithi084.stdout:2/261: creat d5/d1f/f44 x:0 0 0 2024-02-28T23:07:30.446 INFO:tasks.workunit.client.0.smithi057.stdout:8/163: fdatasync d4/f11 0 2024-02-28T23:07:30.447 INFO:tasks.workunit.client.1.smithi084.stdout:2/262: fsync d5/f15 0 2024-02-28T23:07:30.452 INFO:tasks.workunit.client.1.smithi084.stdout:2/263: unlink d5/d7/f3e 0 2024-02-28T23:07:30.452 INFO:tasks.workunit.client.0.smithi057.stdout:6/144: creat d0/d4/d25/f2f x:0 0 0 2024-02-28T23:07:30.453 INFO:tasks.workunit.client.0.smithi057.stdout:4/189: dread d1a/d1b/f23 [0,4194304] 0 2024-02-28T23:07:30.454 INFO:tasks.workunit.client.1.smithi084.stdout:2/264: chown d5/d1f/d42 18585 1 2024-02-28T23:07:30.454 INFO:tasks.workunit.client.0.smithi057.stdout:3/98: rmdir d3/d4/d7 39 2024-02-28T23:07:30.454 INFO:tasks.workunit.client.1.smithi084.stdout:7/265: getdents db 0 2024-02-28T23:07:30.456 INFO:tasks.workunit.client.0.smithi057.stdout:0/136: dread d17/d19/f28 [0,4194304] 0 2024-02-28T23:07:30.456 INFO:tasks.workunit.client.1.smithi084.stdout:7/266: chown db/l4a 4084738 1 2024-02-28T23:07:30.458 INFO:tasks.workunit.client.1.smithi084.stdout:7/267: fsync db/d10/d18/f47 0 2024-02-28T23:07:30.458 INFO:tasks.workunit.client.1.smithi084.stdout:5/313: rename dc/de/f12 to dc/de/df/d16/d34/f6c 0 2024-02-28T23:07:30.460 INFO:tasks.workunit.client.0.smithi057.stdout:1/98: dwrite d2/d7/df/db/f17 [0,4194304] 0 2024-02-28T23:07:30.462 INFO:tasks.workunit.client.0.smithi057.stdout:3/99: mkdir d3/d20 0 2024-02-28T23:07:30.462 INFO:tasks.workunit.client.1.smithi084.stdout:5/314: unlink l8 0 2024-02-28T23:07:30.463 INFO:tasks.workunit.client.0.smithi057.stdout:2/180: sync 2024-02-28T23:07:30.463 INFO:tasks.workunit.client.1.smithi084.stdout:7/268: fdatasync db/ff 0 2024-02-28T23:07:30.464 INFO:tasks.workunit.client.0.smithi057.stdout:6/145: symlink d0/d17/d20/l30 0 2024-02-28T23:07:30.467 INFO:tasks.workunit.client.0.smithi057.stdout:5/95: dwrite d4/d7/fe [0,4194304] 0 2024-02-28T23:07:30.468 INFO:tasks.workunit.client.1.smithi084.stdout:7/269: mknod db/d10/c54 0 2024-02-28T23:07:30.472 INFO:tasks.workunit.client.0.smithi057.stdout:9/165: write dd/ff [89599,48260] 0 2024-02-28T23:07:30.474 INFO:tasks.workunit.client.1.smithi084.stdout:7/270: link db/d11/c34 db/d11/c55 0 2024-02-28T23:07:30.476 INFO:tasks.workunit.client.0.smithi057.stdout:8/164: truncate d4/f17 1105061 0 2024-02-28T23:07:30.477 INFO:tasks.workunit.client.1.smithi084.stdout:7/271: unlink db/d11/f15 0 2024-02-28T23:07:30.477 INFO:tasks.workunit.client.0.smithi057.stdout:0/137: write d17/d19/f28 [622290,38639] 0 2024-02-28T23:07:30.478 INFO:tasks.workunit.client.0.smithi057.stdout:1/99: mkdir d2/d7/df/db/d12/d19 0 2024-02-28T23:07:30.480 INFO:tasks.workunit.client.0.smithi057.stdout:8/165: chown d4/d13/d18/f26 174468 1 2024-02-28T23:07:30.481 INFO:tasks.workunit.client.1.smithi084.stdout:7/272: creat db/d10/f56 x:0 0 0 2024-02-28T23:07:30.481 INFO:tasks.workunit.client.0.smithi057.stdout:9/166: readlink dd/de/d24/l27 0 2024-02-28T23:07:30.484 INFO:tasks.workunit.client.0.smithi057.stdout:0/138: write fe [528745,27299] 0 2024-02-28T23:07:30.484 INFO:tasks.workunit.client.0.smithi057.stdout:2/181: creat d8/d11/f45 x:0 0 0 2024-02-28T23:07:30.487 INFO:tasks.workunit.client.0.smithi057.stdout:1/100: chown d2/d7/df/db 27 1 2024-02-28T23:07:30.488 INFO:tasks.workunit.client.1.smithi084.stdout:7/273: mknod db/d10/c57 0 2024-02-28T23:07:30.490 INFO:tasks.workunit.client.1.smithi084.stdout:5/315: truncate dc/de/df/f20 2952458 0 2024-02-28T23:07:30.492 INFO:tasks.workunit.client.1.smithi084.stdout:7/274: dread - db/d10/d17/d3b/f48 zero size 2024-02-28T23:07:30.492 INFO:tasks.workunit.client.0.smithi057.stdout:0/139: chown l7 793 1 2024-02-28T23:07:30.493 INFO:tasks.workunit.client.1.smithi084.stdout:5/316: readlink dc/de/df/d16/d34/l56 0 2024-02-28T23:07:30.494 INFO:tasks.workunit.client.0.smithi057.stdout:5/96: getdents d4/d9 0 2024-02-28T23:07:30.494 INFO:tasks.workunit.client.0.smithi057.stdout:9/167: symlink dd/de/d24/l30 0 2024-02-28T23:07:30.495 INFO:tasks.workunit.client.1.smithi084.stdout:5/317: write dc/de/f1a [2386621,25443] 0 2024-02-28T23:07:30.496 INFO:tasks.workunit.client.1.smithi084.stdout:7/275: creat db/d10/d17/f58 x:0 0 0 2024-02-28T23:07:30.499 INFO:tasks.workunit.client.1.smithi084.stdout:2/265: dwrite d5/f17 [0,4194304] 0 2024-02-28T23:07:30.502 INFO:tasks.workunit.client.0.smithi057.stdout:4/190: dwrite d1a/d1b/f32 [0,4194304] 0 2024-02-28T23:07:30.505 INFO:tasks.workunit.client.0.smithi057.stdout:9/168: stat dd/de/d14/f1e 0 2024-02-28T23:07:30.506 INFO:tasks.workunit.client.0.smithi057.stdout:1/101: rmdir d2/d7 39 2024-02-28T23:07:30.508 INFO:tasks.workunit.client.1.smithi084.stdout:5/318: rename dc/de/df/c2e to dc/de/df/d16/d34/d54/d5f/d66/c6d 0 2024-02-28T23:07:30.508 INFO:tasks.workunit.client.0.smithi057.stdout:2/182: dread d8/d11/f32 [0,4194304] 0 2024-02-28T23:07:30.508 INFO:tasks.workunit.client.0.smithi057.stdout:9/169: readlink dd/de/d24/l30 0 2024-02-28T23:07:30.509 INFO:tasks.workunit.client.0.smithi057.stdout:3/100: link d3/d4/d7/l1c d3/d4/l21 0 2024-02-28T23:07:30.509 INFO:tasks.workunit.client.0.smithi057.stdout:7/210: sync 2024-02-28T23:07:30.511 INFO:tasks.workunit.client.1.smithi084.stdout:2/266: link d5/d7/d1d/l33 d5/dd/l45 0 2024-02-28T23:07:30.515 INFO:tasks.workunit.client.1.smithi084.stdout:2/267: write d5/d7/f3f [740929,79485] 0 2024-02-28T23:07:30.515 INFO:tasks.workunit.client.0.smithi057.stdout:6/146: dwrite d0/d4/ff [0,4194304] 0 2024-02-28T23:07:30.516 INFO:tasks.workunit.client.1.smithi084.stdout:7/276: dwrite db/d10/d18/f47 [0,4194304] 0 2024-02-28T23:07:30.518 INFO:tasks.workunit.client.1.smithi084.stdout:7/277: symlink db/l59 0 2024-02-28T23:07:30.520 INFO:tasks.workunit.client.0.smithi057.stdout:9/170: readlink dd/de/d24/l30 0 2024-02-28T23:07:30.520 INFO:tasks.workunit.client.0.smithi057.stdout:3/101: chown d3/d4/d7/le 1202825167 1 2024-02-28T23:07:30.520 INFO:tasks.workunit.client.0.smithi057.stdout:1/102: mknod d2/c1a 0 2024-02-28T23:07:30.520 INFO:tasks.workunit.client.0.smithi057.stdout:0/140: link f15 d17/d20/d23/f32 0 2024-02-28T23:07:30.521 INFO:tasks.workunit.client.0.smithi057.stdout:2/183: mknod d8/d11/d39/c46 0 2024-02-28T23:07:30.522 INFO:tasks.workunit.client.0.smithi057.stdout:9/171: chown c9 3927943 1 2024-02-28T23:07:30.525 INFO:tasks.workunit.client.0.smithi057.stdout:3/102: stat d3/d8/f16 0 2024-02-28T23:07:30.531 INFO:tasks.workunit.client.1.smithi084.stdout:5/319: dwrite dc/de/df/d16/d34/d54/f69 [4194304,4194304] 0 2024-02-28T23:07:30.534 INFO:tasks.workunit.client.0.smithi057.stdout:3/103: read d3/fd [1106075,75687] 0 2024-02-28T23:07:30.535 INFO:tasks.workunit.client.0.smithi057.stdout:9/172: truncate dd/de/d14/f29 1010244 0 2024-02-28T23:07:30.536 INFO:tasks.workunit.client.1.smithi084.stdout:2/268: fsync d5/d7/f3f 0 2024-02-28T23:07:30.537 INFO:tasks.workunit.client.1.smithi084.stdout:5/320: dread - dc/de/df/d16/f6a zero size 2024-02-28T23:07:30.539 INFO:tasks.workunit.client.0.smithi057.stdout:0/141: rename cb to d17/d20/d23/c33 0 2024-02-28T23:07:30.540 INFO:tasks.workunit.client.0.smithi057.stdout:6/147: creat d0/d17/d20/f31 x:0 0 0 2024-02-28T23:07:30.541 INFO:tasks.workunit.client.0.smithi057.stdout:5/97: dwrite d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:30.543 INFO:tasks.workunit.client.0.smithi057.stdout:9/173: fdatasync dd/de/d14/f29 0 2024-02-28T23:07:30.544 INFO:tasks.workunit.client.0.smithi057.stdout:2/184: creat d8/d11/d21/d28/f47 x:0 0 0 2024-02-28T23:07:30.544 INFO:tasks.workunit.client.1.smithi084.stdout:5/321: mkdir dc/de/df/d6e 0 2024-02-28T23:07:30.547 INFO:tasks.workunit.client.0.smithi057.stdout:8/166: truncate d4/f9 376088 0 2024-02-28T23:07:30.551 INFO:tasks.workunit.client.0.smithi057.stdout:8/167: fsync d4/d13/f2d 0 2024-02-28T23:07:30.552 INFO:tasks.workunit.client.0.smithi057.stdout:3/104: stat d3/d4/f12 0 2024-02-28T23:07:30.556 INFO:tasks.workunit.client.1.smithi084.stdout:5/322: link dc/de/df/d16/l3e dc/de/df/d50/l6f 0 2024-02-28T23:07:30.556 INFO:tasks.workunit.client.1.smithi084.stdout:2/269: dwrite d5/f17 [0,4194304] 0 2024-02-28T23:07:30.557 INFO:tasks.workunit.client.0.smithi057.stdout:9/174: write dd/de/d14/f1e [4445944,44085] 0 2024-02-28T23:07:30.557 INFO:tasks.workunit.client.0.smithi057.stdout:2/185: mkdir d8/d11/d48 0 2024-02-28T23:07:30.559 INFO:tasks.workunit.client.0.smithi057.stdout:3/105: chown d3/d4 2 1 2024-02-28T23:07:30.563 INFO:tasks.workunit.client.0.smithi057.stdout:8/168: write d4/d13/d18/f26 [455826,92351] 0 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:0/247: sync 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:3/269: sync 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:6/304: sync 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:4/236: sync 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:1/362: sync 2024-02-28T23:07:30.564 INFO:tasks.workunit.client.1.smithi084.stdout:8/242: sync 2024-02-28T23:07:30.565 INFO:tasks.workunit.client.1.smithi084.stdout:2/270: rename d5/d7/d12/d24/d3a/f43 to d5/d7/d2e/f46 0 2024-02-28T23:07:30.565 INFO:tasks.workunit.client.1.smithi084.stdout:6/305: stat d1/c5a 0 2024-02-28T23:07:30.566 INFO:tasks.workunit.client.1.smithi084.stdout:5/323: creat dc/f70 x:0 0 0 2024-02-28T23:07:30.567 INFO:tasks.workunit.client.0.smithi057.stdout:7/211: dwrite d0/d8/d15/f22 [0,4194304] 0 2024-02-28T23:07:30.567 INFO:tasks.workunit.client.0.smithi057.stdout:0/142: symlink d17/d20/d26/d30/l34 0 2024-02-28T23:07:30.567 INFO:tasks.workunit.client.0.smithi057.stdout:5/98: rmdir d4/d9/d16 39 2024-02-28T23:07:30.569 INFO:tasks.workunit.client.1.smithi084.stdout:0/248: mkdir d1/d8/d18/d4c 0 2024-02-28T23:07:30.569 INFO:tasks.workunit.client.1.smithi084.stdout:4/237: write d1/d18/d20/d15/f16 [550660,104897] 0 2024-02-28T23:07:30.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:30 smithi057 ceph-mon[29096]: pgmap v21: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 5.1 MiB/s rd, 41 MiB/s wr, 271 op/s 2024-02-28T23:07:30.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:30 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:30.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:30 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:30.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:30 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:07:30.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:30 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:07:30.571 INFO:tasks.workunit.client.1.smithi084.stdout:1/363: symlink d12/d1b/d3b/l81 0 2024-02-28T23:07:30.574 INFO:tasks.workunit.client.1.smithi084.stdout:1/364: stat d12/d1b/d2e/d30/l39 0 2024-02-28T23:07:30.576 INFO:tasks.workunit.client.1.smithi084.stdout:3/270: symlink d3/d8/d14/l63 0 2024-02-28T23:07:30.577 INFO:tasks.workunit.client.1.smithi084.stdout:1/365: write d12/d1b/d2e/f57 [123718,21065] 0 2024-02-28T23:07:30.579 INFO:tasks.workunit.client.1.smithi084.stdout:2/271: creat d5/d1f/d42/f47 x:0 0 0 2024-02-28T23:07:30.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:30 smithi084 ceph-mon[34826]: pgmap v21: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 5.1 MiB/s rd, 41 MiB/s wr, 271 op/s 2024-02-28T23:07:30.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:30 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:30.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:30 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:30.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:30 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:07:30.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:30 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:07:30.582 INFO:tasks.workunit.client.0.smithi057.stdout:7/212: mkdir d0/d5/d25/d3e 0 2024-02-28T23:07:30.582 INFO:tasks.workunit.client.0.smithi057.stdout:1/103: dwrite d2/d7/df/db/f14 [0,4194304] 0 2024-02-28T23:07:30.582 INFO:tasks.workunit.client.1.smithi084.stdout:8/243: mkdir d0/d9/d1b/d38/d43/d51/d54 0 2024-02-28T23:07:30.582 INFO:tasks.workunit.client.1.smithi084.stdout:1/366: write d12/f28 [2495464,121473] 0 2024-02-28T23:07:30.584 INFO:tasks.workunit.client.0.smithi057.stdout:8/169: unlink d4/d1e/c29 0 2024-02-28T23:07:30.588 INFO:tasks.workunit.client.1.smithi084.stdout:2/272: mkdir d5/d7/d12/d24/d3a/d48 0 2024-02-28T23:07:30.588 INFO:tasks.workunit.client.1.smithi084.stdout:3/271: creat d3/d8/d1a/d29/d41/f64 x:0 0 0 2024-02-28T23:07:30.589 INFO:tasks.workunit.client.1.smithi084.stdout:2/273: fsync d5/d1f/f44 0 2024-02-28T23:07:30.591 INFO:tasks.workunit.client.1.smithi084.stdout:3/272: read d3/d8/d1a/f1f [1409236,111946] 0 2024-02-28T23:07:30.593 INFO:tasks.workunit.client.1.smithi084.stdout:8/244: symlink d0/d44/l55 0 2024-02-28T23:07:30.594 INFO:tasks.workunit.client.1.smithi084.stdout:3/273: readlink d3/d8/l9 0 2024-02-28T23:07:30.594 INFO:tasks.workunit.client.0.smithi057.stdout:6/148: dwrite d0/d4/d25/f2f [0,4194304] 0 2024-02-28T23:07:30.595 INFO:tasks.workunit.client.1.smithi084.stdout:8/245: stat d0/d9/d1b/c22 0 2024-02-28T23:07:30.595 INFO:tasks.workunit.client.1.smithi084.stdout:3/274: write d3/d8/d14/f5c [125459,99017] 0 2024-02-28T23:07:30.599 INFO:tasks.workunit.client.1.smithi084.stdout:5/324: creat dc/de/df/f71 x:0 0 0 2024-02-28T23:07:30.600 INFO:tasks.workunit.client.0.smithi057.stdout:6/149: fdatasync d0/d17/f2c 0 2024-02-28T23:07:30.602 INFO:tasks.workunit.client.1.smithi084.stdout:4/238: getdents d1/d18/d20/d14/d30 0 2024-02-28T23:07:30.605 INFO:tasks.workunit.client.0.smithi057.stdout:7/213: dread d0/d8/d15/f2b [0,4194304] 0 2024-02-28T23:07:30.605 INFO:tasks.workunit.client.1.smithi084.stdout:8/246: symlink d0/d11/l56 0 2024-02-28T23:07:30.606 INFO:tasks.workunit.client.1.smithi084.stdout:1/367: getdents d12/d1b/d63/d7d 0 2024-02-28T23:07:30.606 INFO:tasks.workunit.client.0.smithi057.stdout:3/106: link d3/d4/d7/l1c d3/d4/l22 0 2024-02-28T23:07:30.609 INFO:tasks.workunit.client.0.smithi057.stdout:0/143: dread d17/d20/f27 [0,4194304] 0 2024-02-28T23:07:30.609 INFO:tasks.workunit.client.1.smithi084.stdout:5/325: unlink dc/de/df/d31/c3b 0 2024-02-28T23:07:30.609 INFO:tasks.workunit.client.1.smithi084.stdout:1/368: fsync d12/d1b/d2e/f57 0 2024-02-28T23:07:30.613 INFO:tasks.workunit.client.0.smithi057.stdout:5/99: dread d4/d7/d8/f13 [0,4194304] 0 2024-02-28T23:07:30.616 INFO:tasks.workunit.client.1.smithi084.stdout:2/274: creat d5/d7/d12/d24/d3a/d48/f49 x:0 0 0 2024-02-28T23:07:30.617 INFO:tasks.workunit.client.0.smithi057.stdout:8/170: dwrite d4/d13/f2d [0,4194304] 0 2024-02-28T23:07:30.617 INFO:tasks.workunit.client.0.smithi057.stdout:0/144: readlink d17/l25 0 2024-02-28T23:07:30.620 INFO:tasks.workunit.client.1.smithi084.stdout:3/275: creat d3/d8/d1a/d21/d3d/d43/d61/f65 x:0 0 0 2024-02-28T23:07:30.627 INFO:tasks.workunit.client.1.smithi084.stdout:3/276: write d3/d8/df/f46 [6529219,56132] 0 2024-02-28T23:07:30.630 INFO:tasks.workunit.client.1.smithi084.stdout:4/239: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:30.630 INFO:tasks.workunit.client.1.smithi084.stdout:0/249: truncate d1/d8/f29 2488948 0 2024-02-28T23:07:30.631 INFO:tasks.workunit.client.1.smithi084.stdout:5/326: write dc/de/df/d16/f26 [887292,111032] 0 2024-02-28T23:07:30.632 INFO:tasks.workunit.client.1.smithi084.stdout:1/369: truncate f1 359482 0 2024-02-28T23:07:30.634 INFO:tasks.workunit.client.0.smithi057.stdout:1/104: dwrite d2/f10 [0,4194304] 0 2024-02-28T23:07:30.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/150: link d0/d4/d11/f12 d0/d1b/d22/f32 0 2024-02-28T23:07:30.637 INFO:tasks.workunit.client.0.smithi057.stdout:9/175: truncate dd/de/d14/f21 3239662 0 2024-02-28T23:07:30.638 INFO:tasks.workunit.client.0.smithi057.stdout:8/171: symlink d4/d13/d2c/l38 0 2024-02-28T23:07:30.639 INFO:tasks.workunit.client.0.smithi057.stdout:3/107: dwrite d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:30.639 INFO:tasks.workunit.client.1.smithi084.stdout:8/247: dwrite d0/d9/d1b/f46 [0,4194304] 0 2024-02-28T23:07:30.643 INFO:tasks.workunit.client.1.smithi084.stdout:3/277: creat d3/d50/f66 x:0 0 0 2024-02-28T23:07:30.647 INFO:tasks.workunit.client.1.smithi084.stdout:4/240: mkdir d1/d4a 0 2024-02-28T23:07:30.647 INFO:tasks.workunit.client.0.smithi057.stdout:0/145: symlink d17/d20/d26/l35 0 2024-02-28T23:07:30.650 INFO:tasks.workunit.client.0.smithi057.stdout:9/176: readlink dd/de/d24/l27 0 2024-02-28T23:07:30.650 INFO:tasks.workunit.client.1.smithi084.stdout:0/250: mknod d1/d8/d18/d28/c4d 0 2024-02-28T23:07:30.650 INFO:tasks.workunit.client.1.smithi084.stdout:1/370: mknod d12/d1b/d2e/d32/c82 0 2024-02-28T23:07:30.652 INFO:tasks.workunit.client.1.smithi084.stdout:4/241: mkdir d1/d4b 0 2024-02-28T23:07:30.653 INFO:tasks.workunit.client.0.smithi057.stdout:9/177: fsync dd/de/f1d 0 2024-02-28T23:07:30.654 INFO:tasks.workunit.client.1.smithi084.stdout:2/275: dread d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:30.654 INFO:tasks.workunit.client.1.smithi084.stdout:0/251: truncate d1/d8/f3f 680588 0 2024-02-28T23:07:30.656 INFO:tasks.workunit.client.0.smithi057.stdout:5/100: dwrite d4/f15 [0,4194304] 0 2024-02-28T23:07:30.656 INFO:tasks.workunit.client.1.smithi084.stdout:3/278: creat d3/d8/d1a/d21/d3d/f67 x:0 0 0 2024-02-28T23:07:30.662 INFO:tasks.workunit.client.1.smithi084.stdout:4/242: write d1/f2e [76622,5294] 0 2024-02-28T23:07:30.662 INFO:tasks.workunit.client.1.smithi084.stdout:0/252: fdatasync d1/d8/f27 0 2024-02-28T23:07:30.662 INFO:tasks.workunit.client.0.smithi057.stdout:8/172: creat d4/d13/d1a/f39 x:0 0 0 2024-02-28T23:07:30.663 INFO:tasks.workunit.client.0.smithi057.stdout:0/146: rename l16 to d17/d20/d23/l36 0 2024-02-28T23:07:30.663 INFO:tasks.workunit.client.1.smithi084.stdout:2/276: mkdir d5/d7/d2e/d4a 0 2024-02-28T23:07:30.670 INFO:tasks.workunit.client.0.smithi057.stdout:3/108: creat d3/d20/f23 x:0 0 0 2024-02-28T23:07:30.675 INFO:tasks.workunit.client.0.smithi057.stdout:3/109: read - d3/d20/f23 zero size 2024-02-28T23:07:30.676 INFO:tasks.workunit.client.0.smithi057.stdout:2/186: dread d8/d11/d21/f33 [0,4194304] 0 2024-02-28T23:07:30.678 INFO:tasks.workunit.client.0.smithi057.stdout:5/101: fdatasync d4/d9/d16/f19 0 2024-02-28T23:07:30.678 INFO:tasks.workunit.client.1.smithi084.stdout:5/327: dwrite f7 [0,4194304] 0 2024-02-28T23:07:30.679 INFO:tasks.workunit.client.1.smithi084.stdout:3/279: link d3/d8/d1a/l59 d3/d27/l68 0 2024-02-28T23:07:30.679 INFO:tasks.workunit.client.1.smithi084.stdout:0/253: write d1/d8/f29 [219468,4318] 0 2024-02-28T23:07:30.680 INFO:tasks.workunit.client.0.smithi057.stdout:1/105: getdents d2 0 2024-02-28T23:07:30.681 INFO:tasks.workunit.client.0.smithi057.stdout:2/187: write d8/f1a [5749842,92917] 0 2024-02-28T23:07:30.682 INFO:tasks.workunit.client.1.smithi084.stdout:4/243: mknod d1/d4a/c4c 0 2024-02-28T23:07:30.685 INFO:tasks.workunit.client.0.smithi057.stdout:3/110: creat d3/d20/f24 x:0 0 0 2024-02-28T23:07:30.686 INFO:tasks.workunit.client.0.smithi057.stdout:2/188: stat d8/d11/d21/d28/f47 0 2024-02-28T23:07:30.687 INFO:tasks.workunit.client.1.smithi084.stdout:3/280: write d3/d8/d1a/d21/d3d/f67 [413032,7764] 0 2024-02-28T23:07:30.688 INFO:tasks.workunit.client.1.smithi084.stdout:4/244: mknod d1/d18/c4d 0 2024-02-28T23:07:30.688 INFO:tasks.workunit.client.1.smithi084.stdout:1/371: dwrite d12/d1b/d2e/d32/f47 [0,4194304] 0 2024-02-28T23:07:30.688 INFO:tasks.workunit.client.0.smithi057.stdout:1/106: fdatasync d2/f8 0 2024-02-28T23:07:30.690 INFO:tasks.workunit.client.1.smithi084.stdout:5/328: mknod dc/de/df/d16/d34/d3c/c72 0 2024-02-28T23:07:30.691 INFO:tasks.workunit.client.1.smithi084.stdout:1/372: fdatasync d12/d1b/d3b/f7f 0 2024-02-28T23:07:30.692 INFO:tasks.workunit.client.1.smithi084.stdout:0/254: creat d1/d6/db/d35/d44/f4e x:0 0 0 2024-02-28T23:07:30.693 INFO:tasks.workunit.client.1.smithi084.stdout:4/245: truncate d1/f7 3083293 0 2024-02-28T23:07:30.694 INFO:tasks.workunit.client.0.smithi057.stdout:6/151: dwrite d0/d4/d11/f2d [0,4194304] 0 2024-02-28T23:07:30.694 INFO:tasks.workunit.client.1.smithi084.stdout:0/255: fdatasync d1/d6/f19 0 2024-02-28T23:07:30.694 INFO:tasks.workunit.client.0.smithi057.stdout:8/173: rename d4/d13/d1a/f39 to d4/f3a 0 2024-02-28T23:07:30.695 INFO:tasks.workunit.client.0.smithi057.stdout:2/189: read d8/d11/f29 [3766923,42962] 0 2024-02-28T23:07:30.699 INFO:tasks.workunit.client.0.smithi057.stdout:6/152: mknod d0/d1b/c33 0 2024-02-28T23:07:30.699 INFO:tasks.workunit.client.1.smithi084.stdout:0/256: unlink d1/d8/d18/l2a 0 2024-02-28T23:07:30.700 INFO:tasks.workunit.client.1.smithi084.stdout:1/373: creat d12/d1b/d2e/d30/d77/f83 x:0 0 0 2024-02-28T23:07:30.702 INFO:tasks.workunit.client.1.smithi084.stdout:0/257: write d1/d8/d18/f48 [2357396,3836] 0 2024-02-28T23:07:30.702 INFO:tasks.workunit.client.1.smithi084.stdout:1/374: fsync d12/d1b/d2e/d52/f73 0 2024-02-28T23:07:30.703 INFO:tasks.workunit.client.0.smithi057.stdout:1/107: mkdir d2/d7/df/db/d12/d19/d1b 0 2024-02-28T23:07:30.703 INFO:tasks.workunit.client.1.smithi084.stdout:4/246: link d1/d18/d20/d14/d30/l33 d1/d18/d20/d24/d2f/d3e/l4e 0 2024-02-28T23:07:30.703 INFO:tasks.workunit.client.0.smithi057.stdout:2/190: write d8/d11/d21/d28/f2f [1320609,33320] 0 2024-02-28T23:07:30.704 INFO:tasks.workunit.client.0.smithi057.stdout:0/147: dwrite f4 [4194304,4194304] 0 2024-02-28T23:07:30.705 INFO:tasks.workunit.client.0.smithi057.stdout:8/174: write d4/d13/d18/f27 [2404686,77016] 0 2024-02-28T23:07:30.706 INFO:tasks.workunit.client.1.smithi084.stdout:0/258: creat d1/d6/d2d/f4f x:0 0 0 2024-02-28T23:07:30.706 INFO:tasks.workunit.client.0.smithi057.stdout:6/153: write d0/d4/d25/f2a [774903,82964] 0 2024-02-28T23:07:30.707 INFO:tasks.workunit.client.1.smithi084.stdout:1/375: mkdir d12/d84 0 2024-02-28T23:07:30.707 INFO:tasks.workunit.client.1.smithi084.stdout:4/247: creat d1/d18/d20/d14/f4f x:0 0 0 2024-02-28T23:07:30.708 INFO:tasks.workunit.client.0.smithi057.stdout:2/191: write d8/d11/d21/f36 [746401,26551] 0 2024-02-28T23:07:30.709 INFO:tasks.workunit.client.0.smithi057.stdout:0/148: chown d17/f1e 3413 1 2024-02-28T23:07:30.710 INFO:tasks.workunit.client.1.smithi084.stdout:1/376: stat d12/d1b/d63/f3a 0 2024-02-28T23:07:30.710 INFO:tasks.workunit.client.1.smithi084.stdout:0/259: truncate d1/d6/db/d35/d44/f4e 465103 0 2024-02-28T23:07:30.712 INFO:tasks.workunit.client.1.smithi084.stdout:4/248: mknod d1/d18/d20/d14/c50 0 2024-02-28T23:07:30.714 INFO:tasks.workunit.client.1.smithi084.stdout:1/377: creat d12/d1b/d63/d7d/f85 x:0 0 0 2024-02-28T23:07:30.715 INFO:tasks.workunit.client.1.smithi084.stdout:0/260: symlink d1/d6/db/d35/d4a/l50 0 2024-02-28T23:07:30.715 INFO:tasks.workunit.client.1.smithi084.stdout:9/350: sync 2024-02-28T23:07:30.719 INFO:tasks.workunit.client.0.smithi057.stdout:8/175: write d4/f7 [9305344,7105] 0 2024-02-28T23:07:30.719 INFO:tasks.workunit.client.0.smithi057.stdout:4/191: sync 2024-02-28T23:07:30.719 INFO:tasks.workunit.client.0.smithi057.stdout:6/154: write d0/d4/d11/f2d [1597761,11037] 0 2024-02-28T23:07:30.720 INFO:tasks.workunit.client.1.smithi084.stdout:4/249: creat d1/d18/d20/d14/d30/d47/f51 x:0 0 0 2024-02-28T23:07:30.720 INFO:tasks.workunit.client.0.smithi057.stdout:5/102: dread d4/f15 [0,4194304] 0 2024-02-28T23:07:30.720 INFO:tasks.workunit.client.1.smithi084.stdout:9/351: mknod d4/d6/d15/d12/d41/d55/c74 0 2024-02-28T23:07:30.722 INFO:tasks.workunit.client.0.smithi057.stdout:8/176: read d4/d13/d1a/f32 [2360565,24483] 0 2024-02-28T23:07:30.724 INFO:tasks.workunit.client.0.smithi057.stdout:0/149: mkdir d17/d20/d23/d37 0 2024-02-28T23:07:30.725 INFO:tasks.workunit.client.0.smithi057.stdout:6/155: chown d0/d17/l1a 115796 1 2024-02-28T23:07:30.726 INFO:tasks.workunit.client.0.smithi057.stdout:4/192: unlink d1a/d1b/f23 0 2024-02-28T23:07:30.727 INFO:tasks.workunit.client.1.smithi084.stdout:1/378: getdents d12/d1b/d63 0 2024-02-28T23:07:30.733 INFO:tasks.workunit.client.0.smithi057.stdout:5/103: creat d4/d9/d16/f1b x:0 0 0 2024-02-28T23:07:30.733 INFO:tasks.workunit.client.0.smithi057.stdout:1/108: link d2/d7/df/cc d2/d7/df/db/d12/c1c 0 2024-02-28T23:07:30.735 INFO:tasks.workunit.client.0.smithi057.stdout:4/193: read - d1a/d1b/d24/d2e/d43/f47 zero size 2024-02-28T23:07:30.736 INFO:tasks.workunit.client.0.smithi057.stdout:6/156: mknod d0/d17/d20/c34 0 2024-02-28T23:07:30.736 INFO:tasks.workunit.client.0.smithi057.stdout:2/192: dwrite d8/d11/f17 [0,4194304] 0 2024-02-28T23:07:30.736 INFO:tasks.workunit.client.0.smithi057.stdout:0/150: symlink d17/d20/d26/d30/l38 0 2024-02-28T23:07:30.739 INFO:tasks.workunit.client.1.smithi084.stdout:8/248: truncate d0/d9/d1b/f46 1576018 0 2024-02-28T23:07:30.739 INFO:tasks.workunit.client.1.smithi084.stdout:0/261: dwrite d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:30.740 INFO:tasks.workunit.client.0.smithi057.stdout:1/109: creat d2/d7/df/f1d x:0 0 0 2024-02-28T23:07:30.740 INFO:tasks.workunit.client.0.smithi057.stdout:0/151: write d17/d20/f2b [812985,64860] 0 2024-02-28T23:07:30.741 INFO:tasks.workunit.client.0.smithi057.stdout:4/194: chown f15 1501224 1 2024-02-28T23:07:30.741 INFO:tasks.workunit.client.0.smithi057.stdout:6/157: dread - d0/fd zero size 2024-02-28T23:07:30.743 INFO:tasks.workunit.client.0.smithi057.stdout:1/110: symlink d2/l1e 0 2024-02-28T23:07:30.746 INFO:tasks.workunit.client.1.smithi084.stdout:0/262: symlink d1/d8/l51 0 2024-02-28T23:07:30.747 INFO:tasks.workunit.client.0.smithi057.stdout:5/104: creat d4/d9/f1c x:0 0 0 2024-02-28T23:07:30.749 INFO:tasks.workunit.client.1.smithi084.stdout:9/352: rename d4/d32/d3f/d5a to d4/d6/d15/d75 0 2024-02-28T23:07:30.750 INFO:tasks.workunit.client.0.smithi057.stdout:5/105: dread - d4/fd zero size 2024-02-28T23:07:30.751 INFO:tasks.workunit.client.0.smithi057.stdout:8/177: dread d4/d13/d1a/f32 [0,4194304] 0 2024-02-28T23:07:30.755 INFO:tasks.workunit.client.1.smithi084.stdout:1/379: link d12/d1b/d2e/d32/f47 d12/d1b/d3b/f86 0 2024-02-28T23:07:30.756 INFO:tasks.workunit.client.0.smithi057.stdout:4/195: mkdir d1a/d1b/d2d/d49/d4d 0 2024-02-28T23:07:30.756 INFO:tasks.workunit.client.0.smithi057.stdout:3/111: fsync d3/d20/f23 0 2024-02-28T23:07:30.756 INFO:tasks.workunit.client.0.smithi057.stdout:9/178: write dd/de/d14/f17 [2341244,111017] 0 2024-02-28T23:07:30.758 INFO:tasks.workunit.client.0.smithi057.stdout:3/112: write d3/d4/d1d/f1e [437347,90046] 0 2024-02-28T23:07:30.761 INFO:tasks.workunit.client.0.smithi057.stdout:0/152: getdents d17/d20/d23/d37 0 2024-02-28T23:07:30.762 INFO:tasks.workunit.client.1.smithi084.stdout:5/329: write dc/de/df/f20 [1028350,59196] 0 2024-02-28T23:07:30.764 INFO:tasks.workunit.client.0.smithi057.stdout:8/178: rmdir d4/d13/d1a 39 2024-02-28T23:07:30.764 INFO:tasks.workunit.client.0.smithi057.stdout:0/153: write fe [1441810,43702] 0 2024-02-28T23:07:30.765 INFO:tasks.workunit.client.1.smithi084.stdout:4/250: rename d1/d18/d20/d24/d2f/d35 to d1/d18/d20/d24/d2f/d3e/d52 0 2024-02-28T23:07:30.766 INFO:tasks.workunit.client.1.smithi084.stdout:8/249: write d0/d9/fb [4960363,104503] 0 2024-02-28T23:07:30.766 INFO:tasks.workunit.client.1.smithi084.stdout:9/353: symlink d4/d6/d15/d12/d41/l76 0 2024-02-28T23:07:30.766 INFO:tasks.workunit.client.1.smithi084.stdout:1/380: write d12/d1b/d2e/d52/d5b/f6f [3718963,93412] 0 2024-02-28T23:07:30.768 INFO:tasks.workunit.client.1.smithi084.stdout:9/354: chown d4/f3b 1944529 1 2024-02-28T23:07:30.771 INFO:tasks.workunit.client.1.smithi084.stdout:9/355: dread - d4/d6/d15/f71 zero size 2024-02-28T23:07:30.771 INFO:tasks.workunit.client.0.smithi057.stdout:0/154: write d17/f1e [356748,45568] 0 2024-02-28T23:07:30.772 INFO:tasks.workunit.client.0.smithi057.stdout:4/196: unlink d1a/l27 0 2024-02-28T23:07:30.773 INFO:tasks.workunit.client.1.smithi084.stdout:8/250: read d0/d9/d1b/d20/f25 [390463,65511] 0 2024-02-28T23:07:30.776 INFO:tasks.workunit.client.0.smithi057.stdout:2/193: getdents d8/d11 0 2024-02-28T23:07:30.779 INFO:tasks.workunit.client.0.smithi057.stdout:3/113: symlink d3/d4/l25 0 2024-02-28T23:07:30.781 INFO:tasks.workunit.client.0.smithi057.stdout:5/106: dwrite d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:30.783 INFO:tasks.workunit.client.1.smithi084.stdout:1/381: symlink d12/d1b/d2e/l87 0 2024-02-28T23:07:30.783 INFO:tasks.workunit.client.0.smithi057.stdout:5/107: chown d4/d7/l11 38 1 2024-02-28T23:07:30.784 INFO:tasks.workunit.client.0.smithi057.stdout:2/194: chown d8/d11/d21/c3a 2155883 1 2024-02-28T23:07:30.784 INFO:tasks.workunit.client.0.smithi057.stdout:6/158: getdents d0/d4/d11 0 2024-02-28T23:07:30.785 INFO:tasks.workunit.client.0.smithi057.stdout:7/214: sync 2024-02-28T23:07:30.786 INFO:tasks.workunit.client.0.smithi057.stdout:0/155: creat d17/d20/d26/d30/f39 x:0 0 0 2024-02-28T23:07:30.789 INFO:tasks.workunit.client.0.smithi057.stdout:8/179: symlink d4/d13/l3b 0 2024-02-28T23:07:30.790 INFO:tasks.workunit.client.1.smithi084.stdout:5/330: rename dc/de/df/d16/d34/d54/c5c to dc/d51/c73 0 2024-02-28T23:07:30.792 INFO:tasks.workunit.client.0.smithi057.stdout:0/156: readlink d17/d20/d26/l35 0 2024-02-28T23:07:30.792 INFO:tasks.workunit.client.0.smithi057.stdout:3/114: symlink d3/l26 0 2024-02-28T23:07:30.793 INFO:tasks.workunit.client.0.smithi057.stdout:9/179: creat dd/f31 x:0 0 0 2024-02-28T23:07:30.794 INFO:tasks.workunit.client.0.smithi057.stdout:7/215: write d0/d8/d15/d29/f32 [1080003,44524] 0 2024-02-28T23:07:30.797 INFO:tasks.workunit.client.0.smithi057.stdout:6/159: unlink d0/d4/d11/f12 0 2024-02-28T23:07:30.800 INFO:tasks.workunit.client.0.smithi057.stdout:5/108: mkdir d4/d9/d16/d12/d1d 0 2024-02-28T23:07:30.801 INFO:tasks.workunit.client.1.smithi084.stdout:4/251: rename d1/d18/d20/d24/d2f/d3e/d52/d37/f44 to d1/d18/d20/d14/d30/f53 0 2024-02-28T23:07:30.802 INFO:tasks.workunit.client.1.smithi084.stdout:4/252: readlink d1/lb 0 2024-02-28T23:07:30.803 INFO:tasks.workunit.client.0.smithi057.stdout:8/180: mknod d4/d13/c3c 0 2024-02-28T23:07:30.805 INFO:tasks.workunit.client.0.smithi057.stdout:7/216: read d0/d8/d15/f22 [1818410,35861] 0 2024-02-28T23:07:30.806 INFO:tasks.workunit.client.0.smithi057.stdout:0/157: mknod d17/d19/c3a 0 2024-02-28T23:07:30.807 INFO:tasks.workunit.client.1.smithi084.stdout:1/382: creat d12/d1b/d2e/d52/d61/d74/f88 x:0 0 0 2024-02-28T23:07:30.807 INFO:tasks.workunit.client.0.smithi057.stdout:3/115: mkdir d3/d8/d27 0 2024-02-28T23:07:30.808 INFO:tasks.workunit.client.0.smithi057.stdout:6/160: creat d0/d17/f35 x:0 0 0 2024-02-28T23:07:30.809 INFO:tasks.workunit.client.1.smithi084.stdout:1/383: fdatasync d12/d1b/d63/f3f 0 2024-02-28T23:07:30.809 INFO:tasks.workunit.client.1.smithi084.stdout:5/331: dread dc/de/df/d16/d34/d54/f69 [4194304,4194304] 0 2024-02-28T23:07:30.809 INFO:tasks.workunit.client.1.smithi084.stdout:0/263: write d1/d8/f1c [2867894,128822] 0 2024-02-28T23:07:30.810 INFO:tasks.workunit.client.0.smithi057.stdout:9/180: truncate dd/f12 275110 0 2024-02-28T23:07:30.812 INFO:tasks.workunit.client.0.smithi057.stdout:6/161: write d0/d4/d11/f14 [303265,21924] 0 2024-02-28T23:07:30.813 INFO:tasks.workunit.client.1.smithi084.stdout:5/332: read - dc/de/df/d16/d34/d3c/f5a zero size 2024-02-28T23:07:30.813 INFO:tasks.workunit.client.0.smithi057.stdout:9/181: chown dd/c2a 350 1 2024-02-28T23:07:30.814 INFO:tasks.workunit.client.1.smithi084.stdout:8/251: link d0/d44/l55 d0/l57 0 2024-02-28T23:07:30.815 INFO:tasks.workunit.client.1.smithi084.stdout:4/253: creat d1/d4a/f54 x:0 0 0 2024-02-28T23:07:30.819 INFO:tasks.workunit.client.1.smithi084.stdout:0/264: mknod d1/d8/c52 0 2024-02-28T23:07:30.820 INFO:tasks.workunit.client.0.smithi057.stdout:0/158: mknod d17/c3b 0 2024-02-28T23:07:30.820 INFO:tasks.workunit.client.0.smithi057.stdout:5/109: rename d4/d9/fc to d4/d7/f1e 0 2024-02-28T23:07:30.821 INFO:tasks.workunit.client.1.smithi084.stdout:8/252: creat d0/d24/d4d/f58 x:0 0 0 2024-02-28T23:07:30.823 INFO:tasks.workunit.client.1.smithi084.stdout:9/356: link d4/d32/d3f/l4a d4/d6/d15/d75/l77 0 2024-02-28T23:07:30.824 INFO:tasks.workunit.client.0.smithi057.stdout:4/197: dread f18 [0,4194304] 0 2024-02-28T23:07:30.824 INFO:tasks.workunit.client.1.smithi084.stdout:8/253: fsync d0/d9/f4c 0 2024-02-28T23:07:30.825 INFO:tasks.workunit.client.0.smithi057.stdout:7/217: mkdir d0/d3f 0 2024-02-28T23:07:30.826 INFO:tasks.workunit.client.1.smithi084.stdout:4/254: rename d1/d18/d20/d15 to d1/d18/d20/d24/d2f/d3e/d52/d37/d55 0 2024-02-28T23:07:30.826 INFO:tasks.workunit.client.0.smithi057.stdout:7/218: fdatasync d0/d5/d25/f28 0 2024-02-28T23:07:30.827 INFO:tasks.workunit.client.0.smithi057.stdout:8/181: creat d4/d13/d1a/f3d x:0 0 0 2024-02-28T23:07:30.827 INFO:tasks.workunit.client.1.smithi084.stdout:5/333: creat dc/de/df/d6e/f74 x:0 0 0 2024-02-28T23:07:30.828 INFO:tasks.workunit.client.0.smithi057.stdout:7/219: read d0/f10 [2796634,56466] 0 2024-02-28T23:07:30.829 INFO:tasks.workunit.client.1.smithi084.stdout:5/334: dread - dc/de/df/d6e/f74 zero size 2024-02-28T23:07:30.830 INFO:tasks.workunit.client.0.smithi057.stdout:7/220: read d0/d8/d15/d29/f2a [140088,43148] 0 2024-02-28T23:07:30.831 INFO:tasks.workunit.client.1.smithi084.stdout:5/335: write dc/f27 [715537,114793] 0 2024-02-28T23:07:30.832 INFO:tasks.workunit.client.0.smithi057.stdout:7/221: write d0/d5/f27 [251404,114408] 0 2024-02-28T23:07:30.833 INFO:tasks.workunit.client.0.smithi057.stdout:7/222: readlink d0/d8/lc 0 2024-02-28T23:07:30.833 INFO:tasks.workunit.client.1.smithi084.stdout:4/255: rename d1/d18/d20/d14/l22 to d1/d18/d20/d14/d30/l56 0 2024-02-28T23:07:30.835 INFO:tasks.workunit.client.1.smithi084.stdout:0/265: creat d1/d6/f53 x:0 0 0 2024-02-28T23:07:30.837 INFO:tasks.workunit.client.1.smithi084.stdout:4/256: write d1/d4a/f54 [459848,86068] 0 2024-02-28T23:07:30.837 INFO:tasks.workunit.client.1.smithi084.stdout:5/336: mknod dc/c75 0 2024-02-28T23:07:30.838 INFO:tasks.workunit.client.0.smithi057.stdout:1/111: link d2/c1a d2/d7/df/db/d12/d19/d1b/c1f 0 2024-02-28T23:07:30.841 INFO:tasks.workunit.client.1.smithi084.stdout:0/266: creat d1/d8/d18/d28/f54 x:0 0 0 2024-02-28T23:07:30.842 INFO:tasks.workunit.client.0.smithi057.stdout:0/159: unlink d17/d20/d23/l29 0 2024-02-28T23:07:30.847 INFO:tasks.workunit.client.0.smithi057.stdout:8/182: symlink d4/d13/d1a/l3e 0 2024-02-28T23:07:30.847 INFO:tasks.workunit.client.1.smithi084.stdout:8/254: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:30.847 INFO:tasks.workunit.client.1.smithi084.stdout:4/257: truncate d1/d18/d20/d14/d30/f38 1134620 0 2024-02-28T23:07:30.848 INFO:tasks.workunit.client.0.smithi057.stdout:2/195: dread d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:30.848 INFO:tasks.workunit.client.0.smithi057.stdout:7/223: creat d0/d8/d15/d29/f40 x:0 0 0 2024-02-28T23:07:30.848 INFO:tasks.workunit.client.1.smithi084.stdout:0/267: rename d1/d6/f19 to d1/d8/f55 0 2024-02-28T23:07:30.850 INFO:tasks.workunit.client.1.smithi084.stdout:4/258: mkdir d1/d18/d20/d24/d2f/d3e/d52/d57 0 2024-02-28T23:07:30.851 INFO:tasks.workunit.client.0.smithi057.stdout:5/110: mknod d4/d9/c1f 0 2024-02-28T23:07:30.852 INFO:tasks.workunit.client.0.smithi057.stdout:8/183: dread d4/f5 [0,4194304] 0 2024-02-28T23:07:30.852 INFO:tasks.workunit.client.0.smithi057.stdout:3/116: rename d3/d4/l22 to d3/d8/d1b/l28 0 2024-02-28T23:07:30.853 INFO:tasks.workunit.client.1.smithi084.stdout:0/268: mkdir d1/d6/db/d35/d4a/d56 0 2024-02-28T23:07:30.853 INFO:tasks.workunit.client.1.smithi084.stdout:8/255: truncate d0/d11/f3c 97289 0 2024-02-28T23:07:30.855 INFO:tasks.workunit.client.1.smithi084.stdout:4/259: stat d1/d18/d20/d24/d2f/d3e/l4e 0 2024-02-28T23:07:30.855 INFO:tasks.workunit.client.1.smithi084.stdout:0/269: readlink d1/d6/db/l2c 0 2024-02-28T23:07:30.857 INFO:tasks.workunit.client.0.smithi057.stdout:3/117: write d3/d4/f17 [1970765,126197] 0 2024-02-28T23:07:30.858 INFO:tasks.workunit.client.0.smithi057.stdout:0/160: creat d17/d19/f3c x:0 0 0 2024-02-28T23:07:30.858 INFO:tasks.workunit.client.0.smithi057.stdout:0/161: stat f9 0 2024-02-28T23:07:30.859 INFO:tasks.workunit.client.1.smithi084.stdout:5/337: dwrite dc/de/df/d16/f4a [0,4194304] 0 2024-02-28T23:07:30.860 INFO:tasks.workunit.client.0.smithi057.stdout:4/198: symlink d1a/d1b/d2d/d49/l4e 0 2024-02-28T23:07:30.860 INFO:tasks.workunit.client.1.smithi084.stdout:4/260: creat d1/d18/d20/d14/d30/d47/f58 x:0 0 0 2024-02-28T23:07:30.864 INFO:tasks.workunit.client.0.smithi057.stdout:6/162: truncate d0/d4/d25/f2f 3244947 0 2024-02-28T23:07:30.865 INFO:tasks.workunit.client.1.smithi084.stdout:4/261: dread - d1/d18/d20/d14/d30/d47/f51 zero size 2024-02-28T23:07:30.865 INFO:tasks.workunit.client.0.smithi057.stdout:2/196: unlink d8/d11/d21/d28/l43 0 2024-02-28T23:07:30.868 INFO:tasks.workunit.client.0.smithi057.stdout:6/163: chown d0/d17/f1d 182182 1 2024-02-28T23:07:30.869 INFO:tasks.workunit.client.0.smithi057.stdout:5/111: dread d4/d7/fe [0,4194304] 0 2024-02-28T23:07:30.870 INFO:tasks.workunit.client.0.smithi057.stdout:1/112: unlink d2/d7/df/db/c16 0 2024-02-28T23:07:30.870 INFO:tasks.workunit.client.1.smithi084.stdout:5/338: rename dc/de/df/d50/f67 to dc/d51/f76 0 2024-02-28T23:07:30.874 INFO:tasks.workunit.client.0.smithi057.stdout:4/199: fdatasync d1a/d1b/f2c 0 2024-02-28T23:07:30.874 INFO:tasks.workunit.client.0.smithi057.stdout:5/112: fdatasync d4/fd 0 2024-02-28T23:07:30.874 INFO:tasks.workunit.client.1.smithi084.stdout:0/270: dwrite d1/d8/f29 [0,4194304] 0 2024-02-28T23:07:30.877 INFO:tasks.workunit.client.1.smithi084.stdout:5/339: mkdir dc/de/df/d16/d34/d3c/d77 0 2024-02-28T23:07:30.877 INFO:tasks.workunit.client.0.smithi057.stdout:6/164: chown d0/d17/f35 52 1 2024-02-28T23:07:30.880 INFO:tasks.workunit.client.0.smithi057.stdout:9/182: getdents dd/de/d14 0 2024-02-28T23:07:30.883 INFO:tasks.workunit.client.0.smithi057.stdout:7/224: mknod d0/d5/c41 0 2024-02-28T23:07:30.883 INFO:tasks.workunit.client.0.smithi057.stdout:4/200: fsync d1a/d1b/f1c 0 2024-02-28T23:07:30.883 INFO:tasks.workunit.client.0.smithi057.stdout:8/184: mkdir d4/d1e/d37/d3f 0 2024-02-28T23:07:30.884 INFO:tasks.workunit.client.1.smithi084.stdout:5/340: mknod dc/de/df/d16/d34/d54/d5f/d66/c78 0 2024-02-28T23:07:30.885 INFO:tasks.workunit.client.0.smithi057.stdout:2/197: truncate d8/d11/f1c 3865235 0 2024-02-28T23:07:30.886 INFO:tasks.workunit.client.0.smithi057.stdout:5/113: symlink d4/d7/l20 0 2024-02-28T23:07:30.886 INFO:tasks.workunit.client.0.smithi057.stdout:7/225: dread - d0/d5/f39 zero size 2024-02-28T23:07:30.890 INFO:tasks.workunit.client.1.smithi084.stdout:5/341: creat dc/de/f79 x:0 0 0 2024-02-28T23:07:30.892 INFO:tasks.workunit.client.1.smithi084.stdout:4/262: dwrite d1/d18/d20/d24/d2f/d3e/d52/d37/d55/f16 [4194304,4194304] 0 2024-02-28T23:07:30.893 INFO:tasks.workunit.client.0.smithi057.stdout:0/162: dwrite f10 [0,4194304] 0 2024-02-28T23:07:30.895 INFO:tasks.workunit.client.1.smithi084.stdout:5/342: symlink dc/de/df/d6e/l7a 0 2024-02-28T23:07:30.895 INFO:tasks.workunit.client.1.smithi084.stdout:0/271: dread d1/d8/d18/f48 [0,4194304] 0 2024-02-28T23:07:30.896 INFO:tasks.workunit.client.0.smithi057.stdout:4/201: stat d1a/d1e/l22 0 2024-02-28T23:07:30.902 INFO:tasks.workunit.client.1.smithi084.stdout:5/343: symlink dc/de/df/d50/l7b 0 2024-02-28T23:07:30.905 INFO:tasks.workunit.client.0.smithi057.stdout:2/198: mkdir d8/d11/d21/d28/d49 0 2024-02-28T23:07:30.907 INFO:tasks.workunit.client.1.smithi084.stdout:5/344: write dc/de/df/d16/f4a [1675285,118977] 0 2024-02-28T23:07:30.907 INFO:tasks.workunit.client.1.smithi084.stdout:0/272: unlink d1/d8/f23 0 2024-02-28T23:07:30.908 INFO:tasks.workunit.client.0.smithi057.stdout:9/183: fdatasync dd/f12 0 2024-02-28T23:07:30.912 INFO:tasks.workunit.client.0.smithi057.stdout:6/165: rename d0/d17/d20/c2e to d0/d4/c36 0 2024-02-28T23:07:30.912 INFO:tasks.workunit.client.1.smithi084.stdout:4/263: symlink d1/d18/d20/d24/d2f/d3e/d52/d48/l59 0 2024-02-28T23:07:30.912 INFO:tasks.workunit.client.1.smithi084.stdout:8/256: dwrite d0/d11/f3c [0,4194304] 0 2024-02-28T23:07:30.914 INFO:tasks.workunit.client.1.smithi084.stdout:0/273: write d1/d8/f47 [461734,56602] 0 2024-02-28T23:07:30.914 INFO:tasks.workunit.client.0.smithi057.stdout:8/185: truncate d4/d13/f2b 474502 0 2024-02-28T23:07:30.915 INFO:tasks.workunit.client.0.smithi057.stdout:1/113: dwrite d2/d7/df/db/f18 [0,4194304] 0 2024-02-28T23:07:30.917 INFO:tasks.workunit.client.1.smithi084.stdout:4/264: fdatasync d1/d18/d20/d14/d30/f3d 0 2024-02-28T23:07:30.922 INFO:tasks.workunit.client.0.smithi057.stdout:7/226: creat d0/d8/d15/f42 x:0 0 0 2024-02-28T23:07:30.925 INFO:tasks.workunit.client.0.smithi057.stdout:1/114: rmdir d2 39 2024-02-28T23:07:30.928 INFO:tasks.workunit.client.0.smithi057.stdout:9/184: fsync dd/de/d14/f21 0 2024-02-28T23:07:30.928 INFO:tasks.workunit.client.0.smithi057.stdout:5/114: dwrite d4/d9/d16/f1b [0,4194304] 0 2024-02-28T23:07:30.930 INFO:tasks.workunit.client.1.smithi084.stdout:7/278: sync 2024-02-28T23:07:30.931 INFO:tasks.workunit.client.0.smithi057.stdout:9/185: dread - dd/de/d24/f28 zero size 2024-02-28T23:07:30.932 INFO:tasks.workunit.client.0.smithi057.stdout:5/115: chown d4/d7/d8 1408 1 2024-02-28T23:07:30.932 INFO:tasks.workunit.client.1.smithi084.stdout:5/345: dread dc/de/df/d16/d34/d54/f61 [0,4194304] 0 2024-02-28T23:07:30.938 INFO:tasks.workunit.client.0.smithi057.stdout:3/118: dwrite d3/fd [0,4194304] 0 2024-02-28T23:07:30.939 INFO:tasks.workunit.client.0.smithi057.stdout:4/202: dwrite d1a/d1b/f21 [4194304,4194304] 0 2024-02-28T23:07:30.939 INFO:tasks.workunit.client.0.smithi057.stdout:6/166: mknod d0/d4/c37 0 2024-02-28T23:07:30.940 INFO:tasks.workunit.client.1.smithi084.stdout:0/274: dwrite d1/d8/d18/d28/f54 [0,4194304] 0 2024-02-28T23:07:30.942 INFO:tasks.workunit.client.1.smithi084.stdout:5/346: mkdir dc/d51/d7c 0 2024-02-28T23:07:30.946 INFO:tasks.workunit.client.0.smithi057.stdout:8/186: truncate d4/f17 754444 0 2024-02-28T23:07:30.948 INFO:tasks.workunit.client.1.smithi084.stdout:0/275: unlink d1/d8/d18/l38 0 2024-02-28T23:07:30.957 INFO:tasks.workunit.client.1.smithi084.stdout:0/276: mkdir d1/d6/db/d35/d57 0 2024-02-28T23:07:30.961 INFO:tasks.workunit.client.0.smithi057.stdout:9/186: symlink dd/de/d24/l32 0 2024-02-28T23:07:30.963 INFO:tasks.workunit.client.1.smithi084.stdout:5/347: rename dc/de/df/d16/c39 to dc/c7d 0 2024-02-28T23:07:30.966 INFO:tasks.workunit.client.0.smithi057.stdout:3/119: stat d3/d4/d7/f10 0 2024-02-28T23:07:30.968 INFO:tasks.workunit.client.0.smithi057.stdout:9/187: truncate dd/f2d 994377 0 2024-02-28T23:07:30.971 INFO:tasks.workunit.client.1.smithi084.stdout:7/279: dwrite db/d10/d17/f58 [0,4194304] 0 2024-02-28T23:07:30.971 INFO:tasks.workunit.client.1.smithi084.stdout:8/257: dread d0/d11/f23 [0,4194304] 0 2024-02-28T23:07:30.974 INFO:tasks.workunit.client.0.smithi057.stdout:9/188: read dd/f12 [207872,79594] 0 2024-02-28T23:07:30.976 INFO:tasks.workunit.client.0.smithi057.stdout:6/167: dwrite d0/d4/d11/f14 [0,4194304] 0 2024-02-28T23:07:30.981 INFO:tasks.workunit.client.1.smithi084.stdout:7/280: creat db/d11/d27/d36/f5a x:0 0 0 2024-02-28T23:07:30.981 INFO:tasks.workunit.client.1.smithi084.stdout:4/265: dread d1/d18/d20/d14/f39 [0,4194304] 0 2024-02-28T23:07:30.983 INFO:tasks.workunit.client.0.smithi057.stdout:2/199: link d8/d11/c44 d8/d11/d21/d28/c4a 0 2024-02-28T23:07:30.984 INFO:tasks.workunit.client.1.smithi084.stdout:0/277: dwrite d1/d6/d2d/f30 [0,4194304] 0 2024-02-28T23:07:30.985 INFO:tasks.workunit.client.0.smithi057.stdout:4/203: mkdir d1a/d1e/d4b/d4f 0 2024-02-28T23:07:30.987 INFO:tasks.workunit.client.1.smithi084.stdout:7/281: truncate db/d11/d27/f43 261368 0 2024-02-28T23:07:30.988 INFO:tasks.workunit.client.1.smithi084.stdout:4/266: stat d1/d18/d20/d24/d2f/d3e/d52 0 2024-02-28T23:07:30.988 INFO:tasks.workunit.client.0.smithi057.stdout:8/187: symlink d4/l40 0 2024-02-28T23:07:30.989 INFO:tasks.workunit.client.0.smithi057.stdout:7/227: dwrite d0/d8/d15/f3a [4194304,4194304] 0 2024-02-28T23:07:30.990 INFO:tasks.workunit.client.0.smithi057.stdout:6/168: mkdir d0/d17/d38 0 2024-02-28T23:07:30.995 INFO:tasks.workunit.client.1.smithi084.stdout:0/278: rename d1/d8/f47 to d1/d6/d1b/f58 0 2024-02-28T23:07:30.996 INFO:tasks.workunit.client.1.smithi084.stdout:7/282: creat db/d11/f5b x:0 0 0 2024-02-28T23:07:30.997 INFO:tasks.workunit.client.0.smithi057.stdout:5/116: getdents d4 0 2024-02-28T23:07:30.997 INFO:tasks.workunit.client.0.smithi057.stdout:8/188: stat d4/d1e/f36 0 2024-02-28T23:07:30.997 INFO:tasks.workunit.client.1.smithi084.stdout:0/279: readlink d1/d6/db/d37/l39 0 2024-02-28T23:07:31.001 INFO:tasks.workunit.client.1.smithi084.stdout:0/280: write d1/d8/f1c [545507,99718] 0 2024-02-28T23:07:31.002 INFO:tasks.workunit.client.0.smithi057.stdout:8/189: stat d4/d13/d2c/l38 0 2024-02-28T23:07:31.002 INFO:tasks.workunit.client.0.smithi057.stdout:3/120: creat d3/d8/f29 x:0 0 0 2024-02-28T23:07:31.002 INFO:tasks.workunit.client.1.smithi084.stdout:8/258: dwrite d0/d9/d1b/d38/f3e [4194304,4194304] 0 2024-02-28T23:07:31.002 INFO:tasks.workunit.client.1.smithi084.stdout:7/283: chown db/d11/c34 1502027 1 2024-02-28T23:07:31.003 INFO:tasks.workunit.client.0.smithi057.stdout:5/117: dread - d4/d9/d16/f18 zero size 2024-02-28T23:07:31.009 INFO:tasks.workunit.client.1.smithi084.stdout:7/284: chown db/d11/d27/f33 6901220 1 2024-02-28T23:07:31.009 INFO:tasks.workunit.client.1.smithi084.stdout:8/259: write d0/d9/d1b/f28 [1888627,121611] 0 2024-02-28T23:07:31.009 INFO:tasks.workunit.client.1.smithi084.stdout:7/285: readlink db/d11/l51 0 2024-02-28T23:07:31.011 INFO:tasks.workunit.client.0.smithi057.stdout:7/228: link d0/d5/c41 d0/d5/d25/d3e/c43 0 2024-02-28T23:07:31.014 INFO:tasks.workunit.client.1.smithi084.stdout:0/281: mkdir d1/d6/d59 0 2024-02-28T23:07:31.016 INFO:tasks.workunit.client.0.smithi057.stdout:7/229: truncate d0/d5/f1e 5240666 0 2024-02-28T23:07:31.017 INFO:tasks.workunit.client.0.smithi057.stdout:3/121: readlink d3/d4/d7/l1c 0 2024-02-28T23:07:31.021 INFO:tasks.workunit.client.1.smithi084.stdout:4/267: dwrite d1/d18/f32 [0,4194304] 0 2024-02-28T23:07:31.022 INFO:tasks.workunit.client.0.smithi057.stdout:3/122: read - d3/d8/f29 zero size 2024-02-28T23:07:31.022 INFO:tasks.workunit.client.0.smithi057.stdout:7/230: mkdir d0/d5/d25/d44 0 2024-02-28T23:07:31.023 INFO:tasks.workunit.client.1.smithi084.stdout:0/282: rename d1/d6/f14 to d1/d6/d59/f5a 0 2024-02-28T23:07:31.024 INFO:tasks.workunit.client.0.smithi057.stdout:7/231: stat d0/d5/f30 0 2024-02-28T23:07:31.025 INFO:tasks.workunit.client.1.smithi084.stdout:4/268: truncate d1/d18/d20/d24/f34 344149 0 2024-02-28T23:07:31.025 INFO:tasks.workunit.client.1.smithi084.stdout:0/283: mkdir d1/d6/d59/d5b 0 2024-02-28T23:07:31.026 INFO:tasks.workunit.client.0.smithi057.stdout:9/189: dread dd/de/d14/f1f [0,4194304] 0 2024-02-28T23:07:31.027 INFO:tasks.workunit.client.1.smithi084.stdout:4/269: truncate d1/f2e 1014428 0 2024-02-28T23:07:31.030 INFO:tasks.workunit.client.0.smithi057.stdout:8/190: getdents d4/d13/d18 0 2024-02-28T23:07:31.035 INFO:tasks.workunit.client.1.smithi084.stdout:4/270: creat d1/d18/d20/d24/d2f/d3e/d52/d37/f5a x:0 0 0 2024-02-28T23:07:31.035 INFO:tasks.workunit.client.0.smithi057.stdout:1/115: write d2/d7/df/db/f14 [5037688,51789] 0 2024-02-28T23:07:31.035 INFO:tasks.workunit.client.0.smithi057.stdout:2/200: dwrite d8/f24 [0,4194304] 0 2024-02-28T23:07:31.041 INFO:tasks.workunit.client.0.smithi057.stdout:8/191: stat d4/d13/d1a/f34 0 2024-02-28T23:07:31.041 INFO:tasks.workunit.client.0.smithi057.stdout:9/190: dread dd/de/d14/f1f [0,4194304] 0 2024-02-28T23:07:31.042 INFO:tasks.workunit.client.1.smithi084.stdout:4/271: symlink d1/d18/d20/d24/d2f/d3e/d52/l5b 0 2024-02-28T23:07:31.044 INFO:tasks.workunit.client.1.smithi084.stdout:7/286: write db/d10/f2f [895255,38332] 0 2024-02-28T23:07:31.045 INFO:tasks.workunit.client.1.smithi084.stdout:4/272: chown d1/d18/d20/l19 1 1 2024-02-28T23:07:31.046 INFO:tasks.workunit.client.0.smithi057.stdout:5/118: dwrite d4/fd [0,4194304] 0 2024-02-28T23:07:31.048 INFO:tasks.workunit.client.0.smithi057.stdout:5/119: fdatasync f2 0 2024-02-28T23:07:31.051 INFO:tasks.workunit.client.1.smithi084.stdout:0/284: dwrite d1/d6/db/d35/d44/f4e [0,4194304] 0 2024-02-28T23:07:31.051 INFO:tasks.workunit.client.1.smithi084.stdout:4/273: rmdir d1/d18/d20/d24/d2f 39 2024-02-28T23:07:31.052 INFO:tasks.workunit.client.1.smithi084.stdout:8/260: truncate d0/d9/d30/f42 2302461 0 2024-02-28T23:07:31.056 INFO:tasks.workunit.client.1.smithi084.stdout:7/287: symlink db/d11/d27/d36/l5c 0 2024-02-28T23:07:31.056 INFO:tasks.workunit.client.0.smithi057.stdout:5/120: read d4/d7/fe [3315481,18183] 0 2024-02-28T23:07:31.056 INFO:tasks.workunit.client.0.smithi057.stdout:7/232: truncate d0/f10 2021113 0 2024-02-28T23:07:31.069 INFO:tasks.workunit.client.0.smithi057.stdout:3/123: rmdir d3/d8 39 2024-02-28T23:07:31.071 INFO:tasks.workunit.client.0.smithi057.stdout:6/169: truncate d0/d4/d25/f26 50427 0 2024-02-28T23:07:31.074 INFO:tasks.workunit.client.1.smithi084.stdout:0/285: dread d1/d6/db/d35/d44/f4e [0,4194304] 0 2024-02-28T23:07:31.082 INFO:tasks.workunit.client.1.smithi084.stdout:0/286: getdents d1/d6/db/d35/d44 0 2024-02-28T23:07:31.082 INFO:tasks.workunit.client.1.smithi084.stdout:8/261: dwrite d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:31.083 INFO:tasks.workunit.client.1.smithi084.stdout:2/277: sync 2024-02-28T23:07:31.083 INFO:tasks.workunit.client.1.smithi084.stdout:6/306: sync 2024-02-28T23:07:31.084 INFO:tasks.workunit.client.0.smithi057.stdout:2/201: dwrite d8/d11/f17 [0,4194304] 0 2024-02-28T23:07:31.085 INFO:tasks.workunit.client.0.smithi057.stdout:1/116: dread f0 [0,4194304] 0 2024-02-28T23:07:31.085 INFO:tasks.workunit.client.0.smithi057.stdout:7/233: unlink d0/d5/l14 0 2024-02-28T23:07:31.089 INFO:tasks.workunit.client.0.smithi057.stdout:2/202: write d8/f23 [1603881,96783] 0 2024-02-28T23:07:31.089 INFO:tasks.workunit.client.1.smithi084.stdout:6/307: truncate d1/d9/f28 4675582 0 2024-02-28T23:07:31.089 INFO:tasks.workunit.client.1.smithi084.stdout:0/287: chown d1/d8/d18/d28/f54 39944 1 2024-02-28T23:07:31.090 INFO:tasks.workunit.client.1.smithi084.stdout:0/288: chown d1/d8/d18/f48 1996 1 2024-02-28T23:07:31.092 INFO:tasks.workunit.client.0.smithi057.stdout:5/121: mkdir d4/d21 0 2024-02-28T23:07:31.094 INFO:tasks.workunit.client.0.smithi057.stdout:9/191: dwrite dd/de/f1d [4194304,4194304] 0 2024-02-28T23:07:31.096 INFO:tasks.workunit.client.1.smithi084.stdout:0/289: rename d1/d6/f21 to d1/d6/db/d35/f5c 0 2024-02-28T23:07:31.097 INFO:tasks.workunit.client.0.smithi057.stdout:4/204: dread d1a/d1b/d2d/f3c [0,4194304] 0 2024-02-28T23:07:31.100 INFO:tasks.workunit.client.0.smithi057.stdout:2/203: dread d8/d11/f1b [0,4194304] 0 2024-02-28T23:07:31.102 INFO:tasks.workunit.client.1.smithi084.stdout:0/290: creat d1/d6/db/f5d x:0 0 0 2024-02-28T23:07:31.105 INFO:tasks.workunit.client.0.smithi057.stdout:6/170: write d0/f1c [2374842,119591] 0 2024-02-28T23:07:31.105 INFO:tasks.workunit.client.0.smithi057.stdout:4/205: write d1a/d1e/f42 [497004,103737] 0 2024-02-28T23:07:31.107 INFO:tasks.workunit.client.0.smithi057.stdout:3/124: write d3/d4/d7/f10 [67958,78283] 0 2024-02-28T23:07:31.108 INFO:tasks.workunit.client.0.smithi057.stdout:1/117: rename f0 to d2/d7/df/db/f20 0 2024-02-28T23:07:31.110 INFO:tasks.workunit.client.1.smithi084.stdout:0/291: write d1/d8/d18/d28/f54 [794430,19519] 0 2024-02-28T23:07:31.111 INFO:tasks.workunit.client.1.smithi084.stdout:8/262: dwrite d0/d9/f35 [0,4194304] 0 2024-02-28T23:07:31.113 INFO:tasks.workunit.client.0.smithi057.stdout:9/192: unlink dd/de/f1d 0 2024-02-28T23:07:31.114 INFO:tasks.workunit.client.1.smithi084.stdout:8/263: stat d0/l5 0 2024-02-28T23:07:31.116 INFO:tasks.workunit.client.0.smithi057.stdout:2/204: unlink d8/d11/d21/d28/f47 0 2024-02-28T23:07:31.116 INFO:tasks.workunit.client.1.smithi084.stdout:2/278: dwrite d5/d7/d2e/f2f [0,4194304] 0 2024-02-28T23:07:31.124 INFO:tasks.workunit.client.1.smithi084.stdout:2/279: unlink d5/dd/l27 0 2024-02-28T23:07:31.125 INFO:tasks.workunit.client.0.smithi057.stdout:5/122: dwrite d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:31.126 INFO:tasks.workunit.client.1.smithi084.stdout:8/264: write d0/d9/d30/f42 [558967,35651] 0 2024-02-28T23:07:31.128 INFO:tasks.workunit.client.0.smithi057.stdout:3/125: readlink d3/d8/l1f 0 2024-02-28T23:07:31.130 INFO:tasks.workunit.client.0.smithi057.stdout:3/126: dread - d3/d20/f23 zero size 2024-02-28T23:07:31.131 INFO:tasks.workunit.client.1.smithi084.stdout:0/292: link d1/d6/db/d35/f5c d1/d6/db/d35/f5e 0 2024-02-28T23:07:31.131 INFO:tasks.workunit.client.1.smithi084.stdout:2/280: mknod d5/d7/c4b 0 2024-02-28T23:07:31.132 INFO:tasks.workunit.client.1.smithi084.stdout:8/265: write d0/d11/f23 [2437861,127690] 0 2024-02-28T23:07:31.132 INFO:tasks.workunit.client.0.smithi057.stdout:6/171: mknod d0/d4/d25/c39 0 2024-02-28T23:07:31.133 INFO:tasks.workunit.client.1.smithi084.stdout:0/293: chown d1/d8/d18/l2f 10 1 2024-02-28T23:07:31.135 INFO:tasks.workunit.client.1.smithi084.stdout:8/266: write d0/d9/f21 [5437691,38503] 0 2024-02-28T23:07:31.136 INFO:tasks.workunit.client.0.smithi057.stdout:1/118: dread d2/d7/df/db/f20 [0,4194304] 0 2024-02-28T23:07:31.136 INFO:tasks.workunit.client.1.smithi084.stdout:0/294: mkdir d1/d8/d5f 0 2024-02-28T23:07:31.139 INFO:tasks.workunit.client.0.smithi057.stdout:1/119: truncate d2/d7/df/db/f18 4502589 0 2024-02-28T23:07:31.140 INFO:tasks.workunit.client.1.smithi084.stdout:8/267: rename d0/f34 to d0/d9/d1b/d20/f59 0 2024-02-28T23:07:31.143 INFO:tasks.workunit.client.1.smithi084.stdout:0/295: getdents d1/d6/db/d35/d4a/d56 0 2024-02-28T23:07:31.144 INFO:tasks.workunit.client.1.smithi084.stdout:0/296: chown d1/d8 43 1 2024-02-28T23:07:31.151 INFO:tasks.workunit.client.1.smithi084.stdout:0/297: link d1/d8/d18/l3d d1/d8/d18/d4c/l60 0 2024-02-28T23:07:31.153 INFO:tasks.workunit.client.0.smithi057.stdout:9/193: rename dd to dd/d33 22 2024-02-28T23:07:31.154 INFO:tasks.workunit.client.1.smithi084.stdout:0/298: write d1/d6/db/f5d [275791,107010] 0 2024-02-28T23:07:31.158 INFO:tasks.workunit.client.1.smithi084.stdout:0/299: dread - d1/d6/db/d35/d44/f4b zero size 2024-02-28T23:07:31.159 INFO:tasks.workunit.client.1.smithi084.stdout:0/300: write d1/d6/f13 [207836,126041] 0 2024-02-28T23:07:31.161 INFO:tasks.workunit.client.1.smithi084.stdout:0/301: dread - d1/d6/f53 zero size 2024-02-28T23:07:31.161 INFO:tasks.workunit.client.0.smithi057.stdout:4/206: symlink d1a/l50 0 2024-02-28T23:07:31.161 INFO:tasks.workunit.client.0.smithi057.stdout:0/163: sync 2024-02-28T23:07:31.164 INFO:tasks.workunit.client.1.smithi084.stdout:0/302: unlink d1/d6/d2d/f4f 0 2024-02-28T23:07:31.166 INFO:tasks.workunit.client.1.smithi084.stdout:0/303: truncate d1/d8/f29 4550374 0 2024-02-28T23:07:31.166 INFO:tasks.workunit.client.0.smithi057.stdout:6/172: stat d0/d4/d11/c28 0 2024-02-28T23:07:31.167 INFO:tasks.workunit.client.0.smithi057.stdout:5/123: write f1 [3567378,24147] 0 2024-02-28T23:07:31.167 INFO:tasks.workunit.client.0.smithi057.stdout:5/124: fsync f2 0 2024-02-28T23:07:31.169 INFO:tasks.workunit.client.0.smithi057.stdout:7/234: link d0/c26 d0/d8/d15/d29/c45 0 2024-02-28T23:07:31.170 INFO:tasks.workunit.client.1.smithi084.stdout:0/304: creat d1/d8/d18/f61 x:0 0 0 2024-02-28T23:07:31.172 INFO:tasks.workunit.client.1.smithi084.stdout:0/305: write d1/d6/f16 [1026852,102908] 0 2024-02-28T23:07:31.174 INFO:tasks.workunit.client.1.smithi084.stdout:0/306: readlink d1/d6/db/l2b 0 2024-02-28T23:07:31.174 INFO:tasks.workunit.client.1.smithi084.stdout:6/308: truncate d1/f1b 1127090 0 2024-02-28T23:07:31.177 INFO:tasks.workunit.client.1.smithi084.stdout:0/307: fdatasync d1/d8/f3e 0 2024-02-28T23:07:31.177 INFO:tasks.workunit.client.1.smithi084.stdout:6/309: write d1/d9/d1d/d1e/d33/d38/f54 [860289,48669] 0 2024-02-28T23:07:31.178 INFO:tasks.workunit.client.0.smithi057.stdout:2/205: dwrite d8/f16 [0,4194304] 0 2024-02-28T23:07:31.180 INFO:tasks.workunit.client.0.smithi057.stdout:0/164: rename d17/d20/c2a to d17/d20/d26/c3d 0 2024-02-28T23:07:31.181 INFO:tasks.workunit.client.1.smithi084.stdout:0/308: creat d1/d6/db/d35/d4a/f62 x:0 0 0 2024-02-28T23:07:31.183 INFO:tasks.workunit.client.1.smithi084.stdout:2/281: truncate d5/dd/f16 1693950 0 2024-02-28T23:07:31.183 INFO:tasks.workunit.client.1.smithi084.stdout:0/309: truncate d1/d6/f53 275580 0 2024-02-28T23:07:31.185 INFO:tasks.workunit.client.1.smithi084.stdout:2/282: write d5/d7/f3f [1254588,64984] 0 2024-02-28T23:07:31.185 INFO:tasks.workunit.client.0.smithi057.stdout:5/125: creat d4/d9/d16/d12/f22 x:0 0 0 2024-02-28T23:07:31.186 INFO:tasks.workunit.client.0.smithi057.stdout:2/206: write d8/d11/f17 [2298552,79825] 0 2024-02-28T23:07:31.189 INFO:tasks.workunit.client.1.smithi084.stdout:0/310: rename d1/d8/d18/f48 to d1/d8/d18/f63 0 2024-02-28T23:07:31.190 INFO:tasks.workunit.client.1.smithi084.stdout:2/283: mknod d5/dd/c4c 0 2024-02-28T23:07:31.191 INFO:tasks.workunit.client.1.smithi084.stdout:0/311: readlink d1/d6/d2d/l32 0 2024-02-28T23:07:31.193 INFO:tasks.workunit.client.0.smithi057.stdout:5/126: chown d4/d7/d8/f13 3003 1 2024-02-28T23:07:31.194 INFO:tasks.workunit.client.0.smithi057.stdout:6/173: dwrite d0/d4/ff [0,4194304] 0 2024-02-28T23:07:31.194 INFO:tasks.workunit.client.1.smithi084.stdout:2/284: chown d5/d7/d9/f32 51 1 2024-02-28T23:07:31.195 INFO:tasks.workunit.client.1.smithi084.stdout:6/310: dwrite d1/d9/f48 [0,4194304] 0 2024-02-28T23:07:31.199 INFO:tasks.workunit.client.1.smithi084.stdout:2/285: unlink d5/dd/l45 0 2024-02-28T23:07:31.201 INFO:tasks.workunit.client.1.smithi084.stdout:6/311: symlink d1/d9/d1d/d1e/l66 0 2024-02-28T23:07:31.203 INFO:tasks.workunit.client.0.smithi057.stdout:9/194: mknod dd/c34 0 2024-02-28T23:07:31.204 INFO:tasks.workunit.client.0.smithi057.stdout:0/165: symlink d17/d19/l3e 0 2024-02-28T23:07:31.206 INFO:tasks.workunit.client.1.smithi084.stdout:6/312: mknod d1/d13/d50/d65/c67 0 2024-02-28T23:07:31.207 INFO:tasks.workunit.client.0.smithi057.stdout:7/235: chown d0/d5/d25/d3e 695 1 2024-02-28T23:07:31.209 INFO:tasks.workunit.client.1.smithi084.stdout:8/268: truncate d0/d9/d1b/f28 966317 0 2024-02-28T23:07:31.209 INFO:tasks.workunit.client.0.smithi057.stdout:9/195: fsync dd/de/d14/f17 0 2024-02-28T23:07:31.216 INFO:tasks.workunit.client.0.smithi057.stdout:3/127: getdents d3/d8 0 2024-02-28T23:07:31.217 INFO:tasks.workunit.client.0.smithi057.stdout:4/207: dwrite f18 [0,4194304] 0 2024-02-28T23:07:31.218 INFO:tasks.workunit.client.0.smithi057.stdout:2/207: write d8/d11/d15/f20 [852148,89986] 0 2024-02-28T23:07:31.219 INFO:tasks.workunit.client.1.smithi084.stdout:8/269: link d0/d11/c52 d0/d11/c5a 0 2024-02-28T23:07:31.222 INFO:tasks.workunit.client.0.smithi057.stdout:6/174: unlink d0/d4/l15 0 2024-02-28T23:07:31.224 INFO:tasks.workunit.client.1.smithi084.stdout:8/270: creat d0/d24/f5b x:0 0 0 2024-02-28T23:07:31.225 INFO:tasks.workunit.client.0.smithi057.stdout:0/166: creat d17/d20/d26/d30/f3f x:0 0 0 2024-02-28T23:07:31.226 INFO:tasks.workunit.client.0.smithi057.stdout:7/236: read d0/d5/d36/f3c [556647,10820] 0 2024-02-28T23:07:31.226 INFO:tasks.workunit.client.0.smithi057.stdout:6/175: readlink d0/d4/d25/l29 0 2024-02-28T23:07:31.227 INFO:tasks.workunit.client.1.smithi084.stdout:3/281: sync 2024-02-28T23:07:31.228 INFO:tasks.workunit.client.1.smithi084.stdout:9/357: sync 2024-02-28T23:07:31.228 INFO:tasks.workunit.client.1.smithi084.stdout:1/384: sync 2024-02-28T23:07:31.231 INFO:tasks.workunit.client.1.smithi084.stdout:0/312: dread d1/d8/f1d [0,4194304] 0 2024-02-28T23:07:31.231 INFO:tasks.workunit.client.1.smithi084.stdout:9/358: fdatasync d4/d6/f29 0 2024-02-28T23:07:31.232 INFO:tasks.workunit.client.0.smithi057.stdout:5/127: link l3 d4/d21/l23 0 2024-02-28T23:07:31.232 INFO:tasks.workunit.client.1.smithi084.stdout:6/313: dwrite d1/d9/d1d/d1e/f5b [0,4194304] 0 2024-02-28T23:07:31.234 INFO:tasks.workunit.client.1.smithi084.stdout:0/313: read d1/d8/f1c [559348,55502] 0 2024-02-28T23:07:31.236 INFO:tasks.workunit.client.0.smithi057.stdout:3/128: rename l0 to d3/d8/l2a 0 2024-02-28T23:07:31.237 INFO:tasks.workunit.client.1.smithi084.stdout:8/271: rename d0/d44/f4a to d0/d9/d1b/d38/d43/d51/d54/f5c 0 2024-02-28T23:07:31.239 INFO:tasks.workunit.client.1.smithi084.stdout:9/359: write d4/d6/d15/d12/f48 [69037,96308] 0 2024-02-28T23:07:31.241 INFO:tasks.workunit.client.1.smithi084.stdout:9/360: fdatasync d4/d6/f17 0 2024-02-28T23:07:31.242 INFO:tasks.workunit.client.0.smithi057.stdout:6/176: mknod d0/d4/c3a 0 2024-02-28T23:07:31.243 INFO:tasks.workunit.client.0.smithi057.stdout:6/177: fdatasync d0/d17/f2c 0 2024-02-28T23:07:31.247 INFO:tasks.workunit.client.0.smithi057.stdout:3/129: chown d3/d20/f24 663913266 1 2024-02-28T23:07:31.247 INFO:tasks.workunit.client.1.smithi084.stdout:1/385: mknod d12/d1b/d2e/c89 0 2024-02-28T23:07:31.247 INFO:tasks.workunit.client.1.smithi084.stdout:3/282: rename d3/d8/l13 to d3/d8/d14/d31/l69 0 2024-02-28T23:07:31.250 INFO:tasks.workunit.client.1.smithi084.stdout:0/314: unlink d1/d8/f1c 0 2024-02-28T23:07:31.251 INFO:tasks.workunit.client.0.smithi057.stdout:5/128: chown d4/d21/l23 39732478 1 2024-02-28T23:07:31.251 INFO:tasks.workunit.client.1.smithi084.stdout:3/283: chown d3/d50/c57 216 1 2024-02-28T23:07:31.255 INFO:tasks.workunit.client.1.smithi084.stdout:0/315: readlink d1/d6/db/d37/l39 0 2024-02-28T23:07:31.257 INFO:tasks.workunit.client.0.smithi057.stdout:9/196: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:31.259 INFO:tasks.workunit.client.1.smithi084.stdout:8/272: link d0/d9/f21 d0/d9/d1b/d38/f5d 0 2024-02-28T23:07:31.259 INFO:tasks.workunit.client.1.smithi084.stdout:8/273: chown d0/d9/d1b/d38/d43/d51/d54 432 1 2024-02-28T23:07:31.259 INFO:tasks.workunit.client.1.smithi084.stdout:9/361: unlink d4/d6/d15/f18 0 2024-02-28T23:07:31.259 INFO:tasks.workunit.client.1.smithi084.stdout:3/284: read d3/d8/d14/d31/d34/f2b [3533399,25567] 0 2024-02-28T23:07:31.264 INFO:tasks.workunit.client.0.smithi057.stdout:0/167: link d17/l2d d17/d20/d26/l40 0 2024-02-28T23:07:31.264 INFO:tasks.workunit.client.1.smithi084.stdout:9/362: truncate d4/d6/d15/d12/d41/f64 331939 0 2024-02-28T23:07:31.265 INFO:tasks.workunit.client.0.smithi057.stdout:5/129: rmdir d4/d7/d8 39 2024-02-28T23:07:31.267 INFO:tasks.workunit.client.1.smithi084.stdout:1/386: creat d12/d1b/d2e/d52/d5b/f8a x:0 0 0 2024-02-28T23:07:31.272 INFO:tasks.workunit.client.1.smithi084.stdout:8/274: creat d0/d9/d30/d33/f5e x:0 0 0 2024-02-28T23:07:31.273 INFO:tasks.workunit.client.0.smithi057.stdout:9/197: creat dd/de/d14/f35 x:0 0 0 2024-02-28T23:07:31.275 INFO:tasks.workunit.client.0.smithi057.stdout:0/168: creat d17/d20/d26/f41 x:0 0 0 2024-02-28T23:07:31.276 INFO:tasks.workunit.client.0.smithi057.stdout:6/178: dread d0/d4/ff [0,4194304] 0 2024-02-28T23:07:31.278 INFO:tasks.workunit.client.1.smithi084.stdout:0/316: symlink d1/d6/db/d35/d4a/d56/l64 0 2024-02-28T23:07:31.279 INFO:tasks.workunit.client.0.smithi057.stdout:2/208: dwrite d8/f16 [0,4194304] 0 2024-02-28T23:07:31.279 INFO:tasks.workunit.client.0.smithi057.stdout:9/198: write dd/f2f [87833,118592] 0 2024-02-28T23:07:31.280 INFO:tasks.workunit.client.0.smithi057.stdout:5/130: rename d4/d9/d16/d12 to d4/d21/d24 0 2024-02-28T23:07:31.281 INFO:tasks.workunit.client.1.smithi084.stdout:3/285: mknod d3/d50/d5a/c6a 0 2024-02-28T23:07:31.281 INFO:tasks.workunit.client.1.smithi084.stdout:1/387: write d12/d1b/d2e/d52/f7c [681887,103787] 0 2024-02-28T23:07:31.281 INFO:tasks.workunit.client.0.smithi057.stdout:4/208: dwrite d1a/d1b/d24/f38 [0,4194304] 0 2024-02-28T23:07:31.282 INFO:tasks.workunit.client.1.smithi084.stdout:0/317: write d1/d8/f3f [772142,55278] 0 2024-02-28T23:07:31.286 INFO:tasks.workunit.client.1.smithi084.stdout:0/318: chown d1/la 13 1 2024-02-28T23:07:31.286 INFO:tasks.workunit.client.0.smithi057.stdout:7/237: dwrite d0/d8/d15/f2b [0,4194304] 0 2024-02-28T23:07:31.287 INFO:tasks.workunit.client.0.smithi057.stdout:9/199: chown c6 8523709 1 2024-02-28T23:07:31.287 INFO:tasks.workunit.client.1.smithi084.stdout:1/388: fdatasync d12/d1b/d3b/f42 0 2024-02-28T23:07:31.289 INFO:tasks.workunit.client.1.smithi084.stdout:9/363: rename d4/d6/c27 to d4/d6/c78 0 2024-02-28T23:07:31.290 INFO:tasks.workunit.client.1.smithi084.stdout:1/389: write d12/f40 [3252824,126670] 0 2024-02-28T23:07:31.290 INFO:tasks.workunit.client.0.smithi057.stdout:9/200: read - dd/de/d14/f2b zero size 2024-02-28T23:07:31.295 INFO:tasks.workunit.client.0.smithi057.stdout:9/201: write dd/de/d14/f1e [2640637,98090] 0 2024-02-28T23:07:31.295 INFO:tasks.workunit.client.1.smithi084.stdout:8/275: creat d0/d9/f5f x:0 0 0 2024-02-28T23:07:31.298 INFO:tasks.workunit.client.1.smithi084.stdout:0/319: rename d1/d6/f16 to d1/d8/d18/f65 0 2024-02-28T23:07:31.299 INFO:tasks.workunit.client.0.smithi057.stdout:9/202: read dd/de/d14/f18 [1923808,11240] 0 2024-02-28T23:07:31.299 INFO:tasks.workunit.client.1.smithi084.stdout:8/276: rename d0/d44 to d0/d44/d60 22 2024-02-28T23:07:31.301 INFO:tasks.workunit.client.0.smithi057.stdout:2/209: symlink d8/d11/d15/l4b 0 2024-02-28T23:07:31.303 INFO:tasks.workunit.client.0.smithi057.stdout:4/209: mkdir d1a/d1e/d51 0 2024-02-28T23:07:31.306 INFO:tasks.workunit.client.0.smithi057.stdout:9/203: chown dd/de/c23 4084 1 2024-02-28T23:07:31.307 INFO:tasks.workunit.client.0.smithi057.stdout:0/169: dwrite d17/d20/d26/d30/f39 [0,4194304] 0 2024-02-28T23:07:31.309 INFO:tasks.workunit.client.1.smithi084.stdout:1/390: dwrite d12/f23 [0,4194304] 0 2024-02-28T23:07:31.310 INFO:tasks.workunit.client.0.smithi057.stdout:4/210: chown d1a/d1b/f2c 53325 1 2024-02-28T23:07:31.313 INFO:tasks.workunit.client.0.smithi057.stdout:3/130: rename d3/d8/fb to d3/d4/d7/f2b 0 2024-02-28T23:07:31.313 INFO:tasks.workunit.client.0.smithi057.stdout:2/210: dread d8/d11/f32 [0,4194304] 0 2024-02-28T23:07:31.313 INFO:tasks.workunit.client.0.smithi057.stdout:4/211: fdatasync d1a/d1b/d2d/f35 0 2024-02-28T23:07:31.315 INFO:tasks.workunit.client.0.smithi057.stdout:0/170: creat d17/f42 x:0 0 0 2024-02-28T23:07:31.316 INFO:tasks.workunit.client.0.smithi057.stdout:7/238: creat d0/d8/d15/f46 x:0 0 0 2024-02-28T23:07:31.318 INFO:tasks.workunit.client.1.smithi084.stdout:0/320: dread d1/d8/f29 [0,4194304] 0 2024-02-28T23:07:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:31 smithi057 ceph-mon[29096]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:07:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:31 smithi057 ceph-mon[29096]: Upgrade: Need to upgrade myself (mgr.smithi084.fhkzis) 2024-02-28T23:07:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:31 smithi057 ceph-mon[29096]: Upgrade: Need to upgrade myself (mgr.smithi084.fhkzis) 2024-02-28T23:07:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:31 smithi057 ceph-mon[29096]: pgmap v22: 65 pgs: 65 active+clean; 955 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 10 MiB/s rd, 116 MiB/s wr, 328 op/s 2024-02-28T23:07:31.326 INFO:tasks.workunit.client.0.smithi057.stdout:0/171: creat d17/d20/d23/d37/f43 x:0 0 0 2024-02-28T23:07:31.327 INFO:tasks.workunit.client.1.smithi084.stdout:5/348: sync 2024-02-28T23:07:31.327 INFO:tasks.workunit.client.1.smithi084.stdout:4/274: sync 2024-02-28T23:07:31.327 INFO:tasks.workunit.client.1.smithi084.stdout:7/288: sync 2024-02-28T23:07:31.329 INFO:tasks.workunit.client.0.smithi057.stdout:7/239: fdatasync d0/d8/d15/f2b 0 2024-02-28T23:07:31.329 INFO:tasks.workunit.client.1.smithi084.stdout:3/286: rmdir d3/d8/d14/d31 39 2024-02-28T23:07:31.330 INFO:tasks.workunit.client.1.smithi084.stdout:5/349: chown dc/de/f79 0 1 2024-02-28T23:07:31.331 INFO:tasks.workunit.client.0.smithi057.stdout:0/172: unlink f8 0 2024-02-28T23:07:31.333 INFO:tasks.workunit.client.1.smithi084.stdout:8/277: creat d0/d9/d1b/d38/d43/d51/f61 x:0 0 0 2024-02-28T23:07:31.333 INFO:tasks.workunit.client.0.smithi057.stdout:2/211: rmdir d8 39 2024-02-28T23:07:31.333 INFO:tasks.workunit.client.0.smithi057.stdout:0/173: stat d17/d20/d23/c24 0 2024-02-28T23:07:31.335 INFO:tasks.workunit.client.1.smithi084.stdout:8/278: rename d0 to d0/d9/d1b/d62 22 2024-02-28T23:07:31.335 INFO:tasks.workunit.client.1.smithi084.stdout:0/321: unlink d1/d6/db/d35/d4a/l50 0 2024-02-28T23:07:31.335 INFO:tasks.workunit.client.0.smithi057.stdout:0/174: stat d17/d19 0 2024-02-28T23:07:31.335 INFO:tasks.workunit.client.0.smithi057.stdout:3/131: dwrite d3/d4/d1d/f1e [0,4194304] 0 2024-02-28T23:07:31.336 INFO:tasks.workunit.client.0.smithi057.stdout:0/175: fsync f4 0 2024-02-28T23:07:31.337 INFO:tasks.workunit.client.1.smithi084.stdout:9/364: rmdir d4/d6/d15/d12/d6d 0 2024-02-28T23:07:31.338 INFO:tasks.workunit.client.1.smithi084.stdout:6/314: truncate d1/d9/d1d/d1e/d33/d38/f54 235842 0 2024-02-28T23:07:31.338 INFO:tasks.workunit.client.1.smithi084.stdout:9/365: stat d4/d6/d53/f61 0 2024-02-28T23:07:31.339 INFO:tasks.workunit.client.0.smithi057.stdout:3/132: fdatasync d3/d4/d7/f2b 0 2024-02-28T23:07:31.339 INFO:tasks.workunit.client.0.smithi057.stdout:0/176: chown c2 1076 1 2024-02-28T23:07:31.340 INFO:tasks.workunit.client.1.smithi084.stdout:7/289: truncate db/d11/d27/f35 313193 0 2024-02-28T23:07:31.340 INFO:tasks.workunit.client.0.smithi057.stdout:7/240: rename d0/d8/d15/d29/c45 to d0/d5/d36/c47 0 2024-02-28T23:07:31.342 INFO:tasks.workunit.client.0.smithi057.stdout:3/133: write d3/d4/d1d/f1e [3479255,44690] 0 2024-02-28T23:07:31.342 INFO:tasks.workunit.client.0.smithi057.stdout:0/177: mknod d17/d20/d23/d37/c44 0 2024-02-28T23:07:31.344 INFO:tasks.workunit.client.1.smithi084.stdout:5/350: stat dc/de/df/d16/c23 0 2024-02-28T23:07:31.344 INFO:tasks.workunit.client.1.smithi084.stdout:7/290: stat db/d10/d17/d3b 0 2024-02-28T23:07:31.345 INFO:tasks.workunit.client.0.smithi057.stdout:6/179: dwrite d0/d4/fb [0,4194304] 0 2024-02-28T23:07:31.347 INFO:tasks.workunit.client.1.smithi084.stdout:7/291: write db/d10/f2c [5086736,112192] 0 2024-02-28T23:07:31.348 INFO:tasks.workunit.client.1.smithi084.stdout:1/391: rmdir d12/d1b/d2e 39 2024-02-28T23:07:31.349 INFO:tasks.workunit.client.1.smithi084.stdout:7/292: chown db/l59 34120074 1 2024-02-28T23:07:31.353 INFO:tasks.workunit.client.1.smithi084.stdout:0/322: read - d1/d6/db/d35/d44/f4b zero size 2024-02-28T23:07:31.354 INFO:tasks.workunit.client.1.smithi084.stdout:4/275: mknod d1/d4b/c5c 0 2024-02-28T23:07:31.355 INFO:tasks.workunit.client.0.smithi057.stdout:0/178: link f9 d17/d20/f45 0 2024-02-28T23:07:31.356 INFO:tasks.workunit.client.1.smithi084.stdout:0/323: chown d1/d8/f29 1 1 2024-02-28T23:07:31.356 INFO:tasks.workunit.client.1.smithi084.stdout:4/276: write d1/f4 [3806540,27756] 0 2024-02-28T23:07:31.358 INFO:tasks.workunit.client.0.smithi057.stdout:9/204: write dd/de/d14/f21 [3592437,12506] 0 2024-02-28T23:07:31.361 INFO:tasks.workunit.client.0.smithi057.stdout:3/134: dwrite d3/d20/f23 [0,4194304] 0 2024-02-28T23:07:31.362 INFO:tasks.workunit.client.1.smithi084.stdout:2/286: sync 2024-02-28T23:07:31.365 INFO:tasks.workunit.client.0.smithi057.stdout:0/179: rename f9 to d17/d20/d26/f46 0 2024-02-28T23:07:31.365 INFO:tasks.workunit.client.0.smithi057.stdout:0/180: stat d17 0 2024-02-28T23:07:31.367 INFO:tasks.workunit.client.0.smithi057.stdout:2/212: write d8/d11/f1b [1045713,16293] 0 2024-02-28T23:07:31.368 INFO:tasks.workunit.client.1.smithi084.stdout:3/287: creat d3/d8/d1a/d29/d41/d60/f6b x:0 0 0 2024-02-28T23:07:31.369 INFO:tasks.workunit.client.0.smithi057.stdout:7/241: truncate d0/d5/f2e 128184 0 2024-02-28T23:07:31.369 INFO:tasks.workunit.client.0.smithi057.stdout:2/213: dread - d8/d11/d39/f3c zero size 2024-02-28T23:07:31.370 INFO:tasks.workunit.client.0.smithi057.stdout:6/180: getdents d0/d1b/d22 0 2024-02-28T23:07:31.371 INFO:tasks.workunit.client.1.smithi084.stdout:2/287: write d5/d7/d12/d24/d3a/d48/f49 [113928,59860] 0 2024-02-28T23:07:31.371 INFO:tasks.workunit.client.0.smithi057.stdout:0/181: mkdir d17/d20/d23/d47 0 2024-02-28T23:07:31.372 INFO:tasks.workunit.client.0.smithi057.stdout:7/242: write d0/d5/f39 [1019640,13974] 0 2024-02-28T23:07:31.373 INFO:tasks.workunit.client.1.smithi084.stdout:8/279: mkdir d0/d9/d63 0 2024-02-28T23:07:31.374 INFO:tasks.workunit.client.0.smithi057.stdout:2/214: creat d8/d11/d39/f4c x:0 0 0 2024-02-28T23:07:31.375 INFO:tasks.workunit.client.0.smithi057.stdout:3/135: creat d3/f2c x:0 0 0 2024-02-28T23:07:31.376 INFO:tasks.workunit.client.0.smithi057.stdout:2/215: dread - d8/d11/d39/f3c zero size 2024-02-28T23:07:31.377 INFO:tasks.workunit.client.0.smithi057.stdout:6/181: write d0/d4/ff [4935244,78930] 0 2024-02-28T23:07:31.377 INFO:tasks.workunit.client.0.smithi057.stdout:0/182: mkdir d17/d20/d26/d48 0 2024-02-28T23:07:31.378 INFO:tasks.workunit.client.1.smithi084.stdout:7/293: read db/f1e [6030666,33609] 0 2024-02-28T23:07:31.379 INFO:tasks.workunit.client.0.smithi057.stdout:6/182: chown d0/d4/f19 49 1 2024-02-28T23:07:31.379 INFO:tasks.workunit.client.0.smithi057.stdout:2/216: write d8/d11/d15/f2d [265981,94888] 0 2024-02-28T23:07:31.381 INFO:tasks.workunit.client.1.smithi084.stdout:6/315: chown d1/d9/d1d/d1e/d33/d38/f54 377336 1 2024-02-28T23:07:31.381 INFO:tasks.workunit.client.1.smithi084.stdout:1/392: symlink d12/l8b 0 2024-02-28T23:07:31.382 INFO:tasks.workunit.client.1.smithi084.stdout:8/280: dread - d0/d9/d1b/d38/d43/d51/f61 zero size 2024-02-28T23:07:31.383 INFO:tasks.workunit.client.0.smithi057.stdout:2/217: write d8/d11/d21/f31 [249926,86289] 0 2024-02-28T23:07:31.384 INFO:tasks.workunit.client.1.smithi084.stdout:9/366: dread d4/d6/f8 [0,4194304] 0 2024-02-28T23:07:31.384 INFO:tasks.workunit.client.1.smithi084.stdout:7/294: write f9 [2240692,100910] 0 2024-02-28T23:07:31.384 INFO:tasks.workunit.client.0.smithi057.stdout:3/136: mkdir d3/d20/d2d 0 2024-02-28T23:07:31.384 INFO:tasks.workunit.client.0.smithi057.stdout:2/218: readlink d8/d11/d21/d28/l3f 0 2024-02-28T23:07:31.387 INFO:tasks.workunit.client.0.smithi057.stdout:9/205: write dd/f12 [551842,25216] 0 2024-02-28T23:07:31.388 INFO:tasks.workunit.client.1.smithi084.stdout:8/281: write d0/f8 [4499495,89352] 0 2024-02-28T23:07:31.388 INFO:tasks.workunit.client.1.smithi084.stdout:2/288: creat d5/d7/d1d/f4d x:0 0 0 2024-02-28T23:07:31.391 INFO:tasks.workunit.client.0.smithi057.stdout:0/183: truncate f4 2228491 0 2024-02-28T23:07:31.392 INFO:tasks.workunit.client.0.smithi057.stdout:9/206: symlink dd/de/d24/l36 0 2024-02-28T23:07:31.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/219: symlink d8/d11/d48/l4d 0 2024-02-28T23:07:31.395 INFO:tasks.workunit.client.1.smithi084.stdout:3/288: getdents d3/d8/d1a/d45 0 2024-02-28T23:07:31.396 INFO:tasks.workunit.client.1.smithi084.stdout:4/277: truncate d1/d18/d20/d24/d2f/d3e/d52/d37/d55/f1a 1104423 0 2024-02-28T23:07:31.397 INFO:tasks.workunit.client.0.smithi057.stdout:0/184: symlink d17/l49 0 2024-02-28T23:07:31.398 INFO:tasks.workunit.client.1.smithi084.stdout:8/282: mkdir d0/d9/d63/d64 0 2024-02-28T23:07:31.402 INFO:tasks.workunit.client.0.smithi057.stdout:0/185: dread - d17/d20/d26/d30/f31 zero size 2024-02-28T23:07:31.402 INFO:tasks.workunit.client.0.smithi057.stdout:9/207: mkdir dd/de/d14/d37 0 2024-02-28T23:07:31.402 INFO:tasks.workunit.client.1.smithi084.stdout:6/316: truncate d1/d13/d2e/d41/f37 2434922 0 2024-02-28T23:07:31.403 INFO:tasks.workunit.client.0.smithi057.stdout:0/186: truncate f14 120611 0 2024-02-28T23:07:31.404 INFO:tasks.workunit.client.1.smithi084.stdout:7/295: getdents db/d11/d27 0 2024-02-28T23:07:31.404 INFO:tasks.workunit.client.1.smithi084.stdout:4/278: symlink d1/d4b/l5d 0 2024-02-28T23:07:31.406 INFO:tasks.workunit.client.0.smithi057.stdout:9/208: write dd/de/d24/f28 [840899,69306] 0 2024-02-28T23:07:31.407 INFO:tasks.workunit.client.1.smithi084.stdout:3/289: creat d3/d8/d1a/d21/d3d/d43/f6c x:0 0 0 2024-02-28T23:07:31.407 INFO:tasks.workunit.client.0.smithi057.stdout:3/137: dwrite d3/d4/d1d/f1e [0,4194304] 0 2024-02-28T23:07:31.409 INFO:tasks.workunit.client.1.smithi084.stdout:8/283: symlink d0/d24/l65 0 2024-02-28T23:07:31.411 INFO:tasks.workunit.client.0.smithi057.stdout:2/220: rename f7 to d8/d11/d21/d28/d49/f4e 0 2024-02-28T23:07:31.416 INFO:tasks.workunit.client.1.smithi084.stdout:5/351: dwrite dc/de/df/d16/f45 [0,4194304] 0 2024-02-28T23:07:31.419 INFO:tasks.workunit.client.1.smithi084.stdout:9/367: dread d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:31.420 INFO:tasks.workunit.client.0.smithi057.stdout:3/138: rename f1 to d3/d4/d1d/f2e 0 2024-02-28T23:07:31.421 INFO:tasks.workunit.client.1.smithi084.stdout:2/289: dread d5/dd/f34 [0,4194304] 0 2024-02-28T23:07:31.423 INFO:tasks.workunit.client.1.smithi084.stdout:8/284: write d0/d9/d1b/d38/f3a [210348,88899] 0 2024-02-28T23:07:31.424 INFO:tasks.workunit.client.0.smithi057.stdout:6/183: dwrite d0/d4/d25/f2f [0,4194304] 0 2024-02-28T23:07:31.425 INFO:tasks.workunit.client.0.smithi057.stdout:6/184: rename d0 to d0/d17/d38/d3b 22 2024-02-28T23:07:31.426 INFO:tasks.workunit.client.1.smithi084.stdout:7/296: creat db/d10/f5d x:0 0 0 2024-02-28T23:07:31.426 INFO:tasks.workunit.client.0.smithi057.stdout:9/209: rmdir dd/de/d14/d37 0 2024-02-28T23:07:31.427 INFO:tasks.workunit.client.0.smithi057.stdout:7/243: dread d0/d5/f39 [0,4194304] 0 2024-02-28T23:07:31.428 INFO:tasks.workunit.client.1.smithi084.stdout:3/290: write d3/d8/d1a/d21/d3d/d43/f48 [521610,2786] 0 2024-02-28T23:07:31.428 INFO:tasks.workunit.client.0.smithi057.stdout:6/185: chown d0/d4/d25/f2a 4045 1 2024-02-28T23:07:31.430 INFO:tasks.workunit.client.1.smithi084.stdout:6/317: rename d1/c5a to d1/d13/d2e/c68 0 2024-02-28T23:07:31.431 INFO:tasks.workunit.client.0.smithi057.stdout:2/221: creat d8/f4f x:0 0 0 2024-02-28T23:07:31.433 INFO:tasks.workunit.client.0.smithi057.stdout:9/210: truncate dd/de/d14/f2b 776704 0 2024-02-28T23:07:31.435 INFO:tasks.workunit.client.1.smithi084.stdout:1/393: dwrite d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:31.436 INFO:tasks.workunit.client.0.smithi057.stdout:2/222: mkdir d8/d11/d48/d50 0 2024-02-28T23:07:31.437 INFO:tasks.workunit.client.0.smithi057.stdout:3/139: truncate d3/d8/f16 1590883 0 2024-02-28T23:07:31.437 INFO:tasks.workunit.client.0.smithi057.stdout:9/211: rmdir dd/d1b 39 2024-02-28T23:07:31.440 INFO:tasks.workunit.client.0.smithi057.stdout:2/223: chown d8/d11/f1d 11 1 2024-02-28T23:07:31.441 INFO:tasks.workunit.client.0.smithi057.stdout:9/212: mknod dd/de/d24/c38 0 2024-02-28T23:07:31.442 INFO:tasks.workunit.client.0.smithi057.stdout:2/224: readlink d8/d11/d48/l4d 0 2024-02-28T23:07:31.443 INFO:tasks.workunit.client.1.smithi084.stdout:8/285: symlink d0/d9/d1b/d20/l66 0 2024-02-28T23:07:31.444 INFO:tasks.workunit.client.0.smithi057.stdout:3/140: mknod d3/d4/c2f 0 2024-02-28T23:07:31.447 INFO:tasks.workunit.client.0.smithi057.stdout:0/187: dread d17/d20/f45 [0,4194304] 0 2024-02-28T23:07:31.448 INFO:tasks.workunit.client.1.smithi084.stdout:4/279: dwrite d1/d18/d20/d14/d30/f3d [0,4194304] 0 2024-02-28T23:07:31.449 INFO:tasks.workunit.client.1.smithi084.stdout:1/394: write d12/d1b/d63/f3f [8925270,119593] 0 2024-02-28T23:07:31.451 INFO:tasks.workunit.client.0.smithi057.stdout:0/188: mkdir d17/d20/d4a 0 2024-02-28T23:07:31.451 INFO:tasks.workunit.client.1.smithi084.stdout:9/368: dread d4/d32/d3f/f42 [0,4194304] 0 2024-02-28T23:07:31.451 INFO:tasks.workunit.client.1.smithi084.stdout:8/286: chown d0/d11/c32 23855162 1 2024-02-28T23:07:31.452 INFO:tasks.workunit.client.0.smithi057.stdout:9/213: getdents dd 0 2024-02-28T23:07:31.455 INFO:tasks.workunit.client.0.smithi057.stdout:9/214: symlink dd/d1b/l39 0 2024-02-28T23:07:31.457 INFO:tasks.workunit.client.0.smithi057.stdout:2/225: dread d8/ff [0,4194304] 0 2024-02-28T23:07:31.458 INFO:tasks.workunit.client.0.smithi057.stdout:9/215: mkdir dd/d3a 0 2024-02-28T23:07:31.461 INFO:tasks.workunit.client.1.smithi084.stdout:2/290: dread d5/d7/f3f [0,4194304] 0 2024-02-28T23:07:31.462 INFO:tasks.workunit.client.1.smithi084.stdout:3/291: creat d3/d8/d14/f6d x:0 0 0 2024-02-28T23:07:31.463 INFO:tasks.workunit.client.0.smithi057.stdout:2/226: mkdir d8/d51 0 2024-02-28T23:07:31.463 INFO:tasks.workunit.client.1.smithi084.stdout:8/287: creat d0/d44/f67 x:0 0 0 2024-02-28T23:07:31.464 INFO:tasks.workunit.client.1.smithi084.stdout:7/297: dwrite db/d10/f2e [0,4194304] 0 2024-02-28T23:07:31.465 INFO:tasks.workunit.client.1.smithi084.stdout:1/395: chown d12/d1b/d2e/d30/d77 15047 1 2024-02-28T23:07:31.468 INFO:tasks.workunit.client.1.smithi084.stdout:7/298: fdatasync db/d10/d17/f58 0 2024-02-28T23:07:31.469 INFO:tasks.workunit.client.0.smithi057.stdout:2/227: unlink d8/d11/d21/l26 0 2024-02-28T23:07:31.470 INFO:tasks.workunit.client.1.smithi084.stdout:6/318: rmdir d1/d13 39 2024-02-28T23:07:31.472 INFO:tasks.workunit.client.0.smithi057.stdout:7/244: truncate d0/d5/f27 216894 0 2024-02-28T23:07:31.474 INFO:tasks.workunit.client.0.smithi057.stdout:6/186: truncate d0/d4/d25/f2f 1293100 0 2024-02-28T23:07:31.474 INFO:tasks.workunit.client.0.smithi057.stdout:7/245: rename d0/d5/d25 to d0/d5/d25/d44/d48 22 2024-02-28T23:07:31.475 INFO:tasks.workunit.client.1.smithi084.stdout:1/396: write d12/d1b/d2e/d52/f60 [235465,41191] 0 2024-02-28T23:07:31.481 INFO:tasks.workunit.client.0.smithi057.stdout:6/187: creat d0/f3c x:0 0 0 2024-02-28T23:07:31.481 INFO:tasks.workunit.client.1.smithi084.stdout:1/397: dread - d12/d1b/d2e/d30/f3e zero size 2024-02-28T23:07:31.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:31 smithi084 ceph-mon[34826]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:07:31.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:31 smithi084 ceph-mon[34826]: Upgrade: Need to upgrade myself (mgr.smithi084.fhkzis) 2024-02-28T23:07:31.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:31 smithi084 ceph-mon[34826]: Upgrade: Need to upgrade myself (mgr.smithi084.fhkzis) 2024-02-28T23:07:31.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:31 smithi084 ceph-mon[34826]: pgmap v22: 65 pgs: 65 active+clean; 955 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 10 MiB/s rd, 116 MiB/s wr, 328 op/s 2024-02-28T23:07:31.484 INFO:tasks.workunit.client.0.smithi057.stdout:9/216: write dd/f2f [968269,54144] 0 2024-02-28T23:07:31.485 INFO:tasks.workunit.client.1.smithi084.stdout:3/292: creat d3/d8/d1a/d21/d3d/d43/d61/f6e x:0 0 0 2024-02-28T23:07:31.487 INFO:tasks.workunit.client.0.smithi057.stdout:6/188: mkdir d0/d4/d11/d1f/d3d 0 2024-02-28T23:07:31.490 INFO:tasks.workunit.client.1.smithi084.stdout:4/280: dwrite d1/f2e [0,4194304] 0 2024-02-28T23:07:31.491 INFO:tasks.workunit.client.0.smithi057.stdout:1/120: sync 2024-02-28T23:07:31.491 INFO:tasks.workunit.client.0.smithi057.stdout:8/192: sync 2024-02-28T23:07:31.491 INFO:tasks.workunit.client.0.smithi057.stdout:1/121: stat d2/l1e 0 2024-02-28T23:07:31.491 INFO:tasks.workunit.client.0.smithi057.stdout:6/189: unlink d0/l10 0 2024-02-28T23:07:31.492 INFO:tasks.workunit.client.0.smithi057.stdout:8/193: stat d4/d13/c30 0 2024-02-28T23:07:31.495 INFO:tasks.workunit.client.0.smithi057.stdout:8/194: mknod d4/d1e/c41 0 2024-02-28T23:07:31.496 INFO:tasks.workunit.client.1.smithi084.stdout:2/291: rename d5/d7/fe to d5/d7/d12/d24/d3a/f4e 0 2024-02-28T23:07:31.497 INFO:tasks.workunit.client.0.smithi057.stdout:1/122: read d2/d7/df/db/f17 [1298733,101992] 0 2024-02-28T23:07:31.497 INFO:tasks.workunit.client.1.smithi084.stdout:3/293: fsync d3/d8/d1a/d21/d3d/f67 0 2024-02-28T23:07:31.498 INFO:tasks.workunit.client.1.smithi084.stdout:9/369: dwrite d4/f60 [0,4194304] 0 2024-02-28T23:07:31.498 INFO:tasks.workunit.client.1.smithi084.stdout:8/288: link d0/d9/d1b/d20/f25 d0/d24/d4d/f68 0 2024-02-28T23:07:31.499 INFO:tasks.workunit.client.0.smithi057.stdout:8/195: truncate d4/f5 1223071 0 2024-02-28T23:07:31.502 INFO:tasks.workunit.client.1.smithi084.stdout:3/294: chown d3/d8/d1a/d21/d3d/d43/f6c 3841 1 2024-02-28T23:07:31.502 INFO:tasks.workunit.client.0.smithi057.stdout:8/196: write d4/d1e/f36 [617619,71040] 0 2024-02-28T23:07:31.504 INFO:tasks.workunit.client.0.smithi057.stdout:1/123: creat d2/f21 x:0 0 0 2024-02-28T23:07:31.505 INFO:tasks.workunit.client.0.smithi057.stdout:9/217: dwrite dd/de/d14/f35 [0,4194304] 0 2024-02-28T23:07:31.506 INFO:tasks.workunit.client.0.smithi057.stdout:8/197: fsync d4/d13/f2d 0 2024-02-28T23:07:31.508 INFO:tasks.workunit.client.1.smithi084.stdout:2/292: mkdir d5/d7/d12/d4f 0 2024-02-28T23:07:31.516 INFO:tasks.workunit.client.1.smithi084.stdout:8/289: rmdir d0/d9/d1b/d38/d43/d51/d54 39 2024-02-28T23:07:31.516 INFO:tasks.workunit.client.1.smithi084.stdout:7/299: truncate db/d10/f3e 903634 0 2024-02-28T23:07:31.518 INFO:tasks.workunit.client.1.smithi084.stdout:5/352: dread dc/de/f1a [0,4194304] 0 2024-02-28T23:07:31.519 INFO:tasks.workunit.client.0.smithi057.stdout:8/198: dread d4/f17 [0,4194304] 0 2024-02-28T23:07:31.520 INFO:tasks.workunit.client.1.smithi084.stdout:3/295: mknod d3/d8/d1a/d21/c6f 0 2024-02-28T23:07:31.523 INFO:tasks.workunit.client.1.smithi084.stdout:4/281: dwrite d1/d18/f42 [0,4194304] 0 2024-02-28T23:07:31.525 INFO:tasks.workunit.client.0.smithi057.stdout:7/246: write d0/d8/d15/f1a [1668216,121471] 0 2024-02-28T23:07:31.527 INFO:tasks.workunit.client.1.smithi084.stdout:1/398: write d12/d1b/d2e/d32/f47 [4416465,93398] 0 2024-02-28T23:07:31.529 INFO:tasks.workunit.client.1.smithi084.stdout:9/370: creat d4/f79 x:0 0 0 2024-02-28T23:07:31.531 INFO:tasks.workunit.client.1.smithi084.stdout:7/300: truncate db/d10/d17/f58 2789723 0 2024-02-28T23:07:31.532 INFO:tasks.workunit.client.0.smithi057.stdout:7/247: creat d0/d8/d15/f49 x:0 0 0 2024-02-28T23:07:31.532 INFO:tasks.workunit.client.0.smithi057.stdout:0/189: dread f5 [8388608,4194304] 0 2024-02-28T23:07:31.533 INFO:tasks.workunit.client.1.smithi084.stdout:4/282: write d1/d18/d20/d14/d30/f3d [4652552,69827] 0 2024-02-28T23:07:31.533 INFO:tasks.workunit.client.1.smithi084.stdout:5/353: creat dc/de/df/d31/f7e x:0 0 0 2024-02-28T23:07:31.535 INFO:tasks.workunit.client.1.smithi084.stdout:9/371: fdatasync d4/d6/d15/d75/d2d/f34 0 2024-02-28T23:07:31.535 INFO:tasks.workunit.client.0.smithi057.stdout:6/190: rename d0/d4/d11/d1f to d0/d1b/d3e 0 2024-02-28T23:07:31.537 INFO:tasks.workunit.client.0.smithi057.stdout:6/191: dread - d0/d1b/f27 zero size 2024-02-28T23:07:31.539 INFO:tasks.workunit.client.1.smithi084.stdout:9/372: chown d4 2 1 2024-02-28T23:07:31.539 INFO:tasks.workunit.client.0.smithi057.stdout:6/192: stat d0/d1b/d3e 0 2024-02-28T23:07:31.540 INFO:tasks.workunit.client.1.smithi084.stdout:5/354: write dc/de/df/f20 [3965092,120597] 0 2024-02-28T23:07:31.541 INFO:tasks.workunit.client.1.smithi084.stdout:4/283: dread - d1/d18/d20/f11 zero size 2024-02-28T23:07:31.541 INFO:tasks.workunit.client.1.smithi084.stdout:3/296: fsync d3/d8/d1a/d45/d5e/f62 0 2024-02-28T23:07:31.541 INFO:tasks.workunit.client.0.smithi057.stdout:0/190: rmdir d17/d20/d23/d37 39 2024-02-28T23:07:31.542 INFO:tasks.workunit.client.1.smithi084.stdout:0/324: dread d1/d6/d1b/f22 [0,4194304] 0 2024-02-28T23:07:31.543 INFO:tasks.workunit.client.0.smithi057.stdout:8/199: dwrite d4/f7 [8388608,4194304] 0 2024-02-28T23:07:31.546 INFO:tasks.workunit.client.1.smithi084.stdout:7/301: mkdir db/d5e 0 2024-02-28T23:07:31.548 INFO:tasks.workunit.client.1.smithi084.stdout:9/373: truncate d4/d32/d3f/f42 4349490 0 2024-02-28T23:07:31.548 INFO:tasks.workunit.client.0.smithi057.stdout:9/218: rename dd/de to dd/de/d3b 22 2024-02-28T23:07:31.548 INFO:tasks.workunit.client.0.smithi057.stdout:6/193: symlink d0/d17/d20/l3f 0 2024-02-28T23:07:31.549 INFO:tasks.workunit.client.1.smithi084.stdout:5/355: symlink dc/d51/l7f 0 2024-02-28T23:07:31.552 INFO:tasks.workunit.client.0.smithi057.stdout:6/194: write d0/d17/f35 [636612,103521] 0 2024-02-28T23:07:31.552 INFO:tasks.workunit.client.1.smithi084.stdout:9/374: write d4/d6/d15/f71 [972819,90069] 0 2024-02-28T23:07:31.553 INFO:tasks.workunit.client.1.smithi084.stdout:7/302: stat db/d11/d27/f3a 0 2024-02-28T23:07:31.554 INFO:tasks.workunit.client.1.smithi084.stdout:0/325: unlink d1/d6/db/f5d 0 2024-02-28T23:07:31.554 INFO:tasks.workunit.client.0.smithi057.stdout:8/200: unlink d4/d13/d1a/f3d 0 2024-02-28T23:07:31.555 INFO:tasks.workunit.client.0.smithi057.stdout:6/195: write d0/d17/f2b [270273,23304] 0 2024-02-28T23:07:31.556 INFO:tasks.workunit.client.0.smithi057.stdout:6/196: fdatasync d0/fd 0 2024-02-28T23:07:31.558 INFO:tasks.workunit.client.1.smithi084.stdout:3/297: creat d3/d8/d1a/d21/d3d/d43/f70 x:0 0 0 2024-02-28T23:07:31.559 INFO:tasks.workunit.client.0.smithi057.stdout:8/201: chown d4/d13/f2b 501 1 2024-02-28T23:07:31.563 INFO:tasks.workunit.client.1.smithi084.stdout:3/298: fsync d3/d8/d14/f2e 0 2024-02-28T23:07:31.563 INFO:tasks.workunit.client.1.smithi084.stdout:8/290: dwrite d0/d9/d30/f42 [0,4194304] 0 2024-02-28T23:07:31.563 INFO:tasks.workunit.client.1.smithi084.stdout:0/326: write d1/d8/d18/d28/f54 [3225745,77454] 0 2024-02-28T23:07:31.564 INFO:tasks.workunit.client.0.smithi057.stdout:0/191: dread d17/f18 [0,4194304] 0 2024-02-28T23:07:31.564 INFO:tasks.workunit.client.0.smithi057.stdout:1/124: mkdir d2/d7/df/d22 0 2024-02-28T23:07:31.565 INFO:tasks.workunit.client.0.smithi057.stdout:8/202: truncate d4/d13/d18/f26 1093834 0 2024-02-28T23:07:31.565 INFO:tasks.workunit.client.0.smithi057.stdout:7/248: dread d0/d8/d15/d29/f32 [0,4194304] 0 2024-02-28T23:07:31.565 INFO:tasks.workunit.client.0.smithi057.stdout:9/219: creat dd/f3c x:0 0 0 2024-02-28T23:07:31.566 INFO:tasks.workunit.client.1.smithi084.stdout:3/299: write d3/d8/df/d2f/f3f [1694739,29381] 0 2024-02-28T23:07:31.568 INFO:tasks.workunit.client.1.smithi084.stdout:0/327: fsync d1/d6/db/d35/d4a/f62 0 2024-02-28T23:07:31.569 INFO:tasks.workunit.client.1.smithi084.stdout:6/319: dread d1/d9/d1d/d1e/d33/f64 [0,4194304] 0 2024-02-28T23:07:31.571 INFO:tasks.workunit.client.0.smithi057.stdout:8/203: write d4/d13/d18/f23 [1013274,33314] 0 2024-02-28T23:07:31.571 INFO:tasks.workunit.client.1.smithi084.stdout:0/328: fsync d1/d8/f3f 0 2024-02-28T23:07:31.574 INFO:tasks.workunit.client.0.smithi057.stdout:0/192: mkdir d17/d20/d23/d4b 0 2024-02-28T23:07:31.574 INFO:tasks.workunit.client.1.smithi084.stdout:4/284: link d1/d18/d20/d24/d2f/d3e/d52/d37/d55/f16 d1/d4a/f5e 0 2024-02-28T23:07:31.576 INFO:tasks.workunit.client.1.smithi084.stdout:4/285: dread - d1/d18/d20/f11 zero size 2024-02-28T23:07:31.576 INFO:tasks.workunit.client.1.smithi084.stdout:5/356: creat dc/de/df/d16/d34/d54/f80 x:0 0 0 2024-02-28T23:07:31.580 INFO:tasks.workunit.client.0.smithi057.stdout:7/249: dread d0/d5/f39 [0,4194304] 0 2024-02-28T23:07:31.582 INFO:tasks.workunit.client.0.smithi057.stdout:9/220: link dd/de/d14/f35 dd/de/d14/f3d 0 2024-02-28T23:07:31.582 INFO:tasks.workunit.client.1.smithi084.stdout:9/375: creat d4/d6/d15/d3c/f7a x:0 0 0 2024-02-28T23:07:31.583 INFO:tasks.workunit.client.1.smithi084.stdout:8/291: symlink d0/d9/d63/l69 0 2024-02-28T23:07:31.583 INFO:tasks.workunit.client.1.smithi084.stdout:2/293: dread d5/f15 [0,4194304] 0 2024-02-28T23:07:31.584 INFO:tasks.workunit.client.0.smithi057.stdout:0/193: creat d17/d20/d26/f4c x:0 0 0 2024-02-28T23:07:31.586 INFO:tasks.workunit.client.0.smithi057.stdout:6/197: dwrite d0/f3c [0,4194304] 0 2024-02-28T23:07:31.586 INFO:tasks.workunit.client.0.smithi057.stdout:8/204: mkdir d4/d1e/d37/d3f/d42 0 2024-02-28T23:07:31.588 INFO:tasks.workunit.client.0.smithi057.stdout:7/250: mkdir d0/d5/d4a 0 2024-02-28T23:07:31.588 INFO:tasks.workunit.client.0.smithi057.stdout:0/194: mkdir d17/d19/d4d 0 2024-02-28T23:07:31.589 INFO:tasks.workunit.client.0.smithi057.stdout:8/205: fdatasync d4/d13/d18/f23 0 2024-02-28T23:07:31.592 INFO:tasks.workunit.client.0.smithi057.stdout:6/198: mkdir d0/d1b/d3e/d40 0 2024-02-28T23:07:31.592 INFO:tasks.workunit.client.0.smithi057.stdout:0/195: write d17/d20/d26/d30/f39 [4433780,92923] 0 2024-02-28T23:07:31.594 INFO:tasks.workunit.client.1.smithi084.stdout:3/300: rename d3/d8/d14/f5c to d3/d8/d1a/d29/d41/f71 0 2024-02-28T23:07:31.594 INFO:tasks.workunit.client.0.smithi057.stdout:8/206: mknod d4/d13/c43 0 2024-02-28T23:07:31.596 INFO:tasks.workunit.client.0.smithi057.stdout:0/196: symlink d17/d19/l4e 0 2024-02-28T23:07:31.597 INFO:tasks.workunit.client.0.smithi057.stdout:8/207: chown d4/d13/d1a/f20 4446265 1 2024-02-28T23:07:31.598 INFO:tasks.workunit.client.0.smithi057.stdout:0/197: write d17/f1e [424671,87104] 0 2024-02-28T23:07:31.599 INFO:tasks.workunit.client.0.smithi057.stdout:6/199: mknod d0/d1b/d3e/d40/c41 0 2024-02-28T23:07:31.599 INFO:tasks.workunit.client.1.smithi084.stdout:0/329: creat d1/d6/db/d35/d4a/f66 x:0 0 0 2024-02-28T23:07:31.600 INFO:tasks.workunit.client.1.smithi084.stdout:4/286: unlink d1/d18/d20/c10 0 2024-02-28T23:07:31.601 INFO:tasks.workunit.client.1.smithi084.stdout:8/292: creat d0/d24/d4d/f6a x:0 0 0 2024-02-28T23:07:31.601 INFO:tasks.workunit.client.0.smithi057.stdout:1/125: getdents d2/d7/df 0 2024-02-28T23:07:31.603 INFO:tasks.workunit.client.0.smithi057.stdout:6/200: dread - d0/d17/f2c zero size 2024-02-28T23:07:31.604 INFO:tasks.workunit.client.0.smithi057.stdout:1/126: dread - d2/f21 zero size 2024-02-28T23:07:31.604 INFO:tasks.workunit.client.0.smithi057.stdout:8/208: readlink d4/d13/d1a/l24 0 2024-02-28T23:07:31.604 INFO:tasks.workunit.client.1.smithi084.stdout:6/320: mkdir d1/d13/d47/d69 0 2024-02-28T23:07:31.606 INFO:tasks.workunit.client.0.smithi057.stdout:9/221: dwrite dd/f31 [0,4194304] 0 2024-02-28T23:07:31.606 INFO:tasks.workunit.client.1.smithi084.stdout:9/376: symlink d4/d69/l7b 0 2024-02-28T23:07:31.607 INFO:tasks.workunit.client.1.smithi084.stdout:0/330: symlink d1/d8/d18/l67 0 2024-02-28T23:07:31.609 INFO:tasks.workunit.client.0.smithi057.stdout:8/209: truncate d4/d13/d18/f23 1416546 0 2024-02-28T23:07:31.609 INFO:tasks.workunit.client.0.smithi057.stdout:9/222: write dd/de/d14/f2b [1554608,103751] 0 2024-02-28T23:07:31.610 INFO:tasks.workunit.client.1.smithi084.stdout:7/303: getdents db/d10/d17 0 2024-02-28T23:07:31.611 INFO:tasks.workunit.client.0.smithi057.stdout:1/127: chown d2/f15 6067 1 2024-02-28T23:07:31.611 INFO:tasks.workunit.client.0.smithi057.stdout:6/201: dread d0/d4/d25/f2f [0,4194304] 0 2024-02-28T23:07:31.612 INFO:tasks.workunit.client.1.smithi084.stdout:8/293: chown d0/d9/d1b/d38/d43/d51/d54 4567 1 2024-02-28T23:07:31.612 INFO:tasks.workunit.client.1.smithi084.stdout:5/357: creat dc/de/df/d16/f81 x:0 0 0 2024-02-28T23:07:31.613 INFO:tasks.workunit.client.0.smithi057.stdout:8/210: write d4/d13/d18/f31 [429805,98261] 0 2024-02-28T23:07:31.613 INFO:tasks.workunit.client.1.smithi084.stdout:6/321: write d1/d9/d1d/d1e/f5b [3422964,115770] 0 2024-02-28T23:07:31.614 INFO:tasks.workunit.client.0.smithi057.stdout:9/223: mknod dd/d3a/c3e 0 2024-02-28T23:07:31.616 INFO:tasks.workunit.client.0.smithi057.stdout:1/128: creat d2/d7/df/db/d12/d19/f23 x:0 0 0 2024-02-28T23:07:31.616 INFO:tasks.workunit.client.1.smithi084.stdout:0/331: mknod d1/d6/db/d37/c68 0 2024-02-28T23:07:31.619 INFO:tasks.workunit.client.1.smithi084.stdout:4/287: creat d1/d18/f5f x:0 0 0 2024-02-28T23:07:31.620 INFO:tasks.workunit.client.0.smithi057.stdout:6/202: mkdir d0/d1b/d3e/d3d/d42 0 2024-02-28T23:07:31.621 INFO:tasks.workunit.client.0.smithi057.stdout:1/129: mkdir d2/d7/df/db/d12/d19/d24 0 2024-02-28T23:07:31.623 INFO:tasks.workunit.client.0.smithi057.stdout:9/224: mknod dd/d1b/c3f 0 2024-02-28T23:07:31.624 INFO:tasks.workunit.client.1.smithi084.stdout:4/288: chown d1/d18/d20/d14/c2d 63536291 1 2024-02-28T23:07:31.625 INFO:tasks.workunit.client.1.smithi084.stdout:0/332: fdatasync d1/d6/db/d35/d4a/f62 0 2024-02-28T23:07:31.627 INFO:tasks.workunit.client.1.smithi084.stdout:3/301: dwrite d3/d8/d1a/d21/d3d/d43/f6c [0,4194304] 0 2024-02-28T23:07:31.630 INFO:tasks.workunit.client.1.smithi084.stdout:4/289: fdatasync d1/d18/d20/d14/d30/f3d 0 2024-02-28T23:07:31.635 INFO:tasks.workunit.client.1.smithi084.stdout:2/294: dwrite d5/d7/d2e/f3d [0,4194304] 0 2024-02-28T23:07:31.636 INFO:tasks.workunit.client.1.smithi084.stdout:8/294: rename d0/d9/d1b/d38/d43/d51/d54/f5c to d0/d9/d63/d64/f6b 0 2024-02-28T23:07:31.638 INFO:tasks.workunit.client.0.smithi057.stdout:9/225: dread dd/de/d14/f3d [0,4194304] 0 2024-02-28T23:07:31.641 INFO:tasks.workunit.client.0.smithi057.stdout:0/198: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:31.641 INFO:tasks.workunit.client.1.smithi084.stdout:0/333: rmdir d1/d6/d59 39 2024-02-28T23:07:31.644 INFO:tasks.workunit.client.0.smithi057.stdout:0/199: write d17/d19/f3c [485180,102713] 0 2024-02-28T23:07:31.645 INFO:tasks.workunit.client.0.smithi057.stdout:9/226: mkdir dd/d1b/d40 0 2024-02-28T23:07:31.645 INFO:tasks.workunit.client.1.smithi084.stdout:0/334: chown d1/d6/db/d37/c68 17066 1 2024-02-28T23:07:31.645 INFO:tasks.workunit.client.1.smithi084.stdout:9/377: dwrite d4/f60 [4194304,4194304] 0 2024-02-28T23:07:31.646 INFO:tasks.workunit.client.0.smithi057.stdout:9/227: stat l1 0 2024-02-28T23:07:31.648 INFO:tasks.workunit.client.0.smithi057.stdout:1/130: truncate d2/d7/df/db/f18 2987892 0 2024-02-28T23:07:31.650 INFO:tasks.workunit.client.1.smithi084.stdout:3/302: chown d3/d8/d14/d31 85581 1 2024-02-28T23:07:31.653 INFO:tasks.workunit.client.0.smithi057.stdout:8/211: dwrite d4/f9 [0,4194304] 0 2024-02-28T23:07:31.653 INFO:tasks.workunit.client.0.smithi057.stdout:9/228: mkdir dd/d3a/d41 0 2024-02-28T23:07:31.653 INFO:tasks.workunit.client.0.smithi057.stdout:6/203: dread d0/f1c [0,4194304] 0 2024-02-28T23:07:31.654 INFO:tasks.workunit.client.0.smithi057.stdout:0/200: mknod d17/d20/d4a/c4f 0 2024-02-28T23:07:31.657 INFO:tasks.workunit.client.1.smithi084.stdout:0/335: fdatasync d1/d8/f1d 0 2024-02-28T23:07:31.657 INFO:tasks.workunit.client.0.smithi057.stdout:1/131: fdatasync d2/f8 0 2024-02-28T23:07:31.662 INFO:tasks.workunit.client.0.smithi057.stdout:8/212: creat d4/d1e/d37/d3f/d42/f44 x:0 0 0 2024-02-28T23:07:31.664 INFO:tasks.workunit.client.1.smithi084.stdout:9/378: dwrite d4/d6/f72 [0,4194304] 0 2024-02-28T23:07:31.665 INFO:tasks.workunit.client.0.smithi057.stdout:1/132: mkdir d2/d7/df/db/d12/d25 0 2024-02-28T23:07:31.668 INFO:tasks.workunit.client.0.smithi057.stdout:8/213: mknod d4/d1e/d37/d3f/c45 0 2024-02-28T23:07:31.669 INFO:tasks.workunit.client.1.smithi084.stdout:0/336: mkdir d1/d8/d69 0 2024-02-28T23:07:31.669 INFO:tasks.workunit.client.1.smithi084.stdout:6/322: rmdir d1 39 2024-02-28T23:07:31.675 INFO:tasks.workunit.client.0.smithi057.stdout:8/214: mknod d4/c46 0 2024-02-28T23:07:31.678 INFO:tasks.workunit.client.1.smithi084.stdout:5/358: link dc/de/df/d16/d34/d3c/l58 dc/l82 0 2024-02-28T23:07:31.679 INFO:tasks.workunit.client.0.smithi057.stdout:6/204: write d0/d4/f19 [1329944,86413] 0 2024-02-28T23:07:31.680 INFO:tasks.workunit.client.1.smithi084.stdout:5/359: readlink dc/d51/l7f 0 2024-02-28T23:07:31.683 INFO:tasks.workunit.client.1.smithi084.stdout:0/337: rmdir d1 39 2024-02-28T23:07:31.684 INFO:tasks.workunit.client.0.smithi057.stdout:6/205: readlink d0/d4/d11/l1e 0 2024-02-28T23:07:31.685 INFO:tasks.workunit.client.1.smithi084.stdout:8/295: getdents d0/d9/d30 0 2024-02-28T23:07:31.687 INFO:tasks.workunit.client.0.smithi057.stdout:1/133: dwrite d2/f15 [0,4194304] 0 2024-02-28T23:07:31.688 INFO:tasks.workunit.client.1.smithi084.stdout:3/303: link d3/d8/d1a/d29/f38 d3/d8/f72 0 2024-02-28T23:07:31.689 INFO:tasks.workunit.client.0.smithi057.stdout:0/201: dwrite d17/f18 [0,4194304] 0 2024-02-28T23:07:31.690 INFO:tasks.workunit.client.0.smithi057.stdout:6/206: mknod d0/d1b/d3e/d3d/d42/c43 0 2024-02-28T23:07:31.691 INFO:tasks.workunit.client.1.smithi084.stdout:8/296: symlink d0/d24/l6c 0 2024-02-28T23:07:31.692 INFO:tasks.workunit.client.0.smithi057.stdout:8/215: dread d4/f7 [8388608,4194304] 0 2024-02-28T23:07:31.698 INFO:tasks.workunit.client.0.smithi057.stdout:6/207: symlink d0/d1b/d3e/l44 0 2024-02-28T23:07:31.699 INFO:tasks.workunit.client.0.smithi057.stdout:0/202: mknod d17/d20/d23/c50 0 2024-02-28T23:07:31.700 INFO:tasks.workunit.client.0.smithi057.stdout:8/216: write d4/d13/d18/f31 [1004804,127116] 0 2024-02-28T23:07:31.702 INFO:tasks.workunit.client.0.smithi057.stdout:0/203: creat d17/d20/f51 x:0 0 0 2024-02-28T23:07:31.705 INFO:tasks.workunit.client.0.smithi057.stdout:6/208: link d0/d17/f35 d0/d17/f45 0 2024-02-28T23:07:31.706 INFO:tasks.workunit.client.0.smithi057.stdout:8/217: mknod d4/d1e/d37/d3f/c47 0 2024-02-28T23:07:31.707 INFO:tasks.workunit.client.0.smithi057.stdout:0/204: creat d17/d20/d26/f52 x:0 0 0 2024-02-28T23:07:31.710 INFO:tasks.workunit.client.0.smithi057.stdout:8/218: creat d4/d13/d18/f48 x:0 0 0 2024-02-28T23:07:31.711 INFO:tasks.workunit.client.0.smithi057.stdout:6/209: symlink d0/d4/d11/l46 0 2024-02-28T23:07:31.713 INFO:tasks.workunit.client.0.smithi057.stdout:0/205: creat d17/d19/f53 x:0 0 0 2024-02-28T23:07:31.715 INFO:tasks.workunit.client.0.smithi057.stdout:6/210: symlink d0/d1b/d3e/d40/l47 0 2024-02-28T23:07:31.716 INFO:tasks.workunit.client.0.smithi057.stdout:8/219: creat d4/d13/d2c/f49 x:0 0 0 2024-02-28T23:07:31.718 INFO:tasks.workunit.client.0.smithi057.stdout:0/206: mknod d17/d19/c54 0 2024-02-28T23:07:31.722 INFO:tasks.workunit.client.0.smithi057.stdout:0/207: mkdir d17/d20/d26/d48/d55 0 2024-02-28T23:07:31.725 INFO:tasks.workunit.client.0.smithi057.stdout:0/208: unlink d17/f1e 0 2024-02-28T23:07:31.728 INFO:tasks.workunit.client.0.smithi057.stdout:0/209: write d17/f18 [4369017,72422] 0 2024-02-28T23:07:31.728 INFO:tasks.workunit.client.0.smithi057.stdout:1/134: dread d2/f10 [4194304,4194304] 0 2024-02-28T23:07:31.731 INFO:tasks.workunit.client.0.smithi057.stdout:5/131: sync 2024-02-28T23:07:31.736 INFO:tasks.workunit.client.0.smithi057.stdout:5/132: symlink d4/d7/d8/l25 0 2024-02-28T23:07:31.739 INFO:tasks.workunit.client.0.smithi057.stdout:5/133: creat d4/d7/d8/f26 x:0 0 0 2024-02-28T23:07:31.740 INFO:tasks.workunit.client.0.smithi057.stdout:5/134: stat d4/f6 0 2024-02-28T23:07:31.747 INFO:tasks.workunit.client.0.smithi057.stdout:6/211: dread d0/d1b/d22/f32 [0,4194304] 0 2024-02-28T23:07:31.747 INFO:tasks.workunit.client.0.smithi057.stdout:1/135: dwrite d2/f8 [0,4194304] 0 2024-02-28T23:07:31.747 INFO:tasks.workunit.client.0.smithi057.stdout:5/135: getdents d4/d21 0 2024-02-28T23:07:31.747 INFO:tasks.workunit.client.0.smithi057.stdout:0/210: dwrite d17/d20/d26/d30/f39 [4194304,4194304] 0 2024-02-28T23:07:31.750 INFO:tasks.workunit.client.0.smithi057.stdout:5/136: dread - d4/d21/d24/f22 zero size 2024-02-28T23:07:31.750 INFO:tasks.workunit.client.0.smithi057.stdout:1/136: write d2/f15 [3599986,59522] 0 2024-02-28T23:07:31.751 INFO:tasks.workunit.client.0.smithi057.stdout:0/211: write d17/d20/d26/f41 [357800,47193] 0 2024-02-28T23:07:31.756 INFO:tasks.workunit.client.0.smithi057.stdout:6/212: symlink d0/d1b/d3e/l48 0 2024-02-28T23:07:31.756 INFO:tasks.workunit.client.0.smithi057.stdout:5/137: symlink d4/d21/d24/l27 0 2024-02-28T23:07:31.757 INFO:tasks.workunit.client.0.smithi057.stdout:0/212: symlink d17/d19/l56 0 2024-02-28T23:07:31.760 INFO:tasks.workunit.client.0.smithi057.stdout:0/213: symlink d17/d20/d4a/l57 0 2024-02-28T23:07:31.761 INFO:tasks.workunit.client.0.smithi057.stdout:5/138: truncate d4/d9/d16/f19 691250 0 2024-02-28T23:07:31.764 INFO:tasks.workunit.client.0.smithi057.stdout:5/139: fsync d4/d9/d16/f1a 0 2024-02-28T23:07:31.765 INFO:tasks.workunit.client.0.smithi057.stdout:0/214: write d17/d20/d26/d30/f39 [3280892,67692] 0 2024-02-28T23:07:31.768 INFO:tasks.workunit.client.0.smithi057.stdout:5/140: creat d4/d9/d14/f28 x:0 0 0 2024-02-28T23:07:31.772 INFO:tasks.workunit.client.0.smithi057.stdout:1/137: dwrite d2/d7/df/db/d12/d19/f23 [0,4194304] 0 2024-02-28T23:07:31.773 INFO:tasks.workunit.client.0.smithi057.stdout:5/141: write d4/d7/fe [3031121,45549] 0 2024-02-28T23:07:31.773 INFO:tasks.workunit.client.0.smithi057.stdout:1/138: readlink d2/l1e 0 2024-02-28T23:07:31.778 INFO:tasks.workunit.client.0.smithi057.stdout:6/213: dwrite d0/d4/f19 [0,4194304] 0 2024-02-28T23:07:31.783 INFO:tasks.workunit.client.0.smithi057.stdout:6/214: readlink d0/d1b/d3e/d40/l47 0 2024-02-28T23:07:31.790 INFO:tasks.workunit.client.0.smithi057.stdout:1/139: rename d2/d7/df/db/f14 to d2/d7/df/db/d12/f26 0 2024-02-28T23:07:31.793 INFO:tasks.workunit.client.0.smithi057.stdout:0/215: dwrite f10 [0,4194304] 0 2024-02-28T23:07:31.801 INFO:tasks.workunit.client.0.smithi057.stdout:6/215: getdents d0/d4/d25 0 2024-02-28T23:07:31.801 INFO:tasks.workunit.client.0.smithi057.stdout:5/142: dread d4/d7/d8/f13 [0,4194304] 0 2024-02-28T23:07:31.801 INFO:tasks.workunit.client.0.smithi057.stdout:6/216: creat d0/d17/d20/f49 x:0 0 0 2024-02-28T23:07:31.801 INFO:tasks.workunit.client.0.smithi057.stdout:0/216: link d17/d19/f53 d17/f58 0 2024-02-28T23:07:31.802 INFO:tasks.workunit.client.0.smithi057.stdout:5/143: write d4/d9/d16/f18 [277456,32833] 0 2024-02-28T23:07:31.804 INFO:tasks.workunit.client.0.smithi057.stdout:5/144: readlink d4/d7/d8/l25 0 2024-02-28T23:07:31.808 INFO:tasks.workunit.client.0.smithi057.stdout:6/217: rename d0/d4/fb to d0/f4a 0 2024-02-28T23:07:31.809 INFO:tasks.workunit.client.0.smithi057.stdout:5/145: mknod d4/d9/d16/c29 0 2024-02-28T23:07:31.811 INFO:tasks.workunit.client.0.smithi057.stdout:5/146: stat d4/d7/d8 0 2024-02-28T23:07:31.812 INFO:tasks.workunit.client.1.smithi084.stdout:3/304: rename d3/d8/df/d2f to d3/d50/d5a/d73 0 2024-02-28T23:07:31.812 INFO:tasks.workunit.client.0.smithi057.stdout:5/147: chown d4/d21/d24/d1d 0 1 2024-02-28T23:07:31.813 INFO:tasks.workunit.client.0.smithi057.stdout:6/218: truncate d0/fc 3234952 0 2024-02-28T23:07:31.815 INFO:tasks.workunit.client.0.smithi057.stdout:1/140: dwrite d2/d7/df/db/f20 [0,4194304] 0 2024-02-28T23:07:31.817 INFO:tasks.workunit.client.1.smithi084.stdout:0/338: creat d1/d6/db/d35/d57/f6a x:0 0 0 2024-02-28T23:07:31.817 INFO:tasks.workunit.client.1.smithi084.stdout:6/323: truncate d1/d9/d1d/d1e/d33/d38/f54 343571 0 2024-02-28T23:07:31.818 INFO:tasks.workunit.client.0.smithi057.stdout:1/141: fdatasync d2/f15 0 2024-02-28T23:07:31.818 INFO:tasks.workunit.client.0.smithi057.stdout:6/219: rmdir d0/d1b 39 2024-02-28T23:07:31.818 INFO:tasks.workunit.client.1.smithi084.stdout:9/379: dwrite d4/d6/d15/f2c [0,4194304] 0 2024-02-28T23:07:31.818 INFO:tasks.workunit.client.1.smithi084.stdout:3/305: dread - d3/d8/d1a/d29/d41/f64 zero size 2024-02-28T23:07:31.819 INFO:tasks.workunit.client.0.smithi057.stdout:5/148: creat d4/d9/f2a x:0 0 0 2024-02-28T23:07:31.820 INFO:tasks.workunit.client.0.smithi057.stdout:0/217: dread d17/d20/d26/f46 [0,4194304] 0 2024-02-28T23:07:31.820 INFO:tasks.workunit.client.0.smithi057.stdout:6/220: chown d0/d4/d25/f2f 470 1 2024-02-28T23:07:31.822 INFO:tasks.workunit.client.0.smithi057.stdout:1/142: chown d2/d7/df/db/d12/d19/f23 359 1 2024-02-28T23:07:31.822 INFO:tasks.workunit.client.1.smithi084.stdout:0/339: dread d1/d6/f53 [0,4194304] 0 2024-02-28T23:07:31.823 INFO:tasks.workunit.client.0.smithi057.stdout:6/221: fdatasync d0/d4/ff 0 2024-02-28T23:07:31.824 INFO:tasks.workunit.client.0.smithi057.stdout:0/218: write d17/d19/f2f [748402,54047] 0 2024-02-28T23:07:31.825 INFO:tasks.workunit.client.1.smithi084.stdout:6/324: mknod d1/d13/d50/d65/c6a 0 2024-02-28T23:07:31.828 INFO:tasks.workunit.client.0.smithi057.stdout:5/149: rmdir d4/d21/d24/d1d 0 2024-02-28T23:07:31.829 INFO:tasks.workunit.client.0.smithi057.stdout:0/219: creat d17/d20/d26/f59 x:0 0 0 2024-02-28T23:07:31.829 INFO:tasks.workunit.client.0.smithi057.stdout:5/150: stat f1 0 2024-02-28T23:07:31.830 INFO:tasks.workunit.client.1.smithi084.stdout:8/297: dwrite d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:31.834 INFO:tasks.workunit.client.0.smithi057.stdout:6/222: rmdir d0/d1b/d3e/d40 39 2024-02-28T23:07:31.834 INFO:tasks.workunit.client.1.smithi084.stdout:8/298: chown d0/d9/d63/l69 989 1 2024-02-28T23:07:31.834 INFO:tasks.workunit.client.1.smithi084.stdout:9/380: rename d4/d6/d15/l2f to d4/d6/d15/d75/d2d/l7c 0 2024-02-28T23:07:31.835 INFO:tasks.workunit.client.0.smithi057.stdout:1/143: link d2/l1e d2/l27 0 2024-02-28T23:07:31.837 INFO:tasks.workunit.client.1.smithi084.stdout:0/340: fsync d1/d6/db/d35/f5c 0 2024-02-28T23:07:31.837 INFO:tasks.workunit.client.0.smithi057.stdout:0/220: mknod d17/d20/d23/d47/c5a 0 2024-02-28T23:07:31.838 INFO:tasks.workunit.client.0.smithi057.stdout:5/151: dread d4/d7/f1e [0,4194304] 0 2024-02-28T23:07:31.841 INFO:tasks.workunit.client.0.smithi057.stdout:6/223: dread d0/d17/f2b [0,4194304] 0 2024-02-28T23:07:31.841 INFO:tasks.workunit.client.1.smithi084.stdout:9/381: creat d4/d6/d53/f7d x:0 0 0 2024-02-28T23:07:31.842 INFO:tasks.workunit.client.0.smithi057.stdout:0/221: stat d17/d20/d23/d37/f43 0 2024-02-28T23:07:31.844 INFO:tasks.workunit.client.1.smithi084.stdout:3/306: dwrite d3/fe [0,4194304] 0 2024-02-28T23:07:31.845 INFO:tasks.workunit.client.0.smithi057.stdout:6/224: read d0/d4/d11/f14 [2930353,85774] 0 2024-02-28T23:07:31.847 INFO:tasks.workunit.client.0.smithi057.stdout:6/225: readlink d0/d1b/d3e/l44 0 2024-02-28T23:07:31.847 INFO:tasks.workunit.client.0.smithi057.stdout:1/144: link d2/d7/df/db/d12/d19/f23 d2/d7/df/db/f28 0 2024-02-28T23:07:31.848 INFO:tasks.workunit.client.1.smithi084.stdout:0/341: mkdir d1/d8/d69/d6b 0 2024-02-28T23:07:31.849 INFO:tasks.workunit.client.0.smithi057.stdout:5/152: getdents d4/d7/d8 0 2024-02-28T23:07:31.849 INFO:tasks.workunit.client.1.smithi084.stdout:3/307: mknod d3/c74 0 2024-02-28T23:07:31.850 INFO:tasks.workunit.client.0.smithi057.stdout:5/153: chown d4/d21/d24 12908 1 2024-02-28T23:07:31.850 INFO:tasks.workunit.client.0.smithi057.stdout:6/226: fdatasync d0/fc 0 2024-02-28T23:07:31.851 INFO:tasks.workunit.client.0.smithi057.stdout:1/145: stat d2/f10 0 2024-02-28T23:07:31.853 INFO:tasks.workunit.client.0.smithi057.stdout:6/227: creat d0/d1b/d22/f4b x:0 0 0 2024-02-28T23:07:31.853 INFO:tasks.workunit.client.1.smithi084.stdout:0/342: mkdir d1/d8/d18/d4c/d6c 0 2024-02-28T23:07:31.854 INFO:tasks.workunit.client.1.smithi084.stdout:0/343: stat d1/l15 0 2024-02-28T23:07:31.858 INFO:tasks.workunit.client.1.smithi084.stdout:0/344: mknod d1/d6/db/d35/d44/c6d 0 2024-02-28T23:07:31.858 INFO:tasks.workunit.client.1.smithi084.stdout:8/299: dwrite d0/d9/f4c [0,4194304] 0 2024-02-28T23:07:31.861 INFO:tasks.workunit.client.1.smithi084.stdout:1/399: sync 2024-02-28T23:07:31.861 INFO:tasks.workunit.client.1.smithi084.stdout:0/345: rmdir d1/d8/d18/d28 39 2024-02-28T23:07:31.866 INFO:tasks.workunit.client.0.smithi057.stdout:1/146: getdents d2 0 2024-02-28T23:07:31.866 INFO:tasks.workunit.client.1.smithi084.stdout:9/382: dread d4/d6/d15/f71 [0,4194304] 0 2024-02-28T23:07:31.870 INFO:tasks.workunit.client.0.smithi057.stdout:1/147: symlink d2/d7/df/db/d12/d19/l29 0 2024-02-28T23:07:31.875 INFO:tasks.workunit.client.1.smithi084.stdout:0/346: mknod d1/c6e 0 2024-02-28T23:07:31.876 INFO:tasks.workunit.client.0.smithi057.stdout:5/154: dwrite d4/d9/d16/f18 [0,4194304] 0 2024-02-28T23:07:31.877 INFO:tasks.workunit.client.1.smithi084.stdout:3/308: dwrite d3/d8/d1a/d29/d41/d60/f6b [0,4194304] 0 2024-02-28T23:07:31.877 INFO:tasks.workunit.client.1.smithi084.stdout:9/383: unlink d4/d6/d15/d75/c5d 0 2024-02-28T23:07:31.878 INFO:tasks.workunit.client.0.smithi057.stdout:5/155: write f1 [770339,129264] 0 2024-02-28T23:07:31.878 INFO:tasks.workunit.client.0.smithi057.stdout:5/156: chown d4/fd 290 1 2024-02-28T23:07:31.879 INFO:tasks.workunit.client.0.smithi057.stdout:1/148: rename d2/d7/df/db/d12/d25 to d2/d7/df/db/d12/d19/d1b/d2a 0 2024-02-28T23:07:31.881 INFO:tasks.workunit.client.1.smithi084.stdout:3/309: dread - d3/d8/d1a/d21/d3d/d43/d61/f6e zero size 2024-02-28T23:07:31.884 INFO:tasks.workunit.client.0.smithi057.stdout:0/222: truncate d17/d19/f3c 478650 0 2024-02-28T23:07:31.886 INFO:tasks.workunit.client.1.smithi084.stdout:3/310: rmdir d3/d8/d14 39 2024-02-28T23:07:31.886 INFO:tasks.workunit.client.1.smithi084.stdout:1/400: dread d12/d1b/d2e/d32/f47 [0,4194304] 0 2024-02-28T23:07:31.887 INFO:tasks.workunit.client.0.smithi057.stdout:6/228: truncate d0/f1c 1994987 0 2024-02-28T23:07:31.889 INFO:tasks.workunit.client.0.smithi057.stdout:6/229: mknod d0/d1b/d22/c4c 0 2024-02-28T23:07:31.891 INFO:tasks.workunit.client.1.smithi084.stdout:3/311: mknod d3/d27/d4b/c75 0 2024-02-28T23:07:31.892 INFO:tasks.workunit.client.1.smithi084.stdout:1/401: dread - d12/d1b/d2e/d32/f51 zero size 2024-02-28T23:07:31.893 INFO:tasks.workunit.client.1.smithi084.stdout:3/312: readlink d3/d8/l9 0 2024-02-28T23:07:31.894 INFO:tasks.workunit.client.0.smithi057.stdout:1/149: dread d2/d7/df/db/f17 [0,4194304] 0 2024-02-28T23:07:31.894 INFO:tasks.workunit.client.0.smithi057.stdout:6/230: creat d0/d1b/d3e/d40/f4d x:0 0 0 2024-02-28T23:07:31.895 INFO:tasks.workunit.client.0.smithi057.stdout:0/223: dread f5 [8388608,4194304] 0 2024-02-28T23:07:31.896 INFO:tasks.workunit.client.1.smithi084.stdout:3/313: mkdir d3/d8/d1a/d21/d76 0 2024-02-28T23:07:31.897 INFO:tasks.workunit.client.0.smithi057.stdout:0/224: fdatasync d17/d20/d26/f4c 0 2024-02-28T23:07:31.898 INFO:tasks.workunit.client.0.smithi057.stdout:1/150: mknod d2/d7/df/db/d12/d19/d1b/c2b 0 2024-02-28T23:07:31.898 INFO:tasks.workunit.client.1.smithi084.stdout:1/402: creat d12/d1b/d63/d7d/f8c x:0 0 0 2024-02-28T23:07:31.899 INFO:tasks.workunit.client.0.smithi057.stdout:6/231: mknod d0/d4/d11/c4e 0 2024-02-28T23:07:31.900 INFO:tasks.workunit.client.0.smithi057.stdout:0/225: creat d17/d20/d23/d47/f5b x:0 0 0 2024-02-28T23:07:31.900 INFO:tasks.workunit.client.0.smithi057.stdout:6/232: readlink d0/d1b/d3e/d40/l47 0 2024-02-28T23:07:31.901 INFO:tasks.workunit.client.1.smithi084.stdout:1/403: write d12/d1b/d3b/f42 [122699,32311] 0 2024-02-28T23:07:31.902 INFO:tasks.workunit.client.1.smithi084.stdout:3/314: dread - d3/d8/d14/f2e zero size 2024-02-28T23:07:31.903 INFO:tasks.workunit.client.0.smithi057.stdout:4/212: sync 2024-02-28T23:07:31.903 INFO:tasks.workunit.client.0.smithi057.stdout:3/141: sync 2024-02-28T23:07:31.903 INFO:tasks.workunit.client.0.smithi057.stdout:6/233: mkdir d0/d1b/d3e/d3d/d42/d4f 0 2024-02-28T23:07:31.904 INFO:tasks.workunit.client.0.smithi057.stdout:3/142: fdatasync d3/fd 0 2024-02-28T23:07:31.907 INFO:tasks.workunit.client.1.smithi084.stdout:3/315: symlink d3/d50/l77 0 2024-02-28T23:07:31.907 INFO:tasks.workunit.client.1.smithi084.stdout:0/347: dwrite d1/d6/db/d35/f5c [0,4194304] 0 2024-02-28T23:07:31.909 INFO:tasks.workunit.client.0.smithi057.stdout:3/143: link d3/fd d3/d4/f30 0 2024-02-28T23:07:31.912 INFO:tasks.workunit.client.1.smithi084.stdout:8/300: dread d0/d9/d1b/f39 [0,4194304] 0 2024-02-28T23:07:31.913 INFO:tasks.workunit.client.1.smithi084.stdout:0/348: read d1/d8/f36 [284593,116970] 0 2024-02-28T23:07:31.913 INFO:tasks.workunit.client.1.smithi084.stdout:1/404: link d12/d1b/d63/l2f d12/d84/l8d 0 2024-02-28T23:07:31.915 INFO:tasks.workunit.client.1.smithi084.stdout:8/301: unlink d0/d11/l56 0 2024-02-28T23:07:31.915 INFO:tasks.workunit.client.0.smithi057.stdout:3/144: chown d3/d4/d7/cf 10 1 2024-02-28T23:07:31.919 INFO:tasks.workunit.client.0.smithi057.stdout:5/157: truncate d4/fd 2301116 0 2024-02-28T23:07:31.921 INFO:tasks.workunit.client.1.smithi084.stdout:3/316: rename d3/d8/d1a/d29/f53 to d3/d8/d1a/d45/f78 0 2024-02-28T23:07:31.922 INFO:tasks.workunit.client.0.smithi057.stdout:0/226: dwrite d17/d20/d26/d30/f39 [0,4194304] 0 2024-02-28T23:07:31.922 INFO:tasks.workunit.client.0.smithi057.stdout:5/158: chown d4/d9/d16/f19 0 1 2024-02-28T23:07:31.923 INFO:tasks.workunit.client.1.smithi084.stdout:1/405: getdents d12/d1b/d2e/d30/d50 0 2024-02-28T23:07:31.925 INFO:tasks.workunit.client.1.smithi084.stdout:1/406: dread - d12/d1b/d2e/d52/f6d zero size 2024-02-28T23:07:31.927 INFO:tasks.workunit.client.1.smithi084.stdout:3/317: unlink d3/d8/d14/d31/l69 0 2024-02-28T23:07:31.927 INFO:tasks.workunit.client.0.smithi057.stdout:4/213: truncate f15 315354 0 2024-02-28T23:07:31.928 INFO:tasks.workunit.client.0.smithi057.stdout:1/151: dwrite d2/d7/df/f1d [0,4194304] 0 2024-02-28T23:07:31.933 INFO:tasks.workunit.client.0.smithi057.stdout:6/234: dwrite d0/fd [0,4194304] 0 2024-02-28T23:07:31.936 INFO:tasks.workunit.client.1.smithi084.stdout:3/318: mknod d3/d8/d1a/d21/d3d/c79 0 2024-02-28T23:07:31.936 INFO:tasks.workunit.client.1.smithi084.stdout:8/302: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:31.938 INFO:tasks.workunit.client.0.smithi057.stdout:1/152: creat d2/d7/df/db/d12/d19/d1b/d2a/f2c x:0 0 0 2024-02-28T23:07:31.943 INFO:tasks.workunit.client.1.smithi084.stdout:8/303: dread d0/d9/d1b/d38/f3a [0,4194304] 0 2024-02-28T23:07:31.943 INFO:tasks.workunit.client.0.smithi057.stdout:6/235: rmdir d0/d17 39 2024-02-28T23:07:31.944 INFO:tasks.workunit.client.1.smithi084.stdout:3/319: mkdir d3/d8/d1a/d45/d5e/d7a 0 2024-02-28T23:07:31.944 INFO:tasks.workunit.client.0.smithi057.stdout:6/236: readlink d0/d1b/d3e/l48 0 2024-02-28T23:07:31.948 INFO:tasks.workunit.client.1.smithi084.stdout:1/407: truncate d12/d1b/d2e/d32/f5e 2991401 0 2024-02-28T23:07:31.949 INFO:tasks.workunit.client.0.smithi057.stdout:1/153: mkdir d2/d7/df/db/d12/d19/d24/d2d 0 2024-02-28T23:07:31.951 INFO:tasks.workunit.client.0.smithi057.stdout:1/154: stat d2/d7/df/db 0 2024-02-28T23:07:31.952 INFO:tasks.workunit.client.1.smithi084.stdout:3/320: mkdir d3/d8/d1a/d21/d7b 0 2024-02-28T23:07:31.952 INFO:tasks.workunit.client.1.smithi084.stdout:8/304: rmdir d0/d9/d63/d64 39 2024-02-28T23:07:31.956 INFO:tasks.workunit.client.1.smithi084.stdout:3/321: creat d3/d50/f7c x:0 0 0 2024-02-28T23:07:31.958 INFO:tasks.workunit.client.0.smithi057.stdout:1/155: stat d2/d7/df/db/d12/d19/d24/d2d 0 2024-02-28T23:07:31.960 INFO:tasks.workunit.client.0.smithi057.stdout:4/214: truncate d1a/d1e/f25 3152010 0 2024-02-28T23:07:31.961 INFO:tasks.workunit.client.1.smithi084.stdout:3/322: write d3/d8/d1a/d29/d41/f64 [53914,8998] 0 2024-02-28T23:07:31.961 INFO:tasks.workunit.client.1.smithi084.stdout:0/349: dwrite d1/d6/d1b/f22 [0,4194304] 0 2024-02-28T23:07:31.962 INFO:tasks.workunit.client.1.smithi084.stdout:8/305: rename d0/c19 to d0/d9/d1b/d38/d43/d51/d54/c6d 0 2024-02-28T23:07:31.964 INFO:tasks.workunit.client.1.smithi084.stdout:0/350: stat d1/d6/db/d35/d4a/d56 0 2024-02-28T23:07:31.967 INFO:tasks.workunit.client.0.smithi057.stdout:4/215: creat d1a/d1b/d24/f52 x:0 0 0 2024-02-28T23:07:31.968 INFO:tasks.workunit.client.0.smithi057.stdout:5/159: dwrite d4/d9/d16/f18 [4194304,4194304] 0 2024-02-28T23:07:31.969 INFO:tasks.workunit.client.0.smithi057.stdout:5/160: chown f1 9696568 1 2024-02-28T23:07:31.969 INFO:tasks.workunit.client.0.smithi057.stdout:4/216: chown d1a/d1e/f42 1742285599 1 2024-02-28T23:07:31.970 INFO:tasks.workunit.client.1.smithi084.stdout:3/323: mkdir d3/d27/d7d 0 2024-02-28T23:07:31.970 INFO:tasks.workunit.client.0.smithi057.stdout:6/237: rename d0/d1b/d22/f4b to d0/d4/f50 0 2024-02-28T23:07:31.973 INFO:tasks.workunit.client.1.smithi084.stdout:0/351: creat d1/d6/db/d35/d44/f6f x:0 0 0 2024-02-28T23:07:31.975 INFO:tasks.workunit.client.1.smithi084.stdout:3/324: write d3/d8/df/f19 [950463,12590] 0 2024-02-28T23:07:31.979 INFO:tasks.workunit.client.0.smithi057.stdout:5/161: write d4/d7/d8/f13 [1355551,96417] 0 2024-02-28T23:07:31.980 INFO:tasks.workunit.client.1.smithi084.stdout:0/352: dread d1/d6/f53 [0,4194304] 0 2024-02-28T23:07:31.982 INFO:tasks.workunit.client.0.smithi057.stdout:1/156: dread d2/d7/df/db/f20 [0,4194304] 0 2024-02-28T23:07:31.984 INFO:tasks.workunit.client.0.smithi057.stdout:4/217: creat d1a/d1e/d4b/d4f/f53 x:0 0 0 2024-02-28T23:07:31.984 INFO:tasks.workunit.client.1.smithi084.stdout:3/325: read d3/f6 [187750,68584] 0 2024-02-28T23:07:31.985 INFO:tasks.workunit.client.1.smithi084.stdout:0/353: mkdir d1/d6/d59/d70 0 2024-02-28T23:07:31.986 INFO:tasks.workunit.client.1.smithi084.stdout:3/326: fsync d3/d8/f10 0 2024-02-28T23:07:31.987 INFO:tasks.workunit.client.0.smithi057.stdout:5/162: creat d4/d9/d16/f2b x:0 0 0 2024-02-28T23:07:31.988 INFO:tasks.workunit.client.0.smithi057.stdout:1/157: creat d2/d7/df/db/d12/d19/d1b/d2a/f2e x:0 0 0 2024-02-28T23:07:31.990 INFO:tasks.workunit.client.1.smithi084.stdout:8/306: rmdir d0/d9/d1b 39 2024-02-28T23:07:31.992 INFO:tasks.workunit.client.1.smithi084.stdout:0/354: stat d1/d6/d59/f5a 0 2024-02-28T23:07:31.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/327: write d3/d8/d1a/d21/d3d/f4e [771142,43866] 0 2024-02-28T23:07:31.992 INFO:tasks.workunit.client.0.smithi057.stdout:6/238: dread d0/d17/f45 [0,4194304] 0 2024-02-28T23:07:31.995 INFO:tasks.workunit.client.1.smithi084.stdout:3/328: dread - d3/d8/d1a/f47 zero size 2024-02-28T23:07:31.995 INFO:tasks.workunit.client.1.smithi084.stdout:1/408: dread d12/d1b/f54 [0,4194304] 0 2024-02-28T23:07:31.996 INFO:tasks.workunit.client.0.smithi057.stdout:6/239: truncate d0/d1b/d22/f32 5795542 0 2024-02-28T23:07:31.996 INFO:tasks.workunit.client.1.smithi084.stdout:8/307: mkdir d0/d24/d6e 0 2024-02-28T23:07:31.996 INFO:tasks.workunit.client.1.smithi084.stdout:0/355: chown d1/d8/d18/f63 16716055 1 2024-02-28T23:07:31.997 INFO:tasks.workunit.client.0.smithi057.stdout:5/163: unlink d4/d9/d16/f10 0 2024-02-28T23:07:31.998 INFO:tasks.workunit.client.1.smithi084.stdout:0/356: write d1/d6/d1b/f22 [4445146,117177] 0 2024-02-28T23:07:31.999 INFO:tasks.workunit.client.0.smithi057.stdout:5/164: fsync d4/d7/d8/f26 0 2024-02-28T23:07:32.003 INFO:tasks.workunit.client.1.smithi084.stdout:1/409: creat d12/d84/f8e x:0 0 0 2024-02-28T23:07:32.004 INFO:tasks.workunit.client.0.smithi057.stdout:6/240: rename d0/d4/d11/l46 to d0/d1b/d22/l51 0 2024-02-28T23:07:32.005 INFO:tasks.workunit.client.0.smithi057.stdout:4/218: dwrite d1a/d1b/d24/d2e/f4a [0,4194304] 0 2024-02-28T23:07:32.005 INFO:tasks.workunit.client.1.smithi084.stdout:0/357: rename d1/d6/db/d35/f5e to d1/d6/db/d37/f71 0 2024-02-28T23:07:32.005 INFO:tasks.workunit.client.1.smithi084.stdout:7/304: sync 2024-02-28T23:07:32.006 INFO:tasks.workunit.client.1.smithi084.stdout:0/358: chown d1 557260986 1 2024-02-28T23:07:32.007 INFO:tasks.workunit.client.0.smithi057.stdout:6/241: write d0/d17/f1d [684501,108762] 0 2024-02-28T23:07:32.009 INFO:tasks.workunit.client.0.smithi057.stdout:1/158: dread d2/d7/df/db/f17 [0,4194304] 0 2024-02-28T23:07:32.011 INFO:tasks.workunit.client.1.smithi084.stdout:1/410: mknod d12/d1b/d2e/d52/d61/c8f 0 2024-02-28T23:07:32.011 INFO:tasks.workunit.client.0.smithi057.stdout:4/219: chown c10 11463044 1 2024-02-28T23:07:32.012 INFO:tasks.workunit.client.0.smithi057.stdout:4/220: stat c9 0 2024-02-28T23:07:32.014 INFO:tasks.workunit.client.1.smithi084.stdout:7/305: mkdir db/d11/d27/d5f 0 2024-02-28T23:07:32.014 INFO:tasks.workunit.client.0.smithi057.stdout:4/221: write f0 [816744,58562] 0 2024-02-28T23:07:32.019 INFO:tasks.workunit.client.0.smithi057.stdout:1/159: symlink d2/d7/df/l2f 0 2024-02-28T23:07:32.019 INFO:tasks.workunit.client.1.smithi084.stdout:3/329: dwrite d3/d8/d14/f6d [0,4194304] 0 2024-02-28T23:07:32.021 INFO:tasks.workunit.client.0.smithi057.stdout:1/160: read d2/f10 [4657805,51341] 0 2024-02-28T23:07:32.023 INFO:tasks.workunit.client.0.smithi057.stdout:5/165: getdents d4 0 2024-02-28T23:07:32.024 INFO:tasks.workunit.client.1.smithi084.stdout:1/411: creat d12/d1b/d2e/d30/d77/f90 x:0 0 0 2024-02-28T23:07:32.027 INFO:tasks.workunit.client.1.smithi084.stdout:7/306: creat db/d10/f60 x:0 0 0 2024-02-28T23:07:32.027 INFO:tasks.workunit.client.1.smithi084.stdout:1/412: write d12/f45 [742226,104925] 0 2024-02-28T23:07:32.028 INFO:tasks.workunit.client.0.smithi057.stdout:5/166: write d4/f6 [4713062,125124] 0 2024-02-28T23:07:32.029 INFO:tasks.workunit.client.0.smithi057.stdout:1/161: truncate d2/d7/df/db/d12/d19/d1b/d2a/f2e 864305 0 2024-02-28T23:07:32.030 INFO:tasks.workunit.client.1.smithi084.stdout:7/307: write db/d10/f2f [136606,30438] 0 2024-02-28T23:07:32.032 INFO:tasks.workunit.client.0.smithi057.stdout:5/167: write d4/d9/f2a [292914,112363] 0 2024-02-28T23:07:32.037 INFO:tasks.workunit.client.1.smithi084.stdout:0/359: dwrite d1/d8/d18/f63 [4194304,4194304] 0 2024-02-28T23:07:32.043 INFO:tasks.workunit.client.0.smithi057.stdout:9/229: sync 2024-02-28T23:07:32.043 INFO:tasks.workunit.client.0.smithi057.stdout:2/228: sync 2024-02-28T23:07:32.043 INFO:tasks.workunit.client.0.smithi057.stdout:8/220: sync 2024-02-28T23:07:32.043 INFO:tasks.workunit.client.0.smithi057.stdout:7/251: sync 2024-02-28T23:07:32.043 INFO:tasks.workunit.client.0.smithi057.stdout:5/168: mkdir d4/d9/d16/d2c 0 2024-02-28T23:07:32.044 INFO:tasks.workunit.client.1.smithi084.stdout:7/308: getdents db/d10/d17/d3b 0 2024-02-28T23:07:32.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/162: rename d2/l27 to d2/d7/df/db/d12/l30 0 2024-02-28T23:07:32.047 INFO:tasks.workunit.client.1.smithi084.stdout:3/330: write d3/d8/d1a/d29/f38 [1296702,107924] 0 2024-02-28T23:07:32.047 INFO:tasks.workunit.client.1.smithi084.stdout:7/309: chown db/d5e 78427 1 2024-02-28T23:07:32.048 INFO:tasks.workunit.client.0.smithi057.stdout:2/229: creat d8/d11/d48/f52 x:0 0 0 2024-02-28T23:07:32.051 INFO:tasks.workunit.client.0.smithi057.stdout:8/221: mkdir d4/d13/d1a/d4a 0 2024-02-28T23:07:32.052 INFO:tasks.workunit.client.1.smithi084.stdout:3/331: write d3/f6 [1175700,24282] 0 2024-02-28T23:07:32.054 INFO:tasks.workunit.client.0.smithi057.stdout:9/230: rename l1 to dd/d1b/d40/l42 0 2024-02-28T23:07:32.055 INFO:tasks.workunit.client.0.smithi057.stdout:7/252: creat d0/d5/f4b x:0 0 0 2024-02-28T23:07:32.058 INFO:tasks.workunit.client.1.smithi084.stdout:3/332: read - d3/d8/d1a/d21/d3d/d43/d61/f65 zero size 2024-02-28T23:07:32.058 INFO:tasks.workunit.client.0.smithi057.stdout:6/242: dread d0/d4/d11/f2d [0,4194304] 0 2024-02-28T23:07:32.059 INFO:tasks.workunit.client.0.smithi057.stdout:8/222: creat d4/d13/d1a/f4b x:0 0 0 2024-02-28T23:07:32.061 INFO:tasks.workunit.client.0.smithi057.stdout:8/223: write d4/d13/d18/f31 [1830141,49789] 0 2024-02-28T23:07:32.063 INFO:tasks.workunit.client.0.smithi057.stdout:5/169: rename d4/f15 to d4/d21/f2d 0 2024-02-28T23:07:32.064 INFO:tasks.workunit.client.1.smithi084.stdout:7/310: rmdir db/d11/d27/d5f 0 2024-02-28T23:07:32.064 INFO:tasks.workunit.client.1.smithi084.stdout:0/360: dwrite d1/d6/db/d35/d44/f4b [0,4194304] 0 2024-02-28T23:07:32.064 INFO:tasks.workunit.client.0.smithi057.stdout:8/224: chown d4/d13/d2c/f49 4 1 2024-02-28T23:07:32.065 INFO:tasks.workunit.client.1.smithi084.stdout:7/311: stat db/d11/f29 0 2024-02-28T23:07:32.066 INFO:tasks.workunit.client.0.smithi057.stdout:4/222: truncate fe 1493617 0 2024-02-28T23:07:32.067 INFO:tasks.workunit.client.1.smithi084.stdout:1/413: dread d12/d1b/d3b/f42 [0,4194304] 0 2024-02-28T23:07:32.068 INFO:tasks.workunit.client.0.smithi057.stdout:8/225: truncate d4/f11 1892013 0 2024-02-28T23:07:32.070 INFO:tasks.workunit.client.1.smithi084.stdout:0/361: mkdir d1/d6/db/d37/d72 0 2024-02-28T23:07:32.072 INFO:tasks.workunit.client.1.smithi084.stdout:0/362: creat d1/d6/d59/d70/f73 x:0 0 0 2024-02-28T23:07:32.073 INFO:tasks.workunit.client.1.smithi084.stdout:0/363: readlink d1/la 0 2024-02-28T23:07:32.076 INFO:tasks.workunit.client.0.smithi057.stdout:2/230: rename d8/d11/f45 to d8/d11/d21/d28/d41/f53 0 2024-02-28T23:07:32.077 INFO:tasks.workunit.client.1.smithi084.stdout:0/364: mkdir d1/d8/d18/d4c/d6c/d74 0 2024-02-28T23:07:32.078 INFO:tasks.workunit.client.1.smithi084.stdout:3/333: dwrite d3/d8/d1a/d29/f38 [0,4194304] 0 2024-02-28T23:07:32.079 INFO:tasks.workunit.client.0.smithi057.stdout:9/231: creat dd/de/f43 x:0 0 0 2024-02-28T23:07:32.083 INFO:tasks.workunit.client.0.smithi057.stdout:7/253: creat d0/d3f/f4c x:0 0 0 2024-02-28T23:07:32.084 INFO:tasks.workunit.client.0.smithi057.stdout:8/226: symlink d4/d1e/d37/d3f/d42/l4c 0 2024-02-28T23:07:32.084 INFO:tasks.workunit.client.1.smithi084.stdout:0/365: creat d1/d8/f75 x:0 0 0 2024-02-28T23:07:32.084 INFO:tasks.workunit.client.1.smithi084.stdout:3/334: mkdir d3/d8/d1a/d29/d7e 0 2024-02-28T23:07:32.085 INFO:tasks.workunit.client.1.smithi084.stdout:0/366: fsync d1/d8/f1f 0 2024-02-28T23:07:32.089 INFO:tasks.workunit.client.1.smithi084.stdout:0/367: unlink d1/la 0 2024-02-28T23:07:32.091 INFO:tasks.workunit.client.1.smithi084.stdout:0/368: readlink d1/d8/l51 0 2024-02-28T23:07:32.091 INFO:tasks.workunit.client.1.smithi084.stdout:3/335: symlink d3/d8/d1a/d21/d7b/l7f 0 2024-02-28T23:07:32.092 INFO:tasks.workunit.client.0.smithi057.stdout:6/243: dwrite d0/d4/d25/f2f [0,4194304] 0 2024-02-28T23:07:32.092 INFO:tasks.workunit.client.1.smithi084.stdout:3/336: stat d3/d8/d14/f49 0 2024-02-28T23:07:32.093 INFO:tasks.workunit.client.0.smithi057.stdout:5/170: dwrite d4/d7/d8/f26 [0,4194304] 0 2024-02-28T23:07:32.096 INFO:tasks.workunit.client.1.smithi084.stdout:3/337: mknod d3/d8/d14/c80 0 2024-02-28T23:07:32.097 INFO:tasks.workunit.client.0.smithi057.stdout:9/232: mkdir dd/d1b/d40/d44 0 2024-02-28T23:07:32.099 INFO:tasks.workunit.client.0.smithi057.stdout:7/254: mknod d0/d5/c4d 0 2024-02-28T23:07:32.104 INFO:tasks.workunit.client.0.smithi057.stdout:4/223: dread d1a/d1b/d24/f38 [0,4194304] 0 2024-02-28T23:07:32.105 INFO:tasks.workunit.client.0.smithi057.stdout:6/244: rmdir d0/d4 39 2024-02-28T23:07:32.106 INFO:tasks.workunit.client.1.smithi084.stdout:3/338: rename d3/d27/l68 to d3/d8/d1a/d45/d5e/d7a/l81 0 2024-02-28T23:07:32.107 INFO:tasks.workunit.client.0.smithi057.stdout:9/233: stat dd/de/l1a 0 2024-02-28T23:07:32.111 INFO:tasks.workunit.client.1.smithi084.stdout:3/339: symlink d3/d27/l82 0 2024-02-28T23:07:32.112 INFO:tasks.workunit.client.0.smithi057.stdout:1/163: creat d2/d7/df/db/d12/d19/d24/d2d/f31 x:0 0 0 2024-02-28T23:07:32.112 INFO:tasks.workunit.client.1.smithi084.stdout:0/369: dwrite d1/d6/d59/d70/f73 [0,4194304] 0 2024-02-28T23:07:32.113 INFO:tasks.workunit.client.0.smithi057.stdout:0/227: sync 2024-02-28T23:07:32.114 INFO:tasks.workunit.client.0.smithi057.stdout:3/145: sync 2024-02-28T23:07:32.114 INFO:tasks.workunit.client.0.smithi057.stdout:5/171: mkdir d4/d21/d24/d2e 0 2024-02-28T23:07:32.114 INFO:tasks.workunit.client.0.smithi057.stdout:9/234: read dd/de/d14/f2b [905121,19110] 0 2024-02-28T23:07:32.115 INFO:tasks.workunit.client.1.smithi084.stdout:3/340: write d3/d50/f7c [660908,5493] 0 2024-02-28T23:07:32.117 INFO:tasks.workunit.client.1.smithi084.stdout:0/370: write d1/d8/f1f [3411180,72764] 0 2024-02-28T23:07:32.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/172: read - d4/d9/d14/f28 zero size 2024-02-28T23:07:32.119 INFO:tasks.workunit.client.1.smithi084.stdout:3/341: fdatasync d3/d8/d1a/d21/d3d/d43/f48 0 2024-02-28T23:07:32.119 INFO:tasks.workunit.client.0.smithi057.stdout:3/146: read d3/d4/f30 [2835022,12037] 0 2024-02-28T23:07:32.120 INFO:tasks.workunit.client.0.smithi057.stdout:8/227: dwrite d4/d13/d2c/f49 [0,4194304] 0 2024-02-28T23:07:32.120 INFO:tasks.workunit.client.0.smithi057.stdout:7/255: rename d0/d8 to d0/d5/d36/d4e 0 2024-02-28T23:07:32.122 INFO:tasks.workunit.client.1.smithi084.stdout:0/371: mkdir d1/d6/db/d76 0 2024-02-28T23:07:32.123 INFO:tasks.workunit.client.1.smithi084.stdout:7/312: dread db/f1e [0,4194304] 0 2024-02-28T23:07:32.123 INFO:tasks.workunit.client.0.smithi057.stdout:5/173: stat d4/d7/d8/f13 0 2024-02-28T23:07:32.125 INFO:tasks.workunit.client.0.smithi057.stdout:3/147: write d3/d4/f17 [1473869,66484] 0 2024-02-28T23:07:32.125 INFO:tasks.workunit.client.0.smithi057.stdout:5/174: chown d4/d9/d16 90 1 2024-02-28T23:07:32.126 INFO:tasks.workunit.client.1.smithi084.stdout:7/313: dread - db/d11/d27/d36/f5a zero size 2024-02-28T23:07:32.126 INFO:tasks.workunit.client.1.smithi084.stdout:3/342: creat d3/d8/d1a/d29/d41/d5b/f83 x:0 0 0 2024-02-28T23:07:32.131 INFO:tasks.workunit.client.1.smithi084.stdout:7/314: mknod db/d11/d27/d36/c61 0 2024-02-28T23:07:32.132 INFO:tasks.workunit.client.0.smithi057.stdout:7/256: write d0/d5/d36/d4e/d15/f42 [267803,1086] 0 2024-02-28T23:07:32.133 INFO:tasks.workunit.client.1.smithi084.stdout:3/343: mknod d3/d8/d14/d31/d34/c84 0 2024-02-28T23:07:32.136 INFO:tasks.workunit.client.0.smithi057.stdout:9/235: write dd/f12 [426554,112267] 0 2024-02-28T23:07:32.137 INFO:tasks.workunit.client.1.smithi084.stdout:7/315: unlink db/d10/c54 0 2024-02-28T23:07:32.139 INFO:tasks.workunit.client.0.smithi057.stdout:9/236: readlink dd/de/l1a 0 2024-02-28T23:07:32.140 INFO:tasks.workunit.client.0.smithi057.stdout:0/228: creat d17/d20/d26/d48/f5c x:0 0 0 2024-02-28T23:07:32.144 INFO:tasks.workunit.client.1.smithi084.stdout:0/372: dwrite d1/d8/f3e [0,4194304] 0 2024-02-28T23:07:32.147 INFO:tasks.workunit.client.1.smithi084.stdout:7/316: link db/d10/c57 db/d5e/c62 0 2024-02-28T23:07:32.147 INFO:tasks.workunit.client.1.smithi084.stdout:0/373: dread - d1/d6/db/d35/d4a/f62 zero size 2024-02-28T23:07:32.151 INFO:tasks.workunit.client.1.smithi084.stdout:0/374: chown d1/d8/f1f 176 1 2024-02-28T23:07:32.152 INFO:tasks.workunit.client.0.smithi057.stdout:6/245: creat d0/d4/d25/f52 x:0 0 0 2024-02-28T23:07:32.155 INFO:tasks.workunit.client.0.smithi057.stdout:7/257: unlink d0/d5/d36/d4e/c1f 0 2024-02-28T23:07:32.155 INFO:tasks.workunit.client.0.smithi057.stdout:8/228: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:32.159 INFO:tasks.workunit.client.1.smithi084.stdout:0/375: creat d1/d8/d69/d6b/f77 x:0 0 0 2024-02-28T23:07:32.160 INFO:tasks.workunit.client.0.smithi057.stdout:1/164: dwrite d2/d7/df/db/d12/d19/d24/d2d/f31 [0,4194304] 0 2024-02-28T23:07:32.161 INFO:tasks.workunit.client.1.smithi084.stdout:3/344: dread d3/d8/d1a/d21/d3d/d43/f6c [0,4194304] 0 2024-02-28T23:07:32.161 INFO:tasks.workunit.client.0.smithi057.stdout:8/229: chown d4/d13/d18/c19 18293 1 2024-02-28T23:07:32.162 INFO:tasks.workunit.client.0.smithi057.stdout:1/165: dread - d2/f21 zero size 2024-02-28T23:07:32.163 INFO:tasks.workunit.client.1.smithi084.stdout:0/376: mkdir d1/d8/d78 0 2024-02-28T23:07:32.164 INFO:tasks.workunit.client.0.smithi057.stdout:0/229: rename d17/d20/d26/f59 to d17/d20/f5d 0 2024-02-28T23:07:32.173 INFO:tasks.workunit.client.0.smithi057.stdout:2/231: getdents d8/d11/d39 0 2024-02-28T23:07:32.173 INFO:tasks.workunit.client.1.smithi084.stdout:7/317: dwrite db/d10/d17/d3b/f48 [0,4194304] 0 2024-02-28T23:07:32.175 INFO:tasks.workunit.client.0.smithi057.stdout:4/224: dread f0 [0,4194304] 0 2024-02-28T23:07:32.177 INFO:tasks.workunit.client.0.smithi057.stdout:9/237: rename dd/d3a/c3e to dd/d1b/d40/c45 0 2024-02-28T23:07:32.178 INFO:tasks.workunit.client.1.smithi084.stdout:0/377: rmdir d1/d8/d18/d4c 39 2024-02-28T23:07:32.178 INFO:tasks.workunit.client.0.smithi057.stdout:8/230: creat d4/d1e/d37/d3f/f4d x:0 0 0 2024-02-28T23:07:32.178 INFO:tasks.workunit.client.0.smithi057.stdout:7/258: symlink d0/d5/d36/d4e/d15/d29/d3d/l4f 0 2024-02-28T23:07:32.180 INFO:tasks.workunit.client.1.smithi084.stdout:4/290: sync 2024-02-28T23:07:32.180 INFO:tasks.workunit.client.1.smithi084.stdout:2/295: sync 2024-02-28T23:07:32.181 INFO:tasks.workunit.client.0.smithi057.stdout:3/148: link d3/d8/d1b/l28 d3/d4/d1d/l31 0 2024-02-28T23:07:32.183 INFO:tasks.workunit.client.1.smithi084.stdout:2/296: fsync d5/f15 0 2024-02-28T23:07:32.184 INFO:tasks.workunit.client.0.smithi057.stdout:8/231: stat d4/d1e/d37/d3f/f4d 0 2024-02-28T23:07:32.190 INFO:tasks.workunit.client.0.smithi057.stdout:5/175: write d4/d7/fe [5016151,95953] 0 2024-02-28T23:07:32.193 INFO:tasks.workunit.client.0.smithi057.stdout:6/246: rename d0/d17/d38 to d0/d1b/d3e/d3d/d42/d4f/d53 0 2024-02-28T23:07:32.197 INFO:tasks.workunit.client.0.smithi057.stdout:0/230: link d17/d19/l4e d17/d19/l5e 0 2024-02-28T23:07:32.197 INFO:tasks.workunit.client.0.smithi057.stdout:6/247: chown d0/d4/d11 1685838716 1 2024-02-28T23:07:32.198 INFO:tasks.workunit.client.0.smithi057.stdout:0/231: stat fe 0 2024-02-28T23:07:32.199 INFO:tasks.workunit.client.0.smithi057.stdout:9/238: mkdir dd/d46 0 2024-02-28T23:07:32.201 INFO:tasks.workunit.client.0.smithi057.stdout:5/176: read d4/d21/f2d [3249164,21884] 0 2024-02-28T23:07:32.203 INFO:tasks.workunit.client.1.smithi084.stdout:3/345: dread d3/d8/d1a/d21/f32 [0,4194304] 0 2024-02-28T23:07:32.205 INFO:tasks.workunit.client.1.smithi084.stdout:3/346: chown d3/d50/d5a/d73 16 1 2024-02-28T23:07:32.211 INFO:tasks.workunit.client.0.smithi057.stdout:7/259: creat d0/f50 x:0 0 0 2024-02-28T23:07:32.211 INFO:tasks.workunit.client.0.smithi057.stdout:1/166: link d2/d7/df/db/d12/c1c d2/c32 0 2024-02-28T23:07:32.214 INFO:tasks.workunit.client.0.smithi057.stdout:0/232: symlink d17/d20/d26/d48/l5f 0 2024-02-28T23:07:32.214 INFO:tasks.workunit.client.0.smithi057.stdout:8/232: dwrite d4/d13/f2d [0,4194304] 0 2024-02-28T23:07:32.215 INFO:tasks.workunit.client.0.smithi057.stdout:6/248: rename d0/d1b/d22 to d0/d1b/d54 0 2024-02-28T23:07:32.218 INFO:tasks.workunit.client.1.smithi084.stdout:4/291: creat d1/d18/d20/d24/d2f/d3e/f60 x:0 0 0 2024-02-28T23:07:32.223 INFO:tasks.workunit.client.1.smithi084.stdout:2/297: creat d5/d7/d1d/f50 x:0 0 0 2024-02-28T23:07:32.223 INFO:tasks.workunit.client.0.smithi057.stdout:6/249: read d0/d4/ff [512499,346] 0 2024-02-28T23:07:32.226 INFO:tasks.workunit.client.0.smithi057.stdout:6/250: chown d0 137 1 2024-02-28T23:07:32.226 INFO:tasks.workunit.client.0.smithi057.stdout:7/260: fdatasync d0/f35 0 2024-02-28T23:07:32.228 INFO:tasks.workunit.client.1.smithi084.stdout:3/347: creat d3/d8/d1a/d21/f85 x:0 0 0 2024-02-28T23:07:32.228 INFO:tasks.workunit.client.0.smithi057.stdout:3/149: dwrite d3/f2c [0,4194304] 0 2024-02-28T23:07:32.234 INFO:tasks.workunit.client.0.smithi057.stdout:0/233: mkdir d17/d20/d23/d47/d60 0 2024-02-28T23:07:32.235 INFO:tasks.workunit.client.1.smithi084.stdout:4/292: rename d1/d18/d20/d14/d30/l33 to d1/d18/d20/d24/d2f/d3e/d52/l61 0 2024-02-28T23:07:32.238 INFO:tasks.workunit.client.0.smithi057.stdout:7/261: write d0/d5/f4b [410135,66221] 0 2024-02-28T23:07:32.239 INFO:tasks.workunit.client.1.smithi084.stdout:2/298: creat d5/d7/d12/f51 x:0 0 0 2024-02-28T23:07:32.243 INFO:tasks.workunit.client.1.smithi084.stdout:0/378: link d1/d8/d18/d28/f54 d1/d6/db/d35/d44/f79 0 2024-02-28T23:07:32.244 INFO:tasks.workunit.client.0.smithi057.stdout:2/232: dread d8/f19 [0,4194304] 0 2024-02-28T23:07:32.245 INFO:tasks.workunit.client.0.smithi057.stdout:0/234: write f15 [811362,108795] 0 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:7/318: getdents db/d11/d27 0 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:8/308: sync 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:1/414: sync 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:9/384: sync 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:6/325: sync 2024-02-28T23:07:32.246 INFO:tasks.workunit.client.1.smithi084.stdout:5/360: sync 2024-02-28T23:07:32.247 INFO:tasks.workunit.client.1.smithi084.stdout:4/293: fdatasync d1/f9 0 2024-02-28T23:07:32.250 INFO:tasks.workunit.client.1.smithi084.stdout:5/361: stat dc/d51/l7f 0 2024-02-28T23:07:32.253 INFO:tasks.workunit.client.0.smithi057.stdout:5/177: dwrite d4/d9/f2a [0,4194304] 0 2024-02-28T23:07:32.254 INFO:tasks.workunit.client.1.smithi084.stdout:4/294: dread - d1/d18/d20/d14/f4f zero size 2024-02-28T23:07:32.255 INFO:tasks.workunit.client.0.smithi057.stdout:1/167: rename d2/d7/df/d22 to d2/d33 0 2024-02-28T23:07:32.255 INFO:tasks.workunit.client.0.smithi057.stdout:9/239: dwrite dd/ff [0,4194304] 0 2024-02-28T23:07:32.255 INFO:tasks.workunit.client.0.smithi057.stdout:4/225: dwrite fe [0,4194304] 0 2024-02-28T23:07:32.255 INFO:tasks.workunit.client.1.smithi084.stdout:0/379: creat d1/d8/d69/d6b/f7a x:0 0 0 2024-02-28T23:07:32.256 INFO:tasks.workunit.client.1.smithi084.stdout:4/295: chown d1/d18/d20/d14/f39 132073 1 2024-02-28T23:07:32.257 INFO:tasks.workunit.client.0.smithi057.stdout:7/262: dread - d0/d5/d36/d4e/d15/d29/f40 zero size 2024-02-28T23:07:32.260 INFO:tasks.workunit.client.1.smithi084.stdout:8/309: write d0/d9/d1b/d38/f3e [6651877,15595] 0 2024-02-28T23:07:32.262 INFO:tasks.workunit.client.1.smithi084.stdout:9/385: creat d4/d6/d15/d12/d41/f7e x:0 0 0 2024-02-28T23:07:32.262 INFO:tasks.workunit.client.0.smithi057.stdout:8/233: write d4/d13/f2b [559099,10932] 0 2024-02-28T23:07:32.263 INFO:tasks.workunit.client.1.smithi084.stdout:8/310: fsync d0/d24/d4d/f6a 0 2024-02-28T23:07:32.264 INFO:tasks.workunit.client.1.smithi084.stdout:1/415: dread d12/d1b/d3b/f5d [0,4194304] 0 2024-02-28T23:07:32.266 INFO:tasks.workunit.client.1.smithi084.stdout:9/386: chown d4/d6/f7 6 1 2024-02-28T23:07:32.266 INFO:tasks.workunit.client.0.smithi057.stdout:0/235: mknod d17/d20/d4a/c61 0 2024-02-28T23:07:32.266 INFO:tasks.workunit.client.0.smithi057.stdout:8/234: fsync d4/d13/d1a/f34 0 2024-02-28T23:07:32.269 INFO:tasks.workunit.client.1.smithi084.stdout:7/319: symlink db/l63 0 2024-02-28T23:07:32.269 INFO:tasks.workunit.client.0.smithi057.stdout:8/235: write d4/f11 [603231,94412] 0 2024-02-28T23:07:32.269 INFO:tasks.workunit.client.0.smithi057.stdout:6/251: rename d0/d17/f1d to d0/d1b/d3e/d3d/f55 0 2024-02-28T23:07:32.270 INFO:tasks.workunit.client.0.smithi057.stdout:5/178: creat d4/d21/f2f x:0 0 0 2024-02-28T23:07:32.272 INFO:tasks.workunit.client.0.smithi057.stdout:3/150: dread d3/d8/f16 [0,4194304] 0 2024-02-28T23:07:32.272 INFO:tasks.workunit.client.0.smithi057.stdout:9/240: mknod dd/d1b/c47 0 2024-02-28T23:07:32.272 INFO:tasks.workunit.client.0.smithi057.stdout:8/236: stat d4/d13/d18/f48 0 2024-02-28T23:07:32.273 INFO:tasks.workunit.client.1.smithi084.stdout:7/320: readlink db/d10/d17/l37 0 2024-02-28T23:07:32.274 INFO:tasks.workunit.client.1.smithi084.stdout:7/321: dread - db/d10/f56 zero size 2024-02-28T23:07:32.277 INFO:tasks.workunit.client.1.smithi084.stdout:4/296: mknod d1/d18/d20/d14/d30/c62 0 2024-02-28T23:07:32.278 INFO:tasks.workunit.client.0.smithi057.stdout:3/151: write d3/d20/f23 [3605815,84010] 0 2024-02-28T23:07:32.278 INFO:tasks.workunit.client.0.smithi057.stdout:7/263: rename d0/d5/d25/d44 to d0/d5/d36/d4e/d15/d29/d3d/d51 0 2024-02-28T23:07:32.281 INFO:tasks.workunit.client.0.smithi057.stdout:9/241: read dd/de/d24/f28 [47714,74877] 0 2024-02-28T23:07:32.281 INFO:tasks.workunit.client.1.smithi084.stdout:4/297: fdatasync d1/d18/d20/d14/d30/d47/f51 0 2024-02-28T23:07:32.284 INFO:tasks.workunit.client.1.smithi084.stdout:1/416: rmdir d12/d1b/d2e/d52/d61 39 2024-02-28T23:07:32.287 INFO:tasks.workunit.client.1.smithi084.stdout:0/380: mkdir d1/d6/db/d37/d72/d7b 0 2024-02-28T23:07:32.287 INFO:tasks.workunit.client.1.smithi084.stdout:4/298: dread - d1/d18/d20/d14/d30/f53 zero size 2024-02-28T23:07:32.289 INFO:tasks.workunit.client.0.smithi057.stdout:8/237: stat d4/f5 0 2024-02-28T23:07:32.290 INFO:tasks.workunit.client.0.smithi057.stdout:2/233: getdents d8/d11 0 2024-02-28T23:07:32.290 INFO:tasks.workunit.client.0.smithi057.stdout:0/236: creat d17/d19/d4d/f62 x:0 0 0 2024-02-28T23:07:32.290 INFO:tasks.workunit.client.1.smithi084.stdout:1/417: truncate d12/d1b/d2e/d52/f73 90814 0 2024-02-28T23:07:32.290 INFO:tasks.workunit.client.0.smithi057.stdout:6/252: symlink d0/d1b/d3e/l56 0 2024-02-28T23:07:32.292 INFO:tasks.workunit.client.0.smithi057.stdout:8/238: chown d4/f3a 3847 1 2024-02-28T23:07:32.292 INFO:tasks.workunit.client.1.smithi084.stdout:8/311: creat d0/d9/d63/d64/f6f x:0 0 0 2024-02-28T23:07:32.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/312: fsync d0/d24/d4d/f58 0 2024-02-28T23:07:32.295 INFO:tasks.workunit.client.1.smithi084.stdout:0/381: symlink d1/d6/d59/l7c 0 2024-02-28T23:07:32.296 INFO:tasks.workunit.client.0.smithi057.stdout:9/242: mkdir dd/de/d14/d48 0 2024-02-28T23:07:32.296 INFO:tasks.workunit.client.1.smithi084.stdout:6/326: getdents d1/d13/d50/d65 0 2024-02-28T23:07:32.297 INFO:tasks.workunit.client.0.smithi057.stdout:3/152: mknod d3/d4/d7/c32 0 2024-02-28T23:07:32.298 INFO:tasks.workunit.client.0.smithi057.stdout:6/253: read d0/d4/d25/f26 [22966,87207] 0 2024-02-28T23:07:32.299 INFO:tasks.workunit.client.1.smithi084.stdout:2/299: rename d5/f29 to d5/d7/d12/d24/d2a/f52 0 2024-02-28T23:07:32.299 INFO:tasks.workunit.client.1.smithi084.stdout:0/382: truncate d1/d8/f3c 1151164 0 2024-02-28T23:07:32.299 INFO:tasks.workunit.client.0.smithi057.stdout:0/237: truncate d17/d20/d26/f52 800825 0 2024-02-28T23:07:32.301 INFO:tasks.workunit.client.1.smithi084.stdout:2/300: write d5/f15 [4001518,60076] 0 2024-02-28T23:07:32.301 INFO:tasks.workunit.client.0.smithi057.stdout:7/264: mknod d0/d5/d25/d3e/c52 0 2024-02-28T23:07:32.302 INFO:tasks.workunit.client.0.smithi057.stdout:2/234: fdatasync d8/d11/f29 0 2024-02-28T23:07:32.303 INFO:tasks.workunit.client.0.smithi057.stdout:5/179: dwrite f1 [0,4194304] 0 2024-02-28T23:07:32.304 INFO:tasks.workunit.client.0.smithi057.stdout:2/235: dread - d8/d11/d21/d28/f3d zero size 2024-02-28T23:07:32.306 INFO:tasks.workunit.client.1.smithi084.stdout:6/327: rename d1/d13/d50/d65/c6a to d1/d13/d50/d65/c6b 0 2024-02-28T23:07:32.306 INFO:tasks.workunit.client.0.smithi057.stdout:8/239: mkdir d4/d13/d2c/d4e 0 2024-02-28T23:07:32.307 INFO:tasks.workunit.client.0.smithi057.stdout:2/236: write d8/f4f [613900,118527] 0 2024-02-28T23:07:32.311 INFO:tasks.workunit.client.1.smithi084.stdout:9/387: rename d4/d69/l7b to d4/d6/d15/d3c/l7f 0 2024-02-28T23:07:32.312 INFO:tasks.workunit.client.0.smithi057.stdout:8/240: fdatasync d4/d1e/d37/d3f/d42/f44 0 2024-02-28T23:07:32.315 INFO:tasks.workunit.client.1.smithi084.stdout:6/328: mkdir d1/d13/d2e/d41/d4d/d6c 0 2024-02-28T23:07:32.316 INFO:tasks.workunit.client.0.smithi057.stdout:3/153: symlink d3/d20/l33 0 2024-02-28T23:07:32.317 INFO:tasks.workunit.client.0.smithi057.stdout:7/265: creat d0/d3f/f53 x:0 0 0 2024-02-28T23:07:32.319 INFO:tasks.workunit.client.1.smithi084.stdout:9/388: read d4/d6/f36 [1764439,71639] 0 2024-02-28T23:07:32.319 INFO:tasks.workunit.client.1.smithi084.stdout:4/299: rename d1/d18/d20/d24/d2f/d3e/d52/d37/d55/l1b to d1/d4a/l63 0 2024-02-28T23:07:32.321 INFO:tasks.workunit.client.0.smithi057.stdout:8/241: dread - d4/d13/d18/f48 zero size 2024-02-28T23:07:32.322 INFO:tasks.workunit.client.1.smithi084.stdout:3/348: truncate d3/d8/df/f19 340861 0 2024-02-28T23:07:32.324 INFO:tasks.workunit.client.0.smithi057.stdout:8/242: chown d4/l25 10286430 1 2024-02-28T23:07:32.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/266: unlink d0/d3f/f53 0 2024-02-28T23:07:32.326 INFO:tasks.workunit.client.1.smithi084.stdout:2/301: rmdir d5 39 2024-02-28T23:07:32.326 INFO:tasks.workunit.client.1.smithi084.stdout:7/322: dread db/d10/f2c [0,4194304] 0 2024-02-28T23:07:32.326 INFO:tasks.workunit.client.1.smithi084.stdout:7/323: stat db/d10/d17/f58 0 2024-02-28T23:07:32.327 INFO:tasks.workunit.client.0.smithi057.stdout:8/243: readlink d4/d13/d2c/l2f 0 2024-02-28T23:07:32.327 INFO:tasks.workunit.client.1.smithi084.stdout:7/324: readlink db/l63 0 2024-02-28T23:07:32.328 INFO:tasks.workunit.client.0.smithi057.stdout:6/254: dread d0/fc [0,4194304] 0 2024-02-28T23:07:32.329 INFO:tasks.workunit.client.0.smithi057.stdout:8/244: fsync d4/d13/d18/f26 0 2024-02-28T23:07:32.331 INFO:tasks.workunit.client.1.smithi084.stdout:9/389: dread - d4/f13 zero size 2024-02-28T23:07:32.331 INFO:tasks.workunit.client.1.smithi084.stdout:0/383: dwrite d1/d8/f55 [0,4194304] 0 2024-02-28T23:07:32.332 INFO:tasks.workunit.client.1.smithi084.stdout:1/418: getdents d12/d1b/d2e/d52/d61 0 2024-02-28T23:07:32.333 INFO:tasks.workunit.client.0.smithi057.stdout:5/180: fdatasync f1 0 2024-02-28T23:07:32.334 INFO:tasks.workunit.client.1.smithi084.stdout:5/362: mkdir dc/de/d83 0 2024-02-28T23:07:32.334 INFO:tasks.workunit.client.1.smithi084.stdout:7/325: write db/d11/d27/d36/f5a [89379,66328] 0 2024-02-28T23:07:32.335 INFO:tasks.workunit.client.0.smithi057.stdout:3/154: mknod d3/d20/d2d/c34 0 2024-02-28T23:07:32.337 INFO:tasks.workunit.client.1.smithi084.stdout:6/329: dread d1/d9/d1d/d1e/d33/f64 [0,4194304] 0 2024-02-28T23:07:32.341 INFO:tasks.workunit.client.1.smithi084.stdout:9/390: mknod d4/d32/c80 0 2024-02-28T23:07:32.341 INFO:tasks.workunit.client.1.smithi084.stdout:7/326: mkdir db/d10/d17/d64 0 2024-02-28T23:07:32.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/419: mkdir d12/d1b/d91 0 2024-02-28T23:07:32.343 INFO:tasks.workunit.client.1.smithi084.stdout:7/327: readlink db/d10/d17/l39 0 2024-02-28T23:07:32.344 INFO:tasks.workunit.client.1.smithi084.stdout:5/363: creat dc/d51/d7c/f84 x:0 0 0 2024-02-28T23:07:32.346 INFO:tasks.workunit.client.1.smithi084.stdout:0/384: rename d1/d6/db/d35/d4a/f62 to d1/d6/db/d37/d72/f7d 0 2024-02-28T23:07:32.348 INFO:tasks.workunit.client.1.smithi084.stdout:3/349: unlink d3/d8/d1a/c3b 0 2024-02-28T23:07:32.348 INFO:tasks.workunit.client.1.smithi084.stdout:5/364: write dc/de/df/d6e/f74 [307400,109446] 0 2024-02-28T23:07:32.353 INFO:tasks.workunit.client.1.smithi084.stdout:3/350: write d3/d8/d14/d31/f5d [623741,128092] 0 2024-02-28T23:07:32.354 INFO:tasks.workunit.client.1.smithi084.stdout:9/391: creat d4/d6/d15/d12/f81 x:0 0 0 2024-02-28T23:07:32.356 INFO:tasks.workunit.client.1.smithi084.stdout:5/365: mkdir dc/d51/d85 0 2024-02-28T23:07:32.358 INFO:tasks.workunit.client.1.smithi084.stdout:5/366: dread - dc/d51/d7c/f84 zero size 2024-02-28T23:07:32.362 INFO:tasks.workunit.client.1.smithi084.stdout:3/351: mkdir d3/d8/d1a/d45/d86 0 2024-02-28T23:07:32.365 INFO:tasks.workunit.client.1.smithi084.stdout:6/330: dread d1/d9/d1d/f32 [0,4194304] 0 2024-02-28T23:07:32.367 INFO:tasks.workunit.client.1.smithi084.stdout:9/392: write d4/f1a [5103513,121978] 0 2024-02-28T23:07:32.369 INFO:tasks.workunit.client.1.smithi084.stdout:3/352: write d3/d8/d1a/d21/d3d/d43/f48 [1182960,2746] 0 2024-02-28T23:07:32.372 INFO:tasks.workunit.client.1.smithi084.stdout:1/420: rename d12/d1b/d3b/l64 to d12/l92 0 2024-02-28T23:07:32.374 INFO:tasks.workunit.client.1.smithi084.stdout:4/300: write d1/f6 [762391,8600] 0 2024-02-28T23:07:32.380 INFO:tasks.workunit.client.1.smithi084.stdout:1/421: write d12/d1b/d2e/d52/f73 [796172,52386] 0 2024-02-28T23:07:32.382 INFO:tasks.workunit.client.1.smithi084.stdout:8/313: dwrite d0/d9/d1b/d20/f25 [0,4194304] 0 2024-02-28T23:07:32.383 INFO:tasks.workunit.client.0.smithi057.stdout:0/238: getdents d17/d20/d23 0 2024-02-28T23:07:32.385 INFO:tasks.workunit.client.0.smithi057.stdout:4/226: dwrite d1a/d1e/f29 [0,4194304] 0 2024-02-28T23:07:32.387 INFO:tasks.workunit.client.1.smithi084.stdout:5/367: dwrite dc/de/df/d16/f4a [4194304,4194304] 0 2024-02-28T23:07:32.388 INFO:tasks.workunit.client.0.smithi057.stdout:2/237: creat d8/f54 x:0 0 0 2024-02-28T23:07:32.388 INFO:tasks.workunit.client.0.smithi057.stdout:6/255: mkdir d0/d1b/d3e/d3d/d42/d4f/d53/d57 0 2024-02-28T23:07:32.388 INFO:tasks.workunit.client.0.smithi057.stdout:5/181: dread - d4/d9/d14/f28 zero size 2024-02-28T23:07:32.389 INFO:tasks.workunit.client.0.smithi057.stdout:9/243: dwrite dd/de/d24/f28 [0,4194304] 0 2024-02-28T23:07:32.390 INFO:tasks.workunit.client.0.smithi057.stdout:2/238: fsync d8/d11/d21/f31 0 2024-02-28T23:07:32.392 INFO:tasks.workunit.client.1.smithi084.stdout:4/301: mkdir d1/d18/d20/d24/d2f/d3e/d52/d64 0 2024-02-28T23:07:32.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/239: fdatasync d8/fd 0 2024-02-28T23:07:32.394 INFO:tasks.workunit.client.0.smithi057.stdout:9/244: write dd/ff [335565,113239] 0 2024-02-28T23:07:32.394 INFO:tasks.workunit.client.1.smithi084.stdout:7/328: write db/d10/d17/f3d [334673,91890] 0 2024-02-28T23:07:32.397 INFO:tasks.workunit.client.1.smithi084.stdout:5/368: mknod dc/de/df/d16/d34/d3c/c86 0 2024-02-28T23:07:32.399 INFO:tasks.workunit.client.1.smithi084.stdout:5/369: write f7 [4525697,89785] 0 2024-02-28T23:07:32.400 INFO:tasks.workunit.client.1.smithi084.stdout:7/329: dread - db/d11/f44 zero size 2024-02-28T23:07:32.401 INFO:tasks.workunit.client.1.smithi084.stdout:8/314: symlink d0/d24/d6e/l70 0 2024-02-28T23:07:32.401 INFO:tasks.workunit.client.0.smithi057.stdout:5/182: unlink d4/d9/d16/f1b 0 2024-02-28T23:07:32.403 INFO:tasks.workunit.client.0.smithi057.stdout:5/183: write f2 [1859179,109204] 0 2024-02-28T23:07:32.404 INFO:tasks.workunit.client.0.smithi057.stdout:4/227: mknod d1a/d1e/d51/c54 0 2024-02-28T23:07:32.404 INFO:tasks.workunit.client.1.smithi084.stdout:4/302: rmdir d1/d4a 39 2024-02-28T23:07:32.404 INFO:tasks.workunit.client.0.smithi057.stdout:0/239: creat d17/d20/d23/d47/f63 x:0 0 0 2024-02-28T23:07:32.407 INFO:tasks.workunit.client.1.smithi084.stdout:0/385: rename d1/d6/db/c17 to d1/d6/d1b/c7e 0 2024-02-28T23:07:32.408 INFO:tasks.workunit.client.1.smithi084.stdout:5/370: rmdir dc 39 2024-02-28T23:07:32.410 INFO:tasks.workunit.client.1.smithi084.stdout:1/422: getdents d12/d84 0 2024-02-28T23:07:32.411 INFO:tasks.workunit.client.0.smithi057.stdout:2/240: rmdir d8/d51 0 2024-02-28T23:07:32.413 INFO:tasks.workunit.client.1.smithi084.stdout:4/303: mkdir d1/d18/d20/d65 0 2024-02-28T23:07:32.415 INFO:tasks.workunit.client.1.smithi084.stdout:8/315: rename d0/d9/d63/d64 to d0/d9/d30/d33/d71 0 2024-02-28T23:07:32.417 INFO:tasks.workunit.client.1.smithi084.stdout:1/423: read d12/d1b/d2e/d30/f1c [2158405,104377] 0 2024-02-28T23:07:32.418 INFO:tasks.workunit.client.0.smithi057.stdout:6/256: dwrite d0/d4/d11/f13 [0,4194304] 0 2024-02-28T23:07:32.418 INFO:tasks.workunit.client.1.smithi084.stdout:8/316: chown d0/d24/l37 32737343 1 2024-02-28T23:07:32.419 INFO:tasks.workunit.client.1.smithi084.stdout:0/386: mknod d1/d6/db/d35/d44/c7f 0 2024-02-28T23:07:32.419 INFO:tasks.workunit.client.1.smithi084.stdout:6/331: dwrite d1/d9/d1d/d1e/d33/d38/f54 [0,4194304] 0 2024-02-28T23:07:32.419 INFO:tasks.workunit.client.1.smithi084.stdout:7/330: creat db/d10/f65 x:0 0 0 2024-02-28T23:07:32.420 INFO:tasks.workunit.client.0.smithi057.stdout:4/228: creat d1a/d1b/d2d/d49/d4d/f55 x:0 0 0 2024-02-28T23:07:32.420 INFO:tasks.workunit.client.0.smithi057.stdout:5/184: rename d4/d21/d24/d2e to d4/d21/d24/d30 0 2024-02-28T23:07:32.423 INFO:tasks.workunit.client.1.smithi084.stdout:1/424: dread - d12/d1b/d63/d7d/f8c zero size 2024-02-28T23:07:32.428 INFO:tasks.workunit.client.0.smithi057.stdout:3/155: dwrite d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:32.438 INFO:tasks.workunit.client.0.smithi057.stdout:9/245: write dd/de/d14/f18 [4441700,127262] 0 2024-02-28T23:07:32.440 INFO:tasks.workunit.client.1.smithi084.stdout:9/393: dread d4/d6/d15/d75/d2d/f34 [4194304,4194304] 0 2024-02-28T23:07:32.443 INFO:tasks.workunit.client.0.smithi057.stdout:3/156: mkdir d3/d8/d27/d35 0 2024-02-28T23:07:32.444 INFO:tasks.workunit.client.0.smithi057.stdout:9/246: chown dd/de 76 1 2024-02-28T23:07:32.444 INFO:tasks.workunit.client.1.smithi084.stdout:3/353: dread d3/d8/d1a/f1f [0,4194304] 0 2024-02-28T23:07:32.447 INFO:tasks.workunit.client.0.smithi057.stdout:0/240: dwrite d17/d20/d26/d30/f31 [0,4194304] 0 2024-02-28T23:07:32.447 INFO:tasks.workunit.client.0.smithi057.stdout:2/241: dwrite d8/d11/d21/f31 [0,4194304] 0 2024-02-28T23:07:32.447 INFO:tasks.workunit.client.0.smithi057.stdout:3/157: chown d3/d4/d7/f10 1784582067 1 2024-02-28T23:07:32.450 INFO:tasks.workunit.client.0.smithi057.stdout:9/247: unlink dd/c34 0 2024-02-28T23:07:32.454 INFO:tasks.workunit.client.0.smithi057.stdout:0/241: mknod d17/d20/d26/c64 0 2024-02-28T23:07:32.454 INFO:tasks.workunit.client.1.smithi084.stdout:7/331: dwrite db/d10/d18/f47 [0,4194304] 0 2024-02-28T23:07:32.456 INFO:tasks.workunit.client.0.smithi057.stdout:0/242: rmdir d17/d19 39 2024-02-28T23:07:32.457 INFO:tasks.workunit.client.0.smithi057.stdout:4/229: rmdir d1a/d1e 39 2024-02-28T23:07:32.459 INFO:tasks.workunit.client.1.smithi084.stdout:7/332: write db/d10/f60 [585132,75820] 0 2024-02-28T23:07:32.459 INFO:tasks.workunit.client.0.smithi057.stdout:0/243: write d17/f42 [611459,107877] 0 2024-02-28T23:07:32.461 INFO:tasks.workunit.client.0.smithi057.stdout:4/230: write d1a/d1b/f2c [778017,39283] 0 2024-02-28T23:07:32.465 INFO:tasks.workunit.client.0.smithi057.stdout:2/242: rename d8/l13 to d8/d11/d21/d28/d49/l55 0 2024-02-28T23:07:32.470 INFO:tasks.workunit.client.0.smithi057.stdout:2/243: mkdir d8/d11/d56 0 2024-02-28T23:07:32.471 INFO:tasks.workunit.client.0.smithi057.stdout:4/231: write d1a/d1b/f1c [5051420,45019] 0 2024-02-28T23:07:32.472 INFO:tasks.workunit.client.1.smithi084.stdout:9/394: dwrite d4/d6/d15/d75/d2d/f34 [0,4194304] 0 2024-02-28T23:07:32.473 INFO:tasks.workunit.client.0.smithi057.stdout:4/232: write f16 [395572,13448] 0 2024-02-28T23:07:32.474 INFO:tasks.workunit.client.0.smithi057.stdout:9/248: dread dd/f12 [0,4194304] 0 2024-02-28T23:07:32.475 INFO:tasks.workunit.client.1.smithi084.stdout:9/395: chown d4/d6/f29 334504 1 2024-02-28T23:07:32.478 INFO:tasks.workunit.client.1.smithi084.stdout:8/317: symlink d0/d9/d30/d33/l72 0 2024-02-28T23:07:32.484 INFO:tasks.workunit.client.1.smithi084.stdout:7/333: dwrite db/d10/f60 [0,4194304] 0 2024-02-28T23:07:32.484 INFO:tasks.workunit.client.1.smithi084.stdout:3/354: rmdir d3/d8/d1a/d45/d5e 39 2024-02-28T23:07:32.484 INFO:tasks.workunit.client.0.smithi057.stdout:9/249: write dd/f31 [1636007,52518] 0 2024-02-28T23:07:32.486 INFO:tasks.workunit.client.0.smithi057.stdout:3/158: dwrite d3/d4/d7/f2b [0,4194304] 0 2024-02-28T23:07:32.489 INFO:tasks.workunit.client.0.smithi057.stdout:6/257: dwrite d0/d17/f45 [0,4194304] 0 2024-02-28T23:07:32.495 INFO:tasks.workunit.client.0.smithi057.stdout:0/244: dwrite d17/f42 [0,4194304] 0 2024-02-28T23:07:32.495 INFO:tasks.workunit.client.0.smithi057.stdout:4/233: truncate d1a/d1b/d2d/d49/d4d/f55 844630 0 2024-02-28T23:07:32.496 INFO:tasks.workunit.client.0.smithi057.stdout:3/159: creat d3/d4/d1d/f36 x:0 0 0 2024-02-28T23:07:32.497 INFO:tasks.workunit.client.1.smithi084.stdout:9/396: dread d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:32.499 INFO:tasks.workunit.client.1.smithi084.stdout:0/387: link d1/d6/db/d35/d4a/f66 d1/d6/db/d37/d72/d7b/f80 0 2024-02-28T23:07:32.500 INFO:tasks.workunit.client.0.smithi057.stdout:0/245: mknod d17/d20/d26/d48/c65 0 2024-02-28T23:07:32.505 INFO:tasks.workunit.client.1.smithi084.stdout:3/355: dread d3/d8/d1a/d21/d3d/f67 [0,4194304] 0 2024-02-28T23:07:32.505 INFO:tasks.workunit.client.1.smithi084.stdout:8/318: mknod d0/d44/c73 0 2024-02-28T23:07:32.508 INFO:tasks.workunit.client.1.smithi084.stdout:0/388: mkdir d1/d6/db/d35/d57/d81 0 2024-02-28T23:07:32.515 INFO:tasks.workunit.client.1.smithi084.stdout:9/397: rename d4/d6/d15/d12/d41/l66 to d4/d6/d15/d3c/l82 0 2024-02-28T23:07:32.517 INFO:tasks.workunit.client.0.smithi057.stdout:9/250: dwrite dd/de/d14/f17 [4194304,4194304] 0 2024-02-28T23:07:32.517 INFO:tasks.workunit.client.0.smithi057.stdout:4/234: mknod d1a/d1b/d2d/d49/c56 0 2024-02-28T23:07:32.517 INFO:tasks.workunit.client.0.smithi057.stdout:6/258: link d0/d4/d11/c4e d0/d4/d25/c58 0 2024-02-28T23:07:32.518 INFO:tasks.workunit.client.1.smithi084.stdout:9/398: write d4/d6/ff [2747321,115578] 0 2024-02-28T23:07:32.519 INFO:tasks.workunit.client.1.smithi084.stdout:8/319: read d0/d9/d1b/d38/f5d [120390,126114] 0 2024-02-28T23:07:32.520 INFO:tasks.workunit.client.1.smithi084.stdout:0/389: mkdir d1/d6/db/d35/d82 0 2024-02-28T23:07:32.522 INFO:tasks.workunit.client.0.smithi057.stdout:9/251: chown dd/f2f 2097 1 2024-02-28T23:07:32.523 INFO:tasks.workunit.client.1.smithi084.stdout:8/320: write d0/d24/d4d/f6a [747685,45167] 0 2024-02-28T23:07:32.523 INFO:tasks.workunit.client.0.smithi057.stdout:4/235: mknod d1a/d1b/d2d/d49/c57 0 2024-02-28T23:07:32.526 INFO:tasks.workunit.client.0.smithi057.stdout:6/259: unlink d0/d4/d11/l1e 0 2024-02-28T23:07:32.529 INFO:tasks.workunit.client.0.smithi057.stdout:3/160: rmdir d3/d8/d27/d35 0 2024-02-28T23:07:32.530 INFO:tasks.workunit.client.1.smithi084.stdout:4/304: write d1/f7 [3654197,35850] 0 2024-02-28T23:07:32.531 INFO:tasks.workunit.client.1.smithi084.stdout:8/321: unlink d0/d9/d30/l31 0 2024-02-28T23:07:32.532 INFO:tasks.workunit.client.1.smithi084.stdout:0/390: write d1/d8/f36 [1450573,46566] 0 2024-02-28T23:07:32.532 INFO:tasks.workunit.client.1.smithi084.stdout:9/399: chown d4/fc 107282 1 2024-02-28T23:07:32.533 INFO:tasks.workunit.client.1.smithi084.stdout:7/334: dread db/ff [0,4194304] 0 2024-02-28T23:07:32.535 INFO:tasks.workunit.client.1.smithi084.stdout:9/400: read - d4/d6/f6b zero size 2024-02-28T23:07:32.536 INFO:tasks.workunit.client.0.smithi057.stdout:6/260: dread d0/d17/f2b [0,4194304] 0 2024-02-28T23:07:32.536 INFO:tasks.workunit.client.0.smithi057.stdout:4/236: readlink d1a/d1e/l22 0 2024-02-28T23:07:32.536 INFO:tasks.workunit.client.0.smithi057.stdout:3/161: chown d3/d4/d7/f2b 13778 1 2024-02-28T23:07:32.536 INFO:tasks.workunit.client.0.smithi057.stdout:9/252: mknod dd/de/c49 0 2024-02-28T23:07:32.537 INFO:tasks.workunit.client.0.smithi057.stdout:3/162: write d3/d20/f23 [2059561,18516] 0 2024-02-28T23:07:32.538 INFO:tasks.workunit.client.1.smithi084.stdout:7/335: write db/d11/d27/f33 [386636,120229] 0 2024-02-28T23:07:32.538 INFO:tasks.workunit.client.1.smithi084.stdout:0/391: stat d1/d6/db/d76 0 2024-02-28T23:07:32.538 INFO:tasks.workunit.client.1.smithi084.stdout:4/305: mknod d1/d4b/c66 0 2024-02-28T23:07:32.539 INFO:tasks.workunit.client.1.smithi084.stdout:9/401: read - d4/d32/f5f zero size 2024-02-28T23:07:32.539 INFO:tasks.workunit.client.0.smithi057.stdout:3/163: chown d3/d4/d7 17 1 2024-02-28T23:07:32.540 INFO:tasks.workunit.client.1.smithi084.stdout:4/306: chown l0 1540300388 1 2024-02-28T23:07:32.540 INFO:tasks.workunit.client.1.smithi084.stdout:7/336: read db/d10/d17/f3d [401736,55962] 0 2024-02-28T23:07:32.540 INFO:tasks.workunit.client.0.smithi057.stdout:2/244: truncate d8/d11/f29 3936940 0 2024-02-28T23:07:32.541 INFO:tasks.workunit.client.1.smithi084.stdout:9/402: readlink d4/d6/le 0 2024-02-28T23:07:32.541 INFO:tasks.workunit.client.1.smithi084.stdout:8/322: creat d0/d9/d1b/f74 x:0 0 0 2024-02-28T23:07:32.545 INFO:tasks.workunit.client.0.smithi057.stdout:3/164: write d3/d4/d1d/f2e [4630373,2974] 0 2024-02-28T23:07:32.549 INFO:tasks.workunit.client.1.smithi084.stdout:7/337: mkdir db/d11/d27/d36/d66 0 2024-02-28T23:07:32.549 INFO:tasks.workunit.client.0.smithi057.stdout:6/261: dread d0/d4/f19 [0,4194304] 0 2024-02-28T23:07:32.549 INFO:tasks.workunit.client.1.smithi084.stdout:4/307: mknod d1/d18/d20/d24/d2f/d3e/d52/d48/c67 0 2024-02-28T23:07:32.552 INFO:tasks.workunit.client.1.smithi084.stdout:6/332: write d1/f1b [1398571,37491] 0 2024-02-28T23:07:32.552 INFO:tasks.workunit.client.1.smithi084.stdout:0/392: creat d1/d6/db/d35/d82/f83 x:0 0 0 2024-02-28T23:07:32.554 INFO:tasks.workunit.client.1.smithi084.stdout:8/323: creat d0/d9/d30/d33/d71/f75 x:0 0 0 2024-02-28T23:07:32.555 INFO:tasks.workunit.client.0.smithi057.stdout:5/185: dread f2 [0,4194304] 0 2024-02-28T23:07:32.556 INFO:tasks.workunit.client.1.smithi084.stdout:0/393: dread - d1/d8/d18/f61 zero size 2024-02-28T23:07:32.557 INFO:tasks.workunit.client.0.smithi057.stdout:5/186: symlink d4/d21/d24/d30/l31 0 2024-02-28T23:07:32.557 INFO:tasks.workunit.client.0.smithi057.stdout:3/165: dread d3/d4/f30 [0,4194304] 0 2024-02-28T23:07:32.558 INFO:tasks.workunit.client.1.smithi084.stdout:7/338: mkdir db/d11/d67 0 2024-02-28T23:07:32.558 INFO:tasks.workunit.client.1.smithi084.stdout:8/324: chown d0/d9/d1b/d38/d43/d51 1900 1 2024-02-28T23:07:32.560 INFO:tasks.workunit.client.0.smithi057.stdout:5/187: mkdir d4/d32 0 2024-02-28T23:07:32.560 INFO:tasks.workunit.client.1.smithi084.stdout:4/308: creat d1/d4b/f68 x:0 0 0 2024-02-28T23:07:32.561 INFO:tasks.workunit.client.0.smithi057.stdout:3/166: creat d3/d8/d1b/f37 x:0 0 0 2024-02-28T23:07:32.564 INFO:tasks.workunit.client.1.smithi084.stdout:4/309: write d1/f12 [630188,108153] 0 2024-02-28T23:07:32.565 INFO:tasks.workunit.client.1.smithi084.stdout:4/310: readlink d1/lb 0 2024-02-28T23:07:32.566 INFO:tasks.workunit.client.0.smithi057.stdout:4/237: rmdir d1a 39 2024-02-28T23:07:32.568 INFO:tasks.workunit.client.0.smithi057.stdout:2/245: dread d8/d11/f1b [0,4194304] 0 2024-02-28T23:07:32.568 INFO:tasks.workunit.client.1.smithi084.stdout:0/394: unlink d1/d8/d18/d28/c4d 0 2024-02-28T23:07:32.568 INFO:tasks.workunit.client.0.smithi057.stdout:3/167: mknod d3/d4/c38 0 2024-02-28T23:07:32.569 INFO:tasks.workunit.client.0.smithi057.stdout:6/262: dwrite d0/f4a [0,4194304] 0 2024-02-28T23:07:32.571 INFO:tasks.workunit.client.1.smithi084.stdout:3/356: write d3/d8/d1a/d21/f32 [4200418,101819] 0 2024-02-28T23:07:32.571 INFO:tasks.workunit.client.1.smithi084.stdout:4/311: write d1/d18/d20/d14/d30/d47/f58 [609676,101563] 0 2024-02-28T23:07:32.573 INFO:tasks.workunit.client.1.smithi084.stdout:5/371: sync 2024-02-28T23:07:32.573 INFO:tasks.workunit.client.1.smithi084.stdout:2/302: sync 2024-02-28T23:07:32.573 INFO:tasks.workunit.client.1.smithi084.stdout:0/395: write d1/d8/f55 [2019918,27583] 0 2024-02-28T23:07:32.573 INFO:tasks.workunit.client.0.smithi057.stdout:2/246: fsync d8/fd 0 2024-02-28T23:07:32.574 INFO:tasks.workunit.client.1.smithi084.stdout:7/339: creat db/d11/f68 x:0 0 0 2024-02-28T23:07:32.575 INFO:tasks.workunit.client.0.smithi057.stdout:3/168: unlink d3/d8/f16 0 2024-02-28T23:07:32.576 INFO:tasks.workunit.client.1.smithi084.stdout:9/403: dwrite d4/d6/d15/d12/d41/f64 [0,4194304] 0 2024-02-28T23:07:32.580 INFO:tasks.workunit.client.0.smithi057.stdout:9/253: truncate dd/f31 2031820 0 2024-02-28T23:07:32.581 INFO:tasks.workunit.client.1.smithi084.stdout:7/340: fsync db/d11/f32 0 2024-02-28T23:07:32.581 INFO:tasks.workunit.client.1.smithi084.stdout:9/404: fsync d4/f51 0 2024-02-28T23:07:32.583 INFO:tasks.workunit.client.0.smithi057.stdout:5/188: dwrite d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:32.587 INFO:tasks.workunit.client.0.smithi057.stdout:6/263: mknod d0/d4/d11/c59 0 2024-02-28T23:07:32.588 INFO:tasks.workunit.client.0.smithi057.stdout:4/238: mknod d1a/d48/c58 0 2024-02-28T23:07:32.588 INFO:tasks.workunit.client.0.smithi057.stdout:2/247: mknod d8/d11/d56/c57 0 2024-02-28T23:07:32.591 INFO:tasks.workunit.client.0.smithi057.stdout:6/264: readlink d0/d1b/d3e/l48 0 2024-02-28T23:07:32.591 INFO:tasks.workunit.client.1.smithi084.stdout:3/357: unlink d3/d8/d1a/f1f 0 2024-02-28T23:07:32.595 INFO:tasks.workunit.client.0.smithi057.stdout:6/265: write d0/d17/d20/f31 [78385,16917] 0 2024-02-28T23:07:32.596 INFO:tasks.workunit.client.0.smithi057.stdout:4/239: write d1a/d1b/d2d/f3c [167171,76917] 0 2024-02-28T23:07:32.601 INFO:tasks.workunit.client.0.smithi057.stdout:6/266: write d0/d17/f35 [2644912,29388] 0 2024-02-28T23:07:32.603 INFO:tasks.workunit.client.1.smithi084.stdout:4/312: dwrite d1/f4 [4194304,4194304] 0 2024-02-28T23:07:32.604 INFO:tasks.workunit.client.0.smithi057.stdout:6/267: read - d0/d17/d20/f49 zero size 2024-02-28T23:07:32.604 INFO:tasks.workunit.client.0.smithi057.stdout:2/248: symlink d8/d11/d21/d28/d41/l58 0 2024-02-28T23:07:32.604 INFO:tasks.workunit.client.0.smithi057.stdout:6/268: fdatasync d0/d4/ff 0 2024-02-28T23:07:32.604 INFO:tasks.workunit.client.0.smithi057.stdout:4/240: creat d1a/d1b/d2d/f59 x:0 0 0 2024-02-28T23:07:32.605 INFO:tasks.workunit.client.0.smithi057.stdout:9/254: dread dd/ff [0,4194304] 0 2024-02-28T23:07:32.606 INFO:tasks.workunit.client.1.smithi084.stdout:2/303: unlink d5/d7/d9/f32 0 2024-02-28T23:07:32.607 INFO:tasks.workunit.client.0.smithi057.stdout:5/189: dwrite d4/d7/d8/f13 [0,4194304] 0 2024-02-28T23:07:32.608 INFO:tasks.workunit.client.0.smithi057.stdout:6/269: chown d0/d4/d25/f2a 0 1 2024-02-28T23:07:32.611 INFO:tasks.workunit.client.0.smithi057.stdout:6/270: chown d0/d4/d25 16605161 1 2024-02-28T23:07:32.612 INFO:tasks.workunit.client.1.smithi084.stdout:4/313: write d1/d18/d20/d14/d30/f3d [3872702,108563] 0 2024-02-28T23:07:32.613 INFO:tasks.workunit.client.1.smithi084.stdout:9/405: dwrite d4/d32/f5c [0,4194304] 0 2024-02-28T23:07:32.614 INFO:tasks.workunit.client.1.smithi084.stdout:3/358: creat d3/d50/f87 x:0 0 0 2024-02-28T23:07:32.614 INFO:tasks.workunit.client.1.smithi084.stdout:8/325: getdents d0/d9/d30 0 2024-02-28T23:07:32.615 INFO:tasks.workunit.client.0.smithi057.stdout:2/249: rename d8/d11/d21/l3e to d8/d11/d21/d28/d41/l59 0 2024-02-28T23:07:32.616 INFO:tasks.workunit.client.0.smithi057.stdout:5/190: symlink d4/d9/d16/l33 0 2024-02-28T23:07:32.618 INFO:tasks.workunit.client.0.smithi057.stdout:9/255: dread dd/de/d14/f29 [0,4194304] 0 2024-02-28T23:07:32.618 INFO:tasks.workunit.client.0.smithi057.stdout:6/271: fsync d0/f1c 0 2024-02-28T23:07:32.618 INFO:tasks.workunit.client.1.smithi084.stdout:4/314: dread - d1/d18/d20/d24/d2f/d3e/d52/d37/f5a zero size 2024-02-28T23:07:32.620 INFO:tasks.workunit.client.1.smithi084.stdout:7/341: creat db/d10/d17/d64/f69 x:0 0 0 2024-02-28T23:07:32.620 INFO:tasks.workunit.client.1.smithi084.stdout:9/406: rename d4/d6 to d4/d6/d15/d12/d41/d83 22 2024-02-28T23:07:32.621 INFO:tasks.workunit.client.0.smithi057.stdout:2/250: creat d8/d11/d21/f5a x:0 0 0 2024-02-28T23:07:32.621 INFO:tasks.workunit.client.0.smithi057.stdout:6/272: chown d0/d1b/d3e/d3d/d42/d4f 10 1 2024-02-28T23:07:32.623 INFO:tasks.workunit.client.0.smithi057.stdout:4/241: link d1a/d1e/d51/c54 d1a/d1e/d4b/c5a 0 2024-02-28T23:07:32.629 INFO:tasks.workunit.client.0.smithi057.stdout:6/273: creat d0/d1b/d3e/d3d/f5a x:0 0 0 2024-02-28T23:07:32.630 INFO:tasks.workunit.client.1.smithi084.stdout:1/425: sync 2024-02-28T23:07:32.630 INFO:tasks.workunit.client.1.smithi084.stdout:5/372: mknod dc/de/d83/c87 0 2024-02-28T23:07:32.631 INFO:tasks.workunit.client.0.smithi057.stdout:6/274: truncate d0/d17/f35 4630385 0 2024-02-28T23:07:32.633 INFO:tasks.workunit.client.0.smithi057.stdout:6/275: read - d0/d1b/d3e/d40/f4d zero size 2024-02-28T23:07:32.634 INFO:tasks.workunit.client.0.smithi057.stdout:9/256: dread dd/de/d14/f17 [4194304,4194304] 0 2024-02-28T23:07:32.634 INFO:tasks.workunit.client.0.smithi057.stdout:2/251: creat d8/d11/f5b x:0 0 0 2024-02-28T23:07:32.634 INFO:tasks.workunit.client.0.smithi057.stdout:5/191: link d4/d21/d24/l27 d4/d9/d16/d2c/l34 0 2024-02-28T23:07:32.635 INFO:tasks.workunit.client.1.smithi084.stdout:2/304: mkdir d5/d7/d12/d24/d2a/d53 0 2024-02-28T23:07:32.638 INFO:tasks.workunit.client.0.smithi057.stdout:6/276: fdatasync d0/d4/d11/f13 0 2024-02-28T23:07:32.640 INFO:tasks.workunit.client.1.smithi084.stdout:3/359: creat d3/d8/d1a/d21/d7b/f88 x:0 0 0 2024-02-28T23:07:32.642 INFO:tasks.workunit.client.0.smithi057.stdout:3/169: getdents d3/d4 0 2024-02-28T23:07:32.642 INFO:tasks.workunit.client.1.smithi084.stdout:2/305: truncate d5/f17 5018338 0 2024-02-28T23:07:32.643 INFO:tasks.workunit.client.0.smithi057.stdout:5/192: unlink d4/d7/d8/f13 0 2024-02-28T23:07:32.643 INFO:tasks.workunit.client.0.smithi057.stdout:4/242: dwrite d1a/d2f/f31 [0,4194304] 0 2024-02-28T23:07:32.645 INFO:tasks.workunit.client.0.smithi057.stdout:9/257: mkdir dd/de/d4a 0 2024-02-28T23:07:32.646 INFO:tasks.workunit.client.1.smithi084.stdout:6/333: truncate d1/d9/f3b 3805493 0 2024-02-28T23:07:32.648 INFO:tasks.workunit.client.0.smithi057.stdout:4/243: write d1a/d1b/d2d/d49/d4d/f55 [568915,113155] 0 2024-02-28T23:07:32.650 INFO:tasks.workunit.client.0.smithi057.stdout:6/277: creat d0/d1b/d3e/d3d/d42/d4f/d53/f5b x:0 0 0 2024-02-28T23:07:32.652 INFO:tasks.workunit.client.1.smithi084.stdout:0/396: dread d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:32.653 INFO:tasks.workunit.client.0.smithi057.stdout:6/278: readlink d0/d1b/d3e/l44 0 2024-02-28T23:07:32.654 INFO:tasks.workunit.client.0.smithi057.stdout:5/193: creat d4/d21/f35 x:0 0 0 2024-02-28T23:07:32.655 INFO:tasks.workunit.client.0.smithi057.stdout:9/258: truncate dd/f2d 1528994 0 2024-02-28T23:07:32.656 INFO:tasks.workunit.client.1.smithi084.stdout:1/426: dwrite d12/d1b/d2e/f79 [0,4194304] 0 2024-02-28T23:07:32.661 INFO:tasks.workunit.client.0.smithi057.stdout:3/170: dread d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:32.661 INFO:tasks.workunit.client.0.smithi057.stdout:5/194: rename d4/d7/fe to d4/d9/d14/f36 0 2024-02-28T23:07:32.663 INFO:tasks.workunit.client.0.smithi057.stdout:5/195: stat d4/d9/d14/f28 0 2024-02-28T23:07:32.664 INFO:tasks.workunit.client.1.smithi084.stdout:1/427: chown d12/d1b/d2e/d30/l35 60152907 1 2024-02-28T23:07:32.666 INFO:tasks.workunit.client.0.smithi057.stdout:5/196: truncate d4/d21/f35 337193 0 2024-02-28T23:07:32.670 INFO:tasks.workunit.client.1.smithi084.stdout:7/342: creat db/d5e/f6a x:0 0 0 2024-02-28T23:07:32.671 INFO:tasks.workunit.client.0.smithi057.stdout:6/279: rename d0/fd to d0/d17/d20/f5c 0 2024-02-28T23:07:32.671 INFO:tasks.workunit.client.1.smithi084.stdout:9/407: mknod d4/c84 0 2024-02-28T23:07:32.675 INFO:tasks.workunit.client.0.smithi057.stdout:3/171: creat d3/d4/d1d/f39 x:0 0 0 2024-02-28T23:07:32.679 INFO:tasks.workunit.client.1.smithi084.stdout:9/408: truncate d4/d6/d15/d3c/f7a 567009 0 2024-02-28T23:07:32.681 INFO:tasks.workunit.client.1.smithi084.stdout:1/428: readlink d12/l92 0 2024-02-28T23:07:32.681 INFO:tasks.workunit.client.1.smithi084.stdout:8/326: link d0/c7 d0/d9/d1b/d38/d43/d51/d54/c76 0 2024-02-28T23:07:32.681 INFO:tasks.workunit.client.0.smithi057.stdout:6/280: rmdir d0/d17/d20 39 2024-02-28T23:07:32.682 INFO:tasks.workunit.client.0.smithi057.stdout:4/244: link f15 d1a/d1e/f5b 0 2024-02-28T23:07:32.684 INFO:tasks.workunit.client.0.smithi057.stdout:4/245: chown d1a/d1b/d2d/d49/c56 8941 1 2024-02-28T23:07:32.684 INFO:tasks.workunit.client.1.smithi084.stdout:9/409: truncate d4/d6/f8 4824892 0 2024-02-28T23:07:32.685 INFO:tasks.workunit.client.0.smithi057.stdout:3/172: rename d3/d4/l21 to d3/d20/l3a 0 2024-02-28T23:07:32.686 INFO:tasks.workunit.client.0.smithi057.stdout:2/252: dread d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:32.687 INFO:tasks.workunit.client.1.smithi084.stdout:5/373: dread dc/f2a [0,4194304] 0 2024-02-28T23:07:32.687 INFO:tasks.workunit.client.1.smithi084.stdout:1/429: fdatasync d12/d1b/d63/d7d/f8c 0 2024-02-28T23:07:32.687 INFO:tasks.workunit.client.0.smithi057.stdout:4/246: creat d1a/d1b/f5c x:0 0 0 2024-02-28T23:07:32.688 INFO:tasks.workunit.client.1.smithi084.stdout:4/315: rmdir d1/d18/d20/d24/d2f/d3e/d52/d64 0 2024-02-28T23:07:32.688 INFO:tasks.workunit.client.0.smithi057.stdout:4/247: stat d1a/d1e 0 2024-02-28T23:07:32.689 INFO:tasks.workunit.client.0.smithi057.stdout:6/281: getdents d0/d4/d11 0 2024-02-28T23:07:32.690 INFO:tasks.workunit.client.1.smithi084.stdout:8/327: rename d0/d11/c4b to d0/d44/c77 0 2024-02-28T23:07:32.694 INFO:tasks.workunit.client.0.smithi057.stdout:6/282: rename d0/d1b/d54/f32 to d0/d1b/d54/f5d 0 2024-02-28T23:07:32.695 INFO:tasks.workunit.client.1.smithi084.stdout:5/374: creat dc/d51/f88 x:0 0 0 2024-02-28T23:07:32.696 INFO:tasks.workunit.client.0.smithi057.stdout:2/253: rename d8/d11/f1b to d8/d11/d21/d28/d41/f5c 0 2024-02-28T23:07:32.701 INFO:tasks.workunit.client.1.smithi084.stdout:1/430: rename d12/d1b/d2e/d30/f19 to d12/d1b/d3b/f93 0 2024-02-28T23:07:32.702 INFO:tasks.workunit.client.0.smithi057.stdout:5/197: rmdir d4/d9 39 2024-02-28T23:07:32.703 INFO:tasks.workunit.client.0.smithi057.stdout:9/259: write dd/de/d14/f2b [1699493,58468] 0 2024-02-28T23:07:32.704 INFO:tasks.workunit.client.0.smithi057.stdout:4/248: dwrite d1a/d1b/f3e [0,4194304] 0 2024-02-28T23:07:32.712 INFO:tasks.workunit.client.1.smithi084.stdout:5/375: creat dc/de/df/d31/f89 x:0 0 0 2024-02-28T23:07:32.716 INFO:tasks.workunit.client.1.smithi084.stdout:8/328: dwrite d0/d24/d4d/f58 [0,4194304] 0 2024-02-28T23:07:32.716 INFO:tasks.workunit.client.1.smithi084.stdout:1/431: dread d12/d1b/d2e/d52/f7c [0,4194304] 0 2024-02-28T23:07:32.717 INFO:tasks.workunit.client.1.smithi084.stdout:1/432: stat d12/d1b/d2e/d30/l39 0 2024-02-28T23:07:32.719 INFO:tasks.workunit.client.0.smithi057.stdout:1/168: sync 2024-02-28T23:07:32.722 INFO:tasks.workunit.client.1.smithi084.stdout:9/410: rename d4/f1a to d4/f85 0 2024-02-28T23:07:32.723 INFO:tasks.workunit.client.1.smithi084.stdout:0/397: write d1/d6/db/d37/d72/d7b/f80 [325116,12912] 0 2024-02-28T23:07:32.726 INFO:tasks.workunit.client.1.smithi084.stdout:2/306: truncate d5/dd/f34 4580479 0 2024-02-28T23:07:32.726 INFO:tasks.workunit.client.1.smithi084.stdout:3/360: link d3/d8/d1a/d45/d5e/d7a/l81 d3/d8/d1a/d29/d41/d5b/l89 0 2024-02-28T23:07:32.727 INFO:tasks.workunit.client.1.smithi084.stdout:0/398: write d1/d8/d18/f65 [1818957,34559] 0 2024-02-28T23:07:32.731 INFO:tasks.workunit.client.1.smithi084.stdout:7/343: truncate db/d10/d17/f42 2009631 0 2024-02-28T23:07:32.733 INFO:tasks.workunit.client.1.smithi084.stdout:5/376: symlink dc/d51/l8a 0 2024-02-28T23:07:32.733 INFO:tasks.workunit.client.1.smithi084.stdout:5/377: stat dc 0 2024-02-28T23:07:32.738 INFO:tasks.workunit.client.1.smithi084.stdout:4/316: rename d1/f3b to d1/d18/d20/d24/d2f/d3e/d52/d37/d55/f69 0 2024-02-28T23:07:32.742 INFO:tasks.workunit.client.1.smithi084.stdout:9/411: rmdir d4/d6/d15/d12 39 2024-02-28T23:07:32.744 INFO:tasks.workunit.client.1.smithi084.stdout:9/412: fdatasync d4/d32/f5c 0 2024-02-28T23:07:32.747 INFO:tasks.workunit.client.1.smithi084.stdout:3/361: creat d3/d27/f8a x:0 0 0 2024-02-28T23:07:32.749 INFO:tasks.workunit.client.1.smithi084.stdout:0/399: creat d1/d8/d69/f84 x:0 0 0 2024-02-28T23:07:32.750 INFO:tasks.workunit.client.1.smithi084.stdout:7/344: mknod db/d10/d17/c6b 0 2024-02-28T23:07:32.751 INFO:tasks.workunit.client.1.smithi084.stdout:5/378: rmdir dc/de/df/d16/d34/d3c 39 2024-02-28T23:07:32.753 INFO:tasks.workunit.client.1.smithi084.stdout:8/329: truncate d0/d9/f21 1536744 0 2024-02-28T23:07:32.754 INFO:tasks.workunit.client.1.smithi084.stdout:5/379: write dc/de/df/f71 [269491,60435] 0 2024-02-28T23:07:32.756 INFO:tasks.workunit.client.1.smithi084.stdout:4/317: symlink d1/d18/d20/d14/d30/l6a 0 2024-02-28T23:07:32.759 INFO:tasks.workunit.client.0.smithi057.stdout:5/198: creat d4/d21/f37 x:0 0 0 2024-02-28T23:07:32.759 INFO:tasks.workunit.client.0.smithi057.stdout:4/249: fdatasync d1a/d1b/f32 0 2024-02-28T23:07:32.764 INFO:tasks.workunit.client.1.smithi084.stdout:2/307: creat d5/d7/d2e/d4a/f54 x:0 0 0 2024-02-28T23:07:32.764 INFO:tasks.workunit.client.1.smithi084.stdout:6/334: sync 2024-02-28T23:07:32.765 INFO:tasks.workunit.client.0.smithi057.stdout:5/199: mknod d4/c38 0 2024-02-28T23:07:32.765 INFO:tasks.workunit.client.1.smithi084.stdout:3/362: creat d3/d8/d1a/d29/d41/d60/f8b x:0 0 0 2024-02-28T23:07:32.766 INFO:tasks.workunit.client.0.smithi057.stdout:9/260: rename dd/d3a to dd/de/d14/d48/d4b 0 2024-02-28T23:07:32.766 INFO:tasks.workunit.client.1.smithi084.stdout:0/400: mkdir d1/d8/d69/d6b/d85 0 2024-02-28T23:07:32.767 INFO:tasks.workunit.client.0.smithi057.stdout:4/250: creat d1a/d1e/d4b/f5d x:0 0 0 2024-02-28T23:07:32.772 INFO:tasks.workunit.client.1.smithi084.stdout:0/401: write d1/d8/d18/f63 [5724932,50771] 0 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.0.smithi057.stdout:9/261: write dd/f2d [998391,96307] 0 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.0.smithi057.stdout:1/169: rmdir d2/d7/df/db/d12/d19/d1b 39 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.0.smithi057.stdout:5/200: creat d4/d21/d24/f39 x:0 0 0 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.0.smithi057.stdout:9/262: symlink dd/d1b/d40/l4c 0 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.0.smithi057.stdout:4/251: chown d1a/d1e/d51/c54 8 1 2024-02-28T23:07:32.773 INFO:tasks.workunit.client.1.smithi084.stdout:7/345: fdatasync db/d10/f2c 0 2024-02-28T23:07:32.775 INFO:tasks.workunit.client.0.smithi057.stdout:9/263: fsync dd/ff 0 2024-02-28T23:07:32.778 INFO:tasks.workunit.client.0.smithi057.stdout:2/254: dread d8/d11/d21/d28/d49/f4e [0,4194304] 0 2024-02-28T23:07:32.779 INFO:tasks.workunit.client.1.smithi084.stdout:3/363: chown d3/d8/d1a/d21/d3d/f52 1 1 2024-02-28T23:07:32.780 INFO:tasks.workunit.client.0.smithi057.stdout:5/201: symlink d4/d32/l3a 0 2024-02-28T23:07:32.781 INFO:tasks.workunit.client.1.smithi084.stdout:9/413: unlink d4/d6/d15/l52 0 2024-02-28T23:07:32.782 INFO:tasks.workunit.client.0.smithi057.stdout:9/264: fdatasync dd/de/d14/f1f 0 2024-02-28T23:07:32.782 INFO:tasks.workunit.client.1.smithi084.stdout:8/330: dwrite d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:32.784 INFO:tasks.workunit.client.1.smithi084.stdout:1/433: truncate d12/d1b/f48 795335 0 2024-02-28T23:07:32.785 INFO:tasks.workunit.client.0.smithi057.stdout:5/202: unlink d4/fd 0 2024-02-28T23:07:32.786 INFO:tasks.workunit.client.0.smithi057.stdout:4/252: getdents d1a/d1e/d4b/d4f 0 2024-02-28T23:07:32.787 INFO:tasks.workunit.client.0.smithi057.stdout:2/255: truncate d8/d11/d39/f3c 364645 0 2024-02-28T23:07:32.787 INFO:tasks.workunit.client.0.smithi057.stdout:9/265: write dd/de/d14/f17 [4015537,85217] 0 2024-02-28T23:07:32.788 INFO:tasks.workunit.client.1.smithi084.stdout:9/414: fdatasync d4/d32/f5c 0 2024-02-28T23:07:32.789 INFO:tasks.workunit.client.0.smithi057.stdout:9/266: chown dd/d1b/c47 3857 1 2024-02-28T23:07:32.790 INFO:tasks.workunit.client.0.smithi057.stdout:1/170: dwrite d2/f15 [0,4194304] 0 2024-02-28T23:07:32.791 INFO:tasks.workunit.client.0.smithi057.stdout:1/171: stat d2 0 2024-02-28T23:07:32.794 INFO:tasks.workunit.client.0.smithi057.stdout:4/253: rename d1a/d1b/c26 to d1a/d1e/c5e 0 2024-02-28T23:07:32.801 INFO:tasks.workunit.client.0.smithi057.stdout:4/254: fdatasync d1a/d1b/d24/f38 0 2024-02-28T23:07:32.802 INFO:tasks.workunit.client.1.smithi084.stdout:5/380: rename dc/de/df/d6e/l7a to dc/de/df/d16/d34/l8b 0 2024-02-28T23:07:32.802 INFO:tasks.workunit.client.1.smithi084.stdout:8/331: fsync d0/d9/d1b/d38/d43/d51/f61 0 2024-02-28T23:07:32.802 INFO:tasks.workunit.client.1.smithi084.stdout:4/318: dwrite d1/f6 [0,4194304] 0 2024-02-28T23:07:32.802 INFO:tasks.workunit.client.1.smithi084.stdout:0/402: write d1/d8/f27 [2838256,96986] 0 2024-02-28T23:07:32.804 INFO:tasks.workunit.client.1.smithi084.stdout:6/335: dwrite d1/d13/d47/f4b [0,4194304] 0 2024-02-28T23:07:32.807 INFO:tasks.workunit.client.0.smithi057.stdout:9/267: mkdir dd/d46/d4d 0 2024-02-28T23:07:32.807 INFO:tasks.workunit.client.0.smithi057.stdout:2/256: dwrite d8/f42 [0,4194304] 0 2024-02-28T23:07:32.811 INFO:tasks.workunit.client.1.smithi084.stdout:8/332: dread d0/d9/d1b/d38/f3a [0,4194304] 0 2024-02-28T23:07:32.811 INFO:tasks.workunit.client.1.smithi084.stdout:2/308: dwrite d5/f17 [0,4194304] 0 2024-02-28T23:07:32.811 INFO:tasks.workunit.client.1.smithi084.stdout:9/415: dread d4/d6/d15/f71 [0,4194304] 0 2024-02-28T23:07:32.813 INFO:tasks.workunit.client.0.smithi057.stdout:5/203: getdents d4/d9/d16 0 2024-02-28T23:07:32.813 INFO:tasks.workunit.client.0.smithi057.stdout:1/172: dread d2/d7/df/db/d12/d19/d24/d2d/f31 [0,4194304] 0 2024-02-28T23:07:32.815 INFO:tasks.workunit.client.0.smithi057.stdout:5/204: write f1 [7860278,64261] 0 2024-02-28T23:07:32.815 INFO:tasks.workunit.client.0.smithi057.stdout:2/257: unlink d8/f40 0 2024-02-28T23:07:32.817 INFO:tasks.workunit.client.0.smithi057.stdout:3/173: dread d3/d4/f12 [0,4194304] 0 2024-02-28T23:07:32.821 INFO:tasks.workunit.client.0.smithi057.stdout:9/268: mknod dd/d46/d4d/c4e 0 2024-02-28T23:07:32.821 INFO:tasks.workunit.client.0.smithi057.stdout:4/255: getdents d1a/d2f 0 2024-02-28T23:07:32.823 INFO:tasks.workunit.client.1.smithi084.stdout:2/309: readlink d5/d7/d12/l21 0 2024-02-28T23:07:32.823 INFO:tasks.workunit.client.0.smithi057.stdout:3/174: creat d3/d20/d2d/f3b x:0 0 0 2024-02-28T23:07:32.826 INFO:tasks.workunit.client.0.smithi057.stdout:1/173: link d2/f15 d2/d7/df/db/d12/d19/d24/f34 0 2024-02-28T23:07:32.827 INFO:tasks.workunit.client.0.smithi057.stdout:3/175: write d3/d20/d2d/f3b [345955,30512] 0 2024-02-28T23:07:32.828 INFO:tasks.workunit.client.1.smithi084.stdout:7/346: rename db/l59 to db/d5e/l6c 0 2024-02-28T23:07:32.830 INFO:tasks.workunit.client.1.smithi084.stdout:0/403: symlink d1/d8/d18/d28/l86 0 2024-02-28T23:07:32.831 INFO:tasks.workunit.client.0.smithi057.stdout:2/258: link d8/d11/d21/f36 d8/d11/d15/f5d 0 2024-02-28T23:07:32.833 INFO:tasks.workunit.client.0.smithi057.stdout:4/256: getdents d1a/d1b/d2d 0 2024-02-28T23:07:32.835 INFO:tasks.workunit.client.0.smithi057.stdout:3/176: creat d3/d8/f3c x:0 0 0 2024-02-28T23:07:32.836 INFO:tasks.workunit.client.1.smithi084.stdout:7/347: chown db/d10/f60 5096 1 2024-02-28T23:07:32.837 INFO:tasks.workunit.client.1.smithi084.stdout:7/348: fsync f9 0 2024-02-28T23:07:32.837 INFO:tasks.workunit.client.1.smithi084.stdout:3/364: dwrite d3/d8/d1a/d29/f38 [0,4194304] 0 2024-02-28T23:07:32.838 INFO:tasks.workunit.client.0.smithi057.stdout:2/259: mknod d8/d11/c5e 0 2024-02-28T23:07:32.839 INFO:tasks.workunit.client.0.smithi057.stdout:4/257: mknod d1a/d1b/d24/d2e/d43/c5f 0 2024-02-28T23:07:32.840 INFO:tasks.workunit.client.1.smithi084.stdout:7/349: write db/d10/f56 [690352,95905] 0 2024-02-28T23:07:32.841 INFO:tasks.workunit.client.1.smithi084.stdout:6/336: dwrite d1/f1b [0,4194304] 0 2024-02-28T23:07:32.841 INFO:tasks.workunit.client.0.smithi057.stdout:1/174: link d2/d7/df/f1d d2/f35 0 2024-02-28T23:07:32.841 INFO:tasks.workunit.client.0.smithi057.stdout:2/260: chown d8/d11/d39/f3c 535131965 1 2024-02-28T23:07:32.846 INFO:tasks.workunit.client.1.smithi084.stdout:1/434: dwrite d12/d1b/d2e/d52/f67 [0,4194304] 0 2024-02-28T23:07:32.846 INFO:tasks.workunit.client.1.smithi084.stdout:7/350: stat db/d10/d17/c21 0 2024-02-28T23:07:32.846 INFO:tasks.workunit.client.0.smithi057.stdout:2/261: symlink d8/d11/d15/l5f 0 2024-02-28T23:07:32.848 INFO:tasks.workunit.client.0.smithi057.stdout:9/269: dwrite dd/de/d14/f29 [0,4194304] 0 2024-02-28T23:07:32.849 INFO:tasks.workunit.client.1.smithi084.stdout:7/351: truncate db/d10/f5d 313270 0 2024-02-28T23:07:32.850 INFO:tasks.workunit.client.0.smithi057.stdout:1/175: creat d2/d7/df/db/d12/d19/d1b/f36 x:0 0 0 2024-02-28T23:07:32.851 INFO:tasks.workunit.client.0.smithi057.stdout:2/262: fsync d8/d11/f17 0 2024-02-28T23:07:32.851 INFO:tasks.workunit.client.0.smithi057.stdout:3/177: link d3/l26 d3/d4/d7/l3d 0 2024-02-28T23:07:32.855 INFO:tasks.workunit.client.0.smithi057.stdout:1/176: write d2/f8 [914056,6464] 0 2024-02-28T23:07:32.856 INFO:tasks.workunit.client.0.smithi057.stdout:1/177: stat d2/d7/df/db/d12/d19/d1b/d2a/f2c 0 2024-02-28T23:07:32.858 INFO:tasks.workunit.client.0.smithi057.stdout:1/178: dread - d2/d7/df/db/d12/d19/d1b/d2a/f2c zero size 2024-02-28T23:07:32.859 INFO:tasks.workunit.client.0.smithi057.stdout:1/179: chown d2/f8 1 1 2024-02-28T23:07:32.860 INFO:tasks.workunit.client.0.smithi057.stdout:5/205: read d4/d21/f35 [196056,77168] 0 2024-02-28T23:07:32.863 INFO:tasks.workunit.client.0.smithi057.stdout:5/206: readlink d4/d7/d8/l25 0 2024-02-28T23:07:32.863 INFO:tasks.workunit.client.0.smithi057.stdout:1/180: creat d2/d7/f37 x:0 0 0 2024-02-28T23:07:32.866 INFO:tasks.workunit.client.0.smithi057.stdout:5/207: chown d4/d9/d14/f28 127 1 2024-02-28T23:07:32.870 INFO:tasks.workunit.client.0.smithi057.stdout:3/178: dwrite d3/d20/d2d/f3b [0,4194304] 0 2024-02-28T23:07:32.870 INFO:tasks.workunit.client.1.smithi084.stdout:9/416: symlink d4/d6/d15/d3c/l86 0 2024-02-28T23:07:32.872 INFO:tasks.workunit.client.1.smithi084.stdout:7/352: dwrite db/d11/d27/f33 [0,4194304] 0 2024-02-28T23:07:32.873 INFO:tasks.workunit.client.0.smithi057.stdout:1/181: creat d2/d7/df/db/d12/d19/d1b/f38 x:0 0 0 2024-02-28T23:07:32.880 INFO:tasks.workunit.client.0.smithi057.stdout:5/208: mkdir d4/d9/d14/d3b 0 2024-02-28T23:07:32.884 INFO:tasks.workunit.client.1.smithi084.stdout:2/310: write d5/d7/d2e/f46 [20496,87689] 0 2024-02-28T23:07:32.890 INFO:tasks.workunit.client.0.smithi057.stdout:3/179: creat d3/d8/d27/f3e x:0 0 0 2024-02-28T23:07:32.895 INFO:tasks.workunit.client.0.smithi057.stdout:5/209: dread d4/d9/d16/f19 [0,4194304] 0 2024-02-28T23:07:32.896 INFO:tasks.workunit.client.0.smithi057.stdout:3/180: mkdir d3/d20/d3f 0 2024-02-28T23:07:32.900 INFO:tasks.workunit.client.1.smithi084.stdout:4/319: mknod d1/d18/d20/d24/d2f/d3e/d52/d45/c6b 0 2024-02-28T23:07:32.903 INFO:tasks.workunit.client.0.smithi057.stdout:5/210: readlink l3 0 2024-02-28T23:07:32.903 INFO:tasks.workunit.client.0.smithi057.stdout:3/181: symlink d3/d20/d2d/l40 0 2024-02-28T23:07:32.903 INFO:tasks.workunit.client.0.smithi057.stdout:4/258: rename d1a/d1e/c5e to d1a/d1b/d2d/d49/c60 0 2024-02-28T23:07:32.903 INFO:tasks.workunit.client.1.smithi084.stdout:4/320: readlink d1/d18/d20/l19 0 2024-02-28T23:07:32.904 INFO:tasks.workunit.client.0.smithi057.stdout:4/259: fsync d1a/d1b/f46 0 2024-02-28T23:07:32.906 INFO:tasks.workunit.client.1.smithi084.stdout:4/321: dread - d1/d18/d20/d14/d30/f53 zero size 2024-02-28T23:07:32.907 INFO:tasks.workunit.client.0.smithi057.stdout:4/260: readlink l7 0 2024-02-28T23:07:32.913 INFO:tasks.workunit.client.0.smithi057.stdout:8/245: sync 2024-02-28T23:07:32.913 INFO:tasks.workunit.client.0.smithi057.stdout:7/267: sync 2024-02-28T23:07:32.913 INFO:tasks.workunit.client.0.smithi057.stdout:0/246: sync 2024-02-28T23:07:32.914 INFO:tasks.workunit.client.0.smithi057.stdout:8/246: stat d4/c46 0 2024-02-28T23:07:32.915 INFO:tasks.workunit.client.1.smithi084.stdout:7/353: creat db/d5e/f6d x:0 0 0 2024-02-28T23:07:32.915 INFO:tasks.workunit.client.1.smithi084.stdout:8/333: dread d0/d9/fb [0,4194304] 0 2024-02-28T23:07:32.917 INFO:tasks.workunit.client.0.smithi057.stdout:0/247: write d17/d20/f27 [4399094,47388] 0 2024-02-28T23:07:32.918 INFO:tasks.workunit.client.0.smithi057.stdout:1/182: getdents d2 0 2024-02-28T23:07:32.920 INFO:tasks.workunit.client.1.smithi084.stdout:5/381: rename dc/de/df/d16/l5e to dc/d51/d7c/l8c 0 2024-02-28T23:07:32.922 INFO:tasks.workunit.client.1.smithi084.stdout:2/311: symlink d5/d7/d12/l55 0 2024-02-28T23:07:32.923 INFO:tasks.workunit.client.1.smithi084.stdout:0/404: rmdir d1/d8/d18/d4c 39 2024-02-28T23:07:32.924 INFO:tasks.workunit.client.0.smithi057.stdout:8/247: mknod d4/d1e/d37/c4f 0 2024-02-28T23:07:32.925 INFO:tasks.workunit.client.0.smithi057.stdout:5/211: dread d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:32.926 INFO:tasks.workunit.client.0.smithi057.stdout:5/212: read f2 [2990671,59317] 0 2024-02-28T23:07:32.927 INFO:tasks.workunit.client.1.smithi084.stdout:1/435: mknod d12/c94 0 2024-02-28T23:07:32.929 INFO:tasks.workunit.client.0.smithi057.stdout:7/268: truncate d0/d5/f27 527259 0 2024-02-28T23:07:32.930 INFO:tasks.workunit.client.1.smithi084.stdout:4/322: rmdir d1/d18/d20 39 2024-02-28T23:07:32.931 INFO:tasks.workunit.client.0.smithi057.stdout:9/270: write dd/de/d14/f3d [2554667,119258] 0 2024-02-28T23:07:32.933 INFO:tasks.workunit.client.1.smithi084.stdout:9/417: creat d4/d6/d15/d12/d41/d55/f87 x:0 0 0 2024-02-28T23:07:32.934 INFO:tasks.workunit.client.0.smithi057.stdout:9/271: write dd/f2f [158447,125771] 0 2024-02-28T23:07:32.938 INFO:tasks.workunit.client.1.smithi084.stdout:8/334: rmdir d0 39 2024-02-28T23:07:32.939 INFO:tasks.workunit.client.0.smithi057.stdout:2/263: dread d8/fb [0,4194304] 0 2024-02-28T23:07:32.939 INFO:tasks.workunit.client.0.smithi057.stdout:7/269: rmdir d0/d5/d25 39 2024-02-28T23:07:32.939 INFO:tasks.workunit.client.0.smithi057.stdout:6/283: sync 2024-02-28T23:07:32.942 INFO:tasks.workunit.client.0.smithi057.stdout:6/284: readlink d0/d17/l1a 0 2024-02-28T23:07:32.943 INFO:tasks.workunit.client.0.smithi057.stdout:9/272: chown dd/de/d14/f29 1548 1 2024-02-28T23:07:32.944 INFO:tasks.workunit.client.1.smithi084.stdout:3/365: rename d3/d8/d1a/d21/f32 to d3/d50/f8c 0 2024-02-28T23:07:32.944 INFO:tasks.workunit.client.1.smithi084.stdout:2/312: dwrite d5/f31 [0,4194304] 0 2024-02-28T23:07:32.945 INFO:tasks.workunit.client.0.smithi057.stdout:0/248: getdents d17 0 2024-02-28T23:07:32.945 INFO:tasks.workunit.client.0.smithi057.stdout:1/183: dread d2/d7/df/db/f20 [0,4194304] 0 2024-02-28T23:07:32.945 INFO:tasks.workunit.client.0.smithi057.stdout:8/248: dread d4/d13/d1a/f20 [0,4194304] 0 2024-02-28T23:07:32.945 INFO:tasks.workunit.client.0.smithi057.stdout:9/273: chown dd/d1b 61320380 1 2024-02-28T23:07:32.948 INFO:tasks.workunit.client.1.smithi084.stdout:6/337: creat d1/d9/d1d/d1e/d33/f6d x:0 0 0 2024-02-28T23:07:32.948 INFO:tasks.workunit.client.0.smithi057.stdout:4/261: dread d1a/d1b/f21 [4194304,4194304] 0 2024-02-28T23:07:32.948 INFO:tasks.workunit.client.0.smithi057.stdout:2/264: readlink d8/l14 0 2024-02-28T23:07:32.949 INFO:tasks.workunit.client.1.smithi084.stdout:2/313: truncate d5/d7/d2e/d4a/f54 575027 0 2024-02-28T23:07:32.951 INFO:tasks.workunit.client.1.smithi084.stdout:0/405: chown d1/d6/d1b/f58 155765453 1 2024-02-28T23:07:32.952 INFO:tasks.workunit.client.0.smithi057.stdout:1/184: fsync d2/d7/df/db/d12/d19/d1b/f36 0 2024-02-28T23:07:32.952 INFO:tasks.workunit.client.1.smithi084.stdout:1/436: creat d12/d1b/d63/d7d/f95 x:0 0 0 2024-02-28T23:07:32.954 INFO:tasks.workunit.client.0.smithi057.stdout:7/270: creat d0/d5/d25/d3e/f54 x:0 0 0 2024-02-28T23:07:32.958 INFO:tasks.workunit.client.0.smithi057.stdout:2/265: write d8/ff [589023,51382] 0 2024-02-28T23:07:32.959 INFO:tasks.workunit.client.0.smithi057.stdout:5/213: dwrite d4/f6 [0,4194304] 0 2024-02-28T23:07:32.964 INFO:tasks.workunit.client.1.smithi084.stdout:3/366: symlink d3/d8/d1a/d21/d7b/l8d 0 2024-02-28T23:07:32.965 INFO:tasks.workunit.client.1.smithi084.stdout:5/382: link dc/de/df/d50/f52 dc/de/d83/f8d 0 2024-02-28T23:07:32.966 INFO:tasks.workunit.client.1.smithi084.stdout:6/338: mknod d1/d13/d47/c6e 0 2024-02-28T23:07:32.968 INFO:tasks.workunit.client.0.smithi057.stdout:6/285: rename d0/fc to d0/d17/d20/f5e 0 2024-02-28T23:07:32.969 INFO:tasks.workunit.client.0.smithi057.stdout:8/249: symlink d4/d13/l50 0 2024-02-28T23:07:32.969 INFO:tasks.workunit.client.1.smithi084.stdout:4/323: mknod d1/c6c 0 2024-02-28T23:07:32.970 INFO:tasks.workunit.client.1.smithi084.stdout:1/437: symlink d12/d1b/d2e/d30/d77/l96 0 2024-02-28T23:07:32.971 INFO:tasks.workunit.client.0.smithi057.stdout:1/185: mkdir d2/d7/df/db/d12/d19/d24/d2d/d39 0 2024-02-28T23:07:32.973 INFO:tasks.workunit.client.1.smithi084.stdout:2/314: dwrite d5/d1f/f44 [0,4194304] 0 2024-02-28T23:07:32.974 INFO:tasks.workunit.client.1.smithi084.stdout:8/335: mkdir d0/d24/d78 0 2024-02-28T23:07:32.974 INFO:tasks.workunit.client.1.smithi084.stdout:3/367: symlink d3/l8e 0 2024-02-28T23:07:32.974 INFO:tasks.workunit.client.1.smithi084.stdout:6/339: write d1/d9/d1d/d1e/d33/d38/f4e [258394,26168] 0 2024-02-28T23:07:32.975 INFO:tasks.workunit.client.1.smithi084.stdout:1/438: write f11 [1575542,11079] 0 2024-02-28T23:07:32.977 INFO:tasks.workunit.client.1.smithi084.stdout:0/406: symlink d1/d6/db/d35/d57/d81/l87 0 2024-02-28T23:07:32.979 INFO:tasks.workunit.client.1.smithi084.stdout:3/368: fdatasync d3/d50/f7c 0 2024-02-28T23:07:32.980 INFO:tasks.workunit.client.0.smithi057.stdout:5/214: unlink d4/d21/d24/f39 0 2024-02-28T23:07:32.981 INFO:tasks.workunit.client.1.smithi084.stdout:2/315: chown d5/d7/d2e/d4a/f54 7849 1 2024-02-28T23:07:32.981 INFO:tasks.workunit.client.0.smithi057.stdout:3/182: dread d3/d20/f23 [0,4194304] 0 2024-02-28T23:07:32.983 INFO:tasks.workunit.client.1.smithi084.stdout:2/316: dread - d5/d7/d1d/f50 zero size 2024-02-28T23:07:32.988 INFO:tasks.workunit.client.0.smithi057.stdout:2/266: dwrite d8/f1a [0,4194304] 0 2024-02-28T23:07:32.991 INFO:tasks.workunit.client.1.smithi084.stdout:6/340: mknod d1/d13/d2e/d41/d2c/c6f 0 2024-02-28T23:07:32.995 INFO:tasks.workunit.client.1.smithi084.stdout:6/341: write d1/d9/d1d/d1e/f5b [959779,106065] 0 2024-02-28T23:07:32.997 INFO:tasks.workunit.client.1.smithi084.stdout:9/418: rename d4/d32/d3f/l4a to d4/d6/d15/d75/l88 0 2024-02-28T23:07:32.998 INFO:tasks.workunit.client.0.smithi057.stdout:8/250: dwrite d4/d1e/f2e [0,4194304] 0 2024-02-28T23:07:33.000 INFO:tasks.workunit.client.1.smithi084.stdout:4/324: unlink d1/d18/d20/d14/d30/c62 0 2024-02-28T23:07:33.001 INFO:tasks.workunit.client.0.smithi057.stdout:6/286: symlink d0/d1b/l5f 0 2024-02-28T23:07:33.002 INFO:tasks.workunit.client.1.smithi084.stdout:8/336: dwrite d0/f8 [0,4194304] 0 2024-02-28T23:07:33.003 INFO:tasks.workunit.client.1.smithi084.stdout:4/325: write d1/d18/f42 [1194191,59999] 0 2024-02-28T23:07:33.003 INFO:tasks.workunit.client.0.smithi057.stdout:9/274: creat dd/de/f4f x:0 0 0 2024-02-28T23:07:33.005 INFO:tasks.workunit.client.1.smithi084.stdout:9/419: dread d4/d6/d15/f71 [0,4194304] 0 2024-02-28T23:07:33.006 INFO:tasks.workunit.client.0.smithi057.stdout:6/287: dread - d0/d17/f2c zero size 2024-02-28T23:07:33.006 INFO:tasks.workunit.client.1.smithi084.stdout:4/326: read - d1/d18/d20/d14/d30/d47/f51 zero size 2024-02-28T23:07:33.007 INFO:tasks.workunit.client.0.smithi057.stdout:9/275: chown dd/de/l1a 34 1 2024-02-28T23:07:33.010 INFO:tasks.workunit.client.0.smithi057.stdout:9/276: chown dd/de/l1c 1850 1 2024-02-28T23:07:33.011 INFO:tasks.workunit.client.0.smithi057.stdout:1/186: rename d2/d7/df/db/d12/d19/d1b to d2/d7/df/db/d3a 0 2024-02-28T23:07:33.013 INFO:tasks.workunit.client.0.smithi057.stdout:3/183: stat d3/d8/l1f 0 2024-02-28T23:07:33.016 INFO:tasks.workunit.client.0.smithi057.stdout:0/249: getdents d17 0 2024-02-28T23:07:33.016 INFO:tasks.workunit.client.0.smithi057.stdout:2/267: dwrite d8/f54 [0,4194304] 0 2024-02-28T23:07:33.019 INFO:tasks.workunit.client.1.smithi084.stdout:6/342: creat d1/d13/f70 x:0 0 0 2024-02-28T23:07:33.024 INFO:tasks.workunit.client.1.smithi084.stdout:5/383: rename dc/de/df/d50 to dc/de/df/d31/d8e 0 2024-02-28T23:07:33.028 INFO:tasks.workunit.client.1.smithi084.stdout:7/354: dwrite db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:33.031 INFO:tasks.workunit.client.0.smithi057.stdout:1/187: dread d2/f15 [0,4194304] 0 2024-02-28T23:07:33.032 INFO:tasks.workunit.client.0.smithi057.stdout:6/288: fsync d0/d17/f2b 0 2024-02-28T23:07:33.033 INFO:tasks.workunit.client.1.smithi084.stdout:4/327: dwrite d1/f2e [0,4194304] 0 2024-02-28T23:07:33.035 INFO:tasks.workunit.client.0.smithi057.stdout:0/250: creat d17/d20/f66 x:0 0 0 2024-02-28T23:07:33.038 INFO:tasks.workunit.client.0.smithi057.stdout:9/277: dwrite dd/de/d14/f1e [0,4194304] 0 2024-02-28T23:07:33.039 INFO:tasks.workunit.client.0.smithi057.stdout:8/251: unlink d4/c1f 0 2024-02-28T23:07:33.040 INFO:tasks.workunit.client.0.smithi057.stdout:8/252: stat d4/le 0 2024-02-28T23:07:33.041 INFO:tasks.workunit.client.0.smithi057.stdout:3/184: rename d3/d4/c11 to d3/d8/d1b/c41 0 2024-02-28T23:07:33.042 INFO:tasks.workunit.client.1.smithi084.stdout:3/369: creat d3/d27/d7d/f8f x:0 0 0 2024-02-28T23:07:33.044 INFO:tasks.workunit.client.0.smithi057.stdout:2/268: dwrite d8/f1a [0,4194304] 0 2024-02-28T23:07:33.044 INFO:tasks.workunit.client.0.smithi057.stdout:3/185: readlink d3/d20/l33 0 2024-02-28T23:07:33.044 INFO:tasks.workunit.client.1.smithi084.stdout:3/370: chown d3/d8/d14/d31/f5d 163423461 1 2024-02-28T23:07:33.044 INFO:tasks.workunit.client.1.smithi084.stdout:9/420: rmdir d4/d32/d3f 39 2024-02-28T23:07:33.048 INFO:tasks.workunit.client.1.smithi084.stdout:0/407: link d1/d6/d59/d70/f73 d1/d6/db/d35/d4a/f88 0 2024-02-28T23:07:33.051 INFO:tasks.workunit.client.0.smithi057.stdout:8/253: rename d4/d13/c3c to d4/d1e/c51 0 2024-02-28T23:07:33.051 INFO:tasks.workunit.client.0.smithi057.stdout:1/188: read d2/d7/df/db/d12/d19/f23 [2051894,49135] 0 2024-02-28T23:07:33.051 INFO:tasks.workunit.client.0.smithi057.stdout:7/271: truncate d0/d5/f1e 2619259 0 2024-02-28T23:07:33.052 INFO:tasks.workunit.client.0.smithi057.stdout:3/186: fsync d3/d8/f29 0 2024-02-28T23:07:33.058 INFO:tasks.workunit.client.1.smithi084.stdout:6/343: creat d1/d13/d2e/d41/d2c/f71 x:0 0 0 2024-02-28T23:07:33.060 INFO:tasks.workunit.client.1.smithi084.stdout:6/344: write d1/d13/d2e/d41/d4d/f63 [1007501,107618] 0 2024-02-28T23:07:33.061 INFO:tasks.workunit.client.0.smithi057.stdout:9/278: creat dd/d1b/f50 x:0 0 0 2024-02-28T23:07:33.064 INFO:tasks.workunit.client.1.smithi084.stdout:6/345: read d1/d13/d47/f4b [2908047,76008] 0 2024-02-28T23:07:33.066 INFO:tasks.workunit.client.0.smithi057.stdout:4/262: truncate d1a/f36 1654588 0 2024-02-28T23:07:33.066 INFO:tasks.workunit.client.1.smithi084.stdout:7/355: fsync db/d11/d27/f35 0 2024-02-28T23:07:33.067 INFO:tasks.workunit.client.0.smithi057.stdout:3/187: mknod d3/d20/c42 0 2024-02-28T23:07:33.069 INFO:tasks.workunit.client.0.smithi057.stdout:9/279: creat dd/de/d14/f51 x:0 0 0 2024-02-28T23:07:33.069 INFO:tasks.workunit.client.0.smithi057.stdout:1/189: creat d2/d7/df/db/d3a/f3b x:0 0 0 2024-02-28T23:07:33.072 INFO:tasks.workunit.client.1.smithi084.stdout:7/356: dread db/d11/d27/f43 [0,4194304] 0 2024-02-28T23:07:33.074 INFO:tasks.workunit.client.1.smithi084.stdout:7/357: chown db/d10/d18/f47 89906143 1 2024-02-28T23:07:33.077 INFO:tasks.workunit.client.1.smithi084.stdout:7/358: truncate db/d11/d27/f43 566334 0 2024-02-28T23:07:33.078 INFO:tasks.workunit.client.0.smithi057.stdout:4/263: creat d1a/d48/f61 x:0 0 0 2024-02-28T23:07:33.083 INFO:tasks.workunit.client.1.smithi084.stdout:7/359: write db/d10/d17/d3b/f48 [2003289,49966] 0 2024-02-28T23:07:33.083 INFO:tasks.workunit.client.1.smithi084.stdout:8/337: unlink d0/d9/f35 0 2024-02-28T23:07:33.083 INFO:tasks.workunit.client.0.smithi057.stdout:3/188: symlink d3/d20/d3f/l43 0 2024-02-28T23:07:33.086 INFO:tasks.workunit.client.1.smithi084.stdout:8/338: chown d0/d9/d1b/d38/f3e 2607598 1 2024-02-28T23:07:33.086 INFO:tasks.workunit.client.0.smithi057.stdout:8/254: dwrite d4/d13/d18/f23 [0,4194304] 0 2024-02-28T23:07:33.087 INFO:tasks.workunit.client.1.smithi084.stdout:9/421: rmdir d4/d6/d15/d3c 39 2024-02-28T23:07:33.088 INFO:tasks.workunit.client.1.smithi084.stdout:6/346: dwrite d1/d13/d47/f4b [0,4194304] 0 2024-02-28T23:07:33.089 INFO:tasks.workunit.client.1.smithi084.stdout:9/422: write d4/f3b [1388525,99645] 0 2024-02-28T23:07:33.091 INFO:tasks.workunit.client.0.smithi057.stdout:1/190: creat d2/d7/df/db/d3a/f3c x:0 0 0 2024-02-28T23:07:33.091 INFO:tasks.workunit.client.1.smithi084.stdout:3/371: unlink d3/d50/f7c 0 2024-02-28T23:07:33.091 INFO:tasks.workunit.client.1.smithi084.stdout:6/347: chown d1/d9/d1d/d1e/d33/f4c 7250555 1 2024-02-28T23:07:33.098 INFO:tasks.workunit.client.1.smithi084.stdout:7/360: dread db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:33.098 INFO:tasks.workunit.client.1.smithi084.stdout:0/408: creat d1/d6/db/d35/d4a/d56/f89 x:0 0 0 2024-02-28T23:07:33.099 INFO:tasks.workunit.client.0.smithi057.stdout:2/269: dwrite d8/fb [0,4194304] 0 2024-02-28T23:07:33.099 INFO:tasks.workunit.client.0.smithi057.stdout:7/272: dwrite d0/d5/d36/d4e/f19 [0,4194304] 0 2024-02-28T23:07:33.103 INFO:tasks.workunit.client.1.smithi084.stdout:7/361: chown db/d11/f32 24183 1 2024-02-28T23:07:33.103 INFO:tasks.workunit.client.0.smithi057.stdout:8/255: mknod d4/c52 0 2024-02-28T23:07:33.103 INFO:tasks.workunit.client.0.smithi057.stdout:6/289: dread d0/d1b/d54/f5d [0,4194304] 0 2024-02-28T23:07:33.104 INFO:tasks.workunit.client.1.smithi084.stdout:6/348: dread d1/d9/d1d/d1e/d33/f5e [0,4194304] 0 2024-02-28T23:07:33.104 INFO:tasks.workunit.client.1.smithi084.stdout:1/439: dwrite f1 [0,4194304] 0 2024-02-28T23:07:33.107 INFO:tasks.workunit.client.1.smithi084.stdout:1/440: dread - d12/d1b/d2e/d30/d77/f83 zero size 2024-02-28T23:07:33.107 INFO:tasks.workunit.client.0.smithi057.stdout:2/270: stat d8/d11/d56/c57 0 2024-02-28T23:07:33.108 INFO:tasks.workunit.client.1.smithi084.stdout:5/384: rename dc/de/df/d16/l3e to dc/de/df/d16/d34/d54/d5f/d66/l8f 0 2024-02-28T23:07:33.108 INFO:tasks.workunit.client.1.smithi084.stdout:1/441: readlink d12/d1b/d3b/l49 0 2024-02-28T23:07:33.112 INFO:tasks.workunit.client.0.smithi057.stdout:7/273: rmdir d0/d5/d36 39 2024-02-28T23:07:33.113 INFO:tasks.workunit.client.0.smithi057.stdout:2/271: chown d8/d11/d21/f5a 48069065 1 2024-02-28T23:07:33.113 INFO:tasks.workunit.client.1.smithi084.stdout:4/328: symlink d1/l6d 0 2024-02-28T23:07:33.115 INFO:tasks.workunit.client.1.smithi084.stdout:8/339: mknod d0/d9/d1b/d38/c79 0 2024-02-28T23:07:33.121 INFO:tasks.workunit.client.0.smithi057.stdout:9/280: dwrite dd/de/d14/f3d [0,4194304] 0 2024-02-28T23:07:33.122 INFO:tasks.workunit.client.1.smithi084.stdout:2/317: getdents d5/d7/d2e 0 2024-02-28T23:07:33.122 INFO:tasks.workunit.client.0.smithi057.stdout:4/264: write d1a/f36 [40662,53775] 0 2024-02-28T23:07:33.125 INFO:tasks.workunit.client.1.smithi084.stdout:2/318: readlink d5/d7/d12/l39 0 2024-02-28T23:07:33.127 INFO:tasks.workunit.client.1.smithi084.stdout:7/362: dwrite db/d11/d27/f33 [0,4194304] 0 2024-02-28T23:07:33.128 INFO:tasks.workunit.client.0.smithi057.stdout:8/256: symlink d4/d13/d2c/d4e/l53 0 2024-02-28T23:07:33.130 INFO:tasks.workunit.client.1.smithi084.stdout:0/409: creat d1/d6/db/d35/f8a x:0 0 0 2024-02-28T23:07:33.131 INFO:tasks.workunit.client.1.smithi084.stdout:7/363: dread - db/d11/f5b zero size 2024-02-28T23:07:33.131 INFO:tasks.workunit.client.0.smithi057.stdout:1/191: read d2/d7/df/db/d12/d19/f23 [1571162,81824] 0 2024-02-28T23:07:33.132 INFO:tasks.workunit.client.0.smithi057.stdout:0/251: dwrite f4 [0,4194304] 0 2024-02-28T23:07:33.137 INFO:tasks.workunit.client.0.smithi057.stdout:9/281: write dd/de/f4f [185539,93500] 0 2024-02-28T23:07:33.140 INFO:tasks.workunit.client.1.smithi084.stdout:9/423: mkdir d4/d32/d3f/d89 0 2024-02-28T23:07:33.141 INFO:tasks.workunit.client.0.smithi057.stdout:8/257: rmdir d4/d13/d18 39 2024-02-28T23:07:33.141 INFO:tasks.workunit.client.1.smithi084.stdout:8/340: symlink d0/d9/d1b/l7a 0 2024-02-28T23:07:33.143 INFO:tasks.workunit.client.1.smithi084.stdout:8/341: dread - d0/d44/f67 zero size 2024-02-28T23:07:33.143 INFO:tasks.workunit.client.0.smithi057.stdout:7/274: creat d0/d5/f55 x:0 0 0 2024-02-28T23:07:33.144 INFO:tasks.workunit.client.0.smithi057.stdout:1/192: chown d2/d7/df/db/d3a/d2a/f2c 322 1 2024-02-28T23:07:33.145 INFO:tasks.workunit.client.0.smithi057.stdout:0/252: dread - d17/d20/d23/d47/f63 zero size 2024-02-28T23:07:33.146 INFO:tasks.workunit.client.0.smithi057.stdout:8/258: chown d4/f5 9741983 1 2024-02-28T23:07:33.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/290: dwrite d0/d1b/d3e/d3d/f55 [0,4194304] 0 2024-02-28T23:07:33.148 INFO:tasks.workunit.client.1.smithi084.stdout:2/319: creat d5/d7/d12/d24/f56 x:0 0 0 2024-02-28T23:07:33.156 INFO:tasks.workunit.client.1.smithi084.stdout:0/410: mknod d1/d6/c8b 0 2024-02-28T23:07:33.159 INFO:tasks.workunit.client.1.smithi084.stdout:2/320: fsync d5/d7/d12/d24/d2a/f52 0 2024-02-28T23:07:33.159 INFO:tasks.workunit.client.1.smithi084.stdout:0/411: chown d1/d8/d18/f63 29922391 1 2024-02-28T23:07:33.162 INFO:tasks.workunit.client.1.smithi084.stdout:9/424: dwrite d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:33.167 INFO:tasks.workunit.client.0.smithi057.stdout:6/291: mkdir d0/d4/d25/d60 0 2024-02-28T23:07:33.167 INFO:tasks.workunit.client.0.smithi057.stdout:0/253: creat d17/d20/d26/f67 x:0 0 0 2024-02-28T23:07:33.167 INFO:tasks.workunit.client.0.smithi057.stdout:8/259: mkdir d4/d1e/d37/d3f/d54 0 2024-02-28T23:07:33.167 INFO:tasks.workunit.client.1.smithi084.stdout:5/385: mknod dc/de/df/d16/d34/d3c/d77/c90 0 2024-02-28T23:07:33.168 INFO:tasks.workunit.client.0.smithi057.stdout:0/254: read - d17/f58 zero size 2024-02-28T23:07:33.168 INFO:tasks.workunit.client.1.smithi084.stdout:1/442: rename d12/d1b/d2e/d30/d77/f83 to d12/f97 0 2024-02-28T23:07:33.169 INFO:tasks.workunit.client.0.smithi057.stdout:9/282: rename dd/l26 to dd/de/d14/d48/d4b/d41/l52 0 2024-02-28T23:07:33.169 INFO:tasks.workunit.client.0.smithi057.stdout:0/255: chown c2 2805 1 2024-02-28T23:07:33.171 INFO:tasks.workunit.client.1.smithi084.stdout:4/329: getdents d1/d18/d20/d24 0 2024-02-28T23:07:33.174 INFO:tasks.workunit.client.0.smithi057.stdout:0/256: write d17/d20/d26/d30/f3f [781943,92203] 0 2024-02-28T23:07:33.176 INFO:tasks.workunit.client.0.smithi057.stdout:8/260: write d4/d13/d18/f23 [4425882,62807] 0 2024-02-28T23:07:33.177 INFO:tasks.workunit.client.0.smithi057.stdout:7/275: creat d0/d5/f56 x:0 0 0 2024-02-28T23:07:33.178 INFO:tasks.workunit.client.1.smithi084.stdout:2/321: rmdir d5/dd 39 2024-02-28T23:07:33.178 INFO:tasks.workunit.client.0.smithi057.stdout:4/265: dread f16 [0,4194304] 0 2024-02-28T23:07:33.182 INFO:tasks.workunit.client.0.smithi057.stdout:9/283: symlink dd/de/d14/d48/d4b/l53 0 2024-02-28T23:07:33.185 INFO:tasks.workunit.client.1.smithi084.stdout:7/364: dread db/d10/d17/f3d [0,4194304] 0 2024-02-28T23:07:33.188 INFO:tasks.workunit.client.0.smithi057.stdout:7/276: readlink d0/d5/d36/d4e/la 0 2024-02-28T23:07:33.189 INFO:tasks.workunit.client.1.smithi084.stdout:3/372: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:33.192 INFO:tasks.workunit.client.0.smithi057.stdout:8/261: symlink d4/d13/d2c/d4e/l55 0 2024-02-28T23:07:33.196 INFO:tasks.workunit.client.1.smithi084.stdout:8/342: creat d0/d9/d1b/d38/d43/d51/f7b x:0 0 0 2024-02-28T23:07:33.197 INFO:tasks.workunit.client.1.smithi084.stdout:5/386: dwrite dc/de/df/d31/f89 [0,4194304] 0 2024-02-28T23:07:33.200 INFO:tasks.workunit.client.1.smithi084.stdout:4/330: chown d1/d18/d20/d24/d2f/d3e/l4e 0 1 2024-02-28T23:07:33.206 INFO:tasks.workunit.client.0.smithi057.stdout:6/292: rename d0/d1b/d3e/c24 to d0/c61 0 2024-02-28T23:07:33.206 INFO:tasks.workunit.client.1.smithi084.stdout:7/365: unlink db/f1e 0 2024-02-28T23:07:33.208 INFO:tasks.workunit.client.0.smithi057.stdout:7/277: truncate d0/d5/d36/f3c 2488060 0 2024-02-28T23:07:33.208 INFO:tasks.workunit.client.1.smithi084.stdout:6/349: write d1/d9/d1d/f32 [1789904,60700] 0 2024-02-28T23:07:33.213 INFO:tasks.workunit.client.0.smithi057.stdout:7/278: chown d0/d5/l6 0 1 2024-02-28T23:07:33.218 INFO:tasks.workunit.client.1.smithi084.stdout:8/343: dwrite d0/f1a [0,4194304] 0 2024-02-28T23:07:33.220 INFO:tasks.workunit.client.0.smithi057.stdout:2/272: dwrite d8/d11/f1c [0,4194304] 0 2024-02-28T23:07:33.223 INFO:tasks.workunit.client.1.smithi084.stdout:2/322: rename d5/d7/d12/l26 to d5/d1f/d42/l57 0 2024-02-28T23:07:33.224 INFO:tasks.workunit.client.1.smithi084.stdout:5/387: creat dc/de/df/d16/d34/d54/f91 x:0 0 0 2024-02-28T23:07:33.225 INFO:tasks.workunit.client.1.smithi084.stdout:5/388: readlink dc/de/l44 0 2024-02-28T23:07:33.227 INFO:tasks.workunit.client.0.smithi057.stdout:9/284: dwrite dd/de/d24/f28 [0,4194304] 0 2024-02-28T23:07:33.228 INFO:tasks.workunit.client.0.smithi057.stdout:5/215: sync 2024-02-28T23:07:33.228 INFO:tasks.workunit.client.0.smithi057.stdout:3/189: sync 2024-02-28T23:07:33.228 INFO:tasks.workunit.client.1.smithi084.stdout:5/389: dread - dc/de/df/d16/d34/d3c/f5a zero size 2024-02-28T23:07:33.228 INFO:tasks.workunit.client.0.smithi057.stdout:2/273: fdatasync d8/ff 0 2024-02-28T23:07:33.231 INFO:tasks.workunit.client.1.smithi084.stdout:1/443: truncate d12/d1b/d3b/f44 690798 0 2024-02-28T23:07:33.232 INFO:tasks.workunit.client.1.smithi084.stdout:6/350: creat d1/d13/d2e/d41/d4d/f72 x:0 0 0 2024-02-28T23:07:33.232 INFO:tasks.workunit.client.0.smithi057.stdout:8/262: dread d4/f17 [0,4194304] 0 2024-02-28T23:07:33.237 INFO:tasks.workunit.client.1.smithi084.stdout:4/331: creat d1/f6e x:0 0 0 2024-02-28T23:07:33.238 INFO:tasks.workunit.client.0.smithi057.stdout:2/274: write d8/f19 [261687,124528] 0 2024-02-28T23:07:33.241 INFO:tasks.workunit.client.0.smithi057.stdout:8/263: fsync d4/d13/d1a/f4b 0 2024-02-28T23:07:33.241 INFO:tasks.workunit.client.0.smithi057.stdout:3/190: truncate d3/d8/f3c 697769 0 2024-02-28T23:07:33.242 INFO:tasks.workunit.client.1.smithi084.stdout:3/373: rename d3/d8/d14/d31/d34/c2c to d3/d8/d1a/d29/d41/c90 0 2024-02-28T23:07:33.247 INFO:tasks.workunit.client.0.smithi057.stdout:3/191: write d3/d20/d2d/f3b [1394196,59753] 0 2024-02-28T23:07:33.247 INFO:tasks.workunit.client.1.smithi084.stdout:2/323: creat d5/dd/f58 x:0 0 0 2024-02-28T23:07:33.247 INFO:tasks.workunit.client.0.smithi057.stdout:4/266: truncate d1a/d1b/f21 3417176 0 2024-02-28T23:07:33.252 INFO:tasks.workunit.client.1.smithi084.stdout:7/366: rmdir db/d11/d27/d36/d66 0 2024-02-28T23:07:33.253 INFO:tasks.workunit.client.0.smithi057.stdout:6/293: dread d0/f1c [0,4194304] 0 2024-02-28T23:07:33.253 INFO:tasks.workunit.client.0.smithi057.stdout:1/193: dwrite d2/f35 [0,4194304] 0 2024-02-28T23:07:33.257 INFO:tasks.workunit.client.1.smithi084.stdout:6/351: write d1/d13/d47/f4b [3536321,45946] 0 2024-02-28T23:07:33.257 INFO:tasks.workunit.client.1.smithi084.stdout:7/367: write db/d11/d27/d36/f4c [424475,117792] 0 2024-02-28T23:07:33.260 INFO:tasks.workunit.client.1.smithi084.stdout:1/444: dwrite d12/f97 [0,4194304] 0 2024-02-28T23:07:33.261 INFO:tasks.workunit.client.0.smithi057.stdout:8/264: mkdir d4/d1e/d56 0 2024-02-28T23:07:33.265 INFO:tasks.workunit.client.1.smithi084.stdout:9/425: dread d4/d6/f72 [0,4194304] 0 2024-02-28T23:07:33.267 INFO:tasks.workunit.client.0.smithi057.stdout:9/285: creat dd/de/d14/d48/d4b/f54 x:0 0 0 2024-02-28T23:07:33.267 INFO:tasks.workunit.client.1.smithi084.stdout:9/426: chown d4/d6/d15/d12/d41 200395 1 2024-02-28T23:07:33.269 INFO:tasks.workunit.client.0.smithi057.stdout:6/294: fsync d0/d17/d20/f49 0 2024-02-28T23:07:33.270 INFO:tasks.workunit.client.1.smithi084.stdout:4/332: creat d1/d18/d20/d14/f6f x:0 0 0 2024-02-28T23:07:33.271 INFO:tasks.workunit.client.1.smithi084.stdout:9/427: truncate d4/d6/d15/d12/f81 326087 0 2024-02-28T23:07:33.271 INFO:tasks.workunit.client.1.smithi084.stdout:4/333: fsync d1/f12 0 2024-02-28T23:07:33.274 INFO:tasks.workunit.client.1.smithi084.stdout:9/428: chown d4/d6/d15/d12/f48 10846 1 2024-02-28T23:07:33.274 INFO:tasks.workunit.client.0.smithi057.stdout:9/286: chown cb 421 1 2024-02-28T23:07:33.275 INFO:tasks.workunit.client.1.smithi084.stdout:4/334: chown d1/d18/d20/d24/d2f/d3e/d52/d37 2 1 2024-02-28T23:07:33.276 INFO:tasks.workunit.client.0.smithi057.stdout:7/279: link d0/d5/c4d d0/d5/d25/d3e/c57 0 2024-02-28T23:07:33.276 INFO:tasks.workunit.client.1.smithi084.stdout:9/429: chown d4/d32/c68 3785 1 2024-02-28T23:07:33.280 INFO:tasks.workunit.client.0.smithi057.stdout:0/257: dwrite d17/d19/f3c [0,4194304] 0 2024-02-28T23:07:33.282 INFO:tasks.workunit.client.1.smithi084.stdout:9/430: fsync d4/d6/d15/d12/d41/d55/f5b 0 2024-02-28T23:07:33.288 INFO:tasks.workunit.client.0.smithi057.stdout:6/295: chown d0/d1b/d3e/d3d/f5a 2255 1 2024-02-28T23:07:33.292 INFO:tasks.workunit.client.0.smithi057.stdout:6/296: chown d0/d1b/d3e/l23 3 1 2024-02-28T23:07:33.292 INFO:tasks.workunit.client.1.smithi084.stdout:2/324: rename d5/l6 to d5/d1f/l59 0 2024-02-28T23:07:33.294 INFO:tasks.workunit.client.1.smithi084.stdout:8/344: getdents d0/d9/d30/d33/d71 0 2024-02-28T23:07:33.294 INFO:tasks.workunit.client.0.smithi057.stdout:4/267: rename d1a/c2b to d1a/d2f/c62 0 2024-02-28T23:07:33.297 INFO:tasks.workunit.client.1.smithi084.stdout:1/445: mknod d12/d1b/d2e/d52/d61/d74/c98 0 2024-02-28T23:07:33.298 INFO:tasks.workunit.client.0.smithi057.stdout:5/216: dwrite d4/d9/d14/f28 [0,4194304] 0 2024-02-28T23:07:33.299 INFO:tasks.workunit.client.0.smithi057.stdout:2/275: dwrite d8/d11/d21/f5a [0,4194304] 0 2024-02-28T23:07:33.300 INFO:tasks.workunit.client.1.smithi084.stdout:7/368: dwrite db/d10/f65 [0,4194304] 0 2024-02-28T23:07:33.301 INFO:tasks.workunit.client.0.smithi057.stdout:7/280: write d0/d5/d36/d4e/d15/f3a [1901013,34872] 0 2024-02-28T23:07:33.301 INFO:tasks.workunit.client.0.smithi057.stdout:6/297: write d0/d1b/d3e/d40/f4d [640740,84211] 0 2024-02-28T23:07:33.302 INFO:tasks.workunit.client.0.smithi057.stdout:9/287: creat dd/de/d14/f55 x:0 0 0 2024-02-28T23:07:33.302 INFO:tasks.workunit.client.0.smithi057.stdout:8/265: link d4/d13/d1a/c2a d4/d1e/d56/c57 0 2024-02-28T23:07:33.303 INFO:tasks.workunit.client.1.smithi084.stdout:6/352: rename d1/d9/d1d/d1e/d33/d5c/c36 to d1/d13/d2e/c73 0 2024-02-28T23:07:33.305 INFO:tasks.workunit.client.1.smithi084.stdout:5/390: getdents dc 0 2024-02-28T23:07:33.306 INFO:tasks.workunit.client.1.smithi084.stdout:8/345: write d0/f53 [181018,34483] 0 2024-02-28T23:07:33.308 INFO:tasks.workunit.client.1.smithi084.stdout:2/325: creat d5/d7/d12/d24/d3a/d48/f5a x:0 0 0 2024-02-28T23:07:33.315 INFO:tasks.workunit.client.0.smithi057.stdout:6/298: creat d0/d1b/d3e/d3d/f62 x:0 0 0 2024-02-28T23:07:33.317 INFO:tasks.workunit.client.0.smithi057.stdout:6/299: write d0/d17/f2c [826342,61734] 0 2024-02-28T23:07:33.317 INFO:tasks.workunit.client.1.smithi084.stdout:5/391: symlink dc/d51/d7c/l92 0 2024-02-28T23:07:33.317 INFO:tasks.workunit.client.1.smithi084.stdout:8/346: write d0/d9/d30/d33/f5e [826546,67899] 0 2024-02-28T23:07:33.320 INFO:tasks.workunit.client.0.smithi057.stdout:2/276: symlink d8/d11/d15/l60 0 2024-02-28T23:07:33.324 INFO:tasks.workunit.client.1.smithi084.stdout:7/369: symlink db/d10/l6e 0 2024-02-28T23:07:33.328 INFO:tasks.workunit.client.0.smithi057.stdout:2/277: truncate d8/d11/f5b 1028571 0 2024-02-28T23:07:33.329 INFO:tasks.workunit.client.1.smithi084.stdout:1/446: dwrite d12/d1b/d3b/f59 [0,4194304] 0 2024-02-28T23:07:33.330 INFO:tasks.workunit.client.1.smithi084.stdout:5/392: fsync dc/de/f1a 0 2024-02-28T23:07:33.330 INFO:tasks.workunit.client.1.smithi084.stdout:2/326: dread d5/f31 [0,4194304] 0 2024-02-28T23:07:33.333 INFO:tasks.workunit.client.1.smithi084.stdout:5/393: dread - dc/f48 zero size 2024-02-28T23:07:33.333 INFO:tasks.workunit.client.0.smithi057.stdout:4/268: dwrite d1a/d1b/d24/d2e/f4a [0,4194304] 0 2024-02-28T23:07:33.335 INFO:tasks.workunit.client.0.smithi057.stdout:4/269: readlink l7 0 2024-02-28T23:07:33.336 INFO:tasks.workunit.client.0.smithi057.stdout:3/192: dread d3/d4/d7/f10 [0,4194304] 0 2024-02-28T23:07:33.337 INFO:tasks.workunit.client.1.smithi084.stdout:2/327: truncate d5/dd/f58 842156 0 2024-02-28T23:07:33.337 INFO:tasks.workunit.client.0.smithi057.stdout:9/288: dread dd/de/d14/f3d [0,4194304] 0 2024-02-28T23:07:33.338 INFO:tasks.workunit.client.0.smithi057.stdout:7/281: creat d0/d5/d36/d4e/d15/d29/d3d/d51/f58 x:0 0 0 2024-02-28T23:07:33.338 INFO:tasks.workunit.client.1.smithi084.stdout:8/347: mkdir d0/d9/d1b/d38/d43/d51/d7c 0 2024-02-28T23:07:33.339 INFO:tasks.workunit.client.0.smithi057.stdout:6/300: mknod d0/d17/d20/c63 0 2024-02-28T23:07:33.340 INFO:tasks.workunit.client.1.smithi084.stdout:5/394: write dc/de/df/d16/d34/f4d [527090,44345] 0 2024-02-28T23:07:33.340 INFO:tasks.workunit.client.0.smithi057.stdout:1/194: dwrite d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:33.340 INFO:tasks.workunit.client.0.smithi057.stdout:5/217: mknod d4/d9/d16/d2c/c3c 0 2024-02-28T23:07:33.343 INFO:tasks.workunit.client.1.smithi084.stdout:5/395: truncate dc/f27 1378142 0 2024-02-28T23:07:33.344 INFO:tasks.workunit.client.0.smithi057.stdout:4/270: chown d1a/d1b/d24/d2e/l30 172441287 1 2024-02-28T23:07:33.345 INFO:tasks.workunit.client.0.smithi057.stdout:6/301: creat d0/d17/d20/f64 x:0 0 0 2024-02-28T23:07:33.346 INFO:tasks.workunit.client.0.smithi057.stdout:5/218: fsync d4/d9/d16/f18 0 2024-02-28T23:07:33.347 INFO:tasks.workunit.client.1.smithi084.stdout:7/370: dwrite db/d11/f68 [0,4194304] 0 2024-02-28T23:07:33.348 INFO:tasks.workunit.client.0.smithi057.stdout:7/282: fsync d0/d5/d25/f28 0 2024-02-28T23:07:33.353 INFO:tasks.workunit.client.1.smithi084.stdout:8/348: rename d0/d9/d1b/d38 to d0/d9/d1b/d38/d43/d51/d7d 22 2024-02-28T23:07:33.353 INFO:tasks.workunit.client.0.smithi057.stdout:1/195: write d2/f15 [2155093,14156] 0 2024-02-28T23:07:33.354 INFO:tasks.workunit.client.0.smithi057.stdout:9/289: symlink dd/de/d4a/l56 0 2024-02-28T23:07:33.357 INFO:tasks.workunit.client.0.smithi057.stdout:4/271: chown d1a/d1b/d24/d2e/f3f 8934721 1 2024-02-28T23:07:33.360 INFO:tasks.workunit.client.1.smithi084.stdout:7/371: creat db/d5e/f6f x:0 0 0 2024-02-28T23:07:33.360 INFO:tasks.workunit.client.0.smithi057.stdout:6/302: creat d0/d1b/d54/f65 x:0 0 0 2024-02-28T23:07:33.360 INFO:tasks.workunit.client.0.smithi057.stdout:7/283: chown d0/d3f/f4c 2420 1 2024-02-28T23:07:33.362 INFO:tasks.workunit.client.0.smithi057.stdout:4/272: dread - d1a/d1b/d2d/f35 zero size 2024-02-28T23:07:33.363 INFO:tasks.workunit.client.0.smithi057.stdout:1/196: write d2/d7/df/db/d3a/d2a/f2e [1779635,107972] 0 2024-02-28T23:07:33.363 INFO:tasks.workunit.client.0.smithi057.stdout:2/278: dread d8/d11/d21/d28/d49/f4e [0,4194304] 0 2024-02-28T23:07:33.365 INFO:tasks.workunit.client.0.smithi057.stdout:3/193: unlink d3/d4/d1d/l31 0 2024-02-28T23:07:33.366 INFO:tasks.workunit.client.1.smithi084.stdout:1/447: dwrite d12/d1b/d63/f2a [0,4194304] 0 2024-02-28T23:07:33.373 INFO:tasks.workunit.client.0.smithi057.stdout:9/290: stat dd/de/d14/d48/d4b/d41/l52 0 2024-02-28T23:07:33.374 INFO:tasks.workunit.client.1.smithi084.stdout:5/396: dwrite dc/d51/d7c/f84 [0,4194304] 0 2024-02-28T23:07:33.374 INFO:tasks.workunit.client.0.smithi057.stdout:9/291: chown dd/de/f4f 1 1 2024-02-28T23:07:33.379 INFO:tasks.workunit.client.0.smithi057.stdout:4/273: creat d1a/d1b/d2d/f63 x:0 0 0 2024-02-28T23:07:33.381 INFO:tasks.workunit.client.0.smithi057.stdout:3/194: creat d3/d8/d1b/f44 x:0 0 0 2024-02-28T23:07:33.384 INFO:tasks.workunit.client.0.smithi057.stdout:5/219: dwrite d4/d21/f2f [0,4194304] 0 2024-02-28T23:07:33.385 INFO:tasks.workunit.client.1.smithi084.stdout:7/372: dread db/d10/f60 [0,4194304] 0 2024-02-28T23:07:33.385 INFO:tasks.workunit.client.0.smithi057.stdout:4/274: chown d1a/d1b/d24/f3d 115372739 1 2024-02-28T23:07:33.385 INFO:tasks.workunit.client.0.smithi057.stdout:1/197: creat d2/d7/df/db/d12/d19/d24/d2d/f3d x:0 0 0 2024-02-28T23:07:33.390 INFO:tasks.workunit.client.0.smithi057.stdout:6/303: symlink d0/l66 0 2024-02-28T23:07:33.394 INFO:tasks.workunit.client.0.smithi057.stdout:9/292: rmdir dd/d46/d4d 39 2024-02-28T23:07:33.395 INFO:tasks.workunit.client.0.smithi057.stdout:2/279: dwrite d8/fe [0,4194304] 0 2024-02-28T23:07:33.396 INFO:tasks.workunit.client.1.smithi084.stdout:5/397: dwrite dc/de/df/d31/f4e [0,4194304] 0 2024-02-28T23:07:33.397 INFO:tasks.workunit.client.1.smithi084.stdout:1/448: dwrite d12/d1b/d2e/d32/f4b [0,4194304] 0 2024-02-28T23:07:33.398 INFO:tasks.workunit.client.1.smithi084.stdout:5/398: dread - dc/de/f79 zero size 2024-02-28T23:07:33.402 INFO:tasks.workunit.client.1.smithi084.stdout:7/373: mknod db/d11/d27/d36/c70 0 2024-02-28T23:07:33.405 INFO:tasks.workunit.client.1.smithi084.stdout:7/374: write db/d11/d27/f33 [1558983,5588] 0 2024-02-28T23:07:33.409 INFO:tasks.workunit.client.1.smithi084.stdout:1/449: symlink d12/d1b/d3b/l99 0 2024-02-28T23:07:33.413 INFO:tasks.workunit.client.1.smithi084.stdout:1/450: write d12/d1b/f31 [7889471,34702] 0 2024-02-28T23:07:33.413 INFO:tasks.workunit.client.1.smithi084.stdout:5/399: creat dc/d51/d7c/f93 x:0 0 0 2024-02-28T23:07:33.413 INFO:tasks.workunit.client.0.smithi057.stdout:4/275: read d1a/d1b/d2d/d49/d4d/f55 [769776,12176] 0 2024-02-28T23:07:33.413 INFO:tasks.workunit.client.0.smithi057.stdout:1/198: symlink d2/d7/df/db/d12/l3e 0 2024-02-28T23:07:33.414 INFO:tasks.workunit.client.0.smithi057.stdout:6/304: write d0/d4/d11/f2d [481817,113956] 0 2024-02-28T23:07:33.415 INFO:tasks.workunit.client.1.smithi084.stdout:1/451: stat d12/d1b/d2e/d30/c1e 0 2024-02-28T23:07:33.419 INFO:tasks.workunit.client.0.smithi057.stdout:2/280: mknod d8/d11/d21/c61 0 2024-02-28T23:07:33.419 INFO:tasks.workunit.client.0.smithi057.stdout:5/220: mknod d4/c3d 0 2024-02-28T23:07:33.420 INFO:tasks.workunit.client.0.smithi057.stdout:9/293: mknod dd/de/c57 0 2024-02-28T23:07:33.422 INFO:tasks.workunit.client.0.smithi057.stdout:1/199: mkdir d2/d7/df/db/d12/d3f 0 2024-02-28T23:07:33.424 INFO:tasks.workunit.client.0.smithi057.stdout:6/305: unlink d0/d1b/d3e/l48 0 2024-02-28T23:07:33.424 INFO:tasks.workunit.client.0.smithi057.stdout:8/266: dread d4/f7 [8388608,4194304] 0 2024-02-28T23:07:33.426 INFO:tasks.workunit.client.1.smithi084.stdout:7/375: write db/d10/f60 [633388,39643] 0 2024-02-28T23:07:33.431 INFO:tasks.workunit.client.0.smithi057.stdout:5/221: chown d4/d21/d24/f22 14001 1 2024-02-28T23:07:33.431 INFO:tasks.workunit.client.1.smithi084.stdout:2/328: dread d5/d7/d12/d24/d3a/f4e [0,4194304] 0 2024-02-28T23:07:33.435 INFO:tasks.workunit.client.1.smithi084.stdout:5/400: write dc/de/df/d16/d34/f6c [1300700,78996] 0 2024-02-28T23:07:33.445 INFO:tasks.workunit.client.1.smithi084.stdout:3/374: truncate d3/d27/f2d 7270461 0 2024-02-28T23:07:33.445 INFO:tasks.workunit.client.0.smithi057.stdout:3/195: dwrite d3/d8/f9 [0,4194304] 0 2024-02-28T23:07:33.449 INFO:tasks.workunit.client.0.smithi057.stdout:8/267: creat d4/d13/d1a/f58 x:0 0 0 2024-02-28T23:07:33.450 INFO:tasks.workunit.client.1.smithi084.stdout:5/401: symlink dc/de/l94 0 2024-02-28T23:07:33.452 INFO:tasks.workunit.client.0.smithi057.stdout:6/306: write d0/d1b/d54/f5d [4531877,11368] 0 2024-02-28T23:07:33.453 INFO:tasks.workunit.client.0.smithi057.stdout:1/200: symlink d2/d7/df/db/d12/d19/l40 0 2024-02-28T23:07:33.453 INFO:tasks.workunit.client.0.smithi057.stdout:9/294: dwrite dd/de/d24/f28 [0,4194304] 0 2024-02-28T23:07:33.454 INFO:tasks.workunit.client.1.smithi084.stdout:8/349: dread d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:33.457 INFO:tasks.workunit.client.1.smithi084.stdout:1/452: dwrite d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:33.457 INFO:tasks.workunit.client.1.smithi084.stdout:3/375: symlink d3/d8/d1a/d45/d5e/d7a/l91 0 2024-02-28T23:07:33.460 INFO:tasks.workunit.client.1.smithi084.stdout:4/335: truncate d1/f2e 1926541 0 2024-02-28T23:07:33.462 INFO:tasks.workunit.client.1.smithi084.stdout:5/402: stat dc/d51/d7c/l8c 0 2024-02-28T23:07:33.463 INFO:tasks.workunit.client.0.smithi057.stdout:3/196: truncate d3/d4/f12 5093641 0 2024-02-28T23:07:33.464 INFO:tasks.workunit.client.1.smithi084.stdout:4/336: dread - d1/d18/d20/d14/d30/f31 zero size 2024-02-28T23:07:33.464 INFO:tasks.workunit.client.1.smithi084.stdout:2/329: getdents d5/d7/d12 0 2024-02-28T23:07:33.464 INFO:tasks.workunit.client.1.smithi084.stdout:6/353: truncate d1/d13/d47/f4b 1037518 0 2024-02-28T23:07:33.465 INFO:tasks.workunit.client.0.smithi057.stdout:8/268: write d4/f17 [2551990,31349] 0 2024-02-28T23:07:33.465 INFO:tasks.workunit.client.0.smithi057.stdout:9/295: read dd/de/d14/f2b [430915,55944] 0 2024-02-28T23:07:33.466 INFO:tasks.workunit.client.1.smithi084.stdout:2/330: dread - d5/d7/d1d/f50 zero size 2024-02-28T23:07:33.468 INFO:tasks.workunit.client.1.smithi084.stdout:1/453: mkdir d12/d84/d9a 0 2024-02-28T23:07:33.468 INFO:tasks.workunit.client.0.smithi057.stdout:3/197: dread - d3/d8/d1b/f37 zero size 2024-02-28T23:07:33.468 INFO:tasks.workunit.client.0.smithi057.stdout:5/222: dread d4/f6 [0,4194304] 0 2024-02-28T23:07:33.468 INFO:tasks.workunit.client.0.smithi057.stdout:6/307: unlink d0/d1b/d54/f5d 0 2024-02-28T23:07:33.469 INFO:tasks.workunit.client.1.smithi084.stdout:6/354: write d1/d13/d2e/d41/d2c/f71 [885227,78499] 0 2024-02-28T23:07:33.469 INFO:tasks.workunit.client.1.smithi084.stdout:2/331: readlink d5/d7/d1d/l23 0 2024-02-28T23:07:33.471 INFO:tasks.workunit.client.1.smithi084.stdout:2/332: fdatasync d5/f15 0 2024-02-28T23:07:33.471 INFO:tasks.workunit.client.1.smithi084.stdout:3/376: mknod d3/d8/d1a/d45/d5e/c92 0 2024-02-28T23:07:33.476 INFO:tasks.workunit.client.0.smithi057.stdout:1/201: link d2/f10 d2/d7/f41 0 2024-02-28T23:07:33.477 INFO:tasks.workunit.client.0.smithi057.stdout:7/284: dread d0/d5/d36/f3c [0,4194304] 0 2024-02-28T23:07:33.477 INFO:tasks.workunit.client.1.smithi084.stdout:2/333: readlink d5/d7/d12/l39 0 2024-02-28T23:07:33.477 INFO:tasks.workunit.client.1.smithi084.stdout:3/377: chown d3/d8/df/f12 630907743 1 2024-02-28T23:07:33.477 INFO:tasks.workunit.client.1.smithi084.stdout:3/378: fsync d3/d8/d1a/d21/d3d/d43/d61/f65 0 2024-02-28T23:07:33.479 INFO:tasks.workunit.client.0.smithi057.stdout:8/269: mkdir d4/d1e/d59 0 2024-02-28T23:07:33.482 INFO:tasks.workunit.client.1.smithi084.stdout:4/337: mknod d1/d4b/c70 0 2024-02-28T23:07:33.483 INFO:tasks.workunit.client.0.smithi057.stdout:9/296: rmdir dd/de/d14 39 2024-02-28T23:07:33.487 INFO:tasks.workunit.client.0.smithi057.stdout:5/223: rename d4/d32/l3a to d4/d32/l3e 0 2024-02-28T23:07:33.488 INFO:tasks.workunit.client.0.smithi057.stdout:5/224: dread - d4/d21/d24/f22 zero size 2024-02-28T23:07:33.491 INFO:tasks.workunit.client.0.smithi057.stdout:5/225: chown d4/d9/d16/f2b 0 1 2024-02-28T23:07:33.493 INFO:tasks.workunit.client.0.smithi057.stdout:8/270: symlink d4/d1e/d37/l5a 0 2024-02-28T23:07:33.495 INFO:tasks.workunit.client.1.smithi084.stdout:5/403: rmdir dc/d51/d85 0 2024-02-28T23:07:33.498 INFO:tasks.workunit.client.1.smithi084.stdout:8/350: getdents d0/d9/d63 0 2024-02-28T23:07:33.499 INFO:tasks.workunit.client.1.smithi084.stdout:1/454: dwrite d12/d84/f8e [0,4194304] 0 2024-02-28T23:07:33.500 INFO:tasks.workunit.client.1.smithi084.stdout:2/334: fsync d5/dd/f16 0 2024-02-28T23:07:33.501 INFO:tasks.workunit.client.0.smithi057.stdout:6/308: mkdir d0/d4/d25/d60/d67 0 2024-02-28T23:07:33.502 INFO:tasks.workunit.client.1.smithi084.stdout:5/404: chown dc/de/df/d16/d34/d54/f61 3 1 2024-02-28T23:07:33.504 INFO:tasks.workunit.client.1.smithi084.stdout:1/455: stat d12/d1b/d63/f56 0 2024-02-28T23:07:33.508 INFO:tasks.workunit.client.1.smithi084.stdout:8/351: chown d0/d9/c2b 10561 1 2024-02-28T23:07:33.510 INFO:tasks.workunit.client.0.smithi057.stdout:7/285: creat d0/d5/d36/d4e/d3b/f59 x:0 0 0 2024-02-28T23:07:33.511 INFO:tasks.workunit.client.1.smithi084.stdout:3/379: creat d3/d8/f93 x:0 0 0 2024-02-28T23:07:33.515 INFO:tasks.workunit.client.1.smithi084.stdout:4/338: link l0 d1/d18/d20/d65/l71 0 2024-02-28T23:07:33.519 INFO:tasks.workunit.client.1.smithi084.stdout:1/456: rmdir d12/d1b/d2e 39 2024-02-28T23:07:33.519 INFO:tasks.workunit.client.0.smithi057.stdout:8/271: creat d4/d1e/d37/d3f/d42/f5b x:0 0 0 2024-02-28T23:07:33.519 INFO:tasks.workunit.client.0.smithi057.stdout:1/202: link d2/d7/df/db/d3a/d2a/f2e d2/d7/df/db/d12/d19/d24/f42 0 2024-02-28T23:07:33.520 INFO:tasks.workunit.client.1.smithi084.stdout:4/339: write d1/d18/d20/d14/d30/f3d [2602622,87106] 0 2024-02-28T23:07:33.525 INFO:tasks.workunit.client.0.smithi057.stdout:2/281: dwrite d8/d11/d21/f36 [0,4194304] 0 2024-02-28T23:07:33.525 INFO:tasks.workunit.client.0.smithi057.stdout:1/203: truncate d2/d7/df/db/d3a/f36 151695 0 2024-02-28T23:07:33.529 INFO:tasks.workunit.client.0.smithi057.stdout:3/198: creat d3/d8/f45 x:0 0 0 2024-02-28T23:07:33.529 INFO:tasks.workunit.client.1.smithi084.stdout:3/380: rename d3/d50/c57 to d3/d8/df/c94 0 2024-02-28T23:07:33.529 INFO:tasks.workunit.client.0.smithi057.stdout:6/309: symlink d0/d1b/d54/l68 0 2024-02-28T23:07:33.531 INFO:tasks.workunit.client.1.smithi084.stdout:2/335: mkdir d5/d7/d12/d4f/d5b 0 2024-02-28T23:07:33.534 INFO:tasks.workunit.client.0.smithi057.stdout:8/272: mknod d4/d1e/c5c 0 2024-02-28T23:07:33.534 INFO:tasks.workunit.client.0.smithi057.stdout:8/273: stat d4/d13 0 2024-02-28T23:07:33.536 INFO:tasks.workunit.client.1.smithi084.stdout:6/355: dread d1/d9/f48 [0,4194304] 0 2024-02-28T23:07:33.538 INFO:tasks.workunit.client.0.smithi057.stdout:8/274: truncate d4/d13/d1a/f58 176803 0 2024-02-28T23:07:33.541 INFO:tasks.workunit.client.0.smithi057.stdout:1/204: write d2/d7/df/db/d3a/d2a/f2e [1893621,123922] 0 2024-02-28T23:07:33.543 INFO:tasks.workunit.client.1.smithi084.stdout:8/352: dread d0/d11/f2a [0,4194304] 0 2024-02-28T23:07:33.543 INFO:tasks.workunit.client.1.smithi084.stdout:2/336: mkdir d5/d7/d2e/d4a/d5c 0 2024-02-28T23:07:33.543 INFO:tasks.workunit.client.0.smithi057.stdout:5/226: write d4/d9/d16/f19 [580618,21518] 0 2024-02-28T23:07:33.546 INFO:tasks.workunit.client.0.smithi057.stdout:3/199: rename d3/d8/l1f to d3/l46 0 2024-02-28T23:07:33.546 INFO:tasks.workunit.client.0.smithi057.stdout:8/275: symlink d4/d13/d18/l5d 0 2024-02-28T23:07:33.546 INFO:tasks.workunit.client.1.smithi084.stdout:8/353: fsync d0/d9/d1b/d38/d43/d51/f61 0 2024-02-28T23:07:33.548 INFO:tasks.workunit.client.0.smithi057.stdout:5/227: mknod d4/d9/c3f 0 2024-02-28T23:07:33.549 INFO:tasks.workunit.client.1.smithi084.stdout:6/356: mkdir d1/d13/d50/d74 0 2024-02-28T23:07:33.551 INFO:tasks.workunit.client.0.smithi057.stdout:3/200: dread - d3/d8/f29 zero size 2024-02-28T23:07:33.553 INFO:tasks.workunit.client.0.smithi057.stdout:5/228: creat d4/d9/d16/f40 x:0 0 0 2024-02-28T23:07:33.555 INFO:tasks.workunit.client.0.smithi057.stdout:9/297: rename dd/de/d14/d48/d4b/l53 to dd/d46/d4d/l58 0 2024-02-28T23:07:33.556 INFO:tasks.workunit.client.1.smithi084.stdout:5/405: dread dc/de/df/d16/f1b [0,4194304] 0 2024-02-28T23:07:33.559 INFO:tasks.workunit.client.1.smithi084.stdout:6/357: creat d1/d9/d1d/d1e/d33/f75 x:0 0 0 2024-02-28T23:07:33.560 INFO:tasks.workunit.client.0.smithi057.stdout:9/298: fsync dd/de/d14/f18 0 2024-02-28T23:07:33.562 INFO:tasks.workunit.client.1.smithi084.stdout:8/354: getdents d0/d9/d1b/d20 0 2024-02-28T23:07:33.562 INFO:tasks.workunit.client.1.smithi084.stdout:6/358: fsync d1/d9/f4f 0 2024-02-28T23:07:33.563 INFO:tasks.workunit.client.0.smithi057.stdout:2/282: dwrite d8/d11/f1d [0,4194304] 0 2024-02-28T23:07:33.565 INFO:tasks.workunit.client.1.smithi084.stdout:5/406: rmdir dc/de/df/d16/d34/d3c 39 2024-02-28T23:07:33.566 INFO:tasks.workunit.client.0.smithi057.stdout:6/310: rename d0/d17/d20/f49 to d0/d1b/d3e/d3d/d42/d4f/d53/d57/f69 0 2024-02-28T23:07:33.567 INFO:tasks.workunit.client.1.smithi084.stdout:6/359: write d1/d9/d1d/d1e/f5b [2816713,41672] 0 2024-02-28T23:07:33.569 INFO:tasks.workunit.client.0.smithi057.stdout:8/276: getdents d4/d13 0 2024-02-28T23:07:33.572 INFO:tasks.workunit.client.0.smithi057.stdout:8/277: dread - d4/d13/d1a/f4b zero size 2024-02-28T23:07:33.572 INFO:tasks.workunit.client.0.smithi057.stdout:1/205: dwrite d2/d7/df/db/d12/d19/d24/d2d/f3d [0,4194304] 0 2024-02-28T23:07:33.573 INFO:tasks.workunit.client.1.smithi084.stdout:4/340: dwrite d1/d18/d20/d14/d30/f38 [0,4194304] 0 2024-02-28T23:07:33.573 INFO:tasks.workunit.client.1.smithi084.stdout:8/355: creat d0/d9/d63/f7e x:0 0 0 2024-02-28T23:07:33.574 INFO:tasks.workunit.client.0.smithi057.stdout:8/278: stat d4/d13/c15 0 2024-02-28T23:07:33.577 INFO:tasks.workunit.client.0.smithi057.stdout:5/229: dwrite d4/d7/d8/f26 [4194304,4194304] 0 2024-02-28T23:07:33.577 INFO:tasks.workunit.client.0.smithi057.stdout:2/283: creat d8/d11/d39/f62 x:0 0 0 2024-02-28T23:07:33.580 INFO:tasks.workunit.client.1.smithi084.stdout:6/360: symlink d1/d13/d2e/d41/d4d/l76 0 2024-02-28T23:07:33.580 INFO:tasks.workunit.client.0.smithi057.stdout:9/299: creat dd/d1b/d40/f59 x:0 0 0 2024-02-28T23:07:33.581 INFO:tasks.workunit.client.1.smithi084.stdout:3/381: dread d3/f6 [0,4194304] 0 2024-02-28T23:07:33.585 INFO:tasks.workunit.client.1.smithi084.stdout:4/341: truncate d1/d18/d20/d14/d30/f31 790497 0 2024-02-28T23:07:33.585 INFO:tasks.workunit.client.0.smithi057.stdout:1/206: unlink d2/d7/df/db/c13 0 2024-02-28T23:07:33.588 INFO:tasks.workunit.client.1.smithi084.stdout:0/412: sync 2024-02-28T23:07:33.589 INFO:tasks.workunit.client.1.smithi084.stdout:5/407: creat dc/de/df/d16/f95 x:0 0 0 2024-02-28T23:07:33.589 INFO:tasks.workunit.client.0.smithi057.stdout:3/201: getdents d3/d20 0 2024-02-28T23:07:33.591 INFO:tasks.workunit.client.0.smithi057.stdout:5/230: rename f1 to d4/d32/f41 0 2024-02-28T23:07:33.592 INFO:tasks.workunit.client.0.smithi057.stdout:9/300: unlink dd/de/l1c 0 2024-02-28T23:07:33.594 INFO:tasks.workunit.client.1.smithi084.stdout:4/342: chown d1/d18/d20/d14/d30/l56 443704 1 2024-02-28T23:07:33.595 INFO:tasks.workunit.client.1.smithi084.stdout:5/408: rmdir dc 39 2024-02-28T23:07:33.595 INFO:tasks.workunit.client.0.smithi057.stdout:1/207: creat d2/d7/df/db/d12/d3f/f43 x:0 0 0 2024-02-28T23:07:33.596 INFO:tasks.workunit.client.0.smithi057.stdout:9/301: unlink c5 0 2024-02-28T23:07:33.598 INFO:tasks.workunit.client.1.smithi084.stdout:6/361: getdents d1/d13/d50/d65 0 2024-02-28T23:07:33.599 INFO:tasks.workunit.client.0.smithi057.stdout:9/302: dread - dd/de/f43 zero size 2024-02-28T23:07:33.599 INFO:tasks.workunit.client.0.smithi057.stdout:5/231: truncate d4/d7/f1e 1321206 0 2024-02-28T23:07:33.602 INFO:tasks.workunit.client.0.smithi057.stdout:0/258: sync 2024-02-28T23:07:33.604 INFO:tasks.workunit.client.1.smithi084.stdout:3/382: dwrite d3/d8/d14/f2e [0,4194304] 0 2024-02-28T23:07:33.604 INFO:tasks.workunit.client.0.smithi057.stdout:5/232: stat d4/c5 0 2024-02-28T23:07:33.604 INFO:tasks.workunit.client.0.smithi057.stdout:8/279: dwrite d4/d1e/d37/d3f/f4d [0,4194304] 0 2024-02-28T23:07:33.605 INFO:tasks.workunit.client.1.smithi084.stdout:8/356: dread d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:33.609 INFO:tasks.workunit.client.1.smithi084.stdout:8/357: write d0/d24/f5b [426465,8014] 0 2024-02-28T23:07:33.611 INFO:tasks.workunit.client.1.smithi084.stdout:4/343: creat d1/d18/d20/d65/f72 x:0 0 0 2024-02-28T23:07:33.612 INFO:tasks.workunit.client.1.smithi084.stdout:6/362: rename d1/d13/f70 to d1/d13/d2e/d41/d4d/d6c/f77 0 2024-02-28T23:07:33.617 INFO:tasks.workunit.client.1.smithi084.stdout:3/383: symlink d3/d8/d1a/l95 0 2024-02-28T23:07:33.617 INFO:tasks.workunit.client.0.smithi057.stdout:9/303: rename dd/de/d14/f2b to dd/de/d14/f5a 0 2024-02-28T23:07:33.619 INFO:tasks.workunit.client.1.smithi084.stdout:5/409: rename dc/d51/f76 to dc/de/df/d16/d34/d54/d5f/d66/f96 0 2024-02-28T23:07:33.620 INFO:tasks.workunit.client.0.smithi057.stdout:2/284: dwrite d8/d11/f17 [0,4194304] 0 2024-02-28T23:07:33.621 INFO:tasks.workunit.client.1.smithi084.stdout:4/344: mkdir d1/d18/d20/d24/d2f/d3e/d52/d48/d73 0 2024-02-28T23:07:33.623 INFO:tasks.workunit.client.0.smithi057.stdout:9/304: readlink dd/de/d24/l30 0 2024-02-28T23:07:33.624 INFO:tasks.workunit.client.0.smithi057.stdout:3/202: dwrite d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:33.624 INFO:tasks.workunit.client.1.smithi084.stdout:3/384: unlink d3/d8/d14/d31/d34/f2b 0 2024-02-28T23:07:33.625 INFO:tasks.workunit.client.1.smithi084.stdout:5/410: creat dc/d51/d7c/f97 x:0 0 0 2024-02-28T23:07:33.628 INFO:tasks.workunit.client.1.smithi084.stdout:3/385: fsync d3/d8/d1a/d29/d41/f64 0 2024-02-28T23:07:33.629 INFO:tasks.workunit.client.1.smithi084.stdout:4/345: mknod d1/d4a/c74 0 2024-02-28T23:07:33.631 INFO:tasks.workunit.client.0.smithi057.stdout:0/259: dwrite d17/d20/f51 [0,4194304] 0 2024-02-28T23:07:33.631 INFO:tasks.workunit.client.1.smithi084.stdout:3/386: chown d3/d8/df/c20 40216321 1 2024-02-28T23:07:33.632 INFO:tasks.workunit.client.1.smithi084.stdout:5/411: mkdir dc/de/df/d16/d34/d3c/d98 0 2024-02-28T23:07:33.632 INFO:tasks.workunit.client.0.smithi057.stdout:1/208: getdents d2/d7/df/db/d12/d19/d24/d2d 0 2024-02-28T23:07:33.634 INFO:tasks.workunit.client.1.smithi084.stdout:3/387: write d3/d8/d14/d31/f5d [776108,71308] 0 2024-02-28T23:07:33.635 INFO:tasks.workunit.client.1.smithi084.stdout:5/412: chown dc/de/df/d31/d8e/c62 16683686 1 2024-02-28T23:07:33.636 INFO:tasks.workunit.client.1.smithi084.stdout:4/346: mknod d1/d18/d20/d24/d2f/d3e/d52/d48/d73/c75 0 2024-02-28T23:07:33.639 INFO:tasks.workunit.client.0.smithi057.stdout:5/233: dwrite d4/d21/d24/f22 [0,4194304] 0 2024-02-28T23:07:33.639 INFO:tasks.workunit.client.1.smithi084.stdout:5/413: mknod dc/de/df/d31/d8e/c99 0 2024-02-28T23:07:33.639 INFO:tasks.workunit.client.1.smithi084.stdout:4/347: symlink d1/d4b/l76 0 2024-02-28T23:07:33.640 INFO:tasks.workunit.client.0.smithi057.stdout:5/234: chown d4/d7 77472069 1 2024-02-28T23:07:33.642 INFO:tasks.workunit.client.1.smithi084.stdout:3/388: creat d3/d8/d14/d31/d34/f96 x:0 0 0 2024-02-28T23:07:33.644 INFO:tasks.workunit.client.0.smithi057.stdout:1/209: creat d2/f44 x:0 0 0 2024-02-28T23:07:33.645 INFO:tasks.workunit.client.0.smithi057.stdout:8/280: dread d4/d13/d18/f31 [0,4194304] 0 2024-02-28T23:07:33.645 INFO:tasks.workunit.client.0.smithi057.stdout:5/235: truncate d4/d9/f2a 4619600 0 2024-02-28T23:07:33.646 INFO:tasks.workunit.client.1.smithi084.stdout:4/348: creat d1/d18/f77 x:0 0 0 2024-02-28T23:07:33.647 INFO:tasks.workunit.client.1.smithi084.stdout:5/414: creat dc/de/df/d16/f9a x:0 0 0 2024-02-28T23:07:33.649 INFO:tasks.workunit.client.0.smithi057.stdout:9/305: creat dd/f5b x:0 0 0 2024-02-28T23:07:33.649 INFO:tasks.workunit.client.1.smithi084.stdout:8/358: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:33.651 INFO:tasks.workunit.client.1.smithi084.stdout:5/415: read dc/de/df/f71 [117250,113573] 0 2024-02-28T23:07:33.652 INFO:tasks.workunit.client.1.smithi084.stdout:4/349: unlink d1/d18/d20/d24/d2f/d3e/d52/d45/c6b 0 2024-02-28T23:07:33.654 INFO:tasks.workunit.client.1.smithi084.stdout:8/359: creat d0/d9/d1b/f7f x:0 0 0 2024-02-28T23:07:33.654 INFO:tasks.workunit.client.0.smithi057.stdout:5/236: symlink d4/d7/d8/l42 0 2024-02-28T23:07:33.654 INFO:tasks.workunit.client.0.smithi057.stdout:9/306: chown dd/de/d14/f29 279188 1 2024-02-28T23:07:33.655 INFO:tasks.workunit.client.1.smithi084.stdout:5/416: chown dc/de/l94 394 1 2024-02-28T23:07:33.655 INFO:tasks.workunit.client.0.smithi057.stdout:0/260: dwrite d17/d20/d23/d37/f43 [0,4194304] 0 2024-02-28T23:07:33.656 INFO:tasks.workunit.client.0.smithi057.stdout:5/237: write d4/d7/d8/f26 [4713588,46388] 0 2024-02-28T23:07:33.658 INFO:tasks.workunit.client.0.smithi057.stdout:8/281: rename d4/d13/d1a/f32 to d4/d1e/d37/d3f/d42/f5e 0 2024-02-28T23:07:33.659 INFO:tasks.workunit.client.1.smithi084.stdout:4/350: truncate d1/f2e 2048196 0 2024-02-28T23:07:33.659 INFO:tasks.workunit.client.1.smithi084.stdout:3/389: dwrite d3/d50/f66 [0,4194304] 0 2024-02-28T23:07:33.660 INFO:tasks.workunit.client.0.smithi057.stdout:5/238: write d4/d7/d8/f26 [733957,63944] 0 2024-02-28T23:07:33.660 INFO:tasks.workunit.client.1.smithi084.stdout:4/351: chown d1/f12 1 1 2024-02-28T23:07:33.660 INFO:tasks.workunit.client.0.smithi057.stdout:8/282: chown d4/d13/d1a/d4a 4 1 2024-02-28T23:07:33.661 INFO:tasks.workunit.client.0.smithi057.stdout:0/261: truncate d17/d20/d26/d48/f5c 307573 0 2024-02-28T23:07:33.663 INFO:tasks.workunit.client.1.smithi084.stdout:5/417: mkdir dc/de/df/d16/d34/d54/d5f/d9b 0 2024-02-28T23:07:33.663 INFO:tasks.workunit.client.0.smithi057.stdout:1/210: rename d2/d7/df/db/d12/d19/l29 to d2/d7/df/l45 0 2024-02-28T23:07:33.664 INFO:tasks.workunit.client.1.smithi084.stdout:8/360: creat d0/d9/d1b/d38/d43/d51/d7c/f80 x:0 0 0 2024-02-28T23:07:33.664 INFO:tasks.workunit.client.1.smithi084.stdout:4/352: symlink d1/d18/d20/d14/l78 0 2024-02-28T23:07:33.666 INFO:tasks.workunit.client.0.smithi057.stdout:8/283: creat d4/d1e/d59/f5f x:0 0 0 2024-02-28T23:07:33.667 INFO:tasks.workunit.client.0.smithi057.stdout:9/307: dread dd/de/d14/f1f [0,4194304] 0 2024-02-28T23:07:33.667 INFO:tasks.workunit.client.1.smithi084.stdout:8/361: creat d0/d24/d4d/f81 x:0 0 0 2024-02-28T23:07:33.668 INFO:tasks.workunit.client.0.smithi057.stdout:0/262: getdents d17/d20/d23/d37 0 2024-02-28T23:07:33.668 INFO:tasks.workunit.client.1.smithi084.stdout:4/353: symlink d1/d18/d20/d24/l79 0 2024-02-28T23:07:33.669 INFO:tasks.workunit.client.0.smithi057.stdout:0/263: stat d17/d20/d26/l35 0 2024-02-28T23:07:33.670 INFO:tasks.workunit.client.1.smithi084.stdout:5/418: creat dc/de/df/d16/d34/d54/d5f/d9b/f9c x:0 0 0 2024-02-28T23:07:33.670 INFO:tasks.workunit.client.0.smithi057.stdout:1/211: creat d2/d7/df/db/d12/d19/d24/d2d/f46 x:0 0 0 2024-02-28T23:07:33.671 INFO:tasks.workunit.client.0.smithi057.stdout:0/264: dread - d17/d20/d23/d47/f5b zero size 2024-02-28T23:07:33.672 INFO:tasks.workunit.client.1.smithi084.stdout:5/419: creat dc/d51/f9d x:0 0 0 2024-02-28T23:07:33.673 INFO:tasks.workunit.client.0.smithi057.stdout:1/212: mkdir d2/d7/df/db/d12/d3f/d47 0 2024-02-28T23:07:33.674 INFO:tasks.workunit.client.0.smithi057.stdout:0/265: creat d17/d20/d4a/f68 x:0 0 0 2024-02-28T23:07:33.675 INFO:tasks.workunit.client.1.smithi084.stdout:5/420: mkdir dc/de/d9e 0 2024-02-28T23:07:33.675 INFO:tasks.workunit.client.1.smithi084.stdout:3/390: dwrite d3/d8/f93 [0,4194304] 0 2024-02-28T23:07:33.676 INFO:tasks.workunit.client.0.smithi057.stdout:0/266: write d17/d20/d26/d30/f39 [6962122,49092] 0 2024-02-28T23:07:33.677 INFO:tasks.workunit.client.1.smithi084.stdout:5/421: dread - dc/de/df/d16/f81 zero size 2024-02-28T23:07:33.678 INFO:tasks.workunit.client.0.smithi057.stdout:1/213: mkdir d2/d7/df/db/d12/d19/d48 0 2024-02-28T23:07:33.678 INFO:tasks.workunit.client.0.smithi057.stdout:0/267: chown d17/f58 6687206 1 2024-02-28T23:07:33.680 INFO:tasks.workunit.client.0.smithi057.stdout:0/268: readlink d17/d20/d4a/l57 0 2024-02-28T23:07:33.680 INFO:tasks.workunit.client.1.smithi084.stdout:3/391: getdents d3/d8/d14 0 2024-02-28T23:07:33.681 INFO:tasks.workunit.client.0.smithi057.stdout:8/284: dwrite d4/f11 [0,4194304] 0 2024-02-28T23:07:33.682 INFO:tasks.workunit.client.1.smithi084.stdout:3/392: mkdir d3/d50/d97 0 2024-02-28T23:07:33.683 INFO:tasks.workunit.client.0.smithi057.stdout:0/269: creat d17/d20/d26/f69 x:0 0 0 2024-02-28T23:07:33.684 INFO:tasks.workunit.client.1.smithi084.stdout:3/393: mkdir d3/d8/d1a/d29/d41/d60/d98 0 2024-02-28T23:07:33.686 INFO:tasks.workunit.client.0.smithi057.stdout:8/285: creat d4/d13/d2c/d4e/f60 x:0 0 0 2024-02-28T23:07:33.687 INFO:tasks.workunit.client.1.smithi084.stdout:3/394: truncate d3/d8/d1a/d29/d41/f71 666473 0 2024-02-28T23:07:33.688 INFO:tasks.workunit.client.0.smithi057.stdout:0/270: rename la to d17/d20/d26/d48/l6a 0 2024-02-28T23:07:33.689 INFO:tasks.workunit.client.0.smithi057.stdout:8/286: dread - d4/d1e/d37/d3f/d42/f5b zero size 2024-02-28T23:07:33.689 INFO:tasks.workunit.client.0.smithi057.stdout:0/271: chown d17/d20/d23/d47/d60 0 1 2024-02-28T23:07:33.691 INFO:tasks.workunit.client.0.smithi057.stdout:0/272: dread - d17/d20/d23/f2c zero size 2024-02-28T23:07:33.692 INFO:tasks.workunit.client.0.smithi057.stdout:8/287: mkdir d4/d1e/d37/d3f/d54/d61 0 2024-02-28T23:07:33.696 INFO:tasks.workunit.client.0.smithi057.stdout:8/288: mkdir d4/d13/d18/d62 0 2024-02-28T23:07:33.698 INFO:tasks.workunit.client.0.smithi057.stdout:8/289: mknod d4/d13/d18/d62/c63 0 2024-02-28T23:07:33.702 INFO:tasks.workunit.client.0.smithi057.stdout:8/290: link d4/d13/c30 d4/d13/d2c/c64 0 2024-02-28T23:07:33.704 INFO:tasks.workunit.client.0.smithi057.stdout:8/291: creat d4/d13/d18/f65 x:0 0 0 2024-02-28T23:07:33.708 INFO:tasks.workunit.client.0.smithi057.stdout:8/292: dread d4/f5 [0,4194304] 0 2024-02-28T23:07:33.711 INFO:tasks.workunit.client.0.smithi057.stdout:8/293: mkdir d4/d13/d18/d62/d66 0 2024-02-28T23:07:33.714 INFO:tasks.workunit.client.0.smithi057.stdout:8/294: mknod d4/c67 0 2024-02-28T23:07:33.716 INFO:tasks.workunit.client.0.smithi057.stdout:8/295: chown d4/d13/c15 915 1 2024-02-28T23:07:33.738 INFO:tasks.workunit.client.1.smithi084.stdout:5/422: read dc/de/df/d16/d34/d54/f61 [270221,68437] 0 2024-02-28T23:07:33.741 INFO:tasks.workunit.client.1.smithi084.stdout:5/423: write dc/de/df/d31/f43 [92458,96015] 0 2024-02-28T23:07:33.746 INFO:tasks.workunit.client.1.smithi084.stdout:5/424: write dc/de/f2d [4178063,58611] 0 2024-02-28T23:07:33.753 INFO:tasks.workunit.client.1.smithi084.stdout:8/362: symlink d0/d9/d63/l82 0 2024-02-28T23:07:33.758 INFO:tasks.workunit.client.1.smithi084.stdout:8/363: mkdir d0/d24/d83 0 2024-02-28T23:07:33.768 INFO:tasks.workunit.client.1.smithi084.stdout:6/363: dwrite d1/d13/d47/f4b [0,4194304] 0 2024-02-28T23:07:33.768 INFO:tasks.workunit.client.1.smithi084.stdout:9/431: sync 2024-02-28T23:07:33.770 INFO:tasks.workunit.client.1.smithi084.stdout:9/432: dread d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:33.772 INFO:tasks.workunit.client.0.smithi057.stdout:6/311: write d0/d4/fa [566041,99119] 0 2024-02-28T23:07:33.772 INFO:tasks.workunit.client.1.smithi084.stdout:5/425: fdatasync dc/de/df/d31/f43 0 2024-02-28T23:07:33.773 INFO:tasks.workunit.client.1.smithi084.stdout:9/433: chown d4/d6/d15/d75/l4f 379 1 2024-02-28T23:07:33.774 INFO:tasks.workunit.client.0.smithi057.stdout:5/239: dread d4/d7/f1e [0,4194304] 0 2024-02-28T23:07:33.775 INFO:tasks.workunit.client.1.smithi084.stdout:6/364: unlink d1/d9/d1d/d1e/d33/f5e 0 2024-02-28T23:07:33.775 INFO:tasks.workunit.client.1.smithi084.stdout:9/434: chown d4/c84 2775 1 2024-02-28T23:07:33.780 INFO:tasks.workunit.client.1.smithi084.stdout:9/435: write d4/d6/d15/d12/d41/f7e [907949,410] 0 2024-02-28T23:07:33.784 INFO:tasks.workunit.client.0.smithi057.stdout:2/285: truncate d8/f42 222746 0 2024-02-28T23:07:33.784 INFO:tasks.workunit.client.0.smithi057.stdout:9/308: fdatasync dd/de/d14/f5a 0 2024-02-28T23:07:33.788 INFO:tasks.workunit.client.0.smithi057.stdout:2/286: symlink d8/d11/d15/l63 0 2024-02-28T23:07:33.789 INFO:tasks.workunit.client.1.smithi084.stdout:9/436: rmdir d4 39 2024-02-28T23:07:33.794 INFO:tasks.workunit.client.0.smithi057.stdout:2/287: creat d8/f64 x:0 0 0 2024-02-28T23:07:33.800 INFO:tasks.workunit.client.0.smithi057.stdout:1/214: truncate d2/d7/df/db/f20 3178980 0 2024-02-28T23:07:33.802 INFO:tasks.workunit.client.0.smithi057.stdout:3/203: dwrite d3/d4/f30 [4194304,4194304] 0 2024-02-28T23:07:33.802 INFO:tasks.workunit.client.1.smithi084.stdout:6/365: dwrite d1/d9/d1d/d1e/d33/d38/f54 [0,4194304] 0 2024-02-28T23:07:33.804 INFO:tasks.workunit.client.0.smithi057.stdout:5/240: dwrite f2 [4194304,4194304] 0 2024-02-28T23:07:33.804 INFO:tasks.workunit.client.0.smithi057.stdout:6/312: dwrite d0/d4/d25/f52 [0,4194304] 0 2024-02-28T23:07:33.805 INFO:tasks.workunit.client.1.smithi084.stdout:5/426: link dc/de/df/d16/d34/c49 dc/de/c9f 0 2024-02-28T23:07:33.808 INFO:tasks.workunit.client.1.smithi084.stdout:3/395: truncate d3/f11 4054699 0 2024-02-28T23:07:33.809 INFO:tasks.workunit.client.0.smithi057.stdout:6/313: read d0/f1c [1140110,90097] 0 2024-02-28T23:07:33.813 INFO:tasks.workunit.client.1.smithi084.stdout:5/427: dread - dc/d51/f9d zero size 2024-02-28T23:07:33.816 INFO:tasks.workunit.client.1.smithi084.stdout:5/428: truncate dc/de/f14 4515999 0 2024-02-28T23:07:33.823 INFO:tasks.workunit.client.1.smithi084.stdout:3/396: dread d3/d8/d1a/d29/d41/f64 [0,4194304] 0 2024-02-28T23:07:33.824 INFO:tasks.workunit.client.0.smithi057.stdout:5/241: link d4/d32/f41 d4/d9/d14/f43 0 2024-02-28T23:07:33.828 INFO:tasks.workunit.client.0.smithi057.stdout:6/314: truncate d0/d4/ff 54379 0 2024-02-28T23:07:33.829 INFO:tasks.workunit.client.1.smithi084.stdout:6/366: rename d1/d9/d1d/d1e/d33/d38/f4e to d1/d13/d2e/d41/d2c/f78 0 2024-02-28T23:07:33.829 INFO:tasks.workunit.client.0.smithi057.stdout:5/242: creat d4/d21/d24/f44 x:0 0 0 2024-02-28T23:07:33.831 INFO:tasks.workunit.client.1.smithi084.stdout:8/364: dread d0/d24/d4d/f68 [0,4194304] 0 2024-02-28T23:07:33.837 INFO:tasks.workunit.client.0.smithi057.stdout:6/315: rename d0/d1b/d3e/d3d/d42/d4f/d53/d57 to d0/d4/d25/d60/d6a 0 2024-02-28T23:07:33.844 INFO:tasks.workunit.client.1.smithi084.stdout:4/354: link d1/d18/d20/cf d1/d18/d20/d24/d2f/d3e/d52/d45/c7a 0 2024-02-28T23:07:33.849 INFO:tasks.workunit.client.1.smithi084.stdout:3/397: mknod d3/d8/d1a/d21/d76/c99 0 2024-02-28T23:07:33.851 INFO:tasks.workunit.client.1.smithi084.stdout:4/355: creat d1/d4b/f7b x:0 0 0 2024-02-28T23:07:33.855 INFO:tasks.workunit.client.0.smithi057.stdout:9/309: dread dd/de/d14/f18 [0,4194304] 0 2024-02-28T23:07:33.855 INFO:tasks.workunit.client.0.smithi057.stdout:3/204: dread d3/d4/d1d/f2e [0,4194304] 0 2024-02-28T23:07:33.855 INFO:tasks.workunit.client.1.smithi084.stdout:3/398: dread - d3/d8/d1a/d21/d7b/f88 zero size 2024-02-28T23:07:33.855 INFO:tasks.workunit.client.1.smithi084.stdout:4/356: write d1/d18/f42 [3331072,23846] 0 2024-02-28T23:07:33.855 INFO:tasks.workunit.client.1.smithi084.stdout:9/437: dwrite d4/d6/f7 [4194304,4194304] 0 2024-02-28T23:07:33.863 INFO:tasks.workunit.client.0.smithi057.stdout:2/288: dwrite d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:33.866 INFO:tasks.workunit.client.1.smithi084.stdout:3/399: mknod d3/d50/d5a/c9a 0 2024-02-28T23:07:33.866 INFO:tasks.workunit.client.0.smithi057.stdout:6/316: truncate d0/d4/d11/f2d 2958534 0 2024-02-28T23:07:33.867 INFO:tasks.workunit.client.0.smithi057.stdout:9/310: mkdir dd/d1b/d40/d44/d5c 0 2024-02-28T23:07:33.868 INFO:tasks.workunit.client.0.smithi057.stdout:2/289: write d8/d11/d15/f20 [842827,63641] 0 2024-02-28T23:07:33.869 INFO:tasks.workunit.client.1.smithi084.stdout:5/429: write dc/f2a [1641270,129056] 0 2024-02-28T23:07:33.870 INFO:tasks.workunit.client.1.smithi084.stdout:8/365: getdents d0/d9/d63 0 2024-02-28T23:07:33.873 INFO:tasks.workunit.client.0.smithi057.stdout:6/317: creat d0/d17/d20/f6b x:0 0 0 2024-02-28T23:07:33.875 INFO:tasks.workunit.client.0.smithi057.stdout:2/290: mknod d8/d11/d39/c65 0 2024-02-28T23:07:33.875 INFO:tasks.workunit.client.1.smithi084.stdout:9/438: mknod d4/d32/c8a 0 2024-02-28T23:07:33.877 INFO:tasks.workunit.client.1.smithi084.stdout:6/367: rename d1/d9/d1d/d1e/d33/d5c/c5f to d1/d9/d1d/d1e/d33/c79 0 2024-02-28T23:07:33.878 INFO:tasks.workunit.client.1.smithi084.stdout:5/430: truncate dc/de/df/d31/f7e 599910 0 2024-02-28T23:07:33.883 INFO:tasks.workunit.client.0.smithi057.stdout:9/311: rename dd/de/d4a/l56 to dd/de/l5d 0 2024-02-28T23:07:33.883 INFO:tasks.workunit.client.0.smithi057.stdout:1/215: dwrite d2/d7/df/db/f20 [4194304,4194304] 0 2024-02-28T23:07:33.883 INFO:tasks.workunit.client.0.smithi057.stdout:3/205: dwrite d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:33.883 INFO:tasks.workunit.client.0.smithi057.stdout:6/318: write d0/d4/d11/f13 [3032038,55184] 0 2024-02-28T23:07:33.887 INFO:tasks.workunit.client.0.smithi057.stdout:2/291: read d8/d11/f32 [636819,11575] 0 2024-02-28T23:07:33.890 INFO:tasks.workunit.client.0.smithi057.stdout:3/206: write d3/d4/d1d/f1e [1637667,15941] 0 2024-02-28T23:07:33.892 INFO:tasks.workunit.client.0.smithi057.stdout:9/312: symlink dd/de/l5e 0 2024-02-28T23:07:33.893 INFO:tasks.workunit.client.0.smithi057.stdout:9/313: fsync dd/de/d14/f1f 0 2024-02-28T23:07:33.895 INFO:tasks.workunit.client.1.smithi084.stdout:4/357: link d1/d18/d20/d65/l71 d1/d18/d20/d14/d30/l7c 0 2024-02-28T23:07:33.896 INFO:tasks.workunit.client.1.smithi084.stdout:6/368: read d1/d9/f3b [3269571,48501] 0 2024-02-28T23:07:33.896 INFO:tasks.workunit.client.0.smithi057.stdout:6/319: rmdir d0/d4/d25/d60/d6a 39 2024-02-28T23:07:33.899 INFO:tasks.workunit.client.0.smithi057.stdout:3/207: mkdir d3/d4/d1d/d47 0 2024-02-28T23:07:33.900 INFO:tasks.workunit.client.1.smithi084.stdout:6/369: write d1/d9/d1d/f61 [27163,20289] 0 2024-02-28T23:07:33.904 INFO:tasks.workunit.client.0.smithi057.stdout:1/216: rename d2/f15 to d2/d7/df/db/d12/d19/d24/d2d/f49 0 2024-02-28T23:07:33.906 INFO:tasks.workunit.client.1.smithi084.stdout:7/376: sync 2024-02-28T23:07:33.908 INFO:tasks.workunit.client.1.smithi084.stdout:5/431: creat dc/de/df/d16/fa0 x:0 0 0 2024-02-28T23:07:33.909 INFO:tasks.workunit.client.0.smithi057.stdout:1/217: dread - d2/d7/df/db/d12/d19/d24/d2d/f46 zero size 2024-02-28T23:07:33.910 INFO:tasks.workunit.client.0.smithi057.stdout:6/320: getdents d0/d1b/d3e/d3d 0 2024-02-28T23:07:33.910 INFO:tasks.workunit.client.1.smithi084.stdout:4/358: mknod d1/d18/d20/d24/d2f/d3e/d52/d37/c7d 0 2024-02-28T23:07:33.910 INFO:tasks.workunit.client.0.smithi057.stdout:2/292: dread d8/d11/d21/f36 [0,4194304] 0 2024-02-28T23:07:33.911 INFO:tasks.workunit.client.1.smithi084.stdout:8/366: dwrite d0/d24/d4d/f68 [0,4194304] 0 2024-02-28T23:07:33.911 INFO:tasks.workunit.client.1.smithi084.stdout:7/377: write db/d10/f56 [1163396,114197] 0 2024-02-28T23:07:33.914 INFO:tasks.workunit.client.1.smithi084.stdout:4/359: chown d1/f6 3 1 2024-02-28T23:07:33.916 INFO:tasks.workunit.client.0.smithi057.stdout:9/314: dwrite dd/de/d14/f29 [0,4194304] 0 2024-02-28T23:07:33.916 INFO:tasks.workunit.client.0.smithi057.stdout:2/293: unlink d8/d11/d21/d28/l3f 0 2024-02-28T23:07:33.917 INFO:tasks.workunit.client.1.smithi084.stdout:5/432: write dc/f6b [4935847,34527] 0 2024-02-28T23:07:33.918 INFO:tasks.workunit.client.0.smithi057.stdout:1/218: creat d2/d7/df/db/d12/d19/d24/f4a x:0 0 0 2024-02-28T23:07:33.919 INFO:tasks.workunit.client.0.smithi057.stdout:6/321: unlink d0/d4/d11/c28 0 2024-02-28T23:07:33.923 INFO:tasks.workunit.client.1.smithi084.stdout:8/367: mkdir d0/d9/d30/d33/d71/d84 0 2024-02-28T23:07:33.928 INFO:tasks.workunit.client.1.smithi084.stdout:6/370: dread d1/d13/d2e/d41/d4d/f63 [0,4194304] 0 2024-02-28T23:07:33.928 INFO:tasks.workunit.client.1.smithi084.stdout:9/439: dread d4/d6/d15/d12/d41/f64 [0,4194304] 0 2024-02-28T23:07:33.928 INFO:tasks.workunit.client.0.smithi057.stdout:9/315: write dd/de/d14/f1f [1284083,11396] 0 2024-02-28T23:07:33.928 INFO:tasks.workunit.client.0.smithi057.stdout:1/219: creat d2/d7/df/db/d12/d3f/f4b x:0 0 0 2024-02-28T23:07:33.930 INFO:tasks.workunit.client.0.smithi057.stdout:2/294: rename d8/f54 to d8/d11/d48/d50/f66 0 2024-02-28T23:07:33.931 INFO:tasks.workunit.client.0.smithi057.stdout:3/208: dwrite d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:33.934 INFO:tasks.workunit.client.1.smithi084.stdout:8/368: unlink d0/d9/d30/f42 0 2024-02-28T23:07:33.935 INFO:tasks.workunit.client.0.smithi057.stdout:2/295: write d8/d11/d21/f36 [3485359,126032] 0 2024-02-28T23:07:33.936 INFO:tasks.workunit.client.0.smithi057.stdout:1/220: creat d2/d7/df/db/d12/d19/d24/d2d/f4c x:0 0 0 2024-02-28T23:07:33.938 INFO:tasks.workunit.client.0.smithi057.stdout:3/209: creat d3/d20/d3f/f48 x:0 0 0 2024-02-28T23:07:33.940 INFO:tasks.workunit.client.0.smithi057.stdout:2/296: creat d8/d11/d21/f67 x:0 0 0 2024-02-28T23:07:33.941 INFO:tasks.workunit.client.1.smithi084.stdout:9/440: mknod d4/d6/d15/d12/d41/c8b 0 2024-02-28T23:07:33.941 INFO:tasks.workunit.client.1.smithi084.stdout:8/369: write d0/f53 [176086,6548] 0 2024-02-28T23:07:33.942 INFO:tasks.workunit.client.0.smithi057.stdout:3/210: write d3/d4/d1d/f1e [3173625,92089] 0 2024-02-28T23:07:33.943 INFO:tasks.workunit.client.0.smithi057.stdout:6/322: dread d0/d1b/d3e/d40/f4d [0,4194304] 0 2024-02-28T23:07:33.945 INFO:tasks.workunit.client.0.smithi057.stdout:1/221: rename d2/d7/df/db/d12/d19/d24/f42 to d2/d7/df/f4d 0 2024-02-28T23:07:33.946 INFO:tasks.workunit.client.0.smithi057.stdout:3/211: rmdir d3 39 2024-02-28T23:07:33.947 INFO:tasks.workunit.client.1.smithi084.stdout:6/371: unlink d1/d13/d2e/c68 0 2024-02-28T23:07:33.947 INFO:tasks.workunit.client.1.smithi084.stdout:4/360: dwrite d1/d18/f42 [4194304,4194304] 0 2024-02-28T23:07:33.947 INFO:tasks.workunit.client.1.smithi084.stdout:9/441: read - d4/d6/d15/d12/d41/f65 zero size 2024-02-28T23:07:33.947 INFO:tasks.workunit.client.0.smithi057.stdout:6/323: truncate d0/d4/d25/f2a 1901657 0 2024-02-28T23:07:33.948 INFO:tasks.workunit.client.0.smithi057.stdout:2/297: unlink d8/d11/d21/d28/d49/f4e 0 2024-02-28T23:07:33.950 INFO:tasks.workunit.client.0.smithi057.stdout:6/324: chown d0/d1b/d54/f65 820 1 2024-02-28T23:07:33.950 INFO:tasks.workunit.client.0.smithi057.stdout:1/222: chown d2/c32 0 1 2024-02-28T23:07:33.951 INFO:tasks.workunit.client.0.smithi057.stdout:2/298: read d8/d11/d21/f33 [503941,128691] 0 2024-02-28T23:07:33.952 INFO:tasks.workunit.client.1.smithi084.stdout:3/400: dwrite d3/d8/d1a/d21/d3d/f3c [0,4194304] 0 2024-02-28T23:07:33.953 INFO:tasks.workunit.client.0.smithi057.stdout:3/212: readlink d3/l46 0 2024-02-28T23:07:33.953 INFO:tasks.workunit.client.1.smithi084.stdout:5/433: getdents dc/d51 0 2024-02-28T23:07:33.956 INFO:tasks.workunit.client.0.smithi057.stdout:2/299: creat d8/d11/d21/f68 x:0 0 0 2024-02-28T23:07:33.958 INFO:tasks.workunit.client.0.smithi057.stdout:1/223: symlink d2/d7/df/db/d12/d3f/l4e 0 2024-02-28T23:07:33.958 INFO:tasks.workunit.client.1.smithi084.stdout:6/372: creat d1/d13/d47/f7a x:0 0 0 2024-02-28T23:07:33.959 INFO:tasks.workunit.client.0.smithi057.stdout:2/300: symlink d8/d11/d56/l69 0 2024-02-28T23:07:33.960 INFO:tasks.workunit.client.1.smithi084.stdout:9/442: mknod d4/d6/d15/d75/d2d/c8c 0 2024-02-28T23:07:33.961 INFO:tasks.workunit.client.1.smithi084.stdout:6/373: dread - d1/d13/d2e/d41/d4d/f72 zero size 2024-02-28T23:07:33.963 INFO:tasks.workunit.client.1.smithi084.stdout:9/443: write d4/d32/f5c [14374,88624] 0 2024-02-28T23:07:33.963 INFO:tasks.workunit.client.1.smithi084.stdout:4/361: mknod d1/d18/d20/d24/d2f/d3e/d52/d45/c7e 0 2024-02-28T23:07:33.963 INFO:tasks.workunit.client.1.smithi084.stdout:6/374: fsync d1/d9/f4f 0 2024-02-28T23:07:33.965 INFO:tasks.workunit.client.0.smithi057.stdout:4/276: sync 2024-02-28T23:07:33.965 INFO:tasks.workunit.client.0.smithi057.stdout:7/286: sync 2024-02-28T23:07:33.968 INFO:tasks.workunit.client.1.smithi084.stdout:6/375: chown d1/d13/d2e/d41/d4d/d6c/f77 181806675 1 2024-02-28T23:07:33.969 INFO:tasks.workunit.client.0.smithi057.stdout:3/213: dread d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:33.970 INFO:tasks.workunit.client.0.smithi057.stdout:4/277: write d1a/d1b/f1f [1789941,96122] 0 2024-02-28T23:07:33.970 INFO:tasks.workunit.client.0.smithi057.stdout:6/325: dread d0/d4/d25/f52 [0,4194304] 0 2024-02-28T23:07:33.970 INFO:tasks.workunit.client.0.smithi057.stdout:4/278: stat f14 0 2024-02-28T23:07:33.973 INFO:tasks.workunit.client.0.smithi057.stdout:3/214: creat d3/d4/d1d/f49 x:0 0 0 2024-02-28T23:07:33.973 INFO:tasks.workunit.client.0.smithi057.stdout:6/326: rmdir d0/d17 39 2024-02-28T23:07:33.974 INFO:tasks.workunit.client.1.smithi084.stdout:8/370: rmdir d0/d9/d30/d33 39 2024-02-28T23:07:33.974 INFO:tasks.workunit.client.1.smithi084.stdout:9/444: chown d4/cb 281626 1 2024-02-28T23:07:33.974 INFO:tasks.workunit.client.0.smithi057.stdout:4/279: write d1a/d1b/f32 [3924895,58836] 0 2024-02-28T23:07:33.976 INFO:tasks.workunit.client.0.smithi057.stdout:6/327: fdatasync d0/d1b/f27 0 2024-02-28T23:07:33.978 INFO:tasks.workunit.client.0.smithi057.stdout:4/280: fsync d1a/d1b/d2d/f39 0 2024-02-28T23:07:33.980 INFO:tasks.workunit.client.0.smithi057.stdout:9/316: dread dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:33.981 INFO:tasks.workunit.client.0.smithi057.stdout:4/281: write d1a/d1b/f5c [16296,59143] 0 2024-02-28T23:07:33.982 INFO:tasks.workunit.client.0.smithi057.stdout:6/328: truncate d0/d4/d25/f2f 4590600 0 2024-02-28T23:07:33.983 INFO:tasks.workunit.client.0.smithi057.stdout:9/317: dread - dd/de/d14/f55 zero size 2024-02-28T23:07:33.986 INFO:tasks.workunit.client.1.smithi084.stdout:6/376: rename d1/d9/l5d to d1/d13/d2e/l7b 0 2024-02-28T23:07:33.987 INFO:tasks.workunit.client.0.smithi057.stdout:9/318: chown dd/de/d14/d48/d4b/d41/l52 196 1 2024-02-28T23:07:33.990 INFO:tasks.workunit.client.0.smithi057.stdout:9/319: symlink dd/d1b/l5f 0 2024-02-28T23:07:33.993 INFO:tasks.workunit.client.0.smithi057.stdout:9/320: mkdir dd/de/d4a/d60 0 2024-02-28T23:07:33.996 INFO:tasks.workunit.client.1.smithi084.stdout:6/377: rmdir d1/d13/d47/d69 0 2024-02-28T23:07:33.996 INFO:tasks.workunit.client.0.smithi057.stdout:9/321: rename dd/de/d14/f29 to dd/de/d14/f61 0 2024-02-28T23:07:34.000 INFO:tasks.workunit.client.0.smithi057.stdout:9/322: creat dd/d1b/d40/d44/f62 x:0 0 0 2024-02-28T23:07:34.001 INFO:tasks.workunit.client.0.smithi057.stdout:9/323: fsync dd/f12 0 2024-02-28T23:07:34.003 INFO:tasks.workunit.client.1.smithi084.stdout:9/445: dwrite d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:34.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/378: truncate d1/d13/d2e/d41/f2f 4328638 0 2024-02-28T23:07:34.004 INFO:tasks.workunit.client.0.smithi057.stdout:9/324: mkdir dd/de/d24/d63 0 2024-02-28T23:07:34.005 INFO:tasks.workunit.client.0.smithi057.stdout:5/243: sync 2024-02-28T23:07:34.005 INFO:tasks.workunit.client.0.smithi057.stdout:0/273: sync 2024-02-28T23:07:34.005 INFO:tasks.workunit.client.0.smithi057.stdout:8/296: sync 2024-02-28T23:07:34.006 INFO:tasks.workunit.client.1.smithi084.stdout:4/362: write d1/f1d [4487289,122901] 0 2024-02-28T23:07:34.007 INFO:tasks.workunit.client.0.smithi057.stdout:9/325: symlink dd/de/d14/d48/d4b/l64 0 2024-02-28T23:07:34.008 INFO:tasks.workunit.client.1.smithi084.stdout:9/446: rmdir d4 39 2024-02-28T23:07:34.009 INFO:tasks.workunit.client.0.smithi057.stdout:9/326: write dd/de/d14/f17 [478657,703] 0 2024-02-28T23:07:34.011 INFO:tasks.workunit.client.1.smithi084.stdout:4/363: stat d1/d18/d20/c23 0 2024-02-28T23:07:34.011 INFO:tasks.workunit.client.0.smithi057.stdout:0/274: mkdir d17/d20/d26/d30/d6b 0 2024-02-28T23:07:34.011 INFO:tasks.workunit.client.0.smithi057.stdout:9/327: readlink dd/de/d24/l36 0 2024-02-28T23:07:34.014 INFO:tasks.workunit.client.1.smithi084.stdout:4/364: dread - d1/d18/f5f zero size 2024-02-28T23:07:34.014 INFO:tasks.workunit.client.0.smithi057.stdout:5/244: mknod d4/d9/d14/d3b/c45 0 2024-02-28T23:07:34.016 INFO:tasks.workunit.client.1.smithi084.stdout:6/379: creat d1/d13/d2e/d41/d4d/d6c/f7c x:0 0 0 2024-02-28T23:07:34.017 INFO:tasks.workunit.client.0.smithi057.stdout:9/328: fdatasync dd/de/d14/f35 0 2024-02-28T23:07:34.021 INFO:tasks.workunit.client.1.smithi084.stdout:6/380: chown d1/d9/f28 151 1 2024-02-28T23:07:34.022 INFO:tasks.workunit.client.1.smithi084.stdout:3/401: dwrite d3/d8/d14/f49 [0,4194304] 0 2024-02-28T23:07:34.023 INFO:tasks.workunit.client.0.smithi057.stdout:0/275: dread d17/d20/d26/d48/f5c [0,4194304] 0 2024-02-28T23:07:34.029 INFO:tasks.workunit.client.0.smithi057.stdout:9/329: link l7 dd/d1b/d40/d44/d5c/l65 0 2024-02-28T23:07:34.029 INFO:tasks.workunit.client.1.smithi084.stdout:6/381: dread - d1/d13/d2e/d41/d4d/d6c/f77 zero size 2024-02-28T23:07:34.031 INFO:tasks.workunit.client.0.smithi057.stdout:5/245: dwrite d4/d9/d14/f28 [4194304,4194304] 0 2024-02-28T23:07:34.032 INFO:tasks.workunit.client.0.smithi057.stdout:2/301: getdents d8/d11/d21 0 2024-02-28T23:07:34.033 INFO:tasks.workunit.client.1.smithi084.stdout:4/365: unlink d1/l6d 0 2024-02-28T23:07:34.035 INFO:tasks.workunit.client.0.smithi057.stdout:9/330: write dd/d1b/d40/f59 [992124,78542] 0 2024-02-28T23:07:34.035 INFO:tasks.workunit.client.0.smithi057.stdout:5/246: fdatasync d4/d9/d16/f19 0 2024-02-28T23:07:34.038 INFO:tasks.workunit.client.0.smithi057.stdout:1/224: truncate d2/d7/df/f1d 1017000 0 2024-02-28T23:07:34.038 INFO:tasks.workunit.client.0.smithi057.stdout:5/247: chown d4/d21/l23 24909863 1 2024-02-28T23:07:34.038 INFO:tasks.workunit.client.0.smithi057.stdout:3/215: fsync d3/d4/f12 0 2024-02-28T23:07:34.039 INFO:tasks.workunit.client.1.smithi084.stdout:8/371: dwrite d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:34.039 INFO:tasks.workunit.client.0.smithi057.stdout:4/282: write d1a/d1e/f25 [2071273,43359] 0 2024-02-28T23:07:34.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/225: chown d2/d7/df/db/d3a/d2a 6 1 2024-02-28T23:07:34.046 INFO:tasks.workunit.client.1.smithi084.stdout:3/402: chown d3/l44 6944438 1 2024-02-28T23:07:34.047 INFO:tasks.workunit.client.0.smithi057.stdout:2/302: mknod d8/c6a 0 2024-02-28T23:07:34.047 INFO:tasks.workunit.client.1.smithi084.stdout:1/457: sync 2024-02-28T23:07:34.047 INFO:tasks.workunit.client.1.smithi084.stdout:0/413: sync 2024-02-28T23:07:34.048 INFO:tasks.workunit.client.1.smithi084.stdout:2/337: sync 2024-02-28T23:07:34.050 INFO:tasks.workunit.client.0.smithi057.stdout:2/303: chown d8/d11/d21/c61 2 1 2024-02-28T23:07:34.053 INFO:tasks.workunit.client.1.smithi084.stdout:3/403: chown d3/d8/d1a/d21/d3d/f67 37 1 2024-02-28T23:07:34.053 INFO:tasks.workunit.client.0.smithi057.stdout:7/287: dread d0/d5/d36/d4e/d15/f2b [0,4194304] 0 2024-02-28T23:07:34.054 INFO:tasks.workunit.client.1.smithi084.stdout:6/382: unlink d1/d13/d2e/d41/d4d/f72 0 2024-02-28T23:07:34.057 INFO:tasks.workunit.client.0.smithi057.stdout:2/304: chown d8/f64 4611 1 2024-02-28T23:07:34.057 INFO:tasks.workunit.client.0.smithi057.stdout:9/331: symlink dd/de/d14/d48/d4b/l66 0 2024-02-28T23:07:34.057 INFO:tasks.workunit.client.1.smithi084.stdout:4/366: chown d1/d18/d20/d24/d2f/d3e/d52/d45/c7a 15256501 1 2024-02-28T23:07:34.058 INFO:tasks.workunit.client.1.smithi084.stdout:8/372: rename d0/d9/d1b/l17 to d0/d9/d1b/d20/l85 0 2024-02-28T23:07:34.060 INFO:tasks.workunit.client.0.smithi057.stdout:1/226: mkdir d2/d7/d4f 0 2024-02-28T23:07:34.060 INFO:tasks.workunit.client.0.smithi057.stdout:4/283: rename d1a/d1e/f29 to d1a/d1e/d4b/d4f/f64 0 2024-02-28T23:07:34.062 INFO:tasks.workunit.client.0.smithi057.stdout:8/297: write d4/f5 [1059462,66333] 0 2024-02-28T23:07:34.064 INFO:tasks.workunit.client.1.smithi084.stdout:5/434: dread dc/de/df/d16/f45 [0,4194304] 0 2024-02-28T23:07:34.067 INFO:tasks.workunit.client.1.smithi084.stdout:2/338: creat d5/d7/d12/d24/d2a/d35/f5d x:0 0 0 2024-02-28T23:07:34.068 INFO:tasks.workunit.client.0.smithi057.stdout:1/227: stat d2/d7/df/db/d3a/f3b 0 2024-02-28T23:07:34.070 INFO:tasks.workunit.client.1.smithi084.stdout:5/435: readlink dc/de/df/d16/d34/d54/d5f/d66/l68 0 2024-02-28T23:07:34.070 INFO:tasks.workunit.client.1.smithi084.stdout:6/383: mknod d1/d9/d1d/d1e/d33/d38/c7d 0 2024-02-28T23:07:34.071 INFO:tasks.workunit.client.1.smithi084.stdout:6/384: stat d1/f31 0 2024-02-28T23:07:34.072 INFO:tasks.workunit.client.0.smithi057.stdout:2/305: chown d8/d11/d21/c61 330 1 2024-02-28T23:07:34.075 INFO:tasks.workunit.client.0.smithi057.stdout:7/288: creat d0/d5/d25/d3e/f5a x:0 0 0 2024-02-28T23:07:34.075 INFO:tasks.workunit.client.0.smithi057.stdout:4/284: symlink d1a/d1b/d24/l65 0 2024-02-28T23:07:34.075 INFO:tasks.workunit.client.0.smithi057.stdout:5/248: dwrite d4/d21/d24/f22 [0,4194304] 0 2024-02-28T23:07:34.076 INFO:tasks.workunit.client.1.smithi084.stdout:5/436: truncate dc/d51/d7c/f93 276694 0 2024-02-28T23:07:34.076 INFO:tasks.workunit.client.1.smithi084.stdout:4/367: symlink d1/d18/d20/d14/l7f 0 2024-02-28T23:07:34.077 INFO:tasks.workunit.client.1.smithi084.stdout:4/368: chown d1/d18/d20 1 1 2024-02-28T23:07:34.078 INFO:tasks.workunit.client.1.smithi084.stdout:9/447: link d4/d6/d15/d75/l77 d4/d6/d15/d12/d41/d55/l8d 0 2024-02-28T23:07:34.081 INFO:tasks.workunit.client.1.smithi084.stdout:6/385: mkdir d1/d13/d2e/d7e 0 2024-02-28T23:07:34.082 INFO:tasks.workunit.client.1.smithi084.stdout:4/369: write d1/d18/d20/d24/d2f/d3e/f60 [345208,26348] 0 2024-02-28T23:07:34.083 INFO:tasks.workunit.client.1.smithi084.stdout:9/448: fsync d4/d6/d15/d12/d41/f46 0 2024-02-28T23:07:34.084 INFO:tasks.workunit.client.0.smithi057.stdout:9/332: symlink dd/l67 0 2024-02-28T23:07:34.084 INFO:tasks.workunit.client.1.smithi084.stdout:8/373: unlink d0/d9/d1b/d20/l41 0 2024-02-28T23:07:34.086 INFO:tasks.workunit.client.1.smithi084.stdout:5/437: mknod dc/de/ca1 0 2024-02-28T23:07:34.087 INFO:tasks.workunit.client.0.smithi057.stdout:1/228: chown d2/d7/df/db/d12/d3f/f43 20951 1 2024-02-28T23:07:34.089 INFO:tasks.workunit.client.1.smithi084.stdout:3/404: getdents d3/d8/d1a/d21/d7b 0 2024-02-28T23:07:34.091 INFO:tasks.workunit.client.0.smithi057.stdout:9/333: chown dd/de/d14/f51 14456414 1 2024-02-28T23:07:34.092 INFO:tasks.workunit.client.0.smithi057.stdout:7/289: write d0/d5/d36/d4e/d15/f46 [1028711,86539] 0 2024-02-28T23:07:34.092 INFO:tasks.workunit.client.0.smithi057.stdout:4/285: chown d1a/d1b/d2d/d49/c60 1952 1 2024-02-28T23:07:34.093 INFO:tasks.workunit.client.1.smithi084.stdout:4/370: mknod d1/d18/d20/d14/d30/d47/c80 0 2024-02-28T23:07:34.098 INFO:tasks.workunit.client.1.smithi084.stdout:7/378: sync 2024-02-28T23:07:34.099 INFO:tasks.workunit.client.1.smithi084.stdout:5/438: mknod dc/de/df/d16/d34/d54/d5f/d9b/ca2 0 2024-02-28T23:07:34.099 INFO:tasks.workunit.client.0.smithi057.stdout:2/306: symlink d8/d11/d21/d28/d41/l6b 0 2024-02-28T23:07:34.100 INFO:tasks.workunit.client.0.smithi057.stdout:3/216: dread d3/d4/d7/f2b [0,4194304] 0 2024-02-28T23:07:34.101 INFO:tasks.workunit.client.0.smithi057.stdout:2/307: readlink d8/l2b 0 2024-02-28T23:07:34.101 INFO:tasks.workunit.client.1.smithi084.stdout:6/386: getdents d1/d13/d47 0 2024-02-28T23:07:34.103 INFO:tasks.workunit.client.1.smithi084.stdout:9/449: getdents d4 0 2024-02-28T23:07:34.105 INFO:tasks.workunit.client.1.smithi084.stdout:6/387: fdatasync d1/d13/d2e/f44 0 2024-02-28T23:07:34.107 INFO:tasks.workunit.client.0.smithi057.stdout:4/286: read fe [3714671,107506] 0 2024-02-28T23:07:34.107 INFO:tasks.workunit.client.0.smithi057.stdout:3/217: chown d3/d8/f45 3 1 2024-02-28T23:07:34.107 INFO:tasks.workunit.client.0.smithi057.stdout:0/276: dwrite d17/d19/f53 [0,4194304] 0 2024-02-28T23:07:34.108 INFO:tasks.workunit.client.1.smithi084.stdout:4/371: link d1/d18/d20/d14/d30/f53 d1/d18/d20/d14/d30/f81 0 2024-02-28T23:07:34.109 INFO:tasks.workunit.client.0.smithi057.stdout:8/298: dwrite d4/f17 [4194304,4194304] 0 2024-02-28T23:07:34.110 INFO:tasks.workunit.client.0.smithi057.stdout:4/287: fdatasync d1a/d1b/d2d/f59 0 2024-02-28T23:07:34.111 INFO:tasks.workunit.client.0.smithi057.stdout:5/249: truncate d4/d9/d16/f1a 1297027 0 2024-02-28T23:07:34.112 INFO:tasks.workunit.client.1.smithi084.stdout:4/372: dread - d1/d18/d20/d14/d30/f81 zero size 2024-02-28T23:07:34.113 INFO:tasks.workunit.client.1.smithi084.stdout:2/339: getdents d5/d7/d12/d24/d2a/d35 0 2024-02-28T23:07:34.114 INFO:tasks.workunit.client.0.smithi057.stdout:0/277: symlink d17/d20/d26/d30/l6c 0 2024-02-28T23:07:34.114 INFO:tasks.workunit.client.0.smithi057.stdout:4/288: chown d1a/d1b 510 1 2024-02-28T23:07:34.114 INFO:tasks.workunit.client.0.smithi057.stdout:5/250: chown d4/d32 82 1 2024-02-28T23:07:34.115 INFO:tasks.workunit.client.0.smithi057.stdout:9/334: dread dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:34.118 INFO:tasks.workunit.client.1.smithi084.stdout:4/373: write d1/d18/d20/d14/f4f [764252,35724] 0 2024-02-28T23:07:34.118 INFO:tasks.workunit.client.1.smithi084.stdout:2/340: dread - d5/d7/d12/f51 zero size 2024-02-28T23:07:34.121 INFO:tasks.workunit.client.0.smithi057.stdout:8/299: symlink d4/d1e/d59/l68 0 2024-02-28T23:07:34.123 INFO:tasks.workunit.client.1.smithi084.stdout:4/374: chown d1/d18/d20/l1f 7766848 1 2024-02-28T23:07:34.125 INFO:tasks.workunit.client.0.smithi057.stdout:4/289: write d1a/d1b/d24/f38 [5235917,81521] 0 2024-02-28T23:07:34.127 INFO:tasks.workunit.client.1.smithi084.stdout:0/414: dread d1/d8/d18/f65 [0,4194304] 0 2024-02-28T23:07:34.134 INFO:tasks.workunit.client.0.smithi057.stdout:7/290: dwrite d0/d5/d36/d4e/d15/f49 [0,4194304] 0 2024-02-28T23:07:34.136 INFO:tasks.workunit.client.1.smithi084.stdout:9/450: dwrite d4/d6/ff [4194304,4194304] 0 2024-02-28T23:07:34.138 INFO:tasks.workunit.client.0.smithi057.stdout:9/335: symlink dd/d46/l68 0 2024-02-28T23:07:34.138 INFO:tasks.workunit.client.0.smithi057.stdout:1/229: dwrite d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:34.138 INFO:tasks.workunit.client.0.smithi057.stdout:8/300: rename d4/d13/d18/f48 to d4/d13/d2c/f69 0 2024-02-28T23:07:34.139 INFO:tasks.workunit.client.1.smithi084.stdout:9/451: dread - d4/f79 zero size 2024-02-28T23:07:34.139 INFO:tasks.workunit.client.0.smithi057.stdout:3/218: truncate d3/d4/f12 4213189 0 2024-02-28T23:07:34.141 INFO:tasks.workunit.client.0.smithi057.stdout:4/290: mknod d1a/c66 0 2024-02-28T23:07:34.143 INFO:tasks.workunit.client.0.smithi057.stdout:8/301: readlink d4/d13/d1a/l3e 0 2024-02-28T23:07:34.143 INFO:tasks.workunit.client.0.smithi057.stdout:6/329: sync 2024-02-28T23:07:34.144 INFO:tasks.workunit.client.0.smithi057.stdout:9/336: creat dd/d1b/f69 x:0 0 0 2024-02-28T23:07:34.144 INFO:tasks.workunit.client.0.smithi057.stdout:7/291: chown d0/d5/d36/d4e/d15/d29/f40 1 1 2024-02-28T23:07:34.144 INFO:tasks.workunit.client.0.smithi057.stdout:2/308: dwrite d8/fe [0,4194304] 0 2024-02-28T23:07:34.145 INFO:tasks.workunit.client.1.smithi084.stdout:1/458: dwrite d12/d1b/d3b/f5d [0,4194304] 0 2024-02-28T23:07:34.148 INFO:tasks.workunit.client.0.smithi057.stdout:0/278: dwrite d17/d19/f2f [0,4194304] 0 2024-02-28T23:07:34.148 INFO:tasks.workunit.client.0.smithi057.stdout:6/330: chown d0/d4/d25/f2a 0 1 2024-02-28T23:07:34.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/292: creat d0/d5/d36/d4e/d3b/f5b x:0 0 0 2024-02-28T23:07:34.153 INFO:tasks.workunit.client.0.smithi057.stdout:3/219: creat d3/d8/d1b/f4a x:0 0 0 2024-02-28T23:07:34.153 INFO:tasks.workunit.client.1.smithi084.stdout:4/375: unlink d1/d18/d20/d24/l79 0 2024-02-28T23:07:34.154 INFO:tasks.workunit.client.1.smithi084.stdout:4/376: stat d1/f6e 0 2024-02-28T23:07:34.155 INFO:tasks.workunit.client.1.smithi084.stdout:8/374: dread d0/d9/f4c [0,4194304] 0 2024-02-28T23:07:34.159 INFO:tasks.workunit.client.1.smithi084.stdout:4/377: write d1/f1d [4651298,128756] 0 2024-02-28T23:07:34.159 INFO:tasks.workunit.client.1.smithi084.stdout:5/439: link dc/d51/l8a dc/de/df/d16/d34/d54/la3 0 2024-02-28T23:07:34.162 INFO:tasks.workunit.client.1.smithi084.stdout:6/388: creat d1/d9/d1d/d1e/d33/f7f x:0 0 0 2024-02-28T23:07:34.162 INFO:tasks.workunit.client.1.smithi084.stdout:9/452: readlink d4/d6/d15/d3c/l86 0 2024-02-28T23:07:34.173 INFO:tasks.workunit.client.1.smithi084.stdout:1/459: mkdir d12/d1b/d91/d9b 0 2024-02-28T23:07:34.174 INFO:tasks.workunit.client.1.smithi084.stdout:6/389: creat d1/d13/d50/f80 x:0 0 0 2024-02-28T23:07:34.178 INFO:tasks.workunit.client.1.smithi084.stdout:9/453: readlink d4/d6/d15/d12/d41/l50 0 2024-02-28T23:07:34.178 INFO:tasks.workunit.client.1.smithi084.stdout:0/415: link d1/d6/db/c43 d1/d6/d2d/c8c 0 2024-02-28T23:07:34.179 INFO:tasks.workunit.client.1.smithi084.stdout:6/390: write d1/d13/d50/f80 [673457,48100] 0 2024-02-28T23:07:34.184 INFO:tasks.workunit.client.1.smithi084.stdout:2/341: getdents d5/dd 0 2024-02-28T23:07:34.198 INFO:tasks.workunit.client.1.smithi084.stdout:3/405: dread d3/d8/f72 [0,4194304] 0 2024-02-28T23:07:34.202 INFO:tasks.workunit.client.1.smithi084.stdout:5/440: dwrite dc/de/df/d16/d34/d54/d5f/d66/f96 [0,4194304] 0 2024-02-28T23:07:34.203 INFO:tasks.workunit.client.1.smithi084.stdout:8/375: dwrite d0/d9/d1b/f39 [0,4194304] 0 2024-02-28T23:07:34.215 INFO:tasks.workunit.client.1.smithi084.stdout:7/379: dread db/d10/f2e [0,4194304] 0 2024-02-28T23:07:34.215 INFO:tasks.workunit.client.1.smithi084.stdout:0/416: dwrite d1/d6/db/d37/d72/d7b/f80 [0,4194304] 0 2024-02-28T23:07:34.219 INFO:tasks.workunit.client.1.smithi084.stdout:1/460: getdents d12/d84/d9a 0 2024-02-28T23:07:34.219 INFO:tasks.workunit.client.1.smithi084.stdout:2/342: creat d5/d7/d12/d24/f5e x:0 0 0 2024-02-28T23:07:34.226 INFO:tasks.workunit.client.1.smithi084.stdout:1/461: stat d12/d1b/d3b/c4c 0 2024-02-28T23:07:34.229 INFO:tasks.workunit.client.0.smithi057.stdout:9/337: mknod dd/d1b/d40/c6a 0 2024-02-28T23:07:34.230 INFO:tasks.workunit.client.0.smithi057.stdout:4/291: truncate d1a/f37 160018 0 2024-02-28T23:07:34.231 INFO:tasks.workunit.client.1.smithi084.stdout:0/417: read d1/d6/d59/f5a [110371,100900] 0 2024-02-28T23:07:34.231 INFO:tasks.workunit.client.1.smithi084.stdout:5/441: mkdir dc/de/df/d16/d34/d54/da4 0 2024-02-28T23:07:34.234 INFO:tasks.workunit.client.1.smithi084.stdout:1/462: write d12/d1b/d63/d7d/f85 [315144,54343] 0 2024-02-28T23:07:34.234 INFO:tasks.workunit.client.1.smithi084.stdout:7/380: write db/d11/d27/d36/f5a [4862,48062] 0 2024-02-28T23:07:34.237 INFO:tasks.workunit.client.0.smithi057.stdout:0/279: read d17/d20/d26/d30/f39 [1274834,118645] 0 2024-02-28T23:07:34.239 INFO:tasks.workunit.client.0.smithi057.stdout:4/292: write d1a/d1e/d4b/f5d [574790,28726] 0 2024-02-28T23:07:34.240 INFO:tasks.workunit.client.1.smithi084.stdout:0/418: write d1/d8/d69/d6b/f77 [710044,95963] 0 2024-02-28T23:07:34.241 INFO:tasks.workunit.client.1.smithi084.stdout:3/406: dwrite d3/d8/d1a/d21/d3d/f4e [0,4194304] 0 2024-02-28T23:07:34.243 INFO:tasks.workunit.client.1.smithi084.stdout:3/407: readlink d3/d8/d1a/d45/d5e/d7a/l91 0 2024-02-28T23:07:34.244 INFO:tasks.workunit.client.1.smithi084.stdout:0/419: chown d1/l12 2865 1 2024-02-28T23:07:34.244 INFO:tasks.workunit.client.0.smithi057.stdout:9/338: creat dd/de/f6b x:0 0 0 2024-02-28T23:07:34.244 INFO:tasks.workunit.client.0.smithi057.stdout:7/293: link d0/d5/f56 d0/d5/f5c 0 2024-02-28T23:07:34.245 INFO:tasks.workunit.client.0.smithi057.stdout:5/251: dwrite d4/f6 [4194304,4194304] 0 2024-02-28T23:07:34.245 INFO:tasks.workunit.client.0.smithi057.stdout:4/293: symlink d1a/d1e/d51/l67 0 2024-02-28T23:07:34.247 INFO:tasks.workunit.client.1.smithi084.stdout:4/378: truncate d1/d18/f32 118052 0 2024-02-28T23:07:34.248 INFO:tasks.workunit.client.0.smithi057.stdout:6/331: dread d0/d4/d25/f2a [0,4194304] 0 2024-02-28T23:07:34.248 INFO:tasks.workunit.client.0.smithi057.stdout:9/339: fdatasync dd/de/d14/f61 0 2024-02-28T23:07:34.250 INFO:tasks.workunit.client.0.smithi057.stdout:0/280: getdents d17/d20/d23/d4b 0 2024-02-28T23:07:34.251 INFO:tasks.workunit.client.0.smithi057.stdout:9/340: rmdir dd/de/d4a 39 2024-02-28T23:07:34.252 INFO:tasks.workunit.client.0.smithi057.stdout:0/281: read - d17/d20/f5d zero size 2024-02-28T23:07:34.254 INFO:tasks.workunit.client.0.smithi057.stdout:6/332: truncate d0/d4/d25/f26 218921 0 2024-02-28T23:07:34.255 INFO:tasks.workunit.client.0.smithi057.stdout:5/252: link d4/d9/d14/d3b/c45 d4/d9/d14/c46 0 2024-02-28T23:07:34.256 INFO:tasks.workunit.client.0.smithi057.stdout:9/341: write dd/de/d14/f61 [692670,56730] 0 2024-02-28T23:07:34.257 INFO:tasks.workunit.client.1.smithi084.stdout:5/442: symlink dc/de/df/d31/la5 0 2024-02-28T23:07:34.258 INFO:tasks.workunit.client.0.smithi057.stdout:0/282: mknod d17/d20/d26/d48/c6d 0 2024-02-28T23:07:34.258 INFO:tasks.workunit.client.0.smithi057.stdout:5/253: fsync d4/d21/f35 0 2024-02-28T23:07:34.259 INFO:tasks.workunit.client.0.smithi057.stdout:9/342: readlink dd/de/d24/l36 0 2024-02-28T23:07:34.260 INFO:tasks.workunit.client.1.smithi084.stdout:5/443: fsync dc/de/f79 0 2024-02-28T23:07:34.260 INFO:tasks.workunit.client.1.smithi084.stdout:8/376: link d0/d9/d1b/d20/f25 d0/d24/d78/f86 0 2024-02-28T23:07:34.261 INFO:tasks.workunit.client.0.smithi057.stdout:3/220: dwrite d3/d4/d7/f10 [0,4194304] 0 2024-02-28T23:07:34.263 INFO:tasks.workunit.client.0.smithi057.stdout:6/333: chown d0/d4/d25/d60/d6a/f69 13304927 1 2024-02-28T23:07:34.264 INFO:tasks.workunit.client.1.smithi084.stdout:8/377: write d0/d9/d1b/d38/d43/d51/f7b [849667,77506] 0 2024-02-28T23:07:34.264 INFO:tasks.workunit.client.1.smithi084.stdout:6/391: dwrite d1/d9/f3b [0,4194304] 0 2024-02-28T23:07:34.264 INFO:tasks.workunit.client.1.smithi084.stdout:9/454: getdents d4/d6/d53 0 2024-02-28T23:07:34.264 INFO:tasks.workunit.client.0.smithi057.stdout:0/283: mknod d17/d20/d23/d47/d60/c6e 0 2024-02-28T23:07:34.266 INFO:tasks.workunit.client.0.smithi057.stdout:5/254: creat d4/d9/d14/d3b/f47 x:0 0 0 2024-02-28T23:07:34.268 INFO:tasks.workunit.client.0.smithi057.stdout:0/284: write d17/d20/d23/f2c [928788,103979] 0 2024-02-28T23:07:34.268 INFO:tasks.workunit.client.0.smithi057.stdout:6/334: creat d0/f6c x:0 0 0 2024-02-28T23:07:34.269 INFO:tasks.workunit.client.0.smithi057.stdout:5/255: mkdir d4/d32/d48 0 2024-02-28T23:07:34.271 INFO:tasks.workunit.client.1.smithi084.stdout:2/343: creat d5/d7/d12/d24/d2a/d53/f5f x:0 0 0 2024-02-28T23:07:34.274 INFO:tasks.workunit.client.1.smithi084.stdout:3/408: creat d3/d8/d1a/d45/d5e/d7a/f9b x:0 0 0 2024-02-28T23:07:34.274 INFO:tasks.workunit.client.0.smithi057.stdout:6/335: write d0/d17/f45 [4139080,40776] 0 2024-02-28T23:07:34.275 INFO:tasks.workunit.client.0.smithi057.stdout:0/285: mkdir d17/d20/d23/d47/d60/d6f 0 2024-02-28T23:07:34.275 INFO:tasks.workunit.client.1.smithi084.stdout:4/379: fsync d1/d18/d20/f11 0 2024-02-28T23:07:34.277 INFO:tasks.workunit.client.1.smithi084.stdout:1/463: write d12/d1b/d2e/d32/f47 [621188,19750] 0 2024-02-28T23:07:34.278 INFO:tasks.workunit.client.1.smithi084.stdout:5/444: fsync dc/de/df/d16/d34/f53 0 2024-02-28T23:07:34.279 INFO:tasks.workunit.client.1.smithi084.stdout:4/380: fsync d1/f7 0 2024-02-28T23:07:34.282 INFO:tasks.workunit.client.0.smithi057.stdout:3/221: dwrite d3/d8/d1b/f37 [0,4194304] 0 2024-02-28T23:07:34.283 INFO:tasks.workunit.client.0.smithi057.stdout:9/343: dwrite dd/d1b/d40/d44/f62 [0,4194304] 0 2024-02-28T23:07:34.284 INFO:tasks.workunit.client.1.smithi084.stdout:6/392: rename d1/d13/d2e/d7e to d1/d13/d50/d74/d81 0 2024-02-28T23:07:34.285 INFO:tasks.workunit.client.1.smithi084.stdout:4/381: write d1/d4a/f54 [731830,108246] 0 2024-02-28T23:07:34.286 INFO:tasks.workunit.client.0.smithi057.stdout:3/222: stat d3/d4/c1a 0 2024-02-28T23:07:34.287 INFO:tasks.workunit.client.0.smithi057.stdout:6/336: link d0/d1b/d54/l68 d0/d1b/d3e/d3d/d42/d4f/l6d 0 2024-02-28T23:07:34.288 INFO:tasks.workunit.client.1.smithi084.stdout:7/381: creat db/d10/f71 x:0 0 0 2024-02-28T23:07:34.289 INFO:tasks.workunit.client.0.smithi057.stdout:6/337: stat d0/d1b/d3e/d40/f4d 0 2024-02-28T23:07:34.290 INFO:tasks.workunit.client.1.smithi084.stdout:3/409: truncate d3/d8/d1a/d29/d41/d5b/f83 435287 0 2024-02-28T23:07:34.290 INFO:tasks.workunit.client.0.smithi057.stdout:3/223: write d3/d20/f24 [319520,96876] 0 2024-02-28T23:07:34.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/378: dwrite d0/d9/d1b/f74 [0,4194304] 0 2024-02-28T23:07:34.298 INFO:tasks.workunit.client.0.smithi057.stdout:9/344: getdents dd/d1b/d40/d44 0 2024-02-28T23:07:34.298 INFO:tasks.workunit.client.0.smithi057.stdout:6/338: rmdir d0/d1b/d3e/d3d/d42/d4f 39 2024-02-28T23:07:34.299 INFO:tasks.workunit.client.1.smithi084.stdout:6/393: readlink d1/d9/d1d/d1e/d33/d38/l57 0 2024-02-28T23:07:34.301 INFO:tasks.workunit.client.0.smithi057.stdout:3/224: mknod d3/c4b 0 2024-02-28T23:07:34.301 INFO:tasks.workunit.client.1.smithi084.stdout:4/382: chown d1/d18/d20/d14/d30/l7c 2819 1 2024-02-28T23:07:34.302 INFO:tasks.workunit.client.1.smithi084.stdout:1/464: getdents d12/d1b/d2e/d52/d61 0 2024-02-28T23:07:34.302 INFO:tasks.workunit.client.0.smithi057.stdout:0/286: dwrite d17/d19/f28 [0,4194304] 0 2024-02-28T23:07:34.305 INFO:tasks.workunit.client.0.smithi057.stdout:3/225: chown d3/f2c 754275 1 2024-02-28T23:07:34.305 INFO:tasks.workunit.client.0.smithi057.stdout:0/287: fsync d17/f42 0 2024-02-28T23:07:34.306 INFO:tasks.workunit.client.1.smithi084.stdout:8/379: rename d0/d9/d1b/d38/c79 to d0/d24/d4d/c87 0 2024-02-28T23:07:34.308 INFO:tasks.workunit.client.1.smithi084.stdout:7/382: rename db/d10/d17 to db/d10/d17/d3b/d72 22 2024-02-28T23:07:34.308 INFO:tasks.workunit.client.0.smithi057.stdout:0/288: dread - d17/d20/f5d zero size 2024-02-28T23:07:34.308 INFO:tasks.workunit.client.0.smithi057.stdout:9/345: creat dd/d1b/f6c x:0 0 0 2024-02-28T23:07:34.309 INFO:tasks.workunit.client.0.smithi057.stdout:6/339: mknod d0/d1b/d3e/d3d/d42/d4f/d53/c6e 0 2024-02-28T23:07:34.309 INFO:tasks.workunit.client.1.smithi084.stdout:1/465: chown d12/d1b/d63/f66 43545 1 2024-02-28T23:07:34.310 INFO:tasks.workunit.client.1.smithi084.stdout:6/394: mknod d1/d13/c82 0 2024-02-28T23:07:34.310 INFO:tasks.workunit.client.0.smithi057.stdout:3/226: write d3/d4/d1d/f39 [201105,62233] 0 2024-02-28T23:07:34.314 INFO:tasks.workunit.client.1.smithi084.stdout:3/410: getdents d3 0 2024-02-28T23:07:34.314 INFO:tasks.workunit.client.0.smithi057.stdout:9/346: fdatasync dd/de/f6b 0 2024-02-28T23:07:34.317 INFO:tasks.workunit.client.1.smithi084.stdout:1/466: dread - d12/d1b/d2e/d30/d77/f90 zero size 2024-02-28T23:07:34.318 INFO:tasks.workunit.client.0.smithi057.stdout:9/347: read dd/de/d14/f18 [3034937,41436] 0 2024-02-28T23:07:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:34 smithi057 ceph-mon[29096]: pgmap v23: 65 pgs: 65 active+clean; 955 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 9.5 MiB/s rd, 114 MiB/s wr, 220 op/s 2024-02-28T23:07:34.319 INFO:tasks.workunit.client.0.smithi057.stdout:7/294: fsync d0/d5/d36/d4e/d15/f49 0 2024-02-28T23:07:34.321 INFO:tasks.workunit.client.0.smithi057.stdout:7/295: readlink d0/d5/l6 0 2024-02-28T23:07:34.323 INFO:tasks.workunit.client.1.smithi084.stdout:7/383: unlink db/d11/l51 0 2024-02-28T23:07:34.325 INFO:tasks.workunit.client.1.smithi084.stdout:3/411: mknod d3/d8/d1a/c9c 0 2024-02-28T23:07:34.325 INFO:tasks.workunit.client.1.smithi084.stdout:5/445: dwrite dc/f6b [4194304,4194304] 0 2024-02-28T23:07:34.327 INFO:tasks.workunit.client.1.smithi084.stdout:7/384: stat db/d11/d27/f43 0 2024-02-28T23:07:34.330 INFO:tasks.workunit.client.1.smithi084.stdout:6/395: mknod d1/d13/d2e/d41/d4d/d6c/c83 0 2024-02-28T23:07:34.335 INFO:tasks.workunit.client.0.smithi057.stdout:6/340: dread d0/d4/d25/f52 [0,4194304] 0 2024-02-28T23:07:34.335 INFO:tasks.workunit.client.0.smithi057.stdout:0/289: dwrite d17/d19/f3c [0,4194304] 0 2024-02-28T23:07:34.336 INFO:tasks.workunit.client.1.smithi084.stdout:6/396: write d1/d9/d1d/d1e/d33/f75 [1661,115823] 0 2024-02-28T23:07:34.337 INFO:tasks.workunit.client.1.smithi084.stdout:7/385: rmdir db/d11/d27/d36 39 2024-02-28T23:07:34.339 INFO:tasks.workunit.client.0.smithi057.stdout:6/341: chown d0/d4/c3a 527 1 2024-02-28T23:07:34.341 INFO:tasks.workunit.client.1.smithi084.stdout:6/397: read d1/d13/d2e/d41/d4d/f63 [764481,56949] 0 2024-02-28T23:07:34.345 INFO:tasks.workunit.client.1.smithi084.stdout:7/386: fsync db/d10/f23 0 2024-02-28T23:07:34.350 INFO:tasks.workunit.client.0.smithi057.stdout:0/290: symlink d17/d20/d26/d30/d6b/l70 0 2024-02-28T23:07:34.352 INFO:tasks.workunit.client.1.smithi084.stdout:6/398: read d1/d9/d1d/f61 [33639,118370] 0 2024-02-28T23:07:34.353 INFO:tasks.workunit.client.1.smithi084.stdout:8/380: dwrite d0/d9/d1b/d38/d43/d51/d7c/f80 [0,4194304] 0 2024-02-28T23:07:34.354 INFO:tasks.workunit.client.0.smithi057.stdout:6/342: creat d0/f6f x:0 0 0 2024-02-28T23:07:34.354 INFO:tasks.workunit.client.0.smithi057.stdout:5/256: write d4/d21/f35 [473164,17838] 0 2024-02-28T23:07:34.355 INFO:tasks.workunit.client.0.smithi057.stdout:3/227: fsync d3/d4/d7/f10 0 2024-02-28T23:07:34.360 INFO:tasks.workunit.client.1.smithi084.stdout:8/381: fdatasync d0/d9/d1b/f7f 0 2024-02-28T23:07:34.360 INFO:tasks.workunit.client.0.smithi057.stdout:0/291: mkdir d17/d20/d26/d30/d6b/d71 0 2024-02-28T23:07:34.361 INFO:tasks.workunit.client.1.smithi084.stdout:1/467: dwrite d12/f28 [0,4194304] 0 2024-02-28T23:07:34.363 INFO:tasks.workunit.client.0.smithi057.stdout:1/230: dwrite d2/d7/df/f1d [0,4194304] 0 2024-02-28T23:07:34.364 INFO:tasks.workunit.client.0.smithi057.stdout:3/228: dread - d3/d8/d1b/f4a zero size 2024-02-28T23:07:34.367 INFO:tasks.workunit.client.0.smithi057.stdout:2/309: dread d8/d11/d15/f2d [0,4194304] 0 2024-02-28T23:07:34.371 INFO:tasks.workunit.client.1.smithi084.stdout:7/387: creat db/d5e/f73 x:0 0 0 2024-02-28T23:07:34.372 INFO:tasks.workunit.client.0.smithi057.stdout:1/231: rename d2/c32 to d2/d7/df/db/d12/d3f/d47/c50 0 2024-02-28T23:07:34.374 INFO:tasks.workunit.client.1.smithi084.stdout:4/383: dread d1/f7 [0,4194304] 0 2024-02-28T23:07:34.375 INFO:tasks.workunit.client.1.smithi084.stdout:1/468: rmdir d12/d1b/d2e/d32 39 2024-02-28T23:07:34.378 INFO:tasks.workunit.client.1.smithi084.stdout:1/469: stat d12/d1b/f31 0 2024-02-28T23:07:34.378 INFO:tasks.workunit.client.1.smithi084.stdout:4/384: fsync d1/d18/d20/d14/d30/f38 0 2024-02-28T23:07:34.378 INFO:tasks.workunit.client.0.smithi057.stdout:9/348: dread dd/f2d [0,4194304] 0 2024-02-28T23:07:34.380 INFO:tasks.workunit.client.0.smithi057.stdout:3/229: unlink d3/d4/c2f 0 2024-02-28T23:07:34.380 INFO:tasks.workunit.client.0.smithi057.stdout:5/257: dread f2 [4194304,4194304] 0 2024-02-28T23:07:34.381 INFO:tasks.workunit.client.1.smithi084.stdout:1/470: dread - d12/d1b/d63/d7d/f8c zero size 2024-02-28T23:07:34.382 INFO:tasks.workunit.client.1.smithi084.stdout:7/388: mknod db/d5e/c74 0 2024-02-28T23:07:34.383 INFO:tasks.workunit.client.0.smithi057.stdout:1/232: truncate d2/d7/df/db/d12/d19/d24/f4a 806789 0 2024-02-28T23:07:34.383 INFO:tasks.workunit.client.0.smithi057.stdout:9/349: read dd/de/d14/f35 [1859261,46512] 0 2024-02-28T23:07:34.388 INFO:tasks.workunit.client.0.smithi057.stdout:2/310: fsync d8/d11/d15/f35 0 2024-02-28T23:07:34.389 INFO:tasks.workunit.client.1.smithi084.stdout:6/399: dwrite d1/d13/d47/f56 [4194304,4194304] 0 2024-02-28T23:07:34.391 INFO:tasks.workunit.client.0.smithi057.stdout:0/292: dwrite f5 [4194304,4194304] 0 2024-02-28T23:07:34.397 INFO:tasks.workunit.client.1.smithi084.stdout:7/389: readlink db/l4a 0 2024-02-28T23:07:34.400 INFO:tasks.workunit.client.1.smithi084.stdout:5/446: dread dc/f2a [0,4194304] 0 2024-02-28T23:07:34.400 INFO:tasks.workunit.client.0.smithi057.stdout:3/230: mkdir d3/d20/d3f/d4c 0 2024-02-28T23:07:34.401 INFO:tasks.workunit.client.0.smithi057.stdout:6/343: getdents d0/d17/d20 0 2024-02-28T23:07:34.402 INFO:tasks.workunit.client.0.smithi057.stdout:9/350: mkdir dd/de/d14/d48/d4b/d6d 0 2024-02-28T23:07:34.402 INFO:tasks.workunit.client.0.smithi057.stdout:0/293: truncate d17/d20/d4a/f68 354543 0 2024-02-28T23:07:34.402 INFO:tasks.workunit.client.0.smithi057.stdout:3/231: dread - d3/d8/f45 zero size 2024-02-28T23:07:34.409 INFO:tasks.workunit.client.1.smithi084.stdout:4/385: fsync d1/d4a/f54 0 2024-02-28T23:07:34.410 INFO:tasks.workunit.client.1.smithi084.stdout:1/471: read d12/d1b/d2e/d30/f26 [533412,25482] 0 2024-02-28T23:07:34.410 INFO:tasks.workunit.client.1.smithi084.stdout:7/390: creat db/d10/d17/d64/f75 x:0 0 0 2024-02-28T23:07:34.410 INFO:tasks.workunit.client.0.smithi057.stdout:2/311: symlink d8/l6c 0 2024-02-28T23:07:34.412 INFO:tasks.workunit.client.0.smithi057.stdout:9/351: stat dd/de/d14/f1e 0 2024-02-28T23:07:34.413 INFO:tasks.workunit.client.1.smithi084.stdout:4/386: dread - d1/f6e zero size 2024-02-28T23:07:34.418 INFO:tasks.workunit.client.0.smithi057.stdout:8/302: sync 2024-02-28T23:07:34.418 INFO:tasks.workunit.client.0.smithi057.stdout:1/233: mknod d2/d7/df/db/d12/d19/c51 0 2024-02-28T23:07:34.419 INFO:tasks.workunit.client.1.smithi084.stdout:1/472: read d12/d1b/f31 [1009576,75434] 0 2024-02-28T23:07:34.421 INFO:tasks.workunit.client.0.smithi057.stdout:3/232: mknod d3/d8/d1b/c4d 0 2024-02-28T23:07:34.425 INFO:tasks.workunit.client.0.smithi057.stdout:5/258: dwrite d4/d9/f2a [0,4194304] 0 2024-02-28T23:07:34.426 INFO:tasks.workunit.client.0.smithi057.stdout:4/294: dread d1a/d1b/f1f [4194304,4194304] 0 2024-02-28T23:07:34.426 INFO:tasks.workunit.client.0.smithi057.stdout:0/294: rename c2 to d17/d20/d23/c72 0 2024-02-28T23:07:34.426 INFO:tasks.workunit.client.0.smithi057.stdout:6/344: write d0/d4/d25/f2a [1343346,106910] 0 2024-02-28T23:07:34.427 INFO:tasks.workunit.client.1.smithi084.stdout:0/420: dwrite d1/d6/db/d37/d72/d7b/f80 [4194304,4194304] 0 2024-02-28T23:07:34.428 INFO:tasks.workunit.client.0.smithi057.stdout:1/234: write d2/f21 [292977,55463] 0 2024-02-28T23:07:34.431 INFO:tasks.workunit.client.0.smithi057.stdout:5/259: chown d4/d21/d24 4327 1 2024-02-28T23:07:34.431 INFO:tasks.workunit.client.1.smithi084.stdout:5/447: mkdir dc/de/df/da6 0 2024-02-28T23:07:34.431 INFO:tasks.workunit.client.1.smithi084.stdout:0/421: write d1/d8/f3e [461752,113326] 0 2024-02-28T23:07:34.432 INFO:tasks.workunit.client.0.smithi057.stdout:2/312: creat d8/d11/d48/d50/f6d x:0 0 0 2024-02-28T23:07:34.433 INFO:tasks.workunit.client.1.smithi084.stdout:1/473: read fc [7424383,88031] 0 2024-02-28T23:07:34.434 INFO:tasks.workunit.client.1.smithi084.stdout:5/448: stat dc/de/l94 0 2024-02-28T23:07:34.435 INFO:tasks.workunit.client.0.smithi057.stdout:8/303: write d4/d13/d18/f65 [425675,51078] 0 2024-02-28T23:07:34.437 INFO:tasks.workunit.client.1.smithi084.stdout:2/344: dwrite d5/d7/d9/f2c [0,4194304] 0 2024-02-28T23:07:34.439 INFO:tasks.workunit.client.1.smithi084.stdout:6/400: link d1/d9/c52 d1/d13/c84 0 2024-02-28T23:07:34.439 INFO:tasks.workunit.client.0.smithi057.stdout:3/233: rmdir d3/d20/d2d 39 2024-02-28T23:07:34.441 INFO:tasks.workunit.client.1.smithi084.stdout:0/422: chown d1/d6/db/d35/d44/f79 816 1 2024-02-28T23:07:34.443 INFO:tasks.workunit.client.1.smithi084.stdout:9/455: sync 2024-02-28T23:07:34.445 INFO:tasks.workunit.client.0.smithi057.stdout:6/345: symlink d0/d1b/l70 0 2024-02-28T23:07:34.446 INFO:tasks.workunit.client.0.smithi057.stdout:3/234: truncate d3/d20/d3f/f48 126832 0 2024-02-28T23:07:34.446 INFO:tasks.workunit.client.0.smithi057.stdout:1/235: write d2/d7/df/db/d12/d19/f23 [2074612,12551] 0 2024-02-28T23:07:34.449 INFO:tasks.workunit.client.0.smithi057.stdout:8/304: write d4/f7 [7196806,73936] 0 2024-02-28T23:07:34.451 INFO:tasks.workunit.client.0.smithi057.stdout:5/260: mknod d4/c49 0 2024-02-28T23:07:34.451 INFO:tasks.workunit.client.0.smithi057.stdout:0/295: symlink d17/d20/d23/l73 0 2024-02-28T23:07:34.453 INFO:tasks.workunit.client.1.smithi084.stdout:1/474: mknod d12/d1b/d2e/d52/d5b/c9c 0 2024-02-28T23:07:34.454 INFO:tasks.workunit.client.0.smithi057.stdout:2/313: symlink d8/d11/d39/l6e 0 2024-02-28T23:07:34.457 INFO:tasks.workunit.client.0.smithi057.stdout:7/296: dwrite d0/f10 [0,4194304] 0 2024-02-28T23:07:34.458 INFO:tasks.workunit.client.1.smithi084.stdout:8/382: dread d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:34.458 INFO:tasks.workunit.client.1.smithi084.stdout:3/412: dwrite d3/fb [0,4194304] 0 2024-02-28T23:07:34.459 INFO:tasks.workunit.client.0.smithi057.stdout:9/352: link dd/f5b dd/de/d14/d48/d4b/f6e 0 2024-02-28T23:07:34.459 INFO:tasks.workunit.client.1.smithi084.stdout:6/401: mknod d1/d13/c85 0 2024-02-28T23:07:34.460 INFO:tasks.workunit.client.0.smithi057.stdout:0/296: write d17/d20/f51 [4894191,26499] 0 2024-02-28T23:07:34.460 INFO:tasks.workunit.client.0.smithi057.stdout:3/235: write d3/d20/d3f/f48 [1103734,32280] 0 2024-02-28T23:07:34.462 INFO:tasks.workunit.client.1.smithi084.stdout:6/402: readlink d1/l46 0 2024-02-28T23:07:34.463 INFO:tasks.workunit.client.1.smithi084.stdout:8/383: fsync d0/d9/d1b/d38/d43/d51/d7c/f80 0 2024-02-28T23:07:34.465 INFO:tasks.workunit.client.0.smithi057.stdout:6/346: rename d0/d17/f45 to d0/d4/d25/f71 0 2024-02-28T23:07:34.466 INFO:tasks.workunit.client.0.smithi057.stdout:1/236: creat d2/d7/df/f52 x:0 0 0 2024-02-28T23:07:34.467 INFO:tasks.workunit.client.1.smithi084.stdout:7/391: dwrite db/d11/f12 [4194304,4194304] 0 2024-02-28T23:07:34.468 INFO:tasks.workunit.client.1.smithi084.stdout:6/403: chown d1/d9/f4f 53 1 2024-02-28T23:07:34.468 INFO:tasks.workunit.client.0.smithi057.stdout:7/297: write d0/d5/d36/d4e/d15/f1a [1293379,30274] 0 2024-02-28T23:07:34.469 INFO:tasks.workunit.client.1.smithi084.stdout:0/423: read d1/d6/d1b/f58 [462512,66875] 0 2024-02-28T23:07:34.470 INFO:tasks.workunit.client.1.smithi084.stdout:4/387: getdents d1/d18/d20 0 2024-02-28T23:07:34.475 INFO:tasks.workunit.client.1.smithi084.stdout:3/413: mknod d3/d8/d14/d31/d34/c9d 0 2024-02-28T23:07:34.476 INFO:tasks.workunit.client.0.smithi057.stdout:8/305: dread d4/f5 [0,4194304] 0 2024-02-28T23:07:34.476 INFO:tasks.workunit.client.0.smithi057.stdout:4/295: dwrite f16 [0,4194304] 0 2024-02-28T23:07:34.476 INFO:tasks.workunit.client.0.smithi057.stdout:4/296: fdatasync d1a/d1b/f5c 0 2024-02-28T23:07:34.477 INFO:tasks.workunit.client.1.smithi084.stdout:6/404: truncate d1/d9/f28 5204775 0 2024-02-28T23:07:34.478 INFO:tasks.workunit.client.1.smithi084.stdout:5/449: creat dc/de/df/fa7 x:0 0 0 2024-02-28T23:07:34.479 INFO:tasks.workunit.client.0.smithi057.stdout:5/261: mknod d4/d7/c4a 0 2024-02-28T23:07:34.480 INFO:tasks.workunit.client.0.smithi057.stdout:4/297: write d1a/d1b/d24/f3d [854073,37920] 0 2024-02-28T23:07:34.484 INFO:tasks.workunit.client.1.smithi084.stdout:9/456: rename d4/d32/f5c to d4/d6/d15/f8e 0 2024-02-28T23:07:34.485 INFO:tasks.workunit.client.0.smithi057.stdout:5/262: truncate d4/d7/f1e 2254456 0 2024-02-28T23:07:34.485 INFO:tasks.workunit.client.0.smithi057.stdout:1/237: symlink d2/l53 0 2024-02-28T23:07:34.487 INFO:tasks.workunit.client.1.smithi084.stdout:6/405: chown d1/d9/d1d/d1e/d33/f39 118 1 2024-02-28T23:07:34.487 INFO:tasks.workunit.client.0.smithi057.stdout:6/347: mknod d0/d17/c72 0 2024-02-28T23:07:34.488 INFO:tasks.workunit.client.0.smithi057.stdout:5/263: write d4/d9/f2a [4646256,5021] 0 2024-02-28T23:07:34.488 INFO:tasks.workunit.client.0.smithi057.stdout:1/238: write d2/d7/df/f52 [817820,110070] 0 2024-02-28T23:07:34.489 INFO:tasks.workunit.client.0.smithi057.stdout:3/236: rename d3/d20/l33 to d3/l4e 0 2024-02-28T23:07:34.490 INFO:tasks.workunit.client.0.smithi057.stdout:2/314: dwrite d8/d11/d21/d28/d41/f53 [0,4194304] 0 2024-02-28T23:07:34.490 INFO:tasks.workunit.client.0.smithi057.stdout:6/348: stat d0/d1b/d54/c4c 0 2024-02-28T23:07:34.490 INFO:tasks.workunit.client.0.smithi057.stdout:5/264: chown d4/d9/d14 15518295 1 2024-02-28T23:07:34.497 INFO:tasks.workunit.client.1.smithi084.stdout:6/406: chown d1/d13/d50/d65/c6b 8 1 2024-02-28T23:07:34.498 INFO:tasks.workunit.client.0.smithi057.stdout:9/353: rename dd/de/d14/d48/d4b/d6d to dd/d46/d6f 0 2024-02-28T23:07:34.500 INFO:tasks.workunit.client.1.smithi084.stdout:6/407: fdatasync d1/d13/d47/f7a 0 2024-02-28T23:07:34.500 INFO:tasks.workunit.client.1.smithi084.stdout:8/384: dwrite d0/d24/d4d/f81 [0,4194304] 0 2024-02-28T23:07:34.501 INFO:tasks.workunit.client.0.smithi057.stdout:6/349: getdents d0/d1b/d3e/d3d/d42/d4f/d53 0 2024-02-28T23:07:34.502 INFO:tasks.workunit.client.1.smithi084.stdout:3/414: link d3/d8/d1a/d29/f38 d3/d8/df/f9e 0 2024-02-28T23:07:34.507 INFO:tasks.workunit.client.1.smithi084.stdout:9/457: symlink d4/d6/d15/d3c/l8f 0 2024-02-28T23:07:34.507 INFO:tasks.workunit.client.1.smithi084.stdout:7/392: mknod db/d11/d27/d36/c76 0 2024-02-28T23:07:34.507 INFO:tasks.workunit.client.0.smithi057.stdout:9/354: creat dd/de/d14/d48/d4b/d41/f70 x:0 0 0 2024-02-28T23:07:34.509 INFO:tasks.workunit.client.0.smithi057.stdout:6/350: creat d0/d1b/d3e/d40/f73 x:0 0 0 2024-02-28T23:07:34.510 INFO:tasks.workunit.client.1.smithi084.stdout:1/475: truncate d12/d1b/d2e/d32/f5e 2273331 0 2024-02-28T23:07:34.511 INFO:tasks.workunit.client.1.smithi084.stdout:7/393: chown db/d10/d17/l1f 3678 1 2024-02-28T23:07:34.512 INFO:tasks.workunit.client.0.smithi057.stdout:5/265: dwrite d4/d9/d14/f28 [8388608,4194304] 0 2024-02-28T23:07:34.514 INFO:tasks.workunit.client.1.smithi084.stdout:9/458: write d4/d6/f17 [3655222,23415] 0 2024-02-28T23:07:34.516 INFO:tasks.workunit.client.0.smithi057.stdout:6/351: mkdir d0/d1b/d3e/d3d/d74 0 2024-02-28T23:07:34.519 INFO:tasks.workunit.client.0.smithi057.stdout:5/266: mknod d4/d21/d24/d30/c4b 0 2024-02-28T23:07:34.519 INFO:tasks.workunit.client.1.smithi084.stdout:6/408: readlink d1/l25 0 2024-02-28T23:07:34.520 INFO:tasks.workunit.client.1.smithi084.stdout:8/385: stat d0/la 0 2024-02-28T23:07:34.520 INFO:tasks.workunit.client.1.smithi084.stdout:3/415: unlink d3/d8/f93 0 2024-02-28T23:07:34.522 INFO:tasks.workunit.client.1.smithi084.stdout:5/450: dwrite dc/de/df/d31/f43 [0,4194304] 0 2024-02-28T23:07:34.523 INFO:tasks.workunit.client.1.smithi084.stdout:9/459: truncate d4/d6/d15/f62 221125 0 2024-02-28T23:07:34.527 INFO:tasks.workunit.client.0.smithi057.stdout:6/352: link d0/d1b/d54/c4c d0/d17/c75 0 2024-02-28T23:07:34.529 INFO:tasks.workunit.client.0.smithi057.stdout:5/267: rmdir d4/d32/d48 0 2024-02-28T23:07:34.529 INFO:tasks.workunit.client.1.smithi084.stdout:5/451: read - dc/de/df/d16/d34/d54/f80 zero size 2024-02-28T23:07:34.530 INFO:tasks.workunit.client.0.smithi057.stdout:6/353: write d0/d4/f50 [262193,7795] 0 2024-02-28T23:07:34.534 INFO:tasks.workunit.client.1.smithi084.stdout:0/424: link d1/d8/d18/f63 d1/d6/d59/d70/f8d 0 2024-02-28T23:07:34.535 INFO:tasks.workunit.client.0.smithi057.stdout:6/354: fsync d0/d4/d25/d60/d6a/f69 0 2024-02-28T23:07:34.536 INFO:tasks.workunit.client.0.smithi057.stdout:6/355: fdatasync d0/d4/fa 0 2024-02-28T23:07:34.537 INFO:tasks.workunit.client.0.smithi057.stdout:6/356: fdatasync d0/f3c 0 2024-02-28T23:07:34.543 INFO:tasks.workunit.client.1.smithi084.stdout:7/394: chown db/l4e 1227822 1 2024-02-28T23:07:34.543 INFO:tasks.workunit.client.1.smithi084.stdout:3/416: dread d3/d8/d1a/d21/d3d/f67 [0,4194304] 0 2024-02-28T23:07:34.543 INFO:tasks.workunit.client.1.smithi084.stdout:2/345: dread d5/f17 [0,4194304] 0 2024-02-28T23:07:34.546 INFO:tasks.workunit.client.1.smithi084.stdout:7/395: dread - db/d10/f71 zero size 2024-02-28T23:07:34.552 INFO:tasks.workunit.client.1.smithi084.stdout:1/476: rename d12/d1b/d2e/d52 to d12/d1b/d2e/d52/d9d 22 2024-02-28T23:07:34.554 INFO:tasks.workunit.client.1.smithi084.stdout:9/460: creat d4/f90 x:0 0 0 2024-02-28T23:07:34.555 INFO:tasks.workunit.client.1.smithi084.stdout:5/452: dread dc/f6b [4194304,4194304] 0 2024-02-28T23:07:34.560 INFO:tasks.workunit.client.1.smithi084.stdout:4/388: dread d1/d18/d20/d14/f39 [0,4194304] 0 2024-02-28T23:07:34.566 INFO:tasks.workunit.client.1.smithi084.stdout:6/409: dread d1/d9/d1d/d1e/d33/f39 [0,4194304] 0 2024-02-28T23:07:34.567 INFO:tasks.workunit.client.1.smithi084.stdout:5/453: write dc/de/df/d16/d34/d54/f80 [158352,96118] 0 2024-02-28T23:07:34.567 INFO:tasks.workunit.client.0.smithi057.stdout:3/237: rename d3/d20/d3f to d3/d4f 0 2024-02-28T23:07:34.576 INFO:tasks.workunit.client.1.smithi084.stdout:3/417: symlink d3/d8/d1a/l9f 0 2024-02-28T23:07:34.578 INFO:tasks.workunit.client.1.smithi084.stdout:7/396: mkdir db/d10/d17/d64/d77 0 2024-02-28T23:07:34.579 INFO:tasks.workunit.client.1.smithi084.stdout:3/418: fsync d3/d8/d1a/d21/d3d/d43/d61/f6e 0 2024-02-28T23:07:34.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:34 smithi084 ceph-mon[34826]: pgmap v23: 65 pgs: 65 active+clean; 955 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 9.5 MiB/s rd, 114 MiB/s wr, 220 op/s 2024-02-28T23:07:34.584 INFO:tasks.workunit.client.1.smithi084.stdout:1/477: creat d12/d1b/d2e/d32/f9e x:0 0 0 2024-02-28T23:07:34.586 INFO:tasks.workunit.client.1.smithi084.stdout:1/478: chown d12/d1b/c24 92004 1 2024-02-28T23:07:34.587 INFO:tasks.workunit.client.0.smithi057.stdout:7/298: dread d0/d5/d36/d4e/d15/f3a [4194304,4194304] 0 2024-02-28T23:07:34.590 INFO:tasks.workunit.client.0.smithi057.stdout:7/299: creat d0/d5/d36/d4e/d3b/f5d x:0 0 0 2024-02-28T23:07:34.591 INFO:tasks.workunit.client.1.smithi084.stdout:6/410: symlink d1/d13/d2e/l86 0 2024-02-28T23:07:34.598 INFO:tasks.workunit.client.1.smithi084.stdout:8/386: write d0/d24/d4d/f81 [4246099,87682] 0 2024-02-28T23:07:34.601 INFO:tasks.workunit.client.1.smithi084.stdout:2/346: truncate d5/d7/d9/f3c 1844990 0 2024-02-28T23:07:34.605 INFO:tasks.workunit.client.0.smithi057.stdout:7/300: dwrite d0/d5/d25/f28 [0,4194304] 0 2024-02-28T23:07:34.607 INFO:tasks.workunit.client.1.smithi084.stdout:3/419: rmdir d3/d8/d1a/d45/d5e 39 2024-02-28T23:07:34.607 INFO:tasks.workunit.client.0.smithi057.stdout:0/297: truncate f15 2777999 0 2024-02-28T23:07:34.612 INFO:tasks.workunit.client.0.smithi057.stdout:2/315: write d8/d11/d21/f33 [1472493,28520] 0 2024-02-28T23:07:34.612 INFO:tasks.workunit.client.0.smithi057.stdout:0/298: write d17/d20/d26/f46 [3088968,83081] 0 2024-02-28T23:07:34.612 INFO:tasks.workunit.client.1.smithi084.stdout:3/420: dread - d3/d8/d1a/d29/d41/d60/f8b zero size 2024-02-28T23:07:34.617 INFO:tasks.workunit.client.1.smithi084.stdout:2/347: stat d5/d1f/d42/l57 0 2024-02-28T23:07:34.618 INFO:tasks.workunit.client.0.smithi057.stdout:7/301: getdents d0/d5/d36/d4e/d15/d29/d3d 0 2024-02-28T23:07:34.619 INFO:tasks.workunit.client.1.smithi084.stdout:8/387: dwrite d0/d24/f5b [0,4194304] 0 2024-02-28T23:07:34.622 INFO:tasks.workunit.client.1.smithi084.stdout:5/454: rename dc/de/l44 to dc/de/df/d16/d34/la8 0 2024-02-28T23:07:34.623 INFO:tasks.workunit.client.0.smithi057.stdout:7/302: dread - d0/d5/d36/d4e/d3b/f5d zero size 2024-02-28T23:07:34.624 INFO:tasks.workunit.client.0.smithi057.stdout:0/299: link d17/d20/f2b d17/d20/d23/d37/f74 0 2024-02-28T23:07:34.626 INFO:tasks.workunit.client.0.smithi057.stdout:9/355: truncate dd/de/d14/f21 663534 0 2024-02-28T23:07:34.626 INFO:tasks.workunit.client.1.smithi084.stdout:6/411: dread d1/d13/d2e/d41/f2f [0,4194304] 0 2024-02-28T23:07:34.626 INFO:tasks.workunit.client.1.smithi084.stdout:0/425: getdents d1/d8/d18/d28 0 2024-02-28T23:07:34.628 INFO:tasks.workunit.client.1.smithi084.stdout:4/389: write d1/d18/d20/d14/f36 [840144,39618] 0 2024-02-28T23:07:34.629 INFO:tasks.workunit.client.1.smithi084.stdout:3/421: mkdir d3/d27/d7d/da0 0 2024-02-28T23:07:34.631 INFO:tasks.workunit.client.0.smithi057.stdout:6/357: truncate d0/d17/f35 424039 0 2024-02-28T23:07:34.632 INFO:tasks.workunit.client.0.smithi057.stdout:9/356: mknod dd/de/d14/c71 0 2024-02-28T23:07:34.633 INFO:tasks.workunit.client.0.smithi057.stdout:2/316: dread d8/d11/d21/d28/d41/f53 [0,4194304] 0 2024-02-28T23:07:34.634 INFO:tasks.workunit.client.1.smithi084.stdout:7/397: rmdir db/d11/d67 0 2024-02-28T23:07:34.636 INFO:tasks.workunit.client.0.smithi057.stdout:3/238: truncate d3/d8/f3c 1291056 0 2024-02-28T23:07:34.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/317: chown d8/lc 0 1 2024-02-28T23:07:34.637 INFO:tasks.workunit.client.1.smithi084.stdout:3/422: mknod d3/ca1 0 2024-02-28T23:07:34.638 INFO:tasks.workunit.client.0.smithi057.stdout:2/318: write d8/f4f [143802,4266] 0 2024-02-28T23:07:34.638 INFO:tasks.workunit.client.0.smithi057.stdout:3/239: dread - d3/d4/d1d/f49 zero size 2024-02-28T23:07:34.639 INFO:tasks.workunit.client.0.smithi057.stdout:6/358: mknod d0/d1b/d3e/d3d/d42/c76 0 2024-02-28T23:07:34.640 INFO:tasks.workunit.client.1.smithi084.stdout:7/398: truncate db/d5e/f6a 351727 0 2024-02-28T23:07:34.641 INFO:tasks.workunit.client.0.smithi057.stdout:9/357: getdents dd/d46 0 2024-02-28T23:07:34.642 INFO:tasks.workunit.client.0.smithi057.stdout:2/319: creat d8/d11/d39/f6f x:0 0 0 2024-02-28T23:07:34.643 INFO:tasks.workunit.client.0.smithi057.stdout:0/300: dread d17/d19/f2f [0,4194304] 0 2024-02-28T23:07:34.643 INFO:tasks.workunit.client.1.smithi084.stdout:4/390: link d1/d18/f77 d1/d18/d20/d14/d30/d47/f82 0 2024-02-28T23:07:34.643 INFO:tasks.workunit.client.1.smithi084.stdout:2/348: getdents d5/d7/d2e 0 2024-02-28T23:07:34.644 INFO:tasks.workunit.client.0.smithi057.stdout:9/358: rmdir dd/de/d24 39 2024-02-28T23:07:34.645 INFO:tasks.workunit.client.0.smithi057.stdout:7/303: dread d0/d5/d36/d4e/d15/f2b [0,4194304] 0 2024-02-28T23:07:34.645 INFO:tasks.workunit.client.0.smithi057.stdout:2/320: fdatasync d8/d11/d48/d50/f66 0 2024-02-28T23:07:34.646 INFO:tasks.workunit.client.0.smithi057.stdout:9/359: readlink dd/d1b/l5f 0 2024-02-28T23:07:34.646 INFO:tasks.workunit.client.1.smithi084.stdout:3/423: mknod d3/d50/d5a/ca2 0 2024-02-28T23:07:34.647 INFO:tasks.workunit.client.1.smithi084.stdout:8/388: creat d0/d9/f88 x:0 0 0 2024-02-28T23:07:34.647 INFO:tasks.workunit.client.1.smithi084.stdout:9/461: rename d4/d32/f6c to d4/d6/d15/d12/d41/d55/f91 0 2024-02-28T23:07:34.647 INFO:tasks.workunit.client.0.smithi057.stdout:0/301: creat d17/d20/d23/f75 x:0 0 0 2024-02-28T23:07:34.648 INFO:tasks.workunit.client.0.smithi057.stdout:9/360: fsync dd/d1b/d40/d44/f62 0 2024-02-28T23:07:34.649 INFO:tasks.workunit.client.0.smithi057.stdout:2/321: unlink d8/d11/d21/c61 0 2024-02-28T23:07:34.649 INFO:tasks.workunit.client.0.smithi057.stdout:9/361: dread - dd/f3c zero size 2024-02-28T23:07:34.652 INFO:tasks.workunit.client.0.smithi057.stdout:0/302: creat d17/d19/f76 x:0 0 0 2024-02-28T23:07:34.654 INFO:tasks.workunit.client.0.smithi057.stdout:2/322: mkdir d8/d11/d21/d70 0 2024-02-28T23:07:34.656 INFO:tasks.workunit.client.0.smithi057.stdout:9/362: mkdir dd/de/d24/d72 0 2024-02-28T23:07:34.656 INFO:tasks.workunit.client.1.smithi084.stdout:7/399: dread db/d11/d27/f43 [0,4194304] 0 2024-02-28T23:07:34.657 INFO:tasks.workunit.client.0.smithi057.stdout:0/303: stat d17/d20/d26/l40 0 2024-02-28T23:07:34.658 INFO:tasks.workunit.client.0.smithi057.stdout:2/323: creat d8/d11/d15/f71 x:0 0 0 2024-02-28T23:07:34.660 INFO:tasks.workunit.client.1.smithi084.stdout:0/426: write d1/d6/d2d/f30 [4162617,67232] 0 2024-02-28T23:07:34.661 INFO:tasks.workunit.client.1.smithi084.stdout:8/389: creat d0/d9/d1b/d38/d43/d51/d7c/f89 x:0 0 0 2024-02-28T23:07:34.665 INFO:tasks.workunit.client.1.smithi084.stdout:4/391: rmdir d1/d18/d20/d24/d2f/d3e 39 2024-02-28T23:07:34.665 INFO:tasks.workunit.client.1.smithi084.stdout:5/455: unlink dc/de/df/d16/d34/la8 0 2024-02-28T23:07:34.667 INFO:tasks.workunit.client.0.smithi057.stdout:7/304: dwrite d0/f50 [0,4194304] 0 2024-02-28T23:07:34.673 INFO:tasks.workunit.client.1.smithi084.stdout:4/392: write d1/f9 [4084345,91324] 0 2024-02-28T23:07:34.673 INFO:tasks.workunit.client.1.smithi084.stdout:2/349: dwrite d5/d1f/f44 [0,4194304] 0 2024-02-28T23:07:34.674 INFO:tasks.workunit.client.1.smithi084.stdout:5/456: write dc/d51/d7c/f84 [210731,128698] 0 2024-02-28T23:07:34.674 INFO:tasks.workunit.client.1.smithi084.stdout:8/390: fsync d0/d9/d1b/f46 0 2024-02-28T23:07:34.675 INFO:tasks.workunit.client.1.smithi084.stdout:6/412: dread d1/d9/f28 [0,4194304] 0 2024-02-28T23:07:34.675 INFO:tasks.workunit.client.0.smithi057.stdout:7/305: unlink d0/d5/d36/d4e/c37 0 2024-02-28T23:07:34.678 INFO:tasks.workunit.client.0.smithi057.stdout:7/306: fsync d0/d5/f30 0 2024-02-28T23:07:34.679 INFO:tasks.workunit.client.1.smithi084.stdout:5/457: stat dc/de/f1a 0 2024-02-28T23:07:34.680 INFO:tasks.workunit.client.0.smithi057.stdout:9/363: dwrite dd/de/d14/f1f [0,4194304] 0 2024-02-28T23:07:34.682 INFO:tasks.workunit.client.1.smithi084.stdout:2/350: truncate d5/d7/d12/d24/d2a/d53/f5f 601808 0 2024-02-28T23:07:34.683 INFO:tasks.workunit.client.1.smithi084.stdout:5/458: write dc/de/df/d16/d34/f4d [526754,93779] 0 2024-02-28T23:07:34.683 INFO:tasks.workunit.client.0.smithi057.stdout:0/304: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:34.686 INFO:tasks.workunit.client.1.smithi084.stdout:9/462: rename d4/d6/d15/d12/d41/d55/f91 to d4/d32/f92 0 2024-02-28T23:07:34.688 INFO:tasks.workunit.client.0.smithi057.stdout:7/307: creat d0/d5/d36/d4e/d15/d29/d3d/d51/f5e x:0 0 0 2024-02-28T23:07:34.691 INFO:tasks.workunit.client.1.smithi084.stdout:8/391: chown d0/d9/d30/d33/f5e 0 1 2024-02-28T23:07:34.691 INFO:tasks.workunit.client.0.smithi057.stdout:7/308: readlink d0/d5/d36/d4e/d15/d29/d3d/l4f 0 2024-02-28T23:07:34.691 INFO:tasks.workunit.client.1.smithi084.stdout:7/400: dwrite db/d11/d27/d36/f5a [0,4194304] 0 2024-02-28T23:07:34.692 INFO:tasks.workunit.client.0.smithi057.stdout:9/364: dread dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:34.693 INFO:tasks.workunit.client.0.smithi057.stdout:0/305: rename d17/d19/c1b to d17/d20/d26/d30/c77 0 2024-02-28T23:07:34.694 INFO:tasks.workunit.client.1.smithi084.stdout:2/351: mknod d5/d1f/c60 0 2024-02-28T23:07:34.696 INFO:tasks.workunit.client.1.smithi084.stdout:5/459: mkdir dc/de/df/d31/d8e/da9 0 2024-02-28T23:07:34.697 INFO:tasks.workunit.client.0.smithi057.stdout:0/306: mkdir d17/d20/d4a/d78 0 2024-02-28T23:07:34.697 INFO:tasks.workunit.client.1.smithi084.stdout:2/352: stat d5/d1f/l36 0 2024-02-28T23:07:34.698 INFO:tasks.workunit.client.0.smithi057.stdout:7/309: symlink d0/d5/d36/d4e/d15/d29/l5f 0 2024-02-28T23:07:34.700 INFO:tasks.workunit.client.1.smithi084.stdout:6/413: creat d1/d9/f87 x:0 0 0 2024-02-28T23:07:34.702 INFO:tasks.workunit.client.1.smithi084.stdout:2/353: write d5/d7/d12/d24/d2a/d35/f5d [916788,67576] 0 2024-02-28T23:07:34.703 INFO:tasks.workunit.client.1.smithi084.stdout:8/392: symlink d0/d9/d1b/d38/d43/d51/d7c/l8a 0 2024-02-28T23:07:34.708 INFO:tasks.workunit.client.1.smithi084.stdout:2/354: chown d5/d7/d12/d24/d2a/d35/f40 82465957 1 2024-02-28T23:07:34.710 INFO:tasks.workunit.client.0.smithi057.stdout:0/307: rename f15 to d17/d20/d4a/d78/f79 0 2024-02-28T23:07:34.713 INFO:tasks.workunit.client.0.smithi057.stdout:0/308: mkdir d17/d7a 0 2024-02-28T23:07:34.714 INFO:tasks.workunit.client.1.smithi084.stdout:8/393: creat d0/d24/d6e/f8b x:0 0 0 2024-02-28T23:07:34.715 INFO:tasks.workunit.client.1.smithi084.stdout:5/460: mkdir dc/de/df/d16/d34/daa 0 2024-02-28T23:07:34.715 INFO:tasks.workunit.client.1.smithi084.stdout:3/424: fsync d3/d8/d1a/d29/d41/f71 0 2024-02-28T23:07:34.718 INFO:tasks.workunit.client.1.smithi084.stdout:0/427: write d1/d6/d59/d70/f73 [749446,40763] 0 2024-02-28T23:07:34.722 INFO:tasks.workunit.client.0.smithi057.stdout:9/365: dwrite dd/d1b/f69 [0,4194304] 0 2024-02-28T23:07:34.723 INFO:tasks.workunit.client.0.smithi057.stdout:7/310: dwrite d0/d5/f4b [0,4194304] 0 2024-02-28T23:07:34.723 INFO:tasks.workunit.client.1.smithi084.stdout:5/461: write dc/de/df/d16/d34/d3c/f5a [429473,49339] 0 2024-02-28T23:07:34.727 INFO:tasks.workunit.client.0.smithi057.stdout:2/324: write d8/f42 [889308,90341] 0 2024-02-28T23:07:34.727 INFO:tasks.workunit.client.0.smithi057.stdout:7/311: chown d0/d5/d36/d4e/d15/d29/l5f 371164259 1 2024-02-28T23:07:34.727 INFO:tasks.workunit.client.1.smithi084.stdout:0/428: symlink d1/d6/db/d35/d57/l8e 0 2024-02-28T23:07:34.728 INFO:tasks.workunit.client.1.smithi084.stdout:7/401: dread db/d10/d17/f58 [0,4194304] 0 2024-02-28T23:07:34.728 INFO:tasks.workunit.client.0.smithi057.stdout:0/309: dread d17/d20/d26/f52 [0,4194304] 0 2024-02-28T23:07:34.728 INFO:tasks.workunit.client.0.smithi057.stdout:9/366: mknod dd/d46/c73 0 2024-02-28T23:07:34.728 INFO:tasks.workunit.client.0.smithi057.stdout:2/325: truncate d8/f42 1204985 0 2024-02-28T23:07:34.731 INFO:tasks.workunit.client.0.smithi057.stdout:9/367: dread - dd/f5b zero size 2024-02-28T23:07:34.734 INFO:tasks.workunit.client.1.smithi084.stdout:5/462: mkdir dc/de/df/d31/d8e/da9/dab 0 2024-02-28T23:07:34.734 INFO:tasks.workunit.client.0.smithi057.stdout:0/310: fdatasync d17/d20/d23/d37/f74 0 2024-02-28T23:07:34.734 INFO:tasks.workunit.client.0.smithi057.stdout:7/312: write d0/d5/d36/d4e/d15/d29/f32 [1353294,72930] 0 2024-02-28T23:07:34.736 INFO:tasks.workunit.client.0.smithi057.stdout:9/368: chown dd/de/d14/d48/d4b/l64 4195 1 2024-02-28T23:07:34.737 INFO:tasks.workunit.client.1.smithi084.stdout:8/394: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:34.738 INFO:tasks.workunit.client.0.smithi057.stdout:9/369: dread - dd/f5b zero size 2024-02-28T23:07:34.739 INFO:tasks.workunit.client.0.smithi057.stdout:0/311: write d17/d20/d4a/f68 [992271,54927] 0 2024-02-28T23:07:34.739 INFO:tasks.workunit.client.0.smithi057.stdout:2/326: rename d8/d11/d21/f68 to d8/d11/d15/f72 0 2024-02-28T23:07:34.743 INFO:tasks.workunit.client.0.smithi057.stdout:2/327: rmdir d8/d11/d48/d50 39 2024-02-28T23:07:34.744 INFO:tasks.workunit.client.1.smithi084.stdout:8/395: write d0/d24/d4d/f58 [3909813,54759] 0 2024-02-28T23:07:34.747 INFO:tasks.workunit.client.1.smithi084.stdout:5/463: symlink dc/de/df/d16/d34/d3c/d98/lac 0 2024-02-28T23:07:34.751 INFO:tasks.workunit.client.0.smithi057.stdout:0/312: mkdir d17/d19/d4d/d7b 0 2024-02-28T23:07:34.751 INFO:tasks.workunit.client.0.smithi057.stdout:2/328: creat d8/d11/d21/d28/f73 x:0 0 0 2024-02-28T23:07:34.752 INFO:tasks.workunit.client.1.smithi084.stdout:4/393: truncate d1/d18/f42 4946947 0 2024-02-28T23:07:34.752 INFO:tasks.workunit.client.1.smithi084.stdout:5/464: creat dc/fad x:0 0 0 2024-02-28T23:07:34.753 INFO:tasks.workunit.client.1.smithi084.stdout:4/394: symlink d1/d18/l83 0 2024-02-28T23:07:34.754 INFO:tasks.workunit.client.0.smithi057.stdout:7/313: rename d0/d5/f27 to d0/d5/d36/d4e/d15/f60 0 2024-02-28T23:07:34.757 INFO:tasks.workunit.client.1.smithi084.stdout:9/463: write d4/d6/d15/d12/f3d [1627748,45884] 0 2024-02-28T23:07:34.759 INFO:tasks.workunit.client.0.smithi057.stdout:7/314: mknod d0/d5/d36/d4e/d3b/c61 0 2024-02-28T23:07:34.760 INFO:tasks.workunit.client.1.smithi084.stdout:2/355: write d5/d7/d12/d24/d2a/f52 [359035,64037] 0 2024-02-28T23:07:34.760 INFO:tasks.workunit.client.1.smithi084.stdout:5/465: mkdir dc/de/df/d16/d34/daa/dae 0 2024-02-28T23:07:34.760 INFO:tasks.workunit.client.0.smithi057.stdout:7/315: dread - d0/d3f/f4c zero size 2024-02-28T23:07:34.761 INFO:tasks.workunit.client.1.smithi084.stdout:9/464: chown d4/d6/f72 3913 1 2024-02-28T23:07:34.762 INFO:tasks.workunit.client.0.smithi057.stdout:7/316: rmdir d0 39 2024-02-28T23:07:34.763 INFO:tasks.workunit.client.0.smithi057.stdout:9/370: dwrite dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:34.764 INFO:tasks.workunit.client.1.smithi084.stdout:3/425: write d3/fc [940560,35541] 0 2024-02-28T23:07:34.767 INFO:tasks.workunit.client.0.smithi057.stdout:7/317: rmdir d0/d3f 39 2024-02-28T23:07:34.769 INFO:tasks.workunit.client.1.smithi084.stdout:9/465: write d4/d6/d15/d12/d41/d55/f5b [496706,2933] 0 2024-02-28T23:07:34.769 INFO:tasks.workunit.client.0.smithi057.stdout:9/371: creat dd/de/d14/d48/f74 x:0 0 0 2024-02-28T23:07:34.770 INFO:tasks.workunit.client.1.smithi084.stdout:5/466: rmdir dc/de/df/d16/d34/daa 39 2024-02-28T23:07:34.770 INFO:tasks.workunit.client.1.smithi084.stdout:4/395: rename d1/d4b/f68 to d1/d18/d20/d24/f84 0 2024-02-28T23:07:34.770 INFO:tasks.workunit.client.1.smithi084.stdout:6/414: dwrite d1/d13/d2e/d41/f2f [0,4194304] 0 2024-02-28T23:07:34.771 INFO:tasks.workunit.client.0.smithi057.stdout:9/372: readlink dd/d1b/l20 0 2024-02-28T23:07:34.771 INFO:tasks.workunit.client.0.smithi057.stdout:2/329: dwrite d8/d11/d21/f5a [0,4194304] 0 2024-02-28T23:07:34.773 INFO:tasks.workunit.client.0.smithi057.stdout:2/330: stat d8/d11/d15 0 2024-02-28T23:07:34.775 INFO:tasks.workunit.client.1.smithi084.stdout:5/467: chown dc/de/d83/c87 1110233 1 2024-02-28T23:07:34.776 INFO:tasks.workunit.client.0.smithi057.stdout:7/318: rename d0/d5/f4b to d0/d5/d25/d3e/f62 0 2024-02-28T23:07:34.778 INFO:tasks.workunit.client.1.smithi084.stdout:3/426: mknod d3/d8/d1a/d29/d41/ca3 0 2024-02-28T23:07:34.779 INFO:tasks.workunit.client.1.smithi084.stdout:7/402: dwrite db/fd [0,4194304] 0 2024-02-28T23:07:34.780 INFO:tasks.workunit.client.0.smithi057.stdout:2/331: mknod d8/d11/d39/c74 0 2024-02-28T23:07:34.781 INFO:tasks.workunit.client.0.smithi057.stdout:2/332: write d8/f24 [1260847,109123] 0 2024-02-28T23:07:34.782 INFO:tasks.workunit.client.0.smithi057.stdout:7/319: creat d0/d5/d36/f63 x:0 0 0 2024-02-28T23:07:34.783 INFO:tasks.workunit.client.1.smithi084.stdout:4/396: mknod d1/c85 0 2024-02-28T23:07:34.783 INFO:tasks.workunit.client.1.smithi084.stdout:5/468: creat dc/de/df/d6e/faf x:0 0 0 2024-02-28T23:07:34.784 INFO:tasks.workunit.client.1.smithi084.stdout:6/415: mkdir d1/d88 0 2024-02-28T23:07:34.785 INFO:tasks.workunit.client.0.smithi057.stdout:9/373: creat dd/de/d4a/d60/f75 x:0 0 0 2024-02-28T23:07:34.792 INFO:tasks.workunit.client.0.smithi057.stdout:2/333: creat d8/d11/d21/d28/f75 x:0 0 0 2024-02-28T23:07:34.792 INFO:tasks.workunit.client.1.smithi084.stdout:7/403: mknod db/d10/d17/d3b/c78 0 2024-02-28T23:07:34.793 INFO:tasks.workunit.client.1.smithi084.stdout:5/469: fsync dc/f2a 0 2024-02-28T23:07:34.796 INFO:tasks.workunit.client.0.smithi057.stdout:2/334: write d8/f64 [258819,95170] 0 2024-02-28T23:07:34.799 INFO:tasks.workunit.client.0.smithi057.stdout:2/335: write d8/fd [2571029,41410] 0 2024-02-28T23:07:34.800 INFO:tasks.workunit.client.1.smithi084.stdout:4/397: mknod d1/d18/d20/d24/c86 0 2024-02-28T23:07:34.802 INFO:tasks.workunit.client.0.smithi057.stdout:2/336: chown d8/d11/d21/d28/d41/l58 5294 1 2024-02-28T23:07:34.804 INFO:tasks.workunit.client.0.smithi057.stdout:2/337: fsync d8/fb 0 2024-02-28T23:07:34.814 INFO:tasks.workunit.client.0.smithi057.stdout:2/338: getdents d8/d11/d21 0 2024-02-28T23:07:34.816 INFO:tasks.workunit.client.0.smithi057.stdout:2/339: chown d8/d11/d21/d28/d41/l6b 425513 1 2024-02-28T23:07:34.816 INFO:tasks.workunit.client.1.smithi084.stdout:4/398: mkdir d1/d18/d20/d24/d2f/d3e/d52/d87 0 2024-02-28T23:07:34.817 INFO:tasks.workunit.client.1.smithi084.stdout:6/416: dwrite d1/d9/d1d/d1e/f5b [0,4194304] 0 2024-02-28T23:07:34.817 INFO:tasks.workunit.client.0.smithi057.stdout:2/340: read - d8/d11/d21/d28/f3d zero size 2024-02-28T23:07:34.818 INFO:tasks.workunit.client.0.smithi057.stdout:7/320: rmdir d0/d5/d36 39 2024-02-28T23:07:34.819 INFO:tasks.workunit.client.1.smithi084.stdout:4/399: write d1/f6e [282569,36406] 0 2024-02-28T23:07:34.821 INFO:tasks.workunit.client.0.smithi057.stdout:9/374: getdents dd/de/d4a/d60 0 2024-02-28T23:07:34.822 INFO:tasks.workunit.client.0.smithi057.stdout:9/375: read - dd/f5b zero size 2024-02-28T23:07:34.823 INFO:tasks.workunit.client.0.smithi057.stdout:7/321: fsync d0/d5/d36/d4e/d3b/f59 0 2024-02-28T23:07:34.823 INFO:tasks.workunit.client.0.smithi057.stdout:2/341: symlink d8/d11/d48/d50/l76 0 2024-02-28T23:07:34.823 INFO:tasks.workunit.client.1.smithi084.stdout:7/404: dwrite db/d11/f4f [0,4194304] 0 2024-02-28T23:07:34.828 INFO:tasks.workunit.client.0.smithi057.stdout:7/322: creat d0/d5/d36/d4e/d15/d29/d3d/d51/f64 x:0 0 0 2024-02-28T23:07:34.830 INFO:tasks.workunit.client.1.smithi084.stdout:8/396: dread d0/d9/d1b/f46 [0,4194304] 0 2024-02-28T23:07:34.830 INFO:tasks.workunit.client.0.smithi057.stdout:7/323: chown d0/l34 15895434 1 2024-02-28T23:07:34.832 INFO:tasks.workunit.client.1.smithi084.stdout:9/466: truncate d4/d6/ff 6061887 0 2024-02-28T23:07:34.836 INFO:tasks.workunit.client.1.smithi084.stdout:7/405: truncate db/d11/f4f 4859184 0 2024-02-28T23:07:34.839 INFO:tasks.workunit.client.1.smithi084.stdout:7/406: readlink db/d10/d17/l38 0 2024-02-28T23:07:34.841 INFO:tasks.workunit.client.0.smithi057.stdout:2/342: dwrite d8/d11/f1d [0,4194304] 0 2024-02-28T23:07:34.842 INFO:tasks.workunit.client.0.smithi057.stdout:9/376: dwrite dd/de/d14/d48/d4b/d41/f70 [0,4194304] 0 2024-02-28T23:07:34.842 INFO:tasks.workunit.client.1.smithi084.stdout:8/397: read - d0/d9/d1b/d38/d43/d51/d7c/f89 zero size 2024-02-28T23:07:34.845 INFO:tasks.workunit.client.0.smithi057.stdout:2/343: write d8/d11/d48/f52 [441345,86250] 0 2024-02-28T23:07:34.845 INFO:tasks.workunit.client.1.smithi084.stdout:9/467: write d4/d6/d15/d12/d41/d55/f87 [934378,49457] 0 2024-02-28T23:07:34.848 INFO:tasks.workunit.client.0.smithi057.stdout:9/377: fsync dd/d1b/d40/d44/f62 0 2024-02-28T23:07:34.850 INFO:tasks.workunit.client.0.smithi057.stdout:2/344: mknod d8/d11/d48/d50/c77 0 2024-02-28T23:07:34.851 INFO:tasks.workunit.client.0.smithi057.stdout:2/345: fsync d8/f16 0 2024-02-28T23:07:34.853 INFO:tasks.workunit.client.0.smithi057.stdout:9/378: mkdir dd/de/d14/d48/d76 0 2024-02-28T23:07:34.853 INFO:tasks.workunit.client.1.smithi084.stdout:1/479: sync 2024-02-28T23:07:34.855 INFO:tasks.workunit.client.1.smithi084.stdout:5/470: dwrite dc/de/df/d16/f1b [12582912,4194304] 0 2024-02-28T23:07:34.855 INFO:tasks.workunit.client.1.smithi084.stdout:9/468: chown d4/d6/d15/d12/d41/f46 178257111 1 2024-02-28T23:07:34.856 INFO:tasks.workunit.client.0.smithi057.stdout:9/379: mkdir dd/de/d24/d63/d77 0 2024-02-28T23:07:34.863 INFO:tasks.workunit.client.1.smithi084.stdout:4/400: creat d1/d18/d20/d24/d2f/d3e/d52/d45/f88 x:0 0 0 2024-02-28T23:07:34.863 INFO:tasks.workunit.client.1.smithi084.stdout:2/356: dwrite d5/dd/f1c [0,4194304] 0 2024-02-28T23:07:34.863 INFO:tasks.workunit.client.1.smithi084.stdout:8/398: mknod d0/d24/c8c 0 2024-02-28T23:07:34.865 INFO:tasks.workunit.client.0.smithi057.stdout:2/346: dwrite d8/f19 [0,4194304] 0 2024-02-28T23:07:34.868 INFO:tasks.workunit.client.0.smithi057.stdout:2/347: dread d8/d11/d15/f2d [0,4194304] 0 2024-02-28T23:07:34.871 INFO:tasks.workunit.client.0.smithi057.stdout:2/348: creat d8/d11/d15/f78 x:0 0 0 2024-02-28T23:07:34.873 INFO:tasks.workunit.client.0.smithi057.stdout:2/349: write d8/fb [893342,88524] 0 2024-02-28T23:07:34.874 INFO:tasks.workunit.client.1.smithi084.stdout:1/480: mknod d12/d1b/d63/d7d/c9f 0 2024-02-28T23:07:34.874 INFO:tasks.workunit.client.1.smithi084.stdout:6/417: link d1/f1b d1/d9/d1d/d1e/d33/f89 0 2024-02-28T23:07:34.874 INFO:tasks.workunit.client.0.smithi057.stdout:2/350: stat d8/l6c 0 2024-02-28T23:07:34.875 INFO:tasks.workunit.client.1.smithi084.stdout:4/401: symlink d1/d18/d20/l89 0 2024-02-28T23:07:34.876 INFO:tasks.workunit.client.0.smithi057.stdout:2/351: chown d8/d11/d21/d28/f73 561788 1 2024-02-28T23:07:34.878 INFO:tasks.workunit.client.1.smithi084.stdout:1/481: write f1 [1279469,1184] 0 2024-02-28T23:07:34.881 INFO:tasks.workunit.client.1.smithi084.stdout:4/402: stat d1/d18/f5f 0 2024-02-28T23:07:34.884 INFO:tasks.workunit.client.1.smithi084.stdout:5/471: unlink dc/de/df/d16/d34/d54/d5f/d66/c6d 0 2024-02-28T23:07:34.890 INFO:tasks.workunit.client.1.smithi084.stdout:5/472: dread - dc/de/df/d16/fa0 zero size 2024-02-28T23:07:34.891 INFO:tasks.workunit.client.0.smithi057.stdout:9/380: dread dd/f12 [0,4194304] 0 2024-02-28T23:07:34.892 INFO:tasks.workunit.client.1.smithi084.stdout:1/482: fsync d12/d1b/d2e/d30/f26 0 2024-02-28T23:07:34.898 INFO:tasks.workunit.client.1.smithi084.stdout:1/483: chown d12/d1b/d2e/d30/d77/l96 1768 1 2024-02-28T23:07:34.900 INFO:tasks.workunit.client.1.smithi084.stdout:9/469: dwrite d4/f51 [4194304,4194304] 0 2024-02-28T23:07:34.900 INFO:tasks.workunit.client.1.smithi084.stdout:8/399: dread d0/d11/f3c [0,4194304] 0 2024-02-28T23:07:34.903 INFO:tasks.workunit.client.1.smithi084.stdout:6/418: symlink d1/d88/l8a 0 2024-02-28T23:07:34.913 INFO:tasks.workunit.client.0.smithi057.stdout:9/381: truncate dd/f2f 694232 0 2024-02-28T23:07:34.918 INFO:tasks.workunit.client.0.smithi057.stdout:9/382: read dd/f31 [1710840,82774] 0 2024-02-28T23:07:34.923 INFO:tasks.workunit.client.1.smithi084.stdout:9/470: creat d4/d6/f93 x:0 0 0 2024-02-28T23:07:34.925 INFO:tasks.workunit.client.1.smithi084.stdout:6/419: rename d1/d9/c52 to d1/d9/d1d/d1e/d33/c8b 0 2024-02-28T23:07:34.928 INFO:tasks.workunit.client.1.smithi084.stdout:9/471: fdatasync d4/d6/d15/f2a 0 2024-02-28T23:07:34.930 INFO:tasks.workunit.client.1.smithi084.stdout:6/420: symlink d1/d13/d2e/d41/d2c/l8c 0 2024-02-28T23:07:34.936 INFO:tasks.workunit.client.1.smithi084.stdout:6/421: rename d1/d13/d50/f80 to d1/d13/d50/f8d 0 2024-02-28T23:07:34.937 INFO:tasks.workunit.client.1.smithi084.stdout:2/357: dread d5/f15 [0,4194304] 0 2024-02-28T23:07:34.943 INFO:tasks.workunit.client.1.smithi084.stdout:2/358: symlink d5/d1f/d42/l61 0 2024-02-28T23:07:34.947 INFO:tasks.workunit.client.1.smithi084.stdout:2/359: chown d5/d7/c4b 351003109 1 2024-02-28T23:07:34.947 INFO:tasks.workunit.client.1.smithi084.stdout:6/422: rename d1/d13/d2e/c73 to d1/d13/d2e/d41/c8e 0 2024-02-28T23:07:34.949 INFO:tasks.workunit.client.1.smithi084.stdout:0/429: sync 2024-02-28T23:07:34.951 INFO:tasks.workunit.client.1.smithi084.stdout:8/400: truncate d0/d11/f3c 2906079 0 2024-02-28T23:07:34.952 INFO:tasks.workunit.client.1.smithi084.stdout:7/407: dwrite db/d10/d18/f1b [0,4194304] 0 2024-02-28T23:07:34.961 INFO:tasks.workunit.client.1.smithi084.stdout:0/430: mkdir d1/d8/d78/d8f 0 2024-02-28T23:07:34.964 INFO:tasks.workunit.client.1.smithi084.stdout:2/360: mknod d5/d7/d12/d4f/d5b/c62 0 2024-02-28T23:07:34.965 INFO:tasks.workunit.client.1.smithi084.stdout:9/472: dwrite d4/d6/f7 [8388608,4194304] 0 2024-02-28T23:07:34.969 INFO:tasks.workunit.client.1.smithi084.stdout:7/408: symlink db/d10/d17/d64/d77/l79 0 2024-02-28T23:07:34.971 INFO:tasks.workunit.client.1.smithi084.stdout:8/401: read d0/d9/d1b/d38/f3a [267526,69515] 0 2024-02-28T23:07:34.975 INFO:tasks.workunit.client.1.smithi084.stdout:2/361: symlink d5/dd/l63 0 2024-02-28T23:07:34.975 INFO:tasks.workunit.client.1.smithi084.stdout:1/484: dwrite d12/d1b/f31 [4194304,4194304] 0 2024-02-28T23:07:34.975 INFO:tasks.workunit.client.1.smithi084.stdout:5/473: dwrite dc/de/df/d31/d8e/f52 [4194304,4194304] 0 2024-02-28T23:07:34.981 INFO:tasks.workunit.client.1.smithi084.stdout:7/409: stat db/d11/d27/d36 0 2024-02-28T23:07:34.981 INFO:tasks.workunit.client.1.smithi084.stdout:9/473: fdatasync d4/d6/d15/d12/d41/f7e 0 2024-02-28T23:07:34.981 INFO:tasks.workunit.client.1.smithi084.stdout:1/485: chown d12/d1b/d63/f76 51961 1 2024-02-28T23:07:34.985 INFO:tasks.workunit.client.1.smithi084.stdout:6/423: rename d1/d9/f87 to d1/d9/d1d/d1e/d33/f8f 0 2024-02-28T23:07:34.985 INFO:tasks.workunit.client.1.smithi084.stdout:2/362: mkdir d5/d64 0 2024-02-28T23:07:34.987 INFO:tasks.workunit.client.1.smithi084.stdout:1/486: chown d12/d1b/d3b/l81 150424 1 2024-02-28T23:07:34.992 INFO:tasks.workunit.client.0.smithi057.stdout:8/306: sync 2024-02-28T23:07:34.994 INFO:tasks.workunit.client.1.smithi084.stdout:4/403: dwrite d1/d18/d20/d24/f2a [0,4194304] 0 2024-02-28T23:07:34.999 INFO:tasks.workunit.client.0.smithi057.stdout:8/307: rename d4/d1e/d37/d3f to d4/d1e/d56/d6a 0 2024-02-28T23:07:35.002 INFO:tasks.workunit.client.1.smithi084.stdout:5/474: dwrite dc/d51/d7c/f84 [0,4194304] 0 2024-02-28T23:07:35.005 INFO:tasks.workunit.client.1.smithi084.stdout:9/474: unlink d4/d32/d3f/l63 0 2024-02-28T23:07:35.006 INFO:tasks.workunit.client.1.smithi084.stdout:2/363: dread d5/f17 [0,4194304] 0 2024-02-28T23:07:35.009 INFO:tasks.workunit.client.0.smithi057.stdout:2/352: dread d8/d11/d15/f20 [0,4194304] 0 2024-02-28T23:07:35.011 INFO:tasks.workunit.client.1.smithi084.stdout:0/431: rename d1/d6/d2d/l32 to d1/d6/db/l90 0 2024-02-28T23:07:35.014 INFO:tasks.workunit.client.0.smithi057.stdout:2/353: symlink d8/d11/d21/d28/l79 0 2024-02-28T23:07:35.015 INFO:tasks.workunit.client.1.smithi084.stdout:2/364: fdatasync d5/d7/d12/d24/d2a/d35/f5d 0 2024-02-28T23:07:35.015 INFO:tasks.workunit.client.1.smithi084.stdout:1/487: mkdir d12/d1b/d2e/d30/d77/da0 0 2024-02-28T23:07:35.023 INFO:tasks.workunit.client.1.smithi084.stdout:6/424: dwrite d1/d13/d2e/d41/f2f [0,4194304] 0 2024-02-28T23:07:35.023 INFO:tasks.workunit.client.1.smithi084.stdout:4/404: chown d1/d18/d20/d14/f39 495319 1 2024-02-28T23:07:35.027 INFO:tasks.workunit.client.0.smithi057.stdout:8/308: truncate d4/d1e/d56/d6a/d42/f5e 2941348 0 2024-02-28T23:07:35.029 INFO:tasks.workunit.client.1.smithi084.stdout:7/410: mkdir db/d11/d27/d7a 0 2024-02-28T23:07:35.030 INFO:tasks.workunit.client.1.smithi084.stdout:0/432: fdatasync d1/d6/d1b/f58 0 2024-02-28T23:07:35.031 INFO:tasks.workunit.client.0.smithi057.stdout:8/309: mkdir d4/d1e/d56/d6a/d54/d6b 0 2024-02-28T23:07:35.033 INFO:tasks.workunit.client.1.smithi084.stdout:2/365: rename d5/dd/c4c to d5/d7/d2e/c65 0 2024-02-28T23:07:35.034 INFO:tasks.workunit.client.1.smithi084.stdout:1/488: stat d12/d1b/d2e/d32/f5e 0 2024-02-28T23:07:35.034 INFO:tasks.workunit.client.1.smithi084.stdout:5/475: mkdir dc/de/df/db0 0 2024-02-28T23:07:35.036 INFO:tasks.workunit.client.1.smithi084.stdout:9/475: mknod d4/d6/d15/c94 0 2024-02-28T23:07:35.036 INFO:tasks.workunit.client.1.smithi084.stdout:1/489: chown d12/d1b/d63/d7d 266966819 1 2024-02-28T23:07:35.037 INFO:tasks.workunit.client.0.smithi057.stdout:8/310: creat d4/d1e/d56/d6a/d54/d6b/f6c x:0 0 0 2024-02-28T23:07:35.039 INFO:tasks.workunit.client.1.smithi084.stdout:7/411: creat db/d10/d17/d64/f7b x:0 0 0 2024-02-28T23:07:35.039 INFO:tasks.workunit.client.1.smithi084.stdout:2/366: write d5/d7/d12/d24/d2a/d35/f5d [710369,81063] 0 2024-02-28T23:07:35.042 INFO:tasks.workunit.client.1.smithi084.stdout:6/425: symlink d1/d9/l90 0 2024-02-28T23:07:35.049 INFO:tasks.workunit.client.1.smithi084.stdout:1/490: creat d12/d1b/d3b/fa1 x:0 0 0 2024-02-28T23:07:35.049 INFO:tasks.workunit.client.1.smithi084.stdout:2/367: creat d5/d7/d12/d24/f66 x:0 0 0 2024-02-28T23:07:35.051 INFO:tasks.workunit.client.1.smithi084.stdout:7/412: rename db/d10/f2f to db/d10/d17/d3b/f7c 0 2024-02-28T23:07:35.054 INFO:tasks.workunit.client.1.smithi084.stdout:0/433: creat d1/d8/d18/d4c/d6c/d74/f91 x:0 0 0 2024-02-28T23:07:35.054 INFO:tasks.workunit.client.1.smithi084.stdout:7/413: stat db/d10/d17/f3d 0 2024-02-28T23:07:35.055 INFO:tasks.workunit.client.1.smithi084.stdout:6/426: mkdir d1/d13/d47/d91 0 2024-02-28T23:07:35.060 INFO:tasks.workunit.client.1.smithi084.stdout:2/368: mknod d5/d7/d2e/d4a/d5c/c67 0 2024-02-28T23:07:35.064 INFO:tasks.workunit.client.1.smithi084.stdout:4/405: dwrite d1/d18/d20/d24/d2f/d3e/d52/d37/f5a [0,4194304] 0 2024-02-28T23:07:35.066 INFO:tasks.workunit.client.1.smithi084.stdout:1/491: mkdir d12/da2 0 2024-02-28T23:07:35.067 INFO:tasks.workunit.client.1.smithi084.stdout:9/476: dread d4/d6/f72 [0,4194304] 0 2024-02-28T23:07:35.068 INFO:tasks.workunit.client.1.smithi084.stdout:4/406: chown d1/d4a/f54 728355 1 2024-02-28T23:07:35.070 INFO:tasks.workunit.client.1.smithi084.stdout:6/427: rename d1/f31 to d1/d13/d2e/d41/f92 0 2024-02-28T23:07:35.072 INFO:tasks.workunit.client.1.smithi084.stdout:5/476: dwrite dc/de/df/d6e/faf [0,4194304] 0 2024-02-28T23:07:35.076 INFO:tasks.workunit.client.1.smithi084.stdout:7/414: mknod db/c7d 0 2024-02-28T23:07:35.079 INFO:tasks.workunit.client.1.smithi084.stdout:2/369: symlink d5/d7/d12/d4f/l68 0 2024-02-28T23:07:35.082 INFO:tasks.workunit.client.1.smithi084.stdout:2/370: truncate d5/d7/d2e/f46 839622 0 2024-02-28T23:07:35.084 INFO:tasks.workunit.client.1.smithi084.stdout:3/427: sync 2024-02-28T23:07:35.088 INFO:tasks.workunit.client.1.smithi084.stdout:1/492: creat d12/d1b/d91/d9b/fa3 x:0 0 0 2024-02-28T23:07:35.088 INFO:tasks.workunit.client.1.smithi084.stdout:3/428: write d3/d8/f24 [2721225,38843] 0 2024-02-28T23:07:35.090 INFO:tasks.workunit.client.1.smithi084.stdout:2/371: chown d5/d7/d12/d24/d3a/d48/f49 62162 1 2024-02-28T23:07:35.092 INFO:tasks.workunit.client.1.smithi084.stdout:6/428: creat d1/d13/d47/d91/f93 x:0 0 0 2024-02-28T23:07:35.096 INFO:tasks.workunit.client.1.smithi084.stdout:6/429: read d1/d13/d2e/d41/d2c/f78 [65215,76165] 0 2024-02-28T23:07:35.099 INFO:tasks.workunit.client.1.smithi084.stdout:0/434: write d1/d8/d18/f63 [5139588,77680] 0 2024-02-28T23:07:35.099 INFO:tasks.workunit.client.1.smithi084.stdout:9/477: rename d4/d6/d15/d12/l5e to d4/l95 0 2024-02-28T23:07:35.101 INFO:tasks.workunit.client.1.smithi084.stdout:2/372: mkdir d5/dd/d69 0 2024-02-28T23:07:35.102 INFO:tasks.workunit.client.1.smithi084.stdout:5/477: mkdir dc/de/df/d16/d34/daa/dae/db1 0 2024-02-28T23:07:35.103 INFO:tasks.workunit.client.1.smithi084.stdout:0/435: read - d1/d8/d69/d6b/f7a zero size 2024-02-28T23:07:35.103 INFO:tasks.workunit.client.1.smithi084.stdout:1/493: mknod d12/d1b/d2e/d52/d5b/ca4 0 2024-02-28T23:07:35.104 INFO:tasks.workunit.client.1.smithi084.stdout:3/429: mknod d3/d8/d1a/d29/d41/d60/ca4 0 2024-02-28T23:07:35.105 INFO:tasks.workunit.client.1.smithi084.stdout:1/494: chown d12/d1b/d63 152294 1 2024-02-28T23:07:35.105 INFO:tasks.workunit.client.1.smithi084.stdout:4/407: link d1/d18/d20/c2b d1/d18/d20/d24/d2f/d3e/d52/d57/c8a 0 2024-02-28T23:07:35.106 INFO:tasks.workunit.client.1.smithi084.stdout:5/478: write dc/de/df/d6e/faf [4273124,23749] 0 2024-02-28T23:07:35.110 INFO:tasks.workunit.client.1.smithi084.stdout:6/430: creat d1/d13/d50/d74/d81/f94 x:0 0 0 2024-02-28T23:07:35.111 INFO:tasks.workunit.client.1.smithi084.stdout:9/478: readlink d4/d6/d15/d12/d41/l76 0 2024-02-28T23:07:35.112 INFO:tasks.workunit.client.1.smithi084.stdout:2/373: symlink d5/d64/l6a 0 2024-02-28T23:07:35.119 INFO:tasks.workunit.client.1.smithi084.stdout:5/479: dread - dc/de/df/d16/d34/d54/d5f/d9b/f9c zero size 2024-02-28T23:07:35.120 INFO:tasks.workunit.client.1.smithi084.stdout:7/415: dwrite db/d11/f29 [0,4194304] 0 2024-02-28T23:07:35.120 INFO:tasks.workunit.client.0.smithi057.stdout:4/298: sync 2024-02-28T23:07:35.120 INFO:tasks.workunit.client.0.smithi057.stdout:6/359: sync 2024-02-28T23:07:35.121 INFO:tasks.workunit.client.0.smithi057.stdout:1/239: sync 2024-02-28T23:07:35.121 INFO:tasks.workunit.client.0.smithi057.stdout:5/268: sync 2024-02-28T23:07:35.121 INFO:tasks.workunit.client.1.smithi084.stdout:1/495: truncate d12/d1b/d2e/f4e 110142 0 2024-02-28T23:07:35.122 INFO:tasks.workunit.client.1.smithi084.stdout:7/416: chown db/d10 11 1 2024-02-28T23:07:35.122 INFO:tasks.workunit.client.1.smithi084.stdout:8/402: sync 2024-02-28T23:07:35.123 INFO:tasks.workunit.client.0.smithi057.stdout:4/299: write d1a/f37 [1065689,115589] 0 2024-02-28T23:07:35.125 INFO:tasks.workunit.client.0.smithi057.stdout:5/269: read - d4/d21/f37 zero size 2024-02-28T23:07:35.128 INFO:tasks.workunit.client.1.smithi084.stdout:6/431: rename d1/d9/d1d/d1e/d33/f64 to d1/d13/d50/d65/f95 0 2024-02-28T23:07:35.128 INFO:tasks.workunit.client.0.smithi057.stdout:1/240: chown d2/d7/df/db/d12/d19/d24/d2d/f31 125 1 2024-02-28T23:07:35.128 INFO:tasks.workunit.client.1.smithi084.stdout:4/408: creat d1/d18/d20/d24/f8b x:0 0 0 2024-02-28T23:07:35.130 INFO:tasks.workunit.client.0.smithi057.stdout:6/360: creat d0/d1b/d3e/d3d/d74/f77 x:0 0 0 2024-02-28T23:07:35.132 INFO:tasks.workunit.client.1.smithi084.stdout:5/480: creat dc/de/df/d16/d34/d54/fb2 x:0 0 0 2024-02-28T23:07:35.132 INFO:tasks.workunit.client.1.smithi084.stdout:1/496: mkdir d12/d1b/d91/d9b/da5 0 2024-02-28T23:07:35.134 INFO:tasks.workunit.client.1.smithi084.stdout:3/430: dread d3/f6 [4194304,4194304] 0 2024-02-28T23:07:35.134 INFO:tasks.workunit.client.0.smithi057.stdout:4/300: mkdir d1a/d1b/d24/d68 0 2024-02-28T23:07:35.135 INFO:tasks.workunit.client.1.smithi084.stdout:8/403: creat d0/d9/d30/d33/d71/d84/f8d x:0 0 0 2024-02-28T23:07:35.136 INFO:tasks.workunit.client.1.smithi084.stdout:1/497: chown d12/d1b/d3b/l81 139 1 2024-02-28T23:07:35.137 INFO:tasks.workunit.client.1.smithi084.stdout:1/498: readlink lb 0 2024-02-28T23:07:35.138 INFO:tasks.workunit.client.0.smithi057.stdout:6/361: truncate d0/d1b/d3e/d3d/d42/d4f/d53/f5b 109406 0 2024-02-28T23:07:35.139 INFO:tasks.workunit.client.1.smithi084.stdout:9/479: getdents d4/d6/d15 0 2024-02-28T23:07:35.141 INFO:tasks.workunit.client.1.smithi084.stdout:5/481: creat dc/de/df/d31/d8e/da9/fb3 x:0 0 0 2024-02-28T23:07:35.143 INFO:tasks.workunit.client.1.smithi084.stdout:7/417: link db/l4e db/d5e/l7e 0 2024-02-28T23:07:35.144 INFO:tasks.workunit.client.0.smithi057.stdout:5/270: link d4/d21/d24/l27 d4/d7/l4c 0 2024-02-28T23:07:35.144 INFO:tasks.workunit.client.0.smithi057.stdout:4/301: creat d1a/d48/f69 x:0 0 0 2024-02-28T23:07:35.144 INFO:tasks.workunit.client.1.smithi084.stdout:1/499: symlink d12/d1b/d2e/d30/d77/la6 0 2024-02-28T23:07:35.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/362: rename d0/d17/d20/f6b to d0/d4/d11/f78 0 2024-02-28T23:07:35.146 INFO:tasks.workunit.client.1.smithi084.stdout:7/418: write db/d10/f56 [1328211,74161] 0 2024-02-28T23:07:35.148 INFO:tasks.workunit.client.1.smithi084.stdout:0/436: dread d1/d8/f3e [0,4194304] 0 2024-02-28T23:07:35.148 INFO:tasks.workunit.client.1.smithi084.stdout:7/419: stat db/d11/d27/d36/f5a 0 2024-02-28T23:07:35.149 INFO:tasks.workunit.client.0.smithi057.stdout:5/271: symlink d4/d7/l4d 0 2024-02-28T23:07:35.151 INFO:tasks.workunit.client.1.smithi084.stdout:5/482: write dc/de/df/d16/f29 [1034366,583] 0 2024-02-28T23:07:35.151 INFO:tasks.workunit.client.0.smithi057.stdout:6/363: truncate d0/d1b/d3e/d3d/f62 632476 0 2024-02-28T23:07:35.152 INFO:tasks.workunit.client.0.smithi057.stdout:4/302: rename d1a/d1b/d2d/d49/l4c to d1a/d1e/d51/l6a 0 2024-02-28T23:07:35.152 INFO:tasks.workunit.client.1.smithi084.stdout:7/420: dread - db/d10/d17/d64/f7b zero size 2024-02-28T23:07:35.153 INFO:tasks.workunit.client.1.smithi084.stdout:9/480: creat d4/d32/d3f/f96 x:0 0 0 2024-02-28T23:07:35.154 INFO:tasks.workunit.client.1.smithi084.stdout:5/483: chown dc/f2a 20480389 1 2024-02-28T23:07:35.160 INFO:tasks.workunit.client.0.smithi057.stdout:4/303: rename d1a/d48/f69 to d1a/d1e/f6b 0 2024-02-28T23:07:35.160 INFO:tasks.workunit.client.0.smithi057.stdout:6/364: rename d0/d1b/d3e/d40/c41 to d0/d1b/d3e/d40/c79 0 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.1.smithi084.stdout:0/437: rmdir d1/d6/db/d35/d82 39 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.1.smithi084.stdout:7/421: truncate db/d10/d18/f1b 4509400 0 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.1.smithi084.stdout:8/404: getdents d0/d24 0 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.1.smithi084.stdout:5/484: chown dc/de/df/d31/d8e/c99 0 1 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.1.smithi084.stdout:1/500: symlink d12/da2/la7 0 2024-02-28T23:07:35.161 INFO:tasks.workunit.client.0.smithi057.stdout:4/304: mkdir d1a/d48/d6c 0 2024-02-28T23:07:35.163 INFO:tasks.workunit.client.0.smithi057.stdout:6/365: creat d0/d1b/f7a x:0 0 0 2024-02-28T23:07:35.163 INFO:tasks.workunit.client.1.smithi084.stdout:0/438: write d1/d8/f3c [2015662,67301] 0 2024-02-28T23:07:35.163 INFO:tasks.workunit.client.1.smithi084.stdout:7/422: write db/d10/d17/d64/f75 [912870,15551] 0 2024-02-28T23:07:35.164 INFO:tasks.workunit.client.0.smithi057.stdout:4/305: write f0 [3716776,5839] 0 2024-02-28T23:07:35.167 INFO:tasks.workunit.client.1.smithi084.stdout:6/432: dwrite d1/d9/d1d/d1e/d33/f89 [4194304,4194304] 0 2024-02-28T23:07:35.168 INFO:tasks.workunit.client.1.smithi084.stdout:1/501: dread - d12/d1b/d2e/d30/f3e zero size 2024-02-28T23:07:35.169 INFO:tasks.workunit.client.1.smithi084.stdout:9/481: creat d4/d6/d15/d12/f97 x:0 0 0 2024-02-28T23:07:35.169 INFO:tasks.workunit.client.1.smithi084.stdout:8/405: write d0/d9/d63/f7e [387091,94192] 0 2024-02-28T23:07:35.170 INFO:tasks.workunit.client.0.smithi057.stdout:6/366: creat d0/d1b/d3e/f7b x:0 0 0 2024-02-28T23:07:35.171 INFO:tasks.workunit.client.1.smithi084.stdout:0/439: mkdir d1/d6/db/d35/d44/d92 0 2024-02-28T23:07:35.171 INFO:tasks.workunit.client.1.smithi084.stdout:6/433: stat d1/d13/d50/d65 0 2024-02-28T23:07:35.172 INFO:tasks.workunit.client.1.smithi084.stdout:7/423: chown db/d11/d27/f3a 1593 1 2024-02-28T23:07:35.172 INFO:tasks.workunit.client.1.smithi084.stdout:3/431: dread d3/d8/d14/f49 [0,4194304] 0 2024-02-28T23:07:35.173 INFO:tasks.workunit.client.1.smithi084.stdout:5/485: creat dc/de/df/d31/d8e/da9/dab/fb4 x:0 0 0 2024-02-28T23:07:35.175 INFO:tasks.workunit.client.1.smithi084.stdout:9/482: fdatasync d4/f51 0 2024-02-28T23:07:35.175 INFO:tasks.workunit.client.0.smithi057.stdout:1/241: dread d2/d7/df/db/d12/d19/d24/f34 [0,4194304] 0 2024-02-28T23:07:35.175 INFO:tasks.workunit.client.1.smithi084.stdout:8/406: chown d0/d11/l26 456 1 2024-02-28T23:07:35.176 INFO:tasks.workunit.client.0.smithi057.stdout:1/242: readlink d2/l53 0 2024-02-28T23:07:35.177 INFO:tasks.workunit.client.1.smithi084.stdout:1/502: mkdir d12/d1b/d2e/d30/d50/da8 0 2024-02-28T23:07:35.178 INFO:tasks.workunit.client.0.smithi057.stdout:1/243: write d2/d7/df/db/d3a/f36 [781978,47644] 0 2024-02-28T23:07:35.179 INFO:tasks.workunit.client.1.smithi084.stdout:1/503: chown d12/d1b/d63/d7d/f8c 22708040 1 2024-02-28T23:07:35.180 INFO:tasks.workunit.client.1.smithi084.stdout:8/407: write d0/d9/d30/d33/f5e [137163,126095] 0 2024-02-28T23:07:35.181 INFO:tasks.workunit.client.1.smithi084.stdout:9/483: write d4/d6/d15/d12/f81 [855025,125999] 0 2024-02-28T23:07:35.182 INFO:tasks.workunit.client.0.smithi057.stdout:1/244: write d2/d7/df/db/d12/f26 [1234986,40195] 0 2024-02-28T23:07:35.183 INFO:tasks.workunit.client.0.smithi057.stdout:1/245: fdatasync d2/d7/df/f52 0 2024-02-28T23:07:35.185 INFO:tasks.workunit.client.0.smithi057.stdout:1/246: dread - d2/d7/f37 zero size 2024-02-28T23:07:35.187 INFO:tasks.workunit.client.0.smithi057.stdout:1/247: dread - d2/d7/df/db/d3a/f3c zero size 2024-02-28T23:07:35.188 INFO:tasks.workunit.client.1.smithi084.stdout:5/486: write dc/de/df/d16/f45 [1707384,121749] 0 2024-02-28T23:07:35.191 INFO:tasks.workunit.client.1.smithi084.stdout:5/487: write dc/f2a [1977643,18715] 0 2024-02-28T23:07:35.195 INFO:tasks.workunit.client.1.smithi084.stdout:5/488: write dc/f27 [70296,122541] 0 2024-02-28T23:07:35.197 INFO:tasks.workunit.client.1.smithi084.stdout:1/504: creat d12/da2/fa9 x:0 0 0 2024-02-28T23:07:35.198 INFO:tasks.workunit.client.1.smithi084.stdout:5/489: stat dc/de/d9e 0 2024-02-28T23:07:35.199 INFO:tasks.workunit.client.1.smithi084.stdout:1/505: write f1 [3445050,2592] 0 2024-02-28T23:07:35.199 INFO:tasks.workunit.client.1.smithi084.stdout:6/434: symlink d1/d9/d1d/d1e/d33/l96 0 2024-02-28T23:07:35.201 INFO:tasks.workunit.client.1.smithi084.stdout:0/440: dwrite d1/d8/d69/d6b/f77 [0,4194304] 0 2024-02-28T23:07:35.204 INFO:tasks.workunit.client.0.smithi057.stdout:4/306: rename d1a/d1e/d51/l6a to d1a/d1e/d4b/l6d 0 2024-02-28T23:07:35.205 INFO:tasks.workunit.client.0.smithi057.stdout:1/248: dwrite d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:35.209 INFO:tasks.workunit.client.0.smithi057.stdout:4/307: mkdir d1a/d1e/d4b/d4f/d6e 0 2024-02-28T23:07:35.209 INFO:tasks.workunit.client.1.smithi084.stdout:3/432: write d3/d8/df/f19 [43139,117315] 0 2024-02-28T23:07:35.211 INFO:tasks.workunit.client.1.smithi084.stdout:5/490: mknod dc/d51/cb5 0 2024-02-28T23:07:35.212 INFO:tasks.workunit.client.0.smithi057.stdout:6/367: rename d0/d4/c36 to d0/d4/d11/c7c 0 2024-02-28T23:07:35.213 INFO:tasks.workunit.client.0.smithi057.stdout:1/249: mkdir d2/d33/d54 0 2024-02-28T23:07:35.215 INFO:tasks.workunit.client.0.smithi057.stdout:4/308: truncate d1a/d1b/d2d/f3c 4804348 0 2024-02-28T23:07:35.215 INFO:tasks.workunit.client.1.smithi084.stdout:8/408: mknod d0/d24/d83/c8e 0 2024-02-28T23:07:35.217 INFO:tasks.workunit.client.0.smithi057.stdout:6/368: mkdir d0/d1b/d3e/d3d/d7d 0 2024-02-28T23:07:35.218 INFO:tasks.workunit.client.0.smithi057.stdout:1/250: creat d2/d7/df/db/f55 x:0 0 0 2024-02-28T23:07:35.219 INFO:tasks.workunit.client.1.smithi084.stdout:1/506: fsync d12/d1b/d2e/d52/d5b/f62 0 2024-02-28T23:07:35.220 INFO:tasks.workunit.client.1.smithi084.stdout:5/491: read - dc/de/df/d16/d34/f53 zero size 2024-02-28T23:07:35.220 INFO:tasks.workunit.client.0.smithi057.stdout:4/309: mknod d1a/d1e/d4b/c6f 0 2024-02-28T23:07:35.220 INFO:tasks.workunit.client.0.smithi057.stdout:6/369: read - d0/d17/d20/f64 zero size 2024-02-28T23:07:35.223 INFO:tasks.workunit.client.1.smithi084.stdout:3/433: creat d3/d8/d14/fa5 x:0 0 0 2024-02-28T23:07:35.223 INFO:tasks.workunit.client.0.smithi057.stdout:1/251: symlink d2/d7/df/db/d3a/d2a/l56 0 2024-02-28T23:07:35.224 INFO:tasks.workunit.client.0.smithi057.stdout:0/313: sync 2024-02-28T23:07:35.224 INFO:tasks.workunit.client.0.smithi057.stdout:3/240: sync 2024-02-28T23:07:35.224 INFO:tasks.workunit.client.0.smithi057.stdout:9/383: sync 2024-02-28T23:07:35.225 INFO:tasks.workunit.client.1.smithi084.stdout:7/424: getdents db/d10/d18 0 2024-02-28T23:07:35.226 INFO:tasks.workunit.client.0.smithi057.stdout:0/314: read d17/f58 [2742028,23582] 0 2024-02-28T23:07:35.227 INFO:tasks.workunit.client.1.smithi084.stdout:6/435: rename d1/d9/d1d/d1e/l43 to d1/d13/d2e/d41/l97 0 2024-02-28T23:07:35.233 INFO:tasks.workunit.client.0.smithi057.stdout:4/310: getdents d1a/d1b/d2d 0 2024-02-28T23:07:35.235 INFO:tasks.workunit.client.1.smithi084.stdout:6/436: mknod d1/d13/d2e/c98 0 2024-02-28T23:07:35.235 INFO:tasks.workunit.client.1.smithi084.stdout:0/441: dwrite d1/d8/d18/f61 [0,4194304] 0 2024-02-28T23:07:35.235 INFO:tasks.workunit.client.0.smithi057.stdout:4/311: mknod d1a/d48/c70 0 2024-02-28T23:07:35.236 INFO:tasks.workunit.client.0.smithi057.stdout:4/312: readlink d1a/d1b/d24/l65 0 2024-02-28T23:07:35.239 INFO:tasks.workunit.client.1.smithi084.stdout:3/434: symlink d3/d8/d1a/d29/d41/la6 0 2024-02-28T23:07:35.243 INFO:tasks.workunit.client.1.smithi084.stdout:8/409: link d0/d11/f2a d0/d11/f8f 0 2024-02-28T23:07:35.246 INFO:tasks.workunit.client.1.smithi084.stdout:8/410: fsync d0/d24/f5b 0 2024-02-28T23:07:35.247 INFO:tasks.workunit.client.1.smithi084.stdout:6/437: creat d1/d13/d50/f99 x:0 0 0 2024-02-28T23:07:35.251 INFO:tasks.workunit.client.0.smithi057.stdout:4/313: dwrite d1a/d1b/f46 [0,4194304] 0 2024-02-28T23:07:35.251 INFO:tasks.workunit.client.0.smithi057.stdout:6/370: write d0/d4/d11/f2d [2748715,3606] 0 2024-02-28T23:07:35.255 INFO:tasks.workunit.client.1.smithi084.stdout:5/492: dwrite dc/de/df/d16/f95 [0,4194304] 0 2024-02-28T23:07:35.255 INFO:tasks.workunit.client.0.smithi057.stdout:6/371: dread - d0/d1b/d3e/d40/f73 zero size 2024-02-28T23:07:35.256 INFO:tasks.workunit.client.0.smithi057.stdout:4/314: fdatasync f15 0 2024-02-28T23:07:35.258 INFO:tasks.workunit.client.1.smithi084.stdout:7/425: dwrite db/d10/d17/d3b/f7c [0,4194304] 0 2024-02-28T23:07:35.259 INFO:tasks.workunit.client.0.smithi057.stdout:3/241: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:35.260 INFO:tasks.workunit.client.0.smithi057.stdout:4/315: dread - d1a/d1b/d2d/f39 zero size 2024-02-28T23:07:35.261 INFO:tasks.workunit.client.0.smithi057.stdout:6/372: chown d0/d1b/d3e/d3d/d42/d4f/d53/c6e 435 1 2024-02-28T23:07:35.264 INFO:tasks.workunit.client.1.smithi084.stdout:6/438: creat d1/d9/d1d/d1e/d33/d38/f9a x:0 0 0 2024-02-28T23:07:35.265 INFO:tasks.workunit.client.1.smithi084.stdout:1/507: write d12/d1b/d63/f76 [3579105,25729] 0 2024-02-28T23:07:35.267 INFO:tasks.workunit.client.0.smithi057.stdout:9/384: dwrite dd/f31 [0,4194304] 0 2024-02-28T23:07:35.267 INFO:tasks.workunit.client.1.smithi084.stdout:0/442: getdents d1/d6/db/d35/d57 0 2024-02-28T23:07:35.273 INFO:tasks.workunit.client.0.smithi057.stdout:4/316: rename d1a/f37 to d1a/d48/f71 0 2024-02-28T23:07:35.273 INFO:tasks.workunit.client.0.smithi057.stdout:9/385: dread - dd/de/d14/d48/f74 zero size 2024-02-28T23:07:35.274 INFO:tasks.workunit.client.1.smithi084.stdout:6/439: creat d1/d13/d50/d65/f9b x:0 0 0 2024-02-28T23:07:35.274 INFO:tasks.workunit.client.0.smithi057.stdout:9/386: truncate dd/de/d14/f51 294943 0 2024-02-28T23:07:35.275 INFO:tasks.workunit.client.1.smithi084.stdout:3/435: dwrite d3/d8/d1a/f47 [0,4194304] 0 2024-02-28T23:07:35.277 INFO:tasks.workunit.client.0.smithi057.stdout:9/387: read dd/d1b/d40/d44/f62 [2784049,24603] 0 2024-02-28T23:07:35.277 INFO:tasks.workunit.client.0.smithi057.stdout:4/317: rename d1a/d1e/d4b/c6f to d1a/d1b/d24/d2e/d43/c72 0 2024-02-28T23:07:35.277 INFO:tasks.workunit.client.0.smithi057.stdout:6/373: link d0/d4/d25/f71 d0/d4/d25/d60/d6a/f7e 0 2024-02-28T23:07:35.282 INFO:tasks.workunit.client.1.smithi084.stdout:7/426: rename db/d10/d17/d3b/l45 to db/d10/d17/l7f 0 2024-02-28T23:07:35.282 INFO:tasks.workunit.client.0.smithi057.stdout:1/252: truncate d2/d7/df/db/f17 1208148 0 2024-02-28T23:07:35.283 INFO:tasks.workunit.client.1.smithi084.stdout:7/427: mkdir db/d5e/d80 0 2024-02-28T23:07:35.286 INFO:tasks.workunit.client.0.smithi057.stdout:6/374: symlink d0/d1b/d3e/d3d/d42/d4f/l7f 0 2024-02-28T23:07:35.287 INFO:tasks.workunit.client.0.smithi057.stdout:9/388: rmdir dd/d46 39 2024-02-28T23:07:35.288 INFO:tasks.workunit.client.1.smithi084.stdout:6/440: unlink d1/d13/d2e/l7b 0 2024-02-28T23:07:35.289 INFO:tasks.workunit.client.0.smithi057.stdout:1/253: mknod d2/d7/df/db/d12/d19/d48/c57 0 2024-02-28T23:07:35.290 INFO:tasks.workunit.client.1.smithi084.stdout:7/428: write db/d10/f5d [1012765,65711] 0 2024-02-28T23:07:35.291 INFO:tasks.workunit.client.0.smithi057.stdout:9/389: rename dd/de/d14 to dd/de/d14/d48/d4b/d78 22 2024-02-28T23:07:35.292 INFO:tasks.workunit.client.0.smithi057.stdout:6/375: creat d0/d1b/d3e/d3d/d42/d4f/d53/f80 x:0 0 0 2024-02-28T23:07:35.293 INFO:tasks.workunit.client.1.smithi084.stdout:3/436: rename d3/d8/df/f9e to d3/d8/d1a/d29/d7e/fa7 0 2024-02-28T23:07:35.295 INFO:tasks.workunit.client.1.smithi084.stdout:6/441: stat d1/d13/d2e/d41/d4d/f63 0 2024-02-28T23:07:35.296 INFO:tasks.workunit.client.0.smithi057.stdout:1/254: chown d2/d7/df/db/d12/d19/d48/c57 1790622146 1 2024-02-28T23:07:35.300 INFO:tasks.workunit.client.0.smithi057.stdout:4/318: dwrite d1a/d1b/d24/d2e/f3f [0,4194304] 0 2024-02-28T23:07:35.300 INFO:tasks.workunit.client.0.smithi057.stdout:1/255: write d2/d7/df/db/d12/f26 [5469262,52514] 0 2024-02-28T23:07:35.300 INFO:tasks.workunit.client.0.smithi057.stdout:9/390: creat dd/d46/d6f/f79 x:0 0 0 2024-02-28T23:07:35.303 INFO:tasks.workunit.client.1.smithi084.stdout:8/411: dwrite d0/d9/d1b/d20/f25 [0,4194304] 0 2024-02-28T23:07:35.304 INFO:tasks.workunit.client.0.smithi057.stdout:9/391: fsync dd/de/d14/d48/d4b/d41/f70 0 2024-02-28T23:07:35.305 INFO:tasks.workunit.client.0.smithi057.stdout:5/272: sync 2024-02-28T23:07:35.305 INFO:tasks.workunit.client.0.smithi057.stdout:7/324: sync 2024-02-28T23:07:35.305 INFO:tasks.workunit.client.0.smithi057.stdout:2/354: sync 2024-02-28T23:07:35.305 INFO:tasks.workunit.client.0.smithi057.stdout:8/311: sync 2024-02-28T23:07:35.305 INFO:tasks.workunit.client.0.smithi057.stdout:7/325: stat d0/d5/d36/d4e/la 0 2024-02-28T23:07:35.306 INFO:tasks.workunit.client.0.smithi057.stdout:1/256: symlink d2/d7/df/l58 0 2024-02-28T23:07:35.308 INFO:tasks.workunit.client.0.smithi057.stdout:2/355: stat d8/d11/d21/d28/c2a 0 2024-02-28T23:07:35.309 INFO:tasks.workunit.client.1.smithi084.stdout:8/412: fdatasync d0/d11/f23 0 2024-02-28T23:07:35.309 INFO:tasks.workunit.client.1.smithi084.stdout:3/437: dread - d3/d27/d4b/f55 zero size 2024-02-28T23:07:35.309 INFO:tasks.workunit.client.1.smithi084.stdout:6/442: mknod d1/d13/d2e/d41/d2c/c9c 0 2024-02-28T23:07:35.312 INFO:tasks.workunit.client.1.smithi084.stdout:8/413: chown d0/d9/d1b/d38/f3e 7956144 1 2024-02-28T23:07:35.313 INFO:tasks.workunit.client.0.smithi057.stdout:9/392: rename dd/ff to dd/d1b/d40/d44/f7a 0 2024-02-28T23:07:35.314 INFO:tasks.workunit.client.0.smithi057.stdout:8/312: creat d4/d1e/d56/d6a/f6d x:0 0 0 2024-02-28T23:07:35.315 INFO:tasks.workunit.client.0.smithi057.stdout:5/273: mkdir d4/d9/d16/d4e 0 2024-02-28T23:07:35.317 INFO:tasks.workunit.client.1.smithi084.stdout:4/409: sync 2024-02-28T23:07:35.318 INFO:tasks.workunit.client.1.smithi084.stdout:2/374: sync 2024-02-28T23:07:35.318 INFO:tasks.workunit.client.1.smithi084.stdout:0/443: dread d1/d6/db/d35/d44/f79 [0,4194304] 0 2024-02-28T23:07:35.318 INFO:tasks.workunit.client.0.smithi057.stdout:8/313: write d4/d13/f2d [3813452,118729] 0 2024-02-28T23:07:35.318 INFO:tasks.workunit.client.0.smithi057.stdout:7/326: mknod d0/d5/d25/c65 0 2024-02-28T23:07:35.320 INFO:tasks.workunit.client.1.smithi084.stdout:3/438: rename d3/d27/d4b/f55 to d3/d8/d1a/d29/d41/d60/d98/fa8 0 2024-02-28T23:07:35.321 INFO:tasks.workunit.client.1.smithi084.stdout:6/443: read d1/d9/d1d/f32 [2144735,50717] 0 2024-02-28T23:07:35.322 INFO:tasks.workunit.client.0.smithi057.stdout:7/327: fdatasync d0/d5/d36/d4e/d15/d29/f40 0 2024-02-28T23:07:35.323 INFO:tasks.workunit.client.1.smithi084.stdout:4/410: chown d1/d18/d20/d14/d30/f38 665289 1 2024-02-28T23:07:35.325 INFO:tasks.workunit.client.0.smithi057.stdout:5/274: write d4/d7/f1e [2978163,14337] 0 2024-02-28T23:07:35.325 INFO:tasks.workunit.client.0.smithi057.stdout:9/393: mknod dd/d46/c7b 0 2024-02-28T23:07:35.326 INFO:tasks.workunit.client.1.smithi084.stdout:3/439: rmdir d3/d50 39 2024-02-28T23:07:35.328 INFO:tasks.workunit.client.0.smithi057.stdout:9/394: read - dd/de/d14/f55 zero size 2024-02-28T23:07:35.329 INFO:tasks.workunit.client.0.smithi057.stdout:5/275: write d4/d9/d14/d3b/f47 [49385,66464] 0 2024-02-28T23:07:35.329 INFO:tasks.workunit.client.0.smithi057.stdout:8/314: write d4/d13/d2c/f69 [789736,106438] 0 2024-02-28T23:07:35.329 INFO:tasks.workunit.client.0.smithi057.stdout:7/328: rename d0/d3f to d0/d3f/d66 22 2024-02-28T23:07:35.332 INFO:tasks.workunit.client.1.smithi084.stdout:6/444: truncate d1/d13/d50/d65/f9b 183361 0 2024-02-28T23:07:35.336 INFO:tasks.workunit.client.1.smithi084.stdout:7/429: rmdir db/d5e 39 2024-02-28T23:07:35.339 INFO:tasks.workunit.client.1.smithi084.stdout:5/493: dread dc/de/df/d16/f4a [0,4194304] 0 2024-02-28T23:07:35.340 INFO:tasks.workunit.client.0.smithi057.stdout:5/276: write d4/d21/f2d [2495047,55882] 0 2024-02-28T23:07:35.340 INFO:tasks.workunit.client.0.smithi057.stdout:9/395: unlink dd/de/d14/d48/d4b/f6e 0 2024-02-28T23:07:35.346 INFO:tasks.workunit.client.0.smithi057.stdout:9/396: chown dd/de/f6b 31660 1 2024-02-28T23:07:35.347 INFO:tasks.workunit.client.1.smithi084.stdout:6/445: mkdir d1/d13/d9d 0 2024-02-28T23:07:35.349 INFO:tasks.workunit.client.1.smithi084.stdout:8/414: rename d0/d24/d6e/l70 to d0/l90 0 2024-02-28T23:07:35.350 INFO:tasks.workunit.client.0.smithi057.stdout:8/315: link d4/f9 d4/d13/d1a/d4a/f6e 0 2024-02-28T23:07:35.351 INFO:tasks.workunit.client.0.smithi057.stdout:7/329: creat d0/f67 x:0 0 0 2024-02-28T23:07:35.353 INFO:tasks.workunit.client.0.smithi057.stdout:5/277: mknod d4/d9/d16/d2c/c4f 0 2024-02-28T23:07:35.353 INFO:tasks.workunit.client.0.smithi057.stdout:1/257: dwrite d2/d7/df/db/d3a/d2a/f2e [0,4194304] 0 2024-02-28T23:07:35.356 INFO:tasks.workunit.client.1.smithi084.stdout:7/430: dwrite db/d10/d17/d3b/f7c [4194304,4194304] 0 2024-02-28T23:07:35.357 INFO:tasks.workunit.client.1.smithi084.stdout:6/446: mkdir d1/d13/d50/d74/d9e 0 2024-02-28T23:07:35.360 INFO:tasks.workunit.client.0.smithi057.stdout:6/376: write d0/d4/d11/f14 [2618308,59718] 0 2024-02-28T23:07:35.361 INFO:tasks.workunit.client.0.smithi057.stdout:2/356: dwrite d8/d11/f1c [0,4194304] 0 2024-02-28T23:07:35.361 INFO:tasks.workunit.client.1.smithi084.stdout:3/440: rename d3/d8/d1a/f47 to d3/d8/d1a/d21/d7b/fa9 0 2024-02-28T23:07:35.361 INFO:tasks.workunit.client.1.smithi084.stdout:6/447: chown d1/d9/d1d/d1e/d33 5167 1 2024-02-28T23:07:35.364 INFO:tasks.workunit.client.1.smithi084.stdout:0/444: dread d1/d6/db/d35/d4a/f66 [0,4194304] 0 2024-02-28T23:07:35.364 INFO:tasks.workunit.client.1.smithi084.stdout:5/494: creat dc/de/df/da6/fb6 x:0 0 0 2024-02-28T23:07:35.367 INFO:tasks.workunit.client.0.smithi057.stdout:8/316: dread - d4/d1e/d56/d6a/f6d zero size 2024-02-28T23:07:35.367 INFO:tasks.workunit.client.0.smithi057.stdout:9/397: write dd/de/d4a/d60/f75 [756117,17987] 0 2024-02-28T23:07:35.367 INFO:tasks.workunit.client.1.smithi084.stdout:2/375: getdents d5/d7 0 2024-02-28T23:07:35.370 INFO:tasks.workunit.client.0.smithi057.stdout:7/330: write d0/d5/d36/d4e/d15/d29/d3d/d51/f64 [111703,37960] 0 2024-02-28T23:07:35.372 INFO:tasks.workunit.client.1.smithi084.stdout:4/411: dwrite d1/d18/d20/d14/d30/f81 [0,4194304] 0 2024-02-28T23:07:35.376 INFO:tasks.workunit.client.0.smithi057.stdout:0/315: sync 2024-02-28T23:07:35.380 INFO:tasks.workunit.client.1.smithi084.stdout:0/445: truncate d1/d6/db/d35/d82/f83 709558 0 2024-02-28T23:07:35.382 INFO:tasks.workunit.client.1.smithi084.stdout:8/415: getdents d0/d24/d4d 0 2024-02-28T23:07:35.386 INFO:tasks.workunit.client.0.smithi057.stdout:8/317: mknod d4/d13/d1a/c6f 0 2024-02-28T23:07:35.391 INFO:tasks.workunit.client.0.smithi057.stdout:2/357: creat d8/d11/d48/d50/f7a x:0 0 0 2024-02-28T23:07:35.395 INFO:tasks.workunit.client.1.smithi084.stdout:8/416: creat d0/d24/d6e/f91 x:0 0 0 2024-02-28T23:07:35.395 INFO:tasks.workunit.client.1.smithi084.stdout:6/448: getdents d1/d13/d50/d65 0 2024-02-28T23:07:35.398 INFO:tasks.workunit.client.1.smithi084.stdout:2/376: link d5/d7/d1d/l23 d5/d7/d12/d24/d2a/d35/l6b 0 2024-02-28T23:07:35.399 INFO:tasks.workunit.client.1.smithi084.stdout:8/417: truncate d0/d44/f67 976884 0 2024-02-28T23:07:35.402 INFO:tasks.workunit.client.1.smithi084.stdout:6/449: symlink d1/d13/d2e/d41/d4d/l9f 0 2024-02-28T23:07:35.402 INFO:tasks.workunit.client.0.smithi057.stdout:5/278: dread d4/d7/f1e [0,4194304] 0 2024-02-28T23:07:35.403 INFO:tasks.workunit.client.0.smithi057.stdout:9/398: dwrite dd/d1b/f50 [0,4194304] 0 2024-02-28T23:07:35.403 INFO:tasks.workunit.client.1.smithi084.stdout:2/377: write d5/d7/d2e/f46 [1562646,120384] 0 2024-02-28T23:07:35.405 INFO:tasks.workunit.client.1.smithi084.stdout:7/431: dwrite db/d5e/f6a [0,4194304] 0 2024-02-28T23:07:35.409 INFO:tasks.workunit.client.0.smithi057.stdout:2/358: unlink d8/f24 0 2024-02-28T23:07:35.409 INFO:tasks.workunit.client.0.smithi057.stdout:0/316: dwrite f4 [0,4194304] 0 2024-02-28T23:07:35.413 INFO:tasks.workunit.client.1.smithi084.stdout:2/378: creat d5/d64/f6c x:0 0 0 2024-02-28T23:07:35.413 INFO:tasks.workunit.client.0.smithi057.stdout:0/317: dread - d17/d20/d23/f75 zero size 2024-02-28T23:07:35.416 INFO:tasks.workunit.client.1.smithi084.stdout:7/432: chown db/d10/f60 82970 1 2024-02-28T23:07:35.418 INFO:tasks.workunit.client.0.smithi057.stdout:0/318: fsync d17/d20/d26/f41 0 2024-02-28T23:07:35.418 INFO:tasks.workunit.client.0.smithi057.stdout:5/279: creat d4/d32/f50 x:0 0 0 2024-02-28T23:07:35.419 INFO:tasks.workunit.client.1.smithi084.stdout:2/379: stat d5/d7/d1d/l23 0 2024-02-28T23:07:35.423 INFO:tasks.workunit.client.0.smithi057.stdout:1/258: dread d2/d7/df/db/d12/f26 [0,4194304] 0 2024-02-28T23:07:35.426 INFO:tasks.workunit.client.0.smithi057.stdout:9/399: creat dd/de/d24/d72/f7c x:0 0 0 2024-02-28T23:07:35.427 INFO:tasks.workunit.client.1.smithi084.stdout:3/441: truncate d3/d50/f8c 3892959 0 2024-02-28T23:07:35.428 INFO:tasks.workunit.client.1.smithi084.stdout:2/380: rename d5/d7/ca to d5/d64/c6d 0 2024-02-28T23:07:35.429 INFO:tasks.workunit.client.0.smithi057.stdout:7/331: write d0/d5/d36/d4e/d15/f22 [2327557,70628] 0 2024-02-28T23:07:35.429 INFO:tasks.workunit.client.0.smithi057.stdout:5/280: rename d4/d21/d24/f44 to d4/d21/d24/f51 0 2024-02-28T23:07:35.430 INFO:tasks.workunit.client.0.smithi057.stdout:0/319: read d17/d19/f28 [1737984,22941] 0 2024-02-28T23:07:35.433 INFO:tasks.workunit.client.0.smithi057.stdout:1/259: symlink d2/d7/df/db/d12/l59 0 2024-02-28T23:07:35.433 INFO:tasks.workunit.client.1.smithi084.stdout:7/433: rename db/d10/d17/d64/f7b to db/d10/d17/d64/f81 0 2024-02-28T23:07:35.436 INFO:tasks.workunit.client.1.smithi084.stdout:7/434: chown db/l4a 1556 1 2024-02-28T23:07:35.438 INFO:tasks.workunit.client.0.smithi057.stdout:7/332: truncate d0/d5/d36/d4e/f19 4825176 0 2024-02-28T23:07:35.438 INFO:tasks.workunit.client.0.smithi057.stdout:5/281: getdents d4/d9/d16/d4e 0 2024-02-28T23:07:35.440 INFO:tasks.workunit.client.0.smithi057.stdout:2/359: dwrite d8/d11/d21/d28/f3d [0,4194304] 0 2024-02-28T23:07:35.443 INFO:tasks.workunit.client.1.smithi084.stdout:5/495: dread dc/de/df/d16/d34/d54/d5f/d66/f96 [0,4194304] 0 2024-02-28T23:07:35.444 INFO:tasks.workunit.client.1.smithi084.stdout:8/418: dwrite d0/d9/d30/d33/d71/d84/f8d [0,4194304] 0 2024-02-28T23:07:35.448 INFO:tasks.workunit.client.1.smithi084.stdout:4/412: dwrite d1/d18/d20/d14/d30/d47/f82 [0,4194304] 0 2024-02-28T23:07:35.450 INFO:tasks.workunit.client.0.smithi057.stdout:8/318: truncate d4/d13/f2d 3419757 0 2024-02-28T23:07:35.451 INFO:tasks.workunit.client.0.smithi057.stdout:2/360: mkdir d8/d11/d39/d7b 0 2024-02-28T23:07:35.452 INFO:tasks.workunit.client.1.smithi084.stdout:1/508: sync 2024-02-28T23:07:35.452 INFO:tasks.workunit.client.1.smithi084.stdout:9/484: sync 2024-02-28T23:07:35.452 INFO:tasks.workunit.client.1.smithi084.stdout:6/450: dwrite d1/d13/d50/d74/d81/f94 [0,4194304] 0 2024-02-28T23:07:35.453 INFO:tasks.workunit.client.0.smithi057.stdout:5/282: fdatasync d4/d32/f41 0 2024-02-28T23:07:35.453 INFO:tasks.workunit.client.0.smithi057.stdout:2/361: chown d8/d11/d48/d50 19522 1 2024-02-28T23:07:35.454 INFO:tasks.workunit.client.1.smithi084.stdout:5/496: mknod dc/de/df/d6e/cb7 0 2024-02-28T23:07:35.457 INFO:tasks.workunit.client.1.smithi084.stdout:9/485: stat d4/d6/d15/d12/f48 0 2024-02-28T23:07:35.458 INFO:tasks.workunit.client.0.smithi057.stdout:2/362: dread - d8/d11/d39/f4c zero size 2024-02-28T23:07:35.462 INFO:tasks.workunit.client.0.smithi057.stdout:5/283: unlink d4/d21/d24/d30/l31 0 2024-02-28T23:07:35.462 INFO:tasks.workunit.client.0.smithi057.stdout:2/363: chown d8/f64 0 1 2024-02-28T23:07:35.463 INFO:tasks.workunit.client.1.smithi084.stdout:8/419: creat d0/d9/d30/d33/d71/f92 x:0 0 0 2024-02-28T23:07:35.464 INFO:tasks.workunit.client.0.smithi057.stdout:0/320: dwrite d17/d20/f45 [0,4194304] 0 2024-02-28T23:07:35.465 INFO:tasks.workunit.client.0.smithi057.stdout:3/242: sync 2024-02-28T23:07:35.465 INFO:tasks.workunit.client.1.smithi084.stdout:5/497: fsync dc/f27 0 2024-02-28T23:07:35.470 INFO:tasks.workunit.client.1.smithi084.stdout:1/509: creat d12/d1b/d2e/d30/faa x:0 0 0 2024-02-28T23:07:35.472 INFO:tasks.workunit.client.1.smithi084.stdout:0/446: dread d1/d6/db/d35/d44/f4b [0,4194304] 0 2024-02-28T23:07:35.473 INFO:tasks.workunit.client.1.smithi084.stdout:6/451: mkdir d1/d9/d1d/d1e/d33/d38/da0 0 2024-02-28T23:07:35.473 INFO:tasks.workunit.client.0.smithi057.stdout:9/400: truncate dd/d1b/f50 1747522 0 2024-02-28T23:07:35.473 INFO:tasks.workunit.client.0.smithi057.stdout:5/284: mkdir d4/d9/d16/d52 0 2024-02-28T23:07:35.474 INFO:tasks.workunit.client.0.smithi057.stdout:2/364: readlink d8/d11/d21/d28/d41/l6b 0 2024-02-28T23:07:35.474 INFO:tasks.workunit.client.1.smithi084.stdout:5/498: read dc/de/df/d31/d8e/f52 [959241,12492] 0 2024-02-28T23:07:35.476 INFO:tasks.workunit.client.1.smithi084.stdout:8/420: chown d0/d24/d4d/f81 26 1 2024-02-28T23:07:35.476 INFO:tasks.workunit.client.0.smithi057.stdout:0/321: fdatasync d17/d20/f66 0 2024-02-28T23:07:35.478 INFO:tasks.workunit.client.0.smithi057.stdout:3/243: symlink d3/d4f/d4c/l50 0 2024-02-28T23:07:35.479 INFO:tasks.workunit.client.0.smithi057.stdout:5/285: stat d4/d7/d8/l25 0 2024-02-28T23:07:35.479 INFO:tasks.workunit.client.1.smithi084.stdout:3/442: dwrite d3/d8/d14/f6d [0,4194304] 0 2024-02-28T23:07:35.479 INFO:tasks.workunit.client.1.smithi084.stdout:9/486: write d4/f79 [18941,50109] 0 2024-02-28T23:07:35.481 INFO:tasks.workunit.client.0.smithi057.stdout:9/401: write dd/d1b/d40/f59 [336369,122507] 0 2024-02-28T23:07:35.482 INFO:tasks.workunit.client.1.smithi084.stdout:4/413: rename d1/d18/d20/d14/d30/d47/f51 to d1/d18/d20/d24/d2f/d3e/d52/f8c 0 2024-02-28T23:07:35.482 INFO:tasks.workunit.client.1.smithi084.stdout:0/447: chown d1/d6/db/d35/d57/d81 43 1 2024-02-28T23:07:35.488 INFO:tasks.workunit.client.1.smithi084.stdout:8/421: read d0/d44/f67 [98661,54714] 0 2024-02-28T23:07:35.489 INFO:tasks.workunit.client.1.smithi084.stdout:0/448: chown d1/d6/d59/d5b 1589249 1 2024-02-28T23:07:35.489 INFO:tasks.workunit.client.1.smithi084.stdout:1/510: read d12/d1b/d63/d7d/f85 [359046,119618] 0 2024-02-28T23:07:35.489 INFO:tasks.workunit.client.0.smithi057.stdout:0/322: dread - d17/d20/d26/f69 zero size 2024-02-28T23:07:35.496 INFO:tasks.workunit.client.1.smithi084.stdout:9/487: write d4/f3b [1410609,121190] 0 2024-02-28T23:07:35.496 INFO:tasks.workunit.client.1.smithi084.stdout:5/499: creat dc/de/df/d16/d34/d3c/fb8 x:0 0 0 2024-02-28T23:07:35.499 INFO:tasks.workunit.client.0.smithi057.stdout:9/402: chown dd/d1b/d40/f59 3 1 2024-02-28T23:07:35.499 INFO:tasks.workunit.client.1.smithi084.stdout:2/381: read d5/d7/d12/d24/d3a/d48/f49 [51358,68983] 0 2024-02-28T23:07:35.500 INFO:tasks.workunit.client.1.smithi084.stdout:3/443: creat d3/d8/d1a/d21/d3d/faa x:0 0 0 2024-02-28T23:07:35.503 INFO:tasks.workunit.client.0.smithi057.stdout:9/403: chown c9 0 1 2024-02-28T23:07:35.504 INFO:tasks.workunit.client.0.smithi057.stdout:7/333: dwrite d0/d5/d36/d4e/d15/d29/d3d/d51/f5e [0,4194304] 0 2024-02-28T23:07:35.505 INFO:tasks.workunit.client.1.smithi084.stdout:7/435: dread db/d11/f16 [0,4194304] 0 2024-02-28T23:07:35.506 INFO:tasks.workunit.client.0.smithi057.stdout:5/286: read d4/d9/d14/d3b/f47 [10824,4192] 0 2024-02-28T23:07:35.506 INFO:tasks.workunit.client.0.smithi057.stdout:1/260: dwrite d2/d7/df/db/d3a/f3b [0,4194304] 0 2024-02-28T23:07:35.507 INFO:tasks.workunit.client.1.smithi084.stdout:8/422: creat d0/d9/d30/d33/d71/f93 x:0 0 0 2024-02-28T23:07:35.510 INFO:tasks.workunit.client.1.smithi084.stdout:9/488: creat d4/d6/d15/f98 x:0 0 0 2024-02-28T23:07:35.511 INFO:tasks.workunit.client.1.smithi084.stdout:7/436: fdatasync db/d11/f16 0 2024-02-28T23:07:35.515 INFO:tasks.workunit.client.0.smithi057.stdout:9/404: fsync dd/d46/d6f/f79 0 2024-02-28T23:07:35.515 INFO:tasks.workunit.client.0.smithi057.stdout:0/323: truncate d17/d19/d4d/f62 195035 0 2024-02-28T23:07:35.516 INFO:tasks.workunit.client.0.smithi057.stdout:1/261: readlink d2/d7/df/l45 0 2024-02-28T23:07:35.516 INFO:tasks.workunit.client.1.smithi084.stdout:2/382: symlink d5/d7/d12/d24/d3a/l6e 0 2024-02-28T23:07:35.519 INFO:tasks.workunit.client.1.smithi084.stdout:3/444: write d3/d8/d1a/d29/d41/d60/f8b [813702,69107] 0 2024-02-28T23:07:35.519 INFO:tasks.workunit.client.1.smithi084.stdout:5/500: creat dc/de/df/d16/d34/d54/d5f/d9b/fb9 x:0 0 0 2024-02-28T23:07:35.522 INFO:tasks.workunit.client.1.smithi084.stdout:8/423: fdatasync d0/d9/d1b/d38/d43/d51/d7c/f80 0 2024-02-28T23:07:35.522 INFO:tasks.workunit.client.0.smithi057.stdout:7/334: truncate d0/d5/f56 978882 0 2024-02-28T23:07:35.525 INFO:tasks.workunit.client.0.smithi057.stdout:8/319: dwrite d4/d13/d1a/d4a/f6e [4194304,4194304] 0 2024-02-28T23:07:35.531 INFO:tasks.workunit.client.0.smithi057.stdout:2/365: dwrite d8/d11/f1d [0,4194304] 0 2024-02-28T23:07:35.533 INFO:tasks.workunit.client.1.smithi084.stdout:7/437: creat db/d10/d18/f82 x:0 0 0 2024-02-28T23:07:35.536 INFO:tasks.workunit.client.0.smithi057.stdout:2/366: chown d8/d11/d56 6 1 2024-02-28T23:07:35.538 INFO:tasks.workunit.client.0.smithi057.stdout:6/377: sync 2024-02-28T23:07:35.538 INFO:tasks.workunit.client.0.smithi057.stdout:4/319: sync 2024-02-28T23:07:35.539 INFO:tasks.workunit.client.1.smithi084.stdout:6/452: dwrite d1/d13/d47/f4b [4194304,4194304] 0 2024-02-28T23:07:35.541 INFO:tasks.workunit.client.1.smithi084.stdout:2/383: fdatasync d5/d7/d2e/f46 0 2024-02-28T23:07:35.543 INFO:tasks.workunit.client.0.smithi057.stdout:9/405: symlink dd/de/d24/d63/l7d 0 2024-02-28T23:07:35.544 INFO:tasks.workunit.client.1.smithi084.stdout:4/414: dwrite d1/d18/d20/d14/d30/f38 [4194304,4194304] 0 2024-02-28T23:07:35.546 INFO:tasks.workunit.client.1.smithi084.stdout:7/438: mkdir db/d11/d83 0 2024-02-28T23:07:35.546 INFO:tasks.workunit.client.0.smithi057.stdout:3/244: dwrite d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:35.546 INFO:tasks.workunit.client.0.smithi057.stdout:1/262: mkdir d2/d7/df/db/d3a/d5a 0 2024-02-28T23:07:35.548 INFO:tasks.workunit.client.0.smithi057.stdout:6/378: write d0/d1b/d3e/d3d/d42/d4f/d53/f5b [764310,63017] 0 2024-02-28T23:07:35.552 INFO:tasks.workunit.client.0.smithi057.stdout:2/367: unlink d8/d11/d39/f6f 0 2024-02-28T23:07:35.555 INFO:tasks.workunit.client.1.smithi084.stdout:0/449: dwrite d1/d6/f13 [0,4194304] 0 2024-02-28T23:07:35.556 INFO:tasks.workunit.client.0.smithi057.stdout:4/320: dread - d1a/d1e/d4b/d4f/f53 zero size 2024-02-28T23:07:35.556 INFO:tasks.workunit.client.0.smithi057.stdout:0/324: write d17/f58 [932515,97193] 0 2024-02-28T23:07:35.563 INFO:tasks.workunit.client.0.smithi057.stdout:5/287: dwrite d4/d21/d24/f51 [0,4194304] 0 2024-02-28T23:07:35.567 INFO:tasks.workunit.client.0.smithi057.stdout:9/406: creat dd/de/d4a/d60/f7e x:0 0 0 2024-02-28T23:07:35.567 INFO:tasks.workunit.client.0.smithi057.stdout:5/288: chown d4 226 1 2024-02-28T23:07:35.568 INFO:tasks.workunit.client.1.smithi084.stdout:5/501: dwrite dc/de/df/d16/f9a [0,4194304] 0 2024-02-28T23:07:35.569 INFO:tasks.workunit.client.1.smithi084.stdout:8/424: rename d0/d9/d1b/d38/d43/d51/f61 to d0/d9/d30/d33/f94 0 2024-02-28T23:07:35.570 INFO:tasks.workunit.client.0.smithi057.stdout:0/325: mkdir d17/d20/d26/d48/d7c 0 2024-02-28T23:07:35.572 INFO:tasks.workunit.client.0.smithi057.stdout:6/379: chown d0/d4/d25/f26 19 1 2024-02-28T23:07:35.574 INFO:tasks.workunit.client.0.smithi057.stdout:2/368: rename d8/ff to d8/d11/d56/f7c 0 2024-02-28T23:07:35.577 INFO:tasks.workunit.client.0.smithi057.stdout:1/263: mkdir d2/d7/df/db/d12/d19/d24/d2d/d5b 0 2024-02-28T23:07:35.577 INFO:tasks.workunit.client.0.smithi057.stdout:8/320: dwrite d4/d1e/d56/d6a/f4d [4194304,4194304] 0 2024-02-28T23:07:35.578 INFO:tasks.workunit.client.0.smithi057.stdout:4/321: symlink d1a/d1e/d51/l73 0 2024-02-28T23:07:35.582 INFO:tasks.workunit.client.0.smithi057.stdout:2/369: chown d8/d11/d21/f5a 8569 1 2024-02-28T23:07:35.582 INFO:tasks.workunit.client.0.smithi057.stdout:3/245: unlink d3/l4e 0 2024-02-28T23:07:35.585 INFO:tasks.workunit.client.1.smithi084.stdout:3/445: dwrite d3/f6 [4194304,4194304] 0 2024-02-28T23:07:35.588 INFO:tasks.workunit.client.0.smithi057.stdout:0/326: creat d17/d20/d4a/d78/f7d x:0 0 0 2024-02-28T23:07:35.590 INFO:tasks.workunit.client.0.smithi057.stdout:6/380: symlink d0/d4/d25/d60/d6a/l81 0 2024-02-28T23:07:35.591 INFO:tasks.workunit.client.1.smithi084.stdout:4/415: symlink d1/l8d 0 2024-02-28T23:07:35.595 INFO:tasks.workunit.client.1.smithi084.stdout:6/453: dwrite d1/d9/d1d/d1e/d33/d38/f54 [4194304,4194304] 0 2024-02-28T23:07:35.596 INFO:tasks.workunit.client.1.smithi084.stdout:8/425: creat d0/d11/f95 x:0 0 0 2024-02-28T23:07:35.596 INFO:tasks.workunit.client.0.smithi057.stdout:0/327: write d17/d20/d26/f46 [2891058,14814] 0 2024-02-28T23:07:35.598 INFO:tasks.workunit.client.0.smithi057.stdout:8/321: creat d4/d1e/d56/d6a/d54/f70 x:0 0 0 2024-02-28T23:07:35.599 INFO:tasks.workunit.client.1.smithi084.stdout:5/502: write dc/de/df/d16/f57 [3143004,87729] 0 2024-02-28T23:07:35.601 INFO:tasks.workunit.client.1.smithi084.stdout:1/511: dread f1 [0,4194304] 0 2024-02-28T23:07:35.603 INFO:tasks.workunit.client.0.smithi057.stdout:7/335: write d0/d5/f56 [445024,102849] 0 2024-02-28T23:07:35.604 INFO:tasks.workunit.client.0.smithi057.stdout:1/264: mkdir d2/d7/d4f/d5c 0 2024-02-28T23:07:35.604 INFO:tasks.workunit.client.0.smithi057.stdout:9/407: truncate dd/de/d14/f35 1183892 0 2024-02-28T23:07:35.606 INFO:tasks.workunit.client.1.smithi084.stdout:0/450: dwrite d1/d8/f3f [0,4194304] 0 2024-02-28T23:07:35.607 INFO:tasks.workunit.client.1.smithi084.stdout:3/446: creat d3/d50/d5a/fab x:0 0 0 2024-02-28T23:07:35.607 INFO:tasks.workunit.client.0.smithi057.stdout:8/322: truncate d4/d13/d2c/d4e/f60 578976 0 2024-02-28T23:07:35.608 INFO:tasks.workunit.client.1.smithi084.stdout:4/416: readlink d1/d18/d20/l1f 0 2024-02-28T23:07:35.609 INFO:tasks.workunit.client.0.smithi057.stdout:0/328: rename d17/d20/d23/f32 to d17/d20/d26/d30/d6b/d71/f7e 0 2024-02-28T23:07:35.614 INFO:tasks.workunit.client.0.smithi057.stdout:9/408: rmdir dd/de/d14/d48/d4b 39 2024-02-28T23:07:35.615 INFO:tasks.workunit.client.1.smithi084.stdout:6/454: rename d1/d9/d1d/d1e/d33/l96 to d1/d13/la1 0 2024-02-28T23:07:35.615 INFO:tasks.workunit.client.1.smithi084.stdout:9/489: dread d4/d6/d15/d3c/f7a [0,4194304] 0 2024-02-28T23:07:35.621 INFO:tasks.workunit.client.0.smithi057.stdout:0/329: creat d17/d20/d23/d47/f7f x:0 0 0 2024-02-28T23:07:35.623 INFO:tasks.workunit.client.0.smithi057.stdout:1/265: mknod d2/d7/df/db/d12/d19/d24/d2d/d5b/c5d 0 2024-02-28T23:07:35.624 INFO:tasks.workunit.client.1.smithi084.stdout:9/490: readlink d4/l95 0 2024-02-28T23:07:35.625 INFO:tasks.workunit.client.0.smithi057.stdout:9/409: creat dd/de/d24/d72/f7f x:0 0 0 2024-02-28T23:07:35.627 INFO:tasks.workunit.client.1.smithi084.stdout:0/451: mkdir d1/d8/d5f/d93 0 2024-02-28T23:07:35.632 INFO:tasks.workunit.client.0.smithi057.stdout:2/370: dwrite d8/d11/f1d [0,4194304] 0 2024-02-28T23:07:35.635 INFO:tasks.workunit.client.1.smithi084.stdout:4/417: rename d1/d18/d20/d24/d2f/d3e/d52/d37 to d1/d18/d20/d65/d8e 0 2024-02-28T23:07:35.636 INFO:tasks.workunit.client.0.smithi057.stdout:9/410: rename dd/de/d24/d63/d77 to dd/de/d24/d63/d77/d80 22 2024-02-28T23:07:35.641 INFO:tasks.workunit.client.1.smithi084.stdout:5/503: link dc/de/df/d16/l35 dc/de/d9e/lba 0 2024-02-28T23:07:35.641 INFO:tasks.workunit.client.0.smithi057.stdout:4/322: dwrite fe [4194304,4194304] 0 2024-02-28T23:07:35.643 INFO:tasks.workunit.client.0.smithi057.stdout:2/371: rmdir d8/d11/d21/d28 39 2024-02-28T23:07:35.643 INFO:tasks.workunit.client.0.smithi057.stdout:8/323: dwrite d4/d1e/d56/d6a/f4d [4194304,4194304] 0 2024-02-28T23:07:35.644 INFO:tasks.workunit.client.1.smithi084.stdout:0/452: fsync d1/d8/f55 0 2024-02-28T23:07:35.645 INFO:tasks.workunit.client.1.smithi084.stdout:2/384: truncate d5/d7/d2e/f2f 4176987 0 2024-02-28T23:07:35.646 INFO:tasks.workunit.client.0.smithi057.stdout:7/336: dwrite d0/d5/d36/d4e/d15/d29/d3d/d51/f5e [0,4194304] 0 2024-02-28T23:07:35.647 INFO:tasks.workunit.client.1.smithi084.stdout:7/439: dwrite db/d10/d17/f3d [0,4194304] 0 2024-02-28T23:07:35.648 INFO:tasks.workunit.client.0.smithi057.stdout:9/411: mknod dd/de/d14/d48/d76/c81 0 2024-02-28T23:07:35.650 INFO:tasks.workunit.client.0.smithi057.stdout:4/323: fsync d1a/d2f/f45 0 2024-02-28T23:07:35.652 INFO:tasks.workunit.client.0.smithi057.stdout:8/324: chown d4/d13/d18/f27 647 1 2024-02-28T23:07:35.652 INFO:tasks.workunit.client.1.smithi084.stdout:8/426: dwrite d0/d9/d30/d33/f94 [0,4194304] 0 2024-02-28T23:07:35.654 INFO:tasks.workunit.client.1.smithi084.stdout:3/447: dwrite d3/d8/d1a/d21/d3d/f3c [0,4194304] 0 2024-02-28T23:07:35.655 INFO:tasks.workunit.client.1.smithi084.stdout:9/491: chown d4/d6/d15/d12/l2b 572417 1 2024-02-28T23:07:35.657 INFO:tasks.workunit.client.0.smithi057.stdout:2/372: stat d8/d11/d21/d28/d41/l58 0 2024-02-28T23:07:35.658 INFO:tasks.workunit.client.1.smithi084.stdout:2/385: truncate d5/d1f/d42/f47 896961 0 2024-02-28T23:07:35.660 INFO:tasks.workunit.client.1.smithi084.stdout:9/492: chown d4/d6/d15/d12/d41/c8b 1577535 1 2024-02-28T23:07:35.660 INFO:tasks.workunit.client.1.smithi084.stdout:6/455: dread d1/d13/d47/f4b [4194304,4194304] 0 2024-02-28T23:07:35.663 INFO:tasks.workunit.client.1.smithi084.stdout:6/456: fsync d1/d9/f4f 0 2024-02-28T23:07:35.664 INFO:tasks.workunit.client.1.smithi084.stdout:1/512: read fc [6349856,81331] 0 2024-02-28T23:07:35.664 INFO:tasks.workunit.client.0.smithi057.stdout:9/412: mknod dd/de/d14/c82 0 2024-02-28T23:07:35.666 INFO:tasks.workunit.client.0.smithi057.stdout:5/289: write d4/d9/d14/f43 [428798,98201] 0 2024-02-28T23:07:35.669 INFO:tasks.workunit.client.1.smithi084.stdout:0/453: rmdir d1/d8/d18/d4c 39 2024-02-28T23:07:35.669 INFO:tasks.workunit.client.1.smithi084.stdout:8/427: creat d0/d9/d1b/d38/d43/f96 x:0 0 0 2024-02-28T23:07:35.673 INFO:tasks.workunit.client.1.smithi084.stdout:6/457: read d1/d9/d1d/d1e/d33/f75 [59315,89958] 0 2024-02-28T23:07:35.676 INFO:tasks.workunit.client.0.smithi057.stdout:2/373: unlink d8/d11/d21/f5a 0 2024-02-28T23:07:35.677 INFO:tasks.workunit.client.1.smithi084.stdout:2/386: creat d5/d7/d12/d4f/f6f x:0 0 0 2024-02-28T23:07:35.679 INFO:tasks.workunit.client.1.smithi084.stdout:9/493: write d4/d32/d3f/f42 [2543945,13719] 0 2024-02-28T23:07:35.685 INFO:tasks.workunit.client.0.smithi057.stdout:9/413: creat dd/de/d4a/d60/f83 x:0 0 0 2024-02-28T23:07:35.685 INFO:tasks.workunit.client.0.smithi057.stdout:8/325: dwrite d4/d13/d2c/f49 [0,4194304] 0 2024-02-28T23:07:35.685 INFO:tasks.workunit.client.0.smithi057.stdout:7/337: truncate d0/d5/f2e 565555 0 2024-02-28T23:07:35.686 INFO:tasks.workunit.client.1.smithi084.stdout:1/513: dread f1 [0,4194304] 0 2024-02-28T23:07:35.686 INFO:tasks.workunit.client.1.smithi084.stdout:8/428: unlink d0/d9/d1b/d38/d43/d51/f7b 0 2024-02-28T23:07:35.686 INFO:tasks.workunit.client.0.smithi057.stdout:9/414: stat c6 0 2024-02-28T23:07:35.689 INFO:tasks.workunit.client.1.smithi084.stdout:8/429: dread - d0/d9/f88 zero size 2024-02-28T23:07:35.695 INFO:tasks.workunit.client.1.smithi084.stdout:4/418: rename d1/d4b/f7b to d1/d18/d20/d24/d2f/f8f 0 2024-02-28T23:07:35.697 INFO:tasks.workunit.client.1.smithi084.stdout:4/419: read - d1/d18/d20/d24/f8b zero size 2024-02-28T23:07:35.698 INFO:tasks.workunit.client.0.smithi057.stdout:9/415: write dd/d1b/d40/d44/f7a [1384492,87479] 0 2024-02-28T23:07:35.700 INFO:tasks.workunit.client.1.smithi084.stdout:5/504: rmdir dc/de/df/d16/d34/daa/dae/db1 0 2024-02-28T23:07:35.701 INFO:tasks.workunit.client.0.smithi057.stdout:7/338: mknod d0/d5/d4a/c68 0 2024-02-28T23:07:35.703 INFO:tasks.workunit.client.0.smithi057.stdout:0/330: write d17/d20/d26/d48/f5c [827404,117834] 0 2024-02-28T23:07:35.704 INFO:tasks.workunit.client.0.smithi057.stdout:5/290: link d4/d7/l20 d4/d9/d16/d4e/l53 0 2024-02-28T23:07:35.706 INFO:tasks.workunit.client.0.smithi057.stdout:3/246: dread d3/d4/f30 [0,4194304] 0 2024-02-28T23:07:35.706 INFO:tasks.workunit.client.0.smithi057.stdout:7/339: chown d0/d5/d36/d4e/d15/d29/d3d/d51/f58 204855 1 2024-02-28T23:07:35.706 INFO:tasks.workunit.client.1.smithi084.stdout:2/387: mknod d5/d64/c70 0 2024-02-28T23:07:35.709 INFO:tasks.workunit.client.0.smithi057.stdout:8/326: creat d4/f71 x:0 0 0 2024-02-28T23:07:35.709 INFO:tasks.workunit.client.1.smithi084.stdout:2/388: write d5/d7/d2e/f46 [1194036,36905] 0 2024-02-28T23:07:35.709 INFO:tasks.workunit.client.1.smithi084.stdout:1/514: creat d12/da2/fab x:0 0 0 2024-02-28T23:07:35.711 INFO:tasks.workunit.client.1.smithi084.stdout:6/458: symlink d1/d9/d1d/d1e/d33/la2 0 2024-02-28T23:07:35.711 INFO:tasks.workunit.client.1.smithi084.stdout:8/430: symlink d0/d24/d83/l97 0 2024-02-28T23:07:35.715 INFO:tasks.workunit.client.1.smithi084.stdout:7/440: rename db/d11/c14 to db/d10/d17/d64/c84 0 2024-02-28T23:07:35.715 INFO:tasks.workunit.client.0.smithi057.stdout:5/291: creat d4/d9/d14/f54 x:0 0 0 2024-02-28T23:07:35.716 INFO:tasks.workunit.client.0.smithi057.stdout:9/416: symlink dd/de/l84 0 2024-02-28T23:07:35.716 INFO:tasks.workunit.client.0.smithi057.stdout:5/292: stat d4/d32/f50 0 2024-02-28T23:07:35.716 INFO:tasks.workunit.client.1.smithi084.stdout:1/515: creat d12/d1b/d2e/d30/d50/fac x:0 0 0 2024-02-28T23:07:35.717 INFO:tasks.workunit.client.0.smithi057.stdout:9/417: stat dd/de/d14/l25 0 2024-02-28T23:07:35.718 INFO:tasks.workunit.client.1.smithi084.stdout:8/431: rmdir d0/d24/d78 39 2024-02-28T23:07:35.720 INFO:tasks.workunit.client.1.smithi084.stdout:8/432: truncate d0/f53 1107989 0 2024-02-28T23:07:35.721 INFO:tasks.workunit.client.1.smithi084.stdout:8/433: fsync d0/d9/f15 0 2024-02-28T23:07:35.722 INFO:tasks.workunit.client.0.smithi057.stdout:2/374: rmdir d8/d11/d21 39 2024-02-28T23:07:35.722 INFO:tasks.workunit.client.0.smithi057.stdout:7/340: symlink d0/d5/d36/d4e/d15/d29/l69 0 2024-02-28T23:07:35.722 INFO:tasks.workunit.client.1.smithi084.stdout:9/494: rename d4/d6/f93 to d4/d32/d3f/d89/f99 0 2024-02-28T23:07:35.723 INFO:tasks.workunit.client.1.smithi084.stdout:6/459: mkdir d1/d13/d9d/da3 0 2024-02-28T23:07:35.724 INFO:tasks.workunit.client.0.smithi057.stdout:8/327: symlink d4/d1e/d37/l72 0 2024-02-28T23:07:35.725 INFO:tasks.workunit.client.0.smithi057.stdout:0/331: symlink d17/d19/d4d/d7b/l80 0 2024-02-28T23:07:35.729 INFO:tasks.workunit.client.0.smithi057.stdout:8/328: chown d4/d13/c15 183 1 2024-02-28T23:07:35.729 INFO:tasks.workunit.client.0.smithi057.stdout:0/332: readlink d17/d20/d23/l36 0 2024-02-28T23:07:35.733 INFO:tasks.workunit.client.1.smithi084.stdout:8/434: creat d0/d9/d63/f98 x:0 0 0 2024-02-28T23:07:35.733 INFO:tasks.workunit.client.0.smithi057.stdout:8/329: write d4/d1e/f36 [1156894,59287] 0 2024-02-28T23:07:35.734 INFO:tasks.workunit.client.0.smithi057.stdout:7/341: unlink d0/d5/ce 0 2024-02-28T23:07:35.735 INFO:tasks.workunit.client.1.smithi084.stdout:5/505: rename dc/de/d83/f8d to dc/de/df/d16/d34/daa/dae/fbb 0 2024-02-28T23:07:35.736 INFO:tasks.workunit.client.1.smithi084.stdout:7/441: getdents db/d11/d27/d7a 0 2024-02-28T23:07:35.736 INFO:tasks.workunit.client.0.smithi057.stdout:2/375: rmdir d8/d11/d21 39 2024-02-28T23:07:35.737 INFO:tasks.workunit.client.1.smithi084.stdout:1/516: dwrite d12/d1b/f31 [4194304,4194304] 0 2024-02-28T23:07:35.738 INFO:tasks.workunit.client.1.smithi084.stdout:0/454: dwrite d1/d8/f20 [0,4194304] 0 2024-02-28T23:07:35.738 INFO:tasks.workunit.client.0.smithi057.stdout:8/330: dread - d4/d1e/d56/d6a/d54/d6b/f6c zero size 2024-02-28T23:07:35.740 INFO:tasks.workunit.client.1.smithi084.stdout:7/442: chown db/d10/d17/d64/f69 34104 1 2024-02-28T23:07:35.740 INFO:tasks.workunit.client.0.smithi057.stdout:3/247: dwrite d3/d8/f29 [0,4194304] 0 2024-02-28T23:07:35.741 INFO:tasks.workunit.client.0.smithi057.stdout:2/376: chown d8/d11/d15/c38 390 1 2024-02-28T23:07:35.743 INFO:tasks.workunit.client.0.smithi057.stdout:2/377: chown d8/f19 1609550 1 2024-02-28T23:07:35.744 INFO:tasks.workunit.client.0.smithi057.stdout:9/418: getdents dd/d46 0 2024-02-28T23:07:35.744 INFO:tasks.workunit.client.1.smithi084.stdout:6/460: unlink d1/d13/d47/d91/f93 0 2024-02-28T23:07:35.744 INFO:tasks.workunit.client.1.smithi084.stdout:2/389: link d5/d7/d12/d4f/d5b/c62 d5/dd/d69/c71 0 2024-02-28T23:07:35.744 INFO:tasks.workunit.client.1.smithi084.stdout:9/495: truncate d4/d6/d53/f61 130391 0 2024-02-28T23:07:35.745 INFO:tasks.workunit.client.1.smithi084.stdout:3/448: truncate d3/d8/d14/f2e 490893 0 2024-02-28T23:07:35.746 INFO:tasks.workunit.client.1.smithi084.stdout:7/443: stat db/d10/d17/d64 0 2024-02-28T23:07:35.747 INFO:tasks.workunit.client.1.smithi084.stdout:0/455: mknod d1/d8/d78/c94 0 2024-02-28T23:07:35.748 INFO:tasks.workunit.client.0.smithi057.stdout:8/331: mknod d4/d13/c73 0 2024-02-28T23:07:35.748 INFO:tasks.workunit.client.0.smithi057.stdout:9/419: dread - dd/de/d4a/d60/f7e zero size 2024-02-28T23:07:35.748 INFO:tasks.workunit.client.0.smithi057.stdout:4/324: dread f14 [0,4194304] 0 2024-02-28T23:07:35.750 INFO:tasks.workunit.client.0.smithi057.stdout:3/248: truncate d3/d4f/f48 1366005 0 2024-02-28T23:07:35.750 INFO:tasks.workunit.client.0.smithi057.stdout:5/293: truncate d4/f6 773966 0 2024-02-28T23:07:35.753 INFO:tasks.workunit.client.0.smithi057.stdout:0/333: dread d17/f58 [0,4194304] 0 2024-02-28T23:07:35.753 INFO:tasks.workunit.client.0.smithi057.stdout:5/294: fdatasync d4/d21/f35 0 2024-02-28T23:07:35.754 INFO:tasks.workunit.client.0.smithi057.stdout:3/249: write d3/d4/d1d/f49 [854186,47356] 0 2024-02-28T23:07:35.755 INFO:tasks.workunit.client.0.smithi057.stdout:9/420: creat dd/de/d24/f85 x:0 0 0 2024-02-28T23:07:35.755 INFO:tasks.workunit.client.1.smithi084.stdout:7/444: creat db/d10/d17/d64/d77/f85 x:0 0 0 2024-02-28T23:07:35.760 INFO:tasks.workunit.client.1.smithi084.stdout:3/449: mknod d3/d8/d1a/d21/d3d/cac 0 2024-02-28T23:07:35.760 INFO:tasks.workunit.client.1.smithi084.stdout:2/390: unlink d5/d7/d12/d4f/f6f 0 2024-02-28T23:07:35.761 INFO:tasks.workunit.client.0.smithi057.stdout:0/334: chown d17/d20/d26/d48/d7c 491473403 1 2024-02-28T23:07:35.761 INFO:tasks.workunit.client.0.smithi057.stdout:4/325: write d1a/d1e/f6b [153037,1398] 0 2024-02-28T23:07:35.763 INFO:tasks.workunit.client.1.smithi084.stdout:7/445: creat db/d11/d27/d36/f86 x:0 0 0 2024-02-28T23:07:35.764 INFO:tasks.workunit.client.1.smithi084.stdout:5/506: dread dc/de/df/d16/d34/d54/f5d [0,4194304] 0 2024-02-28T23:07:35.765 INFO:tasks.workunit.client.1.smithi084.stdout:8/435: dwrite d0/d24/d4d/f6a [0,4194304] 0 2024-02-28T23:07:35.765 INFO:tasks.workunit.client.0.smithi057.stdout:0/335: stat d17/d20/d23/d47/f5b 0 2024-02-28T23:07:35.765 INFO:tasks.workunit.client.0.smithi057.stdout:4/326: write d1a/f36 [1507564,86511] 0 2024-02-28T23:07:35.768 INFO:tasks.workunit.client.0.smithi057.stdout:9/421: creat dd/d1b/d40/f86 x:0 0 0 2024-02-28T23:07:35.768 INFO:tasks.workunit.client.0.smithi057.stdout:5/295: link d4/d7/c4a d4/d9/d16/d4e/c55 0 2024-02-28T23:07:35.769 INFO:tasks.workunit.client.1.smithi084.stdout:6/461: rename d1/d13/d47/l60 to d1/d9/la4 0 2024-02-28T23:07:35.772 INFO:tasks.workunit.client.1.smithi084.stdout:8/436: creat d0/d11/f99 x:0 0 0 2024-02-28T23:07:35.772 INFO:tasks.workunit.client.0.smithi057.stdout:4/327: rename d1a/d1b/d2d/f39 to d1a/d1b/f74 0 2024-02-28T23:07:35.775 INFO:tasks.workunit.client.1.smithi084.stdout:1/517: getdents d12/d1b/d91 0 2024-02-28T23:07:35.776 INFO:tasks.workunit.client.0.smithi057.stdout:0/336: getdents d17/d20/d23/d4b 0 2024-02-28T23:07:35.777 INFO:tasks.workunit.client.1.smithi084.stdout:9/496: link d4/d6/d15/d3c/l8f d4/d6/d15/d3c/l9a 0 2024-02-28T23:07:35.778 INFO:tasks.workunit.client.0.smithi057.stdout:5/296: rename d4/la to d4/d9/d14/d3b/l56 0 2024-02-28T23:07:35.778 INFO:tasks.workunit.client.0.smithi057.stdout:2/378: dwrite d8/f19 [4194304,4194304] 0 2024-02-28T23:07:35.779 INFO:tasks.workunit.client.0.smithi057.stdout:9/422: mknod dd/de/d14/d48/d4b/d41/c87 0 2024-02-28T23:07:35.780 INFO:tasks.workunit.client.0.smithi057.stdout:0/337: write d17/d19/f2f [3792802,29932] 0 2024-02-28T23:07:35.781 INFO:tasks.workunit.client.1.smithi084.stdout:2/391: symlink d5/d7/d9/l72 0 2024-02-28T23:07:35.782 INFO:tasks.workunit.client.0.smithi057.stdout:2/379: dread - d8/d11/d39/f62 zero size 2024-02-28T23:07:35.783 INFO:tasks.workunit.client.0.smithi057.stdout:5/297: symlink d4/d21/d24/d30/l57 0 2024-02-28T23:07:35.784 INFO:tasks.workunit.client.0.smithi057.stdout:0/338: readlink d17/d20/d4a/l57 0 2024-02-28T23:07:35.784 INFO:tasks.workunit.client.0.smithi057.stdout:4/328: link d1a/d1b/d24/c44 d1a/d48/c75 0 2024-02-28T23:07:35.784 INFO:tasks.workunit.client.1.smithi084.stdout:2/392: write d5/d7/d12/d24/d2a/d35/f40 [900310,35864] 0 2024-02-28T23:07:35.785 INFO:tasks.workunit.client.1.smithi084.stdout:5/507: creat dc/de/df/d16/d34/d54/da4/fbc x:0 0 0 2024-02-28T23:07:35.787 INFO:tasks.workunit.client.0.smithi057.stdout:2/380: dread - d8/d11/d15/f71 zero size 2024-02-28T23:07:35.789 INFO:tasks.workunit.client.1.smithi084.stdout:0/456: rename d1/d6/db/c49 to d1/d8/d18/d28/c95 0 2024-02-28T23:07:35.790 INFO:tasks.workunit.client.0.smithi057.stdout:4/329: write f18 [4749753,83968] 0 2024-02-28T23:07:35.790 INFO:tasks.workunit.client.0.smithi057.stdout:3/250: dwrite d3/d4/f30 [4194304,4194304] 0 2024-02-28T23:07:35.793 INFO:tasks.workunit.client.1.smithi084.stdout:8/437: chown d0/d11/f2a 47 1 2024-02-28T23:07:35.795 INFO:tasks.workunit.client.1.smithi084.stdout:8/438: fsync d0/d9/d1b/d38/f5d 0 2024-02-28T23:07:35.795 INFO:tasks.workunit.client.1.smithi084.stdout:6/462: symlink d1/d9/d1d/d1e/d33/d38/la5 0 2024-02-28T23:07:35.797 INFO:tasks.workunit.client.0.smithi057.stdout:2/381: write d8/f64 [821723,99817] 0 2024-02-28T23:07:35.797 INFO:tasks.workunit.client.0.smithi057.stdout:0/339: truncate d17/d20/d23/d47/f63 607412 0 2024-02-28T23:07:35.797 INFO:tasks.workunit.client.1.smithi084.stdout:6/463: chown d1/d13/d2e/d41/d2c/f71 386692 1 2024-02-28T23:07:35.798 INFO:tasks.workunit.client.0.smithi057.stdout:3/251: stat d3/d4/d1d/f49 0 2024-02-28T23:07:35.799 INFO:tasks.workunit.client.0.smithi057.stdout:5/298: mkdir d4/d7/d58 0 2024-02-28T23:07:35.799 INFO:tasks.workunit.client.1.smithi084.stdout:8/439: readlink d0/d9/d30/d33/l40 0 2024-02-28T23:07:35.800 INFO:tasks.workunit.client.1.smithi084.stdout:9/497: fsync d4/d6/d15/d3c/f7a 0 2024-02-28T23:07:35.801 INFO:tasks.workunit.client.0.smithi057.stdout:2/382: read d8/d11/d39/f3c [201754,112066] 0 2024-02-28T23:07:35.801 INFO:tasks.workunit.client.1.smithi084.stdout:6/464: fsync d1/d13/d2e/d41/d4d/d6c/f7c 0 2024-02-28T23:07:35.803 INFO:tasks.workunit.client.0.smithi057.stdout:8/332: truncate d4/f5 155454 0 2024-02-28T23:07:35.804 INFO:tasks.workunit.client.1.smithi084.stdout:6/465: read - d1/d13/d50/f99 zero size 2024-02-28T23:07:35.810 INFO:tasks.workunit.client.1.smithi084.stdout:5/508: creat dc/de/df/d16/d34/d3c/d77/fbd x:0 0 0 2024-02-28T23:07:35.812 INFO:tasks.workunit.client.1.smithi084.stdout:5/509: dread - dc/de/df/d16/f81 zero size 2024-02-28T23:07:35.816 INFO:tasks.workunit.client.1.smithi084.stdout:9/498: mknod d4/d32/d3f/d89/c9b 0 2024-02-28T23:07:35.817 INFO:tasks.workunit.client.0.smithi057.stdout:0/340: creat d17/d20/d4a/f81 x:0 0 0 2024-02-28T23:07:35.818 INFO:tasks.workunit.client.1.smithi084.stdout:3/450: getdents d3/d27/d4b 0 2024-02-28T23:07:35.819 INFO:tasks.workunit.client.0.smithi057.stdout:5/299: symlink d4/d21/l59 0 2024-02-28T23:07:35.820 INFO:tasks.workunit.client.1.smithi084.stdout:7/446: getdents db/d10/d17/d3b 0 2024-02-28T23:07:35.820 INFO:tasks.workunit.client.1.smithi084.stdout:0/457: link d1/d8/f3e d1/d6/db/d37/f96 0 2024-02-28T23:07:35.821 INFO:tasks.workunit.client.1.smithi084.stdout:9/499: read d4/d6/f7 [3778762,72572] 0 2024-02-28T23:07:35.821 INFO:tasks.workunit.client.0.smithi057.stdout:2/383: write d8/d11/f32 [158023,70049] 0 2024-02-28T23:07:35.821 INFO:tasks.workunit.client.1.smithi084.stdout:8/440: symlink d0/d24/l9a 0 2024-02-28T23:07:35.822 INFO:tasks.workunit.client.1.smithi084.stdout:7/447: write f9 [730739,12479] 0 2024-02-28T23:07:35.822 INFO:tasks.workunit.client.0.smithi057.stdout:5/300: chown d4/d21/l59 855 1 2024-02-28T23:07:35.825 INFO:tasks.workunit.client.0.smithi057.stdout:0/341: mknod d17/d20/d23/d47/c82 0 2024-02-28T23:07:35.825 INFO:tasks.workunit.client.1.smithi084.stdout:8/441: dread - d0/d24/d6e/f91 zero size 2024-02-28T23:07:35.825 INFO:tasks.workunit.client.1.smithi084.stdout:7/448: write db/d5e/f73 [124989,90505] 0 2024-02-28T23:07:35.827 INFO:tasks.workunit.client.0.smithi057.stdout:3/252: creat d3/f51 x:0 0 0 2024-02-28T23:07:35.830 INFO:tasks.workunit.client.0.smithi057.stdout:2/384: write d8/d11/d15/f72 [503337,7179] 0 2024-02-28T23:07:35.832 INFO:tasks.workunit.client.0.smithi057.stdout:0/342: creat d17/f83 x:0 0 0 2024-02-28T23:07:35.835 INFO:tasks.workunit.client.0.smithi057.stdout:9/423: dwrite dd/d1b/f50 [0,4194304] 0 2024-02-28T23:07:35.835 INFO:tasks.workunit.client.0.smithi057.stdout:4/330: dread d1a/d1b/f1c [0,4194304] 0 2024-02-28T23:07:35.836 INFO:tasks.workunit.client.1.smithi084.stdout:9/500: unlink d4/d6/d15/d12/d41/d55/c74 0 2024-02-28T23:07:35.838 INFO:tasks.workunit.client.1.smithi084.stdout:2/393: write d5/f3b [404505,41374] 0 2024-02-28T23:07:35.838 INFO:tasks.workunit.client.1.smithi084.stdout:7/449: chown db/d5e/c62 41047952 1 2024-02-28T23:07:35.842 INFO:tasks.workunit.client.0.smithi057.stdout:5/301: rename d4/d9/d14/c46 to d4/d9/c5a 0 2024-02-28T23:07:35.843 INFO:tasks.workunit.client.1.smithi084.stdout:7/450: chown db/d10/d17/d64/f81 0 1 2024-02-28T23:07:35.843 INFO:tasks.workunit.client.0.smithi057.stdout:0/343: creat d17/d20/d26/d48/f84 x:0 0 0 2024-02-28T23:07:35.846 INFO:tasks.workunit.client.1.smithi084.stdout:7/451: truncate db/d11/f32 695003 0 2024-02-28T23:07:35.846 INFO:tasks.workunit.client.0.smithi057.stdout:3/253: getdents d3/d4f 0 2024-02-28T23:07:35.847 INFO:tasks.workunit.client.0.smithi057.stdout:3/254: fsync d3/d8/d27/f3e 0 2024-02-28T23:07:35.848 INFO:tasks.workunit.client.0.smithi057.stdout:2/385: creat d8/d11/d21/f7d x:0 0 0 2024-02-28T23:07:35.849 INFO:tasks.workunit.client.0.smithi057.stdout:5/302: mknod d4/d9/d14/c5b 0 2024-02-28T23:07:35.850 INFO:tasks.workunit.client.0.smithi057.stdout:2/386: readlink d8/d11/d48/l4d 0 2024-02-28T23:07:35.850 INFO:tasks.workunit.client.0.smithi057.stdout:4/331: dread f16 [0,4194304] 0 2024-02-28T23:07:35.850 INFO:tasks.workunit.client.0.smithi057.stdout:0/344: mkdir d17/d20/d23/d47/d85 0 2024-02-28T23:07:35.850 INFO:tasks.workunit.client.1.smithi084.stdout:8/442: dread d0/d9/d30/d33/f94 [0,4194304] 0 2024-02-28T23:07:35.850 INFO:tasks.workunit.client.0.smithi057.stdout:3/255: chown d3/d8/d1b/c41 207335 1 2024-02-28T23:07:35.853 INFO:tasks.workunit.client.1.smithi084.stdout:8/443: chown d0/d44/c73 33 1 2024-02-28T23:07:35.855 INFO:tasks.workunit.client.1.smithi084.stdout:1/518: dwrite d12/d1b/d2e/f4e [0,4194304] 0 2024-02-28T23:07:35.859 INFO:tasks.workunit.client.0.smithi057.stdout:9/424: rename c9 to dd/c88 0 2024-02-28T23:07:35.859 INFO:tasks.workunit.client.0.smithi057.stdout:4/332: symlink d1a/d2f/l76 0 2024-02-28T23:07:35.859 INFO:tasks.workunit.client.1.smithi084.stdout:3/451: unlink d3/d8/d14/l1e 0 2024-02-28T23:07:35.860 INFO:tasks.workunit.client.1.smithi084.stdout:5/510: link dc/de/ca1 dc/de/df/d16/d34/d54/d5f/d9b/cbe 0 2024-02-28T23:07:35.861 INFO:tasks.workunit.client.0.smithi057.stdout:8/333: dwrite d4/f5 [0,4194304] 0 2024-02-28T23:07:35.861 INFO:tasks.workunit.client.0.smithi057.stdout:5/303: mkdir d4/d7/d8/d5c 0 2024-02-28T23:07:35.862 INFO:tasks.workunit.client.1.smithi084.stdout:0/458: getdents d1/d8/d5f/d93 0 2024-02-28T23:07:35.864 INFO:tasks.workunit.client.1.smithi084.stdout:2/394: creat d5/d64/f73 x:0 0 0 2024-02-28T23:07:35.864 INFO:tasks.workunit.client.0.smithi057.stdout:0/345: creat d17/d20/d23/d4b/f86 x:0 0 0 2024-02-28T23:07:35.865 INFO:tasks.workunit.client.1.smithi084.stdout:3/452: truncate d3/d8/d1a/d45/f78 644540 0 2024-02-28T23:07:35.865 INFO:tasks.workunit.client.1.smithi084.stdout:7/452: fsync db/d10/f2e 0 2024-02-28T23:07:35.867 INFO:tasks.workunit.client.1.smithi084.stdout:7/453: readlink db/d11/d27/d36/l5c 0 2024-02-28T23:07:35.868 INFO:tasks.workunit.client.0.smithi057.stdout:4/333: creat d1a/d1b/f77 x:0 0 0 2024-02-28T23:07:35.868 INFO:tasks.workunit.client.1.smithi084.stdout:8/444: mknod d0/d9/d30/d33/d71/c9b 0 2024-02-28T23:07:35.868 INFO:tasks.workunit.client.0.smithi057.stdout:0/346: fdatasync d17/d20/d26/f46 0 2024-02-28T23:07:35.869 INFO:tasks.workunit.client.0.smithi057.stdout:9/425: dread dd/de/d14/f1e [4194304,4194304] 0 2024-02-28T23:07:35.870 INFO:tasks.workunit.client.1.smithi084.stdout:7/454: stat db/fd 0 2024-02-28T23:07:35.870 INFO:tasks.workunit.client.1.smithi084.stdout:1/519: rename d12/d1b/d3b/f59 to d12/d84/fad 0 2024-02-28T23:07:35.871 INFO:tasks.workunit.client.0.smithi057.stdout:0/347: readlink d17/d20/d26/d48/l5f 0 2024-02-28T23:07:35.873 INFO:tasks.workunit.client.0.smithi057.stdout:2/387: rename d8/d11/d21/d28/d41/l58 to d8/d11/d15/l7e 0 2024-02-28T23:07:35.874 INFO:tasks.workunit.client.1.smithi084.stdout:5/511: read dc/de/df/d16/d34/daa/dae/fbb [5414452,13467] 0 2024-02-28T23:07:35.876 INFO:tasks.workunit.client.1.smithi084.stdout:9/501: creat d4/f9c x:0 0 0 2024-02-28T23:07:35.877 INFO:tasks.workunit.client.0.smithi057.stdout:9/426: truncate dd/de/d14/d48/f74 423847 0 2024-02-28T23:07:35.878 INFO:tasks.workunit.client.0.smithi057.stdout:4/334: truncate f15 1021066 0 2024-02-28T23:07:35.879 INFO:tasks.workunit.client.0.smithi057.stdout:9/427: chown dd/de/d24/c2e 0 1 2024-02-28T23:07:35.880 INFO:tasks.workunit.client.1.smithi084.stdout:7/455: mkdir db/d5e/d87 0 2024-02-28T23:07:35.881 INFO:tasks.workunit.client.1.smithi084.stdout:0/459: creat d1/d8/d18/d4c/d6c/d74/f97 x:0 0 0 2024-02-28T23:07:35.882 INFO:tasks.workunit.client.0.smithi057.stdout:9/428: read dd/de/d14/f51 [263546,36323] 0 2024-02-28T23:07:35.883 INFO:tasks.workunit.client.1.smithi084.stdout:5/512: creat dc/fbf x:0 0 0 2024-02-28T23:07:35.884 INFO:tasks.workunit.client.0.smithi057.stdout:2/388: mknod d8/d11/d48/d50/c7f 0 2024-02-28T23:07:35.887 INFO:tasks.workunit.client.1.smithi084.stdout:5/513: rmdir dc/de 39 2024-02-28T23:07:35.887 INFO:tasks.workunit.client.1.smithi084.stdout:7/456: mkdir db/d11/d27/d7a/d88 0 2024-02-28T23:07:35.889 INFO:tasks.workunit.client.0.smithi057.stdout:4/335: rename d1a/d1b/f77 to d1a/d1b/d2d/d49/d4d/f78 0 2024-02-28T23:07:35.890 INFO:tasks.workunit.client.0.smithi057.stdout:3/256: link d3/d8/l2a d3/d4/d1d/d47/l52 0 2024-02-28T23:07:35.891 INFO:tasks.workunit.client.1.smithi084.stdout:7/457: mkdir db/d89 0 2024-02-28T23:07:35.894 INFO:tasks.workunit.client.0.smithi057.stdout:0/348: truncate d17/d20/f2b 176140 0 2024-02-28T23:07:35.894 INFO:tasks.workunit.client.0.smithi057.stdout:2/389: write d8/d11/d56/f7c [1419901,17910] 0 2024-02-28T23:07:35.896 INFO:tasks.workunit.client.1.smithi084.stdout:5/514: mkdir dc/de/df/d16/d34/d3c/d98/dc0 0 2024-02-28T23:07:35.898 INFO:tasks.workunit.client.0.smithi057.stdout:2/390: write d8/d11/f29 [3704771,104762] 0 2024-02-28T23:07:35.898 INFO:tasks.workunit.client.1.smithi084.stdout:7/458: mkdir db/d5e/d8a 0 2024-02-28T23:07:35.899 INFO:tasks.workunit.client.0.smithi057.stdout:8/334: mkdir d4/d13/d18/d62/d66/d74 0 2024-02-28T23:07:35.901 INFO:tasks.workunit.client.0.smithi057.stdout:2/391: truncate d8/d11/d15/f78 436301 0 2024-02-28T23:07:35.902 INFO:tasks.workunit.client.0.smithi057.stdout:5/304: link d4/d9/d14/d3b/c45 d4/d21/d24/c5d 0 2024-02-28T23:07:35.903 INFO:tasks.workunit.client.0.smithi057.stdout:2/392: chown d8/d11/d56/c57 113133 1 2024-02-28T23:07:35.905 INFO:tasks.workunit.client.1.smithi084.stdout:5/515: unlink dc/de/df/d31/f4e 0 2024-02-28T23:07:35.906 INFO:tasks.workunit.client.0.smithi057.stdout:0/349: symlink d17/d20/d23/l87 0 2024-02-28T23:07:35.907 INFO:tasks.workunit.client.1.smithi084.stdout:9/502: chown d4/f56 268 1 2024-02-28T23:07:35.908 INFO:tasks.workunit.client.1.smithi084.stdout:3/453: dwrite d3/d8/d1a/d45/d5e/f62 [0,4194304] 0 2024-02-28T23:07:35.908 INFO:tasks.workunit.client.0.smithi057.stdout:9/429: truncate dd/d1b/d40/d44/f62 3692039 0 2024-02-28T23:07:35.911 INFO:tasks.workunit.client.0.smithi057.stdout:2/393: creat d8/d11/d15/f80 x:0 0 0 2024-02-28T23:07:35.911 INFO:tasks.workunit.client.1.smithi084.stdout:0/460: dwrite d1/d8/d18/d4c/d6c/d74/f97 [0,4194304] 0 2024-02-28T23:07:35.911 INFO:tasks.workunit.client.1.smithi084.stdout:8/445: dread d0/d9/d1b/f74 [0,4194304] 0 2024-02-28T23:07:35.913 INFO:tasks.workunit.client.0.smithi057.stdout:8/335: symlink d4/l75 0 2024-02-28T23:07:35.914 INFO:tasks.workunit.client.1.smithi084.stdout:0/461: fsync d1/d8/f75 0 2024-02-28T23:07:35.914 INFO:tasks.workunit.client.0.smithi057.stdout:3/257: link d3/c4b d3/d8/d1b/c53 0 2024-02-28T23:07:35.919 INFO:tasks.workunit.client.0.smithi057.stdout:4/336: getdents d1a/d1e/d4b/d4f 0 2024-02-28T23:07:35.920 INFO:tasks.workunit.client.1.smithi084.stdout:2/395: truncate d5/d7/d12/d24/d2a/d35/f40 1447772 0 2024-02-28T23:07:35.921 INFO:tasks.workunit.client.0.smithi057.stdout:9/430: mkdir dd/de/d89 0 2024-02-28T23:07:35.923 INFO:tasks.workunit.client.0.smithi057.stdout:5/305: getdents d4/d9/d16/d4e 0 2024-02-28T23:07:35.925 INFO:tasks.workunit.client.1.smithi084.stdout:3/454: unlink c2 0 2024-02-28T23:07:35.925 INFO:tasks.workunit.client.1.smithi084.stdout:0/462: read - d1/d8/f40 zero size 2024-02-28T23:07:35.926 INFO:tasks.workunit.client.0.smithi057.stdout:4/337: chown d1a/d1b/d24/d2e/l30 4 1 2024-02-28T23:07:35.929 INFO:tasks.workunit.client.1.smithi084.stdout:2/396: dread - d5/d7/d1d/f50 zero size 2024-02-28T23:07:35.930 INFO:tasks.workunit.client.1.smithi084.stdout:1/520: dwrite d12/d1b/d2e/d52/f7c [0,4194304] 0 2024-02-28T23:07:35.930 INFO:tasks.workunit.client.0.smithi057.stdout:9/431: write dd/de/d14/f18 [1912549,92559] 0 2024-02-28T23:07:35.931 INFO:tasks.workunit.client.0.smithi057.stdout:5/306: creat d4/d9/d14/f5e x:0 0 0 2024-02-28T23:07:35.939 INFO:tasks.workunit.client.0.smithi057.stdout:2/394: dwrite d8/d11/d48/d50/f7a [0,4194304] 0 2024-02-28T23:07:35.939 INFO:tasks.workunit.client.1.smithi084.stdout:9/503: mkdir d4/d6/d15/d75/d9d 0 2024-02-28T23:07:35.941 INFO:tasks.workunit.client.0.smithi057.stdout:5/307: creat d4/d9/d14/f5f x:0 0 0 2024-02-28T23:07:35.943 INFO:tasks.workunit.client.0.smithi057.stdout:4/338: rename d1a/d1b/d24/f3d to d1a/d48/f79 0 2024-02-28T23:07:35.943 INFO:tasks.workunit.client.1.smithi084.stdout:1/521: creat d12/d1b/d2e/d30/d50/fae x:0 0 0 2024-02-28T23:07:35.943 INFO:tasks.workunit.client.1.smithi084.stdout:5/516: link dc/de/c9f dc/de/df/da6/cc1 0 2024-02-28T23:07:35.947 INFO:tasks.workunit.client.0.smithi057.stdout:9/432: creat dd/d1b/d40/d44/d5c/f8a x:0 0 0 2024-02-28T23:07:35.948 INFO:tasks.workunit.client.1.smithi084.stdout:2/397: write d5/d7/d12/d24/d2a/d35/f40 [146411,102410] 0 2024-02-28T23:07:35.948 INFO:tasks.workunit.client.1.smithi084.stdout:8/446: dwrite d0/d9/d30/d33/d71/f92 [0,4194304] 0 2024-02-28T23:07:35.949 INFO:tasks.workunit.client.1.smithi084.stdout:7/459: dread db/d10/f60 [0,4194304] 0 2024-02-28T23:07:35.949 INFO:tasks.workunit.client.1.smithi084.stdout:2/398: readlink d5/d64/l6a 0 2024-02-28T23:07:35.952 INFO:tasks.workunit.client.1.smithi084.stdout:2/399: chown d5/d7/d12/d24/d2a/f52 5 1 2024-02-28T23:07:35.954 INFO:tasks.workunit.client.0.smithi057.stdout:2/395: rename d8/d11/d21/d28/c3b to d8/d11/d15/c81 0 2024-02-28T23:07:35.954 INFO:tasks.workunit.client.0.smithi057.stdout:9/433: mknod dd/d46/d4d/c8b 0 2024-02-28T23:07:35.959 INFO:tasks.workunit.client.1.smithi084.stdout:5/517: symlink dc/de/df/d16/d34/lc2 0 2024-02-28T23:07:35.959 INFO:tasks.workunit.client.0.smithi057.stdout:4/339: rename d1a/d1b/d24/l34 to d1a/d1e/l7a 0 2024-02-28T23:07:35.960 INFO:tasks.workunit.client.0.smithi057.stdout:4/340: chown d1a/d48 2132736862 1 2024-02-28T23:07:35.961 INFO:tasks.workunit.client.1.smithi084.stdout:2/400: write d5/d7/d12/d24/d2a/d35/f40 [1207886,91664] 0 2024-02-28T23:07:35.963 INFO:tasks.workunit.client.0.smithi057.stdout:9/434: write dd/de/d24/f28 [3858498,117133] 0 2024-02-28T23:07:35.963 INFO:tasks.workunit.client.0.smithi057.stdout:4/341: write f0 [4525710,51473] 0 2024-02-28T23:07:35.965 INFO:tasks.workunit.client.0.smithi057.stdout:2/396: creat d8/d11/d21/d28/f82 x:0 0 0 2024-02-28T23:07:35.968 INFO:tasks.workunit.client.1.smithi084.stdout:9/504: creat d4/d6/d15/d12/f9e x:0 0 0 2024-02-28T23:07:35.968 INFO:tasks.workunit.client.1.smithi084.stdout:7/460: rmdir db/d89 0 2024-02-28T23:07:35.969 INFO:tasks.workunit.client.0.smithi057.stdout:9/435: chown cb 24169 1 2024-02-28T23:07:35.971 INFO:tasks.workunit.client.1.smithi084.stdout:2/401: dread d5/d7/d2e/d4a/f54 [0,4194304] 0 2024-02-28T23:07:35.971 INFO:tasks.workunit.client.0.smithi057.stdout:8/336: dwrite d4/d1e/d56/d6a/d42/f5e [0,4194304] 0 2024-02-28T23:07:35.972 INFO:tasks.workunit.client.1.smithi084.stdout:5/518: creat dc/d51/d7c/fc3 x:0 0 0 2024-02-28T23:07:35.972 INFO:tasks.workunit.client.0.smithi057.stdout:2/397: chown d8/d11/d15/c2e 7417 1 2024-02-28T23:07:35.973 INFO:tasks.workunit.client.0.smithi057.stdout:8/337: dread - d4/f3a zero size 2024-02-28T23:07:35.974 INFO:tasks.workunit.client.1.smithi084.stdout:9/505: symlink d4/d32/d3f/l9f 0 2024-02-28T23:07:35.977 INFO:tasks.workunit.client.0.smithi057.stdout:3/258: dread d3/d20/d2d/f3b [0,4194304] 0 2024-02-28T23:07:35.979 INFO:tasks.workunit.client.0.smithi057.stdout:3/259: stat d3/d8/d1b/c4d 0 2024-02-28T23:07:35.980 INFO:tasks.workunit.client.1.smithi084.stdout:4/420: sync 2024-02-28T23:07:35.980 INFO:tasks.workunit.client.0.smithi057.stdout:5/308: dread d4/f6 [0,4194304] 0 2024-02-28T23:07:35.984 INFO:tasks.workunit.client.1.smithi084.stdout:4/421: readlink d1/l8d 0 2024-02-28T23:07:35.987 INFO:tasks.workunit.client.1.smithi084.stdout:5/519: write dc/de/df/d16/d34/d3c/fb8 [561882,69432] 0 2024-02-28T23:07:35.987 INFO:tasks.workunit.client.1.smithi084.stdout:9/506: read d4/d6/d15/d12/f3d [1162385,109387] 0 2024-02-28T23:07:35.989 INFO:tasks.workunit.client.1.smithi084.stdout:1/522: dwrite d12/d1b/d2e/d30/f58 [0,4194304] 0 2024-02-28T23:07:35.989 INFO:tasks.workunit.client.1.smithi084.stdout:2/402: getdents d5/d64 0 2024-02-28T23:07:35.994 INFO:tasks.workunit.client.1.smithi084.stdout:9/507: chown d4/d6/d15/d12 80 1 2024-02-28T23:07:35.994 INFO:tasks.workunit.client.1.smithi084.stdout:4/422: chown d1/d18/d20/d14/d30/d47 3 1 2024-02-28T23:07:35.997 INFO:tasks.workunit.client.0.smithi057.stdout:2/398: dwrite d8/d11/d21/f67 [0,4194304] 0 2024-02-28T23:07:36.001 INFO:tasks.workunit.client.1.smithi084.stdout:4/423: chown d1/c13 975026 1 2024-02-28T23:07:36.002 INFO:tasks.workunit.client.0.smithi057.stdout:2/399: unlink d8/f16 0 2024-02-28T23:07:36.003 INFO:tasks.workunit.client.1.smithi084.stdout:3/455: dread d3/d27/f2d [4194304,4194304] 0 2024-02-28T23:07:36.003 INFO:tasks.workunit.client.1.smithi084.stdout:8/447: dwrite d0/d9/d30/d33/f94 [0,4194304] 0 2024-02-28T23:07:36.003 INFO:tasks.workunit.client.1.smithi084.stdout:5/520: symlink dc/de/df/d16/d34/daa/dae/lc4 0 2024-02-28T23:07:36.004 INFO:tasks.workunit.client.0.smithi057.stdout:3/260: dwrite d3/f2c [0,4194304] 0 2024-02-28T23:07:36.006 INFO:tasks.workunit.client.1.smithi084.stdout:1/523: creat d12/faf x:0 0 0 2024-02-28T23:07:36.007 INFO:tasks.workunit.client.1.smithi084.stdout:7/461: dwrite db/d11/f32 [0,4194304] 0 2024-02-28T23:07:36.013 INFO:tasks.workunit.client.1.smithi084.stdout:4/424: mknod d1/d18/d20/c90 0 2024-02-28T23:07:36.013 INFO:tasks.workunit.client.0.smithi057.stdout:4/342: dread d1a/d1b/f1f [4194304,4194304] 0 2024-02-28T23:07:36.015 INFO:tasks.workunit.client.1.smithi084.stdout:9/508: write d4/d6/d15/d12/d41/d55/f87 [1985683,130662] 0 2024-02-28T23:07:36.015 INFO:tasks.workunit.client.0.smithi057.stdout:5/309: dwrite d4/d9/d14/d3b/f47 [0,4194304] 0 2024-02-28T23:07:36.017 INFO:tasks.workunit.client.1.smithi084.stdout:3/456: mknod d3/d50/d5a/d73/cad 0 2024-02-28T23:07:36.018 INFO:tasks.workunit.client.0.smithi057.stdout:5/310: readlink d4/d21/l59 0 2024-02-28T23:07:36.021 INFO:tasks.workunit.client.0.smithi057.stdout:3/261: rmdir d3/d8 39 2024-02-28T23:07:36.021 INFO:tasks.workunit.client.1.smithi084.stdout:1/524: mknod d12/d1b/d91/cb0 0 2024-02-28T23:07:36.023 INFO:tasks.workunit.client.1.smithi084.stdout:5/521: creat dc/de/df/d16/d34/daa/dae/fc5 x:0 0 0 2024-02-28T23:07:36.024 INFO:tasks.workunit.client.1.smithi084.stdout:2/403: dread d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:36.028 INFO:tasks.workunit.client.0.smithi057.stdout:4/343: creat d1a/d1b/d2d/f7b x:0 0 0 2024-02-28T23:07:36.030 INFO:tasks.workunit.client.0.smithi057.stdout:1/266: sync 2024-02-28T23:07:36.030 INFO:tasks.workunit.client.0.smithi057.stdout:6/381: sync 2024-02-28T23:07:36.031 INFO:tasks.workunit.client.1.smithi084.stdout:8/448: write d0/d9/d1b/d38/f3a [1192978,60393] 0 2024-02-28T23:07:36.031 INFO:tasks.workunit.client.1.smithi084.stdout:5/522: mknod dc/de/df/d16/d34/d3c/d77/cc6 0 2024-02-28T23:07:36.032 INFO:tasks.workunit.client.0.smithi057.stdout:5/311: unlink d4/d7/l11 0 2024-02-28T23:07:36.033 INFO:tasks.workunit.client.0.smithi057.stdout:6/382: stat d0/d1b/d3e/d3d/d74 0 2024-02-28T23:07:36.034 INFO:tasks.workunit.client.1.smithi084.stdout:2/404: write d5/dd/f58 [1784425,48922] 0 2024-02-28T23:07:36.034 INFO:tasks.workunit.client.1.smithi084.stdout:7/462: rmdir db/d11/d27/d7a/d88 0 2024-02-28T23:07:36.037 INFO:tasks.workunit.client.0.smithi057.stdout:3/262: creat d3/d4f/d4c/f54 x:0 0 0 2024-02-28T23:07:36.038 INFO:tasks.workunit.client.1.smithi084.stdout:5/523: write dc/de/df/d16/d34/d3c/f5a [1249551,16402] 0 2024-02-28T23:07:36.039 INFO:tasks.workunit.client.1.smithi084.stdout:8/449: stat d0/d11/c5a 0 2024-02-28T23:07:36.041 INFO:tasks.workunit.client.1.smithi084.stdout:1/525: link d12/d1b/d2e/d32/f47 d12/d1b/d2e/d52/d61/fb1 0 2024-02-28T23:07:36.043 INFO:tasks.workunit.client.1.smithi084.stdout:0/463: dread d1/d6/db/d35/d4a/f88 [0,4194304] 0 2024-02-28T23:07:36.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/267: creat d2/d7/df/db/d12/f5e x:0 0 0 2024-02-28T23:07:36.046 INFO:tasks.workunit.client.0.smithi057.stdout:8/338: dwrite d4/d13/f2d [0,4194304] 0 2024-02-28T23:07:36.049 INFO:tasks.workunit.client.1.smithi084.stdout:0/464: truncate d1/d8/f75 796356 0 2024-02-28T23:07:36.052 INFO:tasks.workunit.client.1.smithi084.stdout:9/509: getdents d4/d32/d3f 0 2024-02-28T23:07:36.052 INFO:tasks.workunit.client.1.smithi084.stdout:8/450: fsync d0/d9/d63/f7e 0 2024-02-28T23:07:36.052 INFO:tasks.workunit.client.1.smithi084.stdout:7/463: getdents db/d5e/d80 0 2024-02-28T23:07:36.053 INFO:tasks.workunit.client.1.smithi084.stdout:5/524: mkdir dc/dc7 0 2024-02-28T23:07:36.055 INFO:tasks.workunit.client.0.smithi057.stdout:1/268: dread - d2/d7/df/db/d3a/f38 zero size 2024-02-28T23:07:36.055 INFO:tasks.workunit.client.0.smithi057.stdout:5/312: mkdir d4/d9/d16/d4e/d60 0 2024-02-28T23:07:36.055 INFO:tasks.workunit.client.0.smithi057.stdout:3/263: readlink d3/d4/d7/l3d 0 2024-02-28T23:07:36.055 INFO:tasks.workunit.client.0.smithi057.stdout:4/344: symlink d1a/l7c 0 2024-02-28T23:07:36.057 INFO:tasks.workunit.client.1.smithi084.stdout:7/464: write db/d5e/f73 [2004,75153] 0 2024-02-28T23:07:36.059 INFO:tasks.workunit.client.0.smithi057.stdout:1/269: mknod d2/d7/df/db/d12/d3f/c5f 0 2024-02-28T23:07:36.060 INFO:tasks.workunit.client.0.smithi057.stdout:2/400: truncate d8/d11/d21/f33 1748900 0 2024-02-28T23:07:36.061 INFO:tasks.workunit.client.1.smithi084.stdout:0/465: creat d1/d6/db/d35/d44/f98 x:0 0 0 2024-02-28T23:07:36.061 INFO:tasks.workunit.client.1.smithi084.stdout:5/525: read dc/d51/d7c/f84 [829889,55771] 0 2024-02-28T23:07:36.062 INFO:tasks.workunit.client.1.smithi084.stdout:7/465: write db/d11/f16 [1582931,5993] 0 2024-02-28T23:07:36.062 INFO:tasks.workunit.client.0.smithi057.stdout:5/313: mkdir d4/d9/d14/d3b/d61 0 2024-02-28T23:07:36.066 INFO:tasks.workunit.client.0.smithi057.stdout:1/270: symlink d2/d7/df/db/d3a/l60 0 2024-02-28T23:07:36.066 INFO:tasks.workunit.client.0.smithi057.stdout:6/383: dwrite d0/f6f [0,4194304] 0 2024-02-28T23:07:36.066 INFO:tasks.workunit.client.0.smithi057.stdout:2/401: mkdir d8/d11/d39/d83 0 2024-02-28T23:07:36.067 INFO:tasks.workunit.client.1.smithi084.stdout:0/466: write d1/d6/d2d/f30 [746947,47576] 0 2024-02-28T23:07:36.069 INFO:tasks.workunit.client.0.smithi057.stdout:5/314: mkdir d4/d21/d24/d30/d62 0 2024-02-28T23:07:36.069 INFO:tasks.workunit.client.1.smithi084.stdout:0/467: fsync d1/d8/f1f 0 2024-02-28T23:07:36.070 INFO:tasks.workunit.client.0.smithi057.stdout:6/384: dread - d0/d1b/d3e/d3d/d42/d4f/d53/f80 zero size 2024-02-28T23:07:36.070 INFO:tasks.workunit.client.0.smithi057.stdout:1/271: write d2/d7/df/db/f28 [723005,44827] 0 2024-02-28T23:07:36.073 INFO:tasks.workunit.client.1.smithi084.stdout:7/466: symlink db/l8b 0 2024-02-28T23:07:36.073 INFO:tasks.workunit.client.0.smithi057.stdout:6/385: fsync d0/d1b/d54/f65 0 2024-02-28T23:07:36.076 INFO:tasks.workunit.client.1.smithi084.stdout:5/526: creat dc/fc8 x:0 0 0 2024-02-28T23:07:36.076 INFO:tasks.workunit.client.0.smithi057.stdout:5/315: mkdir d4/d9/d14/d63 0 2024-02-28T23:07:36.077 INFO:tasks.workunit.client.1.smithi084.stdout:9/510: dwrite d4/f90 [0,4194304] 0 2024-02-28T23:07:36.078 INFO:tasks.workunit.client.1.smithi084.stdout:0/468: creat d1/d6/db/d37/d72/f99 x:0 0 0 2024-02-28T23:07:36.079 INFO:tasks.workunit.client.1.smithi084.stdout:4/425: truncate d1/d18/d20/d14/f36 771338 0 2024-02-28T23:07:36.080 INFO:tasks.workunit.client.1.smithi084.stdout:5/527: truncate dc/f27 1907187 0 2024-02-28T23:07:36.081 INFO:tasks.workunit.client.0.smithi057.stdout:6/386: write d0/d4/d25/f52 [2347703,111029] 0 2024-02-28T23:07:36.083 INFO:tasks.workunit.client.1.smithi084.stdout:7/467: write db/ff [3206084,95298] 0 2024-02-28T23:07:36.084 INFO:tasks.workunit.client.1.smithi084.stdout:0/469: fdatasync d1/d8/d18/d4c/d6c/d74/f91 0 2024-02-28T23:07:36.090 INFO:tasks.workunit.client.0.smithi057.stdout:8/339: truncate d4/d13/d18/f27 360937 0 2024-02-28T23:07:36.092 INFO:tasks.workunit.client.1.smithi084.stdout:4/426: read d1/d18/d20/d14/d30/f31 [530379,68121] 0 2024-02-28T23:07:36.094 INFO:tasks.workunit.client.1.smithi084.stdout:5/528: symlink dc/de/df/da6/lc9 0 2024-02-28T23:07:36.095 INFO:tasks.workunit.client.1.smithi084.stdout:9/511: stat d4/d6/d15/d75/l88 0 2024-02-28T23:07:36.102 INFO:tasks.workunit.client.1.smithi084.stdout:0/470: dread d1/d6/f53 [0,4194304] 0 2024-02-28T23:07:36.102 INFO:tasks.workunit.client.0.smithi057.stdout:6/387: unlink d0/d1b/f7a 0 2024-02-28T23:07:36.103 INFO:tasks.workunit.client.1.smithi084.stdout:2/405: dwrite d5/d1f/f41 [0,4194304] 0 2024-02-28T23:07:36.103 INFO:tasks.workunit.client.0.smithi057.stdout:1/272: getdents d2/d7/df/db/d12 0 2024-02-28T23:07:36.106 INFO:tasks.workunit.client.0.smithi057.stdout:3/264: dwrite d3/d20/d2d/f3b [4194304,4194304] 0 2024-02-28T23:07:36.106 INFO:tasks.workunit.client.1.smithi084.stdout:1/526: dwrite d12/f70 [0,4194304] 0 2024-02-28T23:07:36.108 INFO:tasks.workunit.client.0.smithi057.stdout:1/273: readlink d2/d7/df/db/d12/d3f/l4e 0 2024-02-28T23:07:36.109 INFO:tasks.workunit.client.1.smithi084.stdout:1/527: chown d12/f23 0 1 2024-02-28T23:07:36.110 INFO:tasks.workunit.client.0.smithi057.stdout:3/265: read - d3/d4/d1d/f36 zero size 2024-02-28T23:07:36.110 INFO:tasks.workunit.client.0.smithi057.stdout:4/345: dread d1a/d2f/f31 [0,4194304] 0 2024-02-28T23:07:36.110 INFO:tasks.workunit.client.0.smithi057.stdout:6/388: creat d0/d17/f82 x:0 0 0 2024-02-28T23:07:36.112 INFO:tasks.workunit.client.1.smithi084.stdout:0/471: chown d1/d6/db/d35/d4a 58163218 1 2024-02-28T23:07:36.113 INFO:tasks.workunit.client.1.smithi084.stdout:1/528: dread - d12/da2/fa9 zero size 2024-02-28T23:07:36.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/316: dwrite d4/d9/d14/f36 [0,4194304] 0 2024-02-28T23:07:36.118 INFO:tasks.workunit.client.0.smithi057.stdout:4/346: creat d1a/d1b/d24/d2e/f7d x:0 0 0 2024-02-28T23:07:36.119 INFO:tasks.workunit.client.1.smithi084.stdout:2/406: mkdir d5/d1f/d42/d74 0 2024-02-28T23:07:36.124 INFO:tasks.workunit.client.1.smithi084.stdout:1/529: stat d12/d1b/d2e/f4e 0 2024-02-28T23:07:36.125 INFO:tasks.workunit.client.0.smithi057.stdout:4/347: stat d1a/d1b/d24/c44 0 2024-02-28T23:07:36.128 INFO:tasks.workunit.client.0.smithi057.stdout:3/266: creat d3/d8/d27/f55 x:0 0 0 2024-02-28T23:07:36.130 INFO:tasks.workunit.client.0.smithi057.stdout:4/348: symlink d1a/d48/l7e 0 2024-02-28T23:07:36.135 INFO:tasks.workunit.client.0.smithi057.stdout:3/267: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:36.135 INFO:tasks.workunit.client.0.smithi057.stdout:4/349: mkdir d1a/d1b/d7f 0 2024-02-28T23:07:36.136 INFO:tasks.workunit.client.1.smithi084.stdout:7/468: dwrite db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:36.136 INFO:tasks.workunit.client.1.smithi084.stdout:1/530: stat d12/d1b/d2e/d52/f5a 0 2024-02-28T23:07:36.137 INFO:tasks.workunit.client.0.smithi057.stdout:2/402: dread d8/d11/d48/d50/f66 [0,4194304] 0 2024-02-28T23:07:36.137 INFO:tasks.workunit.client.0.smithi057.stdout:4/350: fdatasync d1a/d1e/d4b/f5d 0 2024-02-28T23:07:36.139 INFO:tasks.workunit.client.0.smithi057.stdout:3/268: creat d3/d20/d2d/f56 x:0 0 0 2024-02-28T23:07:36.140 INFO:tasks.workunit.client.1.smithi084.stdout:5/529: dwrite dc/de/df/d6e/faf [4194304,4194304] 0 2024-02-28T23:07:36.141 INFO:tasks.workunit.client.1.smithi084.stdout:0/472: write d1/d6/db/d35/f5c [303549,56746] 0 2024-02-28T23:07:36.141 INFO:tasks.workunit.client.0.smithi057.stdout:5/317: dread d4/d9/d16/f19 [0,4194304] 0 2024-02-28T23:07:36.142 INFO:tasks.workunit.client.1.smithi084.stdout:7/469: write db/fd [3998407,103398] 0 2024-02-28T23:07:36.142 INFO:tasks.workunit.client.0.smithi057.stdout:4/351: symlink d1a/d1b/d2d/l80 0 2024-02-28T23:07:36.143 INFO:tasks.workunit.client.1.smithi084.stdout:2/407: chown d5/d7/f3f 767 1 2024-02-28T23:07:36.143 INFO:tasks.workunit.client.1.smithi084.stdout:8/451: dread d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:36.144 INFO:tasks.workunit.client.0.smithi057.stdout:3/269: readlink d3/d8/l2a 0 2024-02-28T23:07:36.146 INFO:tasks.workunit.client.1.smithi084.stdout:7/470: chown db/ff 417 1 2024-02-28T23:07:36.147 INFO:tasks.workunit.client.0.smithi057.stdout:2/403: creat d8/f84 x:0 0 0 2024-02-28T23:07:36.147 INFO:tasks.workunit.client.1.smithi084.stdout:4/427: dwrite d1/d18/d20/d24/f8b [0,4194304] 0 2024-02-28T23:07:36.147 INFO:tasks.workunit.client.1.smithi084.stdout:1/531: rmdir d12/d1b/d2e 39 2024-02-28T23:07:36.150 INFO:tasks.workunit.client.0.smithi057.stdout:3/270: mknod d3/d20/d2d/c57 0 2024-02-28T23:07:36.150 INFO:tasks.workunit.client.0.smithi057.stdout:4/352: write d1a/d1e/d4b/d4f/f64 [232750,110372] 0 2024-02-28T23:07:36.152 INFO:tasks.workunit.client.1.smithi084.stdout:4/428: fdatasync d1/d18/d20/d14/d30/f3d 0 2024-02-28T23:07:36.153 INFO:tasks.workunit.client.1.smithi084.stdout:2/408: mkdir d5/d7/d75 0 2024-02-28T23:07:36.154 INFO:tasks.workunit.client.1.smithi084.stdout:6/466: sync 2024-02-28T23:07:36.156 INFO:tasks.workunit.client.0.smithi057.stdout:3/271: write d3/d8/d27/f55 [894782,94858] 0 2024-02-28T23:07:36.157 INFO:tasks.workunit.client.1.smithi084.stdout:4/429: chown d1/d18/d20/d14/d30/f38 6865187 1 2024-02-28T23:07:36.159 INFO:tasks.workunit.client.0.smithi057.stdout:4/353: mkdir d1a/d1e/d81 0 2024-02-28T23:07:36.159 INFO:tasks.workunit.client.1.smithi084.stdout:1/532: readlink d12/d1b/d2e/d30/l2d 0 2024-02-28T23:07:36.160 INFO:tasks.workunit.client.1.smithi084.stdout:9/512: dwrite d4/d6/d15/f98 [0,4194304] 0 2024-02-28T23:07:36.164 INFO:tasks.workunit.client.1.smithi084.stdout:5/530: creat dc/de/fca x:0 0 0 2024-02-28T23:07:36.165 INFO:tasks.workunit.client.0.smithi057.stdout:3/272: symlink d3/d4/l58 0 2024-02-28T23:07:36.167 INFO:tasks.workunit.client.1.smithi084.stdout:6/467: creat d1/d13/d50/fa6 x:0 0 0 2024-02-28T23:07:36.167 INFO:tasks.workunit.client.0.smithi057.stdout:2/404: dwrite d8/fd [4194304,4194304] 0 2024-02-28T23:07:36.167 INFO:tasks.workunit.client.0.smithi057.stdout:1/274: dwrite d2/d7/f41 [0,4194304] 0 2024-02-28T23:07:36.170 INFO:tasks.workunit.client.0.smithi057.stdout:2/405: fdatasync d8/f64 0 2024-02-28T23:07:36.171 INFO:tasks.workunit.client.1.smithi084.stdout:2/409: rename d5/dd/f34 to d5/d1f/d42/d74/f76 0 2024-02-28T23:07:36.172 INFO:tasks.workunit.client.1.smithi084.stdout:4/430: rename d1/d18 to d1/d18/d20/d24/d2f/d3e/d52/d48/d91 22 2024-02-28T23:07:36.173 INFO:tasks.workunit.client.0.smithi057.stdout:2/406: chown d8/d11/c44 30274304 1 2024-02-28T23:07:36.177 INFO:tasks.workunit.client.1.smithi084.stdout:5/531: creat dc/de/df/d31/fcb x:0 0 0 2024-02-28T23:07:36.177 INFO:tasks.workunit.client.1.smithi084.stdout:0/473: getdents d1/d8 0 2024-02-28T23:07:36.179 INFO:tasks.workunit.client.0.smithi057.stdout:1/275: write d2/d7/df/db/d12/d19/f23 [1016064,37450] 0 2024-02-28T23:07:36.183 INFO:tasks.workunit.client.1.smithi084.stdout:0/474: chown d1/d8/d5f 19905047 1 2024-02-28T23:07:36.184 INFO:tasks.workunit.client.1.smithi084.stdout:1/533: dread d12/f45 [0,4194304] 0 2024-02-28T23:07:36.187 INFO:tasks.workunit.client.1.smithi084.stdout:6/468: creat d1/d13/d9d/da3/fa7 x:0 0 0 2024-02-28T23:07:36.187 INFO:tasks.workunit.client.0.smithi057.stdout:1/276: mknod d2/d7/df/db/d3a/c61 0 2024-02-28T23:07:36.190 INFO:tasks.workunit.client.0.smithi057.stdout:1/277: chown d2/d7/df/db/d3a/f3b 3 1 2024-02-28T23:07:36.190 INFO:tasks.workunit.client.0.smithi057.stdout:3/273: dwrite d3/d4/d1d/f2e [0,4194304] 0 2024-02-28T23:07:36.193 INFO:tasks.workunit.client.1.smithi084.stdout:3/457: sync 2024-02-28T23:07:36.194 INFO:tasks.workunit.client.0.smithi057.stdout:1/278: write d2/f21 [987244,32609] 0 2024-02-28T23:07:36.194 INFO:tasks.workunit.client.0.smithi057.stdout:6/389: dwrite d0/d4/ff [0,4194304] 0 2024-02-28T23:07:36.195 INFO:tasks.workunit.client.0.smithi057.stdout:3/274: write d3/d20/f24 [778079,115467] 0 2024-02-28T23:07:36.197 INFO:tasks.workunit.client.1.smithi084.stdout:8/452: dwrite d0/d9/d1b/d38/f5d [0,4194304] 0 2024-02-28T23:07:36.201 INFO:tasks.workunit.client.0.smithi057.stdout:6/390: stat d0/d17/f35 0 2024-02-28T23:07:36.203 INFO:tasks.workunit.client.1.smithi084.stdout:5/532: creat dc/de/df/d31/d8e/da9/fcc x:0 0 0 2024-02-28T23:07:36.205 INFO:tasks.workunit.client.1.smithi084.stdout:7/471: dwrite db/d11/d27/d36/f4c [0,4194304] 0 2024-02-28T23:07:36.209 INFO:tasks.workunit.client.1.smithi084.stdout:1/534: symlink d12/d1b/d2e/d30/d50/lb2 0 2024-02-28T23:07:36.209 INFO:tasks.workunit.client.0.smithi057.stdout:6/391: symlink d0/d17/l83 0 2024-02-28T23:07:36.212 INFO:tasks.workunit.client.0.smithi057.stdout:7/342: sync 2024-02-28T23:07:36.212 INFO:tasks.workunit.client.0.smithi057.stdout:6/392: readlink d0/d1b/d3e/d40/l47 0 2024-02-28T23:07:36.215 INFO:tasks.workunit.client.0.smithi057.stdout:6/393: fsync d0/d17/f82 0 2024-02-28T23:07:36.217 INFO:tasks.workunit.client.1.smithi084.stdout:6/469: mknod d1/d9/ca8 0 2024-02-28T23:07:36.218 INFO:tasks.workunit.client.1.smithi084.stdout:1/535: fsync d12/d1b/d63/f2a 0 2024-02-28T23:07:36.220 INFO:tasks.workunit.client.0.smithi057.stdout:7/343: chown d0/f67 252785 1 2024-02-28T23:07:36.222 INFO:tasks.workunit.client.0.smithi057.stdout:1/279: dwrite d2/d7/df/db/d12/d19/d24/d2d/f4c [0,4194304] 0 2024-02-28T23:07:36.224 INFO:tasks.workunit.client.1.smithi084.stdout:8/453: write d0/d9/d30/d33/f5e [350497,32648] 0 2024-02-28T23:07:36.224 INFO:tasks.workunit.client.1.smithi084.stdout:2/410: dwrite d5/d1f/f41 [0,4194304] 0 2024-02-28T23:07:36.224 INFO:tasks.workunit.client.1.smithi084.stdout:5/533: symlink dc/de/df/d16/d34/lcd 0 2024-02-28T23:07:36.225 INFO:tasks.workunit.client.1.smithi084.stdout:0/475: rename d1/d8/d18/f63 to d1/d8/d18/f9a 0 2024-02-28T23:07:36.225 INFO:tasks.workunit.client.1.smithi084.stdout:5/534: truncate dc/fbf 361502 0 2024-02-28T23:07:36.227 INFO:tasks.workunit.client.0.smithi057.stdout:6/394: creat d0/d17/d20/f84 x:0 0 0 2024-02-28T23:07:36.230 INFO:tasks.workunit.client.1.smithi084.stdout:1/536: fdatasync d12/d1b/d63/d7d/f8c 0 2024-02-28T23:07:36.232 INFO:tasks.workunit.client.1.smithi084.stdout:6/470: creat d1/d13/d2e/d41/fa9 x:0 0 0 2024-02-28T23:07:36.233 INFO:tasks.workunit.client.0.smithi057.stdout:6/395: mknod d0/d17/c85 0 2024-02-28T23:07:36.236 INFO:tasks.workunit.client.0.smithi057.stdout:3/275: dread d3/d8/d27/f55 [0,4194304] 0 2024-02-28T23:07:36.237 INFO:tasks.workunit.client.0.smithi057.stdout:2/407: dread d8/d11/f17 [0,4194304] 0 2024-02-28T23:07:36.238 INFO:tasks.workunit.client.1.smithi084.stdout:0/476: creat d1/d8/d69/d6b/f9b x:0 0 0 2024-02-28T23:07:36.241 INFO:tasks.workunit.client.1.smithi084.stdout:7/472: getdents db/d10/d17/d64/d77 0 2024-02-28T23:07:36.241 INFO:tasks.workunit.client.0.smithi057.stdout:6/396: symlink d0/d1b/d3e/d3d/l86 0 2024-02-28T23:07:36.243 INFO:tasks.workunit.client.1.smithi084.stdout:1/537: fsync d12/d1b/d3b/f42 0 2024-02-28T23:07:36.246 INFO:tasks.workunit.client.1.smithi084.stdout:6/471: unlink d1/d13/d2e/d41/d4d/d6c/f77 0 2024-02-28T23:07:36.248 INFO:tasks.workunit.client.1.smithi084.stdout:7/473: fdatasync db/d10/d17/d64/f69 0 2024-02-28T23:07:36.249 INFO:tasks.workunit.client.1.smithi084.stdout:5/535: creat dc/de/df/d31/d8e/da9/fce x:0 0 0 2024-02-28T23:07:36.253 INFO:tasks.workunit.client.1.smithi084.stdout:3/458: dwrite d3/d8/f10 [0,4194304] 0 2024-02-28T23:07:36.254 INFO:tasks.workunit.client.0.smithi057.stdout:6/397: unlink d0/d4/d25/f2f 0 2024-02-28T23:07:36.259 INFO:tasks.workunit.client.0.smithi057.stdout:2/408: getdents d8/d11 0 2024-02-28T23:07:36.259 INFO:tasks.workunit.client.1.smithi084.stdout:6/472: dread - d1/d9/d1d/d1e/d33/f6d zero size 2024-02-28T23:07:36.259 INFO:tasks.workunit.client.1.smithi084.stdout:3/459: write d3/d50/f66 [2758347,85493] 0 2024-02-28T23:07:36.259 INFO:tasks.workunit.client.0.smithi057.stdout:3/276: getdents d3/d4/d1d/d47 0 2024-02-28T23:07:36.261 INFO:tasks.workunit.client.1.smithi084.stdout:0/477: link d1/d6/db/d35/d4a/f88 d1/d6/db/d76/f9c 0 2024-02-28T23:07:36.263 INFO:tasks.workunit.client.1.smithi084.stdout:7/474: stat db/d11/f16 0 2024-02-28T23:07:36.264 INFO:tasks.workunit.client.0.smithi057.stdout:6/398: chown d0/d17/c75 1738 1 2024-02-28T23:07:36.267 INFO:tasks.workunit.client.0.smithi057.stdout:3/277: write d3/d8/f9 [725094,14464] 0 2024-02-28T23:07:36.267 INFO:tasks.workunit.client.0.smithi057.stdout:1/280: dwrite d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:36.268 INFO:tasks.workunit.client.1.smithi084.stdout:4/431: dread d1/f6 [0,4194304] 0 2024-02-28T23:07:36.268 INFO:tasks.workunit.client.1.smithi084.stdout:1/538: write d12/d1b/d2e/d52/f6d [586790,72041] 0 2024-02-28T23:07:36.270 INFO:tasks.workunit.client.0.smithi057.stdout:7/344: dwrite d0/d5/d36/d4e/d15/f42 [0,4194304] 0 2024-02-28T23:07:36.272 INFO:tasks.workunit.client.1.smithi084.stdout:7/475: creat db/d10/d17/d64/f8c x:0 0 0 2024-02-28T23:07:36.275 INFO:tasks.workunit.client.0.smithi057.stdout:1/281: mkdir d2/d7/df/db/d12/d19/d24/d62 0 2024-02-28T23:07:36.275 INFO:tasks.workunit.client.1.smithi084.stdout:3/460: write d3/d8/d1a/d45/f78 [675057,19149] 0 2024-02-28T23:07:36.275 INFO:tasks.workunit.client.1.smithi084.stdout:8/454: dread d0/d9/d30/d33/f5e [0,4194304] 0 2024-02-28T23:07:36.279 INFO:tasks.workunit.client.1.smithi084.stdout:5/536: rename dc/de/df/d16/d34/d3c/d77/cc6 to dc/de/df/d16/d34/d3c/d77/ccf 0 2024-02-28T23:07:36.279 INFO:tasks.workunit.client.1.smithi084.stdout:7/476: chown db/d11/f12 145685 1 2024-02-28T23:07:36.279 INFO:tasks.workunit.client.1.smithi084.stdout:0/478: getdents d1/d8/d69/d6b/d85 0 2024-02-28T23:07:36.279 INFO:tasks.workunit.client.0.smithi057.stdout:1/282: chown d2/d7/df/db/d3a/d2a 840924710 1 2024-02-28T23:07:36.281 INFO:tasks.workunit.client.1.smithi084.stdout:1/539: creat d12/d1b/d2e/d52/d5b/fb3 x:0 0 0 2024-02-28T23:07:36.281 INFO:tasks.workunit.client.1.smithi084.stdout:8/455: read - d0/d9/f88 zero size 2024-02-28T23:07:36.283 INFO:tasks.workunit.client.0.smithi057.stdout:2/409: dread d8/f64 [0,4194304] 0 2024-02-28T23:07:36.284 INFO:tasks.workunit.client.1.smithi084.stdout:9/513: dread d4/d6/d15/d75/d2d/f34 [0,4194304] 0 2024-02-28T23:07:36.285 INFO:tasks.workunit.client.1.smithi084.stdout:8/456: chown d0/d24 86 1 2024-02-28T23:07:36.286 INFO:tasks.workunit.client.1.smithi084.stdout:4/432: creat d1/d18/d20/d24/d2f/d3e/d52/d45/f92 x:0 0 0 2024-02-28T23:07:36.291 INFO:tasks.workunit.client.0.smithi057.stdout:2/410: unlink d8/lc 0 2024-02-28T23:07:36.293 INFO:tasks.workunit.client.0.smithi057.stdout:7/345: rename d0/f35 to d0/d5/d36/d4e/d15/f6a 0 2024-02-28T23:07:36.294 INFO:tasks.workunit.client.1.smithi084.stdout:3/461: mkdir d3/dae 0 2024-02-28T23:07:36.294 INFO:tasks.workunit.client.1.smithi084.stdout:8/457: read d0/f1a [439331,35653] 0 2024-02-28T23:07:36.295 INFO:tasks.workunit.client.1.smithi084.stdout:1/540: mknod d12/da2/cb4 0 2024-02-28T23:07:36.296 INFO:tasks.workunit.client.0.smithi057.stdout:3/278: dwrite d3/d4/d7/f10 [4194304,4194304] 0 2024-02-28T23:07:36.297 INFO:tasks.workunit.client.0.smithi057.stdout:2/411: mkdir d8/d11/d21/d28/d49/d85 0 2024-02-28T23:07:36.301 INFO:tasks.workunit.client.1.smithi084.stdout:7/477: creat db/d10/d18/f8d x:0 0 0 2024-02-28T23:07:36.302 INFO:tasks.workunit.client.1.smithi084.stdout:4/433: creat d1/d18/d20/d24/d2f/d3e/d52/d45/f93 x:0 0 0 2024-02-28T23:07:36.302 INFO:tasks.workunit.client.0.smithi057.stdout:2/412: write d8/f19 [1960354,107118] 0 2024-02-28T23:07:36.303 INFO:tasks.workunit.client.0.smithi057.stdout:6/399: dwrite d0/d1b/d3e/d3d/d42/d4f/d53/f80 [0,4194304] 0 2024-02-28T23:07:36.303 INFO:tasks.workunit.client.0.smithi057.stdout:2/413: fdatasync d8/d11/d15/f71 0 2024-02-28T23:07:36.306 INFO:tasks.workunit.client.0.smithi057.stdout:7/346: rename d0/d5/d25/d3e/f62 to d0/d5/d36/d4e/d15/d29/d3d/d51/f6b 0 2024-02-28T23:07:36.308 INFO:tasks.workunit.client.0.smithi057.stdout:7/347: stat d0/d5/f55 0 2024-02-28T23:07:36.309 INFO:tasks.workunit.client.1.smithi084.stdout:0/479: creat d1/d8/d78/f9d x:0 0 0 2024-02-28T23:07:36.310 INFO:tasks.workunit.client.1.smithi084.stdout:8/458: creat d0/d9/d1b/d38/d43/d51/d54/f9c x:0 0 0 2024-02-28T23:07:36.311 INFO:tasks.workunit.client.0.smithi057.stdout:3/279: creat d3/d8/f59 x:0 0 0 2024-02-28T23:07:36.313 INFO:tasks.workunit.client.1.smithi084.stdout:7/478: truncate db/d10/d17/d64/f81 315033 0 2024-02-28T23:07:36.313 INFO:tasks.workunit.client.0.smithi057.stdout:2/414: fdatasync d8/fb 0 2024-02-28T23:07:36.317 INFO:tasks.workunit.client.0.smithi057.stdout:2/415: write d8/d11/d48/d50/f6d [76172,99249] 0 2024-02-28T23:07:36.317 INFO:tasks.workunit.client.0.smithi057.stdout:7/348: creat d0/d3f/f6c x:0 0 0 2024-02-28T23:07:36.318 INFO:tasks.workunit.client.1.smithi084.stdout:8/459: fsync d0/d9/f4c 0 2024-02-28T23:07:36.324 INFO:tasks.workunit.client.1.smithi084.stdout:2/411: truncate d5/d7/d9/f2c 1786276 0 2024-02-28T23:07:36.325 INFO:tasks.workunit.client.1.smithi084.stdout:4/434: mknod d1/d18/d20/d24/d2f/d3e/d52/d48/d73/c94 0 2024-02-28T23:07:36.325 INFO:tasks.workunit.client.0.smithi057.stdout:7/349: write d0/d5/d36/d4e/d15/d29/d3d/d51/f5e [5179611,44781] 0 2024-02-28T23:07:36.325 INFO:tasks.workunit.client.0.smithi057.stdout:2/416: dread d8/d11/d39/f3c [0,4194304] 0 2024-02-28T23:07:36.326 INFO:tasks.workunit.client.1.smithi084.stdout:8/460: write d0/d24/d4d/f6a [3225856,123350] 0 2024-02-28T23:07:36.326 INFO:tasks.workunit.client.1.smithi084.stdout:0/480: truncate d1/d8/d18/d4c/d6c/d74/f91 718464 0 2024-02-28T23:07:36.328 INFO:tasks.workunit.client.1.smithi084.stdout:7/479: creat db/d11/d27/d36/f8e x:0 0 0 2024-02-28T23:07:36.330 INFO:tasks.workunit.client.0.smithi057.stdout:6/400: dwrite d0/d1b/d3e/d40/f4d [0,4194304] 0 2024-02-28T23:07:36.330 INFO:tasks.workunit.client.1.smithi084.stdout:1/541: dwrite d12/d1b/d2e/d52/d5b/f8a [0,4194304] 0 2024-02-28T23:07:36.332 INFO:tasks.workunit.client.0.smithi057.stdout:3/280: dread d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:36.333 INFO:tasks.workunit.client.0.smithi057.stdout:7/350: rmdir d0/d5/d36/d4e/d15/d29/d3d/d51 39 2024-02-28T23:07:36.333 INFO:tasks.workunit.client.0.smithi057.stdout:6/401: truncate d0/d17/f2b 1058463 0 2024-02-28T23:07:36.334 INFO:tasks.workunit.client.0.smithi057.stdout:2/417: creat d8/d11/d21/f86 x:0 0 0 2024-02-28T23:07:36.335 INFO:tasks.workunit.client.1.smithi084.stdout:2/412: chown d5/d7/d12/d24/d2a/d35/f5d 29425885 1 2024-02-28T23:07:36.336 INFO:tasks.workunit.client.0.smithi057.stdout:6/402: chown d0/d17/l83 8 1 2024-02-28T23:07:36.337 INFO:tasks.workunit.client.1.smithi084.stdout:2/413: write d5/d64/f6c [479668,3866] 0 2024-02-28T23:07:36.337 INFO:tasks.workunit.client.1.smithi084.stdout:8/461: rmdir d0/d44 39 2024-02-28T23:07:36.338 INFO:tasks.workunit.client.0.smithi057.stdout:3/281: fdatasync d3/d4/f12 0 2024-02-28T23:07:36.339 INFO:tasks.workunit.client.1.smithi084.stdout:9/514: dread d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:36.339 INFO:tasks.workunit.client.0.smithi057.stdout:2/418: fdatasync d8/d11/f17 0 2024-02-28T23:07:36.342 INFO:tasks.workunit.client.0.smithi057.stdout:7/351: mknod d0/d5/d36/d4e/c6d 0 2024-02-28T23:07:36.344 INFO:tasks.workunit.client.0.smithi057.stdout:3/282: creat d3/f5a x:0 0 0 2024-02-28T23:07:36.345 INFO:tasks.workunit.client.1.smithi084.stdout:0/481: symlink d1/d6/db/d37/l9e 0 2024-02-28T23:07:36.347 INFO:tasks.workunit.client.0.smithi057.stdout:3/283: symlink d3/d4/d7/l5b 0 2024-02-28T23:07:36.348 INFO:tasks.workunit.client.1.smithi084.stdout:8/462: read d0/d9/d1b/d38/d43/d51/d7c/f80 [4018341,126071] 0 2024-02-28T23:07:36.350 INFO:tasks.workunit.client.0.smithi057.stdout:3/284: write d3/f5a [414415,107372] 0 2024-02-28T23:07:36.351 INFO:tasks.workunit.client.1.smithi084.stdout:2/414: mknod d5/d1f/d42/d74/c77 0 2024-02-28T23:07:36.355 INFO:tasks.workunit.client.0.smithi057.stdout:2/419: read d8/d11/f5b [554020,30992] 0 2024-02-28T23:07:36.355 INFO:tasks.workunit.client.1.smithi084.stdout:0/482: dread - d1/d6/db/d35/d57/f6a zero size 2024-02-28T23:07:36.358 INFO:tasks.workunit.client.0.smithi057.stdout:2/420: creat d8/d11/d39/f87 x:0 0 0 2024-02-28T23:07:36.359 INFO:tasks.workunit.client.0.smithi057.stdout:2/421: chown d8 1 1 2024-02-28T23:07:36.360 INFO:tasks.workunit.client.1.smithi084.stdout:5/537: dwrite dc/de/df/d16/d34/f53 [0,4194304] 0 2024-02-28T23:07:36.361 INFO:tasks.workunit.client.1.smithi084.stdout:3/462: dread d3/d8/f4f [0,4194304] 0 2024-02-28T23:07:36.361 INFO:tasks.workunit.client.1.smithi084.stdout:4/435: creat d1/d18/d20/d65/d8e/f95 x:0 0 0 2024-02-28T23:07:36.363 INFO:tasks.workunit.client.1.smithi084.stdout:9/515: symlink d4/d6/d15/d12/la0 0 2024-02-28T23:07:36.364 INFO:tasks.workunit.client.1.smithi084.stdout:0/483: fsync d1/d6/db/d35/d44/f98 0 2024-02-28T23:07:36.366 INFO:tasks.workunit.client.1.smithi084.stdout:7/480: truncate db/d10/f2e 889107 0 2024-02-28T23:07:36.368 INFO:tasks.workunit.client.1.smithi084.stdout:5/538: write dc/de/f14 [4966228,103004] 0 2024-02-28T23:07:36.372 INFO:tasks.workunit.client.1.smithi084.stdout:4/436: chown d1/ca 5892103 1 2024-02-28T23:07:36.377 INFO:tasks.workunit.client.1.smithi084.stdout:2/415: truncate d5/d7/d12/d24/d2a/d53/f5f 1152316 0 2024-02-28T23:07:36.377 INFO:tasks.workunit.client.1.smithi084.stdout:6/473: sync 2024-02-28T23:07:36.378 INFO:tasks.workunit.client.1.smithi084.stdout:7/481: creat db/d10/d17/f8f x:0 0 0 2024-02-28T23:07:36.381 INFO:tasks.workunit.client.1.smithi084.stdout:3/463: rename d3/d8/d1a/l95 to d3/d8/d1a/laf 0 2024-02-28T23:07:36.383 INFO:tasks.workunit.client.1.smithi084.stdout:6/474: write d1/d9/f3b [1752444,39059] 0 2024-02-28T23:07:36.384 INFO:tasks.workunit.client.1.smithi084.stdout:8/463: dread d0/d11/f2a [0,4194304] 0 2024-02-28T23:07:36.386 INFO:tasks.workunit.client.1.smithi084.stdout:5/539: write dc/de/df/d31/fcb [31504,10623] 0 2024-02-28T23:07:36.387 INFO:tasks.workunit.client.1.smithi084.stdout:2/416: unlink d5/d1f/d42/f47 0 2024-02-28T23:07:36.390 INFO:tasks.workunit.client.1.smithi084.stdout:4/437: chown d1/d18/d20/d65/d8e/d55/c3a 16 1 2024-02-28T23:07:36.395 INFO:tasks.workunit.client.0.smithi057.stdout:9/436: sync 2024-02-28T23:07:36.395 INFO:tasks.workunit.client.0.smithi057.stdout:0/350: sync 2024-02-28T23:07:36.397 INFO:tasks.workunit.client.1.smithi084.stdout:6/475: read d1/f24 [6913074,86051] 0 2024-02-28T23:07:36.398 INFO:tasks.workunit.client.1.smithi084.stdout:1/542: dwrite f1 [0,4194304] 0 2024-02-28T23:07:36.402 INFO:tasks.workunit.client.1.smithi084.stdout:5/540: stat dc/de/df/d31/d8e/f52 0 2024-02-28T23:07:36.402 INFO:tasks.workunit.client.0.smithi057.stdout:6/403: dwrite d0/f6c [0,4194304] 0 2024-02-28T23:07:36.406 INFO:tasks.workunit.client.1.smithi084.stdout:9/516: dwrite d4/d6/f72 [0,4194304] 0 2024-02-28T23:07:36.407 INFO:tasks.workunit.client.0.smithi057.stdout:0/351: mkdir d17/d20/d26/d48/d55/d88 0 2024-02-28T23:07:36.408 INFO:tasks.workunit.client.0.smithi057.stdout:6/404: symlink d0/d4/d11/l87 0 2024-02-28T23:07:36.408 INFO:tasks.workunit.client.1.smithi084.stdout:3/464: rename d3/d8 to d3/d27/d7d/db0 0 2024-02-28T23:07:36.409 INFO:tasks.workunit.client.0.smithi057.stdout:9/437: getdents dd/de/d4a 0 2024-02-28T23:07:36.409 INFO:tasks.workunit.client.1.smithi084.stdout:0/484: dwrite d1/d8/d18/f9a [8388608,4194304] 0 2024-02-28T23:07:36.410 INFO:tasks.workunit.client.1.smithi084.stdout:6/476: creat d1/d13/d50/d74/faa x:0 0 0 2024-02-28T23:07:36.411 INFO:tasks.workunit.client.1.smithi084.stdout:7/482: getdents db/d10/d18 0 2024-02-28T23:07:36.413 INFO:tasks.workunit.client.1.smithi084.stdout:1/543: mknod d12/d1b/d2e/d30/cb5 0 2024-02-28T23:07:36.414 INFO:tasks.workunit.client.1.smithi084.stdout:4/438: truncate d1/d18/d20/d14/d30/f31 1102202 0 2024-02-28T23:07:36.415 INFO:tasks.workunit.client.0.smithi057.stdout:3/285: dwrite d3/d4/f12 [0,4194304] 0 2024-02-28T23:07:36.417 INFO:tasks.workunit.client.0.smithi057.stdout:6/405: write d0/d1b/d54/f65 [766984,78081] 0 2024-02-28T23:07:36.417 INFO:tasks.workunit.client.0.smithi057.stdout:0/352: rmdir d17/d20/d4a 39 2024-02-28T23:07:36.419 INFO:tasks.workunit.client.1.smithi084.stdout:3/465: write d3/d27/d7d/db0/f24 [2605628,36153] 0 2024-02-28T23:07:36.420 INFO:tasks.workunit.client.1.smithi084.stdout:2/417: creat d5/f78 x:0 0 0 2024-02-28T23:07:36.421 INFO:tasks.workunit.client.1.smithi084.stdout:9/517: read d4/d6/d15/d3c/f7a [326743,9950] 0 2024-02-28T23:07:36.422 INFO:tasks.workunit.client.1.smithi084.stdout:2/418: readlink d5/d1f/d42/l57 0 2024-02-28T23:07:36.423 INFO:tasks.workunit.client.0.smithi057.stdout:0/353: write d17/d20/d26/d30/f31 [3976466,116124] 0 2024-02-28T23:07:36.425 INFO:tasks.workunit.client.0.smithi057.stdout:3/286: unlink d3/d8/f59 0 2024-02-28T23:07:36.425 INFO:tasks.workunit.client.1.smithi084.stdout:6/477: read d1/d9/f4f [897436,51070] 0 2024-02-28T23:07:36.426 INFO:tasks.workunit.client.0.smithi057.stdout:2/422: dwrite d8/d11/d21/d28/f75 [0,4194304] 0 2024-02-28T23:07:36.426 INFO:tasks.workunit.client.0.smithi057.stdout:6/406: rmdir d0/d17 39 2024-02-28T23:07:36.427 INFO:tasks.workunit.client.1.smithi084.stdout:5/541: mknod dc/de/d9e/cd0 0 2024-02-28T23:07:36.429 INFO:tasks.workunit.client.0.smithi057.stdout:3/287: write d3/f51 [736749,10450] 0 2024-02-28T23:07:36.429 INFO:tasks.workunit.client.1.smithi084.stdout:5/542: truncate dc/fc8 509008 0 2024-02-28T23:07:36.429 INFO:tasks.workunit.client.1.smithi084.stdout:8/464: getdents d0/d9/d30 0 2024-02-28T23:07:36.430 INFO:tasks.workunit.client.0.smithi057.stdout:0/354: mknod d17/d20/d23/c89 0 2024-02-28T23:07:36.432 INFO:tasks.workunit.client.0.smithi057.stdout:9/438: truncate dd/de/d14/d48/d4b/d41/f70 1815097 0 2024-02-28T23:07:36.433 INFO:tasks.workunit.client.1.smithi084.stdout:0/485: write d1/d6/db/d37/d72/d7b/f80 [2359207,21345] 0 2024-02-28T23:07:36.435 INFO:tasks.workunit.client.1.smithi084.stdout:7/483: dwrite db/d11/f68 [0,4194304] 0 2024-02-28T23:07:36.436 INFO:tasks.workunit.client.0.smithi057.stdout:3/288: read - d3/d8/f45 zero size 2024-02-28T23:07:36.437 INFO:tasks.workunit.client.1.smithi084.stdout:8/465: write d0/d11/f99 [294257,95202] 0 2024-02-28T23:07:36.437 INFO:tasks.workunit.client.0.smithi057.stdout:2/423: symlink d8/d11/d21/d28/d49/d85/l88 0 2024-02-28T23:07:36.439 INFO:tasks.workunit.client.1.smithi084.stdout:4/439: rename d1/d4b/c5c to d1/d18/d20/d65/d8e/d55/c96 0 2024-02-28T23:07:36.439 INFO:tasks.workunit.client.0.smithi057.stdout:9/439: unlink dd/c88 0 2024-02-28T23:07:36.439 INFO:tasks.workunit.client.0.smithi057.stdout:6/407: dread d0/d1b/d3e/d3d/d42/d4f/d53/f5b [0,4194304] 0 2024-02-28T23:07:36.439 INFO:tasks.workunit.client.0.smithi057.stdout:2/424: fdatasync d8/d11/d48/f52 0 2024-02-28T23:07:36.441 INFO:tasks.workunit.client.1.smithi084.stdout:4/440: chown d1/d4b/l76 28 1 2024-02-28T23:07:36.443 INFO:tasks.workunit.client.0.smithi057.stdout:9/440: mkdir dd/de/d4a/d60/d8c 0 2024-02-28T23:07:36.444 INFO:tasks.workunit.client.1.smithi084.stdout:2/419: fdatasync d5/d7/d12/d24/d3a/d48/f49 0 2024-02-28T23:07:36.446 INFO:tasks.workunit.client.0.smithi057.stdout:9/441: symlink dd/de/d24/d72/l8d 0 2024-02-28T23:07:36.448 INFO:tasks.workunit.client.1.smithi084.stdout:2/420: write d5/d7/d12/d24/d2a/d53/f5f [1215110,44938] 0 2024-02-28T23:07:36.452 INFO:tasks.workunit.client.0.smithi057.stdout:6/408: truncate d0/d4/d25/d60/d6a/f7e 1189224 0 2024-02-28T23:07:36.452 INFO:tasks.workunit.client.1.smithi084.stdout:6/478: creat d1/d13/d50/d74/d81/fab x:0 0 0 2024-02-28T23:07:36.453 INFO:tasks.workunit.client.0.smithi057.stdout:9/442: chown dd/c2a 1279616960 1 2024-02-28T23:07:36.453 INFO:tasks.workunit.client.1.smithi084.stdout:5/543: dread - dc/de/df/d31/f63 zero size 2024-02-28T23:07:36.460 INFO:tasks.workunit.client.1.smithi084.stdout:6/479: readlink d1/d9/d1d/d1e/d33/d38/l57 0 2024-02-28T23:07:36.464 INFO:tasks.workunit.client.1.smithi084.stdout:9/518: symlink d4/la1 0 2024-02-28T23:07:36.464 INFO:tasks.workunit.client.1.smithi084.stdout:1/544: link d12/d1b/d2e/d32/l65 d12/d1b/d2e/d52/d5b/lb6 0 2024-02-28T23:07:36.465 INFO:tasks.workunit.client.0.smithi057.stdout:2/425: dwrite d8/d11/d21/f36 [4194304,4194304] 0 2024-02-28T23:07:36.467 INFO:tasks.workunit.client.0.smithi057.stdout:2/426: read d8/d11/d15/f2d [272416,15018] 0 2024-02-28T23:07:36.468 INFO:tasks.workunit.client.1.smithi084.stdout:2/421: mknod d5/d7/d12/d24/d2a/c79 0 2024-02-28T23:07:36.469 INFO:tasks.workunit.client.1.smithi084.stdout:9/519: chown d4/d32/c80 79 1 2024-02-28T23:07:36.469 INFO:tasks.workunit.client.0.smithi057.stdout:6/409: dwrite d0/d4/f19 [0,4194304] 0 2024-02-28T23:07:36.470 INFO:tasks.workunit.client.0.smithi057.stdout:2/427: write d8/f23 [1364076,117037] 0 2024-02-28T23:07:36.471 INFO:tasks.workunit.client.1.smithi084.stdout:3/466: creat d3/fb1 x:0 0 0 2024-02-28T23:07:36.473 INFO:tasks.workunit.client.1.smithi084.stdout:1/545: chown fc 252 1 2024-02-28T23:07:36.475 INFO:tasks.workunit.client.0.smithi057.stdout:2/428: write d8/d11/d15/f5d [461568,74541] 0 2024-02-28T23:07:36.475 INFO:tasks.workunit.client.0.smithi057.stdout:6/410: rmdir d0/d1b/d3e/d3d/d42/d4f 39 2024-02-28T23:07:36.480 INFO:tasks.workunit.client.1.smithi084.stdout:1/546: chown d12/d1b/d2e/c89 30340906 1 2024-02-28T23:07:36.480 INFO:tasks.workunit.client.1.smithi084.stdout:2/422: write d5/d7/d2e/f46 [935055,32070] 0 2024-02-28T23:07:36.484 INFO:tasks.workunit.client.0.smithi057.stdout:0/355: write d17/d19/f28 [1580958,95861] 0 2024-02-28T23:07:36.486 INFO:tasks.workunit.client.0.smithi057.stdout:6/411: chown d0/d1b/d3e/d40/f4d 51391 1 2024-02-28T23:07:36.487 INFO:tasks.workunit.client.1.smithi084.stdout:7/484: dread db/ff [0,4194304] 0 2024-02-28T23:07:36.488 INFO:tasks.workunit.client.1.smithi084.stdout:8/466: dwrite d0/d24/d4d/f81 [0,4194304] 0 2024-02-28T23:07:36.489 INFO:tasks.workunit.client.1.smithi084.stdout:1/547: chown fc 117184 1 2024-02-28T23:07:36.490 INFO:tasks.workunit.client.0.smithi057.stdout:2/429: write d8/d11/d15/f20 [913871,112550] 0 2024-02-28T23:07:36.493 INFO:tasks.workunit.client.1.smithi084.stdout:9/520: getdents d4/d6 0 2024-02-28T23:07:36.493 INFO:tasks.workunit.client.1.smithi084.stdout:2/423: fdatasync d5/d7/d12/d24/d3a/d48/f49 0 2024-02-28T23:07:36.494 INFO:tasks.workunit.client.0.smithi057.stdout:6/412: symlink d0/d4/l88 0 2024-02-28T23:07:36.495 INFO:tasks.workunit.client.1.smithi084.stdout:7/485: write db/d11/f68 [4033679,109138] 0 2024-02-28T23:07:36.495 INFO:tasks.workunit.client.0.smithi057.stdout:6/413: readlink d0/d4/l88 0 2024-02-28T23:07:36.497 INFO:tasks.workunit.client.0.smithi057.stdout:0/356: creat d17/d20/d26/d48/d7c/f8a x:0 0 0 2024-02-28T23:07:36.500 INFO:tasks.workunit.client.0.smithi057.stdout:2/430: rename d8/d11/d21/l22 to d8/d11/d39/d7b/l89 0 2024-02-28T23:07:36.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/357: creat d17/d20/d23/f8b x:0 0 0 2024-02-28T23:07:36.504 INFO:tasks.workunit.client.1.smithi084.stdout:6/480: dwrite d1/f1b [0,4194304] 0 2024-02-28T23:07:36.504 INFO:tasks.workunit.client.1.smithi084.stdout:7/486: readlink db/d10/d17/l38 0 2024-02-28T23:07:36.507 INFO:tasks.workunit.client.1.smithi084.stdout:1/548: fsync d12/d1b/d2e/d32/f47 0 2024-02-28T23:07:36.510 INFO:tasks.workunit.client.1.smithi084.stdout:3/467: link d3/d27/d7d/db0/f4f d3/d27/d7d/db0/d1a/d21/d76/fb2 0 2024-02-28T23:07:36.517 INFO:tasks.workunit.client.1.smithi084.stdout:4/441: dread d1/f12 [0,4194304] 0 2024-02-28T23:07:36.518 INFO:tasks.workunit.client.0.smithi057.stdout:2/431: dwrite d8/d11/d21/d28/f3d [0,4194304] 0 2024-02-28T23:07:36.519 INFO:tasks.workunit.client.1.smithi084.stdout:8/467: mknod d0/d9/d30/d33/d71/c9d 0 2024-02-28T23:07:36.520 INFO:tasks.workunit.client.0.smithi057.stdout:0/358: dwrite d17/d20/d4a/d78/f79 [0,4194304] 0 2024-02-28T23:07:36.520 INFO:tasks.workunit.client.0.smithi057.stdout:2/432: chown d8/d11/d15/f2d 11804056 1 2024-02-28T23:07:36.523 INFO:tasks.workunit.client.0.smithi057.stdout:0/359: write d17/d20/f45 [2114593,110496] 0 2024-02-28T23:07:36.524 INFO:tasks.workunit.client.1.smithi084.stdout:2/424: dread d5/d7/d12/d24/d3a/d48/f49 [0,4194304] 0 2024-02-28T23:07:36.527 INFO:tasks.workunit.client.1.smithi084.stdout:1/549: rmdir d12/d1b/d2e/d52/d61 39 2024-02-28T23:07:36.527 INFO:tasks.workunit.client.1.smithi084.stdout:5/544: truncate dc/de/df/f71 89085 0 2024-02-28T23:07:36.528 INFO:tasks.workunit.client.1.smithi084.stdout:9/521: rename d4/d6/c25 to d4/d6/d15/d75/ca2 0 2024-02-28T23:07:36.528 INFO:tasks.workunit.client.1.smithi084.stdout:6/481: truncate d1/d13/d2e/d41/d2c/f71 1993699 0 2024-02-28T23:07:36.528 INFO:tasks.workunit.client.0.smithi057.stdout:6/414: dread d0/d4/d25/d60/d6a/f7e [0,4194304] 0 2024-02-28T23:07:36.529 INFO:tasks.workunit.client.0.smithi057.stdout:6/415: chown d0/d1b 2291 1 2024-02-28T23:07:36.530 INFO:tasks.workunit.client.1.smithi084.stdout:9/522: readlink l1 0 2024-02-28T23:07:36.532 INFO:tasks.workunit.client.0.smithi057.stdout:0/360: write d17/d19/f53 [2097151,3696] 0 2024-02-28T23:07:36.535 INFO:tasks.workunit.client.1.smithi084.stdout:7/487: dread db/d10/f5d [0,4194304] 0 2024-02-28T23:07:36.535 INFO:tasks.workunit.client.1.smithi084.stdout:0/486: dread d1/d8/f1d [0,4194304] 0 2024-02-28T23:07:36.539 INFO:tasks.workunit.client.0.smithi057.stdout:8/340: sync 2024-02-28T23:07:36.540 INFO:tasks.workunit.client.1.smithi084.stdout:1/550: fsync d12/d1b/d2e/d32/f51 0 2024-02-28T23:07:36.540 INFO:tasks.workunit.client.1.smithi084.stdout:7/488: fdatasync db/fd 0 2024-02-28T23:07:36.541 INFO:tasks.workunit.client.0.smithi057.stdout:2/433: dwrite d8/d11/f32 [0,4194304] 0 2024-02-28T23:07:36.542 INFO:tasks.workunit.client.1.smithi084.stdout:0/487: readlink d1/d8/d18/l3b 0 2024-02-28T23:07:36.543 INFO:tasks.workunit.client.1.smithi084.stdout:3/468: rename d3/d27/d7d/db0/d1a/d21/c6f to d3/d27/d7d/db0/d14/d31/d34/cb3 0 2024-02-28T23:07:36.543 INFO:tasks.workunit.client.0.smithi057.stdout:8/341: rmdir d4/d1e/d56/d6a/d42 39 2024-02-28T23:07:36.545 INFO:tasks.workunit.client.1.smithi084.stdout:4/442: unlink d1/d18/d20/cf 0 2024-02-28T23:07:36.548 INFO:tasks.workunit.client.0.smithi057.stdout:6/416: dread d0/d4/ff [0,4194304] 0 2024-02-28T23:07:36.548 INFO:tasks.workunit.client.0.smithi057.stdout:8/342: write d4/f17 [261836,77160] 0 2024-02-28T23:07:36.549 INFO:tasks.workunit.client.0.smithi057.stdout:2/434: write d8/d11/d21/d28/d41/f53 [4675812,80573] 0 2024-02-28T23:07:36.554 INFO:tasks.workunit.client.0.smithi057.stdout:0/361: dwrite d17/d20/d26/f41 [0,4194304] 0 2024-02-28T23:07:36.555 INFO:tasks.workunit.client.0.smithi057.stdout:2/435: creat d8/d11/d39/f8a x:0 0 0 2024-02-28T23:07:36.556 INFO:tasks.workunit.client.0.smithi057.stdout:8/343: symlink d4/d1e/d56/d6a/d54/d61/l76 0 2024-02-28T23:07:36.557 INFO:tasks.workunit.client.1.smithi084.stdout:7/489: mknod db/d5e/c90 0 2024-02-28T23:07:36.558 INFO:tasks.workunit.client.0.smithi057.stdout:2/436: read d8/d11/d21/d28/f75 [1625342,101297] 0 2024-02-28T23:07:36.563 INFO:tasks.workunit.client.1.smithi084.stdout:6/482: dwrite d1/d13/d50/d74/d81/f94 [0,4194304] 0 2024-02-28T23:07:36.563 INFO:tasks.workunit.client.1.smithi084.stdout:7/490: write db/d11/d27/d36/f4c [1874017,14458] 0 2024-02-28T23:07:36.563 INFO:tasks.workunit.client.0.smithi057.stdout:6/417: rename d0/f3c to d0/d1b/d3e/d3d/d42/d4f/d53/f89 0 2024-02-28T23:07:36.564 INFO:tasks.workunit.client.1.smithi084.stdout:5/545: rename dc/de/df/d16/d34/d54 to dc/de/df/d31/dd1 0 2024-02-28T23:07:36.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:36 smithi057 ceph-mon[29096]: pgmap v24: 65 pgs: 65 active+clean; 1.6 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 229 MiB/s wr, 419 op/s 2024-02-28T23:07:36.569 INFO:tasks.workunit.client.0.smithi057.stdout:2/437: mkdir d8/d11/d39/d7b/d8b 0 2024-02-28T23:07:36.574 INFO:tasks.workunit.client.0.smithi057.stdout:6/418: creat d0/d1b/d3e/d3d/f8a x:0 0 0 2024-02-28T23:07:36.574 INFO:tasks.workunit.client.1.smithi084.stdout:2/425: link d5/d7/d9/cc d5/d7/d12/d24/d3a/c7a 0 2024-02-28T23:07:36.575 INFO:tasks.workunit.client.0.smithi057.stdout:0/362: link d17/d20/d26/d30/c77 d17/d20/d4a/c8c 0 2024-02-28T23:07:36.576 INFO:tasks.workunit.client.1.smithi084.stdout:3/469: dwrite d3/d50/f66 [0,4194304] 0 2024-02-28T23:07:36.578 INFO:tasks.workunit.client.0.smithi057.stdout:8/344: dread d4/d1e/d56/d6a/f4d [4194304,4194304] 0 2024-02-28T23:07:36.578 INFO:tasks.workunit.client.0.smithi057.stdout:0/363: dread - d17/d20/d4a/f81 zero size 2024-02-28T23:07:36.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:36 smithi084 ceph-mon[34826]: pgmap v24: 65 pgs: 65 active+clean; 1.6 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 229 MiB/s wr, 419 op/s 2024-02-28T23:07:36.583 INFO:tasks.workunit.client.1.smithi084.stdout:7/491: unlink db/d10/l6e 0 2024-02-28T23:07:36.584 INFO:tasks.workunit.client.1.smithi084.stdout:5/546: mkdir dc/de/df/d16/d34/d3c/d77/dd2 0 2024-02-28T23:07:36.587 INFO:tasks.workunit.client.0.smithi057.stdout:8/345: rename d4/d1e/c41 to d4/d1e/d56/d6a/d54/d61/c77 0 2024-02-28T23:07:36.590 INFO:tasks.workunit.client.1.smithi084.stdout:9/523: unlink d4/d6/d15/d12/d41/d55/f5b 0 2024-02-28T23:07:36.591 INFO:tasks.workunit.client.1.smithi084.stdout:1/551: dwrite d12/d1b/f4a [0,4194304] 0 2024-02-28T23:07:36.593 INFO:tasks.workunit.client.0.smithi057.stdout:8/346: mknod d4/d13/d18/d62/d66/d74/c78 0 2024-02-28T23:07:36.594 INFO:tasks.workunit.client.1.smithi084.stdout:9/524: chown d4/d32/f92 30139 1 2024-02-28T23:07:36.600 INFO:tasks.workunit.client.0.smithi057.stdout:2/438: truncate d8/d11/f1d 1906953 0 2024-02-28T23:07:36.600 INFO:tasks.workunit.client.0.smithi057.stdout:8/347: mknod d4/d1e/c79 0 2024-02-28T23:07:36.600 INFO:tasks.workunit.client.1.smithi084.stdout:9/525: read - d4/f9c zero size 2024-02-28T23:07:36.600 INFO:tasks.workunit.client.1.smithi084.stdout:3/470: stat d3/d27/d7d/db0/d14/f2e 0 2024-02-28T23:07:36.601 INFO:tasks.workunit.client.1.smithi084.stdout:0/488: creat d1/d8/d18/f9f x:0 0 0 2024-02-28T23:07:36.603 INFO:tasks.workunit.client.0.smithi057.stdout:2/439: mknod d8/d11/d21/d28/d41/c8c 0 2024-02-28T23:07:36.606 INFO:tasks.workunit.client.1.smithi084.stdout:2/426: dwrite d5/dd/f58 [0,4194304] 0 2024-02-28T23:07:36.606 INFO:tasks.workunit.client.0.smithi057.stdout:8/348: creat d4/d13/d1a/f7a x:0 0 0 2024-02-28T23:07:36.608 INFO:tasks.workunit.client.1.smithi084.stdout:9/526: write d4/d6/d15/d12/d41/f64 [3593535,10151] 0 2024-02-28T23:07:36.609 INFO:tasks.workunit.client.1.smithi084.stdout:1/552: rename d12/l8b to d12/d1b/d91/lb7 0 2024-02-28T23:07:36.611 INFO:tasks.workunit.client.0.smithi057.stdout:2/440: mknod d8/d11/d39/d7b/d8b/c8d 0 2024-02-28T23:07:36.619 INFO:tasks.workunit.client.0.smithi057.stdout:8/349: dread d4/f5 [0,4194304] 0 2024-02-28T23:07:36.619 INFO:tasks.workunit.client.1.smithi084.stdout:5/547: rmdir dc/de/df/d16/d34/d3c/d98/dc0 0 2024-02-28T23:07:36.619 INFO:tasks.workunit.client.1.smithi084.stdout:2/427: symlink d5/d7/d2e/d4a/d5c/l7b 0 2024-02-28T23:07:36.620 INFO:tasks.workunit.client.0.smithi057.stdout:0/364: truncate d17/d20/d26/f46 2716289 0 2024-02-28T23:07:36.620 INFO:tasks.workunit.client.0.smithi057.stdout:6/419: dread d0/d1b/d3e/d3d/f55 [0,4194304] 0 2024-02-28T23:07:36.623 INFO:tasks.workunit.client.0.smithi057.stdout:4/354: sync 2024-02-28T23:07:36.623 INFO:tasks.workunit.client.0.smithi057.stdout:5/318: sync 2024-02-28T23:07:36.623 INFO:tasks.workunit.client.0.smithi057.stdout:1/283: sync 2024-02-28T23:07:36.624 INFO:tasks.workunit.client.0.smithi057.stdout:0/365: mknod d17/d19/d4d/d7b/c8d 0 2024-02-28T23:07:36.626 INFO:tasks.workunit.client.1.smithi084.stdout:4/443: getdents d1/d18/d20/d24/d2f/d3e/d52/d45 0 2024-02-28T23:07:36.626 INFO:tasks.workunit.client.0.smithi057.stdout:8/350: mkdir d4/d13/d1a/d4a/d7b 0 2024-02-28T23:07:36.628 INFO:tasks.workunit.client.0.smithi057.stdout:4/355: chown d1a/d1b/f3e 22 1 2024-02-28T23:07:36.629 INFO:tasks.workunit.client.1.smithi084.stdout:1/553: dread - d12/d1b/d63/f66 zero size 2024-02-28T23:07:36.630 INFO:tasks.workunit.client.1.smithi084.stdout:6/483: dread d1/d9/f28 [0,4194304] 0 2024-02-28T23:07:36.633 INFO:tasks.workunit.client.1.smithi084.stdout:9/527: mknod d4/d6/d15/ca3 0 2024-02-28T23:07:36.633 INFO:tasks.workunit.client.1.smithi084.stdout:1/554: fsync d12/d1b/d3b/f44 0 2024-02-28T23:07:36.634 INFO:tasks.workunit.client.0.smithi057.stdout:4/356: write d1a/d48/f61 [600704,28051] 0 2024-02-28T23:07:36.634 INFO:tasks.workunit.client.0.smithi057.stdout:0/366: read d17/d20/d23/d47/f63 [381538,5235] 0 2024-02-28T23:07:36.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/420: rename d0/d1b/d3e/d3d/d74/f77 to d0/d1b/d3e/d40/f8b 0 2024-02-28T23:07:36.634 INFO:tasks.workunit.client.0.smithi057.stdout:5/319: write d4/d9/d14/d3b/f47 [1515934,71181] 0 2024-02-28T23:07:36.636 INFO:tasks.workunit.client.1.smithi084.stdout:2/428: fsync d5/d7/d9/f2c 0 2024-02-28T23:07:36.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/441: dwrite d8/d11/d48/d50/f7a [0,4194304] 0 2024-02-28T23:07:36.639 INFO:tasks.workunit.client.0.smithi057.stdout:1/284: creat d2/d7/d4f/f63 x:0 0 0 2024-02-28T23:07:36.639 INFO:tasks.workunit.client.0.smithi057.stdout:0/367: readlink d17/d20/d26/d30/l38 0 2024-02-28T23:07:36.639 INFO:tasks.workunit.client.0.smithi057.stdout:0/368: fdatasync f10 0 2024-02-28T23:07:36.642 INFO:tasks.workunit.client.0.smithi057.stdout:1/285: dread - d2/d7/df/db/d3a/f3c zero size 2024-02-28T23:07:36.642 INFO:tasks.workunit.client.0.smithi057.stdout:2/442: dread - d8/d11/d39/f4c zero size 2024-02-28T23:07:36.643 INFO:tasks.workunit.client.0.smithi057.stdout:0/369: chown d17/d20/d23/d47/d60/c6e 183452855 1 2024-02-28T23:07:36.645 INFO:tasks.workunit.client.0.smithi057.stdout:8/351: mkdir d4/d1e/d7c 0 2024-02-28T23:07:36.647 INFO:tasks.workunit.client.0.smithi057.stdout:2/443: fsync d8/d11/d21/f7d 0 2024-02-28T23:07:36.648 INFO:tasks.workunit.client.1.smithi084.stdout:2/429: rename d5/d7/d12/d4f to d5/d7/d9/d7c 0 2024-02-28T23:07:36.648 INFO:tasks.workunit.client.0.smithi057.stdout:8/352: dread - d4/d1e/d56/d6a/d54/f70 zero size 2024-02-28T23:07:36.649 INFO:tasks.workunit.client.1.smithi084.stdout:5/548: dwrite dc/de/df/d31/dd1/f80 [0,4194304] 0 2024-02-28T23:07:36.651 INFO:tasks.workunit.client.0.smithi057.stdout:6/421: unlink d0/f1c 0 2024-02-28T23:07:36.652 INFO:tasks.workunit.client.0.smithi057.stdout:4/357: creat d1a/d1b/d24/f82 x:0 0 0 2024-02-28T23:07:36.657 INFO:tasks.workunit.client.1.smithi084.stdout:5/549: creat dc/de/df/da6/fd3 x:0 0 0 2024-02-28T23:07:36.661 INFO:tasks.workunit.client.0.smithi057.stdout:1/286: read d2/d7/df/db/d12/d19/d24/d2d/f49 [17015,110089] 0 2024-02-28T23:07:36.662 INFO:tasks.workunit.client.0.smithi057.stdout:0/370: unlink d17/d20/d26/d48/d7c/f8a 0 2024-02-28T23:07:36.665 INFO:tasks.workunit.client.0.smithi057.stdout:2/444: symlink d8/d11/d21/d28/d49/d85/l8e 0 2024-02-28T23:07:36.666 INFO:tasks.workunit.client.1.smithi084.stdout:5/550: read dc/f27 [172103,24723] 0 2024-02-28T23:07:36.666 INFO:tasks.workunit.client.1.smithi084.stdout:7/492: dwrite db/d10/d17/d3b/f4d [4194304,4194304] 0 2024-02-28T23:07:36.668 INFO:tasks.workunit.client.0.smithi057.stdout:2/445: fsync d8/d11/f32 0 2024-02-28T23:07:36.677 INFO:tasks.workunit.client.0.smithi057.stdout:4/358: dwrite d1a/d1b/d24/d2e/f3f [0,4194304] 0 2024-02-28T23:07:36.677 INFO:tasks.workunit.client.0.smithi057.stdout:5/320: dwrite f2 [4194304,4194304] 0 2024-02-28T23:07:36.679 INFO:tasks.workunit.client.0.smithi057.stdout:0/371: mknod d17/d20/d23/d37/c8e 0 2024-02-28T23:07:36.680 INFO:tasks.workunit.client.1.smithi084.stdout:3/471: write d3/d27/d7d/db0/fa [607925,96327] 0 2024-02-28T23:07:36.680 INFO:tasks.workunit.client.1.smithi084.stdout:0/489: write d1/d6/d59/d70/f73 [631730,2294] 0 2024-02-28T23:07:36.684 INFO:tasks.workunit.client.1.smithi084.stdout:2/430: dread d5/f17 [0,4194304] 0 2024-02-28T23:07:36.684 INFO:tasks.workunit.client.1.smithi084.stdout:0/490: chown d1/d6/db/c2e 7 1 2024-02-28T23:07:36.685 INFO:tasks.workunit.client.0.smithi057.stdout:6/422: mknod d0/d1b/d3e/c8c 0 2024-02-28T23:07:36.687 INFO:tasks.workunit.client.0.smithi057.stdout:0/372: rename d17/f42 to d17/d19/f8f 0 2024-02-28T23:07:36.690 INFO:tasks.workunit.client.1.smithi084.stdout:7/493: dwrite db/d11/d27/d36/f86 [0,4194304] 0 2024-02-28T23:07:36.692 INFO:tasks.workunit.client.0.smithi057.stdout:3/289: sync 2024-02-28T23:07:36.693 INFO:tasks.workunit.client.0.smithi057.stdout:7/352: sync 2024-02-28T23:07:36.693 INFO:tasks.workunit.client.0.smithi057.stdout:2/446: symlink d8/l8f 0 2024-02-28T23:07:36.694 INFO:tasks.workunit.client.1.smithi084.stdout:3/472: mknod d3/d27/d7d/da0/cb4 0 2024-02-28T23:07:36.695 INFO:tasks.workunit.client.0.smithi057.stdout:5/321: creat d4/d21/d24/f64 x:0 0 0 2024-02-28T23:07:36.695 INFO:tasks.workunit.client.1.smithi084.stdout:2/431: unlink d5/d7/d2e/d4a/d5c/l7b 0 2024-02-28T23:07:36.695 INFO:tasks.workunit.client.0.smithi057.stdout:9/443: sync 2024-02-28T23:07:36.699 INFO:tasks.workunit.client.1.smithi084.stdout:4/444: truncate d1/d18/d20/d24/f8b 960435 0 2024-02-28T23:07:36.699 INFO:tasks.workunit.client.1.smithi084.stdout:5/551: link dc/de/df/d16/d34/l60 dc/de/df/d31/ld4 0 2024-02-28T23:07:36.700 INFO:tasks.workunit.client.0.smithi057.stdout:8/353: getdents d4/d13/d1a/d4a 0 2024-02-28T23:07:36.704 INFO:tasks.workunit.client.0.smithi057.stdout:0/373: creat d17/d20/d4a/d78/f90 x:0 0 0 2024-02-28T23:07:36.705 INFO:tasks.workunit.client.1.smithi084.stdout:6/484: truncate d1/d13/d2e/d41/d2c/f71 1433616 0 2024-02-28T23:07:36.705 INFO:tasks.workunit.client.1.smithi084.stdout:0/491: mknod d1/d8/d69/d6b/d85/ca0 0 2024-02-28T23:07:36.709 INFO:tasks.workunit.client.1.smithi084.stdout:0/492: stat d1/d6/db/l45 0 2024-02-28T23:07:36.710 INFO:tasks.workunit.client.0.smithi057.stdout:9/444: chown dd/de/d4a/d60/f83 0 1 2024-02-28T23:07:36.712 INFO:tasks.workunit.client.0.smithi057.stdout:2/447: mknod d8/d11/d21/d28/c90 0 2024-02-28T23:07:36.714 INFO:tasks.workunit.client.0.smithi057.stdout:8/354: dread d4/d13/d18/f26 [0,4194304] 0 2024-02-28T23:07:36.714 INFO:tasks.workunit.client.1.smithi084.stdout:7/494: dwrite db/d10/d17/d64/f75 [0,4194304] 0 2024-02-28T23:07:36.715 INFO:tasks.workunit.client.1.smithi084.stdout:3/473: rename d3/d27/d7d/db0/d1a/d29/d41/c90 to d3/d27/d7d/db0/d1a/d21/d7b/cb5 0 2024-02-28T23:07:36.720 INFO:tasks.workunit.client.1.smithi084.stdout:7/495: truncate db/d11/d27/d36/f8e 823588 0 2024-02-28T23:07:36.720 INFO:tasks.workunit.client.0.smithi057.stdout:9/445: creat dd/de/d14/f8e x:0 0 0 2024-02-28T23:07:36.722 INFO:tasks.workunit.client.1.smithi084.stdout:2/432: dwrite d5/d64/f6c [0,4194304] 0 2024-02-28T23:07:36.724 INFO:tasks.workunit.client.1.smithi084.stdout:6/485: creat d1/d9/fac x:0 0 0 2024-02-28T23:07:36.725 INFO:tasks.workunit.client.0.smithi057.stdout:5/322: creat d4/d9/d16/d52/f65 x:0 0 0 2024-02-28T23:07:36.728 INFO:tasks.workunit.client.1.smithi084.stdout:6/486: chown d1/d9/d1d/d1e/d33/f75 57094 1 2024-02-28T23:07:36.728 INFO:tasks.workunit.client.1.smithi084.stdout:9/528: rename d4/d6/d15/d12/d41/f46 to d4/d6/d15/d75/fa4 0 2024-02-28T23:07:36.729 INFO:tasks.workunit.client.0.smithi057.stdout:8/355: read d4/d13/d18/f27 [2183,121906] 0 2024-02-28T23:07:36.729 INFO:tasks.workunit.client.0.smithi057.stdout:7/353: truncate d0/d5/d36/d4e/d15/d29/d3d/d51/f58 283899 0 2024-02-28T23:07:36.729 INFO:tasks.workunit.client.1.smithi084.stdout:5/552: getdents dc/de/df/d6e 0 2024-02-28T23:07:36.729 INFO:tasks.workunit.client.0.smithi057.stdout:6/423: rename d0/d1b/l70 to d0/d4/d25/d60/l8d 0 2024-02-28T23:07:36.731 INFO:tasks.workunit.client.0.smithi057.stdout:7/354: readlink d0/d5/l21 0 2024-02-28T23:07:36.732 INFO:tasks.workunit.client.1.smithi084.stdout:3/474: truncate d3/d27/d7d/db0/d1a/d21/d3d/f67 936133 0 2024-02-28T23:07:36.733 INFO:tasks.workunit.client.0.smithi057.stdout:4/359: truncate d1a/d1b/d24/d2e/f4a 3490482 0 2024-02-28T23:07:36.733 INFO:tasks.workunit.client.1.smithi084.stdout:6/487: write d1/d13/d2e/f44 [1615238,99983] 0 2024-02-28T23:07:36.736 INFO:tasks.workunit.client.1.smithi084.stdout:7/496: dread db/ff [0,4194304] 0 2024-02-28T23:07:36.737 INFO:tasks.workunit.client.1.smithi084.stdout:2/433: symlink d5/d7/d12/d24/d2a/d53/l7d 0 2024-02-28T23:07:36.737 INFO:tasks.workunit.client.1.smithi084.stdout:5/553: chown dc/de/df/d31/dd1/d5f/d9b 103729 1 2024-02-28T23:07:36.739 INFO:tasks.workunit.client.0.smithi057.stdout:3/290: link d3/d4f/d4c/l50 d3/d8/d27/l5c 0 2024-02-28T23:07:36.740 INFO:tasks.workunit.client.0.smithi057.stdout:0/374: rename d17/d20/d23/d47/f63 to d17/d20/d26/d48/d7c/f91 0 2024-02-28T23:07:36.740 INFO:tasks.workunit.client.0.smithi057.stdout:8/356: symlink d4/d1e/d56/l7d 0 2024-02-28T23:07:36.741 INFO:tasks.workunit.client.1.smithi084.stdout:5/554: chown dc/de/df/d16/d34/d3c/d98 416927 1 2024-02-28T23:07:36.741 INFO:tasks.workunit.client.1.smithi084.stdout:2/434: write d5/f78 [287032,65732] 0 2024-02-28T23:07:36.742 INFO:tasks.workunit.client.1.smithi084.stdout:3/475: read d3/d27/d7d/db0/d1a/d21/d3d/f4e [3197509,119504] 0 2024-02-28T23:07:36.743 INFO:tasks.workunit.client.0.smithi057.stdout:1/287: dwrite d2/d7/df/db/f17 [0,4194304] 0 2024-02-28T23:07:36.744 INFO:tasks.workunit.client.1.smithi084.stdout:0/493: rename d1/d8/d78/d8f to d1/d8/da1 0 2024-02-28T23:07:36.746 INFO:tasks.workunit.client.0.smithi057.stdout:4/360: write d1a/d1e/f42 [849731,40841] 0 2024-02-28T23:07:36.747 INFO:tasks.workunit.client.0.smithi057.stdout:2/448: dwrite d8/f19 [0,4194304] 0 2024-02-28T23:07:36.747 INFO:tasks.workunit.client.0.smithi057.stdout:6/424: mknod d0/d1b/d3e/d3d/d42/c8e 0 2024-02-28T23:07:36.747 INFO:tasks.workunit.client.1.smithi084.stdout:6/488: creat d1/d13/d9d/fad x:0 0 0 2024-02-28T23:07:36.748 INFO:tasks.workunit.client.0.smithi057.stdout:3/291: unlink d3/d4/l58 0 2024-02-28T23:07:36.750 INFO:tasks.workunit.client.0.smithi057.stdout:0/375: dread - d17/d20/d4a/d78/f90 zero size 2024-02-28T23:07:36.751 INFO:tasks.workunit.client.0.smithi057.stdout:1/288: write d2/d7/df/f4d [3004032,96123] 0 2024-02-28T23:07:36.755 INFO:tasks.workunit.client.1.smithi084.stdout:6/489: truncate d1/d13/d50/d74/faa 587097 0 2024-02-28T23:07:36.756 INFO:tasks.workunit.client.1.smithi084.stdout:5/555: symlink dc/de/df/d31/ld5 0 2024-02-28T23:07:36.761 INFO:tasks.workunit.client.0.smithi057.stdout:3/292: symlink d3/d4/d7/l5d 0 2024-02-28T23:07:36.762 INFO:tasks.workunit.client.1.smithi084.stdout:2/435: creat d5/d7/d9/d7c/d5b/f7e x:0 0 0 2024-02-28T23:07:36.763 INFO:tasks.workunit.client.1.smithi084.stdout:3/476: dread d3/d27/d7d/db0/d1a/d29/d41/f71 [0,4194304] 0 2024-02-28T23:07:36.766 INFO:tasks.workunit.client.1.smithi084.stdout:9/529: rename d4/d32/d3f/f42 to d4/d6/d15/d12/d41/d55/fa5 0 2024-02-28T23:07:36.767 INFO:tasks.workunit.client.0.smithi057.stdout:5/323: dwrite d4/d9/d16/d52/f65 [0,4194304] 0 2024-02-28T23:07:36.770 INFO:tasks.workunit.client.1.smithi084.stdout:4/445: dwrite d1/d18/f42 [0,4194304] 0 2024-02-28T23:07:36.771 INFO:tasks.workunit.client.1.smithi084.stdout:7/497: mknod db/d10/c91 0 2024-02-28T23:07:36.772 INFO:tasks.workunit.client.0.smithi057.stdout:0/376: creat d17/d20/d26/d30/d6b/d71/f92 x:0 0 0 2024-02-28T23:07:36.777 INFO:tasks.workunit.client.0.smithi057.stdout:9/446: rename dd/d46/c73 to dd/de/d14/d48/c8f 0 2024-02-28T23:07:36.778 INFO:tasks.workunit.client.0.smithi057.stdout:7/355: dread d0/d5/d36/f3c [0,4194304] 0 2024-02-28T23:07:36.780 INFO:tasks.workunit.client.1.smithi084.stdout:2/436: stat d5/d1f/l59 0 2024-02-28T23:07:36.781 INFO:tasks.workunit.client.1.smithi084.stdout:7/498: stat db/d10/f71 0 2024-02-28T23:07:36.781 INFO:tasks.workunit.client.0.smithi057.stdout:4/361: dwrite d1a/d1b/f2c [0,4194304] 0 2024-02-28T23:07:36.783 INFO:tasks.workunit.client.1.smithi084.stdout:0/494: dwrite d1/d8/d18/f9f [0,4194304] 0 2024-02-28T23:07:36.785 INFO:tasks.workunit.client.1.smithi084.stdout:4/446: creat d1/f97 x:0 0 0 2024-02-28T23:07:36.785 INFO:tasks.workunit.client.0.smithi057.stdout:2/449: mkdir d8/d11/d21/d70/d91 0 2024-02-28T23:07:36.787 INFO:tasks.workunit.client.1.smithi084.stdout:7/499: mknod db/d5e/d87/c92 0 2024-02-28T23:07:36.787 INFO:tasks.workunit.client.0.smithi057.stdout:3/293: chown d3/d4f/d4c/l50 574122844 1 2024-02-28T23:07:36.787 INFO:tasks.workunit.client.0.smithi057.stdout:6/425: creat d0/d1b/d3e/d3d/d42/f8f x:0 0 0 2024-02-28T23:07:36.790 INFO:tasks.workunit.client.0.smithi057.stdout:3/294: fdatasync d3/f2c 0 2024-02-28T23:07:36.790 INFO:tasks.workunit.client.0.smithi057.stdout:5/324: mknod d4/d9/d14/d3b/d61/c66 0 2024-02-28T23:07:36.791 INFO:tasks.workunit.client.1.smithi084.stdout:0/495: dread - d1/d6/db/d37/d72/f99 zero size 2024-02-28T23:07:36.793 INFO:tasks.workunit.client.1.smithi084.stdout:7/500: creat db/d10/d17/d64/f93 x:0 0 0 2024-02-28T23:07:36.797 INFO:tasks.workunit.client.1.smithi084.stdout:0/496: fsync d1/d6/db/d37/f71 0 2024-02-28T23:07:36.798 INFO:tasks.workunit.client.0.smithi057.stdout:7/356: unlink d0/d5/d36/d4e/d15/d29/d3d/d51/f6b 0 2024-02-28T23:07:36.800 INFO:tasks.workunit.client.1.smithi084.stdout:8/468: sync 2024-02-28T23:07:36.802 INFO:tasks.workunit.client.1.smithi084.stdout:7/501: readlink db/l4e 0 2024-02-28T23:07:36.803 INFO:tasks.workunit.client.1.smithi084.stdout:0/497: fsync d1/d8/d18/f9a 0 2024-02-28T23:07:36.804 INFO:tasks.workunit.client.0.smithi057.stdout:4/362: mknod d1a/d1e/d4b/d4f/c83 0 2024-02-28T23:07:36.804 INFO:tasks.workunit.client.0.smithi057.stdout:7/357: write d0/d5/d36/d4e/d3b/f5d [910632,23670] 0 2024-02-28T23:07:36.804 INFO:tasks.workunit.client.0.smithi057.stdout:9/447: mknod dd/de/d24/d63/d77/c90 0 2024-02-28T23:07:36.805 INFO:tasks.workunit.client.0.smithi057.stdout:6/426: write d0/d17/d20/f84 [120776,122413] 0 2024-02-28T23:07:36.809 INFO:tasks.workunit.client.1.smithi084.stdout:6/490: dwrite d1/d9/d1d/d1e/d33/f8f [0,4194304] 0 2024-02-28T23:07:36.810 INFO:tasks.workunit.client.0.smithi057.stdout:1/289: getdents d2/d7/df/db/d3a 0 2024-02-28T23:07:36.814 INFO:tasks.workunit.client.1.smithi084.stdout:8/469: mkdir d0/d24/d9e 0 2024-02-28T23:07:36.815 INFO:tasks.workunit.client.0.smithi057.stdout:0/377: dread d17/d20/d4a/d78/f79 [0,4194304] 0 2024-02-28T23:07:36.815 INFO:tasks.workunit.client.0.smithi057.stdout:6/427: creat d0/d1b/d54/f90 x:0 0 0 2024-02-28T23:07:36.816 INFO:tasks.workunit.client.0.smithi057.stdout:1/290: fsync d2/d7/df/db/d12/d3f/f4b 0 2024-02-28T23:07:36.817 INFO:tasks.workunit.client.0.smithi057.stdout:9/448: chown dd/d1b/d40/d44/f62 29560 1 2024-02-28T23:07:36.819 INFO:tasks.workunit.client.0.smithi057.stdout:3/295: creat d3/d8/f5e x:0 0 0 2024-02-28T23:07:36.819 INFO:tasks.workunit.client.1.smithi084.stdout:4/447: dwrite d1/d18/f42 [0,4194304] 0 2024-02-28T23:07:36.821 INFO:tasks.workunit.client.1.smithi084.stdout:2/437: rmdir d5/d7/d9/d7c/d5b 39 2024-02-28T23:07:36.821 INFO:tasks.workunit.client.1.smithi084.stdout:8/470: read d0/d11/f99 [77963,125119] 0 2024-02-28T23:07:36.822 INFO:tasks.workunit.client.0.smithi057.stdout:5/325: link d4/d9/f1c d4/d21/f67 0 2024-02-28T23:07:36.823 INFO:tasks.workunit.client.0.smithi057.stdout:3/296: chown d3/d4/d1d/f39 14073941 1 2024-02-28T23:07:36.825 INFO:tasks.workunit.client.1.smithi084.stdout:2/438: chown d5/d7/d12/d24/d3a 7 1 2024-02-28T23:07:36.825 INFO:tasks.workunit.client.1.smithi084.stdout:7/502: creat db/d10/f94 x:0 0 0 2024-02-28T23:07:36.829 INFO:tasks.workunit.client.1.smithi084.stdout:1/555: sync 2024-02-28T23:07:36.831 INFO:tasks.workunit.client.0.smithi057.stdout:2/450: dread d8/f23 [0,4194304] 0 2024-02-28T23:07:36.838 INFO:tasks.workunit.client.0.smithi057.stdout:7/358: rename d0/d5/f5c to d0/d5/d36/d4e/d15/d29/d3d/f6e 0 2024-02-28T23:07:36.838 INFO:tasks.workunit.client.0.smithi057.stdout:6/428: rename d0/d1b/d3e to d0/d1b/d3e/d3d/d74/d91 22 2024-02-28T23:07:36.838 INFO:tasks.workunit.client.0.smithi057.stdout:3/297: creat d3/d8/d27/f5f x:0 0 0 2024-02-28T23:07:36.841 INFO:tasks.workunit.client.0.smithi057.stdout:4/363: dread f15 [0,4194304] 0 2024-02-28T23:07:36.843 INFO:tasks.workunit.client.1.smithi084.stdout:5/556: truncate dc/de/df/d16/d34/f4d 2454300 0 2024-02-28T23:07:36.843 INFO:tasks.workunit.client.1.smithi084.stdout:9/530: write d4/fc [1254775,302] 0 2024-02-28T23:07:36.844 INFO:tasks.workunit.client.1.smithi084.stdout:1/556: rmdir d12/d1b/d2e/d52/d5b 39 2024-02-28T23:07:36.845 INFO:tasks.workunit.client.1.smithi084.stdout:8/471: rename d0/d9/d63 to d0/d44/d9f 0 2024-02-28T23:07:36.846 INFO:tasks.workunit.client.1.smithi084.stdout:2/439: creat d5/d7/d9/d7c/d5b/f7f x:0 0 0 2024-02-28T23:07:36.847 INFO:tasks.workunit.client.1.smithi084.stdout:6/491: getdents d1/d13/d9d 0 2024-02-28T23:07:36.847 INFO:tasks.workunit.client.0.smithi057.stdout:3/298: fdatasync d3/d8/d1b/f4a 0 2024-02-28T23:07:36.847 INFO:tasks.workunit.client.0.smithi057.stdout:0/378: rename d17/d20/d4a/f68 to d17/d20/d26/d48/d55/f93 0 2024-02-28T23:07:36.848 INFO:tasks.workunit.client.1.smithi084.stdout:1/557: readlink d12/d1b/d63/l6e 0 2024-02-28T23:07:36.848 INFO:tasks.workunit.client.0.smithi057.stdout:9/449: getdents dd/d1b/d40/d44 0 2024-02-28T23:07:36.849 INFO:tasks.workunit.client.1.smithi084.stdout:5/557: chown dc/de/df/f20 114956216 1 2024-02-28T23:07:36.851 INFO:tasks.workunit.client.0.smithi057.stdout:4/364: mknod d1a/d1b/d2d/d49/c84 0 2024-02-28T23:07:36.851 INFO:tasks.workunit.client.0.smithi057.stdout:6/429: mkdir d0/d1b/d3e/d3d/d42/d4f/d92 0 2024-02-28T23:07:36.852 INFO:tasks.workunit.client.0.smithi057.stdout:3/299: write d3/d4/d1d/f2e [1259905,35381] 0 2024-02-28T23:07:36.852 INFO:tasks.workunit.client.1.smithi084.stdout:0/498: truncate d1/d6/d59/d70/f73 70850 0 2024-02-28T23:07:36.858 INFO:tasks.workunit.client.0.smithi057.stdout:5/326: dwrite d4/d9/d14/f36 [0,4194304] 0 2024-02-28T23:07:36.860 INFO:tasks.workunit.client.0.smithi057.stdout:3/300: write d3/f5a [1177376,18311] 0 2024-02-28T23:07:36.861 INFO:tasks.workunit.client.0.smithi057.stdout:4/365: write d1a/d1e/d4b/d4f/f53 [570380,98715] 0 2024-02-28T23:07:36.861 INFO:tasks.workunit.client.1.smithi084.stdout:0/499: read d1/d8/f36 [187130,36962] 0 2024-02-28T23:07:36.861 INFO:tasks.workunit.client.1.smithi084.stdout:3/477: dwrite d3/d27/d7d/db0/d1a/d29/d41/d60/d98/fa8 [0,4194304] 0 2024-02-28T23:07:36.861 INFO:tasks.workunit.client.1.smithi084.stdout:7/503: link db/d11/d27/d36/f5a db/d11/f95 0 2024-02-28T23:07:36.862 INFO:tasks.workunit.client.0.smithi057.stdout:9/450: dread dd/de/d14/d48/f74 [0,4194304] 0 2024-02-28T23:07:36.865 INFO:tasks.workunit.client.1.smithi084.stdout:6/492: mknod d1/d13/d2e/cae 0 2024-02-28T23:07:36.866 INFO:tasks.workunit.client.0.smithi057.stdout:4/366: dread - d1a/d1b/d2d/f35 zero size 2024-02-28T23:07:36.866 INFO:tasks.workunit.client.1.smithi084.stdout:8/472: read d0/d11/f99 [307174,128360] 0 2024-02-28T23:07:36.867 INFO:tasks.workunit.client.0.smithi057.stdout:6/430: creat d0/d1b/d3e/d3d/d42/d4f/d53/f93 x:0 0 0 2024-02-28T23:07:36.870 INFO:tasks.workunit.client.0.smithi057.stdout:3/301: symlink d3/d8/d1b/l60 0 2024-02-28T23:07:36.871 INFO:tasks.workunit.client.0.smithi057.stdout:9/451: creat dd/de/d24/d72/f91 x:0 0 0 2024-02-28T23:07:36.871 INFO:tasks.workunit.client.0.smithi057.stdout:1/291: dread d2/d7/df/f4d [0,4194304] 0 2024-02-28T23:07:36.872 INFO:tasks.workunit.client.1.smithi084.stdout:6/493: chown d1/d13/d47/f7a 1 1 2024-02-28T23:07:36.873 INFO:tasks.workunit.client.0.smithi057.stdout:4/367: symlink d1a/d1b/d24/d2e/d43/l85 0 2024-02-28T23:07:36.874 INFO:tasks.workunit.client.1.smithi084.stdout:4/448: getdents d1/d18/d20/d14/d30 0 2024-02-28T23:07:36.874 INFO:tasks.workunit.client.1.smithi084.stdout:3/478: fdatasync d3/d27/d7d/db0/d1a/d21/d3d/f3c 0 2024-02-28T23:07:36.876 INFO:tasks.workunit.client.0.smithi057.stdout:9/452: fdatasync dd/de/d24/d72/f7f 0 2024-02-28T23:07:36.877 INFO:tasks.workunit.client.0.smithi057.stdout:4/368: fdatasync d1a/d1b/d2d/f63 0 2024-02-28T23:07:36.879 INFO:tasks.workunit.client.0.smithi057.stdout:6/431: mkdir d0/d17/d94 0 2024-02-28T23:07:36.881 INFO:tasks.workunit.client.0.smithi057.stdout:0/379: write d17/d20/d26/f46 [1975674,3659] 0 2024-02-28T23:07:36.882 INFO:tasks.workunit.client.0.smithi057.stdout:1/292: truncate d2/d7/df/db/d12/f5e 196486 0 2024-02-28T23:07:36.886 INFO:tasks.workunit.client.0.smithi057.stdout:4/369: dread - d1a/d1b/d24/d2e/d43/f47 zero size 2024-02-28T23:07:36.886 INFO:tasks.workunit.client.1.smithi084.stdout:0/500: rename d1/d8/c52 to d1/d6/db/d76/ca2 0 2024-02-28T23:07:36.889 INFO:tasks.workunit.client.1.smithi084.stdout:8/473: symlink d0/d9/d1b/d20/la0 0 2024-02-28T23:07:36.889 INFO:tasks.workunit.client.0.smithi057.stdout:9/453: write dd/de/d14/f61 [2104170,3921] 0 2024-02-28T23:07:36.890 INFO:tasks.workunit.client.0.smithi057.stdout:1/293: fdatasync d2/d7/df/f4d 0 2024-02-28T23:07:36.891 INFO:tasks.workunit.client.0.smithi057.stdout:9/454: chown dd/d1b/d40 64650 1 2024-02-28T23:07:36.893 INFO:tasks.workunit.client.1.smithi084.stdout:5/558: dwrite dc/de/df/d31/dd1/d5f/d9b/f9c [0,4194304] 0 2024-02-28T23:07:36.901 INFO:tasks.workunit.client.0.smithi057.stdout:6/432: link d0/d17/l83 d0/d1b/d3e/d3d/d7d/l95 0 2024-02-28T23:07:36.901 INFO:tasks.workunit.client.0.smithi057.stdout:5/327: dread d4/d21/f35 [0,4194304] 0 2024-02-28T23:07:36.902 INFO:tasks.workunit.client.0.smithi057.stdout:3/302: dread d3/d4/d7/f10 [4194304,4194304] 0 2024-02-28T23:07:36.903 INFO:tasks.workunit.client.1.smithi084.stdout:3/479: symlink d3/d27/d7d/db0/d14/lb6 0 2024-02-28T23:07:36.907 INFO:tasks.workunit.client.0.smithi057.stdout:2/451: dread d8/d11/d21/d28/d41/f53 [0,4194304] 0 2024-02-28T23:07:36.908 INFO:tasks.workunit.client.1.smithi084.stdout:7/504: getdents db/d10/d17/d64 0 2024-02-28T23:07:36.909 INFO:tasks.workunit.client.1.smithi084.stdout:9/531: dwrite d4/d32/f67 [0,4194304] 0 2024-02-28T23:07:36.910 INFO:tasks.workunit.client.0.smithi057.stdout:6/433: fdatasync d0/d17/d20/f31 0 2024-02-28T23:07:36.913 INFO:tasks.workunit.client.0.smithi057.stdout:3/303: dread - d3/d20/d2d/f56 zero size 2024-02-28T23:07:36.913 INFO:tasks.workunit.client.0.smithi057.stdout:1/294: dread d2/f21 [0,4194304] 0 2024-02-28T23:07:36.914 INFO:tasks.workunit.client.0.smithi057.stdout:4/370: rename d1a/d2f/f31 to d1a/d1b/d2d/d49/f86 0 2024-02-28T23:07:36.916 INFO:tasks.workunit.client.1.smithi084.stdout:2/440: truncate d5/d7/d12/d24/d2a/d53/f5f 464612 0 2024-02-28T23:07:36.918 INFO:tasks.workunit.client.1.smithi084.stdout:1/558: dwrite d12/d1b/f54 [4194304,4194304] 0 2024-02-28T23:07:36.918 INFO:tasks.workunit.client.0.smithi057.stdout:6/434: symlink d0/d1b/d3e/d3d/d42/d4f/l96 0 2024-02-28T23:07:36.920 INFO:tasks.workunit.client.0.smithi057.stdout:0/380: dwrite d17/d20/d4a/f81 [0,4194304] 0 2024-02-28T23:07:36.921 INFO:tasks.workunit.client.0.smithi057.stdout:4/371: chown d1a/d1b/d24/d2e/l30 235 1 2024-02-28T23:07:36.923 INFO:tasks.workunit.client.1.smithi084.stdout:5/559: rmdir dc/de/df/d16/d34/daa 39 2024-02-28T23:07:36.924 INFO:tasks.workunit.client.0.smithi057.stdout:5/328: creat d4/d7/f68 x:0 0 0 2024-02-28T23:07:36.924 INFO:tasks.workunit.client.0.smithi057.stdout:2/452: mkdir d8/d11/d21/d28/d49/d85/d92 0 2024-02-28T23:07:36.924 INFO:tasks.workunit.client.1.smithi084.stdout:8/474: mknod d0/d44/d9f/ca1 0 2024-02-28T23:07:36.927 INFO:tasks.workunit.client.0.smithi057.stdout:5/329: read d4/f6 [267475,19998] 0 2024-02-28T23:07:36.929 INFO:tasks.workunit.client.0.smithi057.stdout:5/330: fdatasync d4/d21/d24/f51 0 2024-02-28T23:07:36.930 INFO:tasks.workunit.client.1.smithi084.stdout:7/505: mknod db/d10/d18/c96 0 2024-02-28T23:07:36.932 INFO:tasks.workunit.client.0.smithi057.stdout:3/304: link d3/d20/d2d/f56 d3/f61 0 2024-02-28T23:07:36.933 INFO:tasks.workunit.client.0.smithi057.stdout:5/331: fsync f2 0 2024-02-28T23:07:36.936 INFO:tasks.workunit.client.1.smithi084.stdout:6/494: dwrite d1/d9/d1d/d1e/d33/f8f [4194304,4194304] 0 2024-02-28T23:07:36.936 INFO:tasks.workunit.client.0.smithi057.stdout:2/453: creat d8/d11/d48/d50/f93 x:0 0 0 2024-02-28T23:07:36.937 INFO:tasks.workunit.client.1.smithi084.stdout:5/560: symlink dc/d51/ld6 0 2024-02-28T23:07:36.943 INFO:tasks.workunit.client.1.smithi084.stdout:7/506: chown db/d10/d17/l1f 2073354 1 2024-02-28T23:07:36.944 INFO:tasks.workunit.client.0.smithi057.stdout:5/332: read d4/d7/d8/f26 [4346812,68741] 0 2024-02-28T23:07:36.946 INFO:tasks.workunit.client.0.smithi057.stdout:6/435: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:36.947 INFO:tasks.workunit.client.1.smithi084.stdout:7/507: fdatasync db/d10/d18/f82 0 2024-02-28T23:07:36.947 INFO:tasks.workunit.client.1.smithi084.stdout:1/559: rename d12/d1b/d2e/d30/c21 to d12/d1b/d2e/d30/d77/cb8 0 2024-02-28T23:07:36.949 INFO:tasks.workunit.client.0.smithi057.stdout:3/305: mkdir d3/d4/d1d/d47/d62 0 2024-02-28T23:07:36.950 INFO:tasks.workunit.client.0.smithi057.stdout:9/455: dwrite dd/de/d14/f21 [0,4194304] 0 2024-02-28T23:07:36.951 INFO:tasks.workunit.client.1.smithi084.stdout:2/441: dread d5/d7/d12/d24/d2a/d35/f5d [0,4194304] 0 2024-02-28T23:07:36.955 INFO:tasks.workunit.client.0.smithi057.stdout:6/436: chown d0/d4/d25/f71 344476576 1 2024-02-28T23:07:36.960 INFO:tasks.workunit.client.0.smithi057.stdout:5/333: mkdir d4/d9/d14/d3b/d61/d69 0 2024-02-28T23:07:36.961 INFO:tasks.workunit.client.1.smithi084.stdout:4/449: dread d1/d18/d20/d14/d30/d47/f49 [0,4194304] 0 2024-02-28T23:07:36.964 INFO:tasks.workunit.client.1.smithi084.stdout:5/561: dread - dc/de/df/d16/d34/daa/dae/fc5 zero size 2024-02-28T23:07:36.965 INFO:tasks.workunit.client.0.smithi057.stdout:1/295: dwrite d2/d7/df/db/d3a/d2a/f2c [0,4194304] 0 2024-02-28T23:07:36.966 INFO:tasks.workunit.client.0.smithi057.stdout:0/381: dwrite d17/d20/d4a/d78/f90 [0,4194304] 0 2024-02-28T23:07:36.968 INFO:tasks.workunit.client.1.smithi084.stdout:7/508: write db/d10/d17/d3b/f7c [1768170,6211] 0 2024-02-28T23:07:36.971 INFO:tasks.workunit.client.1.smithi084.stdout:9/532: dwrite d4/d6/d15/d12/d41/f7e [0,4194304] 0 2024-02-28T23:07:36.971 INFO:tasks.workunit.client.1.smithi084.stdout:0/501: dwrite d1/d6/d59/f5a [0,4194304] 0 2024-02-28T23:07:36.971 INFO:tasks.workunit.client.1.smithi084.stdout:1/560: symlink d12/d1b/d2e/d52/d61/d74/lb9 0 2024-02-28T23:07:36.972 INFO:tasks.workunit.client.0.smithi057.stdout:4/372: dwrite d1a/d1b/f46 [0,4194304] 0 2024-02-28T23:07:36.972 INFO:tasks.workunit.client.0.smithi057.stdout:5/334: creat d4/d9/d14/d3b/d61/d69/f6a x:0 0 0 2024-02-28T23:07:36.979 INFO:tasks.workunit.client.1.smithi084.stdout:2/442: creat d5/d7/d2e/d4a/f80 x:0 0 0 2024-02-28T23:07:36.982 INFO:tasks.workunit.client.0.smithi057.stdout:1/296: creat d2/d7/df/db/d12/d19/d24/d62/f64 x:0 0 0 2024-02-28T23:07:36.984 INFO:tasks.workunit.client.1.smithi084.stdout:9/533: dread - d4/d6/f6b zero size 2024-02-28T23:07:36.984 INFO:tasks.workunit.client.0.smithi057.stdout:0/382: truncate d17/d20/d23/d4b/f86 910014 0 2024-02-28T23:07:36.985 INFO:tasks.workunit.client.1.smithi084.stdout:5/562: creat dc/de/df/d31/fd7 x:0 0 0 2024-02-28T23:07:36.989 INFO:tasks.workunit.client.1.smithi084.stdout:1/561: write d12/f45 [1525774,69649] 0 2024-02-28T23:07:36.989 INFO:tasks.workunit.client.1.smithi084.stdout:9/534: chown d4/d32/d3f/d89 22433 1 2024-02-28T23:07:36.990 INFO:tasks.workunit.client.1.smithi084.stdout:5/563: write dc/d51/f9d [520013,3102] 0 2024-02-28T23:07:36.991 INFO:tasks.workunit.client.0.smithi057.stdout:2/454: dread d8/f42 [0,4194304] 0 2024-02-28T23:07:36.992 INFO:tasks.workunit.client.0.smithi057.stdout:6/437: rename d0/d1b/d3e/d3d/d42/d4f/d53/f89 to d0/d4/d25/d60/f97 0 2024-02-28T23:07:36.992 INFO:tasks.workunit.client.0.smithi057.stdout:3/306: dread d3/f2c [0,4194304] 0 2024-02-28T23:07:36.995 INFO:tasks.workunit.client.1.smithi084.stdout:1/562: chown d12/d1b/d2e/d30/f1d 55242 1 2024-02-28T23:07:36.996 INFO:tasks.workunit.client.1.smithi084.stdout:3/480: dread d3/f11 [0,4194304] 0 2024-02-28T23:07:36.997 INFO:tasks.workunit.client.1.smithi084.stdout:2/443: symlink d5/d7/d12/d24/l81 0 2024-02-28T23:07:37.000 INFO:tasks.workunit.client.0.smithi057.stdout:0/383: creat d17/d19/d4d/d7b/f94 x:0 0 0 2024-02-28T23:07:37.000 INFO:tasks.workunit.client.0.smithi057.stdout:1/297: mkdir d2/d7/df/db/d12/d19/d65 0 2024-02-28T23:07:37.002 INFO:tasks.workunit.client.0.smithi057.stdout:9/456: dwrite dd/d1b/d40/f86 [0,4194304] 0 2024-02-28T23:07:37.003 INFO:tasks.workunit.client.1.smithi084.stdout:8/475: dread d0/d24/f5b [0,4194304] 0 2024-02-28T23:07:37.005 INFO:tasks.workunit.client.1.smithi084.stdout:7/509: creat db/d5e/d80/f97 x:0 0 0 2024-02-28T23:07:37.007 INFO:tasks.workunit.client.1.smithi084.stdout:8/476: truncate d0/d44/d9f/f7e 1186910 0 2024-02-28T23:07:37.007 INFO:tasks.workunit.client.1.smithi084.stdout:4/450: getdents d1/d18/d20/d24/d2f/d3e/d52/d87 0 2024-02-28T23:07:37.007 INFO:tasks.workunit.client.1.smithi084.stdout:1/563: symlink d12/d1b/d2e/d32/lba 0 2024-02-28T23:07:37.011 INFO:tasks.workunit.client.1.smithi084.stdout:2/444: rmdir d5/d7/d9/d7c/d5b 39 2024-02-28T23:07:37.013 INFO:tasks.workunit.client.1.smithi084.stdout:3/481: stat d3/d27/d7d/db0/d14/c42 0 2024-02-28T23:07:37.014 INFO:tasks.workunit.client.1.smithi084.stdout:7/510: truncate db/d11/f5b 119039 0 2024-02-28T23:07:37.016 INFO:tasks.workunit.client.1.smithi084.stdout:8/477: readlink d0/d9/l3d 0 2024-02-28T23:07:37.018 INFO:tasks.workunit.client.1.smithi084.stdout:2/445: fsync d5/d7/d12/d24/f66 0 2024-02-28T23:07:37.019 INFO:tasks.workunit.client.0.smithi057.stdout:0/384: unlink d17/d20/d23/d4b/f86 0 2024-02-28T23:07:37.022 INFO:tasks.workunit.client.1.smithi084.stdout:8/478: fdatasync d0/d9/d30/d33/d71/f92 0 2024-02-28T23:07:37.022 INFO:tasks.workunit.client.0.smithi057.stdout:2/455: mkdir d8/d11/d21/d70/d91/d94 0 2024-02-28T23:07:37.026 INFO:tasks.workunit.client.0.smithi057.stdout:5/335: dwrite d4/d9/d14/d3b/d61/d69/f6a [0,4194304] 0 2024-02-28T23:07:37.029 INFO:tasks.workunit.client.1.smithi084.stdout:0/502: dread d1/d6/db/d35/d44/f79 [0,4194304] 0 2024-02-28T23:07:37.031 INFO:tasks.workunit.client.1.smithi084.stdout:7/511: creat db/f98 x:0 0 0 2024-02-28T23:07:37.031 INFO:tasks.workunit.client.1.smithi084.stdout:0/503: dread - d1/d6/db/d35/d44/f6f zero size 2024-02-28T23:07:37.032 INFO:tasks.workunit.client.0.smithi057.stdout:2/456: creat d8/d11/d15/f95 x:0 0 0 2024-02-28T23:07:37.032 INFO:tasks.workunit.client.0.smithi057.stdout:0/385: dread d17/d20/d26/d48/d7c/f91 [0,4194304] 0 2024-02-28T23:07:37.032 INFO:tasks.workunit.client.0.smithi057.stdout:3/307: dwrite d3/d8/f29 [0,4194304] 0 2024-02-28T23:07:37.034 INFO:tasks.workunit.client.1.smithi084.stdout:7/512: dread - db/d10/d17/d64/f93 zero size 2024-02-28T23:07:37.034 INFO:tasks.workunit.client.0.smithi057.stdout:3/308: write d3/f5a [1699993,82339] 0 2024-02-28T23:07:37.035 INFO:tasks.workunit.client.1.smithi084.stdout:8/479: creat d0/d24/d9e/fa2 x:0 0 0 2024-02-28T23:07:37.037 INFO:tasks.workunit.client.1.smithi084.stdout:2/446: rename d5/d7/d1d/l33 to d5/dd/d69/l82 0 2024-02-28T23:07:37.038 INFO:tasks.workunit.client.1.smithi084.stdout:8/480: fsync d0/d9/d30/d33/f94 0 2024-02-28T23:07:37.039 INFO:tasks.workunit.client.0.smithi057.stdout:3/309: readlink d3/d4/d1d/d47/l52 0 2024-02-28T23:07:37.039 INFO:tasks.workunit.client.0.smithi057.stdout:3/310: chown d3/d8 3348481 1 2024-02-28T23:07:37.040 INFO:tasks.workunit.client.0.smithi057.stdout:5/336: rename d4/d21/d24/l27 to d4/d9/d14/d3b/l6b 0 2024-02-28T23:07:37.041 INFO:tasks.workunit.client.0.smithi057.stdout:3/311: dread - d3/d4/d1d/f36 zero size 2024-02-28T23:07:37.042 INFO:tasks.workunit.client.1.smithi084.stdout:0/504: mkdir d1/d8/d69/da3 0 2024-02-28T23:07:37.044 INFO:tasks.workunit.client.1.smithi084.stdout:9/535: dread d4/d6/f29 [0,4194304] 0 2024-02-28T23:07:37.045 INFO:tasks.workunit.client.0.smithi057.stdout:5/337: write d4/d9/d16/f19 [575256,74941] 0 2024-02-28T23:07:37.049 INFO:tasks.workunit.client.0.smithi057.stdout:3/312: symlink d3/d8/d27/l63 0 2024-02-28T23:07:37.049 INFO:tasks.workunit.client.1.smithi084.stdout:7/513: read db/d10/f23 [90826,15360] 0 2024-02-28T23:07:37.049 INFO:tasks.workunit.client.1.smithi084.stdout:8/481: mknod d0/d24/d4d/ca3 0 2024-02-28T23:07:37.051 INFO:tasks.workunit.client.1.smithi084.stdout:2/447: fdatasync d5/d7/d12/d24/d2a/d35/f5d 0 2024-02-28T23:07:37.053 INFO:tasks.workunit.client.0.smithi057.stdout:3/313: dread d3/d8/d27/f55 [0,4194304] 0 2024-02-28T23:07:37.053 INFO:tasks.workunit.client.1.smithi084.stdout:8/482: truncate d0/d11/f23 5066910 0 2024-02-28T23:07:37.054 INFO:tasks.workunit.client.0.smithi057.stdout:3/314: dread - d3/d8/f45 zero size 2024-02-28T23:07:37.056 INFO:tasks.workunit.client.1.smithi084.stdout:0/505: fdatasync d1/d6/db/d37/f96 0 2024-02-28T23:07:37.057 INFO:tasks.workunit.client.1.smithi084.stdout:8/483: write d0/d11/f95 [228985,111868] 0 2024-02-28T23:07:37.060 INFO:tasks.workunit.client.0.smithi057.stdout:2/457: dwrite d8/d11/d21/d28/f82 [0,4194304] 0 2024-02-28T23:07:37.062 INFO:tasks.workunit.client.1.smithi084.stdout:8/484: write d0/d9/d30/d33/d71/f75 [634536,67510] 0 2024-02-28T23:07:37.064 INFO:tasks.workunit.client.0.smithi057.stdout:5/338: link d4/d32/l3e d4/d9/d14/d63/l6c 0 2024-02-28T23:07:37.065 INFO:tasks.workunit.client.0.smithi057.stdout:3/315: creat d3/f64 x:0 0 0 2024-02-28T23:07:37.067 INFO:tasks.workunit.client.0.smithi057.stdout:4/373: truncate d1a/d1b/f32 3343151 0 2024-02-28T23:07:37.072 INFO:tasks.workunit.client.1.smithi084.stdout:3/482: dread d3/fe [0,4194304] 0 2024-02-28T23:07:37.075 INFO:tasks.workunit.client.0.smithi057.stdout:5/339: rename d4/d7/c4a to d4/d21/d24/c6d 0 2024-02-28T23:07:37.077 INFO:tasks.workunit.client.1.smithi084.stdout:4/451: dread d1/f4 [4194304,4194304] 0 2024-02-28T23:07:37.078 INFO:tasks.workunit.client.0.smithi057.stdout:2/458: truncate d8/f42 771081 0 2024-02-28T23:07:37.079 INFO:tasks.workunit.client.1.smithi084.stdout:2/448: dread d5/dd/f58 [0,4194304] 0 2024-02-28T23:07:37.079 INFO:tasks.workunit.client.0.smithi057.stdout:5/340: write d4/d21/f2f [601404,57738] 0 2024-02-28T23:07:37.080 INFO:tasks.workunit.client.0.smithi057.stdout:3/316: getdents d3/d20/d2d 0 2024-02-28T23:07:37.084 INFO:tasks.workunit.client.1.smithi084.stdout:8/485: creat d0/d9/d30/d33/d71/fa4 x:0 0 0 2024-02-28T23:07:37.085 INFO:tasks.workunit.client.0.smithi057.stdout:5/341: truncate d4/d9/d14/f54 904796 0 2024-02-28T23:07:37.087 INFO:tasks.workunit.client.1.smithi084.stdout:1/564: dread d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:37.087 INFO:tasks.workunit.client.0.smithi057.stdout:0/386: truncate d17/d20/d23/d37/f43 3490844 0 2024-02-28T23:07:37.092 INFO:tasks.workunit.client.1.smithi084.stdout:2/449: mkdir d5/d1f/d83 0 2024-02-28T23:07:37.092 INFO:tasks.workunit.client.1.smithi084.stdout:3/483: creat d3/d27/fb7 x:0 0 0 2024-02-28T23:07:37.094 INFO:tasks.workunit.client.1.smithi084.stdout:1/565: write d12/d1b/d2e/f57 [150501,107301] 0 2024-02-28T23:07:37.095 INFO:tasks.workunit.client.1.smithi084.stdout:7/514: rmdir db/d11/d83 0 2024-02-28T23:07:37.097 INFO:tasks.workunit.client.1.smithi084.stdout:7/515: fdatasync db/d5e/f6f 0 2024-02-28T23:07:37.100 INFO:tasks.workunit.client.1.smithi084.stdout:9/536: rename d4/cb to d4/d6/d15/d75/ca6 0 2024-02-28T23:07:37.101 INFO:tasks.workunit.client.1.smithi084.stdout:7/516: chown db/d10/d17/d3b 46155449 1 2024-02-28T23:07:37.101 INFO:tasks.workunit.client.1.smithi084.stdout:3/484: truncate d3/f28 1920223 0 2024-02-28T23:07:37.103 INFO:tasks.workunit.client.1.smithi084.stdout:7/517: stat db/ff 0 2024-02-28T23:07:37.108 INFO:tasks.workunit.client.1.smithi084.stdout:0/506: rename d1/d8/f1d to d1/d8/d78/fa4 0 2024-02-28T23:07:37.108 INFO:tasks.workunit.client.0.smithi057.stdout:5/342: dwrite d4/d9/d14/d3b/f47 [0,4194304] 0 2024-02-28T23:07:37.111 INFO:tasks.workunit.client.0.smithi057.stdout:3/317: dwrite d3/d4/f17 [0,4194304] 0 2024-02-28T23:07:37.115 INFO:tasks.workunit.client.1.smithi084.stdout:2/450: dread d5/dd/f16 [0,4194304] 0 2024-02-28T23:07:37.118 INFO:tasks.workunit.client.1.smithi084.stdout:3/485: mkdir d3/d50/d5a/d73/db8 0 2024-02-28T23:07:37.118 INFO:tasks.workunit.client.0.smithi057.stdout:0/387: dread d17/d20/d26/d48/f5c [0,4194304] 0 2024-02-28T23:07:37.119 INFO:tasks.workunit.client.0.smithi057.stdout:4/374: dread f14 [0,4194304] 0 2024-02-28T23:07:37.119 INFO:tasks.workunit.client.1.smithi084.stdout:8/486: dwrite d0/d24/d4d/f58 [0,4194304] 0 2024-02-28T23:07:37.121 INFO:tasks.workunit.client.1.smithi084.stdout:8/487: chown d0/d11/f8f 4 1 2024-02-28T23:07:37.124 INFO:tasks.workunit.client.0.smithi057.stdout:0/388: mknod d17/d20/d4a/d78/c95 0 2024-02-28T23:07:37.124 INFO:tasks.workunit.client.1.smithi084.stdout:4/452: rename d1/d18/d20/d24/d2f/d3e/d52/d48/d73/c94 to d1/d18/d20/d65/d8e/d55/c98 0 2024-02-28T23:07:37.125 INFO:tasks.workunit.client.0.smithi057.stdout:4/375: fdatasync d1a/d48/f71 0 2024-02-28T23:07:37.126 INFO:tasks.workunit.client.0.smithi057.stdout:0/389: stat d17/d20/d26/d48/f84 0 2024-02-28T23:07:37.126 INFO:tasks.workunit.client.0.smithi057.stdout:2/459: dread d8/d11/f29 [0,4194304] 0 2024-02-28T23:07:37.127 INFO:tasks.workunit.client.1.smithi084.stdout:7/518: fdatasync db/d10/f2e 0 2024-02-28T23:07:37.127 INFO:tasks.workunit.client.0.smithi057.stdout:7/359: sync 2024-02-28T23:07:37.127 INFO:tasks.workunit.client.0.smithi057.stdout:8/357: sync 2024-02-28T23:07:37.131 INFO:tasks.workunit.client.0.smithi057.stdout:5/343: read d4/d9/f2a [4551667,80656] 0 2024-02-28T23:07:37.131 INFO:tasks.workunit.client.0.smithi057.stdout:3/318: read d3/d4/d7/f10 [971657,116081] 0 2024-02-28T23:07:37.134 INFO:tasks.workunit.client.1.smithi084.stdout:0/507: mkdir d1/d8/d69/d6b/d85/da5 0 2024-02-28T23:07:37.134 INFO:tasks.workunit.client.0.smithi057.stdout:0/390: rmdir d17/d20/d23/d47 39 2024-02-28T23:07:37.135 INFO:tasks.workunit.client.1.smithi084.stdout:2/451: mknod d5/dd/d69/c84 0 2024-02-28T23:07:37.135 INFO:tasks.workunit.client.1.smithi084.stdout:1/566: dwrite d12/d1b/d2e/d30/f1c [0,4194304] 0 2024-02-28T23:07:37.136 INFO:tasks.workunit.client.1.smithi084.stdout:5/564: sync 2024-02-28T23:07:37.137 INFO:tasks.workunit.client.1.smithi084.stdout:6/495: sync 2024-02-28T23:07:37.141 INFO:tasks.workunit.client.0.smithi057.stdout:2/460: creat d8/d11/d39/d7b/d8b/f96 x:0 0 0 2024-02-28T23:07:37.142 INFO:tasks.workunit.client.1.smithi084.stdout:4/453: chown d1/d18/d20/c29 4 1 2024-02-28T23:07:37.142 INFO:tasks.workunit.client.0.smithi057.stdout:0/391: fsync d17/d20/d23/d47/f7f 0 2024-02-28T23:07:37.145 INFO:tasks.workunit.client.0.smithi057.stdout:0/392: chown d17/d20/d26/d48/d55/f93 63348 1 2024-02-28T23:07:37.146 INFO:tasks.workunit.client.1.smithi084.stdout:1/567: creat d12/d84/fbb x:0 0 0 2024-02-28T23:07:37.146 INFO:tasks.workunit.client.1.smithi084.stdout:2/452: write d5/d7/d12/d24/f56 [454551,81580] 0 2024-02-28T23:07:37.148 INFO:tasks.workunit.client.1.smithi084.stdout:0/508: mknod d1/d6/db/d76/ca6 0 2024-02-28T23:07:37.149 INFO:tasks.workunit.client.0.smithi057.stdout:5/344: rmdir d4/d9/d16/d4e/d60 0 2024-02-28T23:07:37.150 INFO:tasks.workunit.client.1.smithi084.stdout:1/568: chown d12/d1b/d63/f56 90 1 2024-02-28T23:07:37.153 INFO:tasks.workunit.client.1.smithi084.stdout:0/509: chown d1/d8/d18/d28/f54 15 1 2024-02-28T23:07:37.163 INFO:tasks.workunit.client.1.smithi084.stdout:5/565: dwrite dc/fc8 [0,4194304] 0 2024-02-28T23:07:37.163 INFO:tasks.workunit.client.1.smithi084.stdout:6/496: dread d1/d13/d50/d65/f9b [0,4194304] 0 2024-02-28T23:07:37.167 INFO:tasks.workunit.client.1.smithi084.stdout:7/519: mknod db/d5e/d8a/c99 0 2024-02-28T23:07:37.171 INFO:tasks.workunit.client.1.smithi084.stdout:4/454: rename d1/l8d to d1/d18/d20/d65/d8e/d55/l99 0 2024-02-28T23:07:37.172 INFO:tasks.workunit.client.1.smithi084.stdout:2/453: creat d5/dd/d69/f85 x:0 0 0 2024-02-28T23:07:37.175 INFO:tasks.workunit.client.1.smithi084.stdout:9/537: dread d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:37.175 INFO:tasks.workunit.client.1.smithi084.stdout:8/488: truncate d0/d9/f21 2097355 0 2024-02-28T23:07:37.176 INFO:tasks.workunit.client.1.smithi084.stdout:3/486: link d3/l44 d3/d27/d7d/db0/d1a/d29/d41/d60/d98/lb9 0 2024-02-28T23:07:37.178 INFO:tasks.workunit.client.1.smithi084.stdout:0/510: dread d1/d8/f3c [0,4194304] 0 2024-02-28T23:07:37.180 INFO:tasks.workunit.client.1.smithi084.stdout:6/497: rmdir d1/d9/d1d/d1e/d33 39 2024-02-28T23:07:37.180 INFO:tasks.workunit.client.0.smithi057.stdout:0/393: rmdir d17/d20/d26 39 2024-02-28T23:07:37.180 INFO:tasks.workunit.client.0.smithi057.stdout:7/360: getdents d0/d5/d36/d4e/d15/d29/d3d/d51 0 2024-02-28T23:07:37.181 INFO:tasks.workunit.client.0.smithi057.stdout:5/345: unlink d4/d32/f41 0 2024-02-28T23:07:37.181 INFO:tasks.workunit.client.1.smithi084.stdout:8/489: read - d0/d9/d1b/f7f zero size 2024-02-28T23:07:37.181 INFO:tasks.workunit.client.1.smithi084.stdout:1/569: dwrite d12/d84/fad [0,4194304] 0 2024-02-28T23:07:37.183 INFO:tasks.workunit.client.1.smithi084.stdout:6/498: fsync d1/d13/d50/d74/faa 0 2024-02-28T23:07:37.187 INFO:tasks.workunit.client.1.smithi084.stdout:0/511: write d1/d6/db/d37/d72/d7b/f80 [9399223,23548] 0 2024-02-28T23:07:37.187 INFO:tasks.workunit.client.0.smithi057.stdout:7/361: mknod d0/d5/d36/d4e/d15/c6f 0 2024-02-28T23:07:37.204 INFO:tasks.workunit.client.1.smithi084.stdout:5/566: dread dc/de/df/d16/f29 [0,4194304] 0 2024-02-28T23:07:37.245 INFO:tasks.workunit.client.1.smithi084.stdout:2/454: creat d5/dd/f86 x:0 0 0 2024-02-28T23:07:37.246 INFO:tasks.workunit.client.1.smithi084.stdout:9/538: dread - d4/f37 zero size 2024-02-28T23:07:37.250 INFO:tasks.workunit.client.1.smithi084.stdout:3/487: creat d3/d27/d7d/da0/fba x:0 0 0 2024-02-28T23:07:37.255 INFO:tasks.workunit.client.1.smithi084.stdout:1/570: write d12/d1b/d63/f3f [9725076,47749] 0 2024-02-28T23:07:37.257 INFO:tasks.workunit.client.1.smithi084.stdout:0/512: mknod d1/d8/ca7 0 2024-02-28T23:07:37.259 INFO:tasks.workunit.client.1.smithi084.stdout:0/513: write d1/d6/db/d35/f5c [2704723,56167] 0 2024-02-28T23:07:37.264 INFO:tasks.workunit.client.1.smithi084.stdout:0/514: fdatasync d1/d6/db/d35/f5c 0 2024-02-28T23:07:37.267 INFO:tasks.workunit.client.1.smithi084.stdout:0/515: write d1/d6/db/d35/d82/f83 [255143,9215] 0 2024-02-28T23:07:37.269 INFO:tasks.workunit.client.1.smithi084.stdout:0/516: stat d1/d6/db/d35 0 2024-02-28T23:07:37.269 INFO:tasks.workunit.client.1.smithi084.stdout:9/539: read d4/d6/ff [2880063,104073] 0 2024-02-28T23:07:37.271 INFO:tasks.workunit.client.1.smithi084.stdout:9/540: chown d4/d6/d15/d75/fa4 26308 1 2024-02-28T23:07:37.278 INFO:tasks.workunit.client.0.smithi057.stdout:5/346: rename d4/d7/f68 to d4/d32/f6e 0 2024-02-28T23:07:37.278 INFO:tasks.workunit.client.0.smithi057.stdout:0/394: read d17/f58 [2560148,13913] 0 2024-02-28T23:07:37.278 INFO:tasks.workunit.client.1.smithi084.stdout:2/455: creat d5/d7/d9/d7c/d5b/f87 x:0 0 0 2024-02-28T23:07:37.280 INFO:tasks.workunit.client.0.smithi057.stdout:2/461: dwrite d8/d11/d21/f31 [4194304,4194304] 0 2024-02-28T23:07:37.284 INFO:tasks.workunit.client.1.smithi084.stdout:0/517: dread d1/d6/d59/d70/f73 [0,4194304] 0 2024-02-28T23:07:37.285 INFO:tasks.workunit.client.1.smithi084.stdout:9/541: symlink d4/d6/d15/d75/d9d/la7 0 2024-02-28T23:07:37.286 INFO:tasks.workunit.client.0.smithi057.stdout:4/376: dwrite d1a/d1b/f1c [0,4194304] 0 2024-02-28T23:07:37.287 INFO:tasks.workunit.client.0.smithi057.stdout:2/462: mkdir d8/d11/d15/d97 0 2024-02-28T23:07:37.289 INFO:tasks.workunit.client.1.smithi084.stdout:4/455: link d1/f8 d1/d18/d20/d24/d2f/d3e/d52/f9a 0 2024-02-28T23:07:37.290 INFO:tasks.workunit.client.0.smithi057.stdout:5/347: write d4/d7/d8/ff [2197127,111877] 0 2024-02-28T23:07:37.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/490: symlink d0/d24/la5 0 2024-02-28T23:07:37.294 INFO:tasks.workunit.client.0.smithi057.stdout:5/348: mknod d4/d9/d16/d4e/c6f 0 2024-02-28T23:07:37.296 INFO:tasks.workunit.client.1.smithi084.stdout:4/456: chown d1/d18/d20/d24/d2f/d3e/d52/d45 8 1 2024-02-28T23:07:37.296 INFO:tasks.workunit.client.1.smithi084.stdout:1/571: link d12/d1b/f48 d12/d1b/d63/fbc 0 2024-02-28T23:07:37.297 INFO:tasks.workunit.client.1.smithi084.stdout:8/491: write d0/d24/d4d/f58 [2253185,87913] 0 2024-02-28T23:07:37.298 INFO:tasks.workunit.client.1.smithi084.stdout:4/457: dread - d1/d18/f5f zero size 2024-02-28T23:07:37.300 INFO:tasks.workunit.client.0.smithi057.stdout:4/377: rename d1a/d1b/d2d/d49/c60 to d1a/d1e/d4b/d4f/c87 0 2024-02-28T23:07:37.303 INFO:tasks.workunit.client.1.smithi084.stdout:8/492: chown d0/d9/d1b/f39 57487 1 2024-02-28T23:07:37.307 INFO:tasks.workunit.client.0.smithi057.stdout:5/349: write d4/d9/d14/d3b/d61/d69/f6a [4352139,3532] 0 2024-02-28T23:07:37.308 INFO:tasks.workunit.client.1.smithi084.stdout:0/518: dwrite d1/d8/f3f [0,4194304] 0 2024-02-28T23:07:37.310 INFO:tasks.workunit.client.0.smithi057.stdout:8/358: dread d4/d1e/d56/d6a/d42/f5e [0,4194304] 0 2024-02-28T23:07:37.311 INFO:tasks.workunit.client.1.smithi084.stdout:4/458: creat d1/d18/d20/d24/d2f/d3e/d52/f9b x:0 0 0 2024-02-28T23:07:37.315 INFO:tasks.workunit.client.0.smithi057.stdout:0/395: dread d17/d20/f27 [0,4194304] 0 2024-02-28T23:07:37.315 INFO:tasks.workunit.client.1.smithi084.stdout:2/456: dread d5/d7/d2e/f46 [0,4194304] 0 2024-02-28T23:07:37.316 INFO:tasks.workunit.client.1.smithi084.stdout:8/493: mknod d0/d9/d30/d33/ca6 0 2024-02-28T23:07:37.317 INFO:tasks.workunit.client.1.smithi084.stdout:5/567: mknod dc/de/df/d16/d34/cd8 0 2024-02-28T23:07:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:37 smithi057 ceph-mon[29096]: pgmap v25: 65 pgs: 65 active+clean; 1.6 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 215 MiB/s wr, 342 op/s 2024-02-28T23:07:37.319 INFO:tasks.workunit.client.0.smithi057.stdout:0/396: dread - d17/d20/d23/f8b zero size 2024-02-28T23:07:37.320 INFO:tasks.workunit.client.1.smithi084.stdout:1/572: truncate d12/d1b/d2e/d32/f5e 346818 0 2024-02-28T23:07:37.322 INFO:tasks.workunit.client.0.smithi057.stdout:2/463: dwrite d8/d11/d21/d28/d41/f5c [0,4194304] 0 2024-02-28T23:07:37.322 INFO:tasks.workunit.client.1.smithi084.stdout:0/519: truncate d1/d6/f53 774003 0 2024-02-28T23:07:37.323 INFO:tasks.workunit.client.1.smithi084.stdout:8/494: readlink d0/d9/d1b/d38/l36 0 2024-02-28T23:07:37.327 INFO:tasks.workunit.client.1.smithi084.stdout:4/459: rmdir d1/d18/d20/d65 39 2024-02-28T23:07:37.327 INFO:tasks.workunit.client.0.smithi057.stdout:0/397: fdatasync d17/d20/d26/f46 0 2024-02-28T23:07:37.332 INFO:tasks.workunit.client.0.smithi057.stdout:2/464: symlink d8/d11/d39/d7b/l98 0 2024-02-28T23:07:37.336 INFO:tasks.workunit.client.1.smithi084.stdout:5/568: dread dc/f28 [0,4194304] 0 2024-02-28T23:07:37.337 INFO:tasks.workunit.client.0.smithi057.stdout:8/359: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:37.338 INFO:tasks.workunit.client.1.smithi084.stdout:5/569: read - dc/de/df/da6/fd3 zero size 2024-02-28T23:07:37.339 INFO:tasks.workunit.client.0.smithi057.stdout:2/465: chown d8/d11/d21/d28/d41/f5c 74385 1 2024-02-28T23:07:37.340 INFO:tasks.workunit.client.0.smithi057.stdout:4/378: dwrite d1a/d1e/f42 [0,4194304] 0 2024-02-28T23:07:37.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/573: fdatasync d12/d1b/f2c 0 2024-02-28T23:07:37.342 INFO:tasks.workunit.client.0.smithi057.stdout:5/350: dread d4/d9/d14/d3b/d61/d69/f6a [0,4194304] 0 2024-02-28T23:07:37.343 INFO:tasks.workunit.client.0.smithi057.stdout:8/360: fdatasync d4/d13/d2c/f49 0 2024-02-28T23:07:37.345 INFO:tasks.workunit.client.0.smithi057.stdout:4/379: write d1a/d1b/f3e [1965089,75188] 0 2024-02-28T23:07:37.346 INFO:tasks.workunit.client.0.smithi057.stdout:2/466: mkdir d8/d11/d21/d70/d99 0 2024-02-28T23:07:37.348 INFO:tasks.workunit.client.0.smithi057.stdout:5/351: chown d4/d9/d16/f19 1614456616 1 2024-02-28T23:07:37.348 INFO:tasks.workunit.client.1.smithi084.stdout:4/460: creat d1/f9c x:0 0 0 2024-02-28T23:07:37.349 INFO:tasks.workunit.client.0.smithi057.stdout:5/352: stat d4/d9/c1f 0 2024-02-28T23:07:37.350 INFO:tasks.workunit.client.1.smithi084.stdout:2/457: getdents d5/d7/d1d 0 2024-02-28T23:07:37.352 INFO:tasks.workunit.client.0.smithi057.stdout:8/361: mknod d4/d1e/d56/d6a/d54/d61/c7e 0 2024-02-28T23:07:37.352 INFO:tasks.workunit.client.0.smithi057.stdout:4/380: truncate d1a/d1b/d2d/f7b 719308 0 2024-02-28T23:07:37.353 INFO:tasks.workunit.client.0.smithi057.stdout:5/353: stat d4/d9/c3f 0 2024-02-28T23:07:37.353 INFO:tasks.workunit.client.1.smithi084.stdout:8/495: getdents d0/d9/d30/d33/d71/d84 0 2024-02-28T23:07:37.354 INFO:tasks.workunit.client.0.smithi057.stdout:2/467: truncate d8/d11/d15/f5d 9416247 0 2024-02-28T23:07:37.357 INFO:tasks.workunit.client.1.smithi084.stdout:1/574: symlink d12/d1b/lbd 0 2024-02-28T23:07:37.357 INFO:tasks.workunit.client.1.smithi084.stdout:4/461: write d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [888450,75538] 0 2024-02-28T23:07:37.360 INFO:tasks.workunit.client.0.smithi057.stdout:8/362: mkdir d4/d13/d7f 0 2024-02-28T23:07:37.360 INFO:tasks.workunit.client.1.smithi084.stdout:1/575: write d12/d1b/d2e/f4e [3663580,61234] 0 2024-02-28T23:07:37.361 INFO:tasks.workunit.client.0.smithi057.stdout:2/468: fdatasync d8/d11/f17 0 2024-02-28T23:07:37.361 INFO:tasks.workunit.client.0.smithi057.stdout:8/363: readlink d4/le 0 2024-02-28T23:07:37.361 INFO:tasks.workunit.client.0.smithi057.stdout:7/362: dread d0/d5/d36/d4e/d15/f22 [0,4194304] 0 2024-02-28T23:07:37.362 INFO:tasks.workunit.client.0.smithi057.stdout:4/381: rename d1a/d1b/f46 to d1a/d1b/d2d/d49/d4d/f88 0 2024-02-28T23:07:37.364 INFO:tasks.workunit.client.1.smithi084.stdout:8/496: rename d0/d44/l4e to d0/d9/d30/la7 0 2024-02-28T23:07:37.366 INFO:tasks.workunit.client.0.smithi057.stdout:4/382: write fe [8246052,91298] 0 2024-02-28T23:07:37.367 INFO:tasks.workunit.client.1.smithi084.stdout:3/488: link d3/d27/d7d/db0/d14/d31/d34/f96 d3/d50/fbb 0 2024-02-28T23:07:37.369 INFO:tasks.workunit.client.1.smithi084.stdout:6/499: unlink d1/c42 0 2024-02-28T23:07:37.370 INFO:tasks.workunit.client.0.smithi057.stdout:7/363: creat d0/d5/d36/d4e/d15/d29/f70 x:0 0 0 2024-02-28T23:07:37.370 INFO:tasks.workunit.client.0.smithi057.stdout:8/364: chown d4/d1e/d56/d6a/d42/f44 145 1 2024-02-28T23:07:37.372 INFO:tasks.workunit.client.1.smithi084.stdout:6/500: dread - d1/d13/d50/f99 zero size 2024-02-28T23:07:37.372 INFO:tasks.workunit.client.1.smithi084.stdout:1/576: truncate d12/f23 2895003 0 2024-02-28T23:07:37.374 INFO:tasks.workunit.client.1.smithi084.stdout:8/497: rmdir d0/d11 39 2024-02-28T23:07:37.374 INFO:tasks.workunit.client.1.smithi084.stdout:1/577: chown d12/d1b/d2e/f57 786 1 2024-02-28T23:07:37.375 INFO:tasks.workunit.client.1.smithi084.stdout:3/489: write d3/d27/d7d/db0/d1a/d29/d41/d60/f8b [40471,32852] 0 2024-02-28T23:07:37.376 INFO:tasks.workunit.client.0.smithi057.stdout:0/398: dwrite f14 [0,4194304] 0 2024-02-28T23:07:37.378 INFO:tasks.workunit.client.0.smithi057.stdout:0/399: write d17/d20/d26/f69 [269885,13835] 0 2024-02-28T23:07:37.379 INFO:tasks.workunit.client.1.smithi084.stdout:6/501: mknod d1/d13/d47/caf 0 2024-02-28T23:07:37.379 INFO:tasks.workunit.client.1.smithi084.stdout:1/578: fsync d12/da2/fa9 0 2024-02-28T23:07:37.379 INFO:tasks.workunit.client.0.smithi057.stdout:8/365: mknod d4/d13/d18/c80 0 2024-02-28T23:07:37.381 INFO:tasks.workunit.client.0.smithi057.stdout:7/364: mkdir d0/d5/d36/d4e/d15/d29/d71 0 2024-02-28T23:07:37.385 INFO:tasks.workunit.client.0.smithi057.stdout:5/354: dwrite d4/d21/f2d [0,4194304] 0 2024-02-28T23:07:37.388 INFO:tasks.workunit.client.1.smithi084.stdout:3/490: creat d3/d27/d7d/db0/d14/d31/fbc x:0 0 0 2024-02-28T23:07:37.388 INFO:tasks.workunit.client.0.smithi057.stdout:0/400: creat d17/d20/d26/d30/f96 x:0 0 0 2024-02-28T23:07:37.389 INFO:tasks.workunit.client.0.smithi057.stdout:4/383: fdatasync d1a/d1b/d24/d2e/f4a 0 2024-02-28T23:07:37.391 INFO:tasks.workunit.client.1.smithi084.stdout:7/520: unlink db/d11/f12 0 2024-02-28T23:07:37.392 INFO:tasks.workunit.client.1.smithi084.stdout:3/491: chown d3/d27/d7d/db0/df/f46 3 1 2024-02-28T23:07:37.394 INFO:tasks.workunit.client.0.smithi057.stdout:0/401: write d17/d20/d23/f8b [776081,30167] 0 2024-02-28T23:07:37.395 INFO:tasks.workunit.client.1.smithi084.stdout:8/498: getdents d0/d9/d1b 0 2024-02-28T23:07:37.395 INFO:tasks.workunit.client.0.smithi057.stdout:8/366: getdents d4/d13/d18/d62 0 2024-02-28T23:07:37.398 INFO:tasks.workunit.client.0.smithi057.stdout:2/469: dwrite d8/d11/f5b [0,4194304] 0 2024-02-28T23:07:37.398 INFO:tasks.workunit.client.0.smithi057.stdout:0/402: fsync d17/d20/f66 0 2024-02-28T23:07:37.401 INFO:tasks.workunit.client.1.smithi084.stdout:8/499: truncate d0/d9/d1b/f46 2104893 0 2024-02-28T23:07:37.402 INFO:tasks.workunit.client.0.smithi057.stdout:4/384: truncate d1a/d1b/d24/d2e/d43/f47 585796 0 2024-02-28T23:07:37.403 INFO:tasks.workunit.client.1.smithi084.stdout:7/521: creat db/d10/d17/f9a x:0 0 0 2024-02-28T23:07:37.403 INFO:tasks.workunit.client.0.smithi057.stdout:8/367: creat d4/d1e/d56/d6a/d54/d6b/f81 x:0 0 0 2024-02-28T23:07:37.403 INFO:tasks.workunit.client.1.smithi084.stdout:8/500: chown d0/d44 1506693519 1 2024-02-28T23:07:37.404 INFO:tasks.workunit.client.0.smithi057.stdout:7/365: dread d0/d5/d36/d4e/d15/f60 [0,4194304] 0 2024-02-28T23:07:37.408 INFO:tasks.workunit.client.0.smithi057.stdout:7/366: readlink d0/d5/l6 0 2024-02-28T23:07:37.409 INFO:tasks.workunit.client.0.smithi057.stdout:0/403: link d17/d20/d26/d48/d55/f93 d17/d19/f97 0 2024-02-28T23:07:37.410 INFO:tasks.workunit.client.1.smithi084.stdout:6/502: dwrite d1/d13/d50/f99 [0,4194304] 0 2024-02-28T23:07:37.411 INFO:tasks.workunit.client.0.smithi057.stdout:8/368: mknod d4/d13/d7f/c82 0 2024-02-28T23:07:37.411 INFO:tasks.workunit.client.1.smithi084.stdout:8/501: mknod d0/d9/d1b/d38/d43/ca8 0 2024-02-28T23:07:37.412 INFO:tasks.workunit.client.1.smithi084.stdout:7/522: creat db/d10/d17/d3b/f9b x:0 0 0 2024-02-28T23:07:37.415 INFO:tasks.workunit.client.1.smithi084.stdout:8/502: write d0/d24/d6e/f8b [333975,111203] 0 2024-02-28T23:07:37.416 INFO:tasks.workunit.client.1.smithi084.stdout:7/523: write db/d10/d17/f9a [455163,86325] 0 2024-02-28T23:07:37.417 INFO:tasks.workunit.client.0.smithi057.stdout:8/369: symlink d4/d13/d2c/l83 0 2024-02-28T23:07:37.418 INFO:tasks.workunit.client.1.smithi084.stdout:5/570: write dc/de/f46 [113093,96471] 0 2024-02-28T23:07:37.419 INFO:tasks.workunit.client.1.smithi084.stdout:0/520: truncate d1/d8/f3f 1175123 0 2024-02-28T23:07:37.421 INFO:tasks.workunit.client.1.smithi084.stdout:0/521: stat d1/d6/db/l2b 0 2024-02-28T23:07:37.421 INFO:tasks.workunit.client.0.smithi057.stdout:8/370: chown d4/d1e/d56/d6a/d42/l4c 1 1 2024-02-28T23:07:37.424 INFO:tasks.workunit.client.1.smithi084.stdout:1/579: dread d12/d1b/d63/f3f [0,4194304] 0 2024-02-28T23:07:37.425 INFO:tasks.workunit.client.1.smithi084.stdout:0/522: write d1/d6/d59/d70/f8d [1704704,18081] 0 2024-02-28T23:07:37.426 INFO:tasks.workunit.client.0.smithi057.stdout:6/438: sync 2024-02-28T23:07:37.428 INFO:tasks.workunit.client.0.smithi057.stdout:0/404: getdents d17/d20/d26/d48/d7c 0 2024-02-28T23:07:37.429 INFO:tasks.workunit.client.0.smithi057.stdout:8/371: creat d4/d13/d2c/f84 x:0 0 0 2024-02-28T23:07:37.429 INFO:tasks.workunit.client.1.smithi084.stdout:5/571: mkdir dc/de/d83/dd9 0 2024-02-28T23:07:37.432 INFO:tasks.workunit.client.0.smithi057.stdout:6/439: write d0/d1b/d3e/d40/f73 [491764,1025] 0 2024-02-28T23:07:37.432 INFO:tasks.workunit.client.0.smithi057.stdout:4/385: dwrite d1a/d1b/d24/f82 [0,4194304] 0 2024-02-28T23:07:37.434 INFO:tasks.workunit.client.1.smithi084.stdout:6/503: truncate d1/d9/d1d/d1e/d33/f4c 328182 0 2024-02-28T23:07:37.435 INFO:tasks.workunit.client.0.smithi057.stdout:8/372: dread - d4/d1e/d56/d6a/d42/f5b zero size 2024-02-28T23:07:37.435 INFO:tasks.workunit.client.0.smithi057.stdout:5/355: dread d4/d21/d24/f22 [0,4194304] 0 2024-02-28T23:07:37.435 INFO:tasks.workunit.client.0.smithi057.stdout:0/405: mkdir d17/d20/d26/d48/d7c/d98 0 2024-02-28T23:07:37.437 INFO:tasks.workunit.client.1.smithi084.stdout:5/572: chown dc/de/d9e/lba 1013088 1 2024-02-28T23:07:37.438 INFO:tasks.workunit.client.1.smithi084.stdout:1/580: mkdir d12/dbe 0 2024-02-28T23:07:37.438 INFO:tasks.workunit.client.0.smithi057.stdout:8/373: chown d4/d13/c73 10 1 2024-02-28T23:07:37.440 INFO:tasks.workunit.client.1.smithi084.stdout:6/504: chown d1/d13/c82 282 1 2024-02-28T23:07:37.440 INFO:tasks.workunit.client.0.smithi057.stdout:7/367: dread d0/d5/d36/d4e/d15/d29/d3d/f6e [0,4194304] 0 2024-02-28T23:07:37.441 INFO:tasks.workunit.client.0.smithi057.stdout:4/386: creat d1a/d1b/d24/f89 x:0 0 0 2024-02-28T23:07:37.443 INFO:tasks.workunit.client.1.smithi084.stdout:5/573: chown dc/de/df/db0 16391624 1 2024-02-28T23:07:37.443 INFO:tasks.workunit.client.1.smithi084.stdout:2/458: dwrite d5/d7/d1d/f4d [0,4194304] 0 2024-02-28T23:07:37.443 INFO:tasks.workunit.client.0.smithi057.stdout:8/374: stat d4/d1e/d37/c4f 0 2024-02-28T23:07:37.444 INFO:tasks.workunit.client.0.smithi057.stdout:5/356: write d4/d21/d24/f22 [3382060,50071] 0 2024-02-28T23:07:37.447 INFO:tasks.workunit.client.0.smithi057.stdout:8/375: stat d4/d13/d2c 0 2024-02-28T23:07:37.447 INFO:tasks.workunit.client.1.smithi084.stdout:9/542: sync 2024-02-28T23:07:37.448 INFO:tasks.workunit.client.0.smithi057.stdout:4/387: read d1a/d1b/d2d/d49/f86 [891386,121856] 0 2024-02-28T23:07:37.449 INFO:tasks.workunit.client.0.smithi057.stdout:5/357: stat d4/d21/f37 0 2024-02-28T23:07:37.451 INFO:tasks.workunit.client.0.smithi057.stdout:7/368: write d0/d5/d36/f63 [736915,43243] 0 2024-02-28T23:07:37.451 INFO:tasks.workunit.client.0.smithi057.stdout:3/319: sync 2024-02-28T23:07:37.451 INFO:tasks.workunit.client.0.smithi057.stdout:1/298: sync 2024-02-28T23:07:37.451 INFO:tasks.workunit.client.0.smithi057.stdout:9/457: sync 2024-02-28T23:07:37.451 INFO:tasks.workunit.client.1.smithi084.stdout:4/462: write d1/d18/d20/d14/f36 [929122,9041] 0 2024-02-28T23:07:37.457 INFO:tasks.workunit.client.1.smithi084.stdout:1/581: symlink d12/d1b/d2e/d52/d61/lbf 0 2024-02-28T23:07:37.458 INFO:tasks.workunit.client.1.smithi084.stdout:3/492: truncate d3/f7 1125604 0 2024-02-28T23:07:37.459 INFO:tasks.workunit.client.0.smithi057.stdout:8/376: write d4/d13/d18/f31 [2034232,107986] 0 2024-02-28T23:07:37.460 INFO:tasks.workunit.client.0.smithi057.stdout:2/470: dread d8/d11/f17 [0,4194304] 0 2024-02-28T23:07:37.460 INFO:tasks.workunit.client.1.smithi084.stdout:2/459: creat d5/d7/d9/f88 x:0 0 0 2024-02-28T23:07:37.462 INFO:tasks.workunit.client.0.smithi057.stdout:7/369: fsync d0/d5/d36/d4e/d15/f49 0 2024-02-28T23:07:37.463 INFO:tasks.workunit.client.1.smithi084.stdout:1/582: chown d12/d1b/f4a 9 1 2024-02-28T23:07:37.464 INFO:tasks.workunit.client.0.smithi057.stdout:1/299: truncate d2/f8 4618493 0 2024-02-28T23:07:37.465 INFO:tasks.workunit.client.0.smithi057.stdout:8/377: dread - d4/d1e/d56/d6a/f6d zero size 2024-02-28T23:07:37.466 INFO:tasks.workunit.client.0.smithi057.stdout:4/388: creat d1a/d1b/f8a x:0 0 0 2024-02-28T23:07:37.467 INFO:tasks.workunit.client.0.smithi057.stdout:2/471: write d8/d11/d15/f5d [62699,2344] 0 2024-02-28T23:07:37.467 INFO:tasks.workunit.client.1.smithi084.stdout:4/463: unlink d1/d18/c4d 0 2024-02-28T23:07:37.468 INFO:tasks.workunit.client.0.smithi057.stdout:6/440: dwrite d0/d17/f2c [0,4194304] 0 2024-02-28T23:07:37.468 INFO:tasks.workunit.client.1.smithi084.stdout:9/543: chown d4/d6/d15/f2c 0 1 2024-02-28T23:07:37.470 INFO:tasks.workunit.client.0.smithi057.stdout:3/320: readlink d3/d4/l25 0 2024-02-28T23:07:37.470 INFO:tasks.workunit.client.0.smithi057.stdout:4/389: dread - d1a/d1b/f8a zero size 2024-02-28T23:07:37.471 INFO:tasks.workunit.client.0.smithi057.stdout:0/406: dread d17/d20/d4a/d78/f79 [0,4194304] 0 2024-02-28T23:07:37.472 INFO:tasks.workunit.client.0.smithi057.stdout:8/378: fsync d4/d1e/d56/d6a/d54/f70 0 2024-02-28T23:07:37.472 INFO:tasks.workunit.client.1.smithi084.stdout:6/505: dwrite d1/d13/d2e/d41/fa9 [0,4194304] 0 2024-02-28T23:07:37.472 INFO:tasks.workunit.client.0.smithi057.stdout:3/321: chown d3/d20/f24 110 1 2024-02-28T23:07:37.474 INFO:tasks.workunit.client.0.smithi057.stdout:2/472: readlink d8/d11/d21/d28/d41/l6b 0 2024-02-28T23:07:37.475 INFO:tasks.workunit.client.0.smithi057.stdout:4/390: dread d1a/d1b/f5c [0,4194304] 0 2024-02-28T23:07:37.476 INFO:tasks.workunit.client.0.smithi057.stdout:7/370: rmdir d0/d5/d25 39 2024-02-28T23:07:37.477 INFO:tasks.workunit.client.1.smithi084.stdout:5/574: link dc/de/l11 dc/de/df/d31/dd1/da4/lda 0 2024-02-28T23:07:37.480 INFO:tasks.workunit.client.0.smithi057.stdout:2/473: chown d8/d11/d21/d28/d41/f53 23 1 2024-02-28T23:07:37.480 INFO:tasks.workunit.client.0.smithi057.stdout:0/407: dread d17/d19/d4d/f62 [0,4194304] 0 2024-02-28T23:07:37.481 INFO:tasks.workunit.client.0.smithi057.stdout:1/300: mkdir d2/d7/df/db/d3a/d66 0 2024-02-28T23:07:37.481 INFO:tasks.workunit.client.1.smithi084.stdout:2/460: getdents d5/d7/d1d 0 2024-02-28T23:07:37.482 INFO:tasks.workunit.client.1.smithi084.stdout:3/493: truncate d3/d27/d7d/db0/d1a/d29/d41/f64 716859 0 2024-02-28T23:07:37.482 INFO:tasks.workunit.client.0.smithi057.stdout:8/379: fdatasync d4/d13/d1a/f20 0 2024-02-28T23:07:37.482 INFO:tasks.workunit.client.1.smithi084.stdout:4/464: write d1/f4 [2515274,15737] 0 2024-02-28T23:07:37.484 INFO:tasks.workunit.client.1.smithi084.stdout:8/503: dwrite d0/d9/d30/d33/d71/f75 [0,4194304] 0 2024-02-28T23:07:37.487 INFO:tasks.workunit.client.0.smithi057.stdout:0/408: unlink f14 0 2024-02-28T23:07:37.490 INFO:tasks.workunit.client.0.smithi057.stdout:1/301: symlink d2/d33/l67 0 2024-02-28T23:07:37.490 INFO:tasks.workunit.client.0.smithi057.stdout:0/409: fdatasync d17/d20/d23/f75 0 2024-02-28T23:07:37.490 INFO:tasks.workunit.client.1.smithi084.stdout:7/524: dread db/d5e/f6a [0,4194304] 0 2024-02-28T23:07:37.494 INFO:tasks.workunit.client.1.smithi084.stdout:1/583: creat d12/d84/d9a/fc0 x:0 0 0 2024-02-28T23:07:37.495 INFO:tasks.workunit.client.1.smithi084.stdout:3/494: write d3/d27/d7d/db0/d1a/d45/f78 [48004,106849] 0 2024-02-28T23:07:37.495 INFO:tasks.workunit.client.1.smithi084.stdout:8/504: read d0/d44/d9f/f7e [725715,71205] 0 2024-02-28T23:07:37.496 INFO:tasks.workunit.client.0.smithi057.stdout:8/380: symlink d4/d1e/d59/l85 0 2024-02-28T23:07:37.496 INFO:tasks.workunit.client.0.smithi057.stdout:3/322: symlink d3/d4/d1d/d47/d62/l65 0 2024-02-28T23:07:37.497 INFO:tasks.workunit.client.0.smithi057.stdout:6/441: link d0/d4/l88 d0/d1b/d3e/d40/l98 0 2024-02-28T23:07:37.498 INFO:tasks.workunit.client.0.smithi057.stdout:6/442: write d0/f6c [1159982,62745] 0 2024-02-28T23:07:37.498 INFO:tasks.workunit.client.1.smithi084.stdout:3/495: chown d3/d27/d7d/db0/d1a 61940044 1 2024-02-28T23:07:37.499 INFO:tasks.workunit.client.0.smithi057.stdout:1/302: mknod d2/d7/df/db/c68 0 2024-02-28T23:07:37.501 INFO:tasks.workunit.client.1.smithi084.stdout:6/506: creat d1/d9/d1d/d1e/d33/d5c/fb0 x:0 0 0 2024-02-28T23:07:37.501 INFO:tasks.workunit.client.1.smithi084.stdout:7/525: mkdir db/d10/d17/d9c 0 2024-02-28T23:07:37.503 INFO:tasks.workunit.client.1.smithi084.stdout:0/523: dread d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:37.503 INFO:tasks.workunit.client.1.smithi084.stdout:5/575: mkdir dc/ddb 0 2024-02-28T23:07:37.505 INFO:tasks.workunit.client.1.smithi084.stdout:4/465: mkdir d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d 0 2024-02-28T23:07:37.508 INFO:tasks.workunit.client.1.smithi084.stdout:6/507: symlink d1/d9/d1d/lb1 0 2024-02-28T23:07:37.509 INFO:tasks.workunit.client.0.smithi057.stdout:4/391: creat d1a/d48/d6c/f8b x:0 0 0 2024-02-28T23:07:37.509 INFO:tasks.workunit.client.0.smithi057.stdout:4/392: dread - d1a/d1b/d2d/f59 zero size 2024-02-28T23:07:37.510 INFO:tasks.workunit.client.0.smithi057.stdout:7/371: link d0/d5/d36/d4e/d15/d29/f70 d0/d5/d36/d4e/d15/d29/d3d/d51/f72 0 2024-02-28T23:07:37.510 INFO:tasks.workunit.client.0.smithi057.stdout:4/393: read - d1a/d1b/d2d/d49/d4d/f78 zero size 2024-02-28T23:07:37.510 INFO:tasks.workunit.client.0.smithi057.stdout:2/474: getdents d8/d11/d21/d70/d99 0 2024-02-28T23:07:37.510 INFO:tasks.workunit.client.0.smithi057.stdout:7/372: chown d0/d5/d36/d4e/c6d 53952 1 2024-02-28T23:07:37.510 INFO:tasks.workunit.client.1.smithi084.stdout:8/505: truncate d0/d24/d4d/f6a 4480631 0 2024-02-28T23:07:37.512 INFO:tasks.workunit.client.0.smithi057.stdout:2/475: readlink d8/d11/d48/l4d 0 2024-02-28T23:07:37.514 INFO:tasks.workunit.client.1.smithi084.stdout:9/544: getdents d4/d6/d15/d75/d9d 0 2024-02-28T23:07:37.515 INFO:tasks.workunit.client.0.smithi057.stdout:9/458: truncate dd/d1b/d40/f86 1768329 0 2024-02-28T23:07:37.517 INFO:tasks.workunit.client.1.smithi084.stdout:2/461: dwrite d5/dd/d69/f85 [0,4194304] 0 2024-02-28T23:07:37.518 INFO:tasks.workunit.client.1.smithi084.stdout:2/462: write d5/d1f/f41 [848058,99860] 0 2024-02-28T23:07:37.519 INFO:tasks.workunit.client.0.smithi057.stdout:5/358: dwrite d4/d9/f1c [0,4194304] 0 2024-02-28T23:07:37.520 INFO:tasks.workunit.client.0.smithi057.stdout:1/303: rename d2/d7/df/db/d12/d3f/f4b to d2/d7/df/db/f69 0 2024-02-28T23:07:37.524 INFO:tasks.workunit.client.1.smithi084.stdout:6/508: creat d1/d9/d1d/fb2 x:0 0 0 2024-02-28T23:07:37.527 INFO:tasks.workunit.client.1.smithi084.stdout:8/506: mknod d0/d9/d30/d33/d71/d84/ca9 0 2024-02-28T23:07:37.528 INFO:tasks.workunit.client.0.smithi057.stdout:8/381: creat d4/d13/d1a/d4a/d7b/f86 x:0 0 0 2024-02-28T23:07:37.530 INFO:tasks.workunit.client.0.smithi057.stdout:3/323: creat d3/d4/f66 x:0 0 0 2024-02-28T23:07:37.530 INFO:tasks.workunit.client.1.smithi084.stdout:0/524: rename d1/d8/d69/d6b/d85/ca0 to d1/d8/d18/d4c/d6c/d74/ca8 0 2024-02-28T23:07:37.530 INFO:tasks.workunit.client.0.smithi057.stdout:6/443: dwrite d0/d4/d11/f14 [0,4194304] 0 2024-02-28T23:07:37.533 INFO:tasks.workunit.client.0.smithi057.stdout:8/382: chown d4/d13/d18/c19 195992 1 2024-02-28T23:07:37.535 INFO:tasks.workunit.client.1.smithi084.stdout:5/576: mkdir dc/de/df/d31/dd1/d5f/ddc 0 2024-02-28T23:07:37.536 INFO:tasks.workunit.client.0.smithi057.stdout:5/359: fdatasync d4/d7/d8/f26 0 2024-02-28T23:07:37.536 INFO:tasks.workunit.client.0.smithi057.stdout:9/459: mkdir dd/d1b/d40/d44/d92 0 2024-02-28T23:07:37.540 INFO:tasks.workunit.client.0.smithi057.stdout:0/410: dwrite d17/d20/d26/d30/f39 [4194304,4194304] 0 2024-02-28T23:07:37.540 INFO:tasks.workunit.client.0.smithi057.stdout:4/394: rename d1a/d48/c70 to d1a/d1b/d2d/d49/c8c 0 2024-02-28T23:07:37.540 INFO:tasks.workunit.client.1.smithi084.stdout:9/545: readlink d4/d6/l1b 0 2024-02-28T23:07:37.541 INFO:tasks.workunit.client.1.smithi084.stdout:4/466: fsync d1/d18/d20/d24/f34 0 2024-02-28T23:07:37.541 INFO:tasks.workunit.client.0.smithi057.stdout:3/324: dread - d3/d8/d1b/f4a zero size 2024-02-28T23:07:37.545 INFO:tasks.workunit.client.0.smithi057.stdout:0/411: read - d17/d19/d4d/d7b/f94 zero size 2024-02-28T23:07:37.545 INFO:tasks.workunit.client.0.smithi057.stdout:5/360: write d4/d9/f1c [2297371,123137] 0 2024-02-28T23:07:37.545 INFO:tasks.workunit.client.0.smithi057.stdout:2/476: symlink d8/d11/d21/d70/d99/l9a 0 2024-02-28T23:07:37.547 INFO:tasks.workunit.client.0.smithi057.stdout:2/477: readlink d8/d11/d39/d7b/l98 0 2024-02-28T23:07:37.548 INFO:tasks.workunit.client.1.smithi084.stdout:0/525: rmdir d1/d6/db/d35/d57/d81 39 2024-02-28T23:07:37.549 INFO:tasks.workunit.client.0.smithi057.stdout:2/478: write d8/f19 [690510,118923] 0 2024-02-28T23:07:37.549 INFO:tasks.workunit.client.0.smithi057.stdout:6/444: rmdir d0/d4/d25 39 2024-02-28T23:07:37.550 INFO:tasks.workunit.client.1.smithi084.stdout:5/577: creat dc/de/df/da6/fdd x:0 0 0 2024-02-28T23:07:37.552 INFO:tasks.workunit.client.1.smithi084.stdout:1/584: link d12/d1b/d3b/f7f d12/d1b/d91/d9b/fc1 0 2024-02-28T23:07:37.554 INFO:tasks.workunit.client.0.smithi057.stdout:8/383: creat d4/d13/d2c/d4e/f87 x:0 0 0 2024-02-28T23:07:37.555 INFO:tasks.workunit.client.0.smithi057.stdout:1/304: creat d2/d7/d4f/d5c/f6a x:0 0 0 2024-02-28T23:07:37.557 INFO:tasks.workunit.client.1.smithi084.stdout:3/496: dwrite d3/f11 [4194304,4194304] 0 2024-02-28T23:07:37.557 INFO:tasks.workunit.client.0.smithi057.stdout:9/460: chown dd/d1b/d40/c45 1348336 1 2024-02-28T23:07:37.559 INFO:tasks.workunit.client.0.smithi057.stdout:3/325: rename d3/d20/c42 to d3/d4/d1d/d47/c67 0 2024-02-28T23:07:37.560 INFO:tasks.workunit.client.1.smithi084.stdout:1/585: chown d12/d1b/d3b/l99 13174 1 2024-02-28T23:07:37.561 INFO:tasks.workunit.client.0.smithi057.stdout:4/395: dread d1a/d1e/d4b/d4f/f53 [0,4194304] 0 2024-02-28T23:07:37.564 INFO:tasks.workunit.client.1.smithi084.stdout:7/526: dwrite db/d10/f2e [0,4194304] 0 2024-02-28T23:07:37.566 INFO:tasks.workunit.client.1.smithi084.stdout:8/507: rename d0/d9/d1b/c1c to d0/caa 0 2024-02-28T23:07:37.567 INFO:tasks.workunit.client.0.smithi057.stdout:7/373: link d0/d5/d36/d4e/d15/d29/c2c d0/d5/d36/d4e/d15/d29/c73 0 2024-02-28T23:07:37.567 INFO:tasks.workunit.client.0.smithi057.stdout:9/461: chown dd/de/d24/d63/l7d 15655923 1 2024-02-28T23:07:37.569 INFO:tasks.workunit.client.1.smithi084.stdout:4/467: mknod d1/d18/d20/d24/d2f/d3e/d52/d87/c9e 0 2024-02-28T23:07:37.570 INFO:tasks.workunit.client.0.smithi057.stdout:0/412: mknod d17/d20/d26/d30/d6b/d71/c99 0 2024-02-28T23:07:37.571 INFO:tasks.workunit.client.1.smithi084.stdout:4/468: chown d1/d4b/l5d 788 1 2024-02-28T23:07:37.571 INFO:tasks.workunit.client.0.smithi057.stdout:2/479: mkdir d8/d11/d21/d28/d49/d85/d9b 0 2024-02-28T23:07:37.572 INFO:tasks.workunit.client.1.smithi084.stdout:0/526: mkdir d1/d6/db/d35/d44/da9 0 2024-02-28T23:07:37.573 INFO:tasks.workunit.client.0.smithi057.stdout:4/396: truncate d1a/d1b/d2d/d49/f86 5039841 0 2024-02-28T23:07:37.574 INFO:tasks.workunit.client.0.smithi057.stdout:8/384: mkdir d4/d1e/d56/d88 0 2024-02-28T23:07:37.575 INFO:tasks.workunit.client.1.smithi084.stdout:2/463: getdents d5 0 2024-02-28T23:07:37.575 INFO:tasks.workunit.client.1.smithi084.stdout:1/586: symlink d12/d1b/d63/d7d/lc2 0 2024-02-28T23:07:37.576 INFO:tasks.workunit.client.0.smithi057.stdout:5/361: mknod d4/d7/c70 0 2024-02-28T23:07:37.578 INFO:tasks.workunit.client.0.smithi057.stdout:9/462: creat dd/de/d14/d48/d4b/d41/f93 x:0 0 0 2024-02-28T23:07:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:37 smithi084 ceph-mon[34826]: pgmap v25: 65 pgs: 65 active+clean; 1.6 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 215 MiB/s wr, 342 op/s 2024-02-28T23:07:37.581 INFO:tasks.workunit.client.1.smithi084.stdout:5/578: mkdir dc/dde 0 2024-02-28T23:07:37.581 INFO:tasks.workunit.client.1.smithi084.stdout:7/527: creat db/d5e/f9d x:0 0 0 2024-02-28T23:07:37.581 INFO:tasks.workunit.client.0.smithi057.stdout:5/362: fdatasync d4/d7/d8/ff 0 2024-02-28T23:07:37.582 INFO:tasks.workunit.client.0.smithi057.stdout:8/385: creat d4/d13/d1a/f89 x:0 0 0 2024-02-28T23:07:37.583 INFO:tasks.workunit.client.1.smithi084.stdout:8/508: creat d0/d9/d1b/d38/d43/d51/d54/fab x:0 0 0 2024-02-28T23:07:37.584 INFO:tasks.workunit.client.0.smithi057.stdout:9/463: readlink dd/d1b/d40/d44/d5c/l65 0 2024-02-28T23:07:37.587 INFO:tasks.workunit.client.0.smithi057.stdout:5/363: chown d4/d21/l23 4083 1 2024-02-28T23:07:37.587 INFO:tasks.workunit.client.0.smithi057.stdout:7/374: rename d0/d5/l6 to d0/l74 0 2024-02-28T23:07:37.588 INFO:tasks.workunit.client.0.smithi057.stdout:3/326: dwrite d3/d20/f24 [0,4194304] 0 2024-02-28T23:07:37.588 INFO:tasks.workunit.client.1.smithi084.stdout:8/509: write d0/d9/d30/d33/d71/fa4 [59895,76203] 0 2024-02-28T23:07:37.588 INFO:tasks.workunit.client.0.smithi057.stdout:0/413: read d17/d20/f2b [144035,46282] 0 2024-02-28T23:07:37.589 INFO:tasks.workunit.client.1.smithi084.stdout:9/546: truncate d4/f90 3752435 0 2024-02-28T23:07:37.590 INFO:tasks.workunit.client.0.smithi057.stdout:8/386: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:37.592 INFO:tasks.workunit.client.1.smithi084.stdout:5/579: read dc/de/df/d31/f43 [3362728,86821] 0 2024-02-28T23:07:37.592 INFO:tasks.workunit.client.0.smithi057.stdout:2/480: rename d8/d11/f1c to d8/d11/d21/d70/d91/d94/f9c 0 2024-02-28T23:07:37.592 INFO:tasks.workunit.client.1.smithi084.stdout:1/587: fdatasync d12/d1b/d2e/d52/d5b/fb3 0 2024-02-28T23:07:37.593 INFO:tasks.workunit.client.0.smithi057.stdout:3/327: write d3/d4/d1d/f1e [4908773,128370] 0 2024-02-28T23:07:37.594 INFO:tasks.workunit.client.0.smithi057.stdout:8/387: chown d4/f9 12185961 1 2024-02-28T23:07:37.594 INFO:tasks.workunit.client.0.smithi057.stdout:5/364: rmdir d4/d32 39 2024-02-28T23:07:37.595 INFO:tasks.workunit.client.1.smithi084.stdout:5/580: fdatasync dc/de/df/d16/f81 0 2024-02-28T23:07:37.596 INFO:tasks.workunit.client.0.smithi057.stdout:8/388: fdatasync d4/d13/d1a/f34 0 2024-02-28T23:07:37.598 INFO:tasks.workunit.client.0.smithi057.stdout:0/414: symlink d17/d7a/l9a 0 2024-02-28T23:07:37.602 INFO:tasks.workunit.client.0.smithi057.stdout:5/365: rename d4/d7/d8/d5c to d4/d9/d14/d3b/d61/d71 0 2024-02-28T23:07:37.602 INFO:tasks.workunit.client.1.smithi084.stdout:1/588: chown d12/d1b/d2e/d32/f47 312566904 1 2024-02-28T23:07:37.603 INFO:tasks.workunit.client.0.smithi057.stdout:0/415: chown d17/d20/d26/f69 6 1 2024-02-28T23:07:37.605 INFO:tasks.workunit.client.1.smithi084.stdout:0/527: dwrite d1/d8/d18/d4c/d6c/d74/f91 [0,4194304] 0 2024-02-28T23:07:37.605 INFO:tasks.workunit.client.0.smithi057.stdout:8/389: creat d4/d13/d1a/d4a/d7b/f8a x:0 0 0 2024-02-28T23:07:37.605 INFO:tasks.workunit.client.1.smithi084.stdout:1/589: write d12/f40 [1779426,103580] 0 2024-02-28T23:07:37.608 INFO:tasks.workunit.client.1.smithi084.stdout:0/528: readlink d1/d6/d59/l7c 0 2024-02-28T23:07:37.609 INFO:tasks.workunit.client.0.smithi057.stdout:8/390: write d4/d13/f2d [175451,53770] 0 2024-02-28T23:07:37.612 INFO:tasks.workunit.client.1.smithi084.stdout:3/497: getdents d3/d27/d7d/db0/d14/d31 0 2024-02-28T23:07:37.613 INFO:tasks.workunit.client.0.smithi057.stdout:0/416: fdatasync d17/d20/d23/d37/f43 0 2024-02-28T23:07:37.614 INFO:tasks.workunit.client.1.smithi084.stdout:7/528: link db/d5e/d80/f97 db/d5e/d8a/f9e 0 2024-02-28T23:07:37.617 INFO:tasks.workunit.client.1.smithi084.stdout:8/510: dread d0/d9/d30/d33/f5e [0,4194304] 0 2024-02-28T23:07:37.619 INFO:tasks.workunit.client.1.smithi084.stdout:0/529: truncate d1/d6/db/d37/d72/f99 169897 0 2024-02-28T23:07:37.619 INFO:tasks.workunit.client.0.smithi057.stdout:8/391: mkdir d4/d13/d18/d62/d66/d74/d8b 0 2024-02-28T23:07:37.621 INFO:tasks.workunit.client.0.smithi057.stdout:7/375: dwrite d0/d5/f55 [0,4194304] 0 2024-02-28T23:07:37.621 INFO:tasks.workunit.client.1.smithi084.stdout:2/464: dwrite d5/f17 [0,4194304] 0 2024-02-28T23:07:37.624 INFO:tasks.workunit.client.0.smithi057.stdout:4/397: dread d1a/d1b/f32 [0,4194304] 0 2024-02-28T23:07:37.625 INFO:tasks.workunit.client.0.smithi057.stdout:4/398: readlink d1a/d1e/l22 0 2024-02-28T23:07:37.626 INFO:tasks.workunit.client.0.smithi057.stdout:0/417: rename d17/d20/d4a/d78/c95 to d17/c9b 0 2024-02-28T23:07:37.627 INFO:tasks.workunit.client.1.smithi084.stdout:9/547: creat d4/d69/fa8 x:0 0 0 2024-02-28T23:07:37.627 INFO:tasks.workunit.client.0.smithi057.stdout:8/392: mkdir d4/d1e/d56/d6a/d54/d61/d8c 0 2024-02-28T23:07:37.627 INFO:tasks.workunit.client.0.smithi057.stdout:0/418: readlink d17/d20/d23/l87 0 2024-02-28T23:07:37.629 INFO:tasks.workunit.client.1.smithi084.stdout:2/465: truncate d5/d7/d12/d24/f5e 162428 0 2024-02-28T23:07:37.630 INFO:tasks.workunit.client.0.smithi057.stdout:8/393: write d4/d13/d18/f23 [3615346,102855] 0 2024-02-28T23:07:37.630 INFO:tasks.workunit.client.1.smithi084.stdout:5/581: dread dc/de/df/d31/dd1/f80 [0,4194304] 0 2024-02-28T23:07:37.631 INFO:tasks.workunit.client.0.smithi057.stdout:1/305: truncate d2/d7/df/db/d12/d19/d24/d2d/f31 1497034 0 2024-02-28T23:07:37.631 INFO:tasks.workunit.client.0.smithi057.stdout:5/366: truncate d4/d7/d8/ff 564726 0 2024-02-28T23:07:37.632 INFO:tasks.workunit.client.0.smithi057.stdout:0/419: creat d17/f9c x:0 0 0 2024-02-28T23:07:37.632 INFO:tasks.workunit.client.0.smithi057.stdout:7/376: read d0/d5/d36/d4e/d15/d29/d3d/d51/f64 [13225,46814] 0 2024-02-28T23:07:37.635 INFO:tasks.workunit.client.1.smithi084.stdout:1/590: mknod d12/d1b/d91/d9b/cc3 0 2024-02-28T23:07:37.635 INFO:tasks.workunit.client.0.smithi057.stdout:7/377: write d0/f10 [1514197,36466] 0 2024-02-28T23:07:37.636 INFO:tasks.workunit.client.1.smithi084.stdout:2/466: read d5/d7/d12/d24/d3a/d48/f49 [137920,3129] 0 2024-02-28T23:07:37.639 INFO:tasks.workunit.client.0.smithi057.stdout:8/394: unlink d4/d13/d18/f27 0 2024-02-28T23:07:37.640 INFO:tasks.workunit.client.0.smithi057.stdout:0/420: unlink d17/d20/d23/d47/c82 0 2024-02-28T23:07:37.641 INFO:tasks.workunit.client.0.smithi057.stdout:3/328: write d3/f61 [101116,48472] 0 2024-02-28T23:07:37.644 INFO:tasks.workunit.client.0.smithi057.stdout:2/481: chown d8/d11/d21/f7d 4 1 2024-02-28T23:07:37.645 INFO:tasks.workunit.client.1.smithi084.stdout:0/530: symlink d1/d6/db/d35/d4a/laa 0 2024-02-28T23:07:37.645 INFO:tasks.workunit.client.1.smithi084.stdout:8/511: rename d0/d9/d1b/f7f to d0/d24/d9e/fac 0 2024-02-28T23:07:37.647 INFO:tasks.workunit.client.1.smithi084.stdout:3/498: dwrite d3/fc [0,4194304] 0 2024-02-28T23:07:37.648 INFO:tasks.workunit.client.0.smithi057.stdout:1/306: unlink d2/d7/df/db/f17 0 2024-02-28T23:07:37.649 INFO:tasks.workunit.client.0.smithi057.stdout:7/378: mkdir d0/d5/d36/d4e/d3b/d75 0 2024-02-28T23:07:37.650 INFO:tasks.workunit.client.1.smithi084.stdout:5/582: creat dc/de/d9e/fdf x:0 0 0 2024-02-28T23:07:37.651 INFO:tasks.workunit.client.0.smithi057.stdout:0/421: mkdir d17/d20/d4a/d78/d9d 0 2024-02-28T23:07:37.655 INFO:tasks.workunit.client.0.smithi057.stdout:7/379: read - d0/d5/d36/d4e/d15/d29/f70 zero size 2024-02-28T23:07:37.655 INFO:tasks.workunit.client.1.smithi084.stdout:7/529: truncate db/d10/f3e 1782808 0 2024-02-28T23:07:37.657 INFO:tasks.workunit.client.1.smithi084.stdout:7/530: chown db/d5e/d80 2605 1 2024-02-28T23:07:37.658 INFO:tasks.workunit.client.1.smithi084.stdout:4/469: dwrite d1/d18/d20/d24/f8b [0,4194304] 0 2024-02-28T23:07:37.658 INFO:tasks.workunit.client.0.smithi057.stdout:4/399: dwrite fe [4194304,4194304] 0 2024-02-28T23:07:37.658 INFO:tasks.workunit.client.0.smithi057.stdout:7/380: chown d0/d5/d36/d4e/d15/d29 87136004 1 2024-02-28T23:07:37.660 INFO:tasks.workunit.client.1.smithi084.stdout:7/531: write db/d11/f5b [962843,85668] 0 2024-02-28T23:07:37.660 INFO:tasks.workunit.client.0.smithi057.stdout:4/400: readlink d1a/d1b/d24/d2e/l30 0 2024-02-28T23:07:37.664 INFO:tasks.workunit.client.0.smithi057.stdout:3/329: symlink d3/d4/d1d/l68 0 2024-02-28T23:07:37.664 INFO:tasks.workunit.client.0.smithi057.stdout:4/401: read d1a/d1e/d4b/f5d [293615,110511] 0 2024-02-28T23:07:37.667 INFO:tasks.workunit.client.1.smithi084.stdout:2/467: link d5/dd/d69/f85 d5/d7/d2e/d4a/d5c/f89 0 2024-02-28T23:07:37.668 INFO:tasks.workunit.client.0.smithi057.stdout:8/395: mknod d4/d1e/d56/d6a/c8d 0 2024-02-28T23:07:37.670 INFO:tasks.workunit.client.0.smithi057.stdout:4/402: chown d1a/d1b/d24/d2e/f4a 33033079 1 2024-02-28T23:07:37.671 INFO:tasks.workunit.client.1.smithi084.stdout:1/591: dwrite d12/d1b/d2e/d52/f7c [0,4194304] 0 2024-02-28T23:07:37.676 INFO:tasks.workunit.client.0.smithi057.stdout:0/422: symlink d17/d20/d4a/l9e 0 2024-02-28T23:07:37.686 INFO:tasks.workunit.client.1.smithi084.stdout:4/470: fdatasync d1/d18/d20/d24/d2f/d3e/d52/f8c 0 2024-02-28T23:07:37.687 INFO:tasks.workunit.client.1.smithi084.stdout:6/509: sync 2024-02-28T23:07:37.687 INFO:tasks.workunit.client.1.smithi084.stdout:9/548: getdents d4/d6/d15/d75 0 2024-02-28T23:07:37.687 INFO:tasks.workunit.client.1.smithi084.stdout:2/468: write d5/d7/d12/d24/d2a/d35/f40 [913046,110989] 0 2024-02-28T23:07:37.687 INFO:tasks.workunit.client.1.smithi084.stdout:7/532: mkdir db/d10/d17/d9c/d9f 0 2024-02-28T23:07:37.687 INFO:tasks.workunit.client.1.smithi084.stdout:2/469: chown d5/d1f/d83 25499 1 2024-02-28T23:07:37.688 INFO:tasks.workunit.client.0.smithi057.stdout:7/381: write d0/d5/d36/d4e/d15/d29/d3d/f6e [751011,82775] 0 2024-02-28T23:07:37.689 INFO:tasks.workunit.client.1.smithi084.stdout:3/499: rename d3/d27/d7d/db0/df/f46 to d3/fbd 0 2024-02-28T23:07:37.691 INFO:tasks.workunit.client.1.smithi084.stdout:7/533: mknod db/d10/d17/ca0 0 2024-02-28T23:07:37.691 INFO:tasks.workunit.client.1.smithi084.stdout:9/549: write d4/d6/d15/d12/f81 [1193039,128665] 0 2024-02-28T23:07:37.695 INFO:tasks.workunit.client.1.smithi084.stdout:9/550: write d4/d6/d15/d12/d41/f65 [19913,42616] 0 2024-02-28T23:07:37.696 INFO:tasks.workunit.client.1.smithi084.stdout:3/500: stat d3/d27/d7d/db0/d1a/d29/d7e/fa7 0 2024-02-28T23:07:37.700 INFO:tasks.workunit.client.0.smithi057.stdout:3/330: dwrite d3/d4/d1d/f1e [0,4194304] 0 2024-02-28T23:07:37.702 INFO:tasks.workunit.client.1.smithi084.stdout:4/471: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f9f x:0 0 0 2024-02-28T23:07:37.703 INFO:tasks.workunit.client.1.smithi084.stdout:4/472: readlink l0 0 2024-02-28T23:07:37.704 INFO:tasks.workunit.client.0.smithi057.stdout:2/482: dread d8/d11/d21/f36 [0,4194304] 0 2024-02-28T23:07:37.704 INFO:tasks.workunit.client.1.smithi084.stdout:0/531: rename d1/d6/db/d35/d57/f6a to d1/d8/d69/d6b/d85/fab 0 2024-02-28T23:07:37.706 INFO:tasks.workunit.client.1.smithi084.stdout:0/532: chown d1/d8/l51 203084390 1 2024-02-28T23:07:37.709 INFO:tasks.workunit.client.0.smithi057.stdout:4/403: readlink d1a/d1e/d4b/l6d 0 2024-02-28T23:07:37.709 INFO:tasks.workunit.client.0.smithi057.stdout:0/423: creat d17/d20/d23/d47/f9f x:0 0 0 2024-02-28T23:07:37.710 INFO:tasks.workunit.client.0.smithi057.stdout:5/367: write d4/d9/d16/f1a [1384892,84849] 0 2024-02-28T23:07:37.710 INFO:tasks.workunit.client.0.smithi057.stdout:0/424: stat d17/l2d 0 2024-02-28T23:07:37.710 INFO:tasks.workunit.client.0.smithi057.stdout:3/331: mkdir d3/d4/d1d/d47/d69 0 2024-02-28T23:07:37.710 INFO:tasks.workunit.client.0.smithi057.stdout:2/483: creat d8/d11/d56/f9d x:0 0 0 2024-02-28T23:07:37.714 INFO:tasks.workunit.client.1.smithi084.stdout:3/501: mknod d3/d27/cbe 0 2024-02-28T23:07:37.717 INFO:tasks.workunit.client.1.smithi084.stdout:9/551: mknod d4/d6/d15/ca9 0 2024-02-28T23:07:37.718 INFO:tasks.workunit.client.0.smithi057.stdout:5/368: write d4/d32/f50 [987324,78182] 0 2024-02-28T23:07:37.721 INFO:tasks.workunit.client.1.smithi084.stdout:0/533: write d1/d6/db/d37/d72/f7d [923654,772] 0 2024-02-28T23:07:37.722 INFO:tasks.workunit.client.0.smithi057.stdout:5/369: write d4/d9/f1c [4473668,98563] 0 2024-02-28T23:07:37.722 INFO:tasks.workunit.client.1.smithi084.stdout:6/510: getdents d1/d9/d1d/d1e 0 2024-02-28T23:07:37.724 INFO:tasks.workunit.client.0.smithi057.stdout:2/484: truncate d8/d11/d39/f3c 1067846 0 2024-02-28T23:07:37.730 INFO:tasks.workunit.client.1.smithi084.stdout:8/512: dwrite d0/d11/f3c [0,4194304] 0 2024-02-28T23:07:37.731 INFO:tasks.workunit.client.1.smithi084.stdout:2/470: rename d5/d7/c4b to d5/d7/d12/c8a 0 2024-02-28T23:07:37.734 INFO:tasks.workunit.client.1.smithi084.stdout:4/473: dread d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [0,4194304] 0 2024-02-28T23:07:37.735 INFO:tasks.workunit.client.0.smithi057.stdout:5/370: mkdir d4/d21/d24/d72 0 2024-02-28T23:07:37.735 INFO:tasks.workunit.client.1.smithi084.stdout:5/583: dread dc/de/df/d16/f95 [0,4194304] 0 2024-02-28T23:07:37.736 INFO:tasks.workunit.client.1.smithi084.stdout:3/502: readlink d3/d27/d7d/db0/d1a/l59 0 2024-02-28T23:07:37.738 INFO:tasks.workunit.client.0.smithi057.stdout:1/307: write d2/d7/df/db/f18 [1596586,85384] 0 2024-02-28T23:07:37.742 INFO:tasks.workunit.client.0.smithi057.stdout:0/425: dwrite d17/d20/d23/d47/f9f [0,4194304] 0 2024-02-28T23:07:37.742 INFO:tasks.workunit.client.0.smithi057.stdout:7/382: write d0/d5/d36/d4e/d15/f60 [1102303,56883] 0 2024-02-28T23:07:37.744 INFO:tasks.workunit.client.0.smithi057.stdout:5/371: stat d4/d32/l3e 0 2024-02-28T23:07:37.744 INFO:tasks.workunit.client.1.smithi084.stdout:0/534: write d1/d8/d78/fa4 [2073758,87274] 0 2024-02-28T23:07:37.745 INFO:tasks.workunit.client.0.smithi057.stdout:8/396: dread d4/f9 [4194304,4194304] 0 2024-02-28T23:07:37.746 INFO:tasks.workunit.client.1.smithi084.stdout:0/535: chown d1/d8/d18/d28/c3a 3 1 2024-02-28T23:07:37.749 INFO:tasks.workunit.client.0.smithi057.stdout:0/426: fsync d17/d20/d23/d47/f5b 0 2024-02-28T23:07:37.750 INFO:tasks.workunit.client.0.smithi057.stdout:4/404: dwrite d1a/d48/d6c/f8b [0,4194304] 0 2024-02-28T23:07:37.753 INFO:tasks.workunit.client.1.smithi084.stdout:7/534: dread db/d10/d18/f1b [0,4194304] 0 2024-02-28T23:07:37.755 INFO:tasks.workunit.client.0.smithi057.stdout:8/397: write d4/d13/d2c/d4e/f60 [426129,67650] 0 2024-02-28T23:07:37.756 INFO:tasks.workunit.client.1.smithi084.stdout:8/513: creat d0/d11/fad x:0 0 0 2024-02-28T23:07:37.756 INFO:tasks.workunit.client.1.smithi084.stdout:2/471: creat d5/d7/d12/d24/d3a/d48/f8b x:0 0 0 2024-02-28T23:07:37.757 INFO:tasks.workunit.client.0.smithi057.stdout:0/427: write f10 [342517,124177] 0 2024-02-28T23:07:37.757 INFO:tasks.workunit.client.0.smithi057.stdout:5/372: creat d4/d7/f73 x:0 0 0 2024-02-28T23:07:37.758 INFO:tasks.workunit.client.1.smithi084.stdout:5/584: creat dc/de/d83/fe0 x:0 0 0 2024-02-28T23:07:37.761 INFO:tasks.workunit.client.0.smithi057.stdout:1/308: getdents d2/d7/df/db/d3a/d2a 0 2024-02-28T23:07:37.763 INFO:tasks.workunit.client.0.smithi057.stdout:0/428: rmdir d17/d19 39 2024-02-28T23:07:37.764 INFO:tasks.workunit.client.1.smithi084.stdout:3/503: unlink d3/d27/d7d/db0/d1a/d29/f38 0 2024-02-28T23:07:37.765 INFO:tasks.workunit.client.0.smithi057.stdout:2/485: dwrite d8/d11/d39/d7b/d8b/f96 [0,4194304] 0 2024-02-28T23:07:37.766 INFO:tasks.workunit.client.0.smithi057.stdout:8/398: write d4/f3a [80637,126202] 0 2024-02-28T23:07:37.768 INFO:tasks.workunit.client.0.smithi057.stdout:1/309: mkdir d2/d7/d4f/d5c/d6b 0 2024-02-28T23:07:37.774 INFO:tasks.workunit.client.1.smithi084.stdout:9/552: mkdir d4/d6/d15/d75/d2d/daa 0 2024-02-28T23:07:37.775 INFO:tasks.workunit.client.1.smithi084.stdout:7/535: mknod db/d5e/ca1 0 2024-02-28T23:07:37.776 INFO:tasks.workunit.client.1.smithi084.stdout:8/514: mkdir d0/d9/d1b/d38/d43/dae 0 2024-02-28T23:07:37.776 INFO:tasks.workunit.client.1.smithi084.stdout:4/474: creat d1/d18/d20/d24/d2f/d3e/d52/d57/fa0 x:0 0 0 2024-02-28T23:07:37.777 INFO:tasks.workunit.client.0.smithi057.stdout:8/399: symlink d4/d13/d18/d62/l8e 0 2024-02-28T23:07:37.777 INFO:tasks.workunit.client.0.smithi057.stdout:1/310: creat d2/d7/df/db/d3a/f6c x:0 0 0 2024-02-28T23:07:37.777 INFO:tasks.workunit.client.1.smithi084.stdout:2/472: rename d5/d1f/d42/l61 to d5/d1f/d83/l8c 0 2024-02-28T23:07:37.783 INFO:tasks.workunit.client.0.smithi057.stdout:2/486: rename d8/f42 to d8/d11/d21/d28/d41/f9e 0 2024-02-28T23:07:37.784 INFO:tasks.workunit.client.1.smithi084.stdout:3/504: fdatasync d3/d27/d7d/db0/d1a/d21/d3d/f52 0 2024-02-28T23:07:37.784 INFO:tasks.workunit.client.1.smithi084.stdout:9/553: mknod d4/d32/d3f/d89/cab 0 2024-02-28T23:07:37.786 INFO:tasks.workunit.client.1.smithi084.stdout:8/515: rmdir d0/d9/d30/d33/d71/d84 39 2024-02-28T23:07:37.791 INFO:tasks.workunit.client.1.smithi084.stdout:9/554: mknod d4/d6/d15/d75/cac 0 2024-02-28T23:07:37.793 INFO:tasks.workunit.client.0.smithi057.stdout:0/429: dread d17/f18 [0,4194304] 0 2024-02-28T23:07:37.793 INFO:tasks.workunit.client.1.smithi084.stdout:4/475: truncate d1/d18/d20/d24/d2f/d3e/d52/d45/f93 306915 0 2024-02-28T23:07:37.794 INFO:tasks.workunit.client.0.smithi057.stdout:1/311: getdents d2/d7/df/db/d12/d19/d65 0 2024-02-28T23:07:37.795 INFO:tasks.workunit.client.0.smithi057.stdout:5/373: dwrite d4/d32/f6e [0,4194304] 0 2024-02-28T23:07:37.797 INFO:tasks.workunit.client.1.smithi084.stdout:3/505: mkdir d3/d27/d7d/db0/d1a/d21/d7b/dbf 0 2024-02-28T23:07:37.809 INFO:tasks.workunit.client.0.smithi057.stdout:5/374: symlink d4/d9/d14/d3b/l74 0 2024-02-28T23:07:37.810 INFO:tasks.workunit.client.1.smithi084.stdout:6/511: truncate d1/d9/f48 3940122 0 2024-02-28T23:07:37.812 INFO:tasks.workunit.client.1.smithi084.stdout:7/536: dread db/d11/d27/f33 [0,4194304] 0 2024-02-28T23:07:37.813 INFO:tasks.workunit.client.0.smithi057.stdout:8/400: dread d4/d1e/f36 [0,4194304] 0 2024-02-28T23:07:37.815 INFO:tasks.workunit.client.0.smithi057.stdout:2/487: dwrite d8/d11/d21/d28/f3d [0,4194304] 0 2024-02-28T23:07:37.815 INFO:tasks.workunit.client.1.smithi084.stdout:7/537: readlink db/l63 0 2024-02-28T23:07:37.816 INFO:tasks.workunit.client.1.smithi084.stdout:7/538: chown db/d11/d27/f3a 80 1 2024-02-28T23:07:37.819 INFO:tasks.workunit.client.1.smithi084.stdout:5/585: dread dc/f6b [4194304,4194304] 0 2024-02-28T23:07:37.821 INFO:tasks.workunit.client.1.smithi084.stdout:5/586: fsync dc/de/df/da6/fdd 0 2024-02-28T23:07:37.822 INFO:tasks.workunit.client.0.smithi057.stdout:8/401: creat d4/d13/d1a/f8f x:0 0 0 2024-02-28T23:07:37.822 INFO:tasks.workunit.client.1.smithi084.stdout:2/473: write d5/d7/d9/f3c [1107798,99088] 0 2024-02-28T23:07:37.824 INFO:tasks.workunit.client.1.smithi084.stdout:3/506: creat d3/fc0 x:0 0 0 2024-02-28T23:07:37.825 INFO:tasks.workunit.client.1.smithi084.stdout:8/516: getdents d0/d9/d1b/d38/d43/d51 0 2024-02-28T23:07:37.825 INFO:tasks.workunit.client.1.smithi084.stdout:2/474: stat d5/d64/l6a 0 2024-02-28T23:07:37.825 INFO:tasks.workunit.client.1.smithi084.stdout:9/555: creat d4/d6/d15/d12/fad x:0 0 0 2024-02-28T23:07:37.828 INFO:tasks.workunit.client.0.smithi057.stdout:2/488: creat d8/d11/d21/d28/d49/d85/f9f x:0 0 0 2024-02-28T23:07:37.828 INFO:tasks.workunit.client.1.smithi084.stdout:7/539: creat db/d5e/d87/fa2 x:0 0 0 2024-02-28T23:07:37.828 INFO:tasks.workunit.client.1.smithi084.stdout:2/475: fsync d5/dd/f1c 0 2024-02-28T23:07:37.828 INFO:tasks.workunit.client.1.smithi084.stdout:4/476: link d1/d18/d20/d65/d8e/d55/f1a d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fa1 0 2024-02-28T23:07:37.829 INFO:tasks.workunit.client.1.smithi084.stdout:6/512: creat d1/d9/d1d/d1e/d33/d38/da0/fb3 x:0 0 0 2024-02-28T23:07:37.829 INFO:tasks.workunit.client.0.smithi057.stdout:8/402: creat d4/d13/d18/f90 x:0 0 0 2024-02-28T23:07:37.831 INFO:tasks.workunit.client.0.smithi057.stdout:0/430: dwrite d17/d20/d26/f67 [0,4194304] 0 2024-02-28T23:07:37.831 INFO:tasks.workunit.client.0.smithi057.stdout:1/312: dwrite d2/d7/df/db/f18 [0,4194304] 0 2024-02-28T23:07:37.832 INFO:tasks.workunit.client.1.smithi084.stdout:5/587: mkdir dc/de/df/d16/d34/daa/dae/de1 0 2024-02-28T23:07:37.832 INFO:tasks.workunit.client.1.smithi084.stdout:4/477: chown d1/c85 1035515 1 2024-02-28T23:07:37.833 INFO:tasks.workunit.client.1.smithi084.stdout:2/476: chown d5/d7/d12/d24/d2a/d35/f5d 16234 1 2024-02-28T23:07:37.834 INFO:tasks.workunit.client.0.smithi057.stdout:8/403: chown d4/f11 2 1 2024-02-28T23:07:37.834 INFO:tasks.workunit.client.0.smithi057.stdout:0/431: read - d17/f83 zero size 2024-02-28T23:07:37.835 INFO:tasks.workunit.client.0.smithi057.stdout:2/489: chown d8/d11/d39/c65 12895 1 2024-02-28T23:07:37.839 INFO:tasks.workunit.client.1.smithi084.stdout:9/556: write d4/d32/f92 [44159,53156] 0 2024-02-28T23:07:37.843 INFO:tasks.workunit.client.0.smithi057.stdout:8/404: creat d4/d1e/d59/f91 x:0 0 0 2024-02-28T23:07:37.846 INFO:tasks.workunit.client.0.smithi057.stdout:0/432: creat d17/d20/d23/d47/d60/fa0 x:0 0 0 2024-02-28T23:07:37.846 INFO:tasks.workunit.client.0.smithi057.stdout:8/405: stat d4/d13/d18/f90 0 2024-02-28T23:07:37.846 INFO:tasks.workunit.client.0.smithi057.stdout:1/313: fsync d2/d7/df/db/d12/d19/d24/d2d/f49 0 2024-02-28T23:07:37.847 INFO:tasks.workunit.client.0.smithi057.stdout:5/375: dread d4/d21/d24/f51 [0,4194304] 0 2024-02-28T23:07:37.847 INFO:tasks.workunit.client.0.smithi057.stdout:8/406: fdatasync d4/f9 0 2024-02-28T23:07:37.849 INFO:tasks.workunit.client.0.smithi057.stdout:5/376: fsync d4/d32/f6e 0 2024-02-28T23:07:37.853 INFO:tasks.workunit.client.1.smithi084.stdout:9/557: dread - d4/d6/d15/d12/f97 zero size 2024-02-28T23:07:37.856 INFO:tasks.workunit.client.0.smithi057.stdout:1/314: creat d2/d7/df/db/d3a/d2a/f6d x:0 0 0 2024-02-28T23:07:37.857 INFO:tasks.workunit.client.0.smithi057.stdout:5/377: mknod d4/d9/d16/c75 0 2024-02-28T23:07:37.859 INFO:tasks.workunit.client.1.smithi084.stdout:7/540: dwrite db/d11/f32 [0,4194304] 0 2024-02-28T23:07:37.860 INFO:tasks.workunit.client.0.smithi057.stdout:5/378: read d4/d9/d16/f18 [7277031,6435] 0 2024-02-28T23:07:37.861 INFO:tasks.workunit.client.1.smithi084.stdout:3/507: dread d3/fc [0,4194304] 0 2024-02-28T23:07:37.862 INFO:tasks.workunit.client.0.smithi057.stdout:5/379: write d4/d9/d16/f19 [230021,123727] 0 2024-02-28T23:07:37.863 INFO:tasks.workunit.client.0.smithi057.stdout:8/407: dwrite d4/d1e/d59/f91 [0,4194304] 0 2024-02-28T23:07:37.865 INFO:tasks.workunit.client.0.smithi057.stdout:1/315: creat d2/d7/df/db/d12/d3f/d47/f6e x:0 0 0 2024-02-28T23:07:37.866 INFO:tasks.workunit.client.1.smithi084.stdout:3/508: readlink d3/d27/d7d/db0/d14/lb6 0 2024-02-28T23:07:37.868 INFO:tasks.workunit.client.0.smithi057.stdout:5/380: rmdir d4/d21 39 2024-02-28T23:07:37.871 INFO:tasks.workunit.client.0.smithi057.stdout:8/408: mknod d4/d1e/d56/d6a/d54/c92 0 2024-02-28T23:07:37.872 INFO:tasks.workunit.client.1.smithi084.stdout:2/477: dwrite d5/dd/f58 [0,4194304] 0 2024-02-28T23:07:37.877 INFO:tasks.workunit.client.1.smithi084.stdout:2/478: write d5/d64/f6c [2644437,73116] 0 2024-02-28T23:07:37.890 INFO:tasks.workunit.client.1.smithi084.stdout:8/517: getdents d0/d9/d30 0 2024-02-28T23:07:37.894 INFO:tasks.workunit.client.1.smithi084.stdout:6/513: rename d1/d13/c84 to d1/d9/d1d/d1e/cb4 0 2024-02-28T23:07:37.898 INFO:tasks.workunit.client.0.smithi057.stdout:2/490: dread d8/d11/d21/d28/f2f [0,4194304] 0 2024-02-28T23:07:37.899 INFO:tasks.workunit.client.1.smithi084.stdout:4/478: creat d1/d18/d20/d24/fa2 x:0 0 0 2024-02-28T23:07:37.899 INFO:tasks.workunit.client.1.smithi084.stdout:9/558: rmdir d4/d6/d15/d12 39 2024-02-28T23:07:37.901 INFO:tasks.workunit.client.1.smithi084.stdout:4/479: write d1/f9c [859886,1472] 0 2024-02-28T23:07:37.905 INFO:tasks.workunit.client.0.smithi057.stdout:5/381: dwrite d4/d21/f37 [0,4194304] 0 2024-02-28T23:07:37.910 INFO:tasks.workunit.client.1.smithi084.stdout:1/592: sync 2024-02-28T23:07:37.910 INFO:tasks.workunit.client.1.smithi084.stdout:7/541: rename db/d10/d17/l1f to db/d11/d27/d7a/la3 0 2024-02-28T23:07:37.914 INFO:tasks.workunit.client.0.smithi057.stdout:1/316: symlink d2/d7/df/db/d12/d19/d24/d2d/d5b/l6f 0 2024-02-28T23:07:37.916 INFO:tasks.workunit.client.1.smithi084.stdout:2/479: dread d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:37.918 INFO:tasks.workunit.client.0.smithi057.stdout:1/317: read - d2/d7/df/db/d12/d19/d24/d2d/f46 zero size 2024-02-28T23:07:37.919 INFO:tasks.workunit.client.0.smithi057.stdout:2/491: write d8/d11/f17 [3979862,40381] 0 2024-02-28T23:07:37.921 INFO:tasks.workunit.client.0.smithi057.stdout:1/318: fsync d2/d7/df/db/d12/f5e 0 2024-02-28T23:07:37.924 INFO:tasks.workunit.client.0.smithi057.stdout:2/492: fsync d8/d11/d15/f20 0 2024-02-28T23:07:37.929 INFO:tasks.workunit.client.1.smithi084.stdout:1/593: dread d12/d1b/d63/f3f [0,4194304] 0 2024-02-28T23:07:37.935 INFO:tasks.workunit.client.0.smithi057.stdout:2/493: rename d8/d11/d39/c74 to d8/d11/d21/ca0 0 2024-02-28T23:07:37.936 INFO:tasks.workunit.client.0.smithi057.stdout:2/494: chown d8/d11/d15/d97 92 1 2024-02-28T23:07:37.938 INFO:tasks.workunit.client.0.smithi057.stdout:2/495: chown d8/d11/d39/f4c 3371153 1 2024-02-28T23:07:37.940 INFO:tasks.workunit.client.0.smithi057.stdout:2/496: chown d8/d11/d39/c46 0 1 2024-02-28T23:07:37.941 INFO:tasks.workunit.client.0.smithi057.stdout:2/497: fsync d8/d11/d15/f80 0 2024-02-28T23:07:37.943 INFO:tasks.workunit.client.1.smithi084.stdout:6/514: truncate d1/d13/d2e/d41/d2c/f71 1832947 0 2024-02-28T23:07:37.945 INFO:tasks.workunit.client.1.smithi084.stdout:3/509: dread d3/d27/d7d/db0/d1a/d45/d5e/f62 [0,4194304] 0 2024-02-28T23:07:37.950 INFO:tasks.workunit.client.1.smithi084.stdout:1/594: creat d12/d1b/d63/fc4 x:0 0 0 2024-02-28T23:07:37.955 INFO:tasks.workunit.client.0.smithi057.stdout:2/498: dwrite d8/d11/d21/d28/f3d [0,4194304] 0 2024-02-28T23:07:37.955 INFO:tasks.workunit.client.1.smithi084.stdout:4/480: dwrite d1/d18/d20/d14/d30/d47/f49 [0,4194304] 0 2024-02-28T23:07:37.958 INFO:tasks.workunit.client.1.smithi084.stdout:8/518: write d0/d9/d1b/f28 [610671,10146] 0 2024-02-28T23:07:37.958 INFO:tasks.workunit.client.1.smithi084.stdout:2/480: write d5/d7/d2e/d4a/f54 [686582,82946] 0 2024-02-28T23:07:37.961 INFO:tasks.workunit.client.0.smithi057.stdout:2/499: creat d8/d11/d21/d70/d99/fa1 x:0 0 0 2024-02-28T23:07:37.962 INFO:tasks.workunit.client.1.smithi084.stdout:4/481: fsync d1/d18/f5f 0 2024-02-28T23:07:37.963 INFO:tasks.workunit.client.1.smithi084.stdout:8/519: readlink d0/d9/d1b/l7a 0 2024-02-28T23:07:37.964 INFO:tasks.workunit.client.0.smithi057.stdout:2/500: truncate d8/d11/d39/f4c 536478 0 2024-02-28T23:07:37.966 INFO:tasks.workunit.client.1.smithi084.stdout:9/559: creat d4/d6/d15/d12/fae x:0 0 0 2024-02-28T23:07:37.966 INFO:tasks.workunit.client.1.smithi084.stdout:6/515: dwrite d1/d13/d2e/f44 [0,4194304] 0 2024-02-28T23:07:37.967 INFO:tasks.workunit.client.0.smithi057.stdout:8/409: rmdir d4/d13/d2c 39 2024-02-28T23:07:37.967 INFO:tasks.workunit.client.1.smithi084.stdout:1/595: symlink d12/d1b/d2e/d30/d77/da0/lc5 0 2024-02-28T23:07:37.969 INFO:tasks.workunit.client.0.smithi057.stdout:8/410: chown d4/d13/d7f/c82 63780177 1 2024-02-28T23:07:37.969 INFO:tasks.workunit.client.0.smithi057.stdout:2/501: mkdir d8/d11/d39/d7b/d8b/da2 0 2024-02-28T23:07:37.970 INFO:tasks.workunit.client.1.smithi084.stdout:1/596: readlink d12/l92 0 2024-02-28T23:07:37.971 INFO:tasks.workunit.client.0.smithi057.stdout:8/411: write d4/d13/f2b [169135,5573] 0 2024-02-28T23:07:37.972 INFO:tasks.workunit.client.1.smithi084.stdout:7/542: dwrite db/d11/d27/f43 [0,4194304] 0 2024-02-28T23:07:37.973 INFO:tasks.workunit.client.1.smithi084.stdout:2/481: unlink d5/d1f/d42/d74/c77 0 2024-02-28T23:07:37.974 INFO:tasks.workunit.client.0.smithi057.stdout:8/412: write d4/d13/d1a/f4b [367701,96784] 0 2024-02-28T23:07:37.980 INFO:tasks.workunit.client.0.smithi057.stdout:8/413: mknod d4/d13/d18/d62/d66/c93 0 2024-02-28T23:07:37.980 INFO:tasks.workunit.client.1.smithi084.stdout:8/520: rmdir d0/d9/d1b/d38/d43 39 2024-02-28T23:07:37.981 INFO:tasks.workunit.client.1.smithi084.stdout:1/597: readlink d12/d1b/d2e/d32/l65 0 2024-02-28T23:07:37.981 INFO:tasks.workunit.client.0.smithi057.stdout:8/414: creat d4/d13/d1a/f94 x:0 0 0 2024-02-28T23:07:37.983 INFO:tasks.workunit.client.1.smithi084.stdout:8/521: stat d0/d24/d9e 0 2024-02-28T23:07:37.983 INFO:tasks.workunit.client.1.smithi084.stdout:2/482: rename d5/f78 to d5/d7/d12/d24/d3a/d48/f8d 0 2024-02-28T23:07:37.984 INFO:tasks.workunit.client.0.smithi057.stdout:8/415: write d4/d13/d18/f23 [886454,98899] 0 2024-02-28T23:07:37.984 INFO:tasks.workunit.client.1.smithi084.stdout:8/522: fdatasync d0/d11/f23 0 2024-02-28T23:07:37.986 INFO:tasks.workunit.client.1.smithi084.stdout:1/598: dread - d12/d1b/d2e/d30/d50/fae zero size 2024-02-28T23:07:37.989 INFO:tasks.workunit.client.0.smithi057.stdout:8/416: creat d4/d1e/d56/d6a/f95 x:0 0 0 2024-02-28T23:07:37.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/510: creat d3/d50/d5a/d73/db8/fc1 x:0 0 0 2024-02-28T23:07:37.992 INFO:tasks.workunit.client.1.smithi084.stdout:7/543: creat db/d10/d17/d9c/d9f/fa4 x:0 0 0 2024-02-28T23:07:37.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/511: stat d3/d27/d7d/db0/d1a/d29/d41/f71 0 2024-02-28T23:07:37.993 INFO:tasks.workunit.client.1.smithi084.stdout:4/482: truncate d1/d18/d20/f11 780927 0 2024-02-28T23:07:37.996 INFO:tasks.workunit.client.1.smithi084.stdout:4/483: write d1/f1d [5655901,30415] 0 2024-02-28T23:07:37.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/560: mknod d4/d6/d15/d75/d9d/caf 0 2024-02-28T23:07:38.000 INFO:tasks.workunit.client.1.smithi084.stdout:1/599: chown d12/d1b/f48 44908588 1 2024-02-28T23:07:38.002 INFO:tasks.workunit.client.0.smithi057.stdout:5/382: creat d4/d21/f76 x:0 0 0 2024-02-28T23:07:38.003 INFO:tasks.workunit.client.1.smithi084.stdout:3/512: truncate d3/d50/d5a/d73/db8/fc1 823219 0 2024-02-28T23:07:38.003 INFO:tasks.workunit.client.0.smithi057.stdout:8/417: dwrite d4/d13/d18/f23 [0,4194304] 0 2024-02-28T23:07:38.004 INFO:tasks.workunit.client.1.smithi084.stdout:7/544: rename db/d11/d27/f43 to db/d10/d17/d9c/d9f/fa5 0 2024-02-28T23:07:38.004 INFO:tasks.workunit.client.1.smithi084.stdout:1/600: chown d12/c94 10210 1 2024-02-28T23:07:38.005 INFO:tasks.workunit.client.0.smithi057.stdout:8/418: rmdir d4/d1e 39 2024-02-28T23:07:38.006 INFO:tasks.workunit.client.1.smithi084.stdout:7/545: read - db/d5e/d80/f97 zero size 2024-02-28T23:07:38.011 INFO:tasks.workunit.client.1.smithi084.stdout:6/516: dwrite d1/f24 [4194304,4194304] 0 2024-02-28T23:07:38.011 INFO:tasks.workunit.client.0.smithi057.stdout:5/383: creat d4/d9/d16/d52/f77 x:0 0 0 2024-02-28T23:07:38.012 INFO:tasks.workunit.client.0.smithi057.stdout:8/419: chown d4/d1e/c51 87451 1 2024-02-28T23:07:38.014 INFO:tasks.workunit.client.0.smithi057.stdout:5/384: truncate d4/f6 665859 0 2024-02-28T23:07:38.014 INFO:tasks.workunit.client.1.smithi084.stdout:7/546: write db/d10/d17/d3b/f48 [1892913,48641] 0 2024-02-28T23:07:38.015 INFO:tasks.workunit.client.0.smithi057.stdout:8/420: dread - d4/d1e/d56/d6a/f6d zero size 2024-02-28T23:07:38.016 INFO:tasks.workunit.client.0.smithi057.stdout:8/421: stat d4/d1e/f2e 0 2024-02-28T23:07:38.021 INFO:tasks.workunit.client.1.smithi084.stdout:1/601: creat d12/d84/d9a/fc6 x:0 0 0 2024-02-28T23:07:38.021 INFO:tasks.workunit.client.1.smithi084.stdout:3/513: symlink d3/d27/d7d/db0/d14/d31/lc2 0 2024-02-28T23:07:38.021 INFO:tasks.workunit.client.1.smithi084.stdout:9/561: getdents d4/d69 0 2024-02-28T23:07:38.022 INFO:tasks.workunit.client.0.smithi057.stdout:5/385: mkdir d4/d7/d58/d78 0 2024-02-28T23:07:38.025 INFO:tasks.workunit.client.0.smithi057.stdout:8/422: truncate d4/f9 650929 0 2024-02-28T23:07:38.025 INFO:tasks.workunit.client.0.smithi057.stdout:5/386: truncate d4/d7/f73 164613 0 2024-02-28T23:07:38.026 INFO:tasks.workunit.client.1.smithi084.stdout:8/523: dwrite d0/d9/d1b/d38/d43/d51/d54/f9c [0,4194304] 0 2024-02-28T23:07:38.026 INFO:tasks.workunit.client.0.smithi057.stdout:8/423: fsync d4/f7 0 2024-02-28T23:07:38.028 INFO:tasks.workunit.client.0.smithi057.stdout:8/424: fdatasync d4/d1e/d56/d6a/f6d 0 2024-02-28T23:07:38.032 INFO:tasks.workunit.client.0.smithi057.stdout:5/387: creat d4/d21/d24/d30/f79 x:0 0 0 2024-02-28T23:07:38.032 INFO:tasks.workunit.client.0.smithi057.stdout:8/425: write d4/d1e/d56/d6a/d42/f5b [609358,120264] 0 2024-02-28T23:07:38.033 INFO:tasks.workunit.client.1.smithi084.stdout:4/484: dread d1/d18/d20/d65/d8e/d55/f1a [0,4194304] 0 2024-02-28T23:07:38.034 INFO:tasks.workunit.client.0.smithi057.stdout:8/426: stat d4/f7 0 2024-02-28T23:07:38.034 INFO:tasks.workunit.client.0.smithi057.stdout:5/388: mkdir d4/d9/d16/d52/d7a 0 2024-02-28T23:07:38.035 INFO:tasks.workunit.client.1.smithi084.stdout:3/514: readlink d3/d27/d7d/db0/d14/l63 0 2024-02-28T23:07:38.036 INFO:tasks.workunit.client.1.smithi084.stdout:8/524: unlink d0/d24/d4d/ca3 0 2024-02-28T23:07:38.036 INFO:tasks.workunit.client.1.smithi084.stdout:4/485: stat d1/d18/d20/d14/d30/l7c 0 2024-02-28T23:07:38.036 INFO:tasks.workunit.client.0.smithi057.stdout:5/389: write d4/d7/f73 [243807,57642] 0 2024-02-28T23:07:38.037 INFO:tasks.workunit.client.0.smithi057.stdout:8/427: mknod d4/d13/d7f/c96 0 2024-02-28T23:07:38.041 INFO:tasks.workunit.client.1.smithi084.stdout:7/547: dread db/d11/d27/f33 [0,4194304] 0 2024-02-28T23:07:38.042 INFO:tasks.workunit.client.1.smithi084.stdout:6/517: symlink d1/d9/d1d/d1e/d33/lb5 0 2024-02-28T23:07:38.042 INFO:tasks.workunit.client.0.smithi057.stdout:5/390: creat d4/d9/d14/d63/f7b x:0 0 0 2024-02-28T23:07:38.043 INFO:tasks.workunit.client.1.smithi084.stdout:3/515: write d3/d27/d7d/f8f [368891,125422] 0 2024-02-28T23:07:38.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/319: unlink d2/d7/df/db/f20 0 2024-02-28T23:07:38.046 INFO:tasks.workunit.client.0.smithi057.stdout:5/391: symlink d4/d7/d58/l7c 0 2024-02-28T23:07:38.047 INFO:tasks.workunit.client.1.smithi084.stdout:9/562: getdents d4/d6/d15 0 2024-02-28T23:07:38.048 INFO:tasks.workunit.client.0.smithi057.stdout:2/502: unlink d8/d11/d21/d28/c4a 0 2024-02-28T23:07:38.049 INFO:tasks.workunit.client.1.smithi084.stdout:6/518: write d1/d13/d50/f99 [1713453,798] 0 2024-02-28T23:07:38.049 INFO:tasks.workunit.client.0.smithi057.stdout:1/320: creat d2/d7/df/db/d3a/f70 x:0 0 0 2024-02-28T23:07:38.050 INFO:tasks.workunit.client.0.smithi057.stdout:6/445: sync 2024-02-28T23:07:38.053 INFO:tasks.workunit.client.1.smithi084.stdout:7/548: write db/d10/d17/f42 [3633748,24769] 0 2024-02-28T23:07:38.054 INFO:tasks.workunit.client.1.smithi084.stdout:3/516: dread - d3/d27/d7d/db0/d1a/d21/d3d/faa zero size 2024-02-28T23:07:38.054 INFO:tasks.workunit.client.1.smithi084.stdout:1/602: dwrite d12/d1b/d2e/d30/d50/fac [0,4194304] 0 2024-02-28T23:07:38.055 INFO:tasks.workunit.client.0.smithi057.stdout:6/446: read - d0/d1b/d54/f90 zero size 2024-02-28T23:07:38.058 INFO:tasks.workunit.client.1.smithi084.stdout:6/519: mkdir d1/d9/d1d/db6 0 2024-02-28T23:07:38.060 INFO:tasks.workunit.client.1.smithi084.stdout:9/563: creat d4/d6/d15/d3c/fb0 x:0 0 0 2024-02-28T23:07:38.061 INFO:tasks.workunit.client.0.smithi057.stdout:6/447: truncate d0/d17/d20/f5c 5061061 0 2024-02-28T23:07:38.063 INFO:tasks.workunit.client.0.smithi057.stdout:5/392: dread d4/d9/d16/f19 [0,4194304] 0 2024-02-28T23:07:38.063 INFO:tasks.workunit.client.0.smithi057.stdout:5/393: chown l3 56 1 2024-02-28T23:07:38.064 INFO:tasks.workunit.client.1.smithi084.stdout:7/549: fdatasync db/d11/f95 0 2024-02-28T23:07:38.066 INFO:tasks.workunit.client.0.smithi057.stdout:1/321: rename d2/d7/df/db/d12/d19/d24 to d2/d71 0 2024-02-28T23:07:38.066 INFO:tasks.workunit.client.1.smithi084.stdout:9/564: write d4/d6/d15/d3c/fb0 [645671,120253] 0 2024-02-28T23:07:38.070 INFO:tasks.workunit.client.1.smithi084.stdout:1/603: unlink d12/d1b/d3b/l81 0 2024-02-28T23:07:38.071 INFO:tasks.workunit.client.1.smithi084.stdout:7/550: write db/d10/d18/f8d [379242,94987] 0 2024-02-28T23:07:38.071 INFO:tasks.workunit.client.0.smithi057.stdout:6/448: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:38.072 INFO:tasks.workunit.client.0.smithi057.stdout:6/449: stat d0/d4 0 2024-02-28T23:07:38.074 INFO:tasks.workunit.client.0.smithi057.stdout:5/394: creat d4/f7d x:0 0 0 2024-02-28T23:07:38.076 INFO:tasks.workunit.client.0.smithi057.stdout:5/395: readlink d4/d32/l3e 0 2024-02-28T23:07:38.076 INFO:tasks.workunit.client.1.smithi084.stdout:3/517: mkdir d3/d27/d7d/db0/d14/dc3 0 2024-02-28T23:07:38.076 INFO:tasks.workunit.client.1.smithi084.stdout:1/604: write d12/d1b/d2e/d32/f4b [5574601,29497] 0 2024-02-28T23:07:38.077 INFO:tasks.workunit.client.1.smithi084.stdout:8/525: link d0/l57 d0/d24/d9e/laf 0 2024-02-28T23:07:38.078 INFO:tasks.workunit.client.0.smithi057.stdout:1/322: creat d2/f72 x:0 0 0 2024-02-28T23:07:38.079 INFO:tasks.workunit.client.1.smithi084.stdout:9/565: truncate d4/f4e 1322051 0 2024-02-28T23:07:38.082 INFO:tasks.workunit.client.1.smithi084.stdout:0/536: sync 2024-02-28T23:07:38.082 INFO:tasks.workunit.client.0.smithi057.stdout:6/450: symlink d0/d4/d11/l99 0 2024-02-28T23:07:38.084 INFO:tasks.workunit.client.0.smithi057.stdout:5/396: chown d4/d7/l20 37753 1 2024-02-28T23:07:38.084 INFO:tasks.workunit.client.1.smithi084.stdout:8/526: rmdir d0 39 2024-02-28T23:07:38.085 INFO:tasks.workunit.client.0.smithi057.stdout:5/397: dread - d4/d21/f76 zero size 2024-02-28T23:07:38.086 INFO:tasks.workunit.client.1.smithi084.stdout:9/566: mknod d4/d6/d15/d12/d41/cb1 0 2024-02-28T23:07:38.089 INFO:tasks.workunit.client.1.smithi084.stdout:7/551: dread db/d10/f56 [0,4194304] 0 2024-02-28T23:07:38.091 INFO:tasks.workunit.client.0.smithi057.stdout:5/398: mknod d4/d9/d14/d63/c7e 0 2024-02-28T23:07:38.092 INFO:tasks.workunit.client.1.smithi084.stdout:4/486: dread d1/d18/d20/d14/d30/f3d [0,4194304] 0 2024-02-28T23:07:38.094 INFO:tasks.workunit.client.1.smithi084.stdout:9/567: chown d4/d6/d15/d75/c35 329418 1 2024-02-28T23:07:38.094 INFO:tasks.workunit.client.1.smithi084.stdout:6/520: dread d1/d13/d50/d65/f95 [0,4194304] 0 2024-02-28T23:07:38.096 INFO:tasks.workunit.client.1.smithi084.stdout:7/552: creat db/d10/d17/d9c/fa6 x:0 0 0 2024-02-28T23:07:38.098 INFO:tasks.workunit.client.1.smithi084.stdout:6/521: readlink d1/d13/d2e/l3a 0 2024-02-28T23:07:38.098 INFO:tasks.workunit.client.1.smithi084.stdout:0/537: link d1/d6/d1b/f58 d1/d6/db/d35/d4a/fac 0 2024-02-28T23:07:38.100 INFO:tasks.workunit.client.1.smithi084.stdout:4/487: truncate d1/d18/d20/d65/d8e/f95 767620 0 2024-02-28T23:07:38.102 INFO:tasks.workunit.client.1.smithi084.stdout:7/553: creat db/d5e/d87/fa7 x:0 0 0 2024-02-28T23:07:38.102 INFO:tasks.workunit.client.1.smithi084.stdout:3/518: link d3/d50/l54 d3/d27/d7d/db0/d1a/d29/d41/d60/d98/lc4 0 2024-02-28T23:07:38.103 INFO:tasks.workunit.client.0.smithi057.stdout:6/451: dwrite d0/d4/ff [0,4194304] 0 2024-02-28T23:07:38.103 INFO:tasks.workunit.client.0.smithi057.stdout:2/503: truncate d8/d11/d15/f2d 339706 0 2024-02-28T23:07:38.105 INFO:tasks.workunit.client.0.smithi057.stdout:2/504: write d8/d11/d39/f4c [1525476,58005] 0 2024-02-28T23:07:38.106 INFO:tasks.workunit.client.1.smithi084.stdout:8/527: creat d0/d44/d9f/fb0 x:0 0 0 2024-02-28T23:07:38.107 INFO:tasks.workunit.client.1.smithi084.stdout:4/488: chown d1/d18/d20/d14/f39 3758241 1 2024-02-28T23:07:38.107 INFO:tasks.workunit.client.0.smithi057.stdout:5/399: creat d4/d9/d16/d52/d7a/f7f x:0 0 0 2024-02-28T23:07:38.108 INFO:tasks.workunit.client.0.smithi057.stdout:5/400: stat d4/d9 0 2024-02-28T23:07:38.109 INFO:tasks.workunit.client.1.smithi084.stdout:0/538: creat d1/d8/d18/d4c/d6c/d74/fad x:0 0 0 2024-02-28T23:07:38.110 INFO:tasks.workunit.client.0.smithi057.stdout:5/401: chown d4/d9/d16/c29 939 1 2024-02-28T23:07:38.110 INFO:tasks.workunit.client.0.smithi057.stdout:2/505: write d8/d11/d15/f5d [1024470,96675] 0 2024-02-28T23:07:38.111 INFO:tasks.workunit.client.1.smithi084.stdout:4/489: chown d1/f12 14271 1 2024-02-28T23:07:38.113 INFO:tasks.workunit.client.1.smithi084.stdout:7/554: readlink db/d10/d17/l7f 0 2024-02-28T23:07:38.113 INFO:tasks.workunit.client.0.smithi057.stdout:5/402: write d4/d21/f35 [1279914,79207] 0 2024-02-28T23:07:38.115 INFO:tasks.workunit.client.1.smithi084.stdout:4/490: stat d1/d18/d20/d24/c86 0 2024-02-28T23:07:38.117 INFO:tasks.workunit.client.0.smithi057.stdout:2/506: creat d8/d11/d39/d83/fa3 x:0 0 0 2024-02-28T23:07:38.118 INFO:tasks.workunit.client.0.smithi057.stdout:8/428: dwrite d4/d13/d1a/d4a/f6e [0,4194304] 0 2024-02-28T23:07:38.121 INFO:tasks.workunit.client.1.smithi084.stdout:9/568: rename d4/c2e to d4/d6/cb2 0 2024-02-28T23:07:38.121 INFO:tasks.workunit.client.1.smithi084.stdout:1/605: dwrite d12/d1b/d3b/f7f [0,4194304] 0 2024-02-28T23:07:38.123 INFO:tasks.workunit.client.1.smithi084.stdout:5/588: sync 2024-02-28T23:07:38.123 INFO:tasks.workunit.client.1.smithi084.stdout:7/555: creat db/d10/d17/d64/d77/fa8 x:0 0 0 2024-02-28T23:07:38.124 INFO:tasks.workunit.client.1.smithi084.stdout:4/491: mknod d1/d18/d20/d14/ca3 0 2024-02-28T23:07:38.125 INFO:tasks.workunit.client.1.smithi084.stdout:7/556: readlink db/l63 0 2024-02-28T23:07:38.125 INFO:tasks.workunit.client.0.smithi057.stdout:8/429: rename d4/d1e/d56/d6a/f6d to d4/d1e/d56/d6a/d54/d61/f97 0 2024-02-28T23:07:38.127 INFO:tasks.workunit.client.0.smithi057.stdout:8/430: chown d4/d1e/d56/d6a/c47 726321609 1 2024-02-28T23:07:38.127 INFO:tasks.workunit.client.1.smithi084.stdout:6/522: dwrite d1/d9/d1d/d1e/d33/f89 [0,4194304] 0 2024-02-28T23:07:38.129 INFO:tasks.workunit.client.0.smithi057.stdout:8/431: truncate d4/d13/d1a/f89 833772 0 2024-02-28T23:07:38.132 INFO:tasks.workunit.client.0.smithi057.stdout:8/432: unlink d4/d13/c15 0 2024-02-28T23:07:38.135 INFO:tasks.workunit.client.1.smithi084.stdout:5/589: creat dc/de/df/d31/dd1/d5f/fe2 x:0 0 0 2024-02-28T23:07:38.138 INFO:tasks.workunit.client.0.smithi057.stdout:8/433: rename d4/d13/d2c/f49 to d4/d1e/d59/f98 0 2024-02-28T23:07:38.143 INFO:tasks.workunit.client.1.smithi084.stdout:4/492: mkdir d1/d18/d20/d14/d30/d47/da4 0 2024-02-28T23:07:38.149 INFO:tasks.workunit.client.0.smithi057.stdout:8/434: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:38.152 INFO:tasks.workunit.client.0.smithi057.stdout:2/507: write d8/d11/d21/f33 [500371,79886] 0 2024-02-28T23:07:38.156 INFO:tasks.workunit.client.0.smithi057.stdout:2/508: mkdir d8/d11/d21/da4 0 2024-02-28T23:07:38.157 INFO:tasks.workunit.client.1.smithi084.stdout:8/528: dwrite d0/d24/d4d/f68 [0,4194304] 0 2024-02-28T23:07:38.158 INFO:tasks.workunit.client.1.smithi084.stdout:1/606: dread d12/d1b/d2e/d52/f73 [0,4194304] 0 2024-02-28T23:07:38.161 INFO:tasks.workunit.client.0.smithi057.stdout:2/509: rename d8/l14 to d8/d11/d21/d28/d41/la5 0 2024-02-28T23:07:38.164 INFO:tasks.workunit.client.1.smithi084.stdout:9/569: dwrite d4/d32/d3f/f96 [0,4194304] 0 2024-02-28T23:07:38.165 INFO:tasks.workunit.client.1.smithi084.stdout:4/493: mkdir d1/d18/d20/d24/d2f/d3e/d52/d87/da5 0 2024-02-28T23:07:38.165 INFO:tasks.workunit.client.1.smithi084.stdout:7/557: write db/d11/d27/d36/f5a [3235263,124627] 0 2024-02-28T23:07:38.165 INFO:tasks.workunit.client.0.smithi057.stdout:2/510: symlink d8/d11/d21/d28/d41/la6 0 2024-02-28T23:07:38.165 INFO:tasks.workunit.client.0.smithi057.stdout:8/435: dwrite d4/d13/f2b [0,4194304] 0 2024-02-28T23:07:38.166 INFO:tasks.workunit.client.0.smithi057.stdout:5/403: truncate d4/d9/d16/f19 193048 0 2024-02-28T23:07:38.168 INFO:tasks.workunit.client.0.smithi057.stdout:2/511: readlink d8/d11/d21/d70/d99/l9a 0 2024-02-28T23:07:38.168 INFO:tasks.workunit.client.0.smithi057.stdout:8/436: chown d4/d1e/d37/l72 302066 1 2024-02-28T23:07:38.172 INFO:tasks.workunit.client.1.smithi084.stdout:6/523: getdents d1/d9/d1d/d1e 0 2024-02-28T23:07:38.173 INFO:tasks.workunit.client.0.smithi057.stdout:2/512: write d8/d11/d15/f72 [129084,123039] 0 2024-02-28T23:07:38.173 INFO:tasks.workunit.client.0.smithi057.stdout:5/404: creat d4/d9/d16/d2c/f80 x:0 0 0 2024-02-28T23:07:38.176 INFO:tasks.workunit.client.0.smithi057.stdout:5/405: fdatasync d4/d9/d14/d3b/f47 0 2024-02-28T23:07:38.176 INFO:tasks.workunit.client.0.smithi057.stdout:8/437: creat d4/d13/d18/d62/d66/f99 x:0 0 0 2024-02-28T23:07:38.177 INFO:tasks.workunit.client.0.smithi057.stdout:2/513: symlink d8/d11/d21/d70/d91/la7 0 2024-02-28T23:07:38.178 INFO:tasks.workunit.client.1.smithi084.stdout:6/524: fdatasync d1/d9/f3b 0 2024-02-28T23:07:38.179 INFO:tasks.workunit.client.1.smithi084.stdout:2/483: sync 2024-02-28T23:07:38.179 INFO:tasks.workunit.client.1.smithi084.stdout:5/590: dwrite dc/d51/d7c/fc3 [0,4194304] 0 2024-02-28T23:07:38.180 INFO:tasks.workunit.client.0.smithi057.stdout:8/438: fsync d4/d1e/d56/d6a/f4d 0 2024-02-28T23:07:38.181 INFO:tasks.workunit.client.1.smithi084.stdout:8/529: truncate d0/d24/d9e/fac 344382 0 2024-02-28T23:07:38.183 INFO:tasks.workunit.client.0.smithi057.stdout:5/406: link d4/d9/d16/l33 d4/d9/d14/d3b/d61/d69/l81 0 2024-02-28T23:07:38.183 INFO:tasks.workunit.client.0.smithi057.stdout:2/514: creat d8/d11/fa8 x:0 0 0 2024-02-28T23:07:38.184 INFO:tasks.workunit.client.0.smithi057.stdout:8/439: mknod d4/d13/d18/d62/d66/c9a 0 2024-02-28T23:07:38.185 INFO:tasks.workunit.client.1.smithi084.stdout:9/570: chown d4/d6/d15/f71 5760 1 2024-02-28T23:07:38.185 INFO:tasks.workunit.client.0.smithi057.stdout:2/515: fsync d8/f23 0 2024-02-28T23:07:38.186 INFO:tasks.workunit.client.0.smithi057.stdout:5/407: write d4/d9/d14/d3b/f47 [755980,18986] 0 2024-02-28T23:07:38.188 INFO:tasks.workunit.client.1.smithi084.stdout:1/607: creat d12/d1b/d2e/d52/fc7 x:0 0 0 2024-02-28T23:07:38.189 INFO:tasks.workunit.client.1.smithi084.stdout:3/519: dread d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c [0,4194304] 0 2024-02-28T23:07:38.189 INFO:tasks.workunit.client.1.smithi084.stdout:0/539: dwrite d1/d8/f40 [0,4194304] 0 2024-02-28T23:07:38.190 INFO:tasks.workunit.client.0.smithi057.stdout:2/516: chown d8/d11/d21/f33 110656 1 2024-02-28T23:07:38.190 INFO:tasks.workunit.client.0.smithi057.stdout:2/517: chown d8/c34 139269329 1 2024-02-28T23:07:38.192 INFO:tasks.workunit.client.0.smithi057.stdout:8/440: rename d4/d13/d1a/c2a to d4/d13/d1a/d4a/d7b/c9b 0 2024-02-28T23:07:38.195 INFO:tasks.workunit.client.1.smithi084.stdout:8/530: write d0/d24/d9e/fac [154091,41815] 0 2024-02-28T23:07:38.195 INFO:tasks.workunit.client.1.smithi084.stdout:6/525: unlink d1/d9/d1d/fb2 0 2024-02-28T23:07:38.195 INFO:tasks.workunit.client.1.smithi084.stdout:1/608: chown d12/d1b/d63/c5c 7302174 1 2024-02-28T23:07:38.195 INFO:tasks.workunit.client.1.smithi084.stdout:7/558: truncate db/d5e/d80/f97 22947 0 2024-02-28T23:07:38.196 INFO:tasks.workunit.client.1.smithi084.stdout:9/571: mkdir d4/d6/d15/db3 0 2024-02-28T23:07:38.197 INFO:tasks.workunit.client.0.smithi057.stdout:5/408: dread d4/d21/f2d [0,4194304] 0 2024-02-28T23:07:38.197 INFO:tasks.workunit.client.0.smithi057.stdout:2/518: symlink d8/d11/d48/d50/la9 0 2024-02-28T23:07:38.203 INFO:tasks.workunit.client.0.smithi057.stdout:2/519: getdents d8/d11/d39/d7b/d8b/da2 0 2024-02-28T23:07:38.205 INFO:tasks.workunit.client.1.smithi084.stdout:2/484: dread - d5/d7/d9/d7c/d5b/f87 zero size 2024-02-28T23:07:38.207 INFO:tasks.workunit.client.1.smithi084.stdout:2/485: readlink d5/d7/d12/l21 0 2024-02-28T23:07:38.207 INFO:tasks.workunit.client.1.smithi084.stdout:4/494: dwrite d1/f1d [0,4194304] 0 2024-02-28T23:07:38.208 INFO:tasks.workunit.client.0.smithi057.stdout:8/441: dwrite d4/d13/d1a/f20 [0,4194304] 0 2024-02-28T23:07:38.209 INFO:tasks.workunit.client.1.smithi084.stdout:1/609: creat d12/d1b/d3b/fc8 x:0 0 0 2024-02-28T23:07:38.212 INFO:tasks.workunit.client.0.smithi057.stdout:5/409: dread d4/d21/f67 [0,4194304] 0 2024-02-28T23:07:38.212 INFO:tasks.workunit.client.0.smithi057.stdout:8/442: write d4/f3a [130375,34375] 0 2024-02-28T23:07:38.212 INFO:tasks.workunit.client.1.smithi084.stdout:8/531: unlink d0/d11/c52 0 2024-02-28T23:07:38.219 INFO:tasks.workunit.client.1.smithi084.stdout:6/526: creat d1/d13/d50/d74/fb7 x:0 0 0 2024-02-28T23:07:38.219 INFO:tasks.workunit.client.0.smithi057.stdout:8/443: mknod d4/d1e/d56/d88/c9c 0 2024-02-28T23:07:38.221 INFO:tasks.workunit.client.0.smithi057.stdout:5/410: creat d4/d21/d24/f82 x:0 0 0 2024-02-28T23:07:38.221 INFO:tasks.workunit.client.1.smithi084.stdout:3/520: creat d3/d27/d7d/db0/d1a/d29/d41/d60/fc5 x:0 0 0 2024-02-28T23:07:38.221 INFO:tasks.workunit.client.1.smithi084.stdout:7/559: truncate db/d10/d17/f58 2225750 0 2024-02-28T23:07:38.224 INFO:tasks.workunit.client.0.smithi057.stdout:5/411: chown d4/d7/l20 57476 1 2024-02-28T23:07:38.224 INFO:tasks.workunit.client.0.smithi057.stdout:2/520: dwrite d8/f64 [0,4194304] 0 2024-02-28T23:07:38.226 INFO:tasks.workunit.client.1.smithi084.stdout:4/495: creat d1/d4b/fa6 x:0 0 0 2024-02-28T23:07:38.226 INFO:tasks.workunit.client.1.smithi084.stdout:2/486: rename d5/d7/d2e/d4a/f80 to d5/d7/d12/d24/d2a/d35/f8e 0 2024-02-28T23:07:38.228 INFO:tasks.workunit.client.0.smithi057.stdout:2/521: chown d8/d11/d15/f20 418 1 2024-02-28T23:07:38.229 INFO:tasks.workunit.client.1.smithi084.stdout:8/532: mknod d0/d9/d1b/d38/d43/d51/d54/cb1 0 2024-02-28T23:07:38.229 INFO:tasks.workunit.client.1.smithi084.stdout:5/591: creat dc/de/fe3 x:0 0 0 2024-02-28T23:07:38.230 INFO:tasks.workunit.client.0.smithi057.stdout:5/412: creat d4/d7/f83 x:0 0 0 2024-02-28T23:07:38.232 INFO:tasks.workunit.client.0.smithi057.stdout:2/522: symlink d8/d11/d21/d28/d49/laa 0 2024-02-28T23:07:38.233 INFO:tasks.workunit.client.0.smithi057.stdout:5/413: creat d4/d9/d14/d3b/f84 x:0 0 0 2024-02-28T23:07:38.235 INFO:tasks.workunit.client.0.smithi057.stdout:5/414: truncate d4/d7/f83 1027412 0 2024-02-28T23:07:38.240 INFO:tasks.workunit.client.0.smithi057.stdout:5/415: mkdir d4/d9/d85 0 2024-02-28T23:07:38.240 INFO:tasks.workunit.client.0.smithi057.stdout:5/416: write d4/d32/f6e [1901700,60940] 0 2024-02-28T23:07:38.240 INFO:tasks.workunit.client.0.smithi057.stdout:5/417: fdatasync d4/d21/d24/f82 0 2024-02-28T23:07:38.241 INFO:tasks.workunit.client.1.smithi084.stdout:3/521: rename d3/d27/d7d/db0/d1a/d29/d41/f64 to d3/d27/d7d/db0/d14/d31/d34/fc6 0 2024-02-28T23:07:38.241 INFO:tasks.workunit.client.1.smithi084.stdout:1/610: dwrite d12/d1b/d2e/d30/f3e [0,4194304] 0 2024-02-28T23:07:38.242 INFO:tasks.workunit.client.0.smithi057.stdout:5/418: write d4/d32/f50 [372718,25905] 0 2024-02-28T23:07:38.243 INFO:tasks.workunit.client.0.smithi057.stdout:5/419: readlink d4/d7/d8/l42 0 2024-02-28T23:07:38.245 INFO:tasks.workunit.client.0.smithi057.stdout:5/420: creat d4/d21/d24/f86 x:0 0 0 2024-02-28T23:07:38.245 INFO:tasks.workunit.client.1.smithi084.stdout:4/496: mkdir d1/d18/d20/d65/d8e/d55/da7 0 2024-02-28T23:07:38.246 INFO:tasks.workunit.client.1.smithi084.stdout:5/592: creat dc/de/d9e/fe4 x:0 0 0 2024-02-28T23:07:38.252 INFO:tasks.workunit.client.1.smithi084.stdout:6/527: dwrite d1/d9/d1d/d1e/d33/f8f [0,4194304] 0 2024-02-28T23:07:38.255 INFO:tasks.workunit.client.1.smithi084.stdout:6/528: fsync d1/d13/d47/f56 0 2024-02-28T23:07:38.261 INFO:tasks.workunit.client.1.smithi084.stdout:6/529: chown d1/d13/c82 2365 1 2024-02-28T23:07:38.265 INFO:tasks.workunit.client.1.smithi084.stdout:2/487: dwrite d5/d7/d12/f51 [0,4194304] 0 2024-02-28T23:07:38.268 INFO:tasks.workunit.client.1.smithi084.stdout:6/530: truncate d1/d13/d9d/da3/fa7 754443 0 2024-02-28T23:07:38.271 INFO:tasks.workunit.client.1.smithi084.stdout:6/531: write d1/d9/fac [839356,2102] 0 2024-02-28T23:07:38.275 INFO:tasks.workunit.client.1.smithi084.stdout:8/533: dread d0/d9/d1b/d20/f25 [0,4194304] 0 2024-02-28T23:07:38.276 INFO:tasks.workunit.client.1.smithi084.stdout:2/488: dread d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:38.277 INFO:tasks.workunit.client.1.smithi084.stdout:0/540: truncate d1/d6/d59/d70/f8d 5175591 0 2024-02-28T23:07:38.278 INFO:tasks.workunit.client.1.smithi084.stdout:7/560: getdents db/d5e/d87 0 2024-02-28T23:07:38.281 INFO:tasks.workunit.client.1.smithi084.stdout:0/541: fdatasync d1/d8/d69/d6b/f7a 0 2024-02-28T23:07:38.284 INFO:tasks.workunit.client.1.smithi084.stdout:7/561: write db/d10/d17/d64/f8c [944254,91096] 0 2024-02-28T23:07:38.287 INFO:tasks.workunit.client.1.smithi084.stdout:4/497: mkdir d1/d18/d20/d24/da8 0 2024-02-28T23:07:38.288 INFO:tasks.workunit.client.0.smithi057.stdout:8/444: truncate d4/f17 1447971 0 2024-02-28T23:07:38.290 INFO:tasks.workunit.client.0.smithi057.stdout:8/445: mkdir d4/d1e/d37/d9d 0 2024-02-28T23:07:38.293 INFO:tasks.workunit.client.0.smithi057.stdout:8/446: mknod d4/d13/d18/c9e 0 2024-02-28T23:07:38.295 INFO:tasks.workunit.client.0.smithi057.stdout:8/447: write d4/d13/d1a/f7a [823446,112618] 0 2024-02-28T23:07:38.296 INFO:tasks.workunit.client.1.smithi084.stdout:8/534: dread d0/d24/d78/f86 [0,4194304] 0 2024-02-28T23:07:38.297 INFO:tasks.workunit.client.0.smithi057.stdout:8/448: dread - d4/d1e/d59/f5f zero size 2024-02-28T23:07:38.298 INFO:tasks.workunit.client.1.smithi084.stdout:6/532: creat d1/d13/d2e/d41/d4d/d6c/fb8 x:0 0 0 2024-02-28T23:07:38.298 INFO:tasks.workunit.client.0.smithi057.stdout:2/523: write d8/d11/d15/f2d [1089920,108678] 0 2024-02-28T23:07:38.299 INFO:tasks.workunit.client.0.smithi057.stdout:8/449: write d4/d13/d18/f26 [1049131,7226] 0 2024-02-28T23:07:38.302 INFO:tasks.workunit.client.0.smithi057.stdout:8/450: mknod d4/d1e/d56/c9f 0 2024-02-28T23:07:38.308 INFO:tasks.workunit.client.0.smithi057.stdout:8/451: creat d4/d1e/d56/d6a/d54/d61/d8c/fa0 x:0 0 0 2024-02-28T23:07:38.309 INFO:tasks.workunit.client.0.smithi057.stdout:8/452: readlink d4/le 0 2024-02-28T23:07:38.309 INFO:tasks.workunit.client.1.smithi084.stdout:7/562: rmdir db/d10/d17/d9c 39 2024-02-28T23:07:38.313 INFO:tasks.workunit.client.1.smithi084.stdout:5/593: dread f7 [0,4194304] 0 2024-02-28T23:07:38.313 INFO:tasks.workunit.client.0.smithi057.stdout:8/453: creat d4/d13/d7f/fa1 x:0 0 0 2024-02-28T23:07:38.314 INFO:tasks.workunit.client.1.smithi084.stdout:0/542: unlink d1/d8/d18/d28/l86 0 2024-02-28T23:07:38.315 INFO:tasks.workunit.client.1.smithi084.stdout:3/522: dread d3/d27/d7d/db0/df/f12 [0,4194304] 0 2024-02-28T23:07:38.318 INFO:tasks.workunit.client.0.smithi057.stdout:5/421: dread d4/d9/d14/f28 [4194304,4194304] 0 2024-02-28T23:07:38.319 INFO:tasks.workunit.client.1.smithi084.stdout:4/498: mknod d1/d18/d20/d24/d2f/d3e/d52/ca9 0 2024-02-28T23:07:38.320 INFO:tasks.workunit.client.0.smithi057.stdout:2/524: dwrite d8/d11/d21/d28/f73 [0,4194304] 0 2024-02-28T23:07:38.325 INFO:tasks.workunit.client.1.smithi084.stdout:2/489: mkdir d5/d1f/d42/d8f 0 2024-02-28T23:07:38.329 INFO:tasks.workunit.client.1.smithi084.stdout:7/563: read - db/d10/d17/d9c/fa6 zero size 2024-02-28T23:07:38.332 INFO:tasks.workunit.client.0.smithi057.stdout:5/422: creat d4/d9/d16/d4e/f87 x:0 0 0 2024-02-28T23:07:38.333 INFO:tasks.workunit.client.1.smithi084.stdout:3/523: write d3/fe [1084011,90743] 0 2024-02-28T23:07:38.336 INFO:tasks.workunit.client.0.smithi057.stdout:5/423: unlink d4/d32/f50 0 2024-02-28T23:07:38.337 INFO:tasks.workunit.client.1.smithi084.stdout:6/533: link d1/d9/d1d/d1e/d33/f8f d1/d13/d50/d74/d9e/fb9 0 2024-02-28T23:07:38.338 INFO:tasks.workunit.client.0.smithi057.stdout:5/424: mkdir d4/d9/d14/d63/d88 0 2024-02-28T23:07:38.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/611: truncate d12/d1b/d2e/d52/d61/d74/f88 1724753 0 2024-02-28T23:07:38.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/612: stat d12/d1b/d2e 0 2024-02-28T23:07:38.345 INFO:tasks.workunit.client.1.smithi084.stdout:8/535: getdents d0/d9/d30 0 2024-02-28T23:07:38.347 INFO:tasks.workunit.client.1.smithi084.stdout:2/490: symlink d5/d7/d75/l90 0 2024-02-28T23:07:38.349 INFO:tasks.workunit.client.1.smithi084.stdout:6/534: unlink d1/d9/d1d/d1e/d33/f7f 0 2024-02-28T23:07:38.350 INFO:tasks.workunit.client.0.smithi057.stdout:8/454: truncate d4/d13/d1a/d4a/f6e 2347937 0 2024-02-28T23:07:38.350 INFO:tasks.workunit.client.0.smithi057.stdout:5/425: write d4/d9/d16/f2b [61747,21837] 0 2024-02-28T23:07:38.351 INFO:tasks.workunit.client.1.smithi084.stdout:5/594: write dc/d51/f88 [945956,57202] 0 2024-02-28T23:07:38.353 INFO:tasks.workunit.client.1.smithi084.stdout:0/543: truncate d1/d6/db/d37/f71 2901435 0 2024-02-28T23:07:38.354 INFO:tasks.workunit.client.1.smithi084.stdout:5/595: dread - dc/de/d9e/fdf zero size 2024-02-28T23:07:38.354 INFO:tasks.workunit.client.0.smithi057.stdout:5/426: creat d4/d7/d58/f89 x:0 0 0 2024-02-28T23:07:38.356 INFO:tasks.workunit.client.0.smithi057.stdout:5/427: chown d4/d9/d16/d2c/f80 294568795 1 2024-02-28T23:07:38.356 INFO:tasks.workunit.client.0.smithi057.stdout:8/455: dread d4/d13/d1a/f7a [0,4194304] 0 2024-02-28T23:07:38.357 INFO:tasks.workunit.client.1.smithi084.stdout:0/544: chown d1/d8/d69/d6b/f9b 0 1 2024-02-28T23:07:38.358 INFO:tasks.workunit.client.0.smithi057.stdout:5/428: chown d4/f6 11187 1 2024-02-28T23:07:38.359 INFO:tasks.workunit.client.1.smithi084.stdout:2/491: fdatasync d5/dd/f86 0 2024-02-28T23:07:38.360 INFO:tasks.workunit.client.1.smithi084.stdout:7/564: dread db/d11/f95 [0,4194304] 0 2024-02-28T23:07:38.362 INFO:tasks.workunit.client.1.smithi084.stdout:3/524: link d3/d27/d7d/db0/d1a/d29/d41/d60/fc5 d3/d27/d7d/db0/d1a/d21/d7b/dbf/fc7 0 2024-02-28T23:07:38.363 INFO:tasks.workunit.client.0.smithi057.stdout:8/456: link d4/d13/d2c/f69 d4/d1e/d59/fa2 0 2024-02-28T23:07:38.364 INFO:tasks.workunit.client.1.smithi084.stdout:7/565: read db/d11/f4f [463832,57374] 0 2024-02-28T23:07:38.364 INFO:tasks.workunit.client.1.smithi084.stdout:2/492: write d5/d7/d12/d24/d3a/d48/f49 [748693,50030] 0 2024-02-28T23:07:38.365 INFO:tasks.workunit.client.1.smithi084.stdout:6/535: rename d1/d13/d50/d65 to d1/d13/d47/d91/dba 0 2024-02-28T23:07:38.365 INFO:tasks.workunit.client.0.smithi057.stdout:5/429: rename d4/d9/d14/c5b to d4/d9/d16/c8a 0 2024-02-28T23:07:38.365 INFO:tasks.workunit.client.0.smithi057.stdout:8/457: stat d4/d1e/d56/d6a/d54/d61/f97 0 2024-02-28T23:07:38.366 INFO:tasks.workunit.client.1.smithi084.stdout:3/525: chown d3/d27/d7d/db0/d1a/d29 2 1 2024-02-28T23:07:38.367 INFO:tasks.workunit.client.1.smithi084.stdout:5/596: mknod dc/de/df/d16/d34/d3c/d77/ce5 0 2024-02-28T23:07:38.368 INFO:tasks.workunit.client.0.smithi057.stdout:8/458: dread - d4/d1e/d56/d6a/d54/d6b/f81 zero size 2024-02-28T23:07:38.370 INFO:tasks.workunit.client.0.smithi057.stdout:5/430: link d4/d21/d24/f22 d4/d9/d16/d4e/f8b 0 2024-02-28T23:07:38.371 INFO:tasks.workunit.client.1.smithi084.stdout:6/536: fsync d1/d9/d1d/d1e/d33/d38/f9a 0 2024-02-28T23:07:38.372 INFO:tasks.workunit.client.1.smithi084.stdout:8/536: creat d0/d9/fb2 x:0 0 0 2024-02-28T23:07:38.372 INFO:tasks.workunit.client.1.smithi084.stdout:7/566: stat db/d10/c20 0 2024-02-28T23:07:38.372 INFO:tasks.workunit.client.1.smithi084.stdout:3/526: chown d3/d27/d7d/db0/d1a/d21/d76/c99 41267 1 2024-02-28T23:07:38.374 INFO:tasks.workunit.client.0.smithi057.stdout:8/459: unlink d4/d1e/d56/d6a/d54/d6b/f81 0 2024-02-28T23:07:38.375 INFO:tasks.workunit.client.0.smithi057.stdout:8/460: chown d4/le 659777 1 2024-02-28T23:07:38.377 INFO:tasks.workunit.client.1.smithi084.stdout:5/597: rmdir dc/de/d83 39 2024-02-28T23:07:38.378 INFO:tasks.workunit.client.1.smithi084.stdout:3/527: read d3/d27/d7d/db0/d1a/d21/d3d/f3c [1747726,36271] 0 2024-02-28T23:07:38.379 INFO:tasks.workunit.client.0.smithi057.stdout:8/461: rename d4/cd to d4/d1e/d56/d6a/d54/d6b/ca3 0 2024-02-28T23:07:38.380 INFO:tasks.workunit.client.1.smithi084.stdout:5/598: dread - dc/de/df/d16/fa0 zero size 2024-02-28T23:07:38.382 INFO:tasks.workunit.client.1.smithi084.stdout:1/613: dread d12/d1b/d2e/d52/d5b/f6f [0,4194304] 0 2024-02-28T23:07:38.385 INFO:tasks.workunit.client.1.smithi084.stdout:3/528: unlink d3/d27/d7d/db0/d1a/d29/d41/ca3 0 2024-02-28T23:07:38.386 INFO:tasks.workunit.client.0.smithi057.stdout:5/431: dwrite d4/d21/d24/d30/f79 [0,4194304] 0 2024-02-28T23:07:38.388 INFO:tasks.workunit.client.1.smithi084.stdout:0/545: truncate d1/d8/d18/d4c/d6c/d74/f97 1329635 0 2024-02-28T23:07:38.389 INFO:tasks.workunit.client.1.smithi084.stdout:3/529: chown d3/d27/d7d/db0/d1a/d29/c40 89090 1 2024-02-28T23:07:38.390 INFO:tasks.workunit.client.1.smithi084.stdout:8/537: link d0/d9/d1b/d38/f3e d0/d9/d1b/d38/d43/d51/fb3 0 2024-02-28T23:07:38.391 INFO:tasks.workunit.client.0.smithi057.stdout:5/432: rename d4/d7/d8/l25 to d4/d21/d24/d30/l8c 0 2024-02-28T23:07:38.392 INFO:tasks.workunit.client.1.smithi084.stdout:1/614: truncate d12/d1b/d63/d7d/f85 73800 0 2024-02-28T23:07:38.393 INFO:tasks.workunit.client.1.smithi084.stdout:6/537: write d1/d9/d1d/d1e/d33/f8f [4297261,78994] 0 2024-02-28T23:07:38.393 INFO:tasks.workunit.client.1.smithi084.stdout:7/567: dwrite db/d10/d17/d3b/f48 [0,4194304] 0 2024-02-28T23:07:38.394 INFO:tasks.workunit.client.1.smithi084.stdout:2/493: link d5/d7/d2e/c65 d5/d7/d9/d7c/c91 0 2024-02-28T23:07:38.394 INFO:tasks.workunit.client.1.smithi084.stdout:3/530: fdatasync d3/d27/d7d/db0/d1a/d21/d3d/f3c 0 2024-02-28T23:07:38.397 INFO:tasks.workunit.client.1.smithi084.stdout:7/568: write db/d10/f23 [4696567,80090] 0 2024-02-28T23:07:38.398 INFO:tasks.workunit.client.0.smithi057.stdout:5/433: creat d4/d7/f8d x:0 0 0 2024-02-28T23:07:38.399 INFO:tasks.workunit.client.1.smithi084.stdout:1/615: dread - d12/d1b/d3b/fc8 zero size 2024-02-28T23:07:38.399 INFO:tasks.workunit.client.1.smithi084.stdout:7/569: chown db/d10/d17 42878001 1 2024-02-28T23:07:38.403 INFO:tasks.workunit.client.1.smithi084.stdout:5/599: dread dc/f27 [0,4194304] 0 2024-02-28T23:07:38.429 INFO:tasks.workunit.client.1.smithi084.stdout:9/572: sync 2024-02-28T23:07:38.443 INFO:tasks.workunit.client.1.smithi084.stdout:2/494: symlink d5/d7/d12/d24/l92 0 2024-02-28T23:07:38.444 INFO:tasks.workunit.client.1.smithi084.stdout:6/538: rename d1/d9/d1d/d1e/d33/d5c to d1/d13/d2e/d41/dbb 0 2024-02-28T23:07:38.449 INFO:tasks.workunit.client.1.smithi084.stdout:7/570: symlink db/d5e/d8a/la9 0 2024-02-28T23:07:38.450 INFO:tasks.workunit.client.1.smithi084.stdout:6/539: write d1/d13/d47/f56 [6395150,30307] 0 2024-02-28T23:07:38.450 INFO:tasks.workunit.client.1.smithi084.stdout:2/495: creat d5/d7/d2e/f93 x:0 0 0 2024-02-28T23:07:38.451 INFO:tasks.workunit.client.1.smithi084.stdout:5/600: unlink dc/de/df/d16/d34/l56 0 2024-02-28T23:07:38.452 INFO:tasks.workunit.client.1.smithi084.stdout:0/546: truncate d1/d6/d59/d70/f8d 3300417 0 2024-02-28T23:07:38.456 INFO:tasks.workunit.client.1.smithi084.stdout:1/616: write d12/d1b/d2e/d52/f6b [1188865,117380] 0 2024-02-28T23:07:38.457 INFO:tasks.workunit.client.1.smithi084.stdout:3/531: dwrite d3/d27/fb7 [0,4194304] 0 2024-02-28T23:07:38.458 INFO:tasks.workunit.client.1.smithi084.stdout:2/496: write d5/d7/d12/d24/f5e [1168964,12681] 0 2024-02-28T23:07:38.461 INFO:tasks.workunit.client.1.smithi084.stdout:0/547: mkdir d1/d6/db/d37/dae 0 2024-02-28T23:07:38.464 INFO:tasks.workunit.client.1.smithi084.stdout:3/532: truncate d3/d50/fbb 665620 0 2024-02-28T23:07:38.464 INFO:tasks.workunit.client.1.smithi084.stdout:7/571: creat db/d10/faa x:0 0 0 2024-02-28T23:07:38.466 INFO:tasks.workunit.client.1.smithi084.stdout:5/601: creat dc/de/df/d31/d8e/da9/dab/fe6 x:0 0 0 2024-02-28T23:07:38.472 INFO:tasks.workunit.client.0.smithi057.stdout:9/464: sync 2024-02-28T23:07:38.475 INFO:tasks.workunit.client.1.smithi084.stdout:3/533: rmdir d3/d27/d7d/db0/df 39 2024-02-28T23:07:38.476 INFO:tasks.workunit.client.1.smithi084.stdout:7/572: creat db/d10/d17/d64/fab x:0 0 0 2024-02-28T23:07:38.483 INFO:tasks.workunit.client.1.smithi084.stdout:3/534: mkdir d3/d50/d5a/d73/dc8 0 2024-02-28T23:07:38.484 INFO:tasks.workunit.client.0.smithi057.stdout:9/465: mknod dd/d1b/d40/d44/d92/c94 0 2024-02-28T23:07:38.484 INFO:tasks.workunit.client.1.smithi084.stdout:6/540: dread d1/d9/d1d/f32 [0,4194304] 0 2024-02-28T23:07:38.486 INFO:tasks.workunit.client.1.smithi084.stdout:3/535: fdatasync d3/f11 0 2024-02-28T23:07:38.487 INFO:tasks.workunit.client.1.smithi084.stdout:7/573: getdents db/d11/d27/d7a 0 2024-02-28T23:07:38.488 INFO:tasks.workunit.client.0.smithi057.stdout:9/466: mknod dd/d1b/d40/d44/c95 0 2024-02-28T23:07:38.488 INFO:tasks.workunit.client.1.smithi084.stdout:9/573: dwrite d4/f90 [0,4194304] 0 2024-02-28T23:07:38.495 INFO:tasks.workunit.client.1.smithi084.stdout:1/617: truncate d12/d1b/d2e/d52/d5b/f8a 1727719 0 2024-02-28T23:07:38.497 INFO:tasks.workunit.client.0.smithi057.stdout:9/467: write dd/d46/d6f/f79 [1028124,2637] 0 2024-02-28T23:07:38.497 INFO:tasks.workunit.client.1.smithi084.stdout:2/497: dwrite d5/d7/d12/d24/d2a/d35/f5d [0,4194304] 0 2024-02-28T23:07:38.499 INFO:tasks.workunit.client.1.smithi084.stdout:7/574: chown db/d10/d17/d64/d77/fa8 0 1 2024-02-28T23:07:38.503 INFO:tasks.workunit.client.0.smithi057.stdout:5/434: dwrite d4/d9/d16/d2c/f80 [0,4194304] 0 2024-02-28T23:07:38.505 INFO:tasks.workunit.client.1.smithi084.stdout:1/618: dread - d12/d84/d9a/fc6 zero size 2024-02-28T23:07:38.505 INFO:tasks.workunit.client.1.smithi084.stdout:7/575: stat db/d11/c1a 0 2024-02-28T23:07:38.507 INFO:tasks.workunit.client.1.smithi084.stdout:2/498: fdatasync d5/dd/f58 0 2024-02-28T23:07:38.508 INFO:tasks.workunit.client.1.smithi084.stdout:3/536: mknod d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/cc9 0 2024-02-28T23:07:38.510 INFO:tasks.workunit.client.1.smithi084.stdout:5/602: dwrite dc/de/df/d31/dd1/d5f/d9b/f9c [0,4194304] 0 2024-02-28T23:07:38.510 INFO:tasks.workunit.client.0.smithi057.stdout:9/468: write dd/de/d14/f5a [1489989,37683] 0 2024-02-28T23:07:38.515 INFO:tasks.workunit.client.1.smithi084.stdout:7/576: write db/d11/f16 [4365580,36485] 0 2024-02-28T23:07:38.515 INFO:tasks.workunit.client.0.smithi057.stdout:5/435: link d4/d9/f2a d4/d21/d24/d72/f8e 0 2024-02-28T23:07:38.516 INFO:tasks.workunit.client.1.smithi084.stdout:0/548: dwrite d1/d6/db/d35/d4a/f88 [0,4194304] 0 2024-02-28T23:07:38.518 INFO:tasks.workunit.client.0.smithi057.stdout:5/436: read - d4/d9/d16/d4e/f87 zero size 2024-02-28T23:07:38.518 INFO:tasks.workunit.client.1.smithi084.stdout:2/499: chown d5/d7/d9/c14 274647 1 2024-02-28T23:07:38.519 INFO:tasks.workunit.client.1.smithi084.stdout:5/603: fdatasync dc/de/df/d31/dd1/d5f/fe2 0 2024-02-28T23:07:38.522 INFO:tasks.workunit.client.0.smithi057.stdout:5/437: rmdir d4/d9/d16/d2c 39 2024-02-28T23:07:38.525 INFO:tasks.workunit.client.1.smithi084.stdout:9/574: symlink d4/d6/d15/d75/d2d/lb4 0 2024-02-28T23:07:38.525 INFO:tasks.workunit.client.1.smithi084.stdout:1/619: creat d12/d1b/d2e/d32/fc9 x:0 0 0 2024-02-28T23:07:38.528 INFO:tasks.workunit.client.1.smithi084.stdout:6/541: dwrite d1/d9/d1d/d1e/d33/f89 [8388608,4194304] 0 2024-02-28T23:07:38.532 INFO:tasks.workunit.client.1.smithi084.stdout:7/577: rename db/d10/f23 to db/d11/d27/d7a/fac 0 2024-02-28T23:07:38.533 INFO:tasks.workunit.client.1.smithi084.stdout:1/620: write d12/d1b/d2e/d30/f1c [3749799,101367] 0 2024-02-28T23:07:38.533 INFO:tasks.workunit.client.0.smithi057.stdout:5/438: mkdir d4/d9/d16/d8f 0 2024-02-28T23:07:38.536 INFO:tasks.workunit.client.0.smithi057.stdout:5/439: truncate d4/d21/d24/f82 340646 0 2024-02-28T23:07:38.536 INFO:tasks.workunit.client.1.smithi084.stdout:3/537: read d3/d27/d7d/db0/d1a/d21/d76/fb2 [516712,13231] 0 2024-02-28T23:07:38.536 INFO:tasks.workunit.client.1.smithi084.stdout:1/621: dread - d12/d1b/d63/fc4 zero size 2024-02-28T23:07:38.538 INFO:tasks.workunit.client.1.smithi084.stdout:6/542: rmdir d1/d88 39 2024-02-28T23:07:38.542 INFO:tasks.workunit.client.1.smithi084.stdout:7/578: mkdir db/d10/d17/d9c/d9f/dad 0 2024-02-28T23:07:38.545 INFO:tasks.workunit.client.1.smithi084.stdout:5/604: dwrite dc/de/df/d31/f7e [0,4194304] 0 2024-02-28T23:07:38.546 INFO:tasks.workunit.client.1.smithi084.stdout:3/538: chown d3/d50/d5a/c9a 142126319 1 2024-02-28T23:07:38.547 INFO:tasks.workunit.client.0.smithi057.stdout:7/383: sync 2024-02-28T23:07:38.548 INFO:tasks.workunit.client.0.smithi057.stdout:4/405: sync 2024-02-28T23:07:38.548 INFO:tasks.workunit.client.0.smithi057.stdout:3/332: sync 2024-02-28T23:07:38.549 INFO:tasks.workunit.client.0.smithi057.stdout:4/406: chown d1a/d1b/d2d 3 1 2024-02-28T23:07:38.552 INFO:tasks.workunit.client.0.smithi057.stdout:7/384: mkdir d0/d3f/d76 0 2024-02-28T23:07:38.553 INFO:tasks.workunit.client.1.smithi084.stdout:9/575: truncate d4/d6/f7 5194534 0 2024-02-28T23:07:38.558 INFO:tasks.workunit.client.1.smithi084.stdout:0/549: dwrite d1/d8/d18/d28/f54 [0,4194304] 0 2024-02-28T23:07:38.558 INFO:tasks.workunit.client.1.smithi084.stdout:5/605: symlink dc/d51/d7c/le7 0 2024-02-28T23:07:38.559 INFO:tasks.workunit.client.0.smithi057.stdout:3/333: creat d3/d4/d7/f6a x:0 0 0 2024-02-28T23:07:38.560 INFO:tasks.workunit.client.1.smithi084.stdout:3/539: fdatasync d3/d27/d7d/db0/d1a/d21/d3d/f52 0 2024-02-28T23:07:38.562 INFO:tasks.workunit.client.0.smithi057.stdout:5/440: dwrite d4/d7/f83 [0,4194304] 0 2024-02-28T23:07:38.564 INFO:tasks.workunit.client.1.smithi084.stdout:4/499: sync 2024-02-28T23:07:38.564 INFO:tasks.workunit.client.1.smithi084.stdout:0/550: readlink d1/d6/db/d37/l9e 0 2024-02-28T23:07:38.566 INFO:tasks.workunit.client.1.smithi084.stdout:1/622: dwrite d12/d1b/f54 [4194304,4194304] 0 2024-02-28T23:07:38.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:38 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:38.569 INFO:tasks.workunit.client.0.smithi057.stdout:7/385: symlink d0/d5/d4a/l77 0 2024-02-28T23:07:38.569 INFO:tasks.workunit.client.1.smithi084.stdout:6/543: creat d1/fbc x:0 0 0 2024-02-28T23:07:38.570 INFO:tasks.workunit.client.0.smithi057.stdout:4/407: creat d1a/d1b/d2d/d49/d4d/f8d x:0 0 0 2024-02-28T23:07:38.570 INFO:tasks.workunit.client.0.smithi057.stdout:3/334: symlink d3/d8/d27/l6b 0 2024-02-28T23:07:38.571 INFO:tasks.workunit.client.1.smithi084.stdout:6/544: stat d1/d13/d2e/d41/fa9 0 2024-02-28T23:07:38.573 INFO:tasks.workunit.client.0.smithi057.stdout:5/441: symlink d4/d21/l90 0 2024-02-28T23:07:38.574 INFO:tasks.workunit.client.0.smithi057.stdout:9/469: dread dd/d1b/f50 [0,4194304] 0 2024-02-28T23:07:38.576 INFO:tasks.workunit.client.1.smithi084.stdout:7/579: creat db/d10/d17/fae x:0 0 0 2024-02-28T23:07:38.576 INFO:tasks.workunit.client.1.smithi084.stdout:2/500: dwrite d5/d7/d12/d24/d2a/d53/f5f [0,4194304] 0 2024-02-28T23:07:38.576 INFO:tasks.workunit.client.1.smithi084.stdout:9/576: creat d4/d6/d15/d12/d41/fb5 x:0 0 0 2024-02-28T23:07:38.576 INFO:tasks.workunit.client.0.smithi057.stdout:5/442: fdatasync d4/d9/d16/f18 0 2024-02-28T23:07:38.578 INFO:tasks.workunit.client.1.smithi084.stdout:0/551: rmdir d1/d6/db/d35/d57 39 2024-02-28T23:07:38.578 INFO:tasks.workunit.client.0.smithi057.stdout:5/443: read - d4/d9/d14/d3b/f84 zero size 2024-02-28T23:07:38.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:38 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:38.580 INFO:tasks.workunit.client.1.smithi084.stdout:1/623: symlink d12/d1b/d63/d7d/lca 0 2024-02-28T23:07:38.582 INFO:tasks.workunit.client.0.smithi057.stdout:9/470: truncate dd/de/f43 766915 0 2024-02-28T23:07:38.583 INFO:tasks.workunit.client.1.smithi084.stdout:6/545: mkdir d1/d13/d50/dbd 0 2024-02-28T23:07:38.583 INFO:tasks.workunit.client.1.smithi084.stdout:7/580: stat db/d5e/c62 0 2024-02-28T23:07:38.585 INFO:tasks.workunit.client.0.smithi057.stdout:2/525: sync 2024-02-28T23:07:38.585 INFO:tasks.workunit.client.0.smithi057.stdout:0/433: sync 2024-02-28T23:07:38.585 INFO:tasks.workunit.client.0.smithi057.stdout:1/323: sync 2024-02-28T23:07:38.585 INFO:tasks.workunit.client.0.smithi057.stdout:7/386: chown d0/d5/d36/d4e/d15/d29/c2c 2854690 1 2024-02-28T23:07:38.586 INFO:tasks.workunit.client.0.smithi057.stdout:8/462: sync 2024-02-28T23:07:38.587 INFO:tasks.workunit.client.0.smithi057.stdout:6/452: sync 2024-02-28T23:07:38.588 INFO:tasks.workunit.client.1.smithi084.stdout:7/581: chown db/d5e/f9d 237 1 2024-02-28T23:07:38.589 INFO:tasks.workunit.client.1.smithi084.stdout:3/540: rmdir d3/d27/d7d/db0/d14/dc3 0 2024-02-28T23:07:38.589 INFO:tasks.workunit.client.0.smithi057.stdout:5/444: symlink d4/d9/d14/d63/d88/l91 0 2024-02-28T23:07:38.591 INFO:tasks.workunit.client.0.smithi057.stdout:9/471: readlink dd/d1b/d40/l42 0 2024-02-28T23:07:38.593 INFO:tasks.workunit.client.1.smithi084.stdout:6/546: unlink d1/d13/d2e/d41/d2c/c9c 0 2024-02-28T23:07:38.598 INFO:tasks.workunit.client.1.smithi084.stdout:0/552: truncate d1/d8/d69/d6b/f7a 434224 0 2024-02-28T23:07:38.598 INFO:tasks.workunit.client.1.smithi084.stdout:2/501: write d5/f31 [1264109,107329] 0 2024-02-28T23:07:38.599 INFO:tasks.workunit.client.0.smithi057.stdout:8/463: symlink d4/d1e/d56/d6a/d42/la4 0 2024-02-28T23:07:38.600 INFO:tasks.workunit.client.1.smithi084.stdout:4/500: dwrite d1/d4a/f54 [0,4194304] 0 2024-02-28T23:07:38.600 INFO:tasks.workunit.client.1.smithi084.stdout:7/582: creat db/d11/d27/d36/faf x:0 0 0 2024-02-28T23:07:38.602 INFO:tasks.workunit.client.0.smithi057.stdout:7/387: dread - d0/d5/d25/d3e/f54 zero size 2024-02-28T23:07:38.607 INFO:tasks.workunit.client.1.smithi084.stdout:3/541: rename d3/d27/d7d/db0/f4f to d3/d27/d7d/da0/fca 0 2024-02-28T23:07:38.607 INFO:tasks.workunit.client.1.smithi084.stdout:8/538: sync 2024-02-28T23:07:38.609 INFO:tasks.workunit.client.1.smithi084.stdout:5/606: dwrite dc/de/df/d16/d34/d3c/f5a [0,4194304] 0 2024-02-28T23:07:38.610 INFO:tasks.workunit.client.0.smithi057.stdout:1/324: chown d2/d7/df/db/d12/l30 29 1 2024-02-28T23:07:38.610 INFO:tasks.workunit.client.1.smithi084.stdout:2/502: mkdir d5/d7/d9/d94 0 2024-02-28T23:07:38.610 INFO:tasks.workunit.client.0.smithi057.stdout:4/408: dwrite d1a/d1e/f42 [0,4194304] 0 2024-02-28T23:07:38.611 INFO:tasks.workunit.client.0.smithi057.stdout:0/434: creat d17/d20/d26/d30/d6b/d71/fa1 x:0 0 0 2024-02-28T23:07:38.612 INFO:tasks.workunit.client.0.smithi057.stdout:9/472: creat dd/de/d89/f96 x:0 0 0 2024-02-28T23:07:38.612 INFO:tasks.workunit.client.0.smithi057.stdout:3/335: dwrite d3/d4/d1d/f39 [0,4194304] 0 2024-02-28T23:07:38.614 INFO:tasks.workunit.client.0.smithi057.stdout:2/526: write d8/d11/f29 [2949173,123657] 0 2024-02-28T23:07:38.614 INFO:tasks.workunit.client.0.smithi057.stdout:8/464: chown d4/d1e/d56/d6a/d42/f5b 4 1 2024-02-28T23:07:38.615 INFO:tasks.workunit.client.0.smithi057.stdout:5/445: creat d4/f92 x:0 0 0 2024-02-28T23:07:38.617 INFO:tasks.workunit.client.0.smithi057.stdout:8/465: fsync d4/f71 0 2024-02-28T23:07:38.619 INFO:tasks.workunit.client.1.smithi084.stdout:0/553: mkdir d1/d6/db/d35/d82/daf 0 2024-02-28T23:07:38.620 INFO:tasks.workunit.client.0.smithi057.stdout:0/435: stat d17/d20/d26/f46 0 2024-02-28T23:07:38.622 INFO:tasks.workunit.client.0.smithi057.stdout:7/388: creat d0/d5/d36/f78 x:0 0 0 2024-02-28T23:07:38.623 INFO:tasks.workunit.client.0.smithi057.stdout:2/527: readlink d8/d11/d39/l6e 0 2024-02-28T23:07:38.625 INFO:tasks.workunit.client.0.smithi057.stdout:4/409: rename d1a/d1e/d51/l67 to d1a/d1e/d4b/l8e 0 2024-02-28T23:07:38.626 INFO:tasks.workunit.client.1.smithi084.stdout:9/577: truncate d4/d6/d15/d12/d41/f64 831562 0 2024-02-28T23:07:38.628 INFO:tasks.workunit.client.0.smithi057.stdout:8/466: dread - d4/d13/d1a/d4a/d7b/f86 zero size 2024-02-28T23:07:38.630 INFO:tasks.workunit.client.0.smithi057.stdout:9/473: mknod dd/de/d14/d48/d4b/c97 0 2024-02-28T23:07:38.631 INFO:tasks.workunit.client.1.smithi084.stdout:2/503: unlink d5/d7/d12/d24/d3a/f4e 0 2024-02-28T23:07:38.633 INFO:tasks.workunit.client.0.smithi057.stdout:0/436: creat d17/d20/d23/d47/fa2 x:0 0 0 2024-02-28T23:07:38.634 INFO:tasks.workunit.client.1.smithi084.stdout:4/501: creat d1/d18/d20/d14/d30/d47/da4/faa x:0 0 0 2024-02-28T23:07:38.634 INFO:tasks.workunit.client.1.smithi084.stdout:8/539: symlink d0/d24/lb4 0 2024-02-28T23:07:38.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/528: write d8/d11/f32 [2788604,48956] 0 2024-02-28T23:07:38.638 INFO:tasks.workunit.client.0.smithi057.stdout:4/410: mkdir d1a/d1b/d24/d2e/d8f 0 2024-02-28T23:07:38.641 INFO:tasks.workunit.client.0.smithi057.stdout:4/411: chown f15 9 1 2024-02-28T23:07:38.643 INFO:tasks.workunit.client.1.smithi084.stdout:8/540: fsync d0/d44/d9f/f98 0 2024-02-28T23:07:38.648 INFO:tasks.workunit.client.1.smithi084.stdout:1/624: dwrite d12/d1b/d2e/d32/fc9 [0,4194304] 0 2024-02-28T23:07:38.648 INFO:tasks.workunit.client.1.smithi084.stdout:5/607: rmdir dc/de/df/d16/d34/daa/dae/de1 0 2024-02-28T23:07:38.650 INFO:tasks.workunit.client.1.smithi084.stdout:9/578: write d4/f85 [927293,5187] 0 2024-02-28T23:07:38.651 INFO:tasks.workunit.client.0.smithi057.stdout:1/325: dwrite d2/d7/df/f4d [0,4194304] 0 2024-02-28T23:07:38.652 INFO:tasks.workunit.client.1.smithi084.stdout:4/502: creat d1/d18/d20/d14/fab x:0 0 0 2024-02-28T23:07:38.653 INFO:tasks.workunit.client.0.smithi057.stdout:5/446: dwrite d4/d9/d14/f54 [0,4194304] 0 2024-02-28T23:07:38.653 INFO:tasks.workunit.client.0.smithi057.stdout:4/412: chown d1a/d1e/d4b/d4f/c87 13709 1 2024-02-28T23:07:38.654 INFO:tasks.workunit.client.1.smithi084.stdout:6/547: dwrite d1/d13/d50/f8d [0,4194304] 0 2024-02-28T23:07:38.657 INFO:tasks.workunit.client.1.smithi084.stdout:8/541: mknod d0/d24/d6e/cb5 0 2024-02-28T23:07:38.657 INFO:tasks.workunit.client.0.smithi057.stdout:2/529: symlink d8/d11/d21/d28/d49/d85/d92/lab 0 2024-02-28T23:07:38.658 INFO:tasks.workunit.client.1.smithi084.stdout:3/542: write d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c [2212448,77272] 0 2024-02-28T23:07:38.658 INFO:tasks.workunit.client.1.smithi084.stdout:7/583: dwrite db/d11/d27/f3a [0,4194304] 0 2024-02-28T23:07:38.659 INFO:tasks.workunit.client.0.smithi057.stdout:5/447: chown d4/d9/d14/f54 494828080 1 2024-02-28T23:07:38.661 INFO:tasks.workunit.client.1.smithi084.stdout:4/503: write d1/f6e [916148,10692] 0 2024-02-28T23:07:38.662 INFO:tasks.workunit.client.0.smithi057.stdout:7/389: dread d0/d5/d36/d4e/d15/d29/f32 [0,4194304] 0 2024-02-28T23:07:38.664 INFO:tasks.workunit.client.1.smithi084.stdout:2/504: creat d5/d7/d9/d7c/f95 x:0 0 0 2024-02-28T23:07:38.667 INFO:tasks.workunit.client.1.smithi084.stdout:1/625: mkdir d12/d1b/d91/d9b/da5/dcb 0 2024-02-28T23:07:38.668 INFO:tasks.workunit.client.0.smithi057.stdout:6/453: write d0/d17/f35 [1545581,60669] 0 2024-02-28T23:07:38.670 INFO:tasks.workunit.client.1.smithi084.stdout:2/505: stat d5/d1f/d83 0 2024-02-28T23:07:38.671 INFO:tasks.workunit.client.0.smithi057.stdout:4/413: write d1a/d1b/d2d/d49/f86 [5524370,111472] 0 2024-02-28T23:07:38.675 INFO:tasks.workunit.client.0.smithi057.stdout:1/326: rmdir d2/d71/d2d 39 2024-02-28T23:07:38.676 INFO:tasks.workunit.client.1.smithi084.stdout:3/543: dread - d3/d27/d7d/db0/d14/f58 zero size 2024-02-28T23:07:38.677 INFO:tasks.workunit.client.1.smithi084.stdout:0/554: dwrite d1/d8/f27 [0,4194304] 0 2024-02-28T23:07:38.677 INFO:tasks.workunit.client.1.smithi084.stdout:6/548: write d1/d13/d2e/d41/f37 [634740,46375] 0 2024-02-28T23:07:38.678 INFO:tasks.workunit.client.0.smithi057.stdout:4/414: chown d1a/d1b/d24/d2e/d43/l85 210 1 2024-02-28T23:07:38.679 INFO:tasks.workunit.client.1.smithi084.stdout:9/579: truncate d4/d6/d53/f61 1122523 0 2024-02-28T23:07:38.680 INFO:tasks.workunit.client.0.smithi057.stdout:2/530: rename d8/d11/d21/d70/d91/d94/f9c to d8/d11/d21/d28/d49/d85/d9b/fac 0 2024-02-28T23:07:38.682 INFO:tasks.workunit.client.0.smithi057.stdout:5/448: dwrite d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:38.684 INFO:tasks.workunit.client.0.smithi057.stdout:6/454: creat d0/d4/d11/f9a x:0 0 0 2024-02-28T23:07:38.685 INFO:tasks.workunit.client.0.smithi057.stdout:7/390: write d0/d5/d36/d4e/d15/f6a [1932636,20283] 0 2024-02-28T23:07:38.692 INFO:tasks.workunit.client.1.smithi084.stdout:4/504: dread d1/d18/d20/d14/d30/f31 [0,4194304] 0 2024-02-28T23:07:38.692 INFO:tasks.workunit.client.1.smithi084.stdout:5/608: dread dc/de/df/d6e/f74 [0,4194304] 0 2024-02-28T23:07:38.692 INFO:tasks.workunit.client.1.smithi084.stdout:6/549: rmdir d1/d13/d47/d91/dba 39 2024-02-28T23:07:38.693 INFO:tasks.workunit.client.0.smithi057.stdout:0/437: link d17/d20/d26/d30/d6b/d71/f92 d17/d19/fa3 0 2024-02-28T23:07:38.694 INFO:tasks.workunit.client.0.smithi057.stdout:7/391: fsync d0/d5/d36/d4e/f19 0 2024-02-28T23:07:38.695 INFO:tasks.workunit.client.0.smithi057.stdout:0/438: chown d17/d19/d4d 102 1 2024-02-28T23:07:38.696 INFO:tasks.workunit.client.0.smithi057.stdout:6/455: write d0/d4/d11/f2d [1585260,37013] 0 2024-02-28T23:07:38.696 INFO:tasks.workunit.client.0.smithi057.stdout:2/531: chown d8/d11/d21/d28/c90 25091072 1 2024-02-28T23:07:38.697 INFO:tasks.workunit.client.0.smithi057.stdout:4/415: rename d1a/d1b/d24/d2e/f3f to d1a/d1e/d51/f90 0 2024-02-28T23:07:38.701 INFO:tasks.workunit.client.1.smithi084.stdout:8/542: rename d0/d9/d1b/d38/d43/d51/d7c to d0/d9/d30/db6 0 2024-02-28T23:07:38.703 INFO:tasks.workunit.client.0.smithi057.stdout:5/449: link d4/d32/f6e d4/d9/d14/d3b/d61/d69/f93 0 2024-02-28T23:07:38.704 INFO:tasks.workunit.client.1.smithi084.stdout:7/584: dwrite db/d10/f94 [0,4194304] 0 2024-02-28T23:07:38.707 INFO:tasks.workunit.client.0.smithi057.stdout:0/439: mknod d17/d19/ca4 0 2024-02-28T23:07:38.707 INFO:tasks.workunit.client.0.smithi057.stdout:2/532: fdatasync d8/d11/d21/f86 0 2024-02-28T23:07:38.707 INFO:tasks.workunit.client.0.smithi057.stdout:7/392: truncate d0/d5/f30 1047910 0 2024-02-28T23:07:38.707 INFO:tasks.workunit.client.0.smithi057.stdout:2/533: read - d8/d11/d39/f8a zero size 2024-02-28T23:07:38.707 INFO:tasks.workunit.client.0.smithi057.stdout:0/440: write d17/d20/d23/d47/d60/fa0 [1022962,28093] 0 2024-02-28T23:07:38.712 INFO:tasks.workunit.client.0.smithi057.stdout:5/450: mknod d4/d9/d16/d8f/c94 0 2024-02-28T23:07:38.715 INFO:tasks.workunit.client.0.smithi057.stdout:0/441: mknod d17/d20/d4a/ca5 0 2024-02-28T23:07:38.715 INFO:tasks.workunit.client.1.smithi084.stdout:4/505: creat d1/d18/d20/d14/d30/fac x:0 0 0 2024-02-28T23:07:38.716 INFO:tasks.workunit.client.1.smithi084.stdout:8/543: dread - d0/d24/d9e/fa2 zero size 2024-02-28T23:07:38.717 INFO:tasks.workunit.client.0.smithi057.stdout:7/393: rename d0/d5/f56 to d0/d5/d36/d4e/d15/d29/f79 0 2024-02-28T23:07:38.719 INFO:tasks.workunit.client.0.smithi057.stdout:5/451: mkdir d4/d9/d14/d3b/d61/d69/d95 0 2024-02-28T23:07:38.721 INFO:tasks.workunit.client.1.smithi084.stdout:7/585: write db/d11/d27/d36/f8e [498826,62527] 0 2024-02-28T23:07:38.722 INFO:tasks.workunit.client.1.smithi084.stdout:2/506: dwrite d5/d7/d12/d24/f66 [0,4194304] 0 2024-02-28T23:07:38.722 INFO:tasks.workunit.client.0.smithi057.stdout:0/442: mkdir d17/d20/d26/d48/d7c/d98/da6 0 2024-02-28T23:07:38.724 INFO:tasks.workunit.client.0.smithi057.stdout:5/452: dread - d4/d21/d24/f64 zero size 2024-02-28T23:07:38.726 INFO:tasks.workunit.client.1.smithi084.stdout:3/544: dwrite d3/f6 [8388608,4194304] 0 2024-02-28T23:07:38.727 INFO:tasks.workunit.client.1.smithi084.stdout:7/586: dread - db/d10/d17/d64/f93 zero size 2024-02-28T23:07:38.727 INFO:tasks.workunit.client.1.smithi084.stdout:9/580: rename d4/d6/d15/d12/f81 to d4/fb6 0 2024-02-28T23:07:38.733 INFO:tasks.workunit.client.0.smithi057.stdout:5/453: write d4/d7/f73 [60985,69803] 0 2024-02-28T23:07:38.733 INFO:tasks.workunit.client.0.smithi057.stdout:7/394: creat d0/d5/d36/d4e/d15/d29/d3d/f7a x:0 0 0 2024-02-28T23:07:38.733 INFO:tasks.workunit.client.0.smithi057.stdout:4/416: dwrite d1a/d2f/f45 [0,4194304] 0 2024-02-28T23:07:38.733 INFO:tasks.workunit.client.1.smithi084.stdout:2/507: truncate d5/d7/d12/d24/d3a/d48/f8b 1030641 0 2024-02-28T23:07:38.736 INFO:tasks.workunit.client.1.smithi084.stdout:4/506: unlink d1/d18/f32 0 2024-02-28T23:07:38.738 INFO:tasks.workunit.client.0.smithi057.stdout:8/467: dread d4/f11 [0,4194304] 0 2024-02-28T23:07:38.740 INFO:tasks.workunit.client.0.smithi057.stdout:4/417: fdatasync d1a/d1e/d51/f90 0 2024-02-28T23:07:38.741 INFO:tasks.workunit.client.0.smithi057.stdout:6/456: dwrite d0/d1b/d3e/d3d/d42/d4f/d53/f93 [0,4194304] 0 2024-02-28T23:07:38.742 INFO:tasks.workunit.client.0.smithi057.stdout:5/454: chown d4/d9/d16/d4e/f87 20 1 2024-02-28T23:07:38.750 INFO:tasks.workunit.client.0.smithi057.stdout:4/418: write d1a/d1b/f3e [166300,8787] 0 2024-02-28T23:07:38.751 INFO:tasks.workunit.client.1.smithi084.stdout:9/581: creat d4/d69/fb7 x:0 0 0 2024-02-28T23:07:38.752 INFO:tasks.workunit.client.0.smithi057.stdout:0/443: dwrite d17/d20/f5d [0,4194304] 0 2024-02-28T23:07:38.754 INFO:tasks.workunit.client.1.smithi084.stdout:3/545: mknod d3/d27/d7d/db0/d1a/ccb 0 2024-02-28T23:07:38.755 INFO:tasks.workunit.client.0.smithi057.stdout:7/395: unlink d0/d5/d36/d4e/d15/d29/d3d/d51/f64 0 2024-02-28T23:07:38.757 INFO:tasks.workunit.client.1.smithi084.stdout:9/582: write d4/d6/d15/d12/d41/d55/f87 [2174685,125974] 0 2024-02-28T23:07:38.760 INFO:tasks.workunit.client.0.smithi057.stdout:5/455: stat d4/d9/d14/d3b/c45 0 2024-02-28T23:07:38.761 INFO:tasks.workunit.client.0.smithi057.stdout:1/327: write d2/d71/d2d/f49 [4925302,80558] 0 2024-02-28T23:07:38.761 INFO:tasks.workunit.client.1.smithi084.stdout:0/555: dread d1/d6/db/d35/f5c [0,4194304] 0 2024-02-28T23:07:38.764 INFO:tasks.workunit.client.0.smithi057.stdout:8/468: dread d4/d13/d18/f23 [0,4194304] 0 2024-02-28T23:07:38.764 INFO:tasks.workunit.client.0.smithi057.stdout:5/456: write d4/d9/d14/f54 [3327434,38548] 0 2024-02-28T23:07:38.764 INFO:tasks.workunit.client.1.smithi084.stdout:2/508: creat d5/d7/d12/d24/d3a/f96 x:0 0 0 2024-02-28T23:07:38.765 INFO:tasks.workunit.client.0.smithi057.stdout:8/469: stat d4/d13/d1a/f58 0 2024-02-28T23:07:38.765 INFO:tasks.workunit.client.0.smithi057.stdout:1/328: dread - d2/d7/df/db/f55 zero size 2024-02-28T23:07:38.767 INFO:tasks.workunit.client.0.smithi057.stdout:4/419: mknod d1a/d1e/d51/c91 0 2024-02-28T23:07:38.768 INFO:tasks.workunit.client.1.smithi084.stdout:4/507: creat d1/d18/d20/d24/d2f/d3e/d52/d57/fad x:0 0 0 2024-02-28T23:07:38.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/444: symlink d17/d7a/la7 0 2024-02-28T23:07:38.771 INFO:tasks.workunit.client.0.smithi057.stdout:7/396: rmdir d0/d5/d25 39 2024-02-28T23:07:38.774 INFO:tasks.workunit.client.1.smithi084.stdout:4/508: write d1/f9c [182198,57664] 0 2024-02-28T23:07:38.775 INFO:tasks.workunit.client.0.smithi057.stdout:6/457: creat d0/d1b/d3e/d3d/d7d/f9b x:0 0 0 2024-02-28T23:07:38.775 INFO:tasks.workunit.client.0.smithi057.stdout:0/445: read d17/d20/d23/d37/f74 [14697,129538] 0 2024-02-28T23:07:38.777 INFO:tasks.workunit.client.1.smithi084.stdout:1/626: dread d12/d1b/d2e/d52/d61/fb1 [0,4194304] 0 2024-02-28T23:07:38.777 INFO:tasks.workunit.client.0.smithi057.stdout:6/458: chown d0/d1b/d54 36483 1 2024-02-28T23:07:38.778 INFO:tasks.workunit.client.0.smithi057.stdout:5/457: readlink d4/d9/d14/d3b/l56 0 2024-02-28T23:07:38.780 INFO:tasks.workunit.client.1.smithi084.stdout:7/587: getdents db/d10/d17/d9c/d9f/dad 0 2024-02-28T23:07:38.781 INFO:tasks.workunit.client.0.smithi057.stdout:6/459: chown d0/d1b/d3e/d3d/d7d 4538 1 2024-02-28T23:07:38.781 INFO:tasks.workunit.client.0.smithi057.stdout:8/470: mknod d4/d1e/d56/d6a/d54/ca5 0 2024-02-28T23:07:38.784 INFO:tasks.workunit.client.0.smithi057.stdout:4/420: write d1a/d1b/f74 [301252,28687] 0 2024-02-28T23:07:38.784 INFO:tasks.workunit.client.0.smithi057.stdout:1/329: dwrite d2/f8 [0,4194304] 0 2024-02-28T23:07:38.784 INFO:tasks.workunit.client.1.smithi084.stdout:9/583: chown d4/l10 4349 1 2024-02-28T23:07:38.786 INFO:tasks.workunit.client.1.smithi084.stdout:5/609: rename dc/de/d9e to dc/de/df/d16/d34/de8 0 2024-02-28T23:07:38.787 INFO:tasks.workunit.client.1.smithi084.stdout:7/588: chown db/d10/d17/d9c/fa6 15810681 1 2024-02-28T23:07:38.789 INFO:tasks.workunit.client.0.smithi057.stdout:6/460: creat d0/d4/d25/f9c x:0 0 0 2024-02-28T23:07:38.790 INFO:tasks.workunit.client.0.smithi057.stdout:5/458: creat d4/d7/d58/f96 x:0 0 0 2024-02-28T23:07:38.792 INFO:tasks.workunit.client.1.smithi084.stdout:2/509: dread d5/d7/d12/d24/f66 [0,4194304] 0 2024-02-28T23:07:38.792 INFO:tasks.workunit.client.0.smithi057.stdout:5/459: dread - d4/d21/f76 zero size 2024-02-28T23:07:38.793 INFO:tasks.workunit.client.1.smithi084.stdout:5/610: write dc/de/df/d16/fa0 [1023754,1368] 0 2024-02-28T23:07:38.796 INFO:tasks.workunit.client.1.smithi084.stdout:9/584: mkdir d4/d32/d3f/d89/db8 0 2024-02-28T23:07:38.797 INFO:tasks.workunit.client.1.smithi084.stdout:4/509: read d1/f2e [1936045,122603] 0 2024-02-28T23:07:38.799 INFO:tasks.workunit.client.0.smithi057.stdout:6/461: rmdir d0/d1b/d3e/d40 39 2024-02-28T23:07:38.801 INFO:tasks.workunit.client.0.smithi057.stdout:0/446: dwrite d17/d19/f28 [0,4194304] 0 2024-02-28T23:07:38.802 INFO:tasks.workunit.client.0.smithi057.stdout:6/462: write d0/d1b/d3e/d3d/f62 [1591601,50229] 0 2024-02-28T23:07:38.803 INFO:tasks.workunit.client.1.smithi084.stdout:1/627: symlink d12/lcc 0 2024-02-28T23:07:38.803 INFO:tasks.workunit.client.1.smithi084.stdout:2/510: write d5/dd/f58 [4816574,114383] 0 2024-02-28T23:07:38.804 INFO:tasks.workunit.client.0.smithi057.stdout:7/397: creat d0/d5/d36/f7b x:0 0 0 2024-02-28T23:07:38.804 INFO:tasks.workunit.client.0.smithi057.stdout:1/330: mknod d2/d33/d54/c73 0 2024-02-28T23:07:38.805 INFO:tasks.workunit.client.1.smithi084.stdout:5/611: rename dc/c2b to dc/d51/ce9 0 2024-02-28T23:07:38.805 INFO:tasks.workunit.client.1.smithi084.stdout:1/628: stat d12/d1b/lbd 0 2024-02-28T23:07:38.806 INFO:tasks.workunit.client.0.smithi057.stdout:8/471: truncate d4/f17 2968596 0 2024-02-28T23:07:38.807 INFO:tasks.workunit.client.1.smithi084.stdout:8/544: link d0/d9/d1b/d38/d43/d51/d54/c6d d0/d9/d1b/d38/d43/cb7 0 2024-02-28T23:07:38.808 INFO:tasks.workunit.client.0.smithi057.stdout:6/463: chown d0/d1b/d3e/d3d/d42/d4f/d53 104316 1 2024-02-28T23:07:38.809 INFO:tasks.workunit.client.1.smithi084.stdout:1/629: read - d12/faf zero size 2024-02-28T23:07:38.811 INFO:tasks.workunit.client.0.smithi057.stdout:4/421: getdents d1a/d1e/d4b/d4f/d6e 0 2024-02-28T23:07:38.811 INFO:tasks.workunit.client.0.smithi057.stdout:8/472: stat d4/d13/d1a/l22 0 2024-02-28T23:07:38.812 INFO:tasks.workunit.client.0.smithi057.stdout:6/464: dread - d0/d1b/d3e/d3d/d42/f8f zero size 2024-02-28T23:07:38.813 INFO:tasks.workunit.client.1.smithi084.stdout:1/630: chown d12/d1b/d3b/f93 51685867 1 2024-02-28T23:07:38.813 INFO:tasks.workunit.client.1.smithi084.stdout:6/550: dread d1/d13/d50/f99 [0,4194304] 0 2024-02-28T23:07:38.813 INFO:tasks.workunit.client.0.smithi057.stdout:0/447: symlink d17/d20/d26/d30/d6b/d71/la8 0 2024-02-28T23:07:38.817 INFO:tasks.workunit.client.1.smithi084.stdout:7/589: mknod db/d10/d17/d9c/d9f/dad/cb0 0 2024-02-28T23:07:38.817 INFO:tasks.workunit.client.0.smithi057.stdout:1/331: rename d2/d7/df/db/d12/d3f/d47/f6e to d2/d71/d62/f74 0 2024-02-28T23:07:38.818 INFO:tasks.workunit.client.0.smithi057.stdout:6/465: write d0/d4/d25/d60/d6a/f69 [367802,104255] 0 2024-02-28T23:07:38.818 INFO:tasks.workunit.client.0.smithi057.stdout:0/448: chown d17/d20/d23/f75 32470 1 2024-02-28T23:07:38.821 INFO:tasks.workunit.client.1.smithi084.stdout:2/511: stat d5/d7/d9/d7c/d5b/c62 0 2024-02-28T23:07:38.823 INFO:tasks.workunit.client.0.smithi057.stdout:0/449: dread - d17/d20/d26/d30/d6b/d71/fa1 zero size 2024-02-28T23:07:38.824 INFO:tasks.workunit.client.1.smithi084.stdout:4/510: read d1/d18/d20/d24/d2f/d3e/d52/f9a [2445311,29644] 0 2024-02-28T23:07:38.824 INFO:tasks.workunit.client.1.smithi084.stdout:6/551: fsync d1/f24 0 2024-02-28T23:07:38.825 INFO:tasks.workunit.client.1.smithi084.stdout:3/546: dread d3/d27/d7d/db0/f10 [0,4194304] 0 2024-02-28T23:07:38.825 INFO:tasks.workunit.client.1.smithi084.stdout:9/585: write d4/d6/d15/f71 [1239941,106568] 0 2024-02-28T23:07:38.826 INFO:tasks.workunit.client.1.smithi084.stdout:1/631: symlink d12/d1b/d2e/d30/d77/lcd 0 2024-02-28T23:07:38.827 INFO:tasks.workunit.client.0.smithi057.stdout:7/398: dwrite d0/d3f/f4c [0,4194304] 0 2024-02-28T23:07:38.829 INFO:tasks.workunit.client.0.smithi057.stdout:8/473: mkdir d4/d1e/d56/da6 0 2024-02-28T23:07:38.830 INFO:tasks.workunit.client.0.smithi057.stdout:6/466: creat d0/d1b/d3e/d3d/f9d x:0 0 0 2024-02-28T23:07:38.830 INFO:tasks.workunit.client.1.smithi084.stdout:6/552: write d1/d13/d2e/d41/f2f [986861,52378] 0 2024-02-28T23:07:38.831 INFO:tasks.workunit.client.1.smithi084.stdout:5/612: rmdir dc/dde 0 2024-02-28T23:07:38.833 INFO:tasks.workunit.client.1.smithi084.stdout:2/512: truncate d5/d7/d1d/f4d 4835184 0 2024-02-28T23:07:38.833 INFO:tasks.workunit.client.1.smithi084.stdout:4/511: fdatasync d1/d18/d20/d24/d2f/f8f 0 2024-02-28T23:07:38.835 INFO:tasks.workunit.client.1.smithi084.stdout:9/586: symlink d4/d6/d15/d12/d41/d55/lb9 0 2024-02-28T23:07:38.835 INFO:tasks.workunit.client.1.smithi084.stdout:4/512: chown d1/f4 3 1 2024-02-28T23:07:38.838 INFO:tasks.workunit.client.0.smithi057.stdout:8/474: mkdir d4/d13/d18/da7 0 2024-02-28T23:07:38.839 INFO:tasks.workunit.client.0.smithi057.stdout:6/467: creat d0/d17/d94/f9e x:0 0 0 2024-02-28T23:07:38.839 INFO:tasks.workunit.client.0.smithi057.stdout:7/399: creat d0/d3f/d76/f7c x:0 0 0 2024-02-28T23:07:38.839 INFO:tasks.workunit.client.1.smithi084.stdout:4/513: truncate d1/d4b/fa6 32306 0 2024-02-28T23:07:38.840 INFO:tasks.workunit.client.1.smithi084.stdout:5/613: truncate dc/de/df/fa7 141244 0 2024-02-28T23:07:38.840 INFO:tasks.workunit.client.0.smithi057.stdout:8/475: fsync d4/d13/d18/f31 0 2024-02-28T23:07:38.842 INFO:tasks.workunit.client.1.smithi084.stdout:5/614: stat dc/d51/d7c/f93 0 2024-02-28T23:07:38.843 INFO:tasks.workunit.client.1.smithi084.stdout:0/556: dread d1/d6/db/d35/d4a/f66 [4194304,4194304] 0 2024-02-28T23:07:38.844 INFO:tasks.workunit.client.0.smithi057.stdout:6/468: chown d0/d1b/d3e/d3d/f8a 1989 1 2024-02-28T23:07:38.847 INFO:tasks.workunit.client.0.smithi057.stdout:1/332: dwrite d2/d7/df/f52 [0,4194304] 0 2024-02-28T23:07:38.847 INFO:tasks.workunit.client.0.smithi057.stdout:8/476: write d4/d13/d1a/d4a/d7b/f8a [117361,34198] 0 2024-02-28T23:07:38.851 INFO:tasks.workunit.client.1.smithi084.stdout:6/553: mknod d1/d9/cbe 0 2024-02-28T23:07:38.851 INFO:tasks.workunit.client.1.smithi084.stdout:1/632: chown d12/d1b/d2e/d30/d77/cb8 1952404367 1 2024-02-28T23:07:38.852 INFO:tasks.workunit.client.0.smithi057.stdout:1/333: fsync d2/d7/df/db/f18 0 2024-02-28T23:07:38.852 INFO:tasks.workunit.client.0.smithi057.stdout:1/334: chown d2/d71/d2d 1981 1 2024-02-28T23:07:38.853 INFO:tasks.workunit.client.0.smithi057.stdout:6/469: write d0/d1b/d3e/d40/f73 [1377235,7444] 0 2024-02-28T23:07:38.855 INFO:tasks.workunit.client.0.smithi057.stdout:8/477: unlink d4/d13/d18/d62/d66/c9a 0 2024-02-28T23:07:38.857 INFO:tasks.workunit.client.0.smithi057.stdout:1/335: readlink d2/d7/df/db/d3a/d2a/l56 0 2024-02-28T23:07:38.862 INFO:tasks.workunit.client.0.smithi057.stdout:5/460: dread d4/d9/d16/f18 [0,4194304] 0 2024-02-28T23:07:38.862 INFO:tasks.workunit.client.0.smithi057.stdout:7/400: getdents d0/d5/d36/d4e/d3b 0 2024-02-28T23:07:38.863 INFO:tasks.workunit.client.1.smithi084.stdout:1/633: write d12/d1b/d3b/f42 [2163644,60286] 0 2024-02-28T23:07:38.864 INFO:tasks.workunit.client.1.smithi084.stdout:4/514: rmdir d1/d18/d20/d24/d2f/d3e/d52/d48 39 2024-02-28T23:07:38.868 INFO:tasks.workunit.client.1.smithi084.stdout:0/557: creat d1/d6/db/d76/fb0 x:0 0 0 2024-02-28T23:07:38.869 INFO:tasks.workunit.client.1.smithi084.stdout:9/587: mknod d4/d6/cba 0 2024-02-28T23:07:38.873 INFO:tasks.workunit.client.1.smithi084.stdout:7/590: dwrite db/d10/d17/d64/f69 [0,4194304] 0 2024-02-28T23:07:38.876 INFO:tasks.workunit.client.0.smithi057.stdout:0/450: truncate d17/d20/d23/d47/f9f 187966 0 2024-02-28T23:07:38.877 INFO:tasks.workunit.client.0.smithi057.stdout:7/401: write d0/d5/d36/d4e/d15/d29/f79 [874450,84394] 0 2024-02-28T23:07:38.879 INFO:tasks.workunit.client.1.smithi084.stdout:8/545: dread d0/d24/f5b [0,4194304] 0 2024-02-28T23:07:38.879 INFO:tasks.workunit.client.0.smithi057.stdout:5/461: symlink d4/d7/l97 0 2024-02-28T23:07:38.881 INFO:tasks.workunit.client.0.smithi057.stdout:8/478: creat d4/d13/fa8 x:0 0 0 2024-02-28T23:07:38.884 INFO:tasks.workunit.client.0.smithi057.stdout:6/470: dwrite d0/d1b/f27 [0,4194304] 0 2024-02-28T23:07:38.884 INFO:tasks.workunit.client.0.smithi057.stdout:1/336: dwrite d2/d7/df/db/d3a/f3c [0,4194304] 0 2024-02-28T23:07:38.889 INFO:tasks.workunit.client.1.smithi084.stdout:3/547: dwrite d3/d27/d7d/db0/d14/d31/d34/fc6 [0,4194304] 0 2024-02-28T23:07:38.891 INFO:tasks.workunit.client.0.smithi057.stdout:8/479: symlink d4/d13/d18/d62/d66/d74/la9 0 2024-02-28T23:07:38.895 INFO:tasks.workunit.client.0.smithi057.stdout:7/402: creat d0/d5/d25/d3e/f7d x:0 0 0 2024-02-28T23:07:38.897 INFO:tasks.workunit.client.0.smithi057.stdout:6/471: symlink d0/d4/d25/l9f 0 2024-02-28T23:07:38.898 INFO:tasks.workunit.client.0.smithi057.stdout:1/337: symlink d2/d7/df/db/d3a/d2a/l75 0 2024-02-28T23:07:38.900 INFO:tasks.workunit.client.1.smithi084.stdout:7/591: dwrite db/d10/f71 [0,4194304] 0 2024-02-28T23:07:38.901 INFO:tasks.workunit.client.0.smithi057.stdout:5/462: creat d4/d9/d14/d3b/d61/d69/d95/f98 x:0 0 0 2024-02-28T23:07:38.905 INFO:tasks.workunit.client.0.smithi057.stdout:8/480: mknod d4/d13/d1a/d4a/d7b/caa 0 2024-02-28T23:07:38.909 INFO:tasks.workunit.client.1.smithi084.stdout:7/592: truncate db/d10/faa 574869 0 2024-02-28T23:07:38.909 INFO:tasks.workunit.client.0.smithi057.stdout:1/338: creat d2/d33/f76 x:0 0 0 2024-02-28T23:07:38.910 INFO:tasks.workunit.client.0.smithi057.stdout:8/481: readlink d4/d13/d2c/d4e/l55 0 2024-02-28T23:07:38.912 INFO:tasks.workunit.client.1.smithi084.stdout:1/634: creat d12/d84/d9a/fce x:0 0 0 2024-02-28T23:07:38.913 INFO:tasks.workunit.client.0.smithi057.stdout:0/451: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:38.915 INFO:tasks.workunit.client.0.smithi057.stdout:1/339: chown d2/d7/df/db/d12/f26 1588 1 2024-02-28T23:07:38.915 INFO:tasks.workunit.client.0.smithi057.stdout:7/403: rename d0/d5/d36/d4e/d15/f42 to d0/f7e 0 2024-02-28T23:07:38.917 INFO:tasks.workunit.client.0.smithi057.stdout:0/452: read f10 [23055,2220] 0 2024-02-28T23:07:38.920 INFO:tasks.workunit.client.1.smithi084.stdout:3/548: creat d3/d27/d7d/db0/d1a/d45/d5e/fcc x:0 0 0 2024-02-28T23:07:38.922 INFO:tasks.workunit.client.0.smithi057.stdout:7/404: creat d0/d3f/f7f x:0 0 0 2024-02-28T23:07:38.927 INFO:tasks.workunit.client.1.smithi084.stdout:1/635: creat d12/d84/fcf x:0 0 0 2024-02-28T23:07:38.927 INFO:tasks.workunit.client.1.smithi084.stdout:8/546: mkdir d0/d9/d1b/d38/d43/dae/db8 0 2024-02-28T23:07:38.927 INFO:tasks.workunit.client.1.smithi084.stdout:9/588: truncate d4/f4e 485669 0 2024-02-28T23:07:38.927 INFO:tasks.workunit.client.0.smithi057.stdout:7/405: write d0/d5/d36/d4e/d15/d29/f40 [752230,124813] 0 2024-02-28T23:07:38.928 INFO:tasks.workunit.client.0.smithi057.stdout:1/340: creat d2/d7/d4f/d5c/d6b/f77 x:0 0 0 2024-02-28T23:07:38.928 INFO:tasks.workunit.client.0.smithi057.stdout:0/453: symlink d17/d20/d26/d48/d7c/d98/la9 0 2024-02-28T23:07:38.929 INFO:tasks.workunit.client.1.smithi084.stdout:2/513: dread d5/d7/d2e/f2f [0,4194304] 0 2024-02-28T23:07:38.930 INFO:tasks.workunit.client.0.smithi057.stdout:8/482: dread d4/d1e/d59/fa2 [0,4194304] 0 2024-02-28T23:07:38.930 INFO:tasks.workunit.client.1.smithi084.stdout:1/636: chown d12/d1b/d2e/d52/d61/f72 120 1 2024-02-28T23:07:38.931 INFO:tasks.workunit.client.1.smithi084.stdout:5/615: rename dc/de/df/d31/f63 to dc/de/df/d16/d34/fea 0 2024-02-28T23:07:38.932 INFO:tasks.workunit.client.0.smithi057.stdout:8/483: fdatasync d4/d13/d1a/f34 0 2024-02-28T23:07:38.932 INFO:tasks.workunit.client.0.smithi057.stdout:0/454: creat d17/d20/d4a/d78/faa x:0 0 0 2024-02-28T23:07:38.933 INFO:tasks.workunit.client.1.smithi084.stdout:1/637: dread - d12/d1b/d2e/d52/fc7 zero size 2024-02-28T23:07:38.933 INFO:tasks.workunit.client.1.smithi084.stdout:2/514: write d5/f31 [2946417,7448] 0 2024-02-28T23:07:38.934 INFO:tasks.workunit.client.0.smithi057.stdout:1/341: truncate d2/d7/d4f/d5c/d6b/f77 396699 0 2024-02-28T23:07:38.935 INFO:tasks.workunit.client.0.smithi057.stdout:0/455: readlink d17/d20/d4a/l9e 0 2024-02-28T23:07:38.936 INFO:tasks.workunit.client.0.smithi057.stdout:7/406: write d0/d5/d36/d4e/d15/f60 [42817,16476] 0 2024-02-28T23:07:38.938 INFO:tasks.workunit.client.1.smithi084.stdout:2/515: dread - d5/d7/d9/f88 zero size 2024-02-28T23:07:38.941 INFO:tasks.workunit.client.1.smithi084.stdout:2/516: write d5/dd/f86 [377296,20884] 0 2024-02-28T23:07:38.941 INFO:tasks.workunit.client.0.smithi057.stdout:1/342: mknod d2/d7/df/db/d3a/c78 0 2024-02-28T23:07:38.941 INFO:tasks.workunit.client.0.smithi057.stdout:8/484: mkdir d4/d1e/d56/d6a/d54/d61/d8c/dab 0 2024-02-28T23:07:38.942 INFO:tasks.workunit.client.0.smithi057.stdout:7/407: truncate d0/d5/d25/d3e/f7d 16070 0 2024-02-28T23:07:38.943 INFO:tasks.workunit.client.0.smithi057.stdout:6/472: dread d0/d17/d20/f5e [0,4194304] 0 2024-02-28T23:07:38.943 INFO:tasks.workunit.client.1.smithi084.stdout:6/554: rename d1/d9/d1d/d1e/d33/d38/da0 to d1/d88/dbf 0 2024-02-28T23:07:38.946 INFO:tasks.workunit.client.0.smithi057.stdout:0/456: dread d17/d20/d26/d48/d7c/f91 [0,4194304] 0 2024-02-28T23:07:38.947 INFO:tasks.workunit.client.0.smithi057.stdout:7/408: mkdir d0/d5/d25/d80 0 2024-02-28T23:07:38.947 INFO:tasks.workunit.client.0.smithi057.stdout:8/485: chown d4/d1e/d56/d6a/f95 122 1 2024-02-28T23:07:38.948 INFO:tasks.workunit.client.1.smithi084.stdout:3/549: symlink d3/d27/d7d/db0/d1a/d45/lcd 0 2024-02-28T23:07:38.948 INFO:tasks.workunit.client.0.smithi057.stdout:7/409: stat d0/l33 0 2024-02-28T23:07:38.948 INFO:tasks.workunit.client.0.smithi057.stdout:6/473: mkdir d0/d17/d94/da0 0 2024-02-28T23:07:38.950 INFO:tasks.workunit.client.0.smithi057.stdout:7/410: stat d0/d5/d36/d4e/lc 0 2024-02-28T23:07:38.950 INFO:tasks.workunit.client.0.smithi057.stdout:0/457: symlink d17/d20/d23/d47/d60/d6f/lab 0 2024-02-28T23:07:38.951 INFO:tasks.workunit.client.0.smithi057.stdout:6/474: readlink d0/d4/d25/l29 0 2024-02-28T23:07:38.955 INFO:tasks.workunit.client.0.smithi057.stdout:0/458: chown d17/d19/f8f 7 1 2024-02-28T23:07:38.956 INFO:tasks.workunit.client.1.smithi084.stdout:9/589: fsync d4/d6/d15/d3c/f47 0 2024-02-28T23:07:38.958 INFO:tasks.workunit.client.0.smithi057.stdout:6/475: unlink d0/d4/d11/f78 0 2024-02-28T23:07:38.961 INFO:tasks.workunit.client.1.smithi084.stdout:3/550: mkdir d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/dce 0 2024-02-28T23:07:38.961 INFO:tasks.workunit.client.1.smithi084.stdout:1/638: dwrite d12/d1b/d63/f2a [0,4194304] 0 2024-02-28T23:07:38.968 INFO:tasks.workunit.client.1.smithi084.stdout:0/558: dread d1/d8/d18/f65 [0,4194304] 0 2024-02-28T23:07:38.969 INFO:tasks.workunit.client.0.smithi057.stdout:8/486: dwrite d4/d13/d1a/f4b [0,4194304] 0 2024-02-28T23:07:38.969 INFO:tasks.workunit.client.0.smithi057.stdout:6/476: rename d0/d4/d25/f52 to d0/d4/d11/fa1 0 2024-02-28T23:07:38.969 INFO:tasks.workunit.client.0.smithi057.stdout:1/343: dwrite d2/d71/d2d/f3d [0,4194304] 0 2024-02-28T23:07:38.970 INFO:tasks.workunit.client.0.smithi057.stdout:8/487: dread - d4/d13/fa8 zero size 2024-02-28T23:07:38.971 INFO:tasks.workunit.client.1.smithi084.stdout:9/590: creat d4/d6/d15/d12/d41/d55/fbb x:0 0 0 2024-02-28T23:07:38.974 INFO:tasks.workunit.client.1.smithi084.stdout:1/639: symlink d12/d1b/d2e/ld0 0 2024-02-28T23:07:38.975 INFO:tasks.workunit.client.0.smithi057.stdout:1/344: chown d2/d7/df/db/d3a/c61 84720586 1 2024-02-28T23:07:38.976 INFO:tasks.workunit.client.0.smithi057.stdout:1/345: stat d2/d33 0 2024-02-28T23:07:38.978 INFO:tasks.workunit.client.0.smithi057.stdout:6/477: rmdir d0/d17/d20 39 2024-02-28T23:07:38.983 INFO:tasks.workunit.client.1.smithi084.stdout:3/551: rename d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/f6e to d3/d27/d7d/db0/d14/d31/d34/fcf 0 2024-02-28T23:07:38.984 INFO:tasks.workunit.client.1.smithi084.stdout:7/593: dread db/d5e/f6a [0,4194304] 0 2024-02-28T23:07:38.984 INFO:tasks.workunit.client.1.smithi084.stdout:9/591: unlink d4/d69/fa8 0 2024-02-28T23:07:38.985 INFO:tasks.workunit.client.0.smithi057.stdout:1/346: rename d2/d7/df/cc to d2/d7/df/db/d12/d3f/c79 0 2024-02-28T23:07:38.985 INFO:tasks.workunit.client.1.smithi084.stdout:6/555: dread d1/d13/d50/d74/d81/f94 [0,4194304] 0 2024-02-28T23:07:38.987 INFO:tasks.workunit.client.0.smithi057.stdout:0/459: dwrite d17/d19/f8f [0,4194304] 0 2024-02-28T23:07:38.988 INFO:tasks.workunit.client.0.smithi057.stdout:8/488: link d4/d1e/d56/c57 d4/d13/d18/d62/d66/d74/d8b/cac 0 2024-02-28T23:07:38.992 INFO:tasks.workunit.client.0.smithi057.stdout:6/478: link d0/d4/d11/f9a d0/d1b/d3e/d3d/d7d/fa2 0 2024-02-28T23:07:38.992 INFO:tasks.workunit.client.1.smithi084.stdout:3/552: fsync d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c 0 2024-02-28T23:07:38.994 INFO:tasks.workunit.client.0.smithi057.stdout:1/347: creat d2/d7/df/db/d3a/d2a/f7a x:0 0 0 2024-02-28T23:07:38.994 INFO:tasks.workunit.client.0.smithi057.stdout:0/460: readlink d17/d19/l1d 0 2024-02-28T23:07:38.994 INFO:tasks.workunit.client.0.smithi057.stdout:8/489: dread d4/d13/d2c/f69 [0,4194304] 0 2024-02-28T23:07:38.995 INFO:tasks.workunit.client.1.smithi084.stdout:0/559: dread d1/d6/db/d37/d72/d7b/f80 [4194304,4194304] 0 2024-02-28T23:07:38.995 INFO:tasks.workunit.client.1.smithi084.stdout:8/547: dread d0/d11/f23 [0,4194304] 0 2024-02-28T23:07:38.996 INFO:tasks.workunit.client.1.smithi084.stdout:1/640: mknod d12/d1b/d2e/d52/cd1 0 2024-02-28T23:07:38.996 INFO:tasks.workunit.client.0.smithi057.stdout:1/348: dread - d2/f72 zero size 2024-02-28T23:07:38.996 INFO:tasks.workunit.client.0.smithi057.stdout:6/479: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:38.996 INFO:tasks.workunit.client.1.smithi084.stdout:8/548: chown d0/d24/d78 1777 1 2024-02-28T23:07:39.001 INFO:tasks.workunit.client.1.smithi084.stdout:5/616: dwrite dc/de/df/d31/dd1/f61 [0,4194304] 0 2024-02-28T23:07:39.004 INFO:tasks.workunit.client.1.smithi084.stdout:0/560: truncate d1/d8/d78/fa4 4479644 0 2024-02-28T23:07:39.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/556: symlink d1/d88/dbf/lc0 0 2024-02-28T23:07:39.004 INFO:tasks.workunit.client.0.smithi057.stdout:6/480: rename d0/f6f to d0/d17/d94/da0/fa3 0 2024-02-28T23:07:39.006 INFO:tasks.workunit.client.1.smithi084.stdout:9/592: creat d4/d6/d15/d12/d41/d55/fbc x:0 0 0 2024-02-28T23:07:39.006 INFO:tasks.workunit.client.1.smithi084.stdout:6/557: fdatasync d1/fbc 0 2024-02-28T23:07:39.011 INFO:tasks.workunit.client.0.smithi057.stdout:6/481: creat d0/d1b/d3e/d3d/d74/fa4 x:0 0 0 2024-02-28T23:07:39.011 INFO:tasks.workunit.client.1.smithi084.stdout:7/594: rename db/d11/d27 to db/d10/d17/d9c/d9f/db1 0 2024-02-28T23:07:39.011 INFO:tasks.workunit.client.1.smithi084.stdout:0/561: stat d1/d6/db/d37/d72/f99 0 2024-02-28T23:07:39.012 INFO:tasks.workunit.client.1.smithi084.stdout:9/593: write d4/d32/f31 [6457917,59065] 0 2024-02-28T23:07:39.012 INFO:tasks.workunit.client.1.smithi084.stdout:6/558: write d1/d88/dbf/fb3 [12796,71185] 0 2024-02-28T23:07:39.013 INFO:tasks.workunit.client.0.smithi057.stdout:8/490: dread d4/f9 [0,4194304] 0 2024-02-28T23:07:39.014 INFO:tasks.workunit.client.1.smithi084.stdout:3/553: creat d3/d27/d7d/db0/d1a/d21/d7b/fd0 x:0 0 0 2024-02-28T23:07:39.021 INFO:tasks.workunit.client.0.smithi057.stdout:8/491: fdatasync d4/d1e/d56/d6a/d42/f5e 0 2024-02-28T23:07:39.021 INFO:tasks.workunit.client.1.smithi084.stdout:7/595: dread - db/d11/f44 zero size 2024-02-28T23:07:39.022 INFO:tasks.workunit.client.0.smithi057.stdout:0/461: dwrite d17/d20/d23/d47/f5b [0,4194304] 0 2024-02-28T23:07:39.024 INFO:tasks.workunit.client.1.smithi084.stdout:6/559: write d1/d13/d47/f7a [797470,125333] 0 2024-02-28T23:07:39.024 INFO:tasks.workunit.client.1.smithi084.stdout:1/641: rename d12/c17 to d12/d1b/d2e/d32/cd2 0 2024-02-28T23:07:39.026 INFO:tasks.workunit.client.1.smithi084.stdout:0/562: symlink d1/d6/d59/lb1 0 2024-02-28T23:07:39.026 INFO:tasks.workunit.client.0.smithi057.stdout:0/462: symlink d17/d7a/lac 0 2024-02-28T23:07:39.029 INFO:tasks.workunit.client.0.smithi057.stdout:8/492: rmdir d4/d1e/d56/d6a/d54/d6b 39 2024-02-28T23:07:39.033 INFO:tasks.workunit.client.0.smithi057.stdout:7/411: dread d0/d5/d36/d4e/d15/f49 [0,4194304] 0 2024-02-28T23:07:39.033 INFO:tasks.workunit.client.1.smithi084.stdout:2/517: dwrite d5/d1f/d42/d74/f76 [0,4194304] 0 2024-02-28T23:07:39.034 INFO:tasks.workunit.client.0.smithi057.stdout:8/493: symlink d4/d1e/d7c/lad 0 2024-02-28T23:07:39.034 INFO:tasks.workunit.client.0.smithi057.stdout:0/463: read d17/d20/d26/d30/d6b/d71/f7e [1367431,86064] 0 2024-02-28T23:07:39.035 INFO:tasks.workunit.client.1.smithi084.stdout:9/594: write d4/d6/d15/d12/f3d [394261,67148] 0 2024-02-28T23:07:39.036 INFO:tasks.workunit.client.0.smithi057.stdout:6/482: dwrite d0/d17/f2c [0,4194304] 0 2024-02-28T23:07:39.037 INFO:tasks.workunit.client.0.smithi057.stdout:0/464: fdatasync d17/d20/d23/d47/fa2 0 2024-02-28T23:07:39.037 INFO:tasks.workunit.client.0.smithi057.stdout:7/412: fdatasync d0/d5/d25/f28 0 2024-02-28T23:07:39.040 INFO:tasks.workunit.client.0.smithi057.stdout:8/494: mkdir d4/d1e/d56/da6/dae 0 2024-02-28T23:07:39.041 INFO:tasks.workunit.client.1.smithi084.stdout:8/549: dwrite d0/d44/d9f/f98 [0,4194304] 0 2024-02-28T23:07:39.041 INFO:tasks.workunit.client.0.smithi057.stdout:7/413: dread - d0/d5/d36/d4e/d3b/f59 zero size 2024-02-28T23:07:39.043 INFO:tasks.workunit.client.0.smithi057.stdout:8/495: fsync d4/d1e/d56/d6a/f95 0 2024-02-28T23:07:39.043 INFO:tasks.workunit.client.0.smithi057.stdout:0/465: creat d17/d20/d26/d30/fad x:0 0 0 2024-02-28T23:07:39.043 INFO:tasks.workunit.client.0.smithi057.stdout:3/336: sync 2024-02-28T23:07:39.046 INFO:tasks.workunit.client.1.smithi084.stdout:7/596: chown db/d10/d18/c28 752152519 1 2024-02-28T23:07:39.046 INFO:tasks.workunit.client.1.smithi084.stdout:6/560: symlink d1/d13/d2e/d41/dbb/lc1 0 2024-02-28T23:07:39.047 INFO:tasks.workunit.client.0.smithi057.stdout:0/466: read - d17/d19/d4d/d7b/f94 zero size 2024-02-28T23:07:39.047 INFO:tasks.workunit.client.0.smithi057.stdout:6/483: unlink d0/d4/d25/d60/l8d 0 2024-02-28T23:07:39.049 INFO:tasks.workunit.client.1.smithi084.stdout:0/563: rename d1/d8/d18/d28 to d1/d8/d69/db2 0 2024-02-28T23:07:39.049 INFO:tasks.workunit.client.1.smithi084.stdout:5/617: dwrite dc/de/df/d31/dd1/d5f/d9b/f9c [4194304,4194304] 0 2024-02-28T23:07:39.050 INFO:tasks.workunit.client.0.smithi057.stdout:0/467: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:39.051 INFO:tasks.workunit.client.0.smithi057.stdout:7/414: mknod d0/d5/d36/d4e/d3b/c81 0 2024-02-28T23:07:39.052 INFO:tasks.workunit.client.0.smithi057.stdout:8/496: fdatasync d4/f9 0 2024-02-28T23:07:39.056 INFO:tasks.workunit.client.1.smithi084.stdout:0/564: readlink d1/d6/db/l2c 0 2024-02-28T23:07:39.056 INFO:tasks.workunit.client.1.smithi084.stdout:6/561: write d1/f1b [13112647,60563] 0 2024-02-28T23:07:39.056 INFO:tasks.workunit.client.0.smithi057.stdout:3/337: creat d3/d4/d7/f6c x:0 0 0 2024-02-28T23:07:39.056 INFO:tasks.workunit.client.0.smithi057.stdout:7/415: chown d0/d5/d36/d4e/d15 212 1 2024-02-28T23:07:39.057 INFO:tasks.workunit.client.0.smithi057.stdout:3/338: read - d3/f64 zero size 2024-02-28T23:07:39.057 INFO:tasks.workunit.client.1.smithi084.stdout:6/562: read - d1/d9/d1d/d1e/d33/f6d zero size 2024-02-28T23:07:39.059 INFO:tasks.workunit.client.1.smithi084.stdout:8/550: readlink d0/d9/d1b/d20/l85 0 2024-02-28T23:07:39.059 INFO:tasks.workunit.client.0.smithi057.stdout:0/468: creat d17/d20/d23/d4b/fae x:0 0 0 2024-02-28T23:07:39.059 INFO:tasks.workunit.client.0.smithi057.stdout:7/416: stat d0/d5/d36/d4e/d3b/f5d 0 2024-02-28T23:07:39.061 INFO:tasks.workunit.client.0.smithi057.stdout:0/469: chown d17/d20/d4a/c4f 14587126 1 2024-02-28T23:07:39.062 INFO:tasks.workunit.client.0.smithi057.stdout:4/422: sync 2024-02-28T23:07:39.062 INFO:tasks.workunit.client.0.smithi057.stdout:8/497: creat d4/d1e/d56/da6/faf x:0 0 0 2024-02-28T23:07:39.062 INFO:tasks.workunit.client.0.smithi057.stdout:2/534: sync 2024-02-28T23:07:39.062 INFO:tasks.workunit.client.0.smithi057.stdout:9/474: sync 2024-02-28T23:07:39.064 INFO:tasks.workunit.client.1.smithi084.stdout:7/597: mkdir db/d10/d17/d9c/d9f/db1/db2 0 2024-02-28T23:07:39.065 INFO:tasks.workunit.client.0.smithi057.stdout:7/417: write d0/d5/d25/f28 [3640346,116746] 0 2024-02-28T23:07:39.067 INFO:tasks.workunit.client.0.smithi057.stdout:3/339: truncate d3/d8/f3c 1016605 0 2024-02-28T23:07:39.068 INFO:tasks.workunit.client.0.smithi057.stdout:0/470: dread - d17/d20/d26/d30/d6b/d71/f92 zero size 2024-02-28T23:07:39.070 INFO:tasks.workunit.client.0.smithi057.stdout:6/484: creat d0/d1b/d3e/fa5 x:0 0 0 2024-02-28T23:07:39.073 INFO:tasks.workunit.client.0.smithi057.stdout:8/498: write d4/d13/d2c/d4e/f87 [167806,117864] 0 2024-02-28T23:07:39.073 INFO:tasks.workunit.client.0.smithi057.stdout:4/423: chown d1a/d1b/d24/d2e/d43/c72 540714525 1 2024-02-28T23:07:39.074 INFO:tasks.workunit.client.1.smithi084.stdout:7/598: rmdir db/d5e/d80 39 2024-02-28T23:07:39.076 INFO:tasks.workunit.client.0.smithi057.stdout:9/475: symlink dd/d1b/d40/d44/l98 0 2024-02-28T23:07:39.078 INFO:tasks.workunit.client.1.smithi084.stdout:7/599: truncate db/d5e/f6a 4290254 0 2024-02-28T23:07:39.080 INFO:tasks.workunit.client.1.smithi084.stdout:6/563: mkdir d1/d13/d50/d74/d9e/dc2 0 2024-02-28T23:07:39.080 INFO:tasks.workunit.client.1.smithi084.stdout:9/595: dwrite d4/d6/d15/d12/f9e [0,4194304] 0 2024-02-28T23:07:39.083 INFO:tasks.workunit.client.1.smithi084.stdout:3/554: read d3/d27/d7d/db0/d14/f49 [4029274,103699] 0 2024-02-28T23:07:39.083 INFO:tasks.workunit.client.0.smithi057.stdout:7/418: chown d0/d5/d36/d4e/d15/d29/c73 0 1 2024-02-28T23:07:39.084 INFO:tasks.workunit.client.0.smithi057.stdout:2/535: mkdir d8/d11/d21/d28/d49/d85/d9b/dad 0 2024-02-28T23:07:39.084 INFO:tasks.workunit.client.0.smithi057.stdout:6/485: creat d0/d1b/d3e/d40/fa6 x:0 0 0 2024-02-28T23:07:39.084 INFO:tasks.workunit.client.0.smithi057.stdout:9/476: write dd/de/d24/f85 [11409,44373] 0 2024-02-28T23:07:39.087 INFO:tasks.workunit.client.1.smithi084.stdout:0/565: rename d1/l15 to d1/d6/db/d35/d57/d81/lb3 0 2024-02-28T23:07:39.087 INFO:tasks.workunit.client.0.smithi057.stdout:1/349: truncate d2/d7/df/db/d12/f26 3745895 0 2024-02-28T23:07:39.090 INFO:tasks.workunit.client.0.smithi057.stdout:8/499: write d4/d13/d2c/d4e/f87 [912414,120784] 0 2024-02-28T23:07:39.092 INFO:tasks.workunit.client.1.smithi084.stdout:6/564: mknod d1/d9/d1d/d1e/d33/d38/cc3 0 2024-02-28T23:07:39.092 INFO:tasks.workunit.client.0.smithi057.stdout:9/477: symlink dd/d1b/l99 0 2024-02-28T23:07:39.094 INFO:tasks.workunit.client.1.smithi084.stdout:9/596: mkdir d4/d6/d15/d75/d2d/dbd 0 2024-02-28T23:07:39.096 INFO:tasks.workunit.client.1.smithi084.stdout:0/566: chown d1/d6/db/d37/l9e 0 1 2024-02-28T23:07:39.096 INFO:tasks.workunit.client.1.smithi084.stdout:7/600: getdents db/d5e/d80 0 2024-02-28T23:07:39.101 INFO:tasks.workunit.client.1.smithi084.stdout:8/551: dwrite d0/d9/fb [4194304,4194304] 0 2024-02-28T23:07:39.102 INFO:tasks.workunit.client.0.smithi057.stdout:1/350: mknod d2/d7/d4f/c7b 0 2024-02-28T23:07:39.102 INFO:tasks.workunit.client.0.smithi057.stdout:7/419: truncate d0/d5/d36/d4e/d15/d29/f2a 4897451 0 2024-02-28T23:07:39.102 INFO:tasks.workunit.client.1.smithi084.stdout:9/597: mknod d4/d69/cbe 0 2024-02-28T23:07:39.103 INFO:tasks.workunit.client.1.smithi084.stdout:5/618: write dc/de/df/d31/dd1/f91 [646042,74427] 0 2024-02-28T23:07:39.107 INFO:tasks.workunit.client.1.smithi084.stdout:6/565: write d1/d13/d2e/d41/d4d/d6c/f7c [295942,17519] 0 2024-02-28T23:07:39.107 INFO:tasks.workunit.client.1.smithi084.stdout:7/601: write db/d10/d17/f42 [3144652,107716] 0 2024-02-28T23:07:39.107 INFO:tasks.workunit.client.1.smithi084.stdout:3/555: dread d3/d27/d7d/db0/d1a/d29/d41/f71 [0,4194304] 0 2024-02-28T23:07:39.107 INFO:tasks.workunit.client.0.smithi057.stdout:6/486: creat d0/d17/fa7 x:0 0 0 2024-02-28T23:07:39.108 INFO:tasks.workunit.client.0.smithi057.stdout:9/478: truncate dd/de/d24/f85 763732 0 2024-02-28T23:07:39.111 INFO:tasks.workunit.client.0.smithi057.stdout:0/471: dwrite d17/d20/d23/f8b [0,4194304] 0 2024-02-28T23:07:39.113 INFO:tasks.workunit.client.1.smithi084.stdout:1/642: dwrite d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:39.113 INFO:tasks.workunit.client.0.smithi057.stdout:3/340: dread d3/f5a [0,4194304] 0 2024-02-28T23:07:39.116 INFO:tasks.workunit.client.1.smithi084.stdout:8/552: chown d0/d9/d1b/f74 100 1 2024-02-28T23:07:39.116 INFO:tasks.workunit.client.1.smithi084.stdout:7/602: chown db/d10/c50 41790296 1 2024-02-28T23:07:39.116 INFO:tasks.workunit.client.0.smithi057.stdout:6/487: write d0/d4/d25/f9c [868010,128866] 0 2024-02-28T23:07:39.117 INFO:tasks.workunit.client.1.smithi084.stdout:1/643: chown d12/d1b/d2e/d52/cd1 4515642 1 2024-02-28T23:07:39.118 INFO:tasks.workunit.client.1.smithi084.stdout:1/644: read - d12/d84/d9a/fce zero size 2024-02-28T23:07:39.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/463: sync 2024-02-28T23:07:39.120 INFO:tasks.workunit.client.0.smithi057.stdout:6/488: chown d0/d17/f2c 20 1 2024-02-28T23:07:39.122 INFO:tasks.workunit.client.0.smithi057.stdout:2/536: dwrite d8/d11/d39/d7b/d8b/f96 [0,4194304] 0 2024-02-28T23:07:39.124 INFO:tasks.workunit.client.0.smithi057.stdout:5/464: write d4/d9/d16/d52/f77 [929879,48720] 0 2024-02-28T23:07:39.126 INFO:tasks.workunit.client.0.smithi057.stdout:1/351: read d2/d7/df/db/d3a/f3b [3338539,32410] 0 2024-02-28T23:07:39.136 INFO:tasks.workunit.client.0.smithi057.stdout:4/424: truncate f16 1379600 0 2024-02-28T23:07:39.136 INFO:tasks.workunit.client.0.smithi057.stdout:9/479: mknod dd/d1b/d40/d44/c9a 0 2024-02-28T23:07:39.142 INFO:tasks.workunit.client.0.smithi057.stdout:4/425: stat d1a/d1e/f25 0 2024-02-28T23:07:39.142 INFO:tasks.workunit.client.0.smithi057.stdout:1/352: unlink d2/d7/d4f/c7b 0 2024-02-28T23:07:39.142 INFO:tasks.workunit.client.0.smithi057.stdout:2/537: unlink d8/d11/d48/d50/f7a 0 2024-02-28T23:07:39.143 INFO:tasks.workunit.client.0.smithi057.stdout:5/465: symlink d4/d32/l99 0 2024-02-28T23:07:39.143 INFO:tasks.workunit.client.0.smithi057.stdout:4/426: chown d1a/d1b/d24/d2e/d43/c72 3973 1 2024-02-28T23:07:39.145 INFO:tasks.workunit.client.0.smithi057.stdout:2/538: write d8/d11/f29 [4064808,72410] 0 2024-02-28T23:07:39.146 INFO:tasks.workunit.client.0.smithi057.stdout:4/427: chown d1a/d1b/d2d/d49/d4d 65009437 1 2024-02-28T23:07:39.154 INFO:tasks.workunit.client.0.smithi057.stdout:8/500: creat d4/d1e/d56/d6a/fb0 x:0 0 0 2024-02-28T23:07:39.154 INFO:tasks.workunit.client.0.smithi057.stdout:2/539: write d8/d11/d21/f36 [2488234,3130] 0 2024-02-28T23:07:39.154 INFO:tasks.workunit.client.0.smithi057.stdout:6/489: dwrite d0/d17/f82 [0,4194304] 0 2024-02-28T23:07:39.158 INFO:tasks.workunit.client.0.smithi057.stdout:7/420: rename d0/d5/d25/d3e/c57 to d0/d5/d36/d4e/d3b/d75/c82 0 2024-02-28T23:07:39.161 INFO:tasks.workunit.client.0.smithi057.stdout:9/480: creat dd/d1b/d40/d44/f9b x:0 0 0 2024-02-28T23:07:39.161 INFO:tasks.workunit.client.0.smithi057.stdout:6/490: write d0/d17/d94/f9e [132234,4425] 0 2024-02-28T23:07:39.164 INFO:tasks.workunit.client.0.smithi057.stdout:3/341: mknod d3/d8/c6d 0 2024-02-28T23:07:39.165 INFO:tasks.workunit.client.0.smithi057.stdout:9/481: write dd/d1b/f69 [4052367,44013] 0 2024-02-28T23:07:39.177 INFO:tasks.workunit.client.0.smithi057.stdout:0/472: rmdir d17/d20/d23/d47/d85 0 2024-02-28T23:07:39.179 INFO:tasks.workunit.client.0.smithi057.stdout:7/421: creat d0/d5/d36/d4e/f83 x:0 0 0 2024-02-28T23:07:39.183 INFO:tasks.workunit.client.0.smithi057.stdout:6/491: mknod d0/d4/d25/d60/d6a/ca8 0 2024-02-28T23:07:39.183 INFO:tasks.workunit.client.0.smithi057.stdout:7/422: write d0/d5/d36/d4e/d15/d29/f40 [963317,699] 0 2024-02-28T23:07:39.185 INFO:tasks.workunit.client.0.smithi057.stdout:9/482: dwrite dd/de/f43 [0,4194304] 0 2024-02-28T23:07:39.185 INFO:tasks.workunit.client.0.smithi057.stdout:7/423: write d0/f67 [306634,78418] 0 2024-02-28T23:07:39.186 INFO:tasks.workunit.client.0.smithi057.stdout:6/492: dread - d0/d1b/d3e/d3d/f9d zero size 2024-02-28T23:07:39.188 INFO:tasks.workunit.client.0.smithi057.stdout:5/466: symlink d4/d9/d85/l9a 0 2024-02-28T23:07:39.190 INFO:tasks.workunit.client.0.smithi057.stdout:7/424: fdatasync d0/d5/d36/d4e/d15/d29/f40 0 2024-02-28T23:07:39.191 INFO:tasks.workunit.client.0.smithi057.stdout:2/540: unlink d8/d11/d21/ca0 0 2024-02-28T23:07:39.197 INFO:tasks.workunit.client.0.smithi057.stdout:4/428: link d1a/d1b/f1f d1a/d48/d6c/f92 0 2024-02-28T23:07:39.198 INFO:tasks.workunit.client.0.smithi057.stdout:0/473: mknod d17/d20/d23/d47/d60/d6f/caf 0 2024-02-28T23:07:39.199 INFO:tasks.workunit.client.0.smithi057.stdout:6/493: creat d0/d4/d25/fa9 x:0 0 0 2024-02-28T23:07:39.209 INFO:tasks.workunit.client.0.smithi057.stdout:9/483: mknod dd/de/c9c 0 2024-02-28T23:07:39.211 INFO:tasks.workunit.client.0.smithi057.stdout:0/474: unlink d17/d20/d23/d47/d60/fa0 0 2024-02-28T23:07:39.214 INFO:tasks.workunit.client.0.smithi057.stdout:8/501: getdents d4/d1e 0 2024-02-28T23:07:39.217 INFO:tasks.workunit.client.0.smithi057.stdout:2/541: mkdir d8/dae 0 2024-02-28T23:07:39.219 INFO:tasks.workunit.client.0.smithi057.stdout:2/542: fdatasync d8/d11/d21/f67 0 2024-02-28T23:07:39.219 INFO:tasks.workunit.client.0.smithi057.stdout:4/429: dwrite d1a/d1e/d51/f90 [0,4194304] 0 2024-02-28T23:07:39.221 INFO:tasks.workunit.client.0.smithi057.stdout:9/484: symlink dd/de/d24/d72/l9d 0 2024-02-28T23:07:39.223 INFO:tasks.workunit.client.0.smithi057.stdout:2/543: write d8/d11/d21/f86 [30332,23512] 0 2024-02-28T23:07:39.230 INFO:tasks.workunit.client.0.smithi057.stdout:0/475: unlink d17/d7a/la7 0 2024-02-28T23:07:39.230 INFO:tasks.workunit.client.0.smithi057.stdout:3/342: truncate d3/d20/d2d/f3b 1448866 0 2024-02-28T23:07:39.231 INFO:tasks.workunit.client.0.smithi057.stdout:8/502: mkdir d4/d13/d2c/db1 0 2024-02-28T23:07:39.233 INFO:tasks.workunit.client.0.smithi057.stdout:4/430: rmdir d1a/d1b/d24 39 2024-02-28T23:07:39.233 INFO:tasks.workunit.client.0.smithi057.stdout:0/476: readlink d17/d20/d26/d30/d6b/d71/la8 0 2024-02-28T23:07:39.236 INFO:tasks.workunit.client.0.smithi057.stdout:9/485: rmdir dd/de/d89 39 2024-02-28T23:07:39.244 INFO:tasks.workunit.client.0.smithi057.stdout:2/544: chown d8/d11/d15/c81 2524 1 2024-02-28T23:07:39.247 INFO:tasks.workunit.client.0.smithi057.stdout:0/477: symlink d17/d20/d23/d47/lb0 0 2024-02-28T23:07:39.250 INFO:tasks.workunit.client.1.smithi084.stdout:7/603: read db/d10/d17/d64/f8c [1009770,122437] 0 2024-02-28T23:07:39.251 INFO:tasks.workunit.client.0.smithi057.stdout:2/545: fsync d8/d11/d56/f9d 0 2024-02-28T23:07:39.251 INFO:tasks.workunit.client.1.smithi084.stdout:1/645: chown d12/d1b/d2e/d52/d5b/fb3 7776 1 2024-02-28T23:07:39.253 INFO:tasks.workunit.client.1.smithi084.stdout:8/553: mknod d0/d44/cb9 0 2024-02-28T23:07:39.253 INFO:tasks.workunit.client.0.smithi057.stdout:4/431: creat d1a/d1b/f93 x:0 0 0 2024-02-28T23:07:39.254 INFO:tasks.workunit.client.1.smithi084.stdout:1/646: chown d12/d1b/d63/fbc 1438 1 2024-02-28T23:07:39.254 INFO:tasks.workunit.client.0.smithi057.stdout:5/467: dwrite d4/d32/f6e [0,4194304] 0 2024-02-28T23:07:39.255 INFO:tasks.workunit.client.1.smithi084.stdout:3/556: creat d3/d27/d7d/db0/df/fd1 x:0 0 0 2024-02-28T23:07:39.257 INFO:tasks.workunit.client.1.smithi084.stdout:0/567: getdents d1/d6/db/d35/d44 0 2024-02-28T23:07:39.258 INFO:tasks.workunit.client.0.smithi057.stdout:0/478: chown d17/d20/d4a/l57 18 1 2024-02-28T23:07:39.260 INFO:tasks.workunit.client.1.smithi084.stdout:8/554: mknod d0/d9/d1b/d20/cba 0 2024-02-28T23:07:39.260 INFO:tasks.workunit.client.1.smithi084.stdout:3/557: write d3/d27/d7d/db0/d1a/d29/d41/f71 [557637,108680] 0 2024-02-28T23:07:39.262 INFO:tasks.workunit.client.0.smithi057.stdout:4/432: chown c10 25935 1 2024-02-28T23:07:39.263 INFO:tasks.workunit.client.1.smithi084.stdout:0/568: chown d1/d8/d69/d6b/f77 1997577 1 2024-02-28T23:07:39.264 INFO:tasks.workunit.client.0.smithi057.stdout:2/546: mkdir d8/d11/d15/daf 0 2024-02-28T23:07:39.265 INFO:tasks.workunit.client.1.smithi084.stdout:5/619: getdents dc/de/df/d16/d34/d3c/d77 0 2024-02-28T23:07:39.266 INFO:tasks.workunit.client.1.smithi084.stdout:7/604: symlink db/d10/d17/d9c/d9f/db1/db2/lb3 0 2024-02-28T23:07:39.267 INFO:tasks.workunit.client.1.smithi084.stdout:9/598: link d4/d6/d15/d75/d2d/c38 d4/d6/d15/db3/cbf 0 2024-02-28T23:07:39.271 INFO:tasks.workunit.client.0.smithi057.stdout:9/486: link dd/de/d14/d48/d4b/l64 dd/d1b/d40/d44/d92/l9e 0 2024-02-28T23:07:39.274 INFO:tasks.workunit.client.1.smithi084.stdout:7/605: mkdir db/d10/d18/db4 0 2024-02-28T23:07:39.275 INFO:tasks.workunit.client.0.smithi057.stdout:7/425: dwrite d0/f23 [0,4194304] 0 2024-02-28T23:07:39.276 INFO:tasks.workunit.client.1.smithi084.stdout:0/569: creat d1/d8/fb4 x:0 0 0 2024-02-28T23:07:39.277 INFO:tasks.workunit.client.0.smithi057.stdout:3/343: dwrite d3/d8/d27/f5f [0,4194304] 0 2024-02-28T23:07:39.278 INFO:tasks.workunit.client.1.smithi084.stdout:9/599: creat d4/d6/d53/fc0 x:0 0 0 2024-02-28T23:07:39.279 INFO:tasks.workunit.client.1.smithi084.stdout:3/558: read d3/d27/d7d/db0/d1a/d21/d76/fb2 [387979,106796] 0 2024-02-28T23:07:39.279 INFO:tasks.workunit.client.0.smithi057.stdout:6/494: dwrite d0/d17/d20/f5e [0,4194304] 0 2024-02-28T23:07:39.280 INFO:tasks.workunit.client.1.smithi084.stdout:5/620: rename dc/de/df/d16/d34/d3c/d77/ccf to dc/de/df/d16/d34/d3c/d77/dd2/ceb 0 2024-02-28T23:07:39.284 INFO:tasks.workunit.client.1.smithi084.stdout:0/570: chown d1/d6/db/c42 3378977 1 2024-02-28T23:07:39.284 INFO:tasks.workunit.client.0.smithi057.stdout:2/547: symlink d8/d11/d39/lb0 0 2024-02-28T23:07:39.285 INFO:tasks.workunit.client.1.smithi084.stdout:9/600: rmdir d4/d6/d15/d75/d2d 39 2024-02-28T23:07:39.286 INFO:tasks.workunit.client.0.smithi057.stdout:8/503: dwrite d4/d1e/d56/d6a/d54/f70 [0,4194304] 0 2024-02-28T23:07:39.286 INFO:tasks.workunit.client.1.smithi084.stdout:5/621: chown dc/de/df/d16/d34/daa/dae/fc5 1100 1 2024-02-28T23:07:39.286 INFO:tasks.workunit.client.1.smithi084.stdout:8/555: dread d0/d44/d9f/f98 [0,4194304] 0 2024-02-28T23:07:39.287 INFO:tasks.workunit.client.0.smithi057.stdout:7/426: symlink d0/d5/d36/d4e/d3b/l84 0 2024-02-28T23:07:39.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/556: stat d0/d9/d1b/d38/d43 0 2024-02-28T23:07:39.293 INFO:tasks.workunit.client.1.smithi084.stdout:3/559: creat d3/d50/d97/fd2 x:0 0 0 2024-02-28T23:07:39.295 INFO:tasks.workunit.client.1.smithi084.stdout:5/622: creat dc/de/df/d16/d34/d3c/fec x:0 0 0 2024-02-28T23:07:39.296 INFO:tasks.workunit.client.1.smithi084.stdout:9/601: symlink d4/d6/d15/d12/lc1 0 2024-02-28T23:07:39.301 INFO:tasks.workunit.client.0.smithi057.stdout:0/479: link d17/d7a/lac d17/d19/d4d/d7b/lb1 0 2024-02-28T23:07:39.301 INFO:tasks.workunit.client.0.smithi057.stdout:8/504: rename d4/d13/d18/f90 to d4/d13/d2c/db1/fb2 0 2024-02-28T23:07:39.305 INFO:tasks.workunit.client.1.smithi084.stdout:3/560: dread - d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/f65 zero size 2024-02-28T23:07:39.307 INFO:tasks.workunit.client.1.smithi084.stdout:9/602: read d4/d6/d15/d12/d41/f65 [3146,108705] 0 2024-02-28T23:07:39.307 INFO:tasks.workunit.client.0.smithi057.stdout:1/353: sync 2024-02-28T23:07:39.308 INFO:tasks.workunit.client.1.smithi084.stdout:3/561: write d3/d50/f66 [1588188,83196] 0 2024-02-28T23:07:39.310 INFO:tasks.workunit.client.1.smithi084.stdout:5/623: creat dc/de/df/d31/dd1/d5f/ddc/fed x:0 0 0 2024-02-28T23:07:39.313 INFO:tasks.workunit.client.1.smithi084.stdout:5/624: stat dc/de/df/fa7 0 2024-02-28T23:07:39.314 INFO:tasks.workunit.client.0.smithi057.stdout:4/433: link d1a/d2f/c62 d1a/d1b/d24/d2e/d43/c94 0 2024-02-28T23:07:39.315 INFO:tasks.workunit.client.1.smithi084.stdout:8/557: getdents d0/d9/d30/d33 0 2024-02-28T23:07:39.315 INFO:tasks.workunit.client.1.smithi084.stdout:3/562: readlink d3/d27/d7d/db0/d14/l63 0 2024-02-28T23:07:39.319 INFO:tasks.workunit.client.0.smithi057.stdout:8/505: mkdir d4/d13/d1a/d4a/db3 0 2024-02-28T23:07:39.320 INFO:tasks.workunit.client.0.smithi057.stdout:4/434: fdatasync d1a/d1b/d24/f38 0 2024-02-28T23:07:39.320 INFO:tasks.workunit.client.0.smithi057.stdout:0/480: mknod d17/d20/d4a/cb2 0 2024-02-28T23:07:39.323 INFO:tasks.workunit.client.1.smithi084.stdout:9/603: mknod d4/d6/d15/d75/d9d/cc2 0 2024-02-28T23:07:39.324 INFO:tasks.workunit.client.0.smithi057.stdout:6/495: getdents d0/d4/d11 0 2024-02-28T23:07:39.325 INFO:tasks.workunit.client.1.smithi084.stdout:1/647: write d12/d1b/d2e/d32/f5e [25684,45803] 0 2024-02-28T23:07:39.329 INFO:tasks.workunit.client.0.smithi057.stdout:3/344: dwrite d3/d4/d1d/f49 [0,4194304] 0 2024-02-28T23:07:39.329 INFO:tasks.workunit.client.0.smithi057.stdout:1/354: creat d2/d71/d2d/d5b/f7c x:0 0 0 2024-02-28T23:07:39.332 INFO:tasks.workunit.client.1.smithi084.stdout:9/604: mkdir d4/d6/d15/d75/d9d/dc3 0 2024-02-28T23:07:39.337 INFO:tasks.workunit.client.1.smithi084.stdout:9/605: dread - d4/d6/d15/d12/d41/d55/fbb zero size 2024-02-28T23:07:39.338 INFO:tasks.workunit.client.1.smithi084.stdout:1/648: rename d12/dbe to d12/d1b/d63/d7d/dd3 0 2024-02-28T23:07:39.338 INFO:tasks.workunit.client.0.smithi057.stdout:6/496: mknod d0/d1b/caa 0 2024-02-28T23:07:39.341 INFO:tasks.workunit.client.1.smithi084.stdout:3/563: write d3/d27/d7d/db0/d14/d31/f5d [231878,124256] 0 2024-02-28T23:07:39.341 INFO:tasks.workunit.client.0.smithi057.stdout:2/548: dwrite d8/d11/d15/f80 [0,4194304] 0 2024-02-28T23:07:39.344 INFO:tasks.workunit.client.1.smithi084.stdout:5/625: dwrite dc/de/df/da6/fd3 [0,4194304] 0 2024-02-28T23:07:39.345 INFO:tasks.workunit.client.0.smithi057.stdout:7/427: dwrite d0/d5/d36/d4e/d15/d29/f79 [0,4194304] 0 2024-02-28T23:07:39.346 INFO:tasks.workunit.client.1.smithi084.stdout:0/571: dread d1/d8/f29 [0,4194304] 0 2024-02-28T23:07:39.347 INFO:tasks.workunit.client.0.smithi057.stdout:8/506: dread d4/d13/d2c/f69 [0,4194304] 0 2024-02-28T23:07:39.349 INFO:tasks.workunit.client.0.smithi057.stdout:6/497: readlink d0/d1b/d3e/d3d/d42/d4f/l96 0 2024-02-28T23:07:39.350 INFO:tasks.workunit.client.1.smithi084.stdout:9/606: unlink d4/d6/d15/d12/d41/fb5 0 2024-02-28T23:07:39.352 INFO:tasks.workunit.client.1.smithi084.stdout:0/572: mknod d1/d8/cb5 0 2024-02-28T23:07:39.354 INFO:tasks.workunit.client.0.smithi057.stdout:4/435: write d1a/d1e/d4b/d4f/f53 [942076,126644] 0 2024-02-28T23:07:39.355 INFO:tasks.workunit.client.0.smithi057.stdout:6/498: write d0/d17/f35 [2329481,21641] 0 2024-02-28T23:07:39.355 INFO:tasks.workunit.client.0.smithi057.stdout:5/468: dwrite d4/d21/d24/d72/f8e [0,4194304] 0 2024-02-28T23:07:39.355 INFO:tasks.workunit.client.0.smithi057.stdout:0/481: symlink d17/d20/d23/d47/d60/d6f/lb3 0 2024-02-28T23:07:39.357 INFO:tasks.workunit.client.1.smithi084.stdout:0/573: fsync d1/d6/db/d35/d82/f83 0 2024-02-28T23:07:39.360 INFO:tasks.workunit.client.1.smithi084.stdout:7/606: dread db/d10/d17/d9c/d9f/db1/f35 [0,4194304] 0 2024-02-28T23:07:39.361 INFO:tasks.workunit.client.0.smithi057.stdout:0/482: stat d17/d20/d4a/c4f 0 2024-02-28T23:07:39.362 INFO:tasks.workunit.client.0.smithi057.stdout:5/469: chown d4/d21/d24/f51 25236 1 2024-02-28T23:07:39.363 INFO:tasks.workunit.client.1.smithi084.stdout:3/564: creat d3/d27/d7d/db0/d1a/d45/fd3 x:0 0 0 2024-02-28T23:07:39.363 INFO:tasks.workunit.client.0.smithi057.stdout:7/428: chown d0/c31 349 1 2024-02-28T23:07:39.364 INFO:tasks.workunit.client.1.smithi084.stdout:7/607: fdatasync db/d5e/f73 0 2024-02-28T23:07:39.365 INFO:tasks.workunit.client.1.smithi084.stdout:0/574: rmdir d1/d6/db/d37 39 2024-02-28T23:07:39.365 INFO:tasks.workunit.client.0.smithi057.stdout:8/507: mknod d4/d1e/d56/d88/cb4 0 2024-02-28T23:07:39.365 INFO:tasks.workunit.client.0.smithi057.stdout:6/499: unlink d0/d17/f2b 0 2024-02-28T23:07:39.366 INFO:tasks.workunit.client.1.smithi084.stdout:9/607: write d4/d6/d15/f8e [2821266,123620] 0 2024-02-28T23:07:39.369 INFO:tasks.workunit.client.0.smithi057.stdout:5/470: read - d4/d9/d16/f40 zero size 2024-02-28T23:07:39.370 INFO:tasks.workunit.client.1.smithi084.stdout:4/515: sync 2024-02-28T23:07:39.376 INFO:tasks.workunit.client.0.smithi057.stdout:0/483: mknod d17/d20/d26/d30/d6b/d71/cb4 0 2024-02-28T23:07:39.376 INFO:tasks.workunit.client.0.smithi057.stdout:3/345: getdents d3/d4f/d4c 0 2024-02-28T23:07:39.376 INFO:tasks.workunit.client.1.smithi084.stdout:7/608: creat db/d10/d17/d64/fb5 x:0 0 0 2024-02-28T23:07:39.378 INFO:tasks.workunit.client.1.smithi084.stdout:7/609: stat db/d11/f5b 0 2024-02-28T23:07:39.379 INFO:tasks.workunit.client.0.smithi057.stdout:6/500: rename d0/d1b/d3e/l23 to d0/d1b/d3e/d3d/d74/lab 0 2024-02-28T23:07:39.380 INFO:tasks.workunit.client.1.smithi084.stdout:7/610: fdatasync db/fd 0 2024-02-28T23:07:39.381 INFO:tasks.workunit.client.0.smithi057.stdout:5/471: readlink d4/d9/d14/d3b/l6b 0 2024-02-28T23:07:39.382 INFO:tasks.workunit.client.0.smithi057.stdout:7/429: creat d0/f85 x:0 0 0 2024-02-28T23:07:39.383 INFO:tasks.workunit.client.0.smithi057.stdout:6/501: write d0/d1b/d3e/d40/f4d [311213,80316] 0 2024-02-28T23:07:39.384 INFO:tasks.workunit.client.1.smithi084.stdout:7/611: chown db/d10/d17/d3b/f4d 105665 1 2024-02-28T23:07:39.387 INFO:tasks.workunit.client.1.smithi084.stdout:1/649: dread d12/f97 [0,4194304] 0 2024-02-28T23:07:39.387 INFO:tasks.workunit.client.0.smithi057.stdout:6/502: readlink d0/d17/l1a 0 2024-02-28T23:07:39.388 INFO:tasks.workunit.client.0.smithi057.stdout:9/487: dread dd/de/d24/f28 [0,4194304] 0 2024-02-28T23:07:39.391 INFO:tasks.workunit.client.1.smithi084.stdout:5/626: write dc/f25 [1851164,101188] 0 2024-02-28T23:07:39.391 INFO:tasks.workunit.client.0.smithi057.stdout:0/484: mkdir d17/d20/d4a/d78/db5 0 2024-02-28T23:07:39.394 INFO:tasks.workunit.client.1.smithi084.stdout:9/608: creat d4/d6/d15/d75/d2d/fc4 x:0 0 0 2024-02-28T23:07:39.395 INFO:tasks.workunit.client.0.smithi057.stdout:5/472: symlink d4/d9/d16/d52/l9b 0 2024-02-28T23:07:39.395 INFO:tasks.workunit.client.1.smithi084.stdout:7/612: mknod db/d11/cb6 0 2024-02-28T23:07:39.396 INFO:tasks.workunit.client.0.smithi057.stdout:0/485: read - d17/d20/d26/d30/fad zero size 2024-02-28T23:07:39.396 INFO:tasks.workunit.client.1.smithi084.stdout:3/565: link d3/d50/d5a/ca2 d3/d50/cd4 0 2024-02-28T23:07:39.397 INFO:tasks.workunit.client.1.smithi084.stdout:9/609: write d4/d6/d15/f71 [2161409,128882] 0 2024-02-28T23:07:39.399 INFO:tasks.workunit.client.1.smithi084.stdout:7/613: dread - db/d10/d18/f82 zero size 2024-02-28T23:07:39.400 INFO:tasks.workunit.client.0.smithi057.stdout:3/346: symlink d3/d8/l6e 0 2024-02-28T23:07:39.401 INFO:tasks.workunit.client.1.smithi084.stdout:0/575: getdents d1/d6/db/d76 0 2024-02-28T23:07:39.403 INFO:tasks.workunit.client.1.smithi084.stdout:3/566: creat d3/d50/fd5 x:0 0 0 2024-02-28T23:07:39.403 INFO:tasks.workunit.client.1.smithi084.stdout:4/516: link d1/ca d1/d18/d20/d24/d2f/d3e/d52/d57/cae 0 2024-02-28T23:07:39.403 INFO:tasks.workunit.client.1.smithi084.stdout:7/614: readlink db/d10/d17/l7f 0 2024-02-28T23:07:39.404 INFO:tasks.workunit.client.1.smithi084.stdout:5/627: truncate dc/de/df/d16/d34/d3c/fb8 910663 0 2024-02-28T23:07:39.404 INFO:tasks.workunit.client.0.smithi057.stdout:8/508: dread d4/d13/d18/f31 [0,4194304] 0 2024-02-28T23:07:39.405 INFO:tasks.workunit.client.1.smithi084.stdout:4/517: readlink d1/lb 0 2024-02-28T23:07:39.405 INFO:tasks.workunit.client.0.smithi057.stdout:9/488: mkdir dd/de/d4a/d9f 0 2024-02-28T23:07:39.405 INFO:tasks.workunit.client.0.smithi057.stdout:1/355: truncate d2/f35 2503876 0 2024-02-28T23:07:39.406 INFO:tasks.workunit.client.1.smithi084.stdout:9/610: creat d4/d6/d15/d12/d41/fc5 x:0 0 0 2024-02-28T23:07:39.406 INFO:tasks.workunit.client.1.smithi084.stdout:5/628: readlink dc/d51/ld6 0 2024-02-28T23:07:39.407 INFO:tasks.workunit.client.0.smithi057.stdout:4/436: dread d1a/d1b/f3e [0,4194304] 0 2024-02-28T23:07:39.408 INFO:tasks.workunit.client.0.smithi057.stdout:0/486: truncate d17/d19/f3c 4919024 0 2024-02-28T23:07:39.409 INFO:tasks.workunit.client.1.smithi084.stdout:3/567: chown d3/d27/d7d/db0/d1a/d21/d7b/dbf/fc7 298311 1 2024-02-28T23:07:39.409 INFO:tasks.workunit.client.1.smithi084.stdout:0/576: getdents d1/d8/d5f/d93 0 2024-02-28T23:07:39.410 INFO:tasks.workunit.client.1.smithi084.stdout:5/629: dread - dc/de/df/d16/f81 zero size 2024-02-28T23:07:39.411 INFO:tasks.workunit.client.0.smithi057.stdout:3/347: write d3/d8/d1b/f37 [139806,19261] 0 2024-02-28T23:07:39.413 INFO:tasks.workunit.client.0.smithi057.stdout:1/356: stat d2/d7/df/db/d3a/d2a/f6d 0 2024-02-28T23:07:39.414 INFO:tasks.workunit.client.1.smithi084.stdout:4/518: chown d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d 105871808 1 2024-02-28T23:07:39.416 INFO:tasks.workunit.client.0.smithi057.stdout:6/503: write d0/d17/d94/da0/fa3 [2657777,104237] 0 2024-02-28T23:07:39.417 INFO:tasks.workunit.client.0.smithi057.stdout:7/430: dwrite d0/d3f/f7f [0,4194304] 0 2024-02-28T23:07:39.424 INFO:tasks.workunit.client.1.smithi084.stdout:0/577: creat d1/d6/db/d37/d72/fb6 x:0 0 0 2024-02-28T23:07:39.424 INFO:tasks.workunit.client.1.smithi084.stdout:7/615: dread db/d10/d17/d64/f8c [0,4194304] 0 2024-02-28T23:07:39.424 INFO:tasks.workunit.client.1.smithi084.stdout:4/519: fdatasync d1/d18/d20/d24/d2f/d3e/d52/d45/f92 0 2024-02-28T23:07:39.424 INFO:tasks.workunit.client.1.smithi084.stdout:0/578: truncate d1/d6/f13 4654324 0 2024-02-28T23:07:39.432 INFO:tasks.workunit.client.1.smithi084.stdout:7/616: mkdir db/d10/d18/db7 0 2024-02-28T23:07:39.433 INFO:tasks.workunit.client.0.smithi057.stdout:4/437: fsync d1a/d1e/f5b 0 2024-02-28T23:07:39.434 INFO:tasks.workunit.client.0.smithi057.stdout:5/473: link d4/d9/d16/c29 d4/d21/c9c 0 2024-02-28T23:07:39.436 INFO:tasks.workunit.client.1.smithi084.stdout:4/520: rename d1/d18/d20/d24/d2f/c3f to d1/d18/d20/d14/d30/d47/da4/caf 0 2024-02-28T23:07:39.436 INFO:tasks.workunit.client.1.smithi084.stdout:5/630: dwrite dc/de/df/d16/f9a [0,4194304] 0 2024-02-28T23:07:39.437 INFO:tasks.workunit.client.1.smithi084.stdout:1/650: dread d12/d1b/d2e/d30/f26 [0,4194304] 0 2024-02-28T23:07:39.438 INFO:tasks.workunit.client.1.smithi084.stdout:0/579: mknod d1/d6/d1b/cb7 0 2024-02-28T23:07:39.438 INFO:tasks.workunit.client.0.smithi057.stdout:8/509: rename d4/d13/d1a/c1d to d4/d1e/d56/d6a/d42/cb5 0 2024-02-28T23:07:39.440 INFO:tasks.workunit.client.0.smithi057.stdout:1/357: creat d2/d71/d2d/d5b/f7d x:0 0 0 2024-02-28T23:07:39.442 INFO:tasks.workunit.client.0.smithi057.stdout:4/438: mknod d1a/d1e/d51/c95 0 2024-02-28T23:07:39.447 INFO:tasks.workunit.client.0.smithi057.stdout:3/348: write d3/d20/f24 [4261107,84231] 0 2024-02-28T23:07:39.447 INFO:tasks.workunit.client.0.smithi057.stdout:2/549: dread d8/d11/d21/d28/d41/f5c [0,4194304] 0 2024-02-28T23:07:39.447 INFO:tasks.workunit.client.0.smithi057.stdout:4/439: readlink d1a/d1b/d24/l65 0 2024-02-28T23:07:39.448 INFO:tasks.workunit.client.0.smithi057.stdout:5/474: symlink d4/d9/d14/d3b/d61/d69/d95/l9d 0 2024-02-28T23:07:39.449 INFO:tasks.workunit.client.0.smithi057.stdout:7/431: mknod d0/d5/d36/d4e/d15/c86 0 2024-02-28T23:07:39.450 INFO:tasks.workunit.client.1.smithi084.stdout:1/651: write d12/d1b/d2e/d52/d61/d74/f88 [2669095,109687] 0 2024-02-28T23:07:39.454 INFO:tasks.workunit.client.1.smithi084.stdout:4/521: unlink d1/d18/d20/d24/d2f/d3e/d52/d48/c67 0 2024-02-28T23:07:39.456 INFO:tasks.workunit.client.1.smithi084.stdout:1/652: mkdir d12/d1b/d91/d9b/da5/dcb/dd4 0 2024-02-28T23:07:39.459 INFO:tasks.workunit.client.1.smithi084.stdout:4/522: write d1/d4a/f54 [1845469,33670] 0 2024-02-28T23:07:39.461 INFO:tasks.workunit.client.1.smithi084.stdout:9/611: dread d4/d6/d15/f2a [0,4194304] 0 2024-02-28T23:07:39.462 INFO:tasks.workunit.client.0.smithi057.stdout:4/440: unlink d1a/d1b/f1f 0 2024-02-28T23:07:39.462 INFO:tasks.workunit.client.0.smithi057.stdout:2/550: rmdir d8/d11/d39/d7b 39 2024-02-28T23:07:39.465 INFO:tasks.workunit.client.0.smithi057.stdout:1/358: creat d2/f7e x:0 0 0 2024-02-28T23:07:39.466 INFO:tasks.workunit.client.1.smithi084.stdout:2/518: sync 2024-02-28T23:07:39.466 INFO:tasks.workunit.client.1.smithi084.stdout:3/568: dwrite d3/fbd [0,4194304] 0 2024-02-28T23:07:39.472 INFO:tasks.workunit.client.1.smithi084.stdout:4/523: symlink d1/d18/d20/d14/d30/d47/lb0 0 2024-02-28T23:07:39.474 INFO:tasks.workunit.client.1.smithi084.stdout:7/617: truncate db/d10/faa 212589 0 2024-02-28T23:07:39.476 INFO:tasks.workunit.client.1.smithi084.stdout:2/519: fsync d5/d64/f6c 0 2024-02-28T23:07:39.476 INFO:tasks.workunit.client.1.smithi084.stdout:5/631: dwrite dc/de/df/d31/d8e/da9/fce [0,4194304] 0 2024-02-28T23:07:39.477 INFO:tasks.workunit.client.0.smithi057.stdout:9/489: dwrite dd/de/d14/d48/d4b/f54 [0,4194304] 0 2024-02-28T23:07:39.478 INFO:tasks.workunit.client.1.smithi084.stdout:4/524: chown d1/d18/d20/d65/d8e/d55/c96 58 1 2024-02-28T23:07:39.480 INFO:tasks.workunit.client.1.smithi084.stdout:3/569: rename d3/d27/d7d/db0/d14/fa5 to d3/d27/d4b/fd6 0 2024-02-28T23:07:39.480 INFO:tasks.workunit.client.0.smithi057.stdout:2/551: rename d8/d11/d15/f2d to d8/d11/d48/d50/fb1 0 2024-02-28T23:07:39.482 INFO:tasks.workunit.client.0.smithi057.stdout:1/359: dread d2/d71/f4a [0,4194304] 0 2024-02-28T23:07:39.483 INFO:tasks.workunit.client.0.smithi057.stdout:7/432: getdents d0/d5 0 2024-02-28T23:07:39.484 INFO:tasks.workunit.client.1.smithi084.stdout:2/520: creat d5/d1f/d83/f97 x:0 0 0 2024-02-28T23:07:39.484 INFO:tasks.workunit.client.1.smithi084.stdout:7/618: symlink db/d10/d17/d3b/lb8 0 2024-02-28T23:07:39.486 INFO:tasks.workunit.client.1.smithi084.stdout:0/580: truncate d1/d6/d59/d70/f73 946969 0 2024-02-28T23:07:39.489 INFO:tasks.workunit.client.1.smithi084.stdout:7/619: read - db/d10/d17/f8f zero size 2024-02-28T23:07:39.490 INFO:tasks.workunit.client.0.smithi057.stdout:0/487: dread d17/d19/f2f [0,4194304] 0 2024-02-28T23:07:39.490 INFO:tasks.workunit.client.0.smithi057.stdout:5/475: dwrite f2 [4194304,4194304] 0 2024-02-28T23:07:39.492 INFO:tasks.workunit.client.1.smithi084.stdout:2/521: write d5/d64/f6c [465741,125437] 0 2024-02-28T23:07:39.492 INFO:tasks.workunit.client.1.smithi084.stdout:9/612: dwrite d4/d6/d15/f62 [0,4194304] 0 2024-02-28T23:07:39.492 INFO:tasks.workunit.client.1.smithi084.stdout:4/525: mknod d1/d18/d20/d24/d2f/d3e/d52/d57/cb1 0 2024-02-28T23:07:39.494 INFO:tasks.workunit.client.1.smithi084.stdout:1/653: truncate d12/f70 1483960 0 2024-02-28T23:07:39.494 INFO:tasks.workunit.client.1.smithi084.stdout:3/570: mknod d3/d27/d7d/db0/d14/d31/d34/cd7 0 2024-02-28T23:07:39.494 INFO:tasks.workunit.client.1.smithi084.stdout:9/613: readlink l1 0 2024-02-28T23:07:39.496 INFO:tasks.workunit.client.1.smithi084.stdout:2/522: chown d5/d1f/d42/d8f 6816 1 2024-02-28T23:07:39.496 INFO:tasks.workunit.client.1.smithi084.stdout:7/620: chown db/d10/d17/d64/f8c 851 1 2024-02-28T23:07:39.497 INFO:tasks.workunit.client.0.smithi057.stdout:3/349: dwrite d3/d8/d1b/f37 [0,4194304] 0 2024-02-28T23:07:39.497 INFO:tasks.workunit.client.0.smithi057.stdout:9/490: creat dd/d46/d6f/fa0 x:0 0 0 2024-02-28T23:07:39.499 INFO:tasks.workunit.client.0.smithi057.stdout:1/360: unlink d2/d7/df/db/f18 0 2024-02-28T23:07:39.499 INFO:tasks.workunit.client.0.smithi057.stdout:2/552: mknod d8/d11/d21/d70/d91/cb2 0 2024-02-28T23:07:39.500 INFO:tasks.workunit.client.1.smithi084.stdout:7/621: stat db/d10/d17/d64/fb5 0 2024-02-28T23:07:39.503 INFO:tasks.workunit.client.1.smithi084.stdout:9/614: read d4/d6/d15/f98 [3533623,120723] 0 2024-02-28T23:07:39.508 INFO:tasks.workunit.client.0.smithi057.stdout:4/441: getdents d1a/d1b/d24/d2e 0 2024-02-28T23:07:39.508 INFO:tasks.workunit.client.0.smithi057.stdout:7/433: mknod d0/d3f/d76/c87 0 2024-02-28T23:07:39.509 INFO:tasks.workunit.client.1.smithi084.stdout:5/632: dread dc/de/f14 [4194304,4194304] 0 2024-02-28T23:07:39.509 INFO:tasks.workunit.client.1.smithi084.stdout:1/654: rmdir d12/d1b/d2e/d32 39 2024-02-28T23:07:39.511 INFO:tasks.workunit.client.0.smithi057.stdout:7/434: chown d0/d5/d4a 11 1 2024-02-28T23:07:39.511 INFO:tasks.workunit.client.0.smithi057.stdout:6/504: dread d0/f4a [0,4194304] 0 2024-02-28T23:07:39.515 INFO:tasks.workunit.client.1.smithi084.stdout:1/655: stat d12/d84/d9a/fc0 0 2024-02-28T23:07:39.518 INFO:tasks.workunit.client.1.smithi084.stdout:6/566: sync 2024-02-28T23:07:39.519 INFO:tasks.workunit.client.1.smithi084.stdout:8/558: sync 2024-02-28T23:07:39.522 INFO:tasks.workunit.client.1.smithi084.stdout:4/526: mknod d1/d18/d20/d65/d8e/d55/cb2 0 2024-02-28T23:07:39.527 INFO:tasks.workunit.client.0.smithi057.stdout:4/442: symlink d1a/d1b/d2d/d49/d4d/l96 0 2024-02-28T23:07:39.528 INFO:tasks.workunit.client.1.smithi084.stdout:7/622: rename db/d10/d17/d9c/d9f/dad to db/d10/d17/d9c/db9 0 2024-02-28T23:07:39.530 INFO:tasks.workunit.client.0.smithi057.stdout:4/443: stat d1a/l50 0 2024-02-28T23:07:39.531 INFO:tasks.workunit.client.0.smithi057.stdout:7/435: write d0/d5/d36/d4e/d15/f3a [3855255,52914] 0 2024-02-28T23:07:39.532 INFO:tasks.workunit.client.1.smithi084.stdout:2/523: dwrite d5/d64/f73 [0,4194304] 0 2024-02-28T23:07:39.536 INFO:tasks.workunit.client.1.smithi084.stdout:5/633: mkdir dc/de/df/d31/dd1/da4/dee 0 2024-02-28T23:07:39.536 INFO:tasks.workunit.client.1.smithi084.stdout:3/571: mkdir d3/d50/d5a/d73/dd8 0 2024-02-28T23:07:39.536 INFO:tasks.workunit.client.0.smithi057.stdout:7/436: dread - d0/f85 zero size 2024-02-28T23:07:39.541 INFO:tasks.workunit.client.1.smithi084.stdout:0/581: dread d1/d6/db/d37/f96 [0,4194304] 0 2024-02-28T23:07:39.541 INFO:tasks.workunit.client.0.smithi057.stdout:5/476: write d4/f6 [1048331,94928] 0 2024-02-28T23:07:39.546 INFO:tasks.workunit.client.0.smithi057.stdout:6/505: dwrite d0/d17/f35 [0,4194304] 0 2024-02-28T23:07:39.548 INFO:tasks.workunit.client.0.smithi057.stdout:5/477: write d4/d9/d14/d3b/f84 [642969,12292] 0 2024-02-28T23:07:39.550 INFO:tasks.workunit.client.0.smithi057.stdout:2/553: rmdir d8/d11/d21/d70/d91/d94 0 2024-02-28T23:07:39.554 INFO:tasks.workunit.client.1.smithi084.stdout:5/634: stat dc/de/df/d16/d34/d3c/d77/dd2/ceb 0 2024-02-28T23:07:39.560 INFO:tasks.workunit.client.1.smithi084.stdout:2/524: creat d5/d7/d9/d94/f98 x:0 0 0 2024-02-28T23:07:39.561 INFO:tasks.workunit.client.0.smithi057.stdout:5/478: read d4/d9/d16/f1a [544461,19005] 0 2024-02-28T23:07:39.561 INFO:tasks.workunit.client.0.smithi057.stdout:6/506: write d0/d4/d25/f71 [3813917,85754] 0 2024-02-28T23:07:39.562 INFO:tasks.workunit.client.0.smithi057.stdout:1/361: dwrite d2/d71/d2d/f49 [4194304,4194304] 0 2024-02-28T23:07:39.562 INFO:tasks.workunit.client.0.smithi057.stdout:4/444: rename d1a/d1b to d1a/d48/d97 0 2024-02-28T23:07:39.562 INFO:tasks.workunit.client.0.smithi057.stdout:0/488: dread d17/d20/d26/d48/d55/f93 [0,4194304] 0 2024-02-28T23:07:39.562 INFO:tasks.workunit.client.1.smithi084.stdout:3/572: truncate d3/d27/d7d/db0/d1a/d29/d41/d60/f6b 4807664 0 2024-02-28T23:07:39.562 INFO:tasks.workunit.client.1.smithi084.stdout:6/567: dwrite d1/d13/d2e/d41/f2f [0,4194304] 0 2024-02-28T23:07:39.567 INFO:tasks.workunit.client.1.smithi084.stdout:1/656: dwrite d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:39.567 INFO:tasks.workunit.client.0.smithi057.stdout:1/362: write d2/d7/df/db/f28 [4729865,95438] 0 2024-02-28T23:07:39.568 INFO:tasks.workunit.client.0.smithi057.stdout:3/350: link d3/d4/d7/l5b d3/d4/l6f 0 2024-02-28T23:07:39.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:39 smithi057 ceph-mon[29096]: pgmap v26: 65 pgs: 65 active+clean; 1.8 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 248 MiB/s wr, 393 op/s 2024-02-28T23:07:39.569 INFO:tasks.workunit.client.0.smithi057.stdout:0/489: write d17/d20/d26/f46 [802054,7142] 0 2024-02-28T23:07:39.569 INFO:tasks.workunit.client.0.smithi057.stdout:1/363: dread - d2/d33/f76 zero size 2024-02-28T23:07:39.571 INFO:tasks.workunit.client.1.smithi084.stdout:9/615: dwrite d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:39.572 INFO:tasks.workunit.client.0.smithi057.stdout:5/479: unlink d4/d21/d24/f22 0 2024-02-28T23:07:39.573 INFO:tasks.workunit.client.0.smithi057.stdout:0/490: write d17/d19/f3c [2424189,94250] 0 2024-02-28T23:07:39.574 INFO:tasks.workunit.client.0.smithi057.stdout:6/507: write d0/d4/d11/fa1 [4494440,1929] 0 2024-02-28T23:07:39.577 INFO:tasks.workunit.client.1.smithi084.stdout:4/527: rename d1/d18/d20/l89 to d1/d18/d20/d24/d2f/d3e/d52/d45/lb3 0 2024-02-28T23:07:39.578 INFO:tasks.workunit.client.0.smithi057.stdout:2/554: unlink d8/d11/c44 0 2024-02-28T23:07:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:39 smithi084 ceph-mon[34826]: pgmap v26: 65 pgs: 65 active+clean; 1.8 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 248 MiB/s wr, 393 op/s 2024-02-28T23:07:39.580 INFO:tasks.workunit.client.1.smithi084.stdout:2/525: mknod d5/dd/d69/c99 0 2024-02-28T23:07:39.581 INFO:tasks.workunit.client.1.smithi084.stdout:5/635: write dc/de/df/d16/f95 [4729213,66506] 0 2024-02-28T23:07:39.583 INFO:tasks.workunit.client.0.smithi057.stdout:6/508: fsync d0/d1b/d3e/d40/f73 0 2024-02-28T23:07:39.586 INFO:tasks.workunit.client.0.smithi057.stdout:6/509: fdatasync d0/f6c 0 2024-02-28T23:07:39.588 INFO:tasks.workunit.client.0.smithi057.stdout:3/351: mkdir d3/d8/d27/d70 0 2024-02-28T23:07:39.588 INFO:tasks.workunit.client.1.smithi084.stdout:3/573: rename d3/d27/d7d/da0 to d3/d27/d7d/dd9 0 2024-02-28T23:07:39.588 INFO:tasks.workunit.client.1.smithi084.stdout:1/657: unlink d12/d1b/f2c 0 2024-02-28T23:07:39.592 INFO:tasks.workunit.client.1.smithi084.stdout:6/568: mkdir d1/d13/d50/dbd/dc4 0 2024-02-28T23:07:39.593 INFO:tasks.workunit.client.1.smithi084.stdout:7/623: dread f9 [0,4194304] 0 2024-02-28T23:07:39.594 INFO:tasks.workunit.client.0.smithi057.stdout:5/480: stat d4/d9/d16/f19 0 2024-02-28T23:07:39.594 INFO:tasks.workunit.client.0.smithi057.stdout:4/445: truncate d1a/d48/d97/d2d/d49/d4d/f55 7636 0 2024-02-28T23:07:39.597 INFO:tasks.workunit.client.0.smithi057.stdout:9/491: dread dd/de/d14/f1f [0,4194304] 0 2024-02-28T23:07:39.598 INFO:tasks.workunit.client.1.smithi084.stdout:2/526: write d5/f17 [4546521,64759] 0 2024-02-28T23:07:39.599 INFO:tasks.workunit.client.1.smithi084.stdout:4/528: rename d1/d18/d20/d65/d8e/d55/da7 to d1/d18/d20/d24/d2f/d3e/d52/d87/da5/db4 0 2024-02-28T23:07:39.602 INFO:tasks.workunit.client.0.smithi057.stdout:7/437: rename d0/d3f/d76/f7c to d0/d5/d36/d4e/d15/d29/d3d/f88 0 2024-02-28T23:07:39.602 INFO:tasks.workunit.client.1.smithi084.stdout:5/636: unlink dc/de/df/d31/dd1/da4/fbc 0 2024-02-28T23:07:39.603 INFO:tasks.workunit.client.0.smithi057.stdout:3/352: symlink d3/d8/d1b/l71 0 2024-02-28T23:07:39.605 INFO:tasks.workunit.client.0.smithi057.stdout:0/491: getdents d17/d20/d26/d48/d7c/d98/da6 0 2024-02-28T23:07:39.608 INFO:tasks.workunit.client.0.smithi057.stdout:7/438: fsync d0/d5/d36/d4e/f83 0 2024-02-28T23:07:39.609 INFO:tasks.workunit.client.1.smithi084.stdout:3/574: mknod d3/d27/d7d/db0/d1a/d29/d41/cda 0 2024-02-28T23:07:39.609 INFO:tasks.workunit.client.0.smithi057.stdout:9/492: read dd/d1b/d40/d44/f7a [284283,113120] 0 2024-02-28T23:07:39.611 INFO:tasks.workunit.client.0.smithi057.stdout:3/353: fdatasync d3/d8/d27/f5f 0 2024-02-28T23:07:39.611 INFO:tasks.workunit.client.1.smithi084.stdout:8/559: dwrite d0/f1a [4194304,4194304] 0 2024-02-28T23:07:39.612 INFO:tasks.workunit.client.1.smithi084.stdout:9/616: truncate d4/d6/f29 1565244 0 2024-02-28T23:07:39.612 INFO:tasks.workunit.client.1.smithi084.stdout:6/569: rmdir d1/d13/d47/d91/dba 39 2024-02-28T23:07:39.618 INFO:tasks.workunit.client.1.smithi084.stdout:2/527: creat d5/d7/d2e/f9a x:0 0 0 2024-02-28T23:07:39.619 INFO:tasks.workunit.client.0.smithi057.stdout:3/354: rmdir d3/d4/d1d/d47 39 2024-02-28T23:07:39.619 INFO:tasks.workunit.client.0.smithi057.stdout:1/364: dwrite d2/f7e [0,4194304] 0 2024-02-28T23:07:39.620 INFO:tasks.workunit.client.1.smithi084.stdout:7/624: creat db/d10/d18/db7/fba x:0 0 0 2024-02-28T23:07:39.620 INFO:tasks.workunit.client.1.smithi084.stdout:6/570: readlink d1/d9/d1d/d1e/d33/lb5 0 2024-02-28T23:07:39.626 INFO:tasks.workunit.client.0.smithi057.stdout:1/365: creat d2/d7/df/db/d3a/f7f x:0 0 0 2024-02-28T23:07:39.627 INFO:tasks.workunit.client.1.smithi084.stdout:3/575: rename d3/d27/d7d/db0/d1a/d45/d5e/d7a/l81 to d3/d27/d7d/db0/d14/d31/ldb 0 2024-02-28T23:07:39.628 INFO:tasks.workunit.client.0.smithi057.stdout:4/446: dread d1a/d48/d97/f3e [0,4194304] 0 2024-02-28T23:07:39.628 INFO:tasks.workunit.client.1.smithi084.stdout:9/617: creat d4/d6/d15/d75/d2d/daa/fc6 x:0 0 0 2024-02-28T23:07:39.629 INFO:tasks.workunit.client.1.smithi084.stdout:7/625: truncate db/d10/d17/d64/f93 844922 0 2024-02-28T23:07:39.634 INFO:tasks.workunit.client.1.smithi084.stdout:7/626: readlink db/d5e/l7e 0 2024-02-28T23:07:39.636 INFO:tasks.workunit.client.0.smithi057.stdout:4/447: truncate d1a/d48/d97/f8a 279881 0 2024-02-28T23:07:39.637 INFO:tasks.workunit.client.1.smithi084.stdout:6/571: dread d1/d13/d2e/d41/d4d/d6c/f7c [0,4194304] 0 2024-02-28T23:07:39.645 INFO:tasks.workunit.client.1.smithi084.stdout:7/627: write db/d10/d17/f42 [3739789,14229] 0 2024-02-28T23:07:39.645 INFO:tasks.workunit.client.1.smithi084.stdout:3/576: creat d3/d50/d5a/d73/db8/fdc x:0 0 0 2024-02-28T23:07:39.646 INFO:tasks.workunit.client.0.smithi057.stdout:9/493: dread dd/de/d24/f28 [0,4194304] 0 2024-02-28T23:07:39.649 INFO:tasks.workunit.client.0.smithi057.stdout:3/355: link d3/d4f/l43 d3/d4/d7/l72 0 2024-02-28T23:07:39.650 INFO:tasks.workunit.client.0.smithi057.stdout:4/448: fsync d1a/d1e/d4b/d4f/f53 0 2024-02-28T23:07:39.651 INFO:tasks.workunit.client.1.smithi084.stdout:2/528: dwrite d5/d7/d1d/f50 [0,4194304] 0 2024-02-28T23:07:39.651 INFO:tasks.workunit.client.1.smithi084.stdout:8/560: link d0/d9/d1b/d38/d43/l4f d0/d44/lbb 0 2024-02-28T23:07:39.653 INFO:tasks.workunit.client.0.smithi057.stdout:6/510: write d0/d4/d11/f9a [729395,63043] 0 2024-02-28T23:07:39.653 INFO:tasks.workunit.client.0.smithi057.stdout:7/439: write d0/d5/f39 [1412413,23745] 0 2024-02-28T23:07:39.653 INFO:tasks.workunit.client.0.smithi057.stdout:1/366: mkdir d2/d7/df/db/d12/d19/d80 0 2024-02-28T23:07:39.658 INFO:tasks.workunit.client.0.smithi057.stdout:3/356: fdatasync d3/d8/d1b/f44 0 2024-02-28T23:07:39.659 INFO:tasks.workunit.client.0.smithi057.stdout:4/449: creat d1a/d48/d97/d24/d2e/f98 x:0 0 0 2024-02-28T23:07:39.660 INFO:tasks.workunit.client.1.smithi084.stdout:0/582: dwrite d1/d6/db/d35/d44/f4b [0,4194304] 0 2024-02-28T23:07:39.660 INFO:tasks.workunit.client.1.smithi084.stdout:7/628: write db/d10/f56 [1934124,120939] 0 2024-02-28T23:07:39.660 INFO:tasks.workunit.client.0.smithi057.stdout:6/511: creat d0/d1b/d3e/d3d/d42/d4f/fac x:0 0 0 2024-02-28T23:07:39.662 INFO:tasks.workunit.client.0.smithi057.stdout:1/367: write d2/d7/df/db/d3a/d2a/f2e [1301892,108144] 0 2024-02-28T23:07:39.667 INFO:tasks.workunit.client.0.smithi057.stdout:9/494: mknod dd/d46/ca1 0 2024-02-28T23:07:39.668 INFO:tasks.workunit.client.1.smithi084.stdout:8/561: mknod d0/d9/d30/d33/d71/cbc 0 2024-02-28T23:07:39.669 INFO:tasks.workunit.client.0.smithi057.stdout:5/481: dread d4/d21/f37 [0,4194304] 0 2024-02-28T23:07:39.671 INFO:tasks.workunit.client.1.smithi084.stdout:7/629: mkdir db/d10/d18/db7/dbb 0 2024-02-28T23:07:39.671 INFO:tasks.workunit.client.1.smithi084.stdout:4/529: dread d1/d18/d20/d65/d8e/f5a [0,4194304] 0 2024-02-28T23:07:39.672 INFO:tasks.workunit.client.0.smithi057.stdout:9/495: write dd/de/d4a/d60/f7e [406788,105826] 0 2024-02-28T23:07:39.673 INFO:tasks.workunit.client.0.smithi057.stdout:7/440: symlink d0/d5/d25/d80/l89 0 2024-02-28T23:07:39.675 INFO:tasks.workunit.client.1.smithi084.stdout:6/572: dwrite d1/d9/d1d/d1e/d33/d38/f9a [0,4194304] 0 2024-02-28T23:07:39.675 INFO:tasks.workunit.client.0.smithi057.stdout:4/450: symlink d1a/d48/d97/d2d/d49/d4d/l99 0 2024-02-28T23:07:39.677 INFO:tasks.workunit.client.1.smithi084.stdout:1/658: dwrite d12/d1b/d3b/f86 [0,4194304] 0 2024-02-28T23:07:39.679 INFO:tasks.workunit.client.1.smithi084.stdout:2/529: rename d5/c37 to d5/d7/d2e/d4a/c9b 0 2024-02-28T23:07:39.679 INFO:tasks.workunit.client.0.smithi057.stdout:9/496: chown dd/de/d14/l25 281744 1 2024-02-28T23:07:39.679 INFO:tasks.workunit.client.0.smithi057.stdout:0/492: dread d17/d20/d4a/f81 [0,4194304] 0 2024-02-28T23:07:39.685 INFO:tasks.workunit.client.0.smithi057.stdout:5/482: mkdir d4/d9/d14/d9e 0 2024-02-28T23:07:39.686 INFO:tasks.workunit.client.1.smithi084.stdout:8/562: chown d0/l90 127528879 1 2024-02-28T23:07:39.686 INFO:tasks.workunit.client.0.smithi057.stdout:5/483: stat d4/d9/d16/c75 0 2024-02-28T23:07:39.687 INFO:tasks.workunit.client.0.smithi057.stdout:4/451: rename c10 to d1a/d48/d97/d2d/c9a 0 2024-02-28T23:07:39.689 INFO:tasks.workunit.client.1.smithi084.stdout:6/573: unlink d1/d13/d9d/da3/fa7 0 2024-02-28T23:07:39.693 INFO:tasks.workunit.client.0.smithi057.stdout:6/512: dwrite d0/d4/d11/f9a [0,4194304] 0 2024-02-28T23:07:39.695 INFO:tasks.workunit.client.1.smithi084.stdout:1/659: truncate d12/d1b/d63/f66 264635 0 2024-02-28T23:07:39.695 INFO:tasks.workunit.client.1.smithi084.stdout:2/530: symlink d5/d7/d12/d24/d3a/d48/l9c 0 2024-02-28T23:07:39.696 INFO:tasks.workunit.client.0.smithi057.stdout:0/493: rmdir d17/d20/d23/d47 39 2024-02-28T23:07:39.696 INFO:tasks.workunit.client.0.smithi057.stdout:5/484: symlink d4/d9/d14/d3b/d61/d69/d95/l9f 0 2024-02-28T23:07:39.696 INFO:tasks.workunit.client.1.smithi084.stdout:0/583: creat d1/d6/db/d35/d44/d92/fb8 x:0 0 0 2024-02-28T23:07:39.697 INFO:tasks.workunit.client.1.smithi084.stdout:4/530: mkdir d1/d18/d20/d24/d2f/d3e/d52/d45/db5 0 2024-02-28T23:07:39.699 INFO:tasks.workunit.client.1.smithi084.stdout:6/574: rename d1/d9/d1d/d1e/l66 to d1/d9/d1d/db6/lc5 0 2024-02-28T23:07:39.700 INFO:tasks.workunit.client.1.smithi084.stdout:7/630: dwrite db/d10/d17/d9c/d9f/db1/d36/f86 [0,4194304] 0 2024-02-28T23:07:39.702 INFO:tasks.workunit.client.0.smithi057.stdout:4/452: getdents d1a/d1e/d51 0 2024-02-28T23:07:39.704 INFO:tasks.workunit.client.1.smithi084.stdout:2/531: mknod d5/d7/d75/c9d 0 2024-02-28T23:07:39.706 INFO:tasks.workunit.client.1.smithi084.stdout:1/660: creat d12/d1b/d91/d9b/da5/dcb/dd4/fd5 x:0 0 0 2024-02-28T23:07:39.707 INFO:tasks.workunit.client.0.smithi057.stdout:6/513: chown d0/d1b/d3e/d3d/d7d/l95 205149724 1 2024-02-28T23:07:39.708 INFO:tasks.workunit.client.1.smithi084.stdout:7/631: read db/d10/f65 [2160462,47045] 0 2024-02-28T23:07:39.708 INFO:tasks.workunit.client.0.smithi057.stdout:1/368: dwrite d2/d7/df/db/f55 [0,4194304] 0 2024-02-28T23:07:39.708 INFO:tasks.workunit.client.1.smithi084.stdout:2/532: dread - d5/d7/d9/d94/f98 zero size 2024-02-28T23:07:39.709 INFO:tasks.workunit.client.1.smithi084.stdout:6/575: symlink d1/d13/d47/lc6 0 2024-02-28T23:07:39.713 INFO:tasks.workunit.client.1.smithi084.stdout:4/531: symlink d1/d18/d20/d24/d2f/d3e/d52/d45/db5/lb6 0 2024-02-28T23:07:39.713 INFO:tasks.workunit.client.1.smithi084.stdout:2/533: chown d5/d64/c70 0 1 2024-02-28T23:07:39.713 INFO:tasks.workunit.client.0.smithi057.stdout:5/485: getdents d4/d9/d16/d52/d7a 0 2024-02-28T23:07:39.714 INFO:tasks.workunit.client.0.smithi057.stdout:8/510: sync 2024-02-28T23:07:39.720 INFO:tasks.workunit.client.1.smithi084.stdout:2/534: write d5/d7/d2e/d4a/f54 [1710598,49822] 0 2024-02-28T23:07:39.720 INFO:tasks.workunit.client.1.smithi084.stdout:7/632: symlink db/d10/d17/d9c/d9f/db1/d7a/lbc 0 2024-02-28T23:07:39.720 INFO:tasks.workunit.client.1.smithi084.stdout:7/633: read db/d10/f94 [2088543,130989] 0 2024-02-28T23:07:39.721 INFO:tasks.workunit.client.0.smithi057.stdout:1/369: mknod d2/d33/c81 0 2024-02-28T23:07:39.721 INFO:tasks.workunit.client.0.smithi057.stdout:5/486: write d4/d9/d16/f2b [696981,11036] 0 2024-02-28T23:07:39.721 INFO:tasks.workunit.client.0.smithi057.stdout:8/511: readlink d4/d1e/d56/d6a/d42/la4 0 2024-02-28T23:07:39.721 INFO:tasks.workunit.client.1.smithi084.stdout:7/634: stat db/d10/d17/d3b/c78 0 2024-02-28T23:07:39.728 INFO:tasks.workunit.client.0.smithi057.stdout:9/497: dwrite dd/de/d14/f1e [0,4194304] 0 2024-02-28T23:07:39.731 INFO:tasks.workunit.client.1.smithi084.stdout:8/563: dread d0/d9/f21 [0,4194304] 0 2024-02-28T23:07:39.732 INFO:tasks.workunit.client.0.smithi057.stdout:1/370: mkdir d2/d7/df/db/d3a/d2a/d82 0 2024-02-28T23:07:39.732 INFO:tasks.workunit.client.0.smithi057.stdout:7/441: truncate d0/d5/d36/d4e/d15/f6a 1679122 0 2024-02-28T23:07:39.739 INFO:tasks.workunit.client.1.smithi084.stdout:3/577: write d3/d27/d7d/db0/d1a/d21/d3d/d43/f70 [713107,18645] 0 2024-02-28T23:07:39.740 INFO:tasks.workunit.client.0.smithi057.stdout:6/514: symlink d0/d1b/d3e/lad 0 2024-02-28T23:07:39.743 INFO:tasks.workunit.client.0.smithi057.stdout:9/498: chown dd/d1b/d40/f59 8039269 1 2024-02-28T23:07:39.744 INFO:tasks.workunit.client.1.smithi084.stdout:2/535: rename d5/d7/d12/d24/f5e to d5/d7/d9/d94/f9e 0 2024-02-28T23:07:39.745 INFO:tasks.workunit.client.0.smithi057.stdout:3/357: dread d3/d4/d1d/f1e [0,4194304] 0 2024-02-28T23:07:39.746 INFO:tasks.workunit.client.1.smithi084.stdout:1/661: link d12/d1b/d2e/d30/l35 d12/d1b/d63/d7d/dd3/ld6 0 2024-02-28T23:07:39.747 INFO:tasks.workunit.client.0.smithi057.stdout:7/442: creat d0/d5/d36/d4e/d3b/d75/f8a x:0 0 0 2024-02-28T23:07:39.747 INFO:tasks.workunit.client.0.smithi057.stdout:8/512: mknod d4/d1e/d56/d6a/d42/cb6 0 2024-02-28T23:07:39.753 INFO:tasks.workunit.client.1.smithi084.stdout:8/564: link d0/d9/d1b/d20/f25 d0/d24/d78/fbd 0 2024-02-28T23:07:39.754 INFO:tasks.workunit.client.0.smithi057.stdout:5/487: read d4/d9/d14/f36 [3480932,56157] 0 2024-02-28T23:07:39.755 INFO:tasks.workunit.client.0.smithi057.stdout:2/555: sync 2024-02-28T23:07:39.755 INFO:tasks.workunit.client.1.smithi084.stdout:4/532: getdents d1/d4a 0 2024-02-28T23:07:39.757 INFO:tasks.workunit.client.1.smithi084.stdout:3/578: link d3/d27/d7d/db0/d1a/d29/d41/f71 d3/d27/d7d/fdd 0 2024-02-28T23:07:39.758 INFO:tasks.workunit.client.0.smithi057.stdout:0/494: truncate d17/f58 2868889 0 2024-02-28T23:07:39.758 INFO:tasks.workunit.client.1.smithi084.stdout:3/579: chown d3/d50/l77 51 1 2024-02-28T23:07:39.758 INFO:tasks.workunit.client.1.smithi084.stdout:8/565: symlink d0/d24/d6e/lbe 0 2024-02-28T23:07:39.761 INFO:tasks.workunit.client.1.smithi084.stdout:4/533: link d1/d18/d20/d24/d2f/d3e/d52/f9b d1/d18/d20/d24/d2f/fb7 0 2024-02-28T23:07:39.763 INFO:tasks.workunit.client.1.smithi084.stdout:3/580: mkdir d3/d27/d7d/db0/d1a/d45/dde 0 2024-02-28T23:07:39.763 INFO:tasks.workunit.client.1.smithi084.stdout:4/534: dread - d1/d18/d20/d24/d2f/fb7 zero size 2024-02-28T23:07:39.766 INFO:tasks.workunit.client.0.smithi057.stdout:2/556: rmdir d8/d11/d21/d70 39 2024-02-28T23:07:39.768 INFO:tasks.workunit.client.1.smithi084.stdout:4/535: creat d1/d18/d20/d65/d8e/d55/fb8 x:0 0 0 2024-02-28T23:07:39.768 INFO:tasks.workunit.client.0.smithi057.stdout:1/371: dwrite d2/d33/f76 [0,4194304] 0 2024-02-28T23:07:39.770 INFO:tasks.workunit.client.0.smithi057.stdout:7/443: creat d0/d5/d36/d4e/d15/d29/d71/f8b x:0 0 0 2024-02-28T23:07:39.771 INFO:tasks.workunit.client.1.smithi084.stdout:3/581: link d3/d27/d7d/db0/d1a/d45/fd3 d3/d27/d7d/db0/d1a/d29/d41/fdf 0 2024-02-28T23:07:39.772 INFO:tasks.workunit.client.1.smithi084.stdout:4/536: mknod d1/d18/d20/d24/d2f/d3e/d52/d87/da5/db4/cb9 0 2024-02-28T23:07:39.772 INFO:tasks.workunit.client.1.smithi084.stdout:3/582: chown d3/d27/d4b/c75 0 1 2024-02-28T23:07:39.773 INFO:tasks.workunit.client.0.smithi057.stdout:3/358: rename d3/d8/f9 to d3/d8/f73 0 2024-02-28T23:07:39.773 INFO:tasks.workunit.client.0.smithi057.stdout:7/444: readlink d0/l33 0 2024-02-28T23:07:39.775 INFO:tasks.workunit.client.0.smithi057.stdout:2/557: chown d8/d11/d39/f8a 117 1 2024-02-28T23:07:39.776 INFO:tasks.workunit.client.0.smithi057.stdout:5/488: write d4/d9/d16/d4e/f87 [215045,53797] 0 2024-02-28T23:07:39.780 INFO:tasks.workunit.client.0.smithi057.stdout:6/515: dread d0/d17/f35 [0,4194304] 0 2024-02-28T23:07:39.780 INFO:tasks.workunit.client.0.smithi057.stdout:3/359: read d3/d4/f12 [890835,23429] 0 2024-02-28T23:07:39.786 INFO:tasks.workunit.client.0.smithi057.stdout:4/453: dwrite d1a/d48/d97/d2d/d49/d4d/f55 [0,4194304] 0 2024-02-28T23:07:39.791 INFO:tasks.workunit.client.1.smithi084.stdout:3/583: dread d3/d27/d7d/db0/d1a/d21/d3d/d43/f48 [0,4194304] 0 2024-02-28T23:07:39.795 INFO:tasks.workunit.client.1.smithi084.stdout:3/584: mknod d3/d27/d7d/ce0 0 2024-02-28T23:07:39.797 INFO:tasks.workunit.client.0.smithi057.stdout:9/499: dwrite dd/de/d14/f55 [0,4194304] 0 2024-02-28T23:07:39.798 INFO:tasks.workunit.client.1.smithi084.stdout:3/585: write d3/d27/d7d/db0/fa [20623,48818] 0 2024-02-28T23:07:39.798 INFO:tasks.workunit.client.0.smithi057.stdout:2/558: mkdir d8/d11/db3 0 2024-02-28T23:07:39.799 INFO:tasks.workunit.client.1.smithi084.stdout:3/586: dread - d3/d50/f87 zero size 2024-02-28T23:07:39.802 INFO:tasks.workunit.client.0.smithi057.stdout:4/454: rename d1a/d1e/d4b to d1a/d2f/d9b 0 2024-02-28T23:07:39.802 INFO:tasks.workunit.client.1.smithi084.stdout:3/587: write d3/d27/d7d/db0/d14/d31/f5d [1109468,16932] 0 2024-02-28T23:07:39.805 INFO:tasks.workunit.client.0.smithi057.stdout:7/445: dwrite d0/f85 [0,4194304] 0 2024-02-28T23:07:39.805 INFO:tasks.workunit.client.0.smithi057.stdout:0/495: dwrite f10 [0,4194304] 0 2024-02-28T23:07:39.809 INFO:tasks.workunit.client.1.smithi084.stdout:3/588: mkdir d3/d27/d7d/db0/d14/de1 0 2024-02-28T23:07:39.812 INFO:tasks.workunit.client.1.smithi084.stdout:9/618: dwrite d4/d6/f29 [0,4194304] 0 2024-02-28T23:07:39.815 INFO:tasks.workunit.client.0.smithi057.stdout:0/496: chown d17/d20/d26/d48/d55/d88 717881 1 2024-02-28T23:07:39.817 INFO:tasks.workunit.client.1.smithi084.stdout:3/589: fdatasync d3/d27/d7d/db0/d1a/d29/d41/f71 0 2024-02-28T23:07:39.820 INFO:tasks.workunit.client.0.smithi057.stdout:9/500: creat dd/d46/d6f/fa2 x:0 0 0 2024-02-28T23:07:39.820 INFO:tasks.workunit.client.0.smithi057.stdout:5/489: write d4/d9/d16/f40 [141229,58287] 0 2024-02-28T23:07:39.820 INFO:tasks.workunit.client.1.smithi084.stdout:5/637: sync 2024-02-28T23:07:39.822 INFO:tasks.workunit.client.1.smithi084.stdout:9/619: symlink d4/d32/d3f/lc7 0 2024-02-28T23:07:39.824 INFO:tasks.workunit.client.0.smithi057.stdout:0/497: rmdir d17/d7a 39 2024-02-28T23:07:39.829 INFO:tasks.workunit.client.1.smithi084.stdout:7/635: write db/d10/d18/f47 [4102227,51615] 0 2024-02-28T23:07:39.831 INFO:tasks.workunit.client.0.smithi057.stdout:6/516: link d0/d1b/c33 d0/d1b/d3e/d3d/d42/d4f/d92/cae 0 2024-02-28T23:07:39.832 INFO:tasks.workunit.client.1.smithi084.stdout:1/662: write d12/d84/f8e [3848259,116370] 0 2024-02-28T23:07:39.834 INFO:tasks.workunit.client.0.smithi057.stdout:4/455: dwrite d1a/d48/d97/d24/d2e/f7d [0,4194304] 0 2024-02-28T23:07:39.838 INFO:tasks.workunit.client.0.smithi057.stdout:7/446: unlink d0/d5/c41 0 2024-02-28T23:07:39.838 INFO:tasks.workunit.client.0.smithi057.stdout:2/559: link d8/d11/d15/c2e d8/d11/d48/d50/cb4 0 2024-02-28T23:07:39.839 INFO:tasks.workunit.client.0.smithi057.stdout:0/498: unlink d17/d19/l3e 0 2024-02-28T23:07:39.840 INFO:tasks.workunit.client.0.smithi057.stdout:7/447: read - d0/d5/d36/f7b zero size 2024-02-28T23:07:39.841 INFO:tasks.workunit.client.1.smithi084.stdout:3/590: unlink d3/d50/l54 0 2024-02-28T23:07:39.842 INFO:tasks.workunit.client.0.smithi057.stdout:0/499: chown d17/d20/d26/f41 5377364 1 2024-02-28T23:07:39.843 INFO:tasks.workunit.client.0.smithi057.stdout:6/517: read d0/d4/f19 [3210413,88085] 0 2024-02-28T23:07:39.844 INFO:tasks.workunit.client.1.smithi084.stdout:9/620: symlink d4/lc8 0 2024-02-28T23:07:39.848 INFO:tasks.workunit.client.0.smithi057.stdout:6/518: dread - d0/d4/d25/fa9 zero size 2024-02-28T23:07:39.850 INFO:tasks.workunit.client.1.smithi084.stdout:9/621: readlink d4/d6/d15/d12/d41/d55/lb9 0 2024-02-28T23:07:39.852 INFO:tasks.workunit.client.1.smithi084.stdout:3/591: chown d3/d27/d7d/db0/d1a/d21/d3d/f67 28049224 1 2024-02-28T23:07:39.853 INFO:tasks.workunit.client.1.smithi084.stdout:5/638: dread dc/de/df/d31/dd1/f91 [0,4194304] 0 2024-02-28T23:07:39.857 INFO:tasks.workunit.client.1.smithi084.stdout:6/576: dwrite d1/d13/d2e/d41/d2c/f71 [0,4194304] 0 2024-02-28T23:07:39.857 INFO:tasks.workunit.client.1.smithi084.stdout:1/663: mkdir d12/d1b/d2e/d52/dd7 0 2024-02-28T23:07:39.872 INFO:tasks.workunit.client.1.smithi084.stdout:5/639: chown l4 43864 1 2024-02-28T23:07:39.872 INFO:tasks.workunit.client.1.smithi084.stdout:4/537: dwrite d1/f2e [0,4194304] 0 2024-02-28T23:07:39.877 INFO:tasks.workunit.client.1.smithi084.stdout:8/566: dread d0/f8 [0,4194304] 0 2024-02-28T23:07:39.877 INFO:tasks.workunit.client.1.smithi084.stdout:7/636: dread db/d10/f3e [0,4194304] 0 2024-02-28T23:07:39.880 INFO:tasks.workunit.client.1.smithi084.stdout:8/567: chown d0/d9/d1b/d38/l36 6587 1 2024-02-28T23:07:39.882 INFO:tasks.workunit.client.1.smithi084.stdout:9/622: symlink d4/d6/d15/d12/d41/lc9 0 2024-02-28T23:07:39.883 INFO:tasks.workunit.client.1.smithi084.stdout:8/568: stat d0/d44/cb9 0 2024-02-28T23:07:39.887 INFO:tasks.workunit.client.1.smithi084.stdout:1/664: creat d12/d1b/d63/fd8 x:0 0 0 2024-02-28T23:07:39.888 INFO:tasks.workunit.client.1.smithi084.stdout:3/592: rename d3/d27/d7d/db0/d1a/d45/fd3 to d3/d27/d7d/db0/d1a/d45/d5e/fe2 0 2024-02-28T23:07:39.891 INFO:tasks.workunit.client.1.smithi084.stdout:2/536: sync 2024-02-28T23:07:39.891 INFO:tasks.workunit.client.1.smithi084.stdout:0/584: sync 2024-02-28T23:07:39.893 INFO:tasks.workunit.client.1.smithi084.stdout:6/577: fsync d1/d9/d1d/d1e/d33/f89 0 2024-02-28T23:07:39.898 INFO:tasks.workunit.client.1.smithi084.stdout:4/538: mknod d1/d18/d20/d24/cba 0 2024-02-28T23:07:39.900 INFO:tasks.workunit.client.1.smithi084.stdout:7/637: mkdir db/dbd 0 2024-02-28T23:07:39.904 INFO:tasks.workunit.client.1.smithi084.stdout:2/537: mkdir d5/d7/d9/d7c/d5b/d9f 0 2024-02-28T23:07:39.904 INFO:tasks.workunit.client.1.smithi084.stdout:3/593: mkdir d3/d27/d7d/db0/d1a/d21/d3d/d43/de3 0 2024-02-28T23:07:39.906 INFO:tasks.workunit.client.1.smithi084.stdout:0/585: dread - d1/d8/d69/f84 zero size 2024-02-28T23:07:39.907 INFO:tasks.workunit.client.1.smithi084.stdout:6/578: truncate d1/d9/d1d/d1e/d33/d38/f54 8724592 0 2024-02-28T23:07:39.908 INFO:tasks.workunit.client.1.smithi084.stdout:5/640: creat dc/de/df/fef x:0 0 0 2024-02-28T23:07:39.908 INFO:tasks.workunit.client.1.smithi084.stdout:9/623: dwrite d4/d6/d15/d12/fad [0,4194304] 0 2024-02-28T23:07:39.913 INFO:tasks.workunit.client.1.smithi084.stdout:3/594: write d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c [3628653,128007] 0 2024-02-28T23:07:39.916 INFO:tasks.workunit.client.1.smithi084.stdout:6/579: chown d1/d13/d2e/d41/fa9 6838 1 2024-02-28T23:07:39.918 INFO:tasks.workunit.client.1.smithi084.stdout:7/638: creat db/d10/d18/db7/fbe x:0 0 0 2024-02-28T23:07:39.918 INFO:tasks.workunit.client.1.smithi084.stdout:8/569: dwrite d0/d9/d30/d33/d71/f6b [0,4194304] 0 2024-02-28T23:07:39.919 INFO:tasks.workunit.client.1.smithi084.stdout:2/538: read d5/dd/f86 [52759,48266] 0 2024-02-28T23:07:39.922 INFO:tasks.workunit.client.1.smithi084.stdout:3/595: chown d3/d27/d7d/db0/d1a/d21/d3d/cac 39573 1 2024-02-28T23:07:39.927 INFO:tasks.workunit.client.0.smithi057.stdout:4/456: rename d1a/d48/d97/d24 to d1a/d48/d9c 0 2024-02-28T23:07:39.928 INFO:tasks.workunit.client.1.smithi084.stdout:9/624: creat d4/d32/d3f/fca x:0 0 0 2024-02-28T23:07:39.928 INFO:tasks.workunit.client.1.smithi084.stdout:1/665: dread d12/d1b/d3b/f42 [0,4194304] 0 2024-02-28T23:07:39.932 INFO:tasks.workunit.client.1.smithi084.stdout:0/586: write d1/d8/f3f [1258015,8917] 0 2024-02-28T23:07:39.933 INFO:tasks.workunit.client.0.smithi057.stdout:4/457: write d1a/d48/d97/f8a [221066,56187] 0 2024-02-28T23:07:39.935 INFO:tasks.workunit.client.1.smithi084.stdout:5/641: read dc/d51/d7c/f93 [160179,19583] 0 2024-02-28T23:07:39.935 INFO:tasks.workunit.client.0.smithi057.stdout:4/458: stat d1a/d48/d9c/l65 0 2024-02-28T23:07:39.936 INFO:tasks.workunit.client.1.smithi084.stdout:6/580: creat d1/d88/dbf/fc7 x:0 0 0 2024-02-28T23:07:39.937 INFO:tasks.workunit.client.1.smithi084.stdout:4/539: write d1/d18/d20/d14/f6f [13308,44115] 0 2024-02-28T23:07:39.937 INFO:tasks.workunit.client.1.smithi084.stdout:7/639: creat db/d10/d17/d64/d77/fbf x:0 0 0 2024-02-28T23:07:39.940 INFO:tasks.workunit.client.1.smithi084.stdout:3/596: rmdir d3/d27/d7d/db0/d1a/d29 39 2024-02-28T23:07:39.940 INFO:tasks.workunit.client.1.smithi084.stdout:7/640: chown db/dbd 59808 1 2024-02-28T23:07:39.942 INFO:tasks.workunit.client.0.smithi057.stdout:2/560: rename d8/d11/d21/d70/d91/cb2 to d8/d11/d21/da4/cb5 0 2024-02-28T23:07:39.943 INFO:tasks.workunit.client.0.smithi057.stdout:5/490: getdents d4/d9/d14/d63/d88 0 2024-02-28T23:07:39.944 INFO:tasks.workunit.client.1.smithi084.stdout:9/625: symlink d4/d6/d53/lcb 0 2024-02-28T23:07:39.946 INFO:tasks.workunit.client.0.smithi057.stdout:4/459: symlink d1a/d48/d97/d2d/l9d 0 2024-02-28T23:07:39.947 INFO:tasks.workunit.client.1.smithi084.stdout:9/626: write d4/d6/d15/d12/fae [939013,106036] 0 2024-02-28T23:07:39.947 INFO:tasks.workunit.client.1.smithi084.stdout:5/642: fdatasync dc/de/f14 0 2024-02-28T23:07:39.949 INFO:tasks.workunit.client.0.smithi057.stdout:5/491: write d4/d9/d14/f54 [3735679,115821] 0 2024-02-28T23:07:39.951 INFO:tasks.workunit.client.1.smithi084.stdout:1/666: mkdir d12/d1b/d2e/d30/d50/da8/dd9 0 2024-02-28T23:07:39.952 INFO:tasks.workunit.client.0.smithi057.stdout:7/448: creat d0/d5/d36/f8c x:0 0 0 2024-02-28T23:07:39.953 INFO:tasks.workunit.client.1.smithi084.stdout:8/570: creat d0/d9/fbf x:0 0 0 2024-02-28T23:07:39.954 INFO:tasks.workunit.client.0.smithi057.stdout:5/492: dread - d4/d21/f76 zero size 2024-02-28T23:07:39.955 INFO:tasks.workunit.client.0.smithi057.stdout:9/501: dwrite dd/de/d14/f3d [0,4194304] 0 2024-02-28T23:07:39.956 INFO:tasks.workunit.client.1.smithi084.stdout:3/597: symlink d3/d27/d7d/db0/d1a/d45/dde/le4 0 2024-02-28T23:07:39.957 INFO:tasks.workunit.client.1.smithi084.stdout:9/627: readlink d4/d6/d15/d12/d41/l50 0 2024-02-28T23:07:39.959 INFO:tasks.workunit.client.1.smithi084.stdout:9/628: write d4/f85 [5334650,97990] 0 2024-02-28T23:07:39.959 INFO:tasks.workunit.client.0.smithi057.stdout:0/500: truncate d17/d20/d26/f67 2024515 0 2024-02-28T23:07:39.960 INFO:tasks.workunit.client.0.smithi057.stdout:9/502: write dd/de/d4a/d60/f75 [475677,12359] 0 2024-02-28T23:07:39.960 INFO:tasks.workunit.client.1.smithi084.stdout:6/581: creat d1/d9/d1d/d1e/fc8 x:0 0 0 2024-02-28T23:07:39.960 INFO:tasks.workunit.client.1.smithi084.stdout:1/667: symlink d12/d1b/d2e/d30/d50/da8/lda 0 2024-02-28T23:07:39.963 INFO:tasks.workunit.client.1.smithi084.stdout:8/571: fsync d0/d24/d6e/f91 0 2024-02-28T23:07:39.964 INFO:tasks.workunit.client.1.smithi084.stdout:4/540: write d1/d18/d20/d65/d8e/d55/f16 [8579039,7749] 0 2024-02-28T23:07:39.965 INFO:tasks.workunit.client.0.smithi057.stdout:6/519: truncate d0/d1b/d3e/d3d/d42/d4f/d53/f80 2044326 0 2024-02-28T23:07:39.966 INFO:tasks.workunit.client.0.smithi057.stdout:5/493: creat d4/d9/d14/d3b/fa0 x:0 0 0 2024-02-28T23:07:39.968 INFO:tasks.workunit.client.1.smithi084.stdout:6/582: creat d1/d13/d50/fc9 x:0 0 0 2024-02-28T23:07:39.969 INFO:tasks.workunit.client.1.smithi084.stdout:9/629: mknod d4/d6/d15/d75/d2d/daa/ccc 0 2024-02-28T23:07:39.969 INFO:tasks.workunit.client.1.smithi084.stdout:5/643: getdents dc/de/df/d16/d34/daa 0 2024-02-28T23:07:39.976 INFO:tasks.workunit.client.1.smithi084.stdout:6/583: creat d1/d13/d50/fca x:0 0 0 2024-02-28T23:07:39.978 INFO:tasks.workunit.client.0.smithi057.stdout:5/494: mkdir d4/d9/d14/d3b/d61/d71/da1 0 2024-02-28T23:07:39.980 INFO:tasks.workunit.client.1.smithi084.stdout:8/572: mkdir d0/d9/d30/d33/d71/d84/dc0 0 2024-02-28T23:07:39.983 INFO:tasks.workunit.client.0.smithi057.stdout:7/449: rename d0/d5/d36/d4e/d3b/d75/c82 to d0/d5/d4a/c8d 0 2024-02-28T23:07:39.983 INFO:tasks.workunit.client.1.smithi084.stdout:0/587: dwrite d1/d6/f53 [0,4194304] 0 2024-02-28T23:07:39.984 INFO:tasks.workunit.client.1.smithi084.stdout:6/584: chown d1/d13/d2e/d41/fa9 244 1 2024-02-28T23:07:39.985 INFO:tasks.workunit.client.0.smithi057.stdout:0/501: mkdir d17/d20/db6 0 2024-02-28T23:07:39.986 INFO:tasks.workunit.client.1.smithi084.stdout:6/585: dread - d1/d13/d50/fc9 zero size 2024-02-28T23:07:39.986 INFO:tasks.workunit.client.1.smithi084.stdout:4/541: dread d1/f2e [0,4194304] 0 2024-02-28T23:07:39.988 INFO:tasks.workunit.client.0.smithi057.stdout:0/502: fdatasync d17/d20/d4a/d78/f7d 0 2024-02-28T23:07:39.989 INFO:tasks.workunit.client.1.smithi084.stdout:7/641: dread db/d10/f60 [0,4194304] 0 2024-02-28T23:07:39.989 INFO:tasks.workunit.client.1.smithi084.stdout:5/644: rename dc/de/f1a to dc/de/df/d16/d34/d3c/d98/ff0 0 2024-02-28T23:07:39.993 INFO:tasks.workunit.client.0.smithi057.stdout:8/513: sync 2024-02-28T23:07:39.994 INFO:tasks.workunit.client.0.smithi057.stdout:2/561: dread d8/d11/d21/d28/d49/d85/d9b/fac [0,4194304] 0 2024-02-28T23:07:39.996 INFO:tasks.workunit.client.1.smithi084.stdout:4/542: write d1/f9c [12068,115211] 0 2024-02-28T23:07:39.996 INFO:tasks.workunit.client.1.smithi084.stdout:5/645: readlink dc/d51/l7f 0 2024-02-28T23:07:39.998 INFO:tasks.workunit.client.1.smithi084.stdout:6/586: read d1/d9/d1d/d1e/d33/f8f [7014014,52759] 0 2024-02-28T23:07:40.000 INFO:tasks.workunit.client.0.smithi057.stdout:5/495: rmdir d4/d9/d16/d52 39 2024-02-28T23:07:40.002 INFO:tasks.workunit.client.1.smithi084.stdout:7/642: creat db/d10/d17/d9c/db9/fc0 x:0 0 0 2024-02-28T23:07:40.003 INFO:tasks.workunit.client.1.smithi084.stdout:5/646: write dc/de/df/da6/fd3 [1740044,122655] 0 2024-02-28T23:07:40.004 INFO:tasks.workunit.client.0.smithi057.stdout:5/496: write d4/d9/d16/f40 [147537,31851] 0 2024-02-28T23:07:40.006 INFO:tasks.workunit.client.0.smithi057.stdout:7/450: mknod d0/d5/d36/d4e/c8e 0 2024-02-28T23:07:40.007 INFO:tasks.workunit.client.0.smithi057.stdout:5/497: fsync d4/d9/d14/d3b/f84 0 2024-02-28T23:07:40.010 INFO:tasks.workunit.client.1.smithi084.stdout:5/647: fdatasync dc/de/df/d16/f45 0 2024-02-28T23:07:40.013 INFO:tasks.workunit.client.0.smithi057.stdout:4/460: getdents d1a/d2f 0 2024-02-28T23:07:40.015 INFO:tasks.workunit.client.0.smithi057.stdout:7/451: dread d0/f67 [0,4194304] 0 2024-02-28T23:07:40.019 INFO:tasks.workunit.client.1.smithi084.stdout:9/630: getdents d4/d6/d15/d12/d41 0 2024-02-28T23:07:40.019 INFO:tasks.workunit.client.1.smithi084.stdout:6/587: chown d1/d9/d1d/d1e/d33/f4c 37 1 2024-02-28T23:07:40.020 INFO:tasks.workunit.client.1.smithi084.stdout:7/643: unlink db/d10/d17/l30 0 2024-02-28T23:07:40.021 INFO:tasks.workunit.client.1.smithi084.stdout:8/573: dwrite d0/d44/d9f/f98 [4194304,4194304] 0 2024-02-28T23:07:40.027 INFO:tasks.workunit.client.0.smithi057.stdout:9/503: link dd/d1b/d40/d44/l98 dd/de/d4a/d9f/la3 0 2024-02-28T23:07:40.027 INFO:tasks.workunit.client.0.smithi057.stdout:8/514: unlink d4/d1e/d37/l5a 0 2024-02-28T23:07:40.028 INFO:tasks.workunit.client.0.smithi057.stdout:5/498: mkdir d4/d21/d24/da2 0 2024-02-28T23:07:40.028 INFO:tasks.workunit.client.0.smithi057.stdout:4/461: unlink d1a/d48/d97/l2a 0 2024-02-28T23:07:40.028 INFO:tasks.workunit.client.0.smithi057.stdout:0/503: creat d17/d7a/fb7 x:0 0 0 2024-02-28T23:07:40.032 INFO:tasks.workunit.client.1.smithi084.stdout:4/543: mknod d1/cbb 0 2024-02-28T23:07:40.032 INFO:tasks.workunit.client.0.smithi057.stdout:1/372: sync 2024-02-28T23:07:40.032 INFO:tasks.workunit.client.0.smithi057.stdout:5/499: chown d4/d9/d16/d4e 1 1 2024-02-28T23:07:40.033 INFO:tasks.workunit.client.1.smithi084.stdout:0/588: truncate d1/d6/db/d35/f5c 3951932 0 2024-02-28T23:07:40.033 INFO:tasks.workunit.client.0.smithi057.stdout:7/452: mknod d0/d5/d36/d4e/d15/d29/d3d/d51/c8f 0 2024-02-28T23:07:40.033 INFO:tasks.workunit.client.1.smithi084.stdout:1/668: dread d12/d1b/d2e/f57 [0,4194304] 0 2024-02-28T23:07:40.035 INFO:tasks.workunit.client.1.smithi084.stdout:7/644: mkdir db/d5e/d87/dc1 0 2024-02-28T23:07:40.036 INFO:tasks.workunit.client.1.smithi084.stdout:6/588: write d1/d13/d50/fca [777560,65831] 0 2024-02-28T23:07:40.036 INFO:tasks.workunit.client.0.smithi057.stdout:4/462: rmdir d1a/d2f 39 2024-02-28T23:07:40.037 INFO:tasks.workunit.client.1.smithi084.stdout:8/574: unlink d0/d9/d30/d33/d71/f6f 0 2024-02-28T23:07:40.038 INFO:tasks.workunit.client.1.smithi084.stdout:9/631: symlink d4/d6/lcd 0 2024-02-28T23:07:40.038 INFO:tasks.workunit.client.0.smithi057.stdout:1/373: dread d2/d7/d4f/d5c/d6b/f77 [0,4194304] 0 2024-02-28T23:07:40.040 INFO:tasks.workunit.client.1.smithi084.stdout:0/589: unlink d1/d6/db/d76/ca6 0 2024-02-28T23:07:40.041 INFO:tasks.workunit.client.0.smithi057.stdout:5/500: creat d4/d9/d85/fa3 x:0 0 0 2024-02-28T23:07:40.043 INFO:tasks.workunit.client.1.smithi084.stdout:5/648: dwrite dc/de/df/d16/fa0 [0,4194304] 0 2024-02-28T23:07:40.044 INFO:tasks.workunit.client.0.smithi057.stdout:5/501: dread - d4/d9/d85/fa3 zero size 2024-02-28T23:07:40.045 INFO:tasks.workunit.client.0.smithi057.stdout:6/520: rename d0/d1b/d3e/d3d/d42/d4f/l6d to d0/d4/d25/d60/laf 0 2024-02-28T23:07:40.045 INFO:tasks.workunit.client.1.smithi084.stdout:8/575: dread - d0/d9/d1b/d38/d43/f96 zero size 2024-02-28T23:07:40.046 INFO:tasks.workunit.client.1.smithi084.stdout:6/589: write d1/d13/d50/d74/faa [160833,52828] 0 2024-02-28T23:07:40.048 INFO:tasks.workunit.client.0.smithi057.stdout:7/453: creat d0/d5/d36/d4e/d15/d29/f90 x:0 0 0 2024-02-28T23:07:40.048 INFO:tasks.workunit.client.0.smithi057.stdout:5/502: dread - d4/d7/d58/f96 zero size 2024-02-28T23:07:40.050 INFO:tasks.workunit.client.1.smithi084.stdout:9/632: mkdir d4/d6/d15/d75/d9d/dc3/dce 0 2024-02-28T23:07:40.051 INFO:tasks.workunit.client.1.smithi084.stdout:0/590: rename d1/d6 to d1/d6/db/d37/d72/db9 22 2024-02-28T23:07:40.052 INFO:tasks.workunit.client.0.smithi057.stdout:2/562: dwrite d8/f1a [4194304,4194304] 0 2024-02-28T23:07:40.054 INFO:tasks.workunit.client.1.smithi084.stdout:0/591: read d1/d8/f29 [873326,52855] 0 2024-02-28T23:07:40.058 INFO:tasks.workunit.client.1.smithi084.stdout:9/633: dread - d4/d32/d3f/d89/f99 zero size 2024-02-28T23:07:40.058 INFO:tasks.workunit.client.0.smithi057.stdout:8/515: link d4/d1e/d56/d6a/d54/d61/d8c/fa0 d4/d13/d2c/db1/fb7 0 2024-02-28T23:07:40.059 INFO:tasks.workunit.client.0.smithi057.stdout:0/504: creat d17/d20/d23/fb8 x:0 0 0 2024-02-28T23:07:40.060 INFO:tasks.workunit.client.0.smithi057.stdout:1/374: rename d2/d7/df/db/d3a/f36 to d2/d7/df/db/d3a/d2a/f83 0 2024-02-28T23:07:40.060 INFO:tasks.workunit.client.1.smithi084.stdout:3/598: sync 2024-02-28T23:07:40.061 INFO:tasks.workunit.client.1.smithi084.stdout:1/669: dread d12/d1b/d2e/d52/f73 [0,4194304] 0 2024-02-28T23:07:40.065 INFO:tasks.workunit.client.1.smithi084.stdout:3/599: readlink d3/d27/d7d/db0/d14/d31/lc2 0 2024-02-28T23:07:40.066 INFO:tasks.workunit.client.1.smithi084.stdout:1/670: write d12/d1b/d2e/d52/d5b/f62 [330759,69719] 0 2024-02-28T23:07:40.066 INFO:tasks.workunit.client.0.smithi057.stdout:4/463: link d1a/d48/d97/d2d/d49/c84 d1a/d48/d97/d7f/c9e 0 2024-02-28T23:07:40.069 INFO:tasks.workunit.client.1.smithi084.stdout:7/645: dread db/d10/f60 [0,4194304] 0 2024-02-28T23:07:40.070 INFO:tasks.workunit.client.1.smithi084.stdout:5/649: creat dc/de/df/d6e/ff1 x:0 0 0 2024-02-28T23:07:40.070 INFO:tasks.workunit.client.0.smithi057.stdout:1/375: unlink d2/d7/df/db/d3a/f7f 0 2024-02-28T23:07:40.071 INFO:tasks.workunit.client.0.smithi057.stdout:7/454: dread d0/d5/d25/d3e/f7d [0,4194304] 0 2024-02-28T23:07:40.076 INFO:tasks.workunit.client.0.smithi057.stdout:6/521: creat d0/d1b/d3e/d3d/d42/fb0 x:0 0 0 2024-02-28T23:07:40.076 INFO:tasks.workunit.client.0.smithi057.stdout:6/522: write d0/d1b/d3e/d3d/d7d/fa2 [3138931,36785] 0 2024-02-28T23:07:40.076 INFO:tasks.workunit.client.1.smithi084.stdout:8/576: dread d0/d9/d1b/d38/f3a [0,4194304] 0 2024-02-28T23:07:40.076 INFO:tasks.workunit.client.1.smithi084.stdout:1/671: dread d12/d1b/d2e/d52/f73 [0,4194304] 0 2024-02-28T23:07:40.076 INFO:tasks.workunit.client.1.smithi084.stdout:4/544: link d1/d18/d20/c23 d1/d18/d20/d14/d30/d47/cbc 0 2024-02-28T23:07:40.077 INFO:tasks.workunit.client.0.smithi057.stdout:8/516: rename d4/d13/d18/d62/c63 to d4/d13/d1a/d4a/cb8 0 2024-02-28T23:07:40.077 INFO:tasks.workunit.client.1.smithi084.stdout:7/646: write db/d10/d17/d64/fab [35612,119582] 0 2024-02-28T23:07:40.080 INFO:tasks.workunit.client.0.smithi057.stdout:8/517: readlink d4/d1e/d56/d6a/d42/l4c 0 2024-02-28T23:07:40.081 INFO:tasks.workunit.client.1.smithi084.stdout:8/577: mkdir d0/d9/d1b/dc1 0 2024-02-28T23:07:40.083 INFO:tasks.workunit.client.0.smithi057.stdout:8/518: write d4/d13/f2b [2442551,39838] 0 2024-02-28T23:07:40.084 INFO:tasks.workunit.client.0.smithi057.stdout:7/455: dread d0/f67 [0,4194304] 0 2024-02-28T23:07:40.088 INFO:tasks.workunit.client.1.smithi084.stdout:3/600: symlink d3/d27/d7d/db0/d1a/d21/d3d/d43/de3/le5 0 2024-02-28T23:07:40.088 INFO:tasks.workunit.client.1.smithi084.stdout:5/650: symlink dc/lf2 0 2024-02-28T23:07:40.089 INFO:tasks.workunit.client.0.smithi057.stdout:8/519: write d4/d13/d1a/d4a/d7b/f8a [1059597,127862] 0 2024-02-28T23:07:40.091 INFO:tasks.workunit.client.1.smithi084.stdout:8/578: write d0/d11/f8f [2448849,91156] 0 2024-02-28T23:07:40.094 INFO:tasks.workunit.client.1.smithi084.stdout:5/651: creat dc/de/df/d6e/ff3 x:0 0 0 2024-02-28T23:07:40.095 INFO:tasks.workunit.client.0.smithi057.stdout:8/520: truncate d4/d13/d1a/f8f 633581 0 2024-02-28T23:07:40.097 INFO:tasks.workunit.client.1.smithi084.stdout:6/590: link d1/d13/d2e/c98 d1/d13/d47/d91/ccb 0 2024-02-28T23:07:40.099 INFO:tasks.workunit.client.0.smithi057.stdout:3/360: sync 2024-02-28T23:07:40.107 INFO:tasks.workunit.client.1.smithi084.stdout:4/545: dwrite d1/d18/d20/d65/d8e/f5a [4194304,4194304] 0 2024-02-28T23:07:40.108 INFO:tasks.workunit.client.0.smithi057.stdout:7/456: mknod d0/d5/d36/d4e/d3b/d75/c91 0 2024-02-28T23:07:40.108 INFO:tasks.workunit.client.0.smithi057.stdout:1/376: dwrite d2/d7/f41 [4194304,4194304] 0 2024-02-28T23:07:40.108 INFO:tasks.workunit.client.0.smithi057.stdout:4/464: dread d1a/d2f/d9b/f5d [0,4194304] 0 2024-02-28T23:07:40.109 INFO:tasks.workunit.client.1.smithi084.stdout:7/647: getdents db/d10/d17/d9c/db9 0 2024-02-28T23:07:40.109 INFO:tasks.workunit.client.1.smithi084.stdout:0/592: write d1/d6/db/d35/d4a/d56/f89 [573460,127292] 0 2024-02-28T23:07:40.111 INFO:tasks.workunit.client.0.smithi057.stdout:5/503: dread d4/d9/d14/d3b/f47 [0,4194304] 0 2024-02-28T23:07:40.112 INFO:tasks.workunit.client.1.smithi084.stdout:8/579: creat d0/d9/d1b/d38/d43/dae/db8/fc2 x:0 0 0 2024-02-28T23:07:40.112 INFO:tasks.workunit.client.1.smithi084.stdout:6/591: rename d1/d13/d2e/d41/d4d/l76 to d1/d88/dbf/lcc 0 2024-02-28T23:07:40.113 INFO:tasks.workunit.client.1.smithi084.stdout:7/648: dread - db/d10/d17/d9c/db9/fc0 zero size 2024-02-28T23:07:40.114 INFO:tasks.workunit.client.0.smithi057.stdout:2/563: write d8/d11/d21/d28/d41/f9e [263310,60609] 0 2024-02-28T23:07:40.114 INFO:tasks.workunit.client.1.smithi084.stdout:9/634: truncate d4/f51 5779001 0 2024-02-28T23:07:40.115 INFO:tasks.workunit.client.0.smithi057.stdout:0/505: getdents d17/d20/d23/d47/d60 0 2024-02-28T23:07:40.116 INFO:tasks.workunit.client.0.smithi057.stdout:4/465: chown d1a/d48/d97/f93 3 1 2024-02-28T23:07:40.117 INFO:tasks.workunit.client.0.smithi057.stdout:1/377: truncate d2/d71/d62/f74 1001323 0 2024-02-28T23:07:40.117 INFO:tasks.workunit.client.1.smithi084.stdout:4/546: read d1/d18/d20/d14/f6f [22593,83923] 0 2024-02-28T23:07:40.117 INFO:tasks.workunit.client.1.smithi084.stdout:2/539: sync 2024-02-28T23:07:40.121 INFO:tasks.workunit.client.1.smithi084.stdout:7/649: dread - db/d10/d17/d64/d77/fa8 zero size 2024-02-28T23:07:40.123 INFO:tasks.workunit.client.1.smithi084.stdout:5/652: dread dc/de/df/da6/fd3 [0,4194304] 0 2024-02-28T23:07:40.125 INFO:tasks.workunit.client.1.smithi084.stdout:8/580: chown d0/d44/l55 5 1 2024-02-28T23:07:40.126 INFO:tasks.workunit.client.1.smithi084.stdout:0/593: symlink d1/d6/db/d37/d72/lba 0 2024-02-28T23:07:40.129 INFO:tasks.workunit.client.1.smithi084.stdout:9/635: readlink d4/d6/le 0 2024-02-28T23:07:40.131 INFO:tasks.workunit.client.1.smithi084.stdout:7/650: write db/d10/d17/d3b/f4d [4395656,107090] 0 2024-02-28T23:07:40.135 INFO:tasks.workunit.client.0.smithi057.stdout:8/521: truncate d4/d13/d1a/d4a/f6e 621004 0 2024-02-28T23:07:40.138 INFO:tasks.workunit.client.1.smithi084.stdout:1/672: dwrite d12/d1b/d2e/d32/f51 [0,4194304] 0 2024-02-28T23:07:40.139 INFO:tasks.workunit.client.0.smithi057.stdout:0/506: creat d17/d20/d23/d4b/fb9 x:0 0 0 2024-02-28T23:07:40.141 INFO:tasks.workunit.client.1.smithi084.stdout:2/540: creat d5/d7/d9/d7c/d5b/d9f/fa0 x:0 0 0 2024-02-28T23:07:40.141 INFO:tasks.workunit.client.1.smithi084.stdout:8/581: creat d0/d9/d1b/d38/d43/d51/d54/fc3 x:0 0 0 2024-02-28T23:07:40.142 INFO:tasks.workunit.client.0.smithi057.stdout:8/522: chown d4/d13/d7f/c96 1386503 1 2024-02-28T23:07:40.143 INFO:tasks.workunit.client.0.smithi057.stdout:6/523: link d0/d1b/d3e/d40/l98 d0/d1b/d3e/lb1 0 2024-02-28T23:07:40.144 INFO:tasks.workunit.client.0.smithi057.stdout:4/466: dread d1a/f36 [0,4194304] 0 2024-02-28T23:07:40.144 INFO:tasks.workunit.client.1.smithi084.stdout:7/651: write db/d10/d17/d9c/d9f/db1/d36/f86 [3114934,61246] 0 2024-02-28T23:07:40.146 INFO:tasks.workunit.client.1.smithi084.stdout:9/636: symlink d4/d6/d15/d75/d2d/daa/lcf 0 2024-02-28T23:07:40.147 INFO:tasks.workunit.client.0.smithi057.stdout:1/378: symlink d2/d7/df/db/d12/d19/d80/l84 0 2024-02-28T23:07:40.147 INFO:tasks.workunit.client.1.smithi084.stdout:0/594: unlink d1/d8/d18/l3d 0 2024-02-28T23:07:40.150 INFO:tasks.workunit.client.0.smithi057.stdout:0/507: mkdir d17/d20/d4a/d78/db5/dba 0 2024-02-28T23:07:40.151 INFO:tasks.workunit.client.1.smithi084.stdout:3/601: dwrite d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/f65 [0,4194304] 0 2024-02-28T23:07:40.151 INFO:tasks.workunit.client.1.smithi084.stdout:2/541: dread - d5/d7/d2e/f93 zero size 2024-02-28T23:07:40.152 INFO:tasks.workunit.client.0.smithi057.stdout:4/467: mkdir d1a/d48/d9c/d2e/d43/d9f 0 2024-02-28T23:07:40.152 INFO:tasks.workunit.client.1.smithi084.stdout:8/582: read d0/d9/d30/d33/d71/d84/f8d [615173,2244] 0 2024-02-28T23:07:40.155 INFO:tasks.workunit.client.0.smithi057.stdout:4/468: dread - d1a/d48/d97/d2d/f59 zero size 2024-02-28T23:07:40.156 INFO:tasks.workunit.client.0.smithi057.stdout:6/524: rename d0/d1b/d3e/d3d/d74/lab to d0/d4/d11/lb2 0 2024-02-28T23:07:40.156 INFO:tasks.workunit.client.1.smithi084.stdout:6/592: rename d1/d9/f48 to d1/d9/d1d/d1e/fcd 0 2024-02-28T23:07:40.157 INFO:tasks.workunit.client.0.smithi057.stdout:8/523: getdents d4/d1e/d56/d6a/d42 0 2024-02-28T23:07:40.158 INFO:tasks.workunit.client.1.smithi084.stdout:1/673: creat d12/d84/d9a/fdb x:0 0 0 2024-02-28T23:07:40.160 INFO:tasks.workunit.client.1.smithi084.stdout:3/602: dread - d3/d50/d5a/d73/db8/fdc zero size 2024-02-28T23:07:40.162 INFO:tasks.workunit.client.1.smithi084.stdout:9/637: fdatasync d4/d32/d3f/d89/f99 0 2024-02-28T23:07:40.164 INFO:tasks.workunit.client.0.smithi057.stdout:6/525: write d0/d4/fa [1224105,79139] 0 2024-02-28T23:07:40.164 INFO:tasks.workunit.client.1.smithi084.stdout:8/583: write d0/d44/d9f/fb0 [581438,28612] 0 2024-02-28T23:07:40.165 INFO:tasks.workunit.client.1.smithi084.stdout:9/638: fsync d4/f90 0 2024-02-28T23:07:40.167 INFO:tasks.workunit.client.1.smithi084.stdout:6/593: chown d1/d9/d1d/d1e/d33/d38/f9a 1728 1 2024-02-28T23:07:40.168 INFO:tasks.workunit.client.1.smithi084.stdout:2/542: truncate d5/d7/d12/d24/d3a/f96 213710 0 2024-02-28T23:07:40.169 INFO:tasks.workunit.client.1.smithi084.stdout:0/595: mknod d1/d8/d69/da3/cbb 0 2024-02-28T23:07:40.170 INFO:tasks.workunit.client.1.smithi084.stdout:7/652: getdents db/d10/d18/db7/dbb 0 2024-02-28T23:07:40.170 INFO:tasks.workunit.client.1.smithi084.stdout:6/594: readlink d1/d13/d47/lc6 0 2024-02-28T23:07:40.171 INFO:tasks.workunit.client.0.smithi057.stdout:2/564: write d8/d11/d15/f35 [992466,60245] 0 2024-02-28T23:07:40.172 INFO:tasks.workunit.client.0.smithi057.stdout:7/457: truncate d0/d5/d36/d4e/d15/d29/f79 1557974 0 2024-02-28T23:07:40.173 INFO:tasks.workunit.client.1.smithi084.stdout:7/653: chown db/d10/d17/d3b 909 1 2024-02-28T23:07:40.176 INFO:tasks.workunit.client.0.smithi057.stdout:1/379: write d2/d7/df/f1d [698319,16673] 0 2024-02-28T23:07:40.177 INFO:tasks.workunit.client.0.smithi057.stdout:3/361: dread d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:40.178 INFO:tasks.workunit.client.0.smithi057.stdout:6/526: rename d0/d4/d25/d60/d67 to d0/d17/d94/da0/db3 0 2024-02-28T23:07:40.183 INFO:tasks.workunit.client.1.smithi084.stdout:1/674: symlink d12/d1b/d2e/d32/ldc 0 2024-02-28T23:07:40.185 INFO:tasks.workunit.client.1.smithi084.stdout:5/653: dread dc/de/df/d16/f95 [0,4194304] 0 2024-02-28T23:07:40.190 INFO:tasks.workunit.client.1.smithi084.stdout:7/654: mkdir db/d10/d18/dc2 0 2024-02-28T23:07:40.190 INFO:tasks.workunit.client.1.smithi084.stdout:0/596: creat d1/d8/d69/db2/fbc x:0 0 0 2024-02-28T23:07:40.196 INFO:tasks.workunit.client.1.smithi084.stdout:3/603: dread d3/d27/f2d [0,4194304] 0 2024-02-28T23:07:40.200 INFO:tasks.workunit.client.1.smithi084.stdout:5/654: stat dc/de/d83/fe0 0 2024-02-28T23:07:40.201 INFO:tasks.workunit.client.1.smithi084.stdout:5/655: readlink dc/d51/d7c/le7 0 2024-02-28T23:07:40.201 INFO:tasks.workunit.client.1.smithi084.stdout:7/655: rmdir db/d10/d17/d9c/d9f/db1/db2 39 2024-02-28T23:07:40.204 INFO:tasks.workunit.client.1.smithi084.stdout:4/547: dread d1/d18/d20/d24/f2a [0,4194304] 0 2024-02-28T23:07:40.207 INFO:tasks.workunit.client.1.smithi084.stdout:6/595: dwrite d1/d13/d50/fca [0,4194304] 0 2024-02-28T23:07:40.209 INFO:tasks.workunit.client.1.smithi084.stdout:0/597: getdents d1/d8/d5f/d93 0 2024-02-28T23:07:40.217 INFO:tasks.workunit.client.1.smithi084.stdout:4/548: chown d1/f9 7174221 1 2024-02-28T23:07:40.217 INFO:tasks.workunit.client.1.smithi084.stdout:3/604: getdents d3 0 2024-02-28T23:07:40.218 INFO:tasks.workunit.client.1.smithi084.stdout:8/584: dwrite d0/f1a [4194304,4194304] 0 2024-02-28T23:07:40.220 INFO:tasks.workunit.client.1.smithi084.stdout:9/639: dwrite d4/f13 [0,4194304] 0 2024-02-28T23:07:40.224 INFO:tasks.workunit.client.1.smithi084.stdout:6/596: symlink d1/d13/d9d/lce 0 2024-02-28T23:07:40.225 INFO:tasks.workunit.client.1.smithi084.stdout:4/549: dread - d1/d18/d20/d24/d2f/d3e/d52/f8c zero size 2024-02-28T23:07:40.227 INFO:tasks.workunit.client.0.smithi057.stdout:7/458: write d0/d5/d36/f63 [883103,21514] 0 2024-02-28T23:07:40.230 INFO:tasks.workunit.client.0.smithi057.stdout:1/380: read - d2/d71/d2d/d5b/f7d zero size 2024-02-28T23:07:40.230 INFO:tasks.workunit.client.0.smithi057.stdout:2/565: creat d8/d11/d21/d70/d91/fb6 x:0 0 0 2024-02-28T23:07:40.235 INFO:tasks.workunit.client.1.smithi084.stdout:7/656: truncate db/d10/d17/f3d 56585 0 2024-02-28T23:07:40.235 INFO:tasks.workunit.client.1.smithi084.stdout:8/585: creat d0/d9/d1b/fc4 x:0 0 0 2024-02-28T23:07:40.236 INFO:tasks.workunit.client.1.smithi084.stdout:1/675: dwrite d12/d1b/d2e/d52/d5b/f6f [0,4194304] 0 2024-02-28T23:07:40.236 INFO:tasks.workunit.client.0.smithi057.stdout:2/566: unlink d8/d11/d56/l69 0 2024-02-28T23:07:40.240 INFO:tasks.workunit.client.0.smithi057.stdout:3/362: creat d3/d8/d27/d70/f74 x:0 0 0 2024-02-28T23:07:40.240 INFO:tasks.workunit.client.1.smithi084.stdout:0/598: creat d1/d8/d18/d4c/d6c/fbd x:0 0 0 2024-02-28T23:07:40.240 INFO:tasks.workunit.client.1.smithi084.stdout:9/640: mkdir d4/d6/d15/d75/d9d/dd0 0 2024-02-28T23:07:40.243 INFO:tasks.workunit.client.1.smithi084.stdout:6/597: symlink d1/d13/d2e/d41/d4d/lcf 0 2024-02-28T23:07:40.245 INFO:tasks.workunit.client.1.smithi084.stdout:9/641: read - d4/f37 zero size 2024-02-28T23:07:40.246 INFO:tasks.workunit.client.1.smithi084.stdout:7/657: write db/d10/d17/d3b/f9b [747958,85203] 0 2024-02-28T23:07:40.247 INFO:tasks.workunit.client.1.smithi084.stdout:5/656: dwrite dc/de/df/d16/f4a [0,4194304] 0 2024-02-28T23:07:40.248 INFO:tasks.workunit.client.0.smithi057.stdout:6/527: creat d0/d4/fb4 x:0 0 0 2024-02-28T23:07:40.248 INFO:tasks.workunit.client.0.smithi057.stdout:3/363: write d3/f61 [43151,20107] 0 2024-02-28T23:07:40.253 INFO:tasks.workunit.client.1.smithi084.stdout:8/586: creat d0/d24/d6e/fc5 x:0 0 0 2024-02-28T23:07:40.254 INFO:tasks.workunit.client.1.smithi084.stdout:0/599: write d1/d8/d78/fa4 [1760772,1314] 0 2024-02-28T23:07:40.255 INFO:tasks.workunit.client.0.smithi057.stdout:7/459: dwrite d0/d5/d36/d4e/d3b/d75/f8a [0,4194304] 0 2024-02-28T23:07:40.255 INFO:tasks.workunit.client.0.smithi057.stdout:2/567: rename d8/fa to d8/d11/d15/daf/fb7 0 2024-02-28T23:07:40.255 INFO:tasks.workunit.client.1.smithi084.stdout:4/550: getdents d1/d18/d20/d24/d2f/d3e/d52/d48 0 2024-02-28T23:07:40.258 INFO:tasks.workunit.client.1.smithi084.stdout:6/598: symlink d1/d13/d50/d74/d9e/dc2/ld0 0 2024-02-28T23:07:40.258 INFO:tasks.workunit.client.0.smithi057.stdout:3/364: mknod d3/d4/d7/c75 0 2024-02-28T23:07:40.261 INFO:tasks.workunit.client.0.smithi057.stdout:8/524: dread d4/d13/d1a/f20 [0,4194304] 0 2024-02-28T23:07:40.266 INFO:tasks.workunit.client.0.smithi057.stdout:6/528: rename d0/d1b/d3e/fa5 to d0/d17/d94/da0/fb5 0 2024-02-28T23:07:40.266 INFO:tasks.workunit.client.1.smithi084.stdout:9/642: getdents d4/d6/d15/d75/d9d/dc3 0 2024-02-28T23:07:40.268 INFO:tasks.workunit.client.0.smithi057.stdout:7/460: symlink d0/d5/d36/d4e/d15/d29/l92 0 2024-02-28T23:07:40.268 INFO:tasks.workunit.client.0.smithi057.stdout:5/504: dread d4/d9/d16/d2c/f80 [0,4194304] 0 2024-02-28T23:07:40.268 INFO:tasks.workunit.client.1.smithi084.stdout:8/587: creat d0/d9/d30/d33/d71/d84/dc0/fc6 x:0 0 0 2024-02-28T23:07:40.268 INFO:tasks.workunit.client.1.smithi084.stdout:8/588: readlink d0/l57 0 2024-02-28T23:07:40.268 INFO:tasks.workunit.client.0.smithi057.stdout:8/525: creat d4/d13/d2c/fb9 x:0 0 0 2024-02-28T23:07:40.269 INFO:tasks.workunit.client.1.smithi084.stdout:6/599: mkdir d1/d13/d50/dbd/dd1 0 2024-02-28T23:07:40.270 INFO:tasks.workunit.client.0.smithi057.stdout:1/381: dwrite d2/d71/d2d/f3d [4194304,4194304] 0 2024-02-28T23:07:40.270 INFO:tasks.workunit.client.0.smithi057.stdout:8/526: readlink d4/le 0 2024-02-28T23:07:40.272 INFO:tasks.workunit.client.1.smithi084.stdout:6/600: fsync d1/d9/f3b 0 2024-02-28T23:07:40.272 INFO:tasks.workunit.client.1.smithi084.stdout:4/551: creat d1/fbd x:0 0 0 2024-02-28T23:07:40.274 INFO:tasks.workunit.client.1.smithi084.stdout:9/643: creat d4/d6/d15/d3c/fd1 x:0 0 0 2024-02-28T23:07:40.275 INFO:tasks.workunit.client.0.smithi057.stdout:6/529: read d0/d17/d20/f84 [138009,47472] 0 2024-02-28T23:07:40.277 INFO:tasks.workunit.client.1.smithi084.stdout:5/657: getdents dc/de/df/d31/dd1 0 2024-02-28T23:07:40.278 INFO:tasks.workunit.client.1.smithi084.stdout:7/658: dwrite db/d10/d17/d9c/d9f/db1/d36/faf [0,4194304] 0 2024-02-28T23:07:40.280 INFO:tasks.workunit.client.0.smithi057.stdout:7/461: mknod d0/d3f/c93 0 2024-02-28T23:07:40.285 INFO:tasks.workunit.client.0.smithi057.stdout:4/469: dread f14 [0,4194304] 0 2024-02-28T23:07:40.285 INFO:tasks.workunit.client.0.smithi057.stdout:8/527: symlink d4/d1e/d56/da6/dae/lba 0 2024-02-28T23:07:40.287 INFO:tasks.workunit.client.1.smithi084.stdout:7/659: stat db/d11/c55 0 2024-02-28T23:07:40.288 INFO:tasks.workunit.client.0.smithi057.stdout:6/530: read d0/d1b/d3e/d40/f4d [2558721,39811] 0 2024-02-28T23:07:40.288 INFO:tasks.workunit.client.1.smithi084.stdout:8/589: link d0/d9/d1b/f74 d0/d9/d1b/dc1/fc7 0 2024-02-28T23:07:40.296 INFO:tasks.workunit.client.1.smithi084.stdout:9/644: unlink d4/d6/d15/d12/lc1 0 2024-02-28T23:07:40.299 INFO:tasks.workunit.client.0.smithi057.stdout:6/531: write d0/d1b/d3e/d3d/f55 [5118311,22678] 0 2024-02-28T23:07:40.299 INFO:tasks.workunit.client.0.smithi057.stdout:4/470: mknod d1a/d48/d9c/d68/ca0 0 2024-02-28T23:07:40.299 INFO:tasks.workunit.client.0.smithi057.stdout:8/528: rename d4/d13/d1a/d4a/d7b to d4/d13/d18/d62/dbb 0 2024-02-28T23:07:40.301 INFO:tasks.workunit.client.1.smithi084.stdout:3/605: write d3/d27/d7d/db0/d1a/d29/d7e/fa7 [3717783,90004] 0 2024-02-28T23:07:40.302 INFO:tasks.workunit.client.1.smithi084.stdout:1/676: dread d12/f28 [0,4194304] 0 2024-02-28T23:07:40.302 INFO:tasks.workunit.client.1.smithi084.stdout:4/552: dread d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fa1 [0,4194304] 0 2024-02-28T23:07:40.304 INFO:tasks.workunit.client.0.smithi057.stdout:9/504: sync 2024-02-28T23:07:40.304 INFO:tasks.workunit.client.1.smithi084.stdout:5/658: chown dc/de/df/d16/d34/l8b 131083632 1 2024-02-28T23:07:40.304 INFO:tasks.workunit.client.1.smithi084.stdout:3/606: readlink d3/d27/d7d/db0/l4a 0 2024-02-28T23:07:40.306 INFO:tasks.workunit.client.1.smithi084.stdout:8/590: unlink d0/d9/d30/db6/l8a 0 2024-02-28T23:07:40.307 INFO:tasks.workunit.client.0.smithi057.stdout:4/471: fsync d1a/d1e/f25 0 2024-02-28T23:07:40.308 INFO:tasks.workunit.client.1.smithi084.stdout:3/607: chown d3/d27/d7d/dd9 0 1 2024-02-28T23:07:40.309 INFO:tasks.workunit.client.0.smithi057.stdout:6/532: creat d0/d1b/d3e/d40/fb6 x:0 0 0 2024-02-28T23:07:40.309 INFO:tasks.workunit.client.1.smithi084.stdout:7/660: mkdir db/d5e/dc3 0 2024-02-28T23:07:40.311 INFO:tasks.workunit.client.0.smithi057.stdout:3/365: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:40.313 INFO:tasks.workunit.client.0.smithi057.stdout:4/472: mkdir d1a/d48/d9c/d2e/da1 0 2024-02-28T23:07:40.313 INFO:tasks.workunit.client.1.smithi084.stdout:4/553: fsync d1/f7 0 2024-02-28T23:07:40.316 INFO:tasks.workunit.client.0.smithi057.stdout:6/533: symlink d0/d1b/d3e/d3d/lb7 0 2024-02-28T23:07:40.316 INFO:tasks.workunit.client.0.smithi057.stdout:8/529: getdents d4/d1e/d56/d6a/d54/d61 0 2024-02-28T23:07:40.318 INFO:tasks.workunit.client.1.smithi084.stdout:8/591: fdatasync d0/d44/d9f/f7e 0 2024-02-28T23:07:40.319 INFO:tasks.workunit.client.1.smithi084.stdout:5/659: mkdir dc/de/df/d16/d34/daa/df4 0 2024-02-28T23:07:40.320 INFO:tasks.workunit.client.1.smithi084.stdout:3/608: creat d3/d27/d7d/fe6 x:0 0 0 2024-02-28T23:07:40.320 INFO:tasks.workunit.client.1.smithi084.stdout:4/554: read d1/d18/d20/d14/d30/f3d [1738221,56323] 0 2024-02-28T23:07:40.321 INFO:tasks.workunit.client.0.smithi057.stdout:6/534: write d0/d1b/d54/f90 [1022431,119905] 0 2024-02-28T23:07:40.322 INFO:tasks.workunit.client.0.smithi057.stdout:8/530: unlink d4/d1e/d56/d6a/d54/ca5 0 2024-02-28T23:07:40.328 INFO:tasks.workunit.client.1.smithi084.stdout:5/660: rename dc/de/df/da6/fb6 to dc/de/df/da6/ff5 0 2024-02-28T23:07:40.329 INFO:tasks.workunit.client.0.smithi057.stdout:4/473: rename d1a/d48/d9c/f82 to d1a/d48/d9c/d2e/fa2 0 2024-02-28T23:07:40.329 INFO:tasks.workunit.client.1.smithi084.stdout:4/555: truncate d1/d18/d20/d14/f4f 1273217 0 2024-02-28T23:07:40.333 INFO:tasks.workunit.client.0.smithi057.stdout:6/535: chown d0/d4/d11/c4e 11 1 2024-02-28T23:07:40.337 INFO:tasks.workunit.client.0.smithi057.stdout:8/531: creat d4/d1e/d56/fbc x:0 0 0 2024-02-28T23:07:40.341 INFO:tasks.workunit.client.1.smithi084.stdout:4/556: truncate d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fa1 2089720 0 2024-02-28T23:07:40.345 INFO:tasks.workunit.client.1.smithi084.stdout:6/601: dread d1/d13/d50/f99 [0,4194304] 0 2024-02-28T23:07:40.345 INFO:tasks.workunit.client.1.smithi084.stdout:5/661: link dc/de/df/d31/dd1/da4/lda dc/de/df/d6e/lf6 0 2024-02-28T23:07:40.346 INFO:tasks.workunit.client.0.smithi057.stdout:3/366: dwrite d3/d4/d1d/f36 [0,4194304] 0 2024-02-28T23:07:40.347 INFO:tasks.workunit.client.0.smithi057.stdout:4/474: rmdir d1a/d48/d9c/d2e/da1 0 2024-02-28T23:07:40.348 INFO:tasks.workunit.client.1.smithi084.stdout:7/661: truncate db/d10/d17/d9c/d9f/db1/d36/f4c 1341337 0 2024-02-28T23:07:40.348 INFO:tasks.workunit.client.1.smithi084.stdout:6/602: chown d1/d9/d1d/db6 142091 1 2024-02-28T23:07:40.348 INFO:tasks.workunit.client.1.smithi084.stdout:3/609: write d3/d27/f8a [1015925,66796] 0 2024-02-28T23:07:40.354 INFO:tasks.workunit.client.1.smithi084.stdout:4/557: truncate d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f9f 744784 0 2024-02-28T23:07:40.357 INFO:tasks.workunit.client.0.smithi057.stdout:7/462: dwrite d0/d5/d25/d3e/f54 [0,4194304] 0 2024-02-28T23:07:40.358 INFO:tasks.workunit.client.1.smithi084.stdout:7/662: unlink db/d10/d17/d64/f69 0 2024-02-28T23:07:40.358 INFO:tasks.workunit.client.1.smithi084.stdout:9/645: dread d4/d6/d15/f98 [0,4194304] 0 2024-02-28T23:07:40.360 INFO:tasks.workunit.client.0.smithi057.stdout:8/532: mkdir d4/d1e/d56/d6a/dbd 0 2024-02-28T23:07:40.361 INFO:tasks.workunit.client.1.smithi084.stdout:3/610: creat d3/dae/fe7 x:0 0 0 2024-02-28T23:07:40.362 INFO:tasks.workunit.client.1.smithi084.stdout:8/592: dwrite d0/d44/f67 [0,4194304] 0 2024-02-28T23:07:40.362 INFO:tasks.workunit.client.1.smithi084.stdout:6/603: unlink d1/d9/d1d/d1e/d33/d38/f54 0 2024-02-28T23:07:40.364 INFO:tasks.workunit.client.0.smithi057.stdout:6/536: dwrite d0/d17/d20/f31 [0,4194304] 0 2024-02-28T23:07:40.365 INFO:tasks.workunit.client.0.smithi057.stdout:4/475: creat d1a/d48/d9c/d2e/fa3 x:0 0 0 2024-02-28T23:07:40.368 INFO:tasks.workunit.client.0.smithi057.stdout:7/463: creat d0/d5/d36/d4e/d3b/d75/f94 x:0 0 0 2024-02-28T23:07:40.369 INFO:tasks.workunit.client.1.smithi084.stdout:7/663: rmdir db/d10/d17/d9c/d9f/db1 39 2024-02-28T23:07:40.369 INFO:tasks.workunit.client.1.smithi084.stdout:9/646: fdatasync d4/d6/d15/f2a 0 2024-02-28T23:07:40.369 INFO:tasks.workunit.client.1.smithi084.stdout:3/611: truncate d3/d27/d7d/db0/df/fd1 446493 0 2024-02-28T23:07:40.372 INFO:tasks.workunit.client.1.smithi084.stdout:3/612: rmdir d3/d27/d7d 39 2024-02-28T23:07:40.375 INFO:tasks.workunit.client.0.smithi057.stdout:6/537: write d0/d4/d11/f14 [1680153,62081] 0 2024-02-28T23:07:40.375 INFO:tasks.workunit.client.0.smithi057.stdout:8/533: symlink d4/d13/d18/d62/dbb/lbe 0 2024-02-28T23:07:40.376 INFO:tasks.workunit.client.0.smithi057.stdout:7/464: rmdir d0/d3f/d76 39 2024-02-28T23:07:40.377 INFO:tasks.workunit.client.1.smithi084.stdout:7/664: symlink db/d10/d17/d9c/db9/lc4 0 2024-02-28T23:07:40.381 INFO:tasks.workunit.client.0.smithi057.stdout:7/465: truncate d0/d5/f39 1830085 0 2024-02-28T23:07:40.384 INFO:tasks.workunit.client.1.smithi084.stdout:6/604: link d1/d13/d47/c62 d1/d13/d50/d74/d81/cd2 0 2024-02-28T23:07:40.386 INFO:tasks.workunit.client.1.smithi084.stdout:9/647: truncate d4/d6/d15/f2c 284770 0 2024-02-28T23:07:40.386 INFO:tasks.workunit.client.0.smithi057.stdout:4/476: rmdir d1a/d48/d9c/d2e/d43/d9f 0 2024-02-28T23:07:40.388 INFO:tasks.workunit.client.1.smithi084.stdout:9/648: read - d4/d6/d15/d3c/fd1 zero size 2024-02-28T23:07:40.390 INFO:tasks.workunit.client.0.smithi057.stdout:8/534: creat d4/d13/d18/d62/d66/fbf x:0 0 0 2024-02-28T23:07:40.393 INFO:tasks.workunit.client.0.smithi057.stdout:4/477: symlink d1a/d48/d97/d2d/d49/d4d/la4 0 2024-02-28T23:07:40.394 INFO:tasks.workunit.client.1.smithi084.stdout:4/558: dwrite d1/d18/d20/d14/fab [0,4194304] 0 2024-02-28T23:07:40.394 INFO:tasks.workunit.client.0.smithi057.stdout:4/478: stat d1a/d48/d97 0 2024-02-28T23:07:40.397 INFO:tasks.workunit.client.1.smithi084.stdout:3/613: link d3/d27/d4b/l56 d3/d27/d7d/db0/d1a/d45/dde/le8 0 2024-02-28T23:07:40.398 INFO:tasks.workunit.client.0.smithi057.stdout:7/466: getdents d0/d5/d36/d4e/d15/d29/d3d 0 2024-02-28T23:07:40.400 INFO:tasks.workunit.client.0.smithi057.stdout:4/479: rename fe to d1a/d48/d97/d7f/fa5 0 2024-02-28T23:07:40.402 INFO:tasks.workunit.client.1.smithi084.stdout:6/605: link d1/d13/d50/d74/d9e/fb9 d1/d13/d2e/fd3 0 2024-02-28T23:07:40.405 INFO:tasks.workunit.client.0.smithi057.stdout:8/535: link d4/d1e/d56/d6a/d42/cb5 d4/d13/d18/da7/cc0 0 2024-02-28T23:07:40.405 INFO:tasks.workunit.client.0.smithi057.stdout:4/480: write d1a/d48/d97/d2d/d49/d4d/f8d [613166,32587] 0 2024-02-28T23:07:40.409 INFO:tasks.workunit.client.1.smithi084.stdout:5/662: dread dc/de/df/d31/f41 [0,4194304] 0 2024-02-28T23:07:40.410 INFO:tasks.workunit.client.1.smithi084.stdout:2/543: sync 2024-02-28T23:07:40.410 INFO:tasks.workunit.client.1.smithi084.stdout:4/559: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fbe x:0 0 0 2024-02-28T23:07:40.412 INFO:tasks.workunit.client.1.smithi084.stdout:3/614: readlink d3/d27/d7d/db0/d1a/d29/d41/d5b/l89 0 2024-02-28T23:07:40.412 INFO:tasks.workunit.client.0.smithi057.stdout:8/536: rename d4/d1e/d37/c4f to d4/d1e/d56/da6/cc1 0 2024-02-28T23:07:40.413 INFO:tasks.workunit.client.1.smithi084.stdout:7/665: link db/d10/d17/d9c/d9f/db1/d36/c61 db/d5e/cc5 0 2024-02-28T23:07:40.415 INFO:tasks.workunit.client.1.smithi084.stdout:2/544: unlink d5/d7/d2e/f2f 0 2024-02-28T23:07:40.417 INFO:tasks.workunit.client.1.smithi084.stdout:7/666: chown db/d10/d17/d9c/d9f/fa5 3151 1 2024-02-28T23:07:40.419 INFO:tasks.workunit.client.1.smithi084.stdout:4/560: rename d1/d18/f42 to d1/d18/d20/d24/d2f/d3e/d52/d87/da5/fbf 0 2024-02-28T23:07:40.420 INFO:tasks.workunit.client.0.smithi057.stdout:4/481: rename d1a/d48/d97/f8a to d1a/d48/d9c/d68/fa6 0 2024-02-28T23:07:40.429 INFO:tasks.workunit.client.1.smithi084.stdout:2/545: creat d5/d1f/d42/d74/fa1 x:0 0 0 2024-02-28T23:07:40.431 INFO:tasks.workunit.client.1.smithi084.stdout:3/615: unlink d3/d27/d7d/db0/d14/d31/d34/cb3 0 2024-02-28T23:07:40.432 INFO:tasks.workunit.client.0.smithi057.stdout:7/467: dwrite d0/d5/d36/d4e/d15/d29/d3d/d51/f5e [0,4194304] 0 2024-02-28T23:07:40.435 INFO:tasks.workunit.client.0.smithi057.stdout:3/367: dwrite d3/d4/f12 [4194304,4194304] 0 2024-02-28T23:07:40.440 INFO:tasks.workunit.client.1.smithi084.stdout:5/663: dwrite dc/de/df/d31/d8e/da9/fb3 [0,4194304] 0 2024-02-28T23:07:40.442 INFO:tasks.workunit.client.1.smithi084.stdout:3/616: write d3/d27/d7d/db0/d1a/d21/d3d/f67 [826287,111215] 0 2024-02-28T23:07:40.442 INFO:tasks.workunit.client.0.smithi057.stdout:0/508: sync 2024-02-28T23:07:40.444 INFO:tasks.workunit.client.1.smithi084.stdout:7/667: dwrite db/d10/d17/d9c/d9f/db1/f53 [4194304,4194304] 0 2024-02-28T23:07:40.444 INFO:tasks.workunit.client.1.smithi084.stdout:2/546: rename d5/d64/f73 to d5/d7/d12/d24/d2a/d35/fa2 0 2024-02-28T23:07:40.445 INFO:tasks.workunit.client.1.smithi084.stdout:3/617: readlink d3/d27/l82 0 2024-02-28T23:07:40.446 INFO:tasks.workunit.client.0.smithi057.stdout:3/368: write d3/d4/d7/f10 [7579769,37372] 0 2024-02-28T23:07:40.448 INFO:tasks.workunit.client.0.smithi057.stdout:6/538: dwrite d0/d17/d20/f84 [0,4194304] 0 2024-02-28T23:07:40.449 INFO:tasks.workunit.client.1.smithi084.stdout:9/649: dwrite d4/f4e [0,4194304] 0 2024-02-28T23:07:40.452 INFO:tasks.workunit.client.1.smithi084.stdout:4/561: dwrite d1/d18/d20/d14/d30/d47/f49 [0,4194304] 0 2024-02-28T23:07:40.455 INFO:tasks.workunit.client.1.smithi084.stdout:2/547: chown d5/d7/d12/d24/d3a/d48/f8b 24912 1 2024-02-28T23:07:40.456 INFO:tasks.workunit.client.0.smithi057.stdout:8/537: dwrite d4/d1e/d56/d6a/d54/f70 [4194304,4194304] 0 2024-02-28T23:07:40.459 INFO:tasks.workunit.client.1.smithi084.stdout:4/562: write d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fbe [506068,116121] 0 2024-02-28T23:07:40.461 INFO:tasks.workunit.client.1.smithi084.stdout:3/618: symlink d3/d27/d7d/db0/d1a/le9 0 2024-02-28T23:07:40.463 INFO:tasks.workunit.client.1.smithi084.stdout:9/650: creat d4/d6/d15/d75/d9d/dd0/fd2 x:0 0 0 2024-02-28T23:07:40.466 INFO:tasks.workunit.client.1.smithi084.stdout:2/548: creat d5/d1f/d42/fa3 x:0 0 0 2024-02-28T23:07:40.471 INFO:tasks.workunit.client.1.smithi084.stdout:7/668: rename db/d11 to db/d5e/dc6 0 2024-02-28T23:07:40.473 INFO:tasks.workunit.client.0.smithi057.stdout:0/509: creat d17/d20/d26/d48/d7c/d98/da6/fbb x:0 0 0 2024-02-28T23:07:40.474 INFO:tasks.workunit.client.1.smithi084.stdout:3/619: creat d3/dae/fea x:0 0 0 2024-02-28T23:07:40.483 INFO:tasks.workunit.client.1.smithi084.stdout:4/563: write d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [561584,43441] 0 2024-02-28T23:07:40.484 INFO:tasks.workunit.client.1.smithi084.stdout:7/669: mkdir db/d5e/dc6/dc7 0 2024-02-28T23:07:40.493 INFO:tasks.workunit.client.1.smithi084.stdout:2/549: dwrite d5/f31 [0,4194304] 0 2024-02-28T23:07:40.493 INFO:tasks.workunit.client.1.smithi084.stdout:4/564: rename d1/d4a to d1/d18/d20/d24/d2f/d3e/d52/d87/dc0 0 2024-02-28T23:07:40.495 INFO:tasks.workunit.client.1.smithi084.stdout:7/670: symlink db/d10/d18/dc2/lc8 0 2024-02-28T23:07:40.496 INFO:tasks.workunit.client.1.smithi084.stdout:3/620: dread d3/d27/d7d/f8f [0,4194304] 0 2024-02-28T23:07:40.496 INFO:tasks.workunit.client.1.smithi084.stdout:2/550: fsync d5/d1f/f41 0 2024-02-28T23:07:40.498 INFO:tasks.workunit.client.0.smithi057.stdout:8/538: dread d4/d1e/d56/d6a/d54/f70 [0,4194304] 0 2024-02-28T23:07:40.504 INFO:tasks.workunit.client.0.smithi057.stdout:8/539: read d4/d13/d18/f65 [120322,127043] 0 2024-02-28T23:07:40.505 INFO:tasks.workunit.client.1.smithi084.stdout:2/551: symlink d5/d1f/d42/d74/la4 0 2024-02-28T23:07:40.506 INFO:tasks.workunit.client.1.smithi084.stdout:3/621: stat d3/d27/d7d/db0/df 0 2024-02-28T23:07:40.507 INFO:tasks.workunit.client.1.smithi084.stdout:4/565: dread d1/d18/d20/d65/d8e/f95 [0,4194304] 0 2024-02-28T23:07:40.508 INFO:tasks.workunit.client.0.smithi057.stdout:4/482: dread d1a/d48/d6c/f92 [4194304,4194304] 0 2024-02-28T23:07:40.509 INFO:tasks.workunit.client.1.smithi084.stdout:2/552: symlink d5/d7/d2e/d4a/la5 0 2024-02-28T23:07:40.509 INFO:tasks.workunit.client.0.smithi057.stdout:8/540: readlink d4/d1e/d56/da6/dae/lba 0 2024-02-28T23:07:40.510 INFO:tasks.workunit.client.1.smithi084.stdout:3/622: mkdir d3/d27/d4b/deb 0 2024-02-28T23:07:40.511 INFO:tasks.workunit.client.1.smithi084.stdout:4/566: chown d1/d18/d20/d24/d2f/d3e/d52/d45 2509521 1 2024-02-28T23:07:40.513 INFO:tasks.workunit.client.1.smithi084.stdout:2/553: mkdir d5/d7/d9/d7c/d5b/da6 0 2024-02-28T23:07:40.514 INFO:tasks.workunit.client.0.smithi057.stdout:8/541: symlink d4/d1e/d56/d88/lc2 0 2024-02-28T23:07:40.515 INFO:tasks.workunit.client.1.smithi084.stdout:3/623: creat d3/d50/fec x:0 0 0 2024-02-28T23:07:40.518 INFO:tasks.workunit.client.0.smithi057.stdout:4/483: link d1a/d48/d97/d2d/f59 d1a/d48/d9c/d68/fa7 0 2024-02-28T23:07:40.519 INFO:tasks.workunit.client.1.smithi084.stdout:4/567: chown d1/d18/d20/d65/d8e/d55/f69 245 1 2024-02-28T23:07:40.519 INFO:tasks.workunit.client.1.smithi084.stdout:7/671: dwrite db/d10/d18/db7/fbe [0,4194304] 0 2024-02-28T23:07:40.523 INFO:tasks.workunit.client.1.smithi084.stdout:3/624: mkdir d3/d27/d7d/db0/df/ded 0 2024-02-28T23:07:40.525 INFO:tasks.workunit.client.1.smithi084.stdout:7/672: truncate db/d10/d17/d64/d77/fa8 1027564 0 2024-02-28T23:07:40.528 INFO:tasks.workunit.client.1.smithi084.stdout:4/568: symlink d1/d18/d20/d14/d30/d47/da4/lc1 0 2024-02-28T23:07:40.532 INFO:tasks.workunit.client.1.smithi084.stdout:4/569: read - d1/d18/d20/d14/d30/d47/da4/faa zero size 2024-02-28T23:07:40.533 INFO:tasks.workunit.client.1.smithi084.stdout:7/673: rename db/d10/d17/d3b/f7c to db/d10/d18/dc2/fc9 0 2024-02-28T23:07:40.534 INFO:tasks.workunit.client.1.smithi084.stdout:5/664: dread dc/de/df/d16/f9a [0,4194304] 0 2024-02-28T23:07:40.536 INFO:tasks.workunit.client.1.smithi084.stdout:3/625: mkdir d3/d27/d7d/db0/d14/dee 0 2024-02-28T23:07:40.545 INFO:tasks.workunit.client.1.smithi084.stdout:4/570: readlink d1/d18/d20/d65/d8e/d55/l99 0 2024-02-28T23:07:40.545 INFO:tasks.workunit.client.1.smithi084.stdout:2/554: dwrite d5/dd/f58 [4194304,4194304] 0 2024-02-28T23:07:40.546 INFO:tasks.workunit.client.0.smithi057.stdout:3/369: dwrite d3/d8/d1b/f44 [0,4194304] 0 2024-02-28T23:07:40.549 INFO:tasks.workunit.client.1.smithi084.stdout:5/665: fdatasync dc/d51/d7c/f84 0 2024-02-28T23:07:40.549 INFO:tasks.workunit.client.1.smithi084.stdout:4/571: dread - d1/d18/d20/d24/f84 zero size 2024-02-28T23:07:40.550 INFO:tasks.workunit.client.0.smithi057.stdout:8/542: dwrite d4/f7 [12582912,4194304] 0 2024-02-28T23:07:40.550 INFO:tasks.workunit.client.1.smithi084.stdout:9/651: dwrite d4/d6/d53/f61 [0,4194304] 0 2024-02-28T23:07:40.551 INFO:tasks.workunit.client.0.smithi057.stdout:6/539: dread d0/d1b/f27 [0,4194304] 0 2024-02-28T23:07:40.553 INFO:tasks.workunit.client.0.smithi057.stdout:4/484: dwrite d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:40.555 INFO:tasks.workunit.client.1.smithi084.stdout:9/652: write d4/d6/d15/d12/fae [1465228,38905] 0 2024-02-28T23:07:40.557 INFO:tasks.workunit.client.1.smithi084.stdout:3/626: unlink d3/d27/d7d/db0/d14/c80 0 2024-02-28T23:07:40.561 INFO:tasks.workunit.client.0.smithi057.stdout:3/370: symlink d3/d4/d1d/l76 0 2024-02-28T23:07:40.561 INFO:tasks.workunit.client.0.smithi057.stdout:6/540: creat d0/d1b/d3e/d3d/d74/fb8 x:0 0 0 2024-02-28T23:07:40.562 INFO:tasks.workunit.client.1.smithi084.stdout:5/666: creat dc/de/df/da6/ff7 x:0 0 0 2024-02-28T23:07:40.564 INFO:tasks.workunit.client.1.smithi084.stdout:5/667: write dc/de/df/d6e/ff3 [716609,34181] 0 2024-02-28T23:07:40.565 INFO:tasks.workunit.client.1.smithi084.stdout:3/627: dread - d3/d27/d7d/db0/d1a/d21/d3d/faa zero size 2024-02-28T23:07:40.567 INFO:tasks.workunit.client.0.smithi057.stdout:4/485: write d1a/d48/d97/d2d/f35 [122950,106194] 0 2024-02-28T23:07:40.567 INFO:tasks.workunit.client.0.smithi057.stdout:2/568: sync 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: Upgrade: Updating mgr.smithi057.zrrldj 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:07:40.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:40 smithi057 ceph-mon[29096]: Deploying daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T23:07:40.571 INFO:tasks.workunit.client.0.smithi057.stdout:0/510: dwrite f5 [4194304,4194304] 0 2024-02-28T23:07:40.572 INFO:tasks.workunit.client.1.smithi084.stdout:5/668: write dc/de/df/d16/d34/d3c/d77/fbd [994097,16614] 0 2024-02-28T23:07:40.574 INFO:tasks.workunit.client.1.smithi084.stdout:2/555: getdents d5/d7/d12/d24/d3a 0 2024-02-28T23:07:40.576 INFO:tasks.workunit.client.0.smithi057.stdout:4/486: read d1a/d48/d97/d2d/d49/d4d/f88 [1754799,68187] 0 2024-02-28T23:07:40.580 INFO:tasks.workunit.client.0.smithi057.stdout:2/569: creat d8/dae/fb8 x:0 0 0 2024-02-28T23:07:40.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: Upgrade: Updating mgr.smithi057.zrrldj 2024-02-28T23:07:40.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:07:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:07:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:07:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:07:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:40 smithi084 ceph-mon[34826]: Deploying daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T23:07:40.582 INFO:tasks.workunit.client.1.smithi084.stdout:9/653: dread d4/f56 [0,4194304] 0 2024-02-28T23:07:40.584 INFO:tasks.workunit.client.0.smithi057.stdout:0/511: chown d17/d20/d23/f8b 79774 1 2024-02-28T23:07:40.584 INFO:tasks.workunit.client.0.smithi057.stdout:6/541: symlink d0/d1b/d3e/d3d/d42/d4f/d92/lb9 0 2024-02-28T23:07:40.587 INFO:tasks.workunit.client.1.smithi084.stdout:3/628: dread d3/d27/f2d [0,4194304] 0 2024-02-28T23:07:40.588 INFO:tasks.workunit.client.0.smithi057.stdout:6/542: chown d0/d1b/d3e/d3d/d42/d4f/d53/f93 0 1 2024-02-28T23:07:40.588 INFO:tasks.workunit.client.0.smithi057.stdout:3/371: dwrite d3/d4f/d4c/f54 [0,4194304] 0 2024-02-28T23:07:40.589 INFO:tasks.workunit.client.1.smithi084.stdout:9/654: truncate d4/d6/d15/d12/fae 2117131 0 2024-02-28T23:07:40.590 INFO:tasks.workunit.client.1.smithi084.stdout:9/655: fdatasync d4/d32/f92 0 2024-02-28T23:07:40.591 INFO:tasks.workunit.client.0.smithi057.stdout:2/570: rmdir d8/d11/d21/d28/d49/d85/d92 39 2024-02-28T23:07:40.596 INFO:tasks.workunit.client.0.smithi057.stdout:3/372: fdatasync d3/d4/f30 0 2024-02-28T23:07:40.596 INFO:tasks.workunit.client.1.smithi084.stdout:3/629: truncate d3/d27/d4b/fd6 638123 0 2024-02-28T23:07:40.596 INFO:tasks.workunit.client.1.smithi084.stdout:9/656: creat d4/d32/d3f/d89/fd3 x:0 0 0 2024-02-28T23:07:40.598 INFO:tasks.workunit.client.1.smithi084.stdout:9/657: dread - d4/d6/d53/fc0 zero size 2024-02-28T23:07:40.600 INFO:tasks.workunit.client.1.smithi084.stdout:4/572: dwrite d1/d18/d20/d65/f72 [0,4194304] 0 2024-02-28T23:07:40.601 INFO:tasks.workunit.client.0.smithi057.stdout:8/543: truncate d4/d13/d2c/d4e/f87 59914 0 2024-02-28T23:07:40.602 INFO:tasks.workunit.client.0.smithi057.stdout:2/571: write d8/d11/d21/d28/d49/d85/f9f [623540,117047] 0 2024-02-28T23:07:40.604 INFO:tasks.workunit.client.1.smithi084.stdout:7/674: truncate db/d10/d17/d9c/d9f/db1/f3a 4138630 0 2024-02-28T23:07:40.604 INFO:tasks.workunit.client.0.smithi057.stdout:3/373: unlink d3/l46 0 2024-02-28T23:07:40.605 INFO:tasks.workunit.client.0.smithi057.stdout:8/544: readlink d4/d1e/d56/d88/lc2 0 2024-02-28T23:07:40.607 INFO:tasks.workunit.client.1.smithi084.stdout:3/630: getdents d3/d27/d7d 0 2024-02-28T23:07:40.608 INFO:tasks.workunit.client.0.smithi057.stdout:4/487: dwrite d1a/d2f/d9b/f5d [0,4194304] 0 2024-02-28T23:07:40.608 INFO:tasks.workunit.client.1.smithi084.stdout:4/573: readlink d1/d18/d20/d65/d8e/l46 0 2024-02-28T23:07:40.609 INFO:tasks.workunit.client.0.smithi057.stdout:6/543: link d0/d1b/d3e/lad d0/d1b/d3e/d3d/d42/d4f/lba 0 2024-02-28T23:07:40.610 INFO:tasks.workunit.client.1.smithi084.stdout:3/631: truncate d3/d50/d5a/fab 858740 0 2024-02-28T23:07:40.611 INFO:tasks.workunit.client.0.smithi057.stdout:6/544: read d0/d4/d11/f9a [1991037,64507] 0 2024-02-28T23:07:40.612 INFO:tasks.workunit.client.1.smithi084.stdout:9/658: creat d4/d6/d15/fd4 x:0 0 0 2024-02-28T23:07:40.613 INFO:tasks.workunit.client.1.smithi084.stdout:4/574: mknod d1/d18/d20/d24/d2f/d3e/d52/d87/da5/cc2 0 2024-02-28T23:07:40.614 INFO:tasks.workunit.client.0.smithi057.stdout:4/488: creat d1a/d1e/fa8 x:0 0 0 2024-02-28T23:07:40.614 INFO:tasks.workunit.client.0.smithi057.stdout:3/374: read d3/d4/d7/f2b [2828797,21588] 0 2024-02-28T23:07:40.616 INFO:tasks.workunit.client.0.smithi057.stdout:6/545: read d0/d4/d25/f26 [164893,126874] 0 2024-02-28T23:07:40.617 INFO:tasks.workunit.client.1.smithi084.stdout:9/659: chown d4/d32/d3f/d89/f99 1 1 2024-02-28T23:07:40.617 INFO:tasks.workunit.client.1.smithi084.stdout:0/600: sync 2024-02-28T23:07:40.619 INFO:tasks.workunit.client.0.smithi057.stdout:4/489: symlink d1a/d1e/la9 0 2024-02-28T23:07:40.621 INFO:tasks.workunit.client.1.smithi084.stdout:2/556: truncate d5/dd/f58 1108026 0 2024-02-28T23:07:40.622 INFO:tasks.workunit.client.0.smithi057.stdout:4/490: readlink d1a/d48/d9c/l65 0 2024-02-28T23:07:40.622 INFO:tasks.workunit.client.1.smithi084.stdout:3/632: mknod d3/d27/d7d/db0/d1a/d45/d86/cef 0 2024-02-28T23:07:40.623 INFO:tasks.workunit.client.0.smithi057.stdout:0/512: write d17/d20/d26/d30/d6b/d71/f92 [444010,130358] 0 2024-02-28T23:07:40.623 INFO:tasks.workunit.client.0.smithi057.stdout:3/375: read d3/d4/f17 [2755363,73890] 0 2024-02-28T23:07:40.625 INFO:tasks.workunit.client.0.smithi057.stdout:5/505: sync 2024-02-28T23:07:40.625 INFO:tasks.workunit.client.0.smithi057.stdout:1/382: sync 2024-02-28T23:07:40.625 INFO:tasks.workunit.client.0.smithi057.stdout:9/505: sync 2024-02-28T23:07:40.625 INFO:tasks.workunit.client.0.smithi057.stdout:7/468: sync 2024-02-28T23:07:40.626 INFO:tasks.workunit.client.1.smithi084.stdout:4/575: mkdir d1/d18/d20/d65/d8e/d55/dc3 0 2024-02-28T23:07:40.627 INFO:tasks.workunit.client.1.smithi084.stdout:3/633: write d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c [1716035,116857] 0 2024-02-28T23:07:40.629 INFO:tasks.workunit.client.1.smithi084.stdout:0/601: creat d1/d8/d69/d6b/fbe x:0 0 0 2024-02-28T23:07:40.629 INFO:tasks.workunit.client.1.smithi084.stdout:7/675: dwrite db/d10/d18/dc2/fc9 [4194304,4194304] 0 2024-02-28T23:07:40.630 INFO:tasks.workunit.client.1.smithi084.stdout:9/660: truncate d4/d6/d15/d12/d41/f64 623730 0 2024-02-28T23:07:40.631 INFO:tasks.workunit.client.0.smithi057.stdout:2/572: dread d8/d11/d15/f35 [0,4194304] 0 2024-02-28T23:07:40.631 INFO:tasks.workunit.client.0.smithi057.stdout:6/546: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:40.632 INFO:tasks.workunit.client.0.smithi057.stdout:4/491: creat d1a/d48/d97/d7f/faa x:0 0 0 2024-02-28T23:07:40.634 INFO:tasks.workunit.client.1.smithi084.stdout:3/634: chown d3/d27/d7d/db0/d1a/d21/d7b 35565028 1 2024-02-28T23:07:40.634 INFO:tasks.workunit.client.1.smithi084.stdout:4/576: unlink d1/d18/d20/d24/d2f/fb7 0 2024-02-28T23:07:40.634 INFO:tasks.workunit.client.1.smithi084.stdout:9/661: rmdir d4/d32 39 2024-02-28T23:07:40.635 INFO:tasks.workunit.client.0.smithi057.stdout:1/383: symlink d2/d33/l85 0 2024-02-28T23:07:40.638 INFO:tasks.workunit.client.0.smithi057.stdout:3/376: unlink d3/d4/d7/l5b 0 2024-02-28T23:07:40.639 INFO:tasks.workunit.client.0.smithi057.stdout:1/384: readlink d2/d33/l85 0 2024-02-28T23:07:40.641 INFO:tasks.workunit.client.0.smithi057.stdout:9/506: truncate dd/f2d 2064682 0 2024-02-28T23:07:40.643 INFO:tasks.workunit.client.1.smithi084.stdout:3/635: mknod d3/d50/d5a/d73/dc8/cf0 0 2024-02-28T23:07:40.645 INFO:tasks.workunit.client.1.smithi084.stdout:4/577: rename d1/d18/d20/d24/d2f/d3e/l4e to d1/d18/d20/d65/lc4 0 2024-02-28T23:07:40.646 INFO:tasks.workunit.client.0.smithi057.stdout:3/377: read d3/d8/d1b/f44 [3219894,65723] 0 2024-02-28T23:07:40.646 INFO:tasks.workunit.client.0.smithi057.stdout:4/492: truncate d1a/d48/d97/d2d/f59 798586 0 2024-02-28T23:07:40.646 INFO:tasks.workunit.client.0.smithi057.stdout:2/573: mkdir d8/d11/d15/db9 0 2024-02-28T23:07:40.647 INFO:tasks.workunit.client.1.smithi084.stdout:4/578: stat d1/f9c 0 2024-02-28T23:07:40.648 INFO:tasks.workunit.client.0.smithi057.stdout:7/469: truncate d0/d5/d36/d4e/d15/d29/f70 738866 0 2024-02-28T23:07:40.649 INFO:tasks.workunit.client.0.smithi057.stdout:6/547: creat d0/d1b/d3e/d3d/d42/d4f/fbb x:0 0 0 2024-02-28T23:07:40.650 INFO:tasks.workunit.client.1.smithi084.stdout:4/579: chown d1/d18/d20/d65 23725607 1 2024-02-28T23:07:40.652 INFO:tasks.workunit.client.1.smithi084.stdout:3/636: symlink d3/d27/d7d/db0/d14/lf1 0 2024-02-28T23:07:40.652 INFO:tasks.workunit.client.1.smithi084.stdout:4/580: readlink d1/d18/d20/d65/d8e/d55/l21 0 2024-02-28T23:07:40.653 INFO:tasks.workunit.client.0.smithi057.stdout:6/548: fdatasync d0/d1b/d3e/d3d/f9d 0 2024-02-28T23:07:40.654 INFO:tasks.workunit.client.1.smithi084.stdout:3/637: chown d3/d27/l33 0 1 2024-02-28T23:07:40.654 INFO:tasks.workunit.client.1.smithi084.stdout:4/581: chown d1/d18/d20/d24/d2f/d3e 3 1 2024-02-28T23:07:40.655 INFO:tasks.workunit.client.0.smithi057.stdout:0/513: getdents d17/d20/d4a/d78 0 2024-02-28T23:07:40.656 INFO:tasks.workunit.client.0.smithi057.stdout:3/378: creat d3/d8/d27/f77 x:0 0 0 2024-02-28T23:07:40.658 INFO:tasks.workunit.client.0.smithi057.stdout:4/493: rmdir d1a/d1e/d51 39 2024-02-28T23:07:40.661 INFO:tasks.workunit.client.0.smithi057.stdout:0/514: rmdir d17/d20/d23 39 2024-02-28T23:07:40.662 INFO:tasks.workunit.client.1.smithi084.stdout:3/638: chown d3/d27/d7d/db0/d1a/d29/d41/d60/f6b 49024 1 2024-02-28T23:07:40.663 INFO:tasks.workunit.client.0.smithi057.stdout:9/507: link dd/d46/d6f/fa0 dd/d1b/d40/d44/fa4 0 2024-02-28T23:07:40.663 INFO:tasks.workunit.client.0.smithi057.stdout:3/379: fsync d3/d4/f30 0 2024-02-28T23:07:40.664 INFO:tasks.workunit.client.0.smithi057.stdout:6/549: write d0/d4/d25/d60/f97 [1036311,74608] 0 2024-02-28T23:07:40.667 INFO:tasks.workunit.client.0.smithi057.stdout:8/545: truncate d4/d1e/d56/d6a/d54/f70 566356 0 2024-02-28T23:07:40.668 INFO:tasks.workunit.client.1.smithi084.stdout:2/557: dread d5/d7/d12/f51 [0,4194304] 0 2024-02-28T23:07:40.668 INFO:tasks.workunit.client.1.smithi084.stdout:9/662: dread d4/f90 [0,4194304] 0 2024-02-28T23:07:40.672 INFO:tasks.workunit.client.1.smithi084.stdout:2/558: fsync d5/d7/d9/d7c/f95 0 2024-02-28T23:07:40.673 INFO:tasks.workunit.client.0.smithi057.stdout:5/506: write d4/d7/d8/ff [855157,55277] 0 2024-02-28T23:07:40.673 INFO:tasks.workunit.client.1.smithi084.stdout:7/676: write db/d10/d17/d9c/d9f/db1/d36/f4c [2286995,29621] 0 2024-02-28T23:07:40.674 INFO:tasks.workunit.client.0.smithi057.stdout:6/550: read - d0/d1b/d3e/d3d/f9d zero size 2024-02-28T23:07:40.675 INFO:tasks.workunit.client.1.smithi084.stdout:0/602: write d1/d6/db/d37/f71 [553298,95282] 0 2024-02-28T23:07:40.676 INFO:tasks.workunit.client.0.smithi057.stdout:4/494: mkdir d1a/d48/d9c/d2e/dab 0 2024-02-28T23:07:40.678 INFO:tasks.workunit.client.1.smithi084.stdout:4/582: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [0,4194304] 0 2024-02-28T23:07:40.681 INFO:tasks.workunit.client.1.smithi084.stdout:2/559: creat d5/d7/d12/d24/fa7 x:0 0 0 2024-02-28T23:07:40.682 INFO:tasks.workunit.client.0.smithi057.stdout:2/574: dwrite d8/d11/d48/d50/f66 [4194304,4194304] 0 2024-02-28T23:07:40.682 INFO:tasks.workunit.client.1.smithi084.stdout:4/583: stat d1/d18/d20/c90 0 2024-02-28T23:07:40.682 INFO:tasks.workunit.client.0.smithi057.stdout:9/508: mkdir dd/d1b/d40/d44/da5 0 2024-02-28T23:07:40.684 INFO:tasks.workunit.client.1.smithi084.stdout:9/663: truncate d4/d6/d15/d75/fa4 742404 0 2024-02-28T23:07:40.686 INFO:tasks.workunit.client.0.smithi057.stdout:9/509: chown dd/de/d4a 228 1 2024-02-28T23:07:40.687 INFO:tasks.workunit.client.0.smithi057.stdout:5/507: creat d4/d9/d14/d3b/d61/d69/fa4 x:0 0 0 2024-02-28T23:07:40.691 INFO:tasks.workunit.client.1.smithi084.stdout:2/560: write d5/d7/d12/d24/d3a/f96 [823807,25549] 0 2024-02-28T23:07:40.692 INFO:tasks.workunit.client.0.smithi057.stdout:2/575: write d8/f19 [3921439,22006] 0 2024-02-28T23:07:40.692 INFO:tasks.workunit.client.0.smithi057.stdout:7/470: dwrite d0/d5/d36/d4e/f19 [0,4194304] 0 2024-02-28T23:07:40.693 INFO:tasks.workunit.client.0.smithi057.stdout:9/510: chown dd/de/l1a 991909405 1 2024-02-28T23:07:40.693 INFO:tasks.workunit.client.0.smithi057.stdout:2/576: chown d8/c2c 2141197616 1 2024-02-28T23:07:40.694 INFO:tasks.workunit.client.1.smithi084.stdout:4/584: symlink d1/d18/d20/d24/d2f/d3e/lc5 0 2024-02-28T23:07:40.695 INFO:tasks.workunit.client.0.smithi057.stdout:3/380: symlink d3/d4/d1d/d47/d69/l78 0 2024-02-28T23:07:40.696 INFO:tasks.workunit.client.1.smithi084.stdout:3/639: dwrite d3/d27/d7d/db0/df/f19 [0,4194304] 0 2024-02-28T23:07:40.698 INFO:tasks.workunit.client.0.smithi057.stdout:1/385: dread d2/d71/d2d/f3d [0,4194304] 0 2024-02-28T23:07:40.698 INFO:tasks.workunit.client.1.smithi084.stdout:4/585: chown d1/c13 11 1 2024-02-28T23:07:40.699 INFO:tasks.workunit.client.0.smithi057.stdout:4/495: mknod d1a/d2f/d9b/d4f/d6e/cac 0 2024-02-28T23:07:40.701 INFO:tasks.workunit.client.1.smithi084.stdout:3/640: write d3/d27/d7d/db0/fa [879065,42336] 0 2024-02-28T23:07:40.701 INFO:tasks.workunit.client.1.smithi084.stdout:9/664: creat d4/d6/d15/d75/d2d/daa/fd5 x:0 0 0 2024-02-28T23:07:40.704 INFO:tasks.workunit.client.1.smithi084.stdout:0/603: dread d1/d6/db/d35/d4a/d56/f89 [0,4194304] 0 2024-02-28T23:07:40.706 INFO:tasks.workunit.client.1.smithi084.stdout:7/677: dread db/d10/d18/dc2/fc9 [4194304,4194304] 0 2024-02-28T23:07:40.708 INFO:tasks.workunit.client.0.smithi057.stdout:9/511: creat dd/de/d24/fa6 x:0 0 0 2024-02-28T23:07:40.708 INFO:tasks.workunit.client.1.smithi084.stdout:7/678: write db/d5e/f6f [709064,73574] 0 2024-02-28T23:07:40.710 INFO:tasks.workunit.client.0.smithi057.stdout:3/381: mknod d3/d8/d27/d70/c79 0 2024-02-28T23:07:40.713 INFO:tasks.workunit.client.1.smithi084.stdout:3/641: symlink d3/d27/d7d/db0/d1a/d21/lf2 0 2024-02-28T23:07:40.713 INFO:tasks.workunit.client.1.smithi084.stdout:0/604: mknod d1/d8/d18/d4c/cbf 0 2024-02-28T23:07:40.714 INFO:tasks.workunit.client.0.smithi057.stdout:6/551: rename d0/d17/d94/da0/fa3 to d0/d17/fbc 0 2024-02-28T23:07:40.720 INFO:tasks.workunit.client.1.smithi084.stdout:7/679: link db/d10/d17/l37 db/d10/d17/d9c/lca 0 2024-02-28T23:07:40.722 INFO:tasks.workunit.client.1.smithi084.stdout:3/642: mknod d3/d27/d7d/db0/df/ded/cf3 0 2024-02-28T23:07:40.726 INFO:tasks.workunit.client.1.smithi084.stdout:2/561: truncate d5/d7/d2e/f46 1421479 0 2024-02-28T23:07:40.727 INFO:tasks.workunit.client.1.smithi084.stdout:4/586: truncate d1/f4 4923790 0 2024-02-28T23:07:40.728 INFO:tasks.workunit.client.1.smithi084.stdout:9/665: write d4/d6/d15/d3c/f47 [183250,13280] 0 2024-02-28T23:07:40.728 INFO:tasks.workunit.client.0.smithi057.stdout:5/508: dwrite d4/d7/f73 [0,4194304] 0 2024-02-28T23:07:40.730 INFO:tasks.workunit.client.1.smithi084.stdout:2/562: fsync d5/d7/d2e/f3d 0 2024-02-28T23:07:40.731 INFO:tasks.workunit.client.0.smithi057.stdout:2/577: dwrite d8/d11/d39/f4c [0,4194304] 0 2024-02-28T23:07:40.733 INFO:tasks.workunit.client.1.smithi084.stdout:3/643: mkdir d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4 0 2024-02-28T23:07:40.733 INFO:tasks.workunit.client.1.smithi084.stdout:4/587: readlink d1/d18/d20/d14/l7f 0 2024-02-28T23:07:40.734 INFO:tasks.workunit.client.1.smithi084.stdout:9/666: chown d4/d32/c80 4546930 1 2024-02-28T23:07:40.735 INFO:tasks.workunit.client.0.smithi057.stdout:1/386: symlink d2/d7/df/db/d12/d3f/d47/l86 0 2024-02-28T23:07:40.736 INFO:tasks.workunit.client.0.smithi057.stdout:9/512: creat dd/d1b/d40/d44/d5c/fa7 x:0 0 0 2024-02-28T23:07:40.737 INFO:tasks.workunit.client.1.smithi084.stdout:4/588: chown d1/d18/d20/d14/d30/d47 1 1 2024-02-28T23:07:40.737 INFO:tasks.workunit.client.0.smithi057.stdout:7/471: rename d0/d5/d36/d4e/c8e to d0/d5/d4a/c95 0 2024-02-28T23:07:40.739 INFO:tasks.workunit.client.0.smithi057.stdout:5/509: chown d4/d9/d14/f36 0 1 2024-02-28T23:07:40.740 INFO:tasks.workunit.client.0.smithi057.stdout:6/552: mkdir d0/d17/d20/dbd 0 2024-02-28T23:07:40.742 INFO:tasks.workunit.client.1.smithi084.stdout:0/605: dread d1/d8/f55 [0,4194304] 0 2024-02-28T23:07:40.742 INFO:tasks.workunit.client.1.smithi084.stdout:2/563: link d5/d7/d9/d7c/d5b/d9f/fa0 d5/d1f/d42/d8f/fa8 0 2024-02-28T23:07:40.744 INFO:tasks.workunit.client.0.smithi057.stdout:5/510: chown d4/c49 1 1 2024-02-28T23:07:40.745 INFO:tasks.workunit.client.1.smithi084.stdout:9/667: creat d4/d6/fd6 x:0 0 0 2024-02-28T23:07:40.745 INFO:tasks.workunit.client.0.smithi057.stdout:0/515: dread f4 [0,4194304] 0 2024-02-28T23:07:40.746 INFO:tasks.workunit.client.1.smithi084.stdout:2/564: fsync d5/d1f/f44 0 2024-02-28T23:07:40.749 INFO:tasks.workunit.client.1.smithi084.stdout:4/589: rename d1/d18/d20/d24/f34 to d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fc6 0 2024-02-28T23:07:40.750 INFO:tasks.workunit.client.0.smithi057.stdout:7/472: symlink d0/d5/d25/d3e/l96 0 2024-02-28T23:07:40.750 INFO:tasks.workunit.client.1.smithi084.stdout:0/606: mkdir d1/dc0 0 2024-02-28T23:07:40.751 INFO:tasks.workunit.client.0.smithi057.stdout:2/578: creat d8/d11/d21/d28/d49/d85/d92/fba x:0 0 0 2024-02-28T23:07:40.753 INFO:tasks.workunit.client.0.smithi057.stdout:0/516: write d17/f9c [811267,109698] 0 2024-02-28T23:07:40.754 INFO:tasks.workunit.client.1.smithi084.stdout:0/607: truncate d1/d8/d69/d6b/fbe 409604 0 2024-02-28T23:07:40.755 INFO:tasks.workunit.client.0.smithi057.stdout:2/579: readlink d8/l8f 0 2024-02-28T23:07:40.755 INFO:tasks.workunit.client.0.smithi057.stdout:4/496: dwrite d1a/d48/d9c/d68/fa6 [0,4194304] 0 2024-02-28T23:07:40.757 INFO:tasks.workunit.client.0.smithi057.stdout:3/382: rename d3/f51 to d3/d4/f7a 0 2024-02-28T23:07:40.758 INFO:tasks.workunit.client.1.smithi084.stdout:0/608: write d1/d8/d69/db2/f54 [2298646,117813] 0 2024-02-28T23:07:40.759 INFO:tasks.workunit.client.0.smithi057.stdout:7/473: chown d0/d5/d36/d4e/l38 3 1 2024-02-28T23:07:40.762 INFO:tasks.workunit.client.1.smithi084.stdout:0/609: chown d1/d6/db/d37/d72/fb6 2810 1 2024-02-28T23:07:40.762 INFO:tasks.workunit.client.1.smithi084.stdout:3/644: write d3/f7 [23802,85783] 0 2024-02-28T23:07:40.763 INFO:tasks.workunit.client.1.smithi084.stdout:9/668: rename d4/d6/ff to d4/d6/d15/d75/d2d/dbd/fd7 0 2024-02-28T23:07:40.764 INFO:tasks.workunit.client.0.smithi057.stdout:2/580: unlink d8/d11/d21/c3a 0 2024-02-28T23:07:40.765 INFO:tasks.workunit.client.1.smithi084.stdout:0/610: chown d1/d8/d18/f9a 63916966 1 2024-02-28T23:07:40.766 INFO:tasks.workunit.client.1.smithi084.stdout:3/645: mknod d3/d50/d5a/d73/cf5 0 2024-02-28T23:07:40.767 INFO:tasks.workunit.client.0.smithi057.stdout:0/517: chown d17/d20/d23/fb8 285110530 1 2024-02-28T23:07:40.769 INFO:tasks.workunit.client.0.smithi057.stdout:4/497: mknod d1a/d2f/d9b/cad 0 2024-02-28T23:07:40.770 INFO:tasks.workunit.client.1.smithi084.stdout:4/590: rename d1/d18/d20/d65/d8e/d55/c98 to d1/d18/d20/d24/da8/cc7 0 2024-02-28T23:07:40.773 INFO:tasks.workunit.client.1.smithi084.stdout:0/611: creat d1/d6/db/d35/d4a/d56/fc1 x:0 0 0 2024-02-28T23:07:40.775 INFO:tasks.workunit.client.1.smithi084.stdout:2/565: write d5/dd/f16 [592081,48526] 0 2024-02-28T23:07:40.775 INFO:tasks.workunit.client.1.smithi084.stdout:4/591: chown d1/d18/d20/d14/d30/c3c 1827359 1 2024-02-28T23:07:40.776 INFO:tasks.workunit.client.0.smithi057.stdout:1/387: dread d2/d71/d2d/f31 [0,4194304] 0 2024-02-28T23:07:40.778 INFO:tasks.workunit.client.0.smithi057.stdout:6/553: dwrite d0/d4/d25/d60/d6a/f7e [0,4194304] 0 2024-02-28T23:07:40.778 INFO:tasks.workunit.client.0.smithi057.stdout:2/581: write d8/d11/f5b [4133595,109199] 0 2024-02-28T23:07:40.779 INFO:tasks.workunit.client.0.smithi057.stdout:7/474: mknod d0/d5/d36/d4e/d15/d29/d3d/c97 0 2024-02-28T23:07:40.780 INFO:tasks.workunit.client.1.smithi084.stdout:4/592: dread - d1/d18/d20/d24/d2f/d3e/d52/d45/f92 zero size 2024-02-28T23:07:40.782 INFO:tasks.workunit.client.0.smithi057.stdout:1/388: chown d2/d7/df/db/d3a/f70 0 1 2024-02-28T23:07:40.784 INFO:tasks.workunit.client.1.smithi084.stdout:3/646: dread d3/fbd [0,4194304] 0 2024-02-28T23:07:40.786 INFO:tasks.workunit.client.1.smithi084.stdout:9/669: dread d4/d6/d15/f98 [0,4194304] 0 2024-02-28T23:07:40.786 INFO:tasks.workunit.client.0.smithi057.stdout:4/498: chown d1a/d2f/c62 0 1 2024-02-28T23:07:40.786 INFO:tasks.workunit.client.0.smithi057.stdout:1/389: dread - d2/d7/f37 zero size 2024-02-28T23:07:40.787 INFO:tasks.workunit.client.0.smithi057.stdout:0/518: mknod d17/d20/d23/d4b/cbc 0 2024-02-28T23:07:40.787 INFO:tasks.workunit.client.0.smithi057.stdout:3/383: dread d3/d4f/d4c/f54 [0,4194304] 0 2024-02-28T23:07:40.787 INFO:tasks.workunit.client.0.smithi057.stdout:4/499: stat d1a/d1e 0 2024-02-28T23:07:40.787 INFO:tasks.workunit.client.0.smithi057.stdout:4/500: stat d1a/d48/d97/d2d/f59 0 2024-02-28T23:07:40.789 INFO:tasks.workunit.client.1.smithi084.stdout:3/647: creat d3/d50/ff6 x:0 0 0 2024-02-28T23:07:40.789 INFO:tasks.workunit.client.1.smithi084.stdout:4/593: read d1/d18/d20/d14/d30/f31 [716385,85260] 0 2024-02-28T23:07:40.793 INFO:tasks.workunit.client.1.smithi084.stdout:3/648: dread - d3/d27/d7d/db0/d1a/d21/d3d/faa zero size 2024-02-28T23:07:40.796 INFO:tasks.workunit.client.1.smithi084.stdout:4/594: mknod d1/d18/d20/d24/d2f/cc8 0 2024-02-28T23:07:40.796 INFO:tasks.workunit.client.0.smithi057.stdout:0/519: unlink d17/d20/d23/f8b 0 2024-02-28T23:07:40.801 INFO:tasks.workunit.client.0.smithi057.stdout:3/384: chown d3/d8/d1b/c53 725156 1 2024-02-28T23:07:40.802 INFO:tasks.workunit.client.1.smithi084.stdout:4/595: mknod d1/d18/d20/d24/d2f/cc9 0 2024-02-28T23:07:40.802 INFO:tasks.workunit.client.0.smithi057.stdout:2/582: dwrite d8/fb [0,4194304] 0 2024-02-28T23:07:40.802 INFO:tasks.workunit.client.0.smithi057.stdout:4/501: symlink d1a/d48/d97/d2d/d49/lae 0 2024-02-28T23:07:40.803 INFO:tasks.workunit.client.0.smithi057.stdout:2/583: chown d8/d11/d21/d28 60873 1 2024-02-28T23:07:40.804 INFO:tasks.workunit.client.1.smithi084.stdout:9/670: dwrite d4/f4e [0,4194304] 0 2024-02-28T23:07:40.808 INFO:tasks.workunit.client.0.smithi057.stdout:0/520: rmdir d17/d20/d26/d48/d7c/d98/da6 39 2024-02-28T23:07:40.810 INFO:tasks.workunit.client.0.smithi057.stdout:0/521: fsync d17/d20/d23/d47/f7f 0 2024-02-28T23:07:40.811 INFO:tasks.workunit.client.0.smithi057.stdout:0/522: dread - d17/f83 zero size 2024-02-28T23:07:40.811 INFO:tasks.workunit.client.1.smithi084.stdout:4/596: link d1/f9c d1/d4b/fca 0 2024-02-28T23:07:40.812 INFO:tasks.workunit.client.0.smithi057.stdout:4/502: chown d1a/d1e/l7a 432881 1 2024-02-28T23:07:40.815 INFO:tasks.workunit.client.0.smithi057.stdout:9/513: truncate dd/de/f43 297007 0 2024-02-28T23:07:40.815 INFO:tasks.workunit.client.1.smithi084.stdout:9/671: creat d4/d6/d15/d75/d9d/dc3/fd8 x:0 0 0 2024-02-28T23:07:40.815 INFO:tasks.workunit.client.1.smithi084.stdout:4/597: write d1/d18/f5f [422904,104469] 0 2024-02-28T23:07:40.815 INFO:tasks.workunit.client.1.smithi084.stdout:3/649: dwrite d3/d27/d7d/db0/d1a/d45/d5e/f62 [0,4194304] 0 2024-02-28T23:07:40.817 INFO:tasks.workunit.client.0.smithi057.stdout:1/390: rename d2/d71/f4a to d2/f87 0 2024-02-28T23:07:40.820 INFO:tasks.workunit.client.0.smithi057.stdout:3/385: read d3/d4/d7/f10 [7743004,87678] 0 2024-02-28T23:07:40.821 INFO:tasks.workunit.client.1.smithi084.stdout:2/566: dwrite d5/d7/d2e/d4a/d5c/f89 [4194304,4194304] 0 2024-02-28T23:07:40.822 INFO:tasks.workunit.client.1.smithi084.stdout:3/650: read - d3/d27/d7d/db0/d1a/d21/d7b/f88 zero size 2024-02-28T23:07:40.822 INFO:tasks.workunit.client.0.smithi057.stdout:4/503: unlink d1a/d1e/f6b 0 2024-02-28T23:07:40.826 INFO:tasks.workunit.client.0.smithi057.stdout:9/514: dread - dd/de/f6b zero size 2024-02-28T23:07:40.826 INFO:tasks.workunit.client.0.smithi057.stdout:5/511: truncate d4/d21/f2d 1402775 0 2024-02-28T23:07:40.828 INFO:tasks.workunit.client.1.smithi084.stdout:3/651: creat d3/d50/d5a/d73/dc8/ff7 x:0 0 0 2024-02-28T23:07:40.829 INFO:tasks.workunit.client.1.smithi084.stdout:2/567: mknod d5/ca9 0 2024-02-28T23:07:40.829 INFO:tasks.workunit.client.0.smithi057.stdout:0/523: mknod d17/d20/d26/d48/d7c/d98/cbd 0 2024-02-28T23:07:40.832 INFO:tasks.workunit.client.1.smithi084.stdout:2/568: write d5/d7/d9/f88 [820818,63654] 0 2024-02-28T23:07:40.833 INFO:tasks.workunit.client.1.smithi084.stdout:9/672: link d4/d6/lcd d4/d6/d15/d75/d9d/dd0/ld9 0 2024-02-28T23:07:40.835 INFO:tasks.workunit.client.1.smithi084.stdout:9/673: read d4/f90 [1775895,50189] 0 2024-02-28T23:07:40.836 INFO:tasks.workunit.client.0.smithi057.stdout:2/584: dwrite d8/d11/d21/d28/d41/f53 [0,4194304] 0 2024-02-28T23:07:40.836 INFO:tasks.workunit.client.1.smithi084.stdout:3/652: symlink d3/d27/d7d/db0/d1a/d45/d86/lf8 0 2024-02-28T23:07:40.837 INFO:tasks.workunit.client.1.smithi084.stdout:2/569: chown d5/d7/d9/d94/f98 23665 1 2024-02-28T23:07:40.837 INFO:tasks.workunit.client.0.smithi057.stdout:6/554: truncate d0/d17/d20/f31 2255164 0 2024-02-28T23:07:40.839 INFO:tasks.workunit.client.1.smithi084.stdout:1/677: sync 2024-02-28T23:07:40.841 INFO:tasks.workunit.client.1.smithi084.stdout:2/570: chown d5/d1f/d42/l57 189118 1 2024-02-28T23:07:40.843 INFO:tasks.workunit.client.0.smithi057.stdout:1/391: mknod d2/d71/d2d/d39/c88 0 2024-02-28T23:07:40.845 INFO:tasks.workunit.client.1.smithi084.stdout:2/571: chown d5/d7/d12/d24/d2a/c79 66578521 1 2024-02-28T23:07:40.847 INFO:tasks.workunit.client.0.smithi057.stdout:9/515: truncate dd/d1b/f6c 601170 0 2024-02-28T23:07:40.850 INFO:tasks.workunit.client.0.smithi057.stdout:0/524: rename d17/d20/f45 to d17/d20/d26/d48/d55/fbe 0 2024-02-28T23:07:40.850 INFO:tasks.workunit.client.0.smithi057.stdout:9/516: chown dd/de/d24/d72/l8d 884368900 1 2024-02-28T23:07:40.851 INFO:tasks.workunit.client.1.smithi084.stdout:2/572: mknod d5/d7/d9/d7c/d5b/caa 0 2024-02-28T23:07:40.853 INFO:tasks.workunit.client.1.smithi084.stdout:1/678: getdents d12/d1b/d2e/d52/d61/d74 0 2024-02-28T23:07:40.854 INFO:tasks.workunit.client.1.smithi084.stdout:3/653: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:40.856 INFO:tasks.workunit.client.0.smithi057.stdout:3/386: link d3/d4/d7/c75 d3/d8/d1b/c7b 0 2024-02-28T23:07:40.859 INFO:tasks.workunit.client.0.smithi057.stdout:1/392: creat d2/d33/d54/f89 x:0 0 0 2024-02-28T23:07:40.860 INFO:tasks.workunit.client.0.smithi057.stdout:4/504: rmdir d1a/d48/d9c/d2e/dab 0 2024-02-28T23:07:40.866 INFO:tasks.workunit.client.0.smithi057.stdout:9/517: fdatasync dd/f2d 0 2024-02-28T23:07:40.866 INFO:tasks.workunit.client.0.smithi057.stdout:3/387: read d3/d8/d27/f3e [2421110,8900] 0 2024-02-28T23:07:40.867 INFO:tasks.workunit.client.0.smithi057.stdout:1/393: write d2/d7/df/db/d3a/f6c [885296,118250] 0 2024-02-28T23:07:40.868 INFO:tasks.workunit.client.1.smithi084.stdout:3/654: mkdir d3/d27/d7d/db0/d1a/df9 0 2024-02-28T23:07:40.869 INFO:tasks.workunit.client.1.smithi084.stdout:1/679: link lb d12/d1b/d2e/d52/d5b/ldd 0 2024-02-28T23:07:40.871 INFO:tasks.workunit.client.0.smithi057.stdout:4/505: symlink d1a/d2f/laf 0 2024-02-28T23:07:40.872 INFO:tasks.workunit.client.1.smithi084.stdout:9/674: write d4/d6/d15/d3c/f70 [548009,44650] 0 2024-02-28T23:07:40.872 INFO:tasks.workunit.client.0.smithi057.stdout:9/518: chown dd/de/d14/f51 54920027 1 2024-02-28T23:07:40.874 INFO:tasks.workunit.client.1.smithi084.stdout:4/598: dread d1/f7 [0,4194304] 0 2024-02-28T23:07:40.874 INFO:tasks.workunit.client.0.smithi057.stdout:2/585: dwrite d8/d11/d21/d28/d41/f53 [0,4194304] 0 2024-02-28T23:07:40.877 INFO:tasks.workunit.client.0.smithi057.stdout:3/388: fdatasync d3/d4/d7/f10 0 2024-02-28T23:07:40.877 INFO:tasks.workunit.client.0.smithi057.stdout:1/394: symlink d2/d7/df/l8a 0 2024-02-28T23:07:40.882 INFO:tasks.workunit.client.0.smithi057.stdout:6/555: dwrite d0/d17/d20/f5c [4194304,4194304] 0 2024-02-28T23:07:40.883 INFO:tasks.workunit.client.0.smithi057.stdout:7/475: write d0/d5/d36/d4e/d15/d29/d3d/f6e [2314577,10089] 0 2024-02-28T23:07:40.887 INFO:tasks.workunit.client.0.smithi057.stdout:9/519: creat dd/d1b/d40/d44/d92/fa8 x:0 0 0 2024-02-28T23:07:40.889 INFO:tasks.workunit.client.0.smithi057.stdout:2/586: mkdir d8/d11/d21/d28/d41/dbb 0 2024-02-28T23:07:40.891 INFO:tasks.workunit.client.1.smithi084.stdout:9/675: rmdir d4/d6/d15/d75/d2d/daa 39 2024-02-28T23:07:40.892 INFO:tasks.workunit.client.0.smithi057.stdout:5/512: dread d4/d9/f1c [0,4194304] 0 2024-02-28T23:07:40.892 INFO:tasks.workunit.client.0.smithi057.stdout:7/476: unlink d0/d5/d36/d4e/c6d 0 2024-02-28T23:07:40.896 INFO:tasks.workunit.client.0.smithi057.stdout:1/395: rename d2/d7/df/f1d to d2/d33/d54/f8b 0 2024-02-28T23:07:40.896 INFO:tasks.workunit.client.1.smithi084.stdout:1/680: rename d12/d1b/d2e/d52/d61/fb1 to d12/d1b/d91/d9b/fde 0 2024-02-28T23:07:40.897 INFO:tasks.workunit.client.1.smithi084.stdout:4/599: unlink l0 0 2024-02-28T23:07:40.899 INFO:tasks.workunit.client.1.smithi084.stdout:1/681: chown d12/f28 63575 1 2024-02-28T23:07:40.900 INFO:tasks.workunit.client.1.smithi084.stdout:9/676: creat d4/d32/d3f/d89/db8/fda x:0 0 0 2024-02-28T23:07:40.903 INFO:tasks.workunit.client.1.smithi084.stdout:9/677: chown d4/d6/d15/f8e 80326367 1 2024-02-28T23:07:40.903 INFO:tasks.workunit.client.1.smithi084.stdout:2/573: dread d5/d7/f3f [0,4194304] 0 2024-02-28T23:07:40.905 INFO:tasks.workunit.client.0.smithi057.stdout:8/546: sync 2024-02-28T23:07:40.905 INFO:tasks.workunit.client.1.smithi084.stdout:1/682: mknod d12/d1b/d2e/d52/dd7/cdf 0 2024-02-28T23:07:40.905 INFO:tasks.workunit.client.1.smithi084.stdout:2/574: rmdir d5/d7 39 2024-02-28T23:07:40.907 INFO:tasks.workunit.client.1.smithi084.stdout:1/683: fdatasync d12/d1b/d2e/d30/f1c 0 2024-02-28T23:07:40.908 INFO:tasks.workunit.client.1.smithi084.stdout:9/678: mknod d4/d6/d15/d12/d41/cdb 0 2024-02-28T23:07:40.908 INFO:tasks.workunit.client.1.smithi084.stdout:2/575: symlink d5/d64/lab 0 2024-02-28T23:07:40.909 INFO:tasks.workunit.client.0.smithi057.stdout:0/525: dwrite d17/d20/d26/d48/f5c [0,4194304] 0 2024-02-28T23:07:40.910 INFO:tasks.workunit.client.1.smithi084.stdout:1/684: dread - d12/d84/d9a/fc6 zero size 2024-02-28T23:07:40.913 INFO:tasks.workunit.client.1.smithi084.stdout:1/685: dread - d12/d84/fbb zero size 2024-02-28T23:07:40.913 INFO:tasks.workunit.client.1.smithi084.stdout:2/576: chown d5/d7/d9/d7c/d5b/caa 10136 1 2024-02-28T23:07:40.913 INFO:tasks.workunit.client.1.smithi084.stdout:4/600: dread d1/d18/d20/d14/fab [0,4194304] 0 2024-02-28T23:07:40.914 INFO:tasks.workunit.client.0.smithi057.stdout:5/513: mknod d4/d9/d14/d63/ca5 0 2024-02-28T23:07:40.916 INFO:tasks.workunit.client.1.smithi084.stdout:9/679: creat d4/fdc x:0 0 0 2024-02-28T23:07:40.916 INFO:tasks.workunit.client.0.smithi057.stdout:7/477: write d0/d5/d36/d4e/d15/d29/d3d/f6e [1692175,71898] 0 2024-02-28T23:07:40.917 INFO:tasks.workunit.client.0.smithi057.stdout:9/520: rename dd/d1b/d40/c6a to dd/de/d14/d48/d76/ca9 0 2024-02-28T23:07:40.918 INFO:tasks.workunit.client.1.smithi084.stdout:1/686: creat d12/d1b/d91/fe0 x:0 0 0 2024-02-28T23:07:40.918 INFO:tasks.workunit.client.0.smithi057.stdout:4/506: dwrite d1a/d48/d97/f21 [0,4194304] 0 2024-02-28T23:07:40.920 INFO:tasks.workunit.client.1.smithi084.stdout:2/577: creat d5/d7/d75/fac x:0 0 0 2024-02-28T23:07:40.925 INFO:tasks.workunit.client.0.smithi057.stdout:0/526: unlink d17/d20/d26/d30/l38 0 2024-02-28T23:07:40.926 INFO:tasks.workunit.client.1.smithi084.stdout:9/680: mkdir d4/d6/d15/d12/ddd 0 2024-02-28T23:07:40.927 INFO:tasks.workunit.client.0.smithi057.stdout:2/587: dwrite d8/dae/fb8 [0,4194304] 0 2024-02-28T23:07:40.933 INFO:tasks.workunit.client.0.smithi057.stdout:3/389: getdents d3/d20 0 2024-02-28T23:07:40.934 INFO:tasks.workunit.client.1.smithi084.stdout:2/578: getdents d5/d7/d9/d94 0 2024-02-28T23:07:40.937 INFO:tasks.workunit.client.1.smithi084.stdout:5/669: sync 2024-02-28T23:07:40.937 INFO:tasks.workunit.client.1.smithi084.stdout:7/680: sync 2024-02-28T23:07:40.937 INFO:tasks.workunit.client.1.smithi084.stdout:8/593: sync 2024-02-28T23:07:40.937 INFO:tasks.workunit.client.1.smithi084.stdout:6/606: sync 2024-02-28T23:07:40.938 INFO:tasks.workunit.client.0.smithi057.stdout:1/396: rename d2/d7/d4f/d5c/f6a to d2/d7/df/db/d12/d3f/f8c 0 2024-02-28T23:07:40.938 INFO:tasks.workunit.client.0.smithi057.stdout:9/521: mkdir dd/d1b/d40/d44/d5c/daa 0 2024-02-28T23:07:40.940 INFO:tasks.workunit.client.0.smithi057.stdout:8/547: getdents d4/d1e/d56/d6a/d54/d61/d8c/dab 0 2024-02-28T23:07:40.940 INFO:tasks.workunit.client.1.smithi084.stdout:4/601: dwrite d1/f97 [0,4194304] 0 2024-02-28T23:07:40.943 INFO:tasks.workunit.client.0.smithi057.stdout:1/397: fdatasync d2/d7/df/db/d3a/f38 0 2024-02-28T23:07:40.946 INFO:tasks.workunit.client.1.smithi084.stdout:2/579: mknod d5/d7/d12/cad 0 2024-02-28T23:07:40.947 INFO:tasks.workunit.client.0.smithi057.stdout:3/390: fsync d3/d8/f73 0 2024-02-28T23:07:40.947 INFO:tasks.workunit.client.0.smithi057.stdout:4/507: rename d1a/d2f/d9b to d1a/d48/db0 0 2024-02-28T23:07:40.948 INFO:tasks.workunit.client.0.smithi057.stdout:5/514: creat d4/d9/fa6 x:0 0 0 2024-02-28T23:07:40.949 INFO:tasks.workunit.client.1.smithi084.stdout:8/594: creat d0/d9/d30/db6/fc8 x:0 0 0 2024-02-28T23:07:40.949 INFO:tasks.workunit.client.0.smithi057.stdout:9/522: write dd/de/d14/f3d [2969260,38987] 0 2024-02-28T23:07:40.951 INFO:tasks.workunit.client.1.smithi084.stdout:6/607: mknod d1/d13/d50/dbd/cd4 0 2024-02-28T23:07:40.953 INFO:tasks.workunit.client.0.smithi057.stdout:8/548: fdatasync d4/f9 0 2024-02-28T23:07:40.955 INFO:tasks.workunit.client.1.smithi084.stdout:4/602: chown d1/d18/d20/c2b 3250 1 2024-02-28T23:07:40.956 INFO:tasks.workunit.client.0.smithi057.stdout:7/478: dwrite d0/d5/d36/d4e/d3b/d75/f8a [0,4194304] 0 2024-02-28T23:07:40.957 INFO:tasks.workunit.client.1.smithi084.stdout:2/580: mkdir d5/d7/d2e/d4a/dae 0 2024-02-28T23:07:40.961 INFO:tasks.workunit.client.0.smithi057.stdout:1/398: symlink d2/d7/df/db/d12/l8d 0 2024-02-28T23:07:40.961 INFO:tasks.workunit.client.0.smithi057.stdout:5/515: stat d4/d9/d14/f5e 0 2024-02-28T23:07:40.961 INFO:tasks.workunit.client.0.smithi057.stdout:3/391: write d3/d4f/d4c/f54 [4642665,37599] 0 2024-02-28T23:07:40.961 INFO:tasks.workunit.client.1.smithi084.stdout:3/655: dwrite d3/d27/d7d/db0/d14/f58 [0,4194304] 0 2024-02-28T23:07:40.963 INFO:tasks.workunit.client.1.smithi084.stdout:5/670: rename dc/de/df/d31/dd1/d5f/ddc to dc/de/df/d16/d34/d3c/d77/df8 0 2024-02-28T23:07:40.965 INFO:tasks.workunit.client.0.smithi057.stdout:7/479: read d0/d5/f39 [1287177,96267] 0 2024-02-28T23:07:40.967 INFO:tasks.workunit.client.1.smithi084.stdout:7/681: dread db/d10/d18/db7/fbe [0,4194304] 0 2024-02-28T23:07:40.969 INFO:tasks.workunit.client.0.smithi057.stdout:0/527: rename d17/l49 to d17/d20/d23/d4b/lbf 0 2024-02-28T23:07:40.971 INFO:tasks.workunit.client.1.smithi084.stdout:2/581: creat d5/d1f/d83/faf x:0 0 0 2024-02-28T23:07:40.972 INFO:tasks.workunit.client.1.smithi084.stdout:5/671: symlink dc/d51/lf9 0 2024-02-28T23:07:40.973 INFO:tasks.workunit.client.0.smithi057.stdout:8/549: mkdir d4/d1e/d59/dc3 0 2024-02-28T23:07:40.975 INFO:tasks.workunit.client.0.smithi057.stdout:0/528: chown l7 5 1 2024-02-28T23:07:40.975 INFO:tasks.workunit.client.1.smithi084.stdout:3/656: creat d3/d27/d4b/deb/ffa x:0 0 0 2024-02-28T23:07:40.976 INFO:tasks.workunit.client.1.smithi084.stdout:8/595: rename d0/d24/d6e/cb5 to d0/d9/d1b/d38/d43/d51/cc9 0 2024-02-28T23:07:40.977 INFO:tasks.workunit.client.0.smithi057.stdout:1/399: creat d2/d7/df/db/d12/d3f/d47/f8e x:0 0 0 2024-02-28T23:07:40.978 INFO:tasks.workunit.client.0.smithi057.stdout:4/508: dwrite f0 [0,4194304] 0 2024-02-28T23:07:40.980 INFO:tasks.workunit.client.0.smithi057.stdout:0/529: chown l7 78 1 2024-02-28T23:07:40.980 INFO:tasks.workunit.client.0.smithi057.stdout:8/550: write d4/d13/d18/f31 [324877,112959] 0 2024-02-28T23:07:40.982 INFO:tasks.workunit.client.0.smithi057.stdout:9/523: creat dd/de/d4a/d9f/fab x:0 0 0 2024-02-28T23:07:40.982 INFO:tasks.workunit.client.0.smithi057.stdout:5/516: truncate d4/d7/d8/f26 7153880 0 2024-02-28T23:07:40.984 INFO:tasks.workunit.client.1.smithi084.stdout:1/687: dwrite d12/d1b/d63/fbc [0,4194304] 0 2024-02-28T23:07:40.985 INFO:tasks.workunit.client.1.smithi084.stdout:2/582: symlink d5/d7/d9/d7c/lb0 0 2024-02-28T23:07:40.985 INFO:tasks.workunit.client.0.smithi057.stdout:3/392: write d3/d4/f30 [3772218,109377] 0 2024-02-28T23:07:40.986 INFO:tasks.workunit.client.1.smithi084.stdout:0/612: sync 2024-02-28T23:07:40.987 INFO:tasks.workunit.client.1.smithi084.stdout:9/681: truncate d4/d6/d15/d12/d41/d55/fa5 2891164 0 2024-02-28T23:07:40.990 INFO:tasks.workunit.client.0.smithi057.stdout:0/530: symlink d17/d20/d23/d4b/lc0 0 2024-02-28T23:07:40.990 INFO:tasks.workunit.client.1.smithi084.stdout:3/657: unlink d3/d27/d7d/db0/d1a/d29/d7e/fa7 0 2024-02-28T23:07:40.991 INFO:tasks.workunit.client.1.smithi084.stdout:6/608: rmdir d1/d13/d50 39 2024-02-28T23:07:40.991 INFO:tasks.workunit.client.1.smithi084.stdout:8/596: read d0/d11/f23 [102781,37030] 0 2024-02-28T23:07:40.992 INFO:tasks.workunit.client.0.smithi057.stdout:9/524: chown dd/d1b/d40/d44/d5c/f8a 890997 1 2024-02-28T23:07:40.992 INFO:tasks.workunit.client.0.smithi057.stdout:3/393: chown d3/d8/d27/d70 6082597 1 2024-02-28T23:07:40.992 INFO:tasks.workunit.client.0.smithi057.stdout:1/400: mkdir d2/d7/df/db/d3a/d8f 0 2024-02-28T23:07:40.993 INFO:tasks.workunit.client.1.smithi084.stdout:3/658: read d3/fbd [240886,74509] 0 2024-02-28T23:07:40.993 INFO:tasks.workunit.client.0.smithi057.stdout:5/517: chown d4/d9/d16/d2c/f80 8862 1 2024-02-28T23:07:40.993 INFO:tasks.workunit.client.0.smithi057.stdout:2/588: dwrite d8/d11/d39/f4c [4194304,4194304] 0 2024-02-28T23:07:40.993 INFO:tasks.workunit.client.0.smithi057.stdout:8/551: creat d4/d13/d18/d62/d66/fc4 x:0 0 0 2024-02-28T23:07:40.994 INFO:tasks.workunit.client.0.smithi057.stdout:4/509: read d1a/d48/d97/d2d/d49/d4d/f88 [3111703,115594] 0 2024-02-28T23:07:40.996 INFO:tasks.workunit.client.1.smithi084.stdout:5/672: getdents dc/de/df/d16/d34 0 2024-02-28T23:07:41.001 INFO:tasks.workunit.client.0.smithi057.stdout:0/531: symlink d17/d20/d26/lc1 0 2024-02-28T23:07:41.002 INFO:tasks.workunit.client.0.smithi057.stdout:5/518: dread - d4/d9/d14/f5f zero size 2024-02-28T23:07:41.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/609: write d1/d13/d47/f7a [1542268,61374] 0 2024-02-28T23:07:41.007 INFO:tasks.workunit.client.1.smithi084.stdout:4/603: readlink d1/d18/d20/d14/d30/l7c 0 2024-02-28T23:07:41.008 INFO:tasks.workunit.client.1.smithi084.stdout:6/610: stat d1/d88/l8a 0 2024-02-28T23:07:41.008 INFO:tasks.workunit.client.0.smithi057.stdout:4/510: unlink d1a/d48/d9c/d2e/f4a 0 2024-02-28T23:07:41.008 INFO:tasks.workunit.client.0.smithi057.stdout:5/519: read d4/d9/d14/d3b/f84 [354281,94721] 0 2024-02-28T23:07:41.009 INFO:tasks.workunit.client.0.smithi057.stdout:1/401: mkdir d2/d90 0 2024-02-28T23:07:41.009 INFO:tasks.workunit.client.1.smithi084.stdout:0/613: rename d1/d6/db/d35/f5c to d1/d6/db/d35/d57/d81/fc2 0 2024-02-28T23:07:41.010 INFO:tasks.workunit.client.0.smithi057.stdout:8/552: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:41.010 INFO:tasks.workunit.client.0.smithi057.stdout:2/589: readlink d8/d11/d39/d7b/l98 0 2024-02-28T23:07:41.011 INFO:tasks.workunit.client.0.smithi057.stdout:7/480: dread d0/d5/d36/d4e/d15/f1a [0,4194304] 0 2024-02-28T23:07:41.014 INFO:tasks.workunit.client.1.smithi084.stdout:9/682: truncate d4/d6/d15/d75/d2d/f34 3416741 0 2024-02-28T23:07:41.017 INFO:tasks.workunit.client.1.smithi084.stdout:4/604: unlink d1/d18/d20/c90 0 2024-02-28T23:07:41.022 INFO:tasks.workunit.client.1.smithi084.stdout:5/673: creat dc/de/df/d16/ffa x:0 0 0 2024-02-28T23:07:41.023 INFO:tasks.workunit.client.1.smithi084.stdout:2/583: dwrite d5/d7/d1d/f50 [0,4194304] 0 2024-02-28T23:07:41.023 INFO:tasks.workunit.client.1.smithi084.stdout:1/688: dwrite d12/d1b/d63/fd8 [0,4194304] 0 2024-02-28T23:07:41.024 INFO:tasks.workunit.client.0.smithi057.stdout:4/511: rmdir d1a/d48/d9c 39 2024-02-28T23:07:41.027 INFO:tasks.workunit.client.0.smithi057.stdout:1/402: symlink d2/d7/df/db/d12/l91 0 2024-02-28T23:07:41.029 INFO:tasks.workunit.client.1.smithi084.stdout:0/614: read d1/d6/db/d37/f96 [2180280,90511] 0 2024-02-28T23:07:41.031 INFO:tasks.workunit.client.0.smithi057.stdout:1/403: write d2/d7/f41 [8085003,66296] 0 2024-02-28T23:07:41.034 INFO:tasks.workunit.client.0.smithi057.stdout:4/512: dread d1a/f36 [0,4194304] 0 2024-02-28T23:07:41.037 INFO:tasks.workunit.client.1.smithi084.stdout:4/605: symlink d1/d18/d20/d24/d2f/d3e/d52/d45/db5/lcb 0 2024-02-28T23:07:41.039 INFO:tasks.workunit.client.1.smithi084.stdout:7/682: dread db/d5e/dc6/f32 [0,4194304] 0 2024-02-28T23:07:41.040 INFO:tasks.workunit.client.1.smithi084.stdout:2/584: truncate d5/d7/d12/d24/d2a/d35/f5d 4434006 0 2024-02-28T23:07:41.040 INFO:tasks.workunit.client.1.smithi084.stdout:5/674: dread dc/d51/d7c/f93 [0,4194304] 0 2024-02-28T23:07:41.043 INFO:tasks.workunit.client.1.smithi084.stdout:7/683: dread - db/d10/d18/f82 zero size 2024-02-28T23:07:41.045 INFO:tasks.workunit.client.0.smithi057.stdout:0/532: rename d17/d19/fa3 to d17/d20/d26/d48/d7c/d98/fc2 0 2024-02-28T23:07:41.049 INFO:tasks.workunit.client.0.smithi057.stdout:1/404: creat d2/d7/df/f92 x:0 0 0 2024-02-28T23:07:41.049 INFO:tasks.workunit.client.1.smithi084.stdout:6/611: truncate d1/d9/d1d/d1e/d33/f8f 4266784 0 2024-02-28T23:07:41.054 INFO:tasks.workunit.client.0.smithi057.stdout:4/513: mkdir d1a/d48/db0/db1 0 2024-02-28T23:07:41.055 INFO:tasks.workunit.client.1.smithi084.stdout:4/606: unlink d1/d18/d20/d24/d2f/f8f 0 2024-02-28T23:07:41.056 INFO:tasks.workunit.client.1.smithi084.stdout:0/615: mknod d1/d6/d1b/cc3 0 2024-02-28T23:07:41.058 INFO:tasks.workunit.client.0.smithi057.stdout:8/553: rename d4/d13/d1a/l3e to d4/d13/d2c/db1/lc5 0 2024-02-28T23:07:41.058 INFO:tasks.workunit.client.0.smithi057.stdout:2/590: dread d8/d11/f1d [0,4194304] 0 2024-02-28T23:07:41.058 INFO:tasks.workunit.client.0.smithi057.stdout:1/405: write d2/d7/df/db/d3a/f6c [1472519,77698] 0 2024-02-28T23:07:41.060 INFO:tasks.workunit.client.0.smithi057.stdout:3/394: truncate d3/d4/f17 4182581 0 2024-02-28T23:07:41.063 INFO:tasks.workunit.client.1.smithi084.stdout:9/683: getdents d4/d6/d15/d12 0 2024-02-28T23:07:41.064 INFO:tasks.workunit.client.0.smithi057.stdout:7/481: getdents d0/d5/d36/d4e/d15 0 2024-02-28T23:07:41.064 INFO:tasks.workunit.client.1.smithi084.stdout:7/684: symlink db/d5e/d87/dc1/lcb 0 2024-02-28T23:07:41.064 INFO:tasks.workunit.client.0.smithi057.stdout:8/554: dread - d4/d1e/d56/d6a/f95 zero size 2024-02-28T23:07:41.067 INFO:tasks.workunit.client.0.smithi057.stdout:5/520: getdents d4/d9/d16/d52 0 2024-02-28T23:07:41.069 INFO:tasks.workunit.client.0.smithi057.stdout:2/591: rmdir d8/d11/d48 39 2024-02-28T23:07:41.069 INFO:tasks.workunit.client.1.smithi084.stdout:8/597: dwrite d0/f53 [0,4194304] 0 2024-02-28T23:07:41.069 INFO:tasks.workunit.client.0.smithi057.stdout:9/525: dread dd/d1b/d40/f86 [0,4194304] 0 2024-02-28T23:07:41.071 INFO:tasks.workunit.client.1.smithi084.stdout:4/607: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fcc x:0 0 0 2024-02-28T23:07:41.072 INFO:tasks.workunit.client.0.smithi057.stdout:9/526: write dd/de/d14/f1e [3980130,7290] 0 2024-02-28T23:07:41.073 INFO:tasks.workunit.client.1.smithi084.stdout:6/612: creat d1/d9/fd5 x:0 0 0 2024-02-28T23:07:41.075 INFO:tasks.workunit.client.0.smithi057.stdout:4/514: rename d1a/d48/d97/d2d to d1a/d2f/db2 0 2024-02-28T23:07:41.078 INFO:tasks.workunit.client.0.smithi057.stdout:4/515: readlink d1a/d48/db0/l6d 0 2024-02-28T23:07:41.079 INFO:tasks.workunit.client.1.smithi084.stdout:9/684: chown d4/d6/d15/d3c/l7f 5658 1 2024-02-28T23:07:41.081 INFO:tasks.workunit.client.1.smithi084.stdout:2/585: dwrite d5/d1f/d42/d74/f76 [0,4194304] 0 2024-02-28T23:07:41.082 INFO:tasks.workunit.client.1.smithi084.stdout:9/685: chown d4/d32/d3f/f96 1609946 1 2024-02-28T23:07:41.089 INFO:tasks.workunit.client.1.smithi084.stdout:3/659: dread d3/d27/d7d/db0/d1a/d21/d76/fb2 [0,4194304] 0 2024-02-28T23:07:41.096 INFO:tasks.workunit.client.1.smithi084.stdout:6/613: symlink d1/d13/d50/d74/d9e/dc2/ld6 0 2024-02-28T23:07:41.096 INFO:tasks.workunit.client.1.smithi084.stdout:3/660: fsync d3/d27/d7d/db0/d1a/d21/d3d/f3c 0 2024-02-28T23:07:41.097 INFO:tasks.workunit.client.0.smithi057.stdout:7/482: dwrite d0/f85 [0,4194304] 0 2024-02-28T23:07:41.100 INFO:tasks.workunit.client.1.smithi084.stdout:6/614: readlink d1/d9/d1d/d1e/d33/la2 0 2024-02-28T23:07:41.101 INFO:tasks.workunit.client.0.smithi057.stdout:9/527: mknod dd/de/d24/cac 0 2024-02-28T23:07:41.101 INFO:tasks.workunit.client.0.smithi057.stdout:4/516: creat d1a/d48/db0/d4f/fb3 x:0 0 0 2024-02-28T23:07:41.101 INFO:tasks.workunit.client.0.smithi057.stdout:2/592: mkdir d8/d11/d39/dbc 0 2024-02-28T23:07:41.104 INFO:tasks.workunit.client.0.smithi057.stdout:4/517: chown d1a/d2f/db2/d49/c57 938483 1 2024-02-28T23:07:41.105 INFO:tasks.workunit.client.1.smithi084.stdout:3/661: stat d3/d27/d7d/db0/d14/l63 0 2024-02-28T23:07:41.109 INFO:tasks.workunit.client.1.smithi084.stdout:8/598: rename d0/d9/d1b/d20/cba to d0/d44/d9f/cca 0 2024-02-28T23:07:41.109 INFO:tasks.workunit.client.1.smithi084.stdout:3/662: read - d3/d27/d4b/deb/ffa zero size 2024-02-28T23:07:41.112 INFO:tasks.workunit.client.0.smithi057.stdout:8/555: rename d4/d13/d2c/db1/lc5 to d4/d1e/lc6 0 2024-02-28T23:07:41.113 INFO:tasks.workunit.client.1.smithi084.stdout:4/608: creat d1/d18/d20/d65/d8e/d55/fcd x:0 0 0 2024-02-28T23:07:41.113 INFO:tasks.workunit.client.0.smithi057.stdout:2/593: creat d8/d11/d21/d70/fbd x:0 0 0 2024-02-28T23:07:41.114 INFO:tasks.workunit.client.0.smithi057.stdout:7/483: mkdir d0/d98 0 2024-02-28T23:07:41.114 INFO:tasks.workunit.client.0.smithi057.stdout:4/518: write d1a/d48/d97/f5c [836315,87105] 0 2024-02-28T23:07:41.117 INFO:tasks.workunit.client.1.smithi084.stdout:1/689: dread d12/d1b/d63/f76 [0,4194304] 0 2024-02-28T23:07:41.119 INFO:tasks.workunit.client.1.smithi084.stdout:0/616: truncate d1/d8/f40 721060 0 2024-02-28T23:07:41.120 INFO:tasks.workunit.client.0.smithi057.stdout:0/533: dwrite d17/f58 [0,4194304] 0 2024-02-28T23:07:41.120 INFO:tasks.workunit.client.0.smithi057.stdout:2/594: chown d8/d11/d39/d7b/d8b/f96 15049462 1 2024-02-28T23:07:41.120 INFO:tasks.workunit.client.0.smithi057.stdout:8/556: write d4/d13/d18/f31 [2341461,109816] 0 2024-02-28T23:07:41.126 INFO:tasks.workunit.client.0.smithi057.stdout:3/395: dwrite d3/d4/d1d/f1e [0,4194304] 0 2024-02-28T23:07:41.126 INFO:tasks.workunit.client.1.smithi084.stdout:9/686: dwrite d4/d6/d15/d12/d41/d55/fbc [0,4194304] 0 2024-02-28T23:07:41.127 INFO:tasks.workunit.client.1.smithi084.stdout:7/685: truncate db/d10/d17/d9c/d9f/db1/d36/f5a 1830242 0 2024-02-28T23:07:41.132 INFO:tasks.workunit.client.1.smithi084.stdout:0/617: symlink d1/d6/d2d/lc4 0 2024-02-28T23:07:41.134 INFO:tasks.workunit.client.0.smithi057.stdout:1/406: getdents d2/d7/df/db/d12 0 2024-02-28T23:07:41.136 INFO:tasks.workunit.client.0.smithi057.stdout:4/519: fsync d1a/d1e/f5b 0 2024-02-28T23:07:41.138 INFO:tasks.workunit.client.0.smithi057.stdout:4/520: stat d1a/d2f/db2/f35 0 2024-02-28T23:07:41.138 INFO:tasks.workunit.client.1.smithi084.stdout:5/675: dwrite dc/de/df/d16/d34/f6c [0,4194304] 0 2024-02-28T23:07:41.139 INFO:tasks.workunit.client.1.smithi084.stdout:1/690: read d12/d1b/d2e/d30/d50/fac [4085242,48905] 0 2024-02-28T23:07:41.141 INFO:tasks.workunit.client.0.smithi057.stdout:2/595: write d8/d11/d15/daf/fb7 [8432709,26937] 0 2024-02-28T23:07:41.143 INFO:tasks.workunit.client.0.smithi057.stdout:7/484: dread - d0/d5/d25/d3e/f5a zero size 2024-02-28T23:07:41.143 INFO:tasks.workunit.client.0.smithi057.stdout:8/557: creat d4/d13/d18/d62/d66/d74/fc7 x:0 0 0 2024-02-28T23:07:41.144 INFO:tasks.workunit.client.0.smithi057.stdout:2/596: dread - d8/d11/d21/f7d zero size 2024-02-28T23:07:41.146 INFO:tasks.workunit.client.0.smithi057.stdout:7/485: readlink d0/d5/d36/d4e/d15/d29/l5f 0 2024-02-28T23:07:41.149 INFO:tasks.workunit.client.1.smithi084.stdout:1/691: write d12/d1b/d2e/d52/d5b/f6f [5149913,43625] 0 2024-02-28T23:07:41.149 INFO:tasks.workunit.client.0.smithi057.stdout:4/521: mkdir d1a/d48/db0/d4f/d6e/db4 0 2024-02-28T23:07:41.150 INFO:tasks.workunit.client.1.smithi084.stdout:8/599: dread d0/d9/d1b/f46 [0,4194304] 0 2024-02-28T23:07:41.151 INFO:tasks.workunit.client.0.smithi057.stdout:0/534: link d17/d20/d26/d30/d6b/d71/f92 d17/d20/d26/d30/d6b/fc3 0 2024-02-28T23:07:41.152 INFO:tasks.workunit.client.1.smithi084.stdout:3/663: creat d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb x:0 0 0 2024-02-28T23:07:41.152 INFO:tasks.workunit.client.1.smithi084.stdout:0/618: rmdir d1/d8 39 2024-02-28T23:07:41.155 INFO:tasks.workunit.client.1.smithi084.stdout:9/687: mkdir d4/d6/d15/d75/d2d/dbd/dde 0 2024-02-28T23:07:41.160 INFO:tasks.workunit.client.1.smithi084.stdout:9/688: truncate d4/d6/d15/d12/f9e 4932540 0 2024-02-28T23:07:41.161 INFO:tasks.workunit.client.0.smithi057.stdout:7/486: mknod d0/d5/d4a/c99 0 2024-02-28T23:07:41.161 INFO:tasks.workunit.client.0.smithi057.stdout:4/522: mkdir d1a/d48/db0/d4f/db5 0 2024-02-28T23:07:41.162 INFO:tasks.workunit.client.0.smithi057.stdout:8/558: symlink d4/d1e/d56/d6a/d54/d6b/lc8 0 2024-02-28T23:07:41.164 INFO:tasks.workunit.client.0.smithi057.stdout:2/597: mkdir d8/d11/dbe 0 2024-02-28T23:07:41.168 INFO:tasks.workunit.client.1.smithi084.stdout:0/619: fsync d1/d6/d59/d70/f8d 0 2024-02-28T23:07:41.169 INFO:tasks.workunit.client.1.smithi084.stdout:3/664: mkdir d3/d27/d7d/db0/d1a/d45/dde/dfc 0 2024-02-28T23:07:41.170 INFO:tasks.workunit.client.0.smithi057.stdout:0/535: dread d17/d20/d26/d48/d7c/d98/fc2 [0,4194304] 0 2024-02-28T23:07:41.170 INFO:tasks.workunit.client.0.smithi057.stdout:8/559: mkdir d4/d1e/d56/dc9 0 2024-02-28T23:07:41.171 INFO:tasks.workunit.client.1.smithi084.stdout:8/600: creat d0/d24/d4d/fcb x:0 0 0 2024-02-28T23:07:41.172 INFO:tasks.workunit.client.0.smithi057.stdout:2/598: creat d8/d11/d15/db9/fbf x:0 0 0 2024-02-28T23:07:41.173 INFO:tasks.workunit.client.1.smithi084.stdout:8/601: readlink d0/d24/l6c 0 2024-02-28T23:07:41.174 INFO:tasks.workunit.client.0.smithi057.stdout:4/523: read f16 [530907,124935] 0 2024-02-28T23:07:41.174 INFO:tasks.workunit.client.1.smithi084.stdout:9/689: symlink d4/d6/d15/d75/d2d/dbd/dde/ldf 0 2024-02-28T23:07:41.177 INFO:tasks.workunit.client.1.smithi084.stdout:4/609: dwrite d1/f4 [0,4194304] 0 2024-02-28T23:07:41.177 INFO:tasks.workunit.client.1.smithi084.stdout:3/665: truncate d3/f28 2200216 0 2024-02-28T23:07:41.178 INFO:tasks.workunit.client.0.smithi057.stdout:4/524: write d1a/d2f/db2/f63 [885197,23673] 0 2024-02-28T23:07:41.178 INFO:tasks.workunit.client.1.smithi084.stdout:8/602: chown d0/d24/d83/c8e 3147993 1 2024-02-28T23:07:41.179 INFO:tasks.workunit.client.0.smithi057.stdout:8/560: mkdir d4/d1e/d56/d6a/d54/d6b/dca 0 2024-02-28T23:07:41.180 INFO:tasks.workunit.client.0.smithi057.stdout:2/599: rename d8/d11/d39/f4c to d8/d11/d15/daf/fc0 0 2024-02-28T23:07:41.180 INFO:tasks.workunit.client.0.smithi057.stdout:0/536: mknod d17/d20/d26/d48/cc4 0 2024-02-28T23:07:41.185 INFO:tasks.workunit.client.0.smithi057.stdout:3/396: write d3/d4/d7/f2b [2697463,118373] 0 2024-02-28T23:07:41.187 INFO:tasks.workunit.client.0.smithi057.stdout:8/561: chown d4/f3a 1 1 2024-02-28T23:07:41.187 INFO:tasks.workunit.client.1.smithi084.stdout:3/666: rename d3/d27/f2d to d3/d27/d7d/db0/df/ded/ffd 0 2024-02-28T23:07:41.187 INFO:tasks.workunit.client.0.smithi057.stdout:0/537: rename d17 to d17/d20/d26/d48/d7c/d98/da6/dc5 22 2024-02-28T23:07:41.189 INFO:tasks.workunit.client.0.smithi057.stdout:1/407: write d2/f44 [118520,127403] 0 2024-02-28T23:07:41.190 INFO:tasks.workunit.client.1.smithi084.stdout:9/690: unlink d4/d6/d15/d75/l77 0 2024-02-28T23:07:41.190 INFO:tasks.workunit.client.0.smithi057.stdout:0/538: stat d17/d19/d4d 0 2024-02-28T23:07:41.193 INFO:tasks.workunit.client.1.smithi084.stdout:5/676: dwrite dc/de/df/d16/f9a [0,4194304] 0 2024-02-28T23:07:41.193 INFO:tasks.workunit.client.1.smithi084.stdout:4/610: dread d1/d18/d20/d24/d2f/d3e/d52/d45/f93 [0,4194304] 0 2024-02-28T23:07:41.195 INFO:tasks.workunit.client.0.smithi057.stdout:9/528: dwrite dd/d1b/f6c [0,4194304] 0 2024-02-28T23:07:41.196 INFO:tasks.workunit.client.0.smithi057.stdout:2/600: write d8/d11/d21/d28/d49/d85/d9b/fac [1468968,2226] 0 2024-02-28T23:07:41.197 INFO:tasks.workunit.client.1.smithi084.stdout:3/667: mkdir d3/d27/d7d/db0/d1a/df9/dfe 0 2024-02-28T23:07:41.201 INFO:tasks.workunit.client.1.smithi084.stdout:9/691: read d4/d6/d15/d12/d41/d55/fa5 [1773424,52890] 0 2024-02-28T23:07:41.205 INFO:tasks.workunit.client.1.smithi084.stdout:4/611: fdatasync d1/d18/d20/d65/f72 0 2024-02-28T23:07:41.207 INFO:tasks.workunit.client.0.smithi057.stdout:0/539: symlink d17/d20/d23/d47/lc6 0 2024-02-28T23:07:41.208 INFO:tasks.workunit.client.0.smithi057.stdout:3/397: read d3/d8/d1b/f37 [2840724,53509] 0 2024-02-28T23:07:41.212 INFO:tasks.workunit.client.0.smithi057.stdout:2/601: creat d8/d11/d21/d70/fc1 x:0 0 0 2024-02-28T23:07:41.213 INFO:tasks.workunit.client.1.smithi084.stdout:9/692: mkdir d4/d6/d15/de0 0 2024-02-28T23:07:41.213 INFO:tasks.workunit.client.1.smithi084.stdout:4/612: unlink d1/d18/d20/d24/d2f/cc8 0 2024-02-28T23:07:41.214 INFO:tasks.workunit.client.1.smithi084.stdout:1/692: write f11 [341557,124418] 0 2024-02-28T23:07:41.216 INFO:tasks.workunit.client.0.smithi057.stdout:2/602: stat d8/d11/d39/c65 0 2024-02-28T23:07:41.217 INFO:tasks.workunit.client.0.smithi057.stdout:7/487: dwrite d0/d5/d36/d4e/d15/f6a [0,4194304] 0 2024-02-28T23:07:41.218 INFO:tasks.workunit.client.1.smithi084.stdout:9/693: fdatasync d4/d69/fb7 0 2024-02-28T23:07:41.218 INFO:tasks.workunit.client.0.smithi057.stdout:9/529: rename dd/de/d24/d72/f7f to dd/de/d14/d48/d4b/d41/fad 0 2024-02-28T23:07:41.219 INFO:tasks.workunit.client.0.smithi057.stdout:2/603: readlink d8/d11/d21/d28/d49/d85/l8e 0 2024-02-28T23:07:41.221 INFO:tasks.workunit.client.1.smithi084.stdout:7/686: dwrite db/d10/faa [0,4194304] 0 2024-02-28T23:07:41.224 INFO:tasks.workunit.client.0.smithi057.stdout:3/398: symlink d3/d8/d27/l7c 0 2024-02-28T23:07:41.224 INFO:tasks.workunit.client.0.smithi057.stdout:7/488: chown d0/d5/d36/d4e/d3b/f5b 1 1 2024-02-28T23:07:41.225 INFO:tasks.workunit.client.1.smithi084.stdout:8/603: write d0/d9/d1b/f74 [1935502,42529] 0 2024-02-28T23:07:41.233 INFO:tasks.workunit.client.0.smithi057.stdout:9/530: mknod dd/d46/d6f/cae 0 2024-02-28T23:07:41.233 INFO:tasks.workunit.client.1.smithi084.stdout:5/677: dwrite dc/de/df/d16/fa0 [4194304,4194304] 0 2024-02-28T23:07:41.233 INFO:tasks.workunit.client.1.smithi084.stdout:4/613: unlink d1/d18/d20/d14/d30/f53 0 2024-02-28T23:07:41.236 INFO:tasks.workunit.client.1.smithi084.stdout:7/687: symlink db/d10/d17/d64/d77/lcc 0 2024-02-28T23:07:41.236 INFO:tasks.workunit.client.1.smithi084.stdout:1/693: chown d12/d1b/d2e/d52/d5b/ldd 246 1 2024-02-28T23:07:41.239 INFO:tasks.workunit.client.1.smithi084.stdout:1/694: readlink d12/d1b/l20 0 2024-02-28T23:07:41.240 INFO:tasks.workunit.client.0.smithi057.stdout:3/399: link d3/f2c d3/d4f/f7d 0 2024-02-28T23:07:41.243 INFO:tasks.workunit.client.0.smithi057.stdout:1/408: truncate d2/f7e 2609576 0 2024-02-28T23:07:41.244 INFO:tasks.workunit.client.1.smithi084.stdout:9/694: creat d4/d6/d15/d75/d2d/daa/fe1 x:0 0 0 2024-02-28T23:07:41.247 INFO:tasks.workunit.client.1.smithi084.stdout:0/620: dwrite d1/d6/db/d35/d4a/f88 [0,4194304] 0 2024-02-28T23:07:41.247 INFO:tasks.workunit.client.0.smithi057.stdout:8/562: dread d4/f5 [0,4194304] 0 2024-02-28T23:07:41.250 INFO:tasks.workunit.client.0.smithi057.stdout:4/525: dread d1a/d1e/f42 [0,4194304] 0 2024-02-28T23:07:41.250 INFO:tasks.workunit.client.1.smithi084.stdout:1/695: unlink d12/d1b/l27 0 2024-02-28T23:07:41.251 INFO:tasks.workunit.client.1.smithi084.stdout:4/614: symlink d1/d18/d20/d65/lce 0 2024-02-28T23:07:41.253 INFO:tasks.workunit.client.0.smithi057.stdout:3/400: link d3/d20/f23 d3/d8/d1b/f7e 0 2024-02-28T23:07:41.256 INFO:tasks.workunit.client.1.smithi084.stdout:1/696: write d12/d1b/d2e/d52/d5b/f6f [3161410,21730] 0 2024-02-28T23:07:41.257 INFO:tasks.workunit.client.0.smithi057.stdout:1/409: mknod d2/d7/df/db/d12/d19/d48/c93 0 2024-02-28T23:07:41.257 INFO:tasks.workunit.client.0.smithi057.stdout:8/563: chown d4/d1e/d56/l7d 75504 1 2024-02-28T23:07:41.262 INFO:tasks.workunit.client.1.smithi084.stdout:0/621: write d1/d8/d18/d4c/d6c/d74/f91 [2884683,57057] 0 2024-02-28T23:07:41.262 INFO:tasks.workunit.client.0.smithi057.stdout:2/604: dwrite d8/d11/d21/d28/d41/f5c [0,4194304] 0 2024-02-28T23:07:41.264 INFO:tasks.workunit.client.0.smithi057.stdout:3/401: creat d3/d4/d7/f7f x:0 0 0 2024-02-28T23:07:41.266 INFO:tasks.workunit.client.0.smithi057.stdout:9/531: dwrite dd/de/d24/fa6 [0,4194304] 0 2024-02-28T23:07:41.266 INFO:tasks.workunit.client.0.smithi057.stdout:1/410: write d2/d7/df/db/d3a/d2a/f6d [585364,125539] 0 2024-02-28T23:07:41.267 INFO:tasks.workunit.client.1.smithi084.stdout:3/668: dread d3/fc [0,4194304] 0 2024-02-28T23:07:41.268 INFO:tasks.workunit.client.0.smithi057.stdout:8/564: unlink d4/d13/d7f/c96 0 2024-02-28T23:07:41.270 INFO:tasks.workunit.client.1.smithi084.stdout:8/604: getdents d0/d24 0 2024-02-28T23:07:41.273 INFO:tasks.workunit.client.1.smithi084.stdout:9/695: getdents d4/d6/d15/d75/d9d/dc3 0 2024-02-28T23:07:41.274 INFO:tasks.workunit.client.0.smithi057.stdout:2/605: write d8/d11/d15/f5d [6909053,33404] 0 2024-02-28T23:07:41.275 INFO:tasks.workunit.client.1.smithi084.stdout:3/669: stat d3/d27/d7d/db0/d1a/d45 0 2024-02-28T23:07:41.275 INFO:tasks.workunit.client.1.smithi084.stdout:7/688: write db/d10/f94 [1829441,114473] 0 2024-02-28T23:07:41.276 INFO:tasks.workunit.client.1.smithi084.stdout:1/697: creat d12/d1b/d91/d9b/da5/dcb/dd4/fe1 x:0 0 0 2024-02-28T23:07:41.281 INFO:tasks.workunit.client.1.smithi084.stdout:5/678: dwrite dc/f27 [0,4194304] 0 2024-02-28T23:07:41.289 INFO:tasks.workunit.client.0.smithi057.stdout:9/532: read dd/de/d14/d48/d4b/f54 [3807816,13096] 0 2024-02-28T23:07:41.289 INFO:tasks.workunit.client.0.smithi057.stdout:8/565: symlink d4/d13/d18/d62/d66/d74/d8b/lcb 0 2024-02-28T23:07:41.289 INFO:tasks.workunit.client.1.smithi084.stdout:7/689: write db/d10/d17/d64/d77/fbf [688902,130298] 0 2024-02-28T23:07:41.289 INFO:tasks.workunit.client.1.smithi084.stdout:8/605: rmdir d0/d24/d78 39 2024-02-28T23:07:41.292 INFO:tasks.workunit.client.0.smithi057.stdout:4/526: dwrite d1a/d2f/db2/d49/d4d/f8d [0,4194304] 0 2024-02-28T23:07:41.294 INFO:tasks.workunit.client.1.smithi084.stdout:2/586: sync 2024-02-28T23:07:41.295 INFO:tasks.workunit.client.1.smithi084.stdout:9/696: rmdir d4/d32/d3f/d89 39 2024-02-28T23:07:41.296 INFO:tasks.workunit.client.0.smithi057.stdout:2/606: getdents d8/d11/d21 0 2024-02-28T23:07:41.298 INFO:tasks.workunit.client.1.smithi084.stdout:2/587: chown d5/d7/d12/l55 3800742 1 2024-02-28T23:07:41.298 INFO:tasks.workunit.client.1.smithi084.stdout:9/697: read - d4/f9c zero size 2024-02-28T23:07:41.299 INFO:tasks.workunit.client.1.smithi084.stdout:4/615: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/f93 [0,4194304] 0 2024-02-28T23:07:41.303 INFO:tasks.workunit.client.1.smithi084.stdout:3/670: creat d3/d27/d7d/db0/d1a/d29/d41/fff x:0 0 0 2024-02-28T23:07:41.304 INFO:tasks.workunit.client.0.smithi057.stdout:8/566: fdatasync d4/d13/d1a/f89 0 2024-02-28T23:07:41.305 INFO:tasks.workunit.client.1.smithi084.stdout:1/698: creat d12/d1b/d2e/d52/dd7/fe2 x:0 0 0 2024-02-28T23:07:41.307 INFO:tasks.workunit.client.0.smithi057.stdout:3/402: dwrite d3/d4/d1d/f39 [0,4194304] 0 2024-02-28T23:07:41.308 INFO:tasks.workunit.client.0.smithi057.stdout:9/533: mknod dd/de/d24/d72/caf 0 2024-02-28T23:07:41.311 INFO:tasks.workunit.client.0.smithi057.stdout:9/534: readlink dd/de/d24/l36 0 2024-02-28T23:07:41.315 INFO:tasks.workunit.client.1.smithi084.stdout:2/588: rmdir d5/d7/d9/d94 39 2024-02-28T23:07:41.315 INFO:tasks.workunit.client.0.smithi057.stdout:0/540: dread d17/d20/d26/f46 [0,4194304] 0 2024-02-28T23:07:41.317 INFO:tasks.workunit.client.0.smithi057.stdout:8/567: chown d4/d1e/d56/d6a/d54 9139 1 2024-02-28T23:07:41.318 INFO:tasks.workunit.client.1.smithi084.stdout:5/679: creat dc/de/d83/dd9/ffb x:0 0 0 2024-02-28T23:07:41.321 INFO:tasks.workunit.client.1.smithi084.stdout:3/671: rename d3/d27/d7d/db0/d14/d31/d34/fc6 to d3/d27/d7d/db0/d1a/d45/d5e/d7a/f100 0 2024-02-28T23:07:41.326 INFO:tasks.workunit.client.0.smithi057.stdout:1/411: dwrite d2/d33/f76 [0,4194304] 0 2024-02-28T23:07:41.328 INFO:tasks.workunit.client.0.smithi057.stdout:3/403: symlink d3/d4/d1d/d47/d69/l80 0 2024-02-28T23:07:41.329 INFO:tasks.workunit.client.1.smithi084.stdout:6/615: sync 2024-02-28T23:07:41.330 INFO:tasks.workunit.client.0.smithi057.stdout:4/527: creat d1a/d48/db0/db1/fb6 x:0 0 0 2024-02-28T23:07:41.330 INFO:tasks.workunit.client.1.smithi084.stdout:0/622: getdents d1/d6/db/d37/d72 0 2024-02-28T23:07:41.331 INFO:tasks.workunit.client.1.smithi084.stdout:7/690: symlink db/d10/d18/db7/dbb/lcd 0 2024-02-28T23:07:41.332 INFO:tasks.workunit.client.0.smithi057.stdout:0/541: rmdir d17/d20/d26/d48/d55 39 2024-02-28T23:07:41.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/489: dread d0/d5/d36/d4e/d15/d29/f2a [0,4194304] 0 2024-02-28T23:07:41.332 INFO:tasks.workunit.client.1.smithi084.stdout:9/698: symlink d4/d6/d15/de0/le2 0 2024-02-28T23:07:41.335 INFO:tasks.workunit.client.1.smithi084.stdout:4/616: creat d1/d18/fcf x:0 0 0 2024-02-28T23:07:41.335 INFO:tasks.workunit.client.1.smithi084.stdout:9/699: chown d4/d69 0 1 2024-02-28T23:07:41.336 INFO:tasks.workunit.client.0.smithi057.stdout:9/535: symlink dd/de/d89/lb0 0 2024-02-28T23:07:41.336 INFO:tasks.workunit.client.0.smithi057.stdout:1/412: fsync d2/d7/df/db/f69 0 2024-02-28T23:07:41.336 INFO:tasks.workunit.client.0.smithi057.stdout:3/404: creat d3/d8/d27/f81 x:0 0 0 2024-02-28T23:07:41.338 INFO:tasks.workunit.client.0.smithi057.stdout:3/405: fdatasync d3/d8/f29 0 2024-02-28T23:07:41.338 INFO:tasks.workunit.client.1.smithi084.stdout:1/699: creat d12/d1b/d2e/d52/fe3 x:0 0 0 2024-02-28T23:07:41.340 INFO:tasks.workunit.client.0.smithi057.stdout:7/490: mknod d0/d5/d36/d4e/d15/c9a 0 2024-02-28T23:07:41.341 INFO:tasks.workunit.client.0.smithi057.stdout:3/406: fdatasync d3/d8/f29 0 2024-02-28T23:07:41.342 INFO:tasks.workunit.client.0.smithi057.stdout:9/536: mknod dd/de/d24/d63/cb1 0 2024-02-28T23:07:41.344 INFO:tasks.workunit.client.0.smithi057.stdout:9/537: write dd/de/d14/f1e [1350692,53191] 0 2024-02-28T23:07:41.348 INFO:tasks.workunit.client.0.smithi057.stdout:3/407: rename d3/d8/f45 to d3/d20/d2d/f82 0 2024-02-28T23:07:41.348 INFO:tasks.workunit.client.1.smithi084.stdout:6/616: mknod d1/d9/d1d/db6/cd7 0 2024-02-28T23:07:41.349 INFO:tasks.workunit.client.1.smithi084.stdout:2/589: mknod d5/d7/d9/d7c/cb1 0 2024-02-28T23:07:41.350 INFO:tasks.workunit.client.1.smithi084.stdout:7/691: rmdir db/d10/d17/d9c 39 2024-02-28T23:07:41.353 INFO:tasks.workunit.client.0.smithi057.stdout:4/528: mkdir d1a/d48/d9c/d2e/d8f/db7 0 2024-02-28T23:07:41.353 INFO:tasks.workunit.client.0.smithi057.stdout:1/413: creat d2/d7/df/db/d3a/d8f/f94 x:0 0 0 2024-02-28T23:07:41.353 INFO:tasks.workunit.client.0.smithi057.stdout:3/408: write d3/d4f/d4c/f54 [3284729,109640] 0 2024-02-28T23:07:41.354 INFO:tasks.workunit.client.0.smithi057.stdout:0/542: getdents d17/d19/d4d 0 2024-02-28T23:07:41.354 INFO:tasks.workunit.client.0.smithi057.stdout:9/538: creat dd/d1b/d40/fb2 x:0 0 0 2024-02-28T23:07:41.355 INFO:tasks.workunit.client.1.smithi084.stdout:1/700: write d12/d1b/d2e/d32/f47 [941775,11231] 0 2024-02-28T23:07:41.358 INFO:tasks.workunit.client.0.smithi057.stdout:1/414: chown d2/d71/d2d/f49 2707 1 2024-02-28T23:07:41.361 INFO:tasks.workunit.client.1.smithi084.stdout:0/623: creat d1/d6/fc5 x:0 0 0 2024-02-28T23:07:41.363 INFO:tasks.workunit.client.1.smithi084.stdout:4/617: dwrite d1/d18/d20/d65/d8e/f5a [4194304,4194304] 0 2024-02-28T23:07:41.363 INFO:tasks.workunit.client.1.smithi084.stdout:8/606: link d0/d24/d78/fbd d0/d9/d30/d33/d71/d84/fcc 0 2024-02-28T23:07:41.365 INFO:tasks.workunit.client.0.smithi057.stdout:0/543: rename d17/d20/d26/d48/d7c/f91 to d17/d20/d4a/d78/fc7 0 2024-02-28T23:07:41.367 INFO:tasks.workunit.client.1.smithi084.stdout:0/624: dread - d1/d6/db/d35/f8a zero size 2024-02-28T23:07:41.368 INFO:tasks.workunit.client.1.smithi084.stdout:7/692: mkdir db/d5e/dce 0 2024-02-28T23:07:41.369 INFO:tasks.workunit.client.1.smithi084.stdout:8/607: dread - d0/d9/fbf zero size 2024-02-28T23:07:41.373 INFO:tasks.workunit.client.1.smithi084.stdout:4/618: mknod d1/d18/d20/d65/d8e/cd0 0 2024-02-28T23:07:41.377 INFO:tasks.workunit.client.1.smithi084.stdout:6/617: dread d1/d13/d2e/d41/d2c/f78 [0,4194304] 0 2024-02-28T23:07:41.380 INFO:tasks.workunit.client.0.smithi057.stdout:0/544: link d17/d20/d4a/c8c d17/d20/d23/d47/d60/d6f/cc8 0 2024-02-28T23:07:41.382 INFO:tasks.workunit.client.0.smithi057.stdout:3/409: dwrite d3/d8/d1b/f37 [0,4194304] 0 2024-02-28T23:07:41.382 INFO:tasks.workunit.client.0.smithi057.stdout:7/491: mknod d0/d5/d36/d4e/d15/d29/d71/c9b 0 2024-02-28T23:07:41.382 INFO:tasks.workunit.client.1.smithi084.stdout:1/701: rename d12/d1b/d63/f66 to d12/da2/fe4 0 2024-02-28T23:07:41.385 INFO:tasks.workunit.client.0.smithi057.stdout:7/492: write d0/d5/d36/d4e/f83 [585437,73402] 0 2024-02-28T23:07:41.386 INFO:tasks.workunit.client.1.smithi084.stdout:0/625: creat d1/d6/db/d35/d82/daf/fc6 x:0 0 0 2024-02-28T23:07:41.389 INFO:tasks.workunit.client.0.smithi057.stdout:9/539: write dd/de/f6b [568378,8175] 0 2024-02-28T23:07:41.389 INFO:tasks.workunit.client.1.smithi084.stdout:5/680: dread dc/f25 [0,4194304] 0 2024-02-28T23:07:41.391 INFO:tasks.workunit.client.1.smithi084.stdout:0/626: chown d1/d6/db/d35/d4a/laa 2 1 2024-02-28T23:07:41.392 INFO:tasks.workunit.client.1.smithi084.stdout:4/619: rename d1/d18/d20/d24/d2f/d3e/d52/ca9 to d1/d18/d20/d24/d2f/d3e/d52/d87/cd1 0 2024-02-28T23:07:41.392 INFO:tasks.workunit.client.0.smithi057.stdout:3/410: fsync d3/d20/f23 0 2024-02-28T23:07:41.393 INFO:tasks.workunit.client.1.smithi084.stdout:5/681: write dc/de/df/da6/fdd [885879,75449] 0 2024-02-28T23:07:41.394 INFO:tasks.workunit.client.0.smithi057.stdout:1/415: truncate d2/f35 1936858 0 2024-02-28T23:07:41.396 INFO:tasks.workunit.client.0.smithi057.stdout:1/416: fsync d2/d71/d2d/f46 0 2024-02-28T23:07:41.397 INFO:tasks.workunit.client.1.smithi084.stdout:1/702: rmdir d12/d1b/d2e/d52/dd7 39 2024-02-28T23:07:41.398 INFO:tasks.workunit.client.0.smithi057.stdout:4/529: dread d1a/d48/db0/d4f/f64 [0,4194304] 0 2024-02-28T23:07:41.398 INFO:tasks.workunit.client.0.smithi057.stdout:8/568: dread d4/d13/d1a/f4b [0,4194304] 0 2024-02-28T23:07:41.401 INFO:tasks.workunit.client.0.smithi057.stdout:9/540: write dd/de/d24/f28 [881861,127909] 0 2024-02-28T23:07:41.402 INFO:tasks.workunit.client.1.smithi084.stdout:6/618: symlink d1/d9/d1d/d1e/d33/ld8 0 2024-02-28T23:07:41.404 INFO:tasks.workunit.client.0.smithi057.stdout:3/411: dread - d3/d4/f66 zero size 2024-02-28T23:07:41.404 INFO:tasks.workunit.client.0.smithi057.stdout:1/417: write d2/d7/df/db/d3a/f6c [240671,86188] 0 2024-02-28T23:07:41.405 INFO:tasks.workunit.client.0.smithi057.stdout:8/569: write d4/d13/d18/d62/dbb/f86 [602563,59485] 0 2024-02-28T23:07:41.406 INFO:tasks.workunit.client.1.smithi084.stdout:9/700: write d4/f60 [4795537,81196] 0 2024-02-28T23:07:41.408 INFO:tasks.workunit.client.0.smithi057.stdout:3/412: chown d3/c4b 1597748 1 2024-02-28T23:07:41.416 INFO:tasks.workunit.client.0.smithi057.stdout:3/413: creat d3/d4f/f83 x:0 0 0 2024-02-28T23:07:41.418 INFO:tasks.workunit.client.1.smithi084.stdout:4/620: read d1/d18/d20/d14/d30/f81 [1277993,52110] 0 2024-02-28T23:07:41.419 INFO:tasks.workunit.client.1.smithi084.stdout:8/608: dread d0/d11/f3c [0,4194304] 0 2024-02-28T23:07:41.422 INFO:tasks.workunit.client.1.smithi084.stdout:6/619: mknod d1/d9/d1d/d1e/cd9 0 2024-02-28T23:07:41.422 INFO:tasks.workunit.client.1.smithi084.stdout:2/590: dread d5/d7/d12/d24/d2a/d35/fa2 [0,4194304] 0 2024-02-28T23:07:41.422 INFO:tasks.workunit.client.0.smithi057.stdout:8/570: mknod d4/d1e/d56/d6a/d54/ccc 0 2024-02-28T23:07:41.424 INFO:tasks.workunit.client.0.smithi057.stdout:3/414: creat d3/d4/d7/f84 x:0 0 0 2024-02-28T23:07:41.425 INFO:tasks.workunit.client.1.smithi084.stdout:7/693: getdents db/d10/d17/d9c/d9f/db1/db2 0 2024-02-28T23:07:41.426 INFO:tasks.workunit.client.1.smithi084.stdout:2/591: fdatasync d5/d7/d2e/f9a 0 2024-02-28T23:07:41.428 INFO:tasks.workunit.client.0.smithi057.stdout:7/493: link d0/d5/d36/d4e/d15/d29/d3d/f88 d0/d5/d36/d4e/d15/d29/d3d/f9c 0 2024-02-28T23:07:41.429 INFO:tasks.workunit.client.1.smithi084.stdout:7/694: chown db/d10/d18/f82 129447958 1 2024-02-28T23:07:41.429 INFO:tasks.workunit.client.0.smithi057.stdout:0/545: write d17/d20/d4a/d78/fc7 [995772,31769] 0 2024-02-28T23:07:41.430 INFO:tasks.workunit.client.1.smithi084.stdout:9/701: dwrite d4/f60 [0,4194304] 0 2024-02-28T23:07:41.430 INFO:tasks.workunit.client.1.smithi084.stdout:0/627: truncate d1/d6/f13 3936797 0 2024-02-28T23:07:41.432 INFO:tasks.workunit.client.0.smithi057.stdout:3/415: creat d3/d20/d2d/f85 x:0 0 0 2024-02-28T23:07:41.432 INFO:tasks.workunit.client.1.smithi084.stdout:0/628: fdatasync d1/d8/d69/d6b/f9b 0 2024-02-28T23:07:41.433 INFO:tasks.workunit.client.1.smithi084.stdout:9/702: chown d4/d6/d15/d12/ddd 27778 1 2024-02-28T23:07:41.435 INFO:tasks.workunit.client.1.smithi084.stdout:6/620: symlink d1/d9/d1d/d1e/d33/d38/lda 0 2024-02-28T23:07:41.436 INFO:tasks.workunit.client.0.smithi057.stdout:9/541: write dd/f12 [1529418,103025] 0 2024-02-28T23:07:41.439 INFO:tasks.workunit.client.0.smithi057.stdout:7/494: creat d0/d5/d36/d4e/d3b/f9d x:0 0 0 2024-02-28T23:07:41.439 INFO:tasks.workunit.client.1.smithi084.stdout:8/609: symlink d0/d44/d9f/lcd 0 2024-02-28T23:07:41.440 INFO:tasks.workunit.client.0.smithi057.stdout:4/530: truncate d1a/d48/f79 375301 0 2024-02-28T23:07:41.442 INFO:tasks.workunit.client.1.smithi084.stdout:5/682: dread dc/de/df/d6e/faf [4194304,4194304] 0 2024-02-28T23:07:41.443 INFO:tasks.workunit.client.0.smithi057.stdout:9/542: creat dd/de/d4a/d9f/fb3 x:0 0 0 2024-02-28T23:07:41.443 INFO:tasks.workunit.client.1.smithi084.stdout:0/629: symlink d1/d6/db/d35/d44/d92/lc7 0 2024-02-28T23:07:41.444 INFO:tasks.workunit.client.1.smithi084.stdout:1/703: dread d12/d1b/d3b/f93 [4194304,4194304] 0 2024-02-28T23:07:41.444 INFO:tasks.workunit.client.0.smithi057.stdout:7/495: rename d0/d5/d36/d4e/f83 to d0/d5/d36/d4e/d3b/f9e 0 2024-02-28T23:07:41.445 INFO:tasks.workunit.client.0.smithi057.stdout:3/416: read d3/f2c [2214740,75403] 0 2024-02-28T23:07:41.447 INFO:tasks.workunit.client.1.smithi084.stdout:8/610: fdatasync d0/d24/f5b 0 2024-02-28T23:07:41.450 INFO:tasks.workunit.client.1.smithi084.stdout:1/704: write d12/faf [121442,96878] 0 2024-02-28T23:07:41.450 INFO:tasks.workunit.client.1.smithi084.stdout:2/592: getdents d5/d7/d2e/d4a 0 2024-02-28T23:07:41.452 INFO:tasks.workunit.client.0.smithi057.stdout:1/418: dread d2/d7/df/db/d3a/d2a/f2c [0,4194304] 0 2024-02-28T23:07:41.454 INFO:tasks.workunit.client.0.smithi057.stdout:9/543: creat dd/de/d24/d63/d77/fb4 x:0 0 0 2024-02-28T23:07:41.455 INFO:tasks.workunit.client.1.smithi084.stdout:5/683: rmdir dc/de/df/d16/d34/daa/dae 39 2024-02-28T23:07:41.457 INFO:tasks.workunit.client.0.smithi057.stdout:0/546: truncate d17/d20/d4a/f81 1598403 0 2024-02-28T23:07:41.459 INFO:tasks.workunit.client.0.smithi057.stdout:7/496: chown d0/d5/d36/d4e/d15/d29/d3d/l4f 1 1 2024-02-28T23:07:41.463 INFO:tasks.workunit.client.1.smithi084.stdout:2/593: mknod d5/dd/cb2 0 2024-02-28T23:07:41.464 INFO:tasks.workunit.client.1.smithi084.stdout:9/703: dwrite d4/d6/d15/d75/fa4 [0,4194304] 0 2024-02-28T23:07:41.464 INFO:tasks.workunit.client.0.smithi057.stdout:1/419: symlink d2/d7/df/db/d3a/d66/l95 0 2024-02-28T23:07:41.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/531: getdents d1a/d48/db0/d4f 0 2024-02-28T23:07:41.466 INFO:tasks.workunit.client.0.smithi057.stdout:9/544: fsync dd/d1b/d40/d44/f62 0 2024-02-28T23:07:41.467 INFO:tasks.workunit.client.1.smithi084.stdout:7/695: write db/d10/d17/d3b/f3f [308957,123033] 0 2024-02-28T23:07:41.469 INFO:tasks.workunit.client.1.smithi084.stdout:0/630: truncate d1/d6/d2d/f30 2262697 0 2024-02-28T23:07:41.470 INFO:tasks.workunit.client.0.smithi057.stdout:6/556: sync 2024-02-28T23:07:41.470 INFO:tasks.workunit.client.0.smithi057.stdout:5/521: sync 2024-02-28T23:07:41.472 INFO:tasks.workunit.client.1.smithi084.stdout:3/672: sync 2024-02-28T23:07:41.472 INFO:tasks.workunit.client.0.smithi057.stdout:7/497: mkdir d0/d5/d25/d80/d9f 0 2024-02-28T23:07:41.472 INFO:tasks.workunit.client.0.smithi057.stdout:0/547: dread d17/d20/d26/f69 [0,4194304] 0 2024-02-28T23:07:41.473 INFO:tasks.workunit.client.1.smithi084.stdout:7/696: chown db/d10/d17/d9c/d9f/db1/d36 116 1 2024-02-28T23:07:41.473 INFO:tasks.workunit.client.1.smithi084.stdout:8/611: getdents d0/d11 0 2024-02-28T23:07:41.474 INFO:tasks.workunit.client.1.smithi084.stdout:4/621: dread d1/d18/d20/d14/d30/f38 [0,4194304] 0 2024-02-28T23:07:41.474 INFO:tasks.workunit.client.1.smithi084.stdout:5/684: symlink dc/lfc 0 2024-02-28T23:07:41.478 INFO:tasks.workunit.client.1.smithi084.stdout:2/594: chown d5/d7/d12/d24/d2a/f52 243 1 2024-02-28T23:07:41.478 INFO:tasks.workunit.client.1.smithi084.stdout:0/631: fsync d1/d8/d69/d6b/d85/fab 0 2024-02-28T23:07:41.479 INFO:tasks.workunit.client.0.smithi057.stdout:9/545: mknod dd/de/d14/d48/d4b/cb5 0 2024-02-28T23:07:41.480 INFO:tasks.workunit.client.1.smithi084.stdout:9/704: truncate d4/d6/d15/f98 2671967 0 2024-02-28T23:07:41.482 INFO:tasks.workunit.client.0.smithi057.stdout:3/417: dwrite d3/d4/d1d/f1e [4194304,4194304] 0 2024-02-28T23:07:41.483 INFO:tasks.workunit.client.1.smithi084.stdout:5/685: read - dc/fad zero size 2024-02-28T23:07:41.484 INFO:tasks.workunit.client.0.smithi057.stdout:5/522: chown d4/d9/d14/d3b/fa0 3055 1 2024-02-28T23:07:41.485 INFO:tasks.workunit.client.1.smithi084.stdout:0/632: creat d1/d6/d2d/fc8 x:0 0 0 2024-02-28T23:07:41.486 INFO:tasks.workunit.client.1.smithi084.stdout:4/622: readlink d1/d18/d20/d24/d2f/d3e/d52/d45/db5/lcb 0 2024-02-28T23:07:41.487 INFO:tasks.workunit.client.0.smithi057.stdout:8/571: dwrite d4/d1e/d56/d6a/d54/f70 [0,4194304] 0 2024-02-28T23:07:41.487 INFO:tasks.workunit.client.0.smithi057.stdout:9/546: chown dd/de/d24/d63 19 1 2024-02-28T23:07:41.488 INFO:tasks.workunit.client.1.smithi084.stdout:9/705: readlink d4/d6/d15/d3c/l7f 0 2024-02-28T23:07:41.488 INFO:tasks.workunit.client.1.smithi084.stdout:8/612: mkdir d0/d9/d30/d33/d71/d84/dce 0 2024-02-28T23:07:41.489 INFO:tasks.workunit.client.0.smithi057.stdout:6/557: creat d0/d4/d25/fbe x:0 0 0 2024-02-28T23:07:41.489 INFO:tasks.workunit.client.1.smithi084.stdout:5/686: fsync dc/de/d83/dd9/ffb 0 2024-02-28T23:07:41.490 INFO:tasks.workunit.client.1.smithi084.stdout:2/595: fdatasync d5/f15 0 2024-02-28T23:07:41.493 INFO:tasks.workunit.client.1.smithi084.stdout:1/705: dwrite d12/d1b/d2e/d32/f47 [4194304,4194304] 0 2024-02-28T23:07:41.496 INFO:tasks.workunit.client.1.smithi084.stdout:6/621: dread d1/d9/d1d/d1e/d33/f89 [4194304,4194304] 0 2024-02-28T23:07:41.497 INFO:tasks.workunit.client.1.smithi084.stdout:8/613: mknod d0/d9/d30/db6/ccf 0 2024-02-28T23:07:41.498 INFO:tasks.workunit.client.0.smithi057.stdout:0/548: creat d17/d20/db6/fc9 x:0 0 0 2024-02-28T23:07:41.501 INFO:tasks.workunit.client.0.smithi057.stdout:6/558: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:41.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/549: stat d17/d20/d26/d48/f84 0 2024-02-28T23:07:41.503 INFO:tasks.workunit.client.1.smithi084.stdout:9/706: symlink d4/d32/d3f/le3 0 2024-02-28T23:07:41.504 INFO:tasks.workunit.client.1.smithi084.stdout:9/707: chown d4/d6/d15/f71 1 1 2024-02-28T23:07:41.504 INFO:tasks.workunit.client.1.smithi084.stdout:2/596: rename d5/dd/d69/c71 to d5/d7/d75/cb3 0 2024-02-28T23:07:41.508 INFO:tasks.workunit.client.0.smithi057.stdout:7/498: mkdir d0/d5/d36/d4e/d15/da0 0 2024-02-28T23:07:41.508 INFO:tasks.workunit.client.0.smithi057.stdout:6/559: dread d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:41.508 INFO:tasks.workunit.client.0.smithi057.stdout:0/550: fdatasync d17/d19/f2f 0 2024-02-28T23:07:41.508 INFO:tasks.workunit.client.1.smithi084.stdout:6/622: fsync d1/d13/d2e/d41/d4d/d6c/f7c 0 2024-02-28T23:07:41.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/633: mknod d1/d8/d69/d6b/d85/da5/cc9 0 2024-02-28T23:07:41.510 INFO:tasks.workunit.client.0.smithi057.stdout:3/418: creat d3/d8/f86 x:0 0 0 2024-02-28T23:07:41.510 INFO:tasks.workunit.client.0.smithi057.stdout:9/547: truncate dd/de/d14/d48/d4b/d41/f93 582547 0 2024-02-28T23:07:41.511 INFO:tasks.workunit.client.0.smithi057.stdout:4/532: dwrite f18 [0,4194304] 0 2024-02-28T23:07:41.514 INFO:tasks.workunit.client.1.smithi084.stdout:6/623: chown d1/d9/d1d/d1e/d33/f39 11109 1 2024-02-28T23:07:41.516 INFO:tasks.workunit.client.0.smithi057.stdout:1/420: truncate d2/d7/df/f4d 1087765 0 2024-02-28T23:07:41.519 INFO:tasks.workunit.client.1.smithi084.stdout:9/708: creat d4/d6/d15/d75/d9d/dd0/fe4 x:0 0 0 2024-02-28T23:07:41.520 INFO:tasks.workunit.client.1.smithi084.stdout:5/687: link dc/de/df/d16/d34/lcd dc/de/df/d31/dd1/da4/lfd 0 2024-02-28T23:07:41.522 INFO:tasks.workunit.client.0.smithi057.stdout:9/548: dread - dd/f5b zero size 2024-02-28T23:07:41.522 INFO:tasks.workunit.client.0.smithi057.stdout:5/523: dwrite d4/d9/d14/d3b/d61/d69/f93 [0,4194304] 0 2024-02-28T23:07:41.522 INFO:tasks.workunit.client.0.smithi057.stdout:2/607: sync 2024-02-28T23:07:41.523 INFO:tasks.workunit.client.0.smithi057.stdout:6/560: fdatasync d0/d4/d25/f2a 0 2024-02-28T23:07:41.523 INFO:tasks.workunit.client.0.smithi057.stdout:7/499: mkdir d0/d3f/da1 0 2024-02-28T23:07:41.526 INFO:tasks.workunit.client.0.smithi057.stdout:1/421: fsync d2/d7/df/db/d12/d3f/f8c 0 2024-02-28T23:07:41.527 INFO:tasks.workunit.client.1.smithi084.stdout:7/697: write db/d5e/f6d [832662,20809] 0 2024-02-28T23:07:41.529 INFO:tasks.workunit.client.1.smithi084.stdout:4/623: dwrite d1/d18/d20/d24/d2f/d3e/d52/d87/da5/fbf [4194304,4194304] 0 2024-02-28T23:07:41.530 INFO:tasks.workunit.client.0.smithi057.stdout:6/561: write d0/d4/d25/fa9 [862230,99905] 0 2024-02-28T23:07:41.531 INFO:tasks.workunit.client.1.smithi084.stdout:0/634: readlink d1/d6/db/l90 0 2024-02-28T23:07:41.532 INFO:tasks.workunit.client.0.smithi057.stdout:7/500: creat d0/d98/fa2 x:0 0 0 2024-02-28T23:07:41.534 INFO:tasks.workunit.client.1.smithi084.stdout:7/698: write db/d10/f94 [623804,46507] 0 2024-02-28T23:07:41.535 INFO:tasks.workunit.client.1.smithi084.stdout:0/635: stat d1/d8/f3c 0 2024-02-28T23:07:41.536 INFO:tasks.workunit.client.0.smithi057.stdout:6/562: mknod d0/d17/d94/cbf 0 2024-02-28T23:07:41.537 INFO:tasks.workunit.client.0.smithi057.stdout:8/572: dwrite d4/d1e/d56/d6a/d54/f70 [0,4194304] 0 2024-02-28T23:07:41.539 INFO:tasks.workunit.client.1.smithi084.stdout:0/636: chown d1/d6/db/d76/ca2 48 1 2024-02-28T23:07:41.540 INFO:tasks.workunit.client.0.smithi057.stdout:2/608: rename d8/c6a to d8/d11/d21/d28/cc2 0 2024-02-28T23:07:41.542 INFO:tasks.workunit.client.0.smithi057.stdout:8/573: chown d4/d13/d7f/fa1 92840 1 2024-02-28T23:07:41.542 INFO:tasks.workunit.client.1.smithi084.stdout:3/673: dread d3/d27/d7d/db0/df/f12 [0,4194304] 0 2024-02-28T23:07:41.542 INFO:tasks.workunit.client.1.smithi084.stdout:0/637: write d1/d8/d69/db2/f54 [5016152,127117] 0 2024-02-28T23:07:41.543 INFO:tasks.workunit.client.1.smithi084.stdout:9/709: dwrite d4/d6/d15/d12/f3d [0,4194304] 0 2024-02-28T23:07:41.545 INFO:tasks.workunit.client.0.smithi057.stdout:1/422: rename d2/d7/df/db/d3a/d2a/f2c to d2/d7/df/db/d12/d19/d65/f96 0 2024-02-28T23:07:41.546 INFO:tasks.workunit.client.1.smithi084.stdout:5/688: mkdir dc/de/df/d31/dd1/d5f/d9b/dfe 0 2024-02-28T23:07:41.547 INFO:tasks.workunit.client.0.smithi057.stdout:2/609: chown d8/d11/d21/d28/d49/d85/l88 130966 1 2024-02-28T23:07:41.550 INFO:tasks.workunit.client.1.smithi084.stdout:3/674: dread - d3/d27/d7d/db0/d1a/d29/d41/d60/fc5 zero size 2024-02-28T23:07:41.550 INFO:tasks.workunit.client.0.smithi057.stdout:2/610: readlink d8/d11/d39/lb0 0 2024-02-28T23:07:41.550 INFO:tasks.workunit.client.0.smithi057.stdout:8/574: creat d4/d1e/d56/d6a/d42/fcd x:0 0 0 2024-02-28T23:07:41.550 INFO:tasks.workunit.client.0.smithi057.stdout:7/501: write d0/d5/d36/d4e/d15/d29/f32 [1852422,114099] 0 2024-02-28T23:07:41.551 INFO:tasks.workunit.client.1.smithi084.stdout:3/675: fsync f1 0 2024-02-28T23:07:41.553 INFO:tasks.workunit.client.0.smithi057.stdout:1/423: symlink d2/d7/df/db/d12/d3f/l97 0 2024-02-28T23:07:41.557 INFO:tasks.workunit.client.0.smithi057.stdout:2/611: unlink d8/d11/f5b 0 2024-02-28T23:07:41.558 INFO:tasks.workunit.client.1.smithi084.stdout:4/624: creat d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fd2 x:0 0 0 2024-02-28T23:07:41.560 INFO:tasks.workunit.client.1.smithi084.stdout:1/706: rename d12/d1b/d63/fc4 to d12/d1b/d2e/fe5 0 2024-02-28T23:07:41.560 INFO:tasks.workunit.client.0.smithi057.stdout:8/575: read d4/d13/d18/d62/dbb/f86 [293415,1881] 0 2024-02-28T23:07:41.561 INFO:tasks.workunit.client.1.smithi084.stdout:3/676: write d3/d27/d7d/db0/d1a/d21/d3d/f67 [275861,29244] 0 2024-02-28T23:07:41.563 INFO:tasks.workunit.client.1.smithi084.stdout:2/597: write d5/d1f/d42/d8f/fa8 [700525,15881] 0 2024-02-28T23:07:41.564 INFO:tasks.workunit.client.0.smithi057.stdout:2/612: rename d8/d11/d39 to d8/d11/d39/d7b/d8b/da2/dc3 22 2024-02-28T23:07:41.565 INFO:tasks.workunit.client.0.smithi057.stdout:3/419: rmdir d3/d8 39 2024-02-28T23:07:41.568 INFO:tasks.workunit.client.1.smithi084.stdout:8/614: dwrite d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:41.569 INFO:tasks.workunit.client.0.smithi057.stdout:8/576: creat d4/d1e/d56/d6a/d42/fce x:0 0 0 2024-02-28T23:07:41.569 INFO:tasks.workunit.client.0.smithi057.stdout:0/551: write d17/d20/f2b [206854,22908] 0 2024-02-28T23:07:41.569 INFO:tasks.workunit.client.0.smithi057.stdout:7/502: creat d0/d5/d25/d3e/fa3 x:0 0 0 2024-02-28T23:07:41.574 INFO:tasks.workunit.client.1.smithi084.stdout:6/624: rmdir d1/d13/d9d/da3 0 2024-02-28T23:07:41.574 INFO:tasks.workunit.client.1.smithi084.stdout:8/615: stat d0/d9/d1b/d38/d43/d51/d54 0 2024-02-28T23:07:41.575 INFO:tasks.workunit.client.0.smithi057.stdout:3/420: symlink d3/d4/d1d/d47/l87 0 2024-02-28T23:07:41.575 INFO:tasks.workunit.client.0.smithi057.stdout:9/549: write dd/f2d [1910926,130100] 0 2024-02-28T23:07:41.579 INFO:tasks.workunit.client.1.smithi084.stdout:3/677: fsync d3/d27/d7d/db0/d1a/d45/f78 0 2024-02-28T23:07:41.579 INFO:tasks.workunit.client.0.smithi057.stdout:1/424: dwrite d2/d7/df/db/d3a/f3c [0,4194304] 0 2024-02-28T23:07:41.580 INFO:tasks.workunit.client.1.smithi084.stdout:6/625: write d1/d88/dbf/fc7 [646140,75441] 0 2024-02-28T23:07:41.580 INFO:tasks.workunit.client.1.smithi084.stdout:9/710: dwrite d4/d6/d15/d12/f48 [4194304,4194304] 0 2024-02-28T23:07:41.585 INFO:tasks.workunit.client.1.smithi084.stdout:0/638: rename d1/d8/d78/c94 to d1/d8/d18/d4c/d6c/cca 0 2024-02-28T23:07:41.585 INFO:tasks.workunit.client.0.smithi057.stdout:0/552: symlink d17/d20/d4a/d78/lca 0 2024-02-28T23:07:41.586 INFO:tasks.workunit.client.1.smithi084.stdout:4/625: mkdir d1/d18/d20/d14/dd3 0 2024-02-28T23:07:41.586 INFO:tasks.workunit.client.0.smithi057.stdout:9/550: symlink dd/d1b/d40/d44/lb6 0 2024-02-28T23:07:41.587 INFO:tasks.workunit.client.0.smithi057.stdout:3/421: creat d3/d4/d7/f88 x:0 0 0 2024-02-28T23:07:41.588 INFO:tasks.workunit.client.1.smithi084.stdout:5/689: creat dc/de/df/d31/dd1/da4/dee/fff x:0 0 0 2024-02-28T23:07:41.588 INFO:tasks.workunit.client.0.smithi057.stdout:6/563: truncate d0/d1b/d54/f65 771556 0 2024-02-28T23:07:41.588 INFO:tasks.workunit.client.0.smithi057.stdout:8/577: symlink d4/d13/d1a/d4a/db3/lcf 0 2024-02-28T23:07:41.589 INFO:tasks.workunit.client.1.smithi084.stdout:1/707: creat d12/d1b/d91/d9b/da5/dcb/dd4/fe6 x:0 0 0 2024-02-28T23:07:41.591 INFO:tasks.workunit.client.0.smithi057.stdout:0/553: write d17/d20/d23/d47/fa2 [897408,9377] 0 2024-02-28T23:07:41.592 INFO:tasks.workunit.client.0.smithi057.stdout:6/564: fdatasync d0/d17/d94/f9e 0 2024-02-28T23:07:41.592 INFO:tasks.workunit.client.1.smithi084.stdout:2/598: mkdir d5/d7/d75/db4 0 2024-02-28T23:07:41.592 INFO:tasks.workunit.client.0.smithi057.stdout:3/422: creat d3/d4f/d4c/f89 x:0 0 0 2024-02-28T23:07:41.597 INFO:tasks.workunit.client.1.smithi084.stdout:2/599: read d5/d1f/d42/d8f/fa8 [168187,2079] 0 2024-02-28T23:07:41.598 INFO:tasks.workunit.client.0.smithi057.stdout:0/554: stat cf 0 2024-02-28T23:07:41.598 INFO:tasks.workunit.client.0.smithi057.stdout:3/423: fdatasync d3/d20/d2d/f56 0 2024-02-28T23:07:41.598 INFO:tasks.workunit.client.0.smithi057.stdout:9/551: rmdir dd/d46 39 2024-02-28T23:07:41.600 INFO:tasks.workunit.client.1.smithi084.stdout:0/639: symlink d1/d8/d69/d6b/lcb 0 2024-02-28T23:07:41.601 INFO:tasks.workunit.client.0.smithi057.stdout:8/578: unlink d4/d1e/lc6 0 2024-02-28T23:07:41.603 INFO:tasks.workunit.client.1.smithi084.stdout:6/626: dwrite d1/d13/d47/f56 [0,4194304] 0 2024-02-28T23:07:41.605 INFO:tasks.workunit.client.0.smithi057.stdout:6/565: unlink d0/d1b/d3e/d3d/d42/d4f/d53/c6e 0 2024-02-28T23:07:41.607 INFO:tasks.workunit.client.1.smithi084.stdout:9/711: creat d4/d6/d15/d12/d41/fe5 x:0 0 0 2024-02-28T23:07:41.608 INFO:tasks.workunit.client.0.smithi057.stdout:1/425: link d2/d71/d62/f74 d2/d7/df/db/d12/d19/d65/f98 0 2024-02-28T23:07:41.608 INFO:tasks.workunit.client.0.smithi057.stdout:3/424: readlink d3/d8/d27/l5c 0 2024-02-28T23:07:41.609 INFO:tasks.workunit.client.0.smithi057.stdout:2/613: truncate d8/f19 541886 0 2024-02-28T23:07:41.610 INFO:tasks.workunit.client.0.smithi057.stdout:8/579: symlink d4/d13/d2c/d4e/ld0 0 2024-02-28T23:07:41.612 INFO:tasks.workunit.client.0.smithi057.stdout:6/566: mknod d0/d1b/d3e/d3d/d7d/cc0 0 2024-02-28T23:07:41.612 INFO:tasks.workunit.client.1.smithi084.stdout:7/699: truncate db/d5e/dc6/f68 1684878 0 2024-02-28T23:07:41.613 INFO:tasks.workunit.client.1.smithi084.stdout:5/690: mknod dc/de/df/db0/c100 0 2024-02-28T23:07:41.613 INFO:tasks.workunit.client.1.smithi084.stdout:1/708: mknod d12/d1b/d2e/ce7 0 2024-02-28T23:07:41.616 INFO:tasks.workunit.client.1.smithi084.stdout:5/691: read - dc/de/df/d16/d34/d3c/fec zero size 2024-02-28T23:07:41.620 INFO:tasks.workunit.client.0.smithi057.stdout:2/614: creat d8/d11/d15/fc4 x:0 0 0 2024-02-28T23:07:41.621 INFO:tasks.workunit.client.0.smithi057.stdout:8/580: creat d4/d1e/d56/d88/fd1 x:0 0 0 2024-02-28T23:07:41.624 INFO:tasks.workunit.client.1.smithi084.stdout:0/640: dwrite d1/d8/d69/d6b/f7a [0,4194304] 0 2024-02-28T23:07:41.625 INFO:tasks.workunit.client.1.smithi084.stdout:5/692: symlink dc/de/d83/l101 0 2024-02-28T23:07:41.627 INFO:tasks.workunit.client.0.smithi057.stdout:1/426: creat d2/d7/df/db/d3a/f99 x:0 0 0 2024-02-28T23:07:41.627 INFO:tasks.workunit.client.1.smithi084.stdout:4/626: rename d1/d18/d20/d24/d2f/d3e/d52/d87 to d1/d18/d20/d24/d2f/d3e/d52/d45/dd4 0 2024-02-28T23:07:41.627 INFO:tasks.workunit.client.1.smithi084.stdout:2/600: dwrite d5/d7/d9/f3c [0,4194304] 0 2024-02-28T23:07:41.629 INFO:tasks.workunit.client.0.smithi057.stdout:0/555: dwrite d17/d20/d23/d4b/fb9 [0,4194304] 0 2024-02-28T23:07:41.629 INFO:tasks.workunit.client.1.smithi084.stdout:8/616: rename d0/d24 to d0/d24/d4d/dd0 22 2024-02-28T23:07:41.634 INFO:tasks.workunit.client.0.smithi057.stdout:7/503: dwrite d0/f50 [0,4194304] 0 2024-02-28T23:07:41.635 INFO:tasks.workunit.client.0.smithi057.stdout:8/581: creat d4/d13/d1a/d4a/db3/fd2 x:0 0 0 2024-02-28T23:07:41.638 INFO:tasks.workunit.client.0.smithi057.stdout:2/615: stat d8/d11/d48/d50/cb4 0 2024-02-28T23:07:41.640 INFO:tasks.workunit.client.1.smithi084.stdout:1/709: link d12/d1b/d91/d9b/fc1 d12/d1b/d2e/d30/fe8 0 2024-02-28T23:07:41.640 INFO:tasks.workunit.client.1.smithi084.stdout:0/641: write d1/d8/d69/db2/fbc [121688,114508] 0 2024-02-28T23:07:41.640 INFO:tasks.workunit.client.1.smithi084.stdout:5/693: unlink dc/de/df/d31/dd1/d5f/d66/f96 0 2024-02-28T23:07:41.641 INFO:tasks.workunit.client.0.smithi057.stdout:0/556: write f4 [647771,128651] 0 2024-02-28T23:07:41.644 INFO:tasks.workunit.client.0.smithi057.stdout:8/582: read d4/f7 [16186945,124788] 0 2024-02-28T23:07:41.644 INFO:tasks.workunit.client.0.smithi057.stdout:1/427: mknod d2/d7/d4f/d5c/d6b/c9a 0 2024-02-28T23:07:41.644 INFO:tasks.workunit.client.0.smithi057.stdout:2/616: fdatasync d8/d11/d39/d7b/d8b/f96 0 2024-02-28T23:07:41.645 INFO:tasks.workunit.client.0.smithi057.stdout:1/428: stat d2/l1e 0 2024-02-28T23:07:41.648 INFO:tasks.workunit.client.1.smithi084.stdout:8/617: truncate d0/d9/d30/d33/d71/fa4 1143208 0 2024-02-28T23:07:41.649 INFO:tasks.workunit.client.1.smithi084.stdout:3/678: dwrite d3/d27/d7d/db0/d14/f2e [0,4194304] 0 2024-02-28T23:07:41.649 INFO:tasks.workunit.client.1.smithi084.stdout:1/710: chown l5 27 1 2024-02-28T23:07:41.650 INFO:tasks.workunit.client.1.smithi084.stdout:9/712: rename d4/d32/d3f/l43 to d4/d32/d3f/d89/le6 0 2024-02-28T23:07:41.651 INFO:tasks.workunit.client.0.smithi057.stdout:7/504: symlink d0/d5/d25/d80/d9f/la4 0 2024-02-28T23:07:41.654 INFO:tasks.workunit.client.0.smithi057.stdout:0/557: write d17/d20/d26/f69 [521915,115184] 0 2024-02-28T23:07:41.654 INFO:tasks.workunit.client.0.smithi057.stdout:8/583: truncate d4/d13/d1a/f4b 4770456 0 2024-02-28T23:07:41.658 INFO:tasks.workunit.client.1.smithi084.stdout:1/711: chown d12/d1b/d3b/f86 2513534 1 2024-02-28T23:07:41.658 INFO:tasks.workunit.client.1.smithi084.stdout:5/694: rename dc/de/df/d16/f29 to dc/de/df/d16/d34/de8/f102 0 2024-02-28T23:07:41.661 INFO:tasks.workunit.client.1.smithi084.stdout:1/712: fdatasync d12/d1b/d2e/d30/f1c 0 2024-02-28T23:07:41.661 INFO:tasks.workunit.client.1.smithi084.stdout:5/695: chown dc/de/df/d16/d34/fea 8330 1 2024-02-28T23:07:41.662 INFO:tasks.workunit.client.1.smithi084.stdout:8/618: read d0/d9/d30/d33/d71/d84/fcc [2386609,13204] 0 2024-02-28T23:07:41.662 INFO:tasks.workunit.client.1.smithi084.stdout:3/679: symlink d3/d27/d7d/db0/d1a/d29/d41/l101 0 2024-02-28T23:07:41.664 INFO:tasks.workunit.client.1.smithi084.stdout:4/627: creat d1/d18/d20/d24/d2f/d3e/d52/d45/fd5 x:0 0 0 2024-02-28T23:07:41.668 INFO:tasks.workunit.client.1.smithi084.stdout:9/713: mkdir d4/d6/d15/db3/de7 0 2024-02-28T23:07:41.675 INFO:tasks.workunit.client.0.smithi057.stdout:1/429: dwrite d2/d33/d54/f89 [0,4194304] 0 2024-02-28T23:07:41.675 INFO:tasks.workunit.client.0.smithi057.stdout:0/558: read d17/d20/d26/d30/d6b/d71/f7e [3768644,584] 0 2024-02-28T23:07:41.678 INFO:tasks.workunit.client.1.smithi084.stdout:2/601: getdents d5/d7/d9/d7c 0 2024-02-28T23:07:41.678 INFO:tasks.workunit.client.1.smithi084.stdout:3/680: mknod d3/d27/d7d/db0/d14/d31/c102 0 2024-02-28T23:07:41.680 INFO:tasks.workunit.client.0.smithi057.stdout:1/430: dread - d2/d7/f37 zero size 2024-02-28T23:07:41.681 INFO:tasks.workunit.client.0.smithi057.stdout:8/584: rename d4/d13/d2c/l2f to d4/ld3 0 2024-02-28T23:07:41.682 INFO:tasks.workunit.client.0.smithi057.stdout:1/431: chown d2/d7/df/db/d12/d3f/f8c 76966 1 2024-02-28T23:07:41.682 INFO:tasks.workunit.client.1.smithi084.stdout:5/696: unlink dc/de/df/d16/f81 0 2024-02-28T23:07:41.685 INFO:tasks.workunit.client.0.smithi057.stdout:1/432: chown d2/d7/f37 43 1 2024-02-28T23:07:41.685 INFO:tasks.workunit.client.1.smithi084.stdout:1/713: dread d12/d1b/d2e/d52/f60 [0,4194304] 0 2024-02-28T23:07:41.686 INFO:tasks.workunit.client.0.smithi057.stdout:8/585: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:41.687 INFO:tasks.workunit.client.1.smithi084.stdout:9/714: chown d4/d6/d15/f98 4071 1 2024-02-28T23:07:41.691 INFO:tasks.workunit.client.0.smithi057.stdout:3/425: write d3/d20/f23 [5216933,98848] 0 2024-02-28T23:07:41.691 INFO:tasks.workunit.client.0.smithi057.stdout:2/617: dread d8/fd [0,4194304] 0 2024-02-28T23:07:41.692 INFO:tasks.workunit.client.1.smithi084.stdout:5/697: creat dc/d51/f103 x:0 0 0 2024-02-28T23:07:41.694 INFO:tasks.workunit.client.0.smithi057.stdout:2/618: write d8/d11/d15/db9/fbf [944317,59347] 0 2024-02-28T23:07:41.695 INFO:tasks.workunit.client.1.smithi084.stdout:3/681: rename d3/d27/d7d/db0/d1a/d45/lcd to d3/d27/d7d/db0/d1a/d45/d86/l103 0 2024-02-28T23:07:41.696 INFO:tasks.workunit.client.1.smithi084.stdout:0/642: write d1/d8/f3e [2132132,39381] 0 2024-02-28T23:07:41.696 INFO:tasks.workunit.client.1.smithi084.stdout:9/715: write d4/f13 [3668001,7315] 0 2024-02-28T23:07:41.698 INFO:tasks.workunit.client.0.smithi057.stdout:1/433: mknod d2/d7/df/db/d3a/d8f/c9b 0 2024-02-28T23:07:41.698 INFO:tasks.workunit.client.0.smithi057.stdout:2/619: write d8/d11/d21/d70/d99/fa1 [131979,19] 0 2024-02-28T23:07:41.702 INFO:tasks.workunit.client.0.smithi057.stdout:6/567: write d0/d4/d25/f2a [2148707,64412] 0 2024-02-28T23:07:41.703 INFO:tasks.workunit.client.1.smithi084.stdout:9/716: fdatasync d4/d6/d15/d3c/f70 0 2024-02-28T23:07:41.703 INFO:tasks.workunit.client.0.smithi057.stdout:8/586: write d4/d1e/d56/d6a/d54/d61/f97 [962463,79907] 0 2024-02-28T23:07:41.706 INFO:tasks.workunit.client.0.smithi057.stdout:3/426: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:41.707 INFO:tasks.workunit.client.1.smithi084.stdout:7/700: dwrite db/d10/d17/f3d [0,4194304] 0 2024-02-28T23:07:41.712 INFO:tasks.workunit.client.1.smithi084.stdout:8/619: write d0/d24/d4d/f68 [2537948,27156] 0 2024-02-28T23:07:41.713 INFO:tasks.workunit.client.0.smithi057.stdout:1/434: creat d2/d90/f9c x:0 0 0 2024-02-28T23:07:41.713 INFO:tasks.workunit.client.1.smithi084.stdout:2/602: dread d5/dd/f58 [0,4194304] 0 2024-02-28T23:07:41.714 INFO:tasks.workunit.client.0.smithi057.stdout:8/587: creat d4/d13/d1a/d4a/fd4 x:0 0 0 2024-02-28T23:07:41.723 INFO:tasks.workunit.client.0.smithi057.stdout:6/568: creat d0/d4/fc1 x:0 0 0 2024-02-28T23:07:41.723 INFO:tasks.workunit.client.1.smithi084.stdout:7/701: creat db/d10/d17/d9c/d9f/db1/db2/fcf x:0 0 0 2024-02-28T23:07:41.726 INFO:tasks.workunit.client.1.smithi084.stdout:1/714: dwrite d12/d1b/d2e/d52/d61/d74/f88 [0,4194304] 0 2024-02-28T23:07:41.727 INFO:tasks.workunit.client.0.smithi057.stdout:2/620: write d8/d11/d21/d28/f75 [4883201,22156] 0 2024-02-28T23:07:41.727 INFO:tasks.workunit.client.1.smithi084.stdout:3/682: getdents d3/d27/d7d/db0/d1a/d45/dde/dfc 0 2024-02-28T23:07:41.729 INFO:tasks.workunit.client.1.smithi084.stdout:2/603: dread - d5/d7/d75/fac zero size 2024-02-28T23:07:41.733 INFO:tasks.workunit.client.0.smithi057.stdout:0/559: dwrite d17/d20/d23/d47/f9f [0,4194304] 0 2024-02-28T23:07:41.733 INFO:tasks.workunit.client.1.smithi084.stdout:1/715: chown d12/d1b/d2e/l87 2603384 1 2024-02-28T23:07:41.736 INFO:tasks.workunit.client.0.smithi057.stdout:8/588: getdents d4/d1e/d56 0 2024-02-28T23:07:41.736 INFO:tasks.workunit.client.0.smithi057.stdout:6/569: read d0/d4/d11/f9a [3684311,65645] 0 2024-02-28T23:07:41.738 INFO:tasks.workunit.client.1.smithi084.stdout:2/604: dread - d5/d7/d2e/f93 zero size 2024-02-28T23:07:41.738 INFO:tasks.workunit.client.0.smithi057.stdout:3/427: dwrite d3/d4/d1d/f1e [4194304,4194304] 0 2024-02-28T23:07:41.738 INFO:tasks.workunit.client.1.smithi084.stdout:4/628: dwrite d1/f9c [0,4194304] 0 2024-02-28T23:07:41.740 INFO:tasks.workunit.client.0.smithi057.stdout:6/570: chown d0/d17/c85 958247 1 2024-02-28T23:07:41.741 INFO:tasks.workunit.client.1.smithi084.stdout:1/716: chown d12/d1b/d3b/f86 54629086 1 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.1.smithi084.stdout:5/698: dwrite dc/de/df/d16/f95 [0,4194304] 0 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.1.smithi084.stdout:4/629: readlink d1/d18/d20/d14/l78 0 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.0.smithi057.stdout:0/560: fdatasync d17/d20/d26/d48/d7c/d98/fc2 0 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.0.smithi057.stdout:3/428: chown d3/d4/f17 14 1 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.0.smithi057.stdout:1/435: rename d2/d7/df/db/d3a/d2a/f2e to d2/d7/df/db/d12/d19/f9d 0 2024-02-28T23:07:41.749 INFO:tasks.workunit.client.1.smithi084.stdout:9/717: link d4/f4e d4/d6/d15/fe8 0 2024-02-28T23:07:41.751 INFO:tasks.workunit.client.0.smithi057.stdout:6/571: mknod d0/d1b/d3e/d3d/d42/d4f/d92/cc2 0 2024-02-28T23:07:41.753 INFO:tasks.workunit.client.1.smithi084.stdout:8/620: creat d0/d44/d9f/fd1 x:0 0 0 2024-02-28T23:07:41.754 INFO:tasks.workunit.client.0.smithi057.stdout:8/589: creat d4/d1e/d56/d6a/d54/d61/d8c/dab/fd5 x:0 0 0 2024-02-28T23:07:41.755 INFO:tasks.workunit.client.0.smithi057.stdout:3/429: mknod d3/d4/d7/c8a 0 2024-02-28T23:07:41.755 INFO:tasks.workunit.client.1.smithi084.stdout:2/605: chown d5/dd/d69/c84 197597 1 2024-02-28T23:07:41.756 INFO:tasks.workunit.client.0.smithi057.stdout:6/572: chown d0/d1b/d3e/d3d/d42/c8e 28511 1 2024-02-28T23:07:41.758 INFO:tasks.workunit.client.1.smithi084.stdout:2/606: chown d5/d7/d2e/d4a/d5c/f89 0 1 2024-02-28T23:07:41.758 INFO:tasks.workunit.client.0.smithi057.stdout:2/621: dwrite d8/d11/d21/d70/fc1 [0,4194304] 0 2024-02-28T23:07:41.759 INFO:tasks.workunit.client.0.smithi057.stdout:8/590: write d4/d13/d18/f26 [1430030,91052] 0 2024-02-28T23:07:41.761 INFO:tasks.workunit.client.0.smithi057.stdout:1/436: creat d2/d71/f9e x:0 0 0 2024-02-28T23:07:41.762 INFO:tasks.workunit.client.1.smithi084.stdout:0/643: dwrite d1/d6/d59/d70/f8d [0,4194304] 0 2024-02-28T23:07:41.762 INFO:tasks.workunit.client.0.smithi057.stdout:0/561: write d17/d20/d26/d48/d55/fbe [1590025,27526] 0 2024-02-28T23:07:41.763 INFO:tasks.workunit.client.0.smithi057.stdout:3/430: mknod d3/d8/d27/d70/c8b 0 2024-02-28T23:07:41.765 INFO:tasks.workunit.client.1.smithi084.stdout:4/630: unlink d1/d18/d20/d65/f72 0 2024-02-28T23:07:41.765 INFO:tasks.workunit.client.1.smithi084.stdout:1/717: link d12/d1b/d2e/d32/f51 d12/d1b/d91/d9b/da5/dcb/dd4/fe9 0 2024-02-28T23:07:41.767 INFO:tasks.workunit.client.1.smithi084.stdout:4/631: chown d1/d4b/l76 571735 1 2024-02-28T23:07:41.767 INFO:tasks.workunit.client.1.smithi084.stdout:8/621: creat d0/d24/d78/fd2 x:0 0 0 2024-02-28T23:07:41.768 INFO:tasks.workunit.client.0.smithi057.stdout:8/591: readlink d4/d13/d1a/d4a/db3/lcf 0 2024-02-28T23:07:41.768 INFO:tasks.workunit.client.0.smithi057.stdout:2/622: write d8/d11/d15/f5d [9947433,34906] 0 2024-02-28T23:07:41.770 INFO:tasks.workunit.client.0.smithi057.stdout:2/623: chown d8/d11/f32 38064 1 2024-02-28T23:07:41.770 INFO:tasks.workunit.client.1.smithi084.stdout:2/607: truncate d5/d7/d9/d7c/d5b/f7f 613116 0 2024-02-28T23:07:41.770 INFO:tasks.workunit.client.1.smithi084.stdout:3/683: rename d3/d27/d7d/db0/d1a/d45/d86/l103 to d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/l104 0 2024-02-28T23:07:41.771 INFO:tasks.workunit.client.0.smithi057.stdout:1/437: write d2/d7/df/db/d3a/f70 [1030571,97558] 0 2024-02-28T23:07:41.774 INFO:tasks.workunit.client.1.smithi084.stdout:2/608: fdatasync d5/dd/f1c 0 2024-02-28T23:07:41.774 INFO:tasks.workunit.client.1.smithi084.stdout:8/622: fsync d0/d9/d1b/d38/d43/f96 0 2024-02-28T23:07:41.776 INFO:tasks.workunit.client.0.smithi057.stdout:0/562: dread - d17/d20/d26/f4c zero size 2024-02-28T23:07:41.777 INFO:tasks.workunit.client.1.smithi084.stdout:0/644: mknod d1/d8/d69/da3/ccc 0 2024-02-28T23:07:41.779 INFO:tasks.workunit.client.0.smithi057.stdout:3/431: creat d3/d4/d1d/d47/d62/f8c x:0 0 0 2024-02-28T23:07:41.782 INFO:tasks.workunit.client.1.smithi084.stdout:1/718: mknod d12/d1b/d2e/d52/d61/d74/cea 0 2024-02-28T23:07:41.785 INFO:tasks.workunit.client.0.smithi057.stdout:2/624: mkdir d8/d11/d21/d28/d49/d85/d92/dc5 0 2024-02-28T23:07:41.786 INFO:tasks.workunit.client.1.smithi084.stdout:8/623: stat d0/d9/d1b/d38/d43/d51/fb3 0 2024-02-28T23:07:41.793 INFO:tasks.workunit.client.1.smithi084.stdout:0/645: rename d1/d6/db/c42 to d1/d6/db/d37/d72/d7b/ccd 0 2024-02-28T23:07:41.793 INFO:tasks.workunit.client.1.smithi084.stdout:0/646: chown d1/dc0 3 1 2024-02-28T23:07:41.797 INFO:tasks.workunit.client.1.smithi084.stdout:1/719: rmdir d12/d1b/d63/d7d/dd3 39 2024-02-28T23:07:41.799 INFO:tasks.workunit.client.0.smithi057.stdout:0/563: getdents d17/d20/d26/d48/d55 0 2024-02-28T23:07:41.801 INFO:tasks.workunit.client.1.smithi084.stdout:5/699: write dc/de/df/d16/d34/d3c/d98/ff0 [3395000,3224] 0 2024-02-28T23:07:41.802 INFO:tasks.workunit.client.1.smithi084.stdout:4/632: write d1/f8 [8205783,38452] 0 2024-02-28T23:07:41.802 INFO:tasks.workunit.client.0.smithi057.stdout:1/438: link d2/d7/df/db/d12/d19/d65/f98 d2/d7/df/db/d12/d3f/d47/f9f 0 2024-02-28T23:07:41.804 INFO:tasks.workunit.client.1.smithi084.stdout:1/720: creat d12/d1b/d2e/d30/d77/feb x:0 0 0 2024-02-28T23:07:41.808 INFO:tasks.workunit.client.0.smithi057.stdout:8/592: dwrite d4/d13/d2c/db1/fb2 [0,4194304] 0 2024-02-28T23:07:41.808 INFO:tasks.workunit.client.1.smithi084.stdout:0/647: symlink d1/d6/db/d37/dae/lce 0 2024-02-28T23:07:41.810 INFO:tasks.workunit.client.0.smithi057.stdout:1/439: symlink d2/d71/la0 0 2024-02-28T23:07:41.810 INFO:tasks.workunit.client.1.smithi084.stdout:1/721: dread - d12/d84/fbb zero size 2024-02-28T23:07:41.813 INFO:tasks.workunit.client.1.smithi084.stdout:4/633: write d1/d18/d20/d24/d2f/d3e/d52/d45/f92 [500751,131034] 0 2024-02-28T23:07:41.815 INFO:tasks.workunit.client.0.smithi057.stdout:6/573: dwrite d0/d4/d11/f13 [0,4194304] 0 2024-02-28T23:07:41.815 INFO:tasks.workunit.client.0.smithi057.stdout:2/625: truncate d8/fb 3114224 0 2024-02-28T23:07:41.816 INFO:tasks.workunit.client.0.smithi057.stdout:3/432: dwrite d3/d4/d1d/f36 [0,4194304] 0 2024-02-28T23:07:41.817 INFO:tasks.workunit.client.0.smithi057.stdout:8/593: write d4/d13/d18/d62/d66/fc4 [936437,108955] 0 2024-02-28T23:07:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:41 smithi057 ceph-mon[29096]: pgmap v27: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 75 MiB/s rd, 305 MiB/s wr, 525 op/s 2024-02-28T23:07:41.819 INFO:tasks.workunit.client.0.smithi057.stdout:6/574: chown d0/d17/f35 8951806 1 2024-02-28T23:07:41.821 INFO:tasks.workunit.client.1.smithi084.stdout:1/722: truncate d12/d84/d9a/fc6 548264 0 2024-02-28T23:07:41.821 INFO:tasks.workunit.client.1.smithi084.stdout:8/624: dwrite d0/d11/f8f [0,4194304] 0 2024-02-28T23:07:41.822 INFO:tasks.workunit.client.0.smithi057.stdout:8/594: mknod d4/d1e/d37/cd6 0 2024-02-28T23:07:41.822 INFO:tasks.workunit.client.1.smithi084.stdout:0/648: mkdir d1/d6/db/d35/d44/d92/dcf 0 2024-02-28T23:07:41.824 INFO:tasks.workunit.client.1.smithi084.stdout:1/723: chown d12/d1b/d63/d7d/lca 290100 1 2024-02-28T23:07:41.824 INFO:tasks.workunit.client.1.smithi084.stdout:8/625: stat d0/d11/f23 0 2024-02-28T23:07:41.824 INFO:tasks.workunit.client.0.smithi057.stdout:2/626: truncate d8/d11/d15/fc4 661124 0 2024-02-28T23:07:41.826 INFO:tasks.workunit.client.1.smithi084.stdout:3/684: dread d3/d27/d7d/db0/d1a/d21/d3d/d43/f6c [0,4194304] 0 2024-02-28T23:07:41.829 INFO:tasks.workunit.client.1.smithi084.stdout:4/634: rmdir d1/d18/d20/d24/d2f/d3e/d52/d48/d73 39 2024-02-28T23:07:41.829 INFO:tasks.workunit.client.0.smithi057.stdout:6/575: creat d0/d17/d20/fc3 x:0 0 0 2024-02-28T23:07:41.829 INFO:tasks.workunit.client.0.smithi057.stdout:0/564: dwrite d17/d20/f51 [0,4194304] 0 2024-02-28T23:07:41.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:41 smithi084 ceph-mon[34826]: pgmap v27: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 75 MiB/s rd, 305 MiB/s wr, 525 op/s 2024-02-28T23:07:41.833 INFO:tasks.workunit.client.0.smithi057.stdout:8/595: write d4/d13/d1a/f20 [230783,103945] 0 2024-02-28T23:07:41.834 INFO:tasks.workunit.client.1.smithi084.stdout:0/649: mkdir d1/d8/da1/dd0 0 2024-02-28T23:07:41.835 INFO:tasks.workunit.client.0.smithi057.stdout:0/565: rmdir d17/d19 39 2024-02-28T23:07:41.835 INFO:tasks.workunit.client.1.smithi084.stdout:2/609: dread d5/d7/d2e/f3d [0,4194304] 0 2024-02-28T23:07:41.836 INFO:tasks.workunit.client.1.smithi084.stdout:1/724: symlink d12/d1b/d2e/d52/d61/lec 0 2024-02-28T23:07:41.836 INFO:tasks.workunit.client.1.smithi084.stdout:8/626: mknod d0/d9/d1b/d38/d43/d51/d54/cd3 0 2024-02-28T23:07:41.836 INFO:tasks.workunit.client.0.smithi057.stdout:6/576: unlink d0/d1b/d3e/d3d/f8a 0 2024-02-28T23:07:41.841 INFO:tasks.workunit.client.0.smithi057.stdout:0/566: readlink d17/d19/l1d 0 2024-02-28T23:07:41.844 INFO:tasks.workunit.client.0.smithi057.stdout:8/596: symlink d4/d1e/d59/dc3/ld7 0 2024-02-28T23:07:41.844 INFO:tasks.workunit.client.0.smithi057.stdout:3/433: dwrite d3/d4/d7/f2b [0,4194304] 0 2024-02-28T23:07:41.844 INFO:tasks.workunit.client.1.smithi084.stdout:8/627: chown d0/d9/d30 7752963 1 2024-02-28T23:07:41.844 INFO:tasks.workunit.client.1.smithi084.stdout:3/685: creat d3/d27/d7d/db0/d1a/d21/d7b/f105 x:0 0 0 2024-02-28T23:07:41.844 INFO:tasks.workunit.client.1.smithi084.stdout:0/650: symlink d1/d8/d69/d6b/d85/da5/ld1 0 2024-02-28T23:07:41.847 INFO:tasks.workunit.client.0.smithi057.stdout:6/577: creat d0/d1b/d3e/d3d/fc4 x:0 0 0 2024-02-28T23:07:41.851 INFO:tasks.workunit.client.0.smithi057.stdout:8/597: write d4/d13/d18/d62/d66/fbf [308917,25017] 0 2024-02-28T23:07:41.852 INFO:tasks.workunit.client.1.smithi084.stdout:5/700: dwrite dc/de/df/d31/dd1/f61 [0,4194304] 0 2024-02-28T23:07:41.854 INFO:tasks.workunit.client.1.smithi084.stdout:0/651: unlink d1/d8/l26 0 2024-02-28T23:07:41.855 INFO:tasks.workunit.client.0.smithi057.stdout:0/567: rename d17/d20/d26/d30/f39 to d17/d20/d4a/d78/db5/fcb 0 2024-02-28T23:07:41.855 INFO:tasks.workunit.client.0.smithi057.stdout:8/598: write d4/d13/d1a/f20 [1212425,67535] 0 2024-02-28T23:07:41.856 INFO:tasks.workunit.client.1.smithi084.stdout:8/628: mkdir d0/d9/d1b/d38/dd4 0 2024-02-28T23:07:41.857 INFO:tasks.workunit.client.0.smithi057.stdout:3/434: symlink d3/d4/d1d/d47/d69/l8d 0 2024-02-28T23:07:41.858 INFO:tasks.workunit.client.0.smithi057.stdout:3/435: fdatasync d3/d4/d7/f6c 0 2024-02-28T23:07:41.858 INFO:tasks.workunit.client.1.smithi084.stdout:0/652: chown d1/d8/f55 7470 1 2024-02-28T23:07:41.859 INFO:tasks.workunit.client.0.smithi057.stdout:3/436: chown d3/d8/c6d 5358 1 2024-02-28T23:07:41.859 INFO:tasks.workunit.client.0.smithi057.stdout:6/578: link d0/d17/d94/f9e d0/d17/d94/da0/fc5 0 2024-02-28T23:07:41.860 INFO:tasks.workunit.client.1.smithi084.stdout:3/686: creat d3/d27/d7d/db0/d1a/d45/d86/f106 x:0 0 0 2024-02-28T23:07:41.861 INFO:tasks.workunit.client.1.smithi084.stdout:1/725: dwrite f11 [0,4194304] 0 2024-02-28T23:07:41.863 INFO:tasks.workunit.client.0.smithi057.stdout:6/579: creat d0/d1b/d3e/d40/fc6 x:0 0 0 2024-02-28T23:07:41.864 INFO:tasks.workunit.client.1.smithi084.stdout:5/701: write dc/de/df/d16/f9a [2315446,42389] 0 2024-02-28T23:07:41.864 INFO:tasks.workunit.client.1.smithi084.stdout:3/687: dread - d3/d27/d7d/dd9/fba zero size 2024-02-28T23:07:41.866 INFO:tasks.workunit.client.1.smithi084.stdout:2/610: dread d5/d7/d9/f3c [0,4194304] 0 2024-02-28T23:07:41.874 INFO:tasks.workunit.client.0.smithi057.stdout:8/599: dwrite d4/d1e/d56/da6/faf [0,4194304] 0 2024-02-28T23:07:41.874 INFO:tasks.workunit.client.0.smithi057.stdout:1/440: write d2/d71/d2d/f31 [1098266,113675] 0 2024-02-28T23:07:41.879 INFO:tasks.workunit.client.1.smithi084.stdout:4/635: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [0,4194304] 0 2024-02-28T23:07:41.880 INFO:tasks.workunit.client.0.smithi057.stdout:1/441: fsync d2/d33/d54/f89 0 2024-02-28T23:07:41.880 INFO:tasks.workunit.client.1.smithi084.stdout:0/653: symlink d1/d8/ld2 0 2024-02-28T23:07:41.881 INFO:tasks.workunit.client.1.smithi084.stdout:6/627: sync 2024-02-28T23:07:41.882 INFO:tasks.workunit.client.1.smithi084.stdout:5/702: chown dc/l82 72046020 1 2024-02-28T23:07:41.884 INFO:tasks.workunit.client.0.smithi057.stdout:1/442: symlink d2/d71/d2d/la1 0 2024-02-28T23:07:41.884 INFO:tasks.workunit.client.0.smithi057.stdout:3/437: dwrite d3/d8/f5e [0,4194304] 0 2024-02-28T23:07:41.886 INFO:tasks.workunit.client.0.smithi057.stdout:8/600: truncate d4/d1e/d59/fa2 1683682 0 2024-02-28T23:07:41.886 INFO:tasks.workunit.client.1.smithi084.stdout:1/726: creat d12/d1b/d2e/d30/d77/da0/fed x:0 0 0 2024-02-28T23:07:41.886 INFO:tasks.workunit.client.1.smithi084.stdout:6/628: chown d1/d13/d50/dbd/cd4 1331 1 2024-02-28T23:07:41.887 INFO:tasks.workunit.client.0.smithi057.stdout:8/601: write d4/d13/d2c/f84 [733743,25108] 0 2024-02-28T23:07:41.898 INFO:tasks.workunit.client.0.smithi057.stdout:3/438: creat d3/f8e x:0 0 0 2024-02-28T23:07:41.899 INFO:tasks.workunit.client.0.smithi057.stdout:8/602: creat d4/d13/d18/d62/d66/fd8 x:0 0 0 2024-02-28T23:07:41.900 INFO:tasks.workunit.client.0.smithi057.stdout:2/627: truncate d8/dae/fb8 2711140 0 2024-02-28T23:07:41.904 INFO:tasks.workunit.client.1.smithi084.stdout:3/688: dwrite d3/d27/d7d/db0/d1a/d45/f78 [0,4194304] 0 2024-02-28T23:07:41.905 INFO:tasks.workunit.client.0.smithi057.stdout:0/568: truncate d17/d20/d4a/d78/fc7 884346 0 2024-02-28T23:07:41.907 INFO:tasks.workunit.client.0.smithi057.stdout:1/443: dwrite d2/d33/f76 [4194304,4194304] 0 2024-02-28T23:07:41.908 INFO:tasks.workunit.client.0.smithi057.stdout:8/603: rename d4/f9 to d4/d1e/d56/da6/fd9 0 2024-02-28T23:07:41.908 INFO:tasks.workunit.client.1.smithi084.stdout:3/689: write d3/d27/d7d/db0/d1a/d21/d3d/d43/f70 [413605,60648] 0 2024-02-28T23:07:41.916 INFO:tasks.workunit.client.1.smithi084.stdout:3/690: read d3/d27/d7d/db0/d14/f2e [128876,8723] 0 2024-02-28T23:07:41.917 INFO:tasks.workunit.client.0.smithi057.stdout:2/628: mknod d8/d11/d21/d28/d49/d85/d9b/dad/cc6 0 2024-02-28T23:07:41.917 INFO:tasks.workunit.client.0.smithi057.stdout:0/569: creat d17/d20/d26/d48/d55/d88/fcc x:0 0 0 2024-02-28T23:07:41.917 INFO:tasks.workunit.client.0.smithi057.stdout:1/444: mknod d2/d7/df/ca2 0 2024-02-28T23:07:41.919 INFO:tasks.workunit.client.0.smithi057.stdout:3/439: read d3/f2c [1642138,57091] 0 2024-02-28T23:07:41.920 INFO:tasks.workunit.client.0.smithi057.stdout:0/570: chown d17/d20/d26/d30/d6b/d71 11230 1 2024-02-28T23:07:41.920 INFO:tasks.workunit.client.0.smithi057.stdout:8/604: link d4/f11 d4/d1e/d56/d6a/d54/d6b/dca/fda 0 2024-02-28T23:07:41.925 INFO:tasks.workunit.client.0.smithi057.stdout:1/445: unlink d2/d7/df/db/d12/d3f/l97 0 2024-02-28T23:07:41.926 INFO:tasks.workunit.client.1.smithi084.stdout:0/654: creat d1/d8/d69/da3/fd3 x:0 0 0 2024-02-28T23:07:41.927 INFO:tasks.workunit.client.0.smithi057.stdout:3/440: write d3/d20/d2d/f82 [778278,100333] 0 2024-02-28T23:07:41.927 INFO:tasks.workunit.client.0.smithi057.stdout:8/605: truncate d4/d1e/f2e 3353410 0 2024-02-28T23:07:41.930 INFO:tasks.workunit.client.1.smithi084.stdout:5/703: mkdir dc/de/df/db0/d104 0 2024-02-28T23:07:41.930 INFO:tasks.workunit.client.0.smithi057.stdout:0/571: creat d17/d20/d23/fcd x:0 0 0 2024-02-28T23:07:41.931 INFO:tasks.workunit.client.1.smithi084.stdout:4/636: creat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/fd6 x:0 0 0 2024-02-28T23:07:41.933 INFO:tasks.workunit.client.0.smithi057.stdout:1/446: mkdir d2/d71/d2d/d39/da3 0 2024-02-28T23:07:41.936 INFO:tasks.workunit.client.1.smithi084.stdout:0/655: creat d1/d6/db/d37/dae/fd4 x:0 0 0 2024-02-28T23:07:41.939 INFO:tasks.workunit.client.0.smithi057.stdout:8/606: rename d4/d13/f2d to d4/d13/d1a/fdb 0 2024-02-28T23:07:41.939 INFO:tasks.workunit.client.1.smithi084.stdout:1/727: truncate d12/f70 860645 0 2024-02-28T23:07:41.939 INFO:tasks.workunit.client.1.smithi084.stdout:4/637: chown d1/d18/d20/d14/f4f 140816231 1 2024-02-28T23:07:41.940 INFO:tasks.workunit.client.0.smithi057.stdout:2/629: dwrite d8/d11/f32 [0,4194304] 0 2024-02-28T23:07:41.941 INFO:tasks.workunit.client.1.smithi084.stdout:0/656: truncate d1/d8/d69/db2/fbc 349253 0 2024-02-28T23:07:41.942 INFO:tasks.workunit.client.1.smithi084.stdout:6/629: creat d1/fdb x:0 0 0 2024-02-28T23:07:41.943 INFO:tasks.workunit.client.1.smithi084.stdout:5/704: mknod dc/de/df/d16/c105 0 2024-02-28T23:07:41.945 INFO:tasks.workunit.client.1.smithi084.stdout:6/630: stat d1/f24 0 2024-02-28T23:07:41.946 INFO:tasks.workunit.client.1.smithi084.stdout:0/657: creat d1/d8/d69/d6b/d85/fd5 x:0 0 0 2024-02-28T23:07:41.949 INFO:tasks.workunit.client.0.smithi057.stdout:8/607: rename d4/d13/d18/d62/dbb/f8a to d4/d13/d1a/d4a/fdc 0 2024-02-28T23:07:41.952 INFO:tasks.workunit.client.1.smithi084.stdout:1/728: rename d12/d1b/d2e/d30/fe8 to d12/d1b/d2e/d30/d50/da8/dd9/fee 0 2024-02-28T23:07:41.953 INFO:tasks.workunit.client.0.smithi057.stdout:3/441: dwrite d3/d8/d1b/f7e [4194304,4194304] 0 2024-02-28T23:07:41.955 INFO:tasks.workunit.client.1.smithi084.stdout:5/705: write dc/de/df/d31/fcb [360936,60868] 0 2024-02-28T23:07:41.959 INFO:tasks.workunit.client.1.smithi084.stdout:4/638: mkdir d1/d18/d20/d24/d2f/d3e/d52/d48/dd7 0 2024-02-28T23:07:41.959 INFO:tasks.workunit.client.0.smithi057.stdout:1/447: dwrite d2/d7/df/db/d3a/f99 [0,4194304] 0 2024-02-28T23:07:41.961 INFO:tasks.workunit.client.1.smithi084.stdout:0/658: creat d1/d6/db/d37/dae/fd6 x:0 0 0 2024-02-28T23:07:41.962 INFO:tasks.workunit.client.1.smithi084.stdout:7/702: sync 2024-02-28T23:07:41.963 INFO:tasks.workunit.client.1.smithi084.stdout:0/659: fdatasync d1/d8/d69/db2/f54 0 2024-02-28T23:07:41.964 INFO:tasks.workunit.client.0.smithi057.stdout:3/442: fdatasync d3/d8/f3c 0 2024-02-28T23:07:41.967 INFO:tasks.workunit.client.0.smithi057.stdout:1/448: dread d2/f44 [0,4194304] 0 2024-02-28T23:07:41.967 INFO:tasks.workunit.client.1.smithi084.stdout:8/629: write d0/d9/d30/db6/f89 [940606,114347] 0 2024-02-28T23:07:41.971 INFO:tasks.workunit.client.1.smithi084.stdout:5/706: mknod dc/de/df/d31/d8e/c106 0 2024-02-28T23:07:41.972 INFO:tasks.workunit.client.1.smithi084.stdout:8/630: write d0/d9/d1b/fc4 [523354,109128] 0 2024-02-28T23:07:41.973 INFO:tasks.workunit.client.1.smithi084.stdout:5/707: chown c9 3470 1 2024-02-28T23:07:41.978 INFO:tasks.workunit.client.1.smithi084.stdout:0/660: fsync d1/d6/db/d35/d44/f4b 0 2024-02-28T23:07:41.979 INFO:tasks.workunit.client.0.smithi057.stdout:0/572: dwrite d17/d19/d4d/f62 [0,4194304] 0 2024-02-28T23:07:41.980 INFO:tasks.workunit.client.0.smithi057.stdout:1/449: mkdir d2/d7/df/db/d12/d19/da4 0 2024-02-28T23:07:41.980 INFO:tasks.workunit.client.1.smithi084.stdout:4/639: symlink d1/d18/d20/d14/d30/ld8 0 2024-02-28T23:07:41.982 INFO:tasks.workunit.client.1.smithi084.stdout:6/631: rename d1/d13/d9d to d1/d9/ddc 0 2024-02-28T23:07:41.983 INFO:tasks.workunit.client.1.smithi084.stdout:7/703: symlink db/d5e/dc3/ld0 0 2024-02-28T23:07:41.984 INFO:tasks.workunit.client.0.smithi057.stdout:3/443: getdents d3/d4/d1d/d47/d69 0 2024-02-28T23:07:41.984 INFO:tasks.workunit.client.0.smithi057.stdout:1/450: truncate d2/d71/d62/f64 452286 0 2024-02-28T23:07:41.985 INFO:tasks.workunit.client.1.smithi084.stdout:8/631: creat d0/d24/d78/fd5 x:0 0 0 2024-02-28T23:07:41.986 INFO:tasks.workunit.client.1.smithi084.stdout:7/704: readlink db/d10/d17/d9c/db9/lc4 0 2024-02-28T23:07:41.990 INFO:tasks.workunit.client.0.smithi057.stdout:3/444: creat d3/d4/d7/f8f x:0 0 0 2024-02-28T23:07:41.991 INFO:tasks.workunit.client.1.smithi084.stdout:4/640: mkdir d1/dd9 0 2024-02-28T23:07:41.991 INFO:tasks.workunit.client.0.smithi057.stdout:1/451: mknod d2/d7/df/db/d12/d19/d80/ca5 0 2024-02-28T23:07:41.992 INFO:tasks.workunit.client.0.smithi057.stdout:3/445: dread - d3/d4/d1d/d47/d62/f8c zero size 2024-02-28T23:07:41.993 INFO:tasks.workunit.client.1.smithi084.stdout:7/705: mknod db/d5e/d80/cd1 0 2024-02-28T23:07:41.994 INFO:tasks.workunit.client.1.smithi084.stdout:4/641: write d1/f1d [737961,24100] 0 2024-02-28T23:07:41.994 INFO:tasks.workunit.client.1.smithi084.stdout:5/708: dwrite dc/de/df/d31/dd1/d5f/fe2 [0,4194304] 0 2024-02-28T23:07:41.995 INFO:tasks.workunit.client.0.smithi057.stdout:1/452: creat d2/d71/d62/fa6 x:0 0 0 2024-02-28T23:07:41.996 INFO:tasks.workunit.client.1.smithi084.stdout:6/632: creat d1/d13/d50/dbd/dc4/fdd x:0 0 0 2024-02-28T23:07:41.996 INFO:tasks.workunit.client.0.smithi057.stdout:3/446: write d3/d8/d27/f81 [793571,70699] 0 2024-02-28T23:07:41.999 INFO:tasks.workunit.client.0.smithi057.stdout:1/453: truncate d2/f72 1011989 0 2024-02-28T23:07:42.001 INFO:tasks.workunit.client.1.smithi084.stdout:8/632: rename d0/d9/d30/d33/d71/d84/fcc to d0/d9/d1b/d38/d43/dae/db8/fd6 0 2024-02-28T23:07:42.003 INFO:tasks.workunit.client.1.smithi084.stdout:9/718: sync 2024-02-28T23:07:42.004 INFO:tasks.workunit.client.1.smithi084.stdout:6/633: creat d1/d88/dbf/fde x:0 0 0 2024-02-28T23:07:42.005 INFO:tasks.workunit.client.1.smithi084.stdout:6/634: dread - d1/d88/dbf/fde zero size 2024-02-28T23:07:42.006 INFO:tasks.workunit.client.0.smithi057.stdout:1/454: mknod d2/d7/df/db/d3a/d5a/ca7 0 2024-02-28T23:07:42.007 INFO:tasks.workunit.client.0.smithi057.stdout:3/447: link d3/d4/f17 d3/d8/d27/f90 0 2024-02-28T23:07:42.008 INFO:tasks.workunit.client.1.smithi084.stdout:7/706: symlink db/dbd/ld2 0 2024-02-28T23:07:42.010 INFO:tasks.workunit.client.1.smithi084.stdout:8/633: creat d0/d9/d30/db6/fd7 x:0 0 0 2024-02-28T23:07:42.011 INFO:tasks.workunit.client.0.smithi057.stdout:3/448: chown d3/d8/d27/f5f 7 1 2024-02-28T23:07:42.013 INFO:tasks.workunit.client.0.smithi057.stdout:0/573: dwrite d17/d20/d26/d30/d6b/d71/f92 [0,4194304] 0 2024-02-28T23:07:42.017 INFO:tasks.workunit.client.0.smithi057.stdout:3/449: mkdir d3/d4f/d4c/d91 0 2024-02-28T23:07:42.017 INFO:tasks.workunit.client.1.smithi084.stdout:6/635: creat d1/d13/d50/fdf x:0 0 0 2024-02-28T23:07:42.021 INFO:tasks.workunit.client.1.smithi084.stdout:2/611: write d5/d1f/f44 [2513084,7776] 0 2024-02-28T23:07:42.024 INFO:tasks.workunit.client.1.smithi084.stdout:6/636: dread - d1/d13/d50/fc9 zero size 2024-02-28T23:07:42.027 INFO:tasks.workunit.client.0.smithi057.stdout:1/455: dwrite d2/d7/df/db/f55 [0,4194304] 0 2024-02-28T23:07:42.027 INFO:tasks.workunit.client.1.smithi084.stdout:8/634: dread d0/d9/d1b/f46 [0,4194304] 0 2024-02-28T23:07:42.030 INFO:tasks.workunit.client.1.smithi084.stdout:2/612: fdatasync d5/d1f/d42/d8f/fa8 0 2024-02-28T23:07:42.032 INFO:tasks.workunit.client.0.smithi057.stdout:0/574: mknod d17/d20/d26/d48/d55/d88/cce 0 2024-02-28T23:07:42.032 INFO:tasks.workunit.client.1.smithi084.stdout:6/637: readlink d1/d9/d1d/d1e/d33/d38/l3d 0 2024-02-28T23:07:42.034 INFO:tasks.workunit.client.1.smithi084.stdout:9/719: dwrite d4/d6/d15/f8e [0,4194304] 0 2024-02-28T23:07:42.034 INFO:tasks.workunit.client.1.smithi084.stdout:3/691: truncate d3/d27/d7d/f8f 28633 0 2024-02-28T23:07:42.037 INFO:tasks.workunit.client.0.smithi057.stdout:1/456: getdents d2/d7/df/db/d3a/d66 0 2024-02-28T23:07:42.042 INFO:tasks.workunit.client.1.smithi084.stdout:6/638: mknod d1/d13/d50/d74/ce0 0 2024-02-28T23:07:42.043 INFO:tasks.workunit.client.0.smithi057.stdout:3/450: dwrite d3/d4/d7/f84 [0,4194304] 0 2024-02-28T23:07:42.046 INFO:tasks.workunit.client.1.smithi084.stdout:2/613: symlink d5/d7/d9/d7c/d5b/d9f/lb5 0 2024-02-28T23:07:42.048 INFO:tasks.workunit.client.0.smithi057.stdout:3/451: chown d3/d4/f7a 1893 1 2024-02-28T23:07:42.050 INFO:tasks.workunit.client.1.smithi084.stdout:3/692: fsync d3/f7 0 2024-02-28T23:07:42.054 INFO:tasks.workunit.client.0.smithi057.stdout:3/452: symlink d3/d4/d1d/d47/l92 0 2024-02-28T23:07:42.054 INFO:tasks.workunit.client.1.smithi084.stdout:6/639: chown d1/d13/d2e/d41/c8e 6989 1 2024-02-28T23:07:42.061 INFO:tasks.workunit.client.0.smithi057.stdout:1/457: dwrite d2/f21 [0,4194304] 0 2024-02-28T23:07:42.065 INFO:tasks.workunit.client.1.smithi084.stdout:6/640: rename d1/d13/d47/d91 to d1/d13/d47/d91/de1 22 2024-02-28T23:07:42.068 INFO:tasks.workunit.client.1.smithi084.stdout:6/641: chown d1/d9/f4f 87337687 1 2024-02-28T23:07:42.069 INFO:tasks.workunit.client.0.smithi057.stdout:1/458: rename d2/d33/d54/c73 to d2/d7/df/db/ca8 0 2024-02-28T23:07:42.070 INFO:tasks.workunit.client.0.smithi057.stdout:1/459: truncate d2/f44 587328 0 2024-02-28T23:07:42.071 INFO:tasks.workunit.client.1.smithi084.stdout:7/707: dread db/d10/d17/d3b/f48 [0,4194304] 0 2024-02-28T23:07:42.073 INFO:tasks.workunit.client.1.smithi084.stdout:0/661: write d1/d8/d78/f9d [391387,92633] 0 2024-02-28T23:07:42.074 INFO:tasks.workunit.client.1.smithi084.stdout:2/614: link d5/d7/d12/d24/l81 d5/d1f/lb6 0 2024-02-28T23:07:42.074 INFO:tasks.workunit.client.0.smithi057.stdout:1/460: mkdir d2/d7/df/db/da9 0 2024-02-28T23:07:42.077 INFO:tasks.workunit.client.0.smithi057.stdout:1/461: chown d2/d7/df/db/d3a/d2a/l75 2420804 1 2024-02-28T23:07:42.077 INFO:tasks.workunit.client.0.smithi057.stdout:0/575: dread d17/d20/d26/f41 [0,4194304] 0 2024-02-28T23:07:42.079 INFO:tasks.workunit.client.0.smithi057.stdout:1/462: readlink d2/d7/df/db/d3a/d66/l95 0 2024-02-28T23:07:42.079 INFO:tasks.workunit.client.1.smithi084.stdout:7/708: chown db/d5e/d8a/la9 13622808 1 2024-02-28T23:07:42.082 INFO:tasks.workunit.client.1.smithi084.stdout:6/642: chown d1/d13/d2e/d41/d4d/d6c/c83 6212 1 2024-02-28T23:07:42.082 INFO:tasks.workunit.client.1.smithi084.stdout:2/615: fsync d5/d7/d9/d7c/d5b/f87 0 2024-02-28T23:07:42.082 INFO:tasks.workunit.client.0.smithi057.stdout:0/576: creat d17/d20/d26/d48/d7c/d98/fcf x:0 0 0 2024-02-28T23:07:42.083 INFO:tasks.workunit.client.1.smithi084.stdout:1/729: dwrite d12/d1b/d3b/fa1 [0,4194304] 0 2024-02-28T23:07:42.084 INFO:tasks.workunit.client.1.smithi084.stdout:6/643: dread - d1/d88/dbf/fde zero size 2024-02-28T23:07:42.084 INFO:tasks.workunit.client.1.smithi084.stdout:0/662: mkdir d1/d8/d69/da3/dd7 0 2024-02-28T23:07:42.085 INFO:tasks.workunit.client.1.smithi084.stdout:2/616: stat d5/d7/d2e/f9a 0 2024-02-28T23:07:42.086 INFO:tasks.workunit.client.0.smithi057.stdout:0/577: mkdir d17/d20/d26/d30/dd0 0 2024-02-28T23:07:42.089 INFO:tasks.workunit.client.1.smithi084.stdout:2/617: chown d5/dd/d69/c84 31182283 1 2024-02-28T23:07:42.092 INFO:tasks.workunit.client.1.smithi084.stdout:8/635: dread d0/d9/fb [4194304,4194304] 0 2024-02-28T23:07:42.092 INFO:tasks.workunit.client.0.smithi057.stdout:3/453: dread d3/d4f/d4c/f54 [0,4194304] 0 2024-02-28T23:07:42.095 INFO:tasks.workunit.client.1.smithi084.stdout:6/644: mkdir d1/d88/dbf/de2 0 2024-02-28T23:07:42.096 INFO:tasks.workunit.client.1.smithi084.stdout:8/636: fdatasync d0/d9/fb2 0 2024-02-28T23:07:42.096 INFO:tasks.workunit.client.0.smithi057.stdout:0/578: creat d17/d20/d4a/d78/d9d/fd1 x:0 0 0 2024-02-28T23:07:42.099 INFO:tasks.workunit.client.1.smithi084.stdout:1/730: truncate d12/d1b/d63/d7d/f95 560301 0 2024-02-28T23:07:42.101 INFO:tasks.workunit.client.0.smithi057.stdout:1/463: dwrite d2/d7/df/db/d3a/d2a/f83 [0,4194304] 0 2024-02-28T23:07:42.102 INFO:tasks.workunit.client.0.smithi057.stdout:0/579: unlink d17/d20/d26/d48/f84 0 2024-02-28T23:07:42.107 INFO:tasks.workunit.client.0.smithi057.stdout:0/580: fsync d17/d20/d23/d47/f5b 0 2024-02-28T23:07:42.109 INFO:tasks.workunit.client.1.smithi084.stdout:4/642: dwrite d1/d18/d20/d65/d8e/d55/f1a [0,4194304] 0 2024-02-28T23:07:42.121 INFO:tasks.workunit.client.1.smithi084.stdout:4/643: read - d1/d18/fcf zero size 2024-02-28T23:07:42.121 INFO:tasks.workunit.client.1.smithi084.stdout:2/618: rename d5/d7/d9/d7c/c91 to d5/d7/d12/d24/d3a/cb7 0 2024-02-28T23:07:42.121 INFO:tasks.workunit.client.1.smithi084.stdout:5/709: dwrite dc/de/df/d31/d8e/da9/fcc [0,4194304] 0 2024-02-28T23:07:42.123 INFO:tasks.workunit.client.1.smithi084.stdout:4/644: write d1/f9c [10594,74841] 0 2024-02-28T23:07:42.128 INFO:tasks.workunit.client.1.smithi084.stdout:1/731: mkdir d12/d1b/d2e/d30/d50/da8/dd9/def 0 2024-02-28T23:07:42.130 INFO:tasks.workunit.client.1.smithi084.stdout:0/663: write d1/d6/db/d35/d4a/fac [142017,92736] 0 2024-02-28T23:07:42.131 INFO:tasks.workunit.client.0.smithi057.stdout:0/581: write d17/d20/d23/d47/f5b [1850231,65052] 0 2024-02-28T23:07:42.136 INFO:tasks.workunit.client.1.smithi084.stdout:7/709: dread db/d10/d17/d64/f75 [0,4194304] 0 2024-02-28T23:07:42.136 INFO:tasks.workunit.client.0.smithi057.stdout:3/454: dwrite d3/d4/d7/f84 [0,4194304] 0 2024-02-28T23:07:42.138 INFO:tasks.workunit.client.1.smithi084.stdout:4/645: read - d1/d18/fcf zero size 2024-02-28T23:07:42.141 INFO:tasks.workunit.client.1.smithi084.stdout:8/637: dwrite d0/d11/fad [0,4194304] 0 2024-02-28T23:07:42.146 INFO:tasks.workunit.client.1.smithi084.stdout:0/664: chown d1/d6/db/d35/f8a 314 1 2024-02-28T23:07:42.146 INFO:tasks.workunit.client.0.smithi057.stdout:1/464: dwrite d2/d7/df/db/d3a/d2a/f83 [0,4194304] 0 2024-02-28T23:07:42.146 INFO:tasks.workunit.client.1.smithi084.stdout:9/720: dwrite d4/d32/f31 [0,4194304] 0 2024-02-28T23:07:42.147 INFO:tasks.workunit.client.1.smithi084.stdout:1/732: mkdir d12/d1b/d2e/d30/d50/da8/dd9/df0 0 2024-02-28T23:07:42.150 INFO:tasks.workunit.client.1.smithi084.stdout:3/693: dwrite d3/d50/f8c [0,4194304] 0 2024-02-28T23:07:42.155 INFO:tasks.workunit.client.1.smithi084.stdout:4/646: symlink d1/dd9/lda 0 2024-02-28T23:07:42.158 INFO:tasks.workunit.client.1.smithi084.stdout:7/710: chown db/d10/d17/d3b/f3f 28736808 1 2024-02-28T23:07:42.159 INFO:tasks.workunit.client.1.smithi084.stdout:2/619: link d5/dd/f1c d5/d7/d9/fb8 0 2024-02-28T23:07:42.164 INFO:tasks.workunit.client.1.smithi084.stdout:0/665: rmdir d1/d6/db/d37/dae 39 2024-02-28T23:07:42.164 INFO:tasks.workunit.client.1.smithi084.stdout:6/645: dwrite d1/d9/f4f [0,4194304] 0 2024-02-28T23:07:42.165 INFO:tasks.workunit.client.1.smithi084.stdout:3/694: write d3/d27/d4b/fd6 [523724,1638] 0 2024-02-28T23:07:42.170 INFO:tasks.workunit.client.0.smithi057.stdout:1/465: dwrite d2/d7/df/db/d3a/f3c [0,4194304] 0 2024-02-28T23:07:42.172 INFO:tasks.workunit.client.1.smithi084.stdout:7/711: write db/d10/d17/d9c/d9f/db1/d36/f4c [966852,61681] 0 2024-02-28T23:07:42.174 INFO:tasks.workunit.client.0.smithi057.stdout:0/582: mkdir d17/d19/dd2 0 2024-02-28T23:07:42.177 INFO:tasks.workunit.client.1.smithi084.stdout:7/712: stat db/d10/d17/d64/d77/f85 0 2024-02-28T23:07:42.178 INFO:tasks.workunit.client.1.smithi084.stdout:8/638: dwrite d0/f1a [0,4194304] 0 2024-02-28T23:07:42.180 INFO:tasks.workunit.client.0.smithi057.stdout:3/455: creat d3/d8/f93 x:0 0 0 2024-02-28T23:07:42.181 INFO:tasks.workunit.client.0.smithi057.stdout:1/466: chown d2/d7/df/c11 15982909 1 2024-02-28T23:07:42.181 INFO:tasks.workunit.client.1.smithi084.stdout:0/666: mkdir d1/d8/d18/d4c/d6c/d74/dd8 0 2024-02-28T23:07:42.188 INFO:tasks.workunit.client.1.smithi084.stdout:7/713: readlink db/d10/d17/d9c/d9f/db1/d36/l5c 0 2024-02-28T23:07:42.199 INFO:tasks.workunit.client.0.smithi057.stdout:1/467: getdents d2/d7/df/db/d3a/d2a 0 2024-02-28T23:07:42.203 INFO:tasks.workunit.client.1.smithi084.stdout:4/647: dwrite d1/d18/d20/d14/d30/f3d [0,4194304] 0 2024-02-28T23:07:42.205 INFO:tasks.workunit.client.0.smithi057.stdout:0/583: dwrite d17/d20/f66 [0,4194304] 0 2024-02-28T23:07:42.206 INFO:tasks.workunit.client.0.smithi057.stdout:1/468: rmdir d2/d7/d4f 39 2024-02-28T23:07:42.207 INFO:tasks.workunit.client.1.smithi084.stdout:5/710: dread dc/f6b [0,4194304] 0 2024-02-28T23:07:42.208 INFO:tasks.workunit.client.0.smithi057.stdout:1/469: read - d2/d90/f9c zero size 2024-02-28T23:07:42.208 INFO:tasks.workunit.client.1.smithi084.stdout:4/648: write d1/d18/d20/d24/fa2 [482448,33111] 0 2024-02-28T23:07:42.211 INFO:tasks.workunit.client.1.smithi084.stdout:4/649: fsync d1/d18/d20/d24/f8b 0 2024-02-28T23:07:42.212 INFO:tasks.workunit.client.0.smithi057.stdout:1/470: stat d2/d7/df/db/d3a/d2a/f6d 0 2024-02-28T23:07:42.216 INFO:tasks.workunit.client.0.smithi057.stdout:3/456: dwrite d3/d20/f24 [0,4194304] 0 2024-02-28T23:07:42.223 INFO:tasks.workunit.client.0.smithi057.stdout:1/471: mkdir d2/d33/daa 0 2024-02-28T23:07:42.226 INFO:tasks.workunit.client.1.smithi084.stdout:4/650: truncate d1/d4b/fa6 53003 0 2024-02-28T23:07:42.230 INFO:tasks.workunit.client.0.smithi057.stdout:0/584: rename d17/d20/d26/d48/d7c/d98/fc2 to d17/d20/d23/d47/d60/fd3 0 2024-02-28T23:07:42.236 INFO:tasks.workunit.client.0.smithi057.stdout:6/580: sync 2024-02-28T23:07:42.237 INFO:tasks.workunit.client.0.smithi057.stdout:7/505: sync 2024-02-28T23:07:42.237 INFO:tasks.workunit.client.0.smithi057.stdout:4/533: sync 2024-02-28T23:07:42.237 INFO:tasks.workunit.client.0.smithi057.stdout:8/608: sync 2024-02-28T23:07:42.240 INFO:tasks.workunit.client.0.smithi057.stdout:2/630: sync 2024-02-28T23:07:42.241 INFO:tasks.workunit.client.0.smithi057.stdout:9/552: sync 2024-02-28T23:07:42.242 INFO:tasks.workunit.client.0.smithi057.stdout:5/524: sync 2024-02-28T23:07:42.242 INFO:tasks.workunit.client.0.smithi057.stdout:1/472: chown d2/d7/df/db/d3a/c1f 17317837 1 2024-02-28T23:07:42.250 INFO:tasks.workunit.client.0.smithi057.stdout:0/585: mkdir d17/d20/d4a/d78/dd4 0 2024-02-28T23:07:42.258 INFO:tasks.workunit.client.0.smithi057.stdout:7/506: unlink d0/d5/d25/d3e/l96 0 2024-02-28T23:07:42.258 INFO:tasks.workunit.client.1.smithi084.stdout:1/733: creat d12/d1b/d63/d7d/ff1 x:0 0 0 2024-02-28T23:07:42.260 INFO:tasks.workunit.client.0.smithi057.stdout:7/507: chown d0/f10 2 1 2024-02-28T23:07:42.263 INFO:tasks.workunit.client.1.smithi084.stdout:9/721: link d4/d6/d15/d3c/l7f d4/d6/le9 0 2024-02-28T23:07:42.264 INFO:tasks.workunit.client.0.smithi057.stdout:9/553: fdatasync dd/de/d14/f17 0 2024-02-28T23:07:42.266 INFO:tasks.workunit.client.0.smithi057.stdout:1/473: symlink d2/d71/d2d/lab 0 2024-02-28T23:07:42.266 INFO:tasks.workunit.client.1.smithi084.stdout:6/646: creat d1/d13/d47/d91/dba/fe3 x:0 0 0 2024-02-28T23:07:42.268 INFO:tasks.workunit.client.1.smithi084.stdout:8/639: creat d0/d9/d1b/d38/d43/dae/db8/fd8 x:0 0 0 2024-02-28T23:07:42.270 INFO:tasks.workunit.client.1.smithi084.stdout:2/620: fsync d5/dd/f1c 0 2024-02-28T23:07:42.270 INFO:tasks.workunit.client.1.smithi084.stdout:6/647: dread - d1/d13/d50/d74/fb7 zero size 2024-02-28T23:07:42.274 INFO:tasks.workunit.client.1.smithi084.stdout:3/695: dread d3/fb [0,4194304] 0 2024-02-28T23:07:42.275 INFO:tasks.workunit.client.1.smithi084.stdout:0/667: fdatasync d1/d8/d69/f84 0 2024-02-28T23:07:42.275 INFO:tasks.workunit.client.1.smithi084.stdout:7/714: mknod db/d10/d17/d64/d77/cd3 0 2024-02-28T23:07:42.277 INFO:tasks.workunit.client.1.smithi084.stdout:2/621: dread d5/dd/f58 [0,4194304] 0 2024-02-28T23:07:42.278 INFO:tasks.workunit.client.0.smithi057.stdout:5/525: creat d4/d9/d14/d3b/d61/d71/fa7 x:0 0 0 2024-02-28T23:07:42.279 INFO:tasks.workunit.client.0.smithi057.stdout:0/586: fdatasync d17/d20/d23/d37/f43 0 2024-02-28T23:07:42.287 INFO:tasks.workunit.client.1.smithi084.stdout:4/651: truncate d1/d18/d20/d14/d30/d47/da4/faa 179871 0 2024-02-28T23:07:42.288 INFO:tasks.workunit.client.0.smithi057.stdout:0/587: fdatasync d17/d20/d26/d30/d6b/d71/fa1 0 2024-02-28T23:07:42.288 INFO:tasks.workunit.client.0.smithi057.stdout:6/581: symlink d0/lc7 0 2024-02-28T23:07:42.294 INFO:tasks.workunit.client.0.smithi057.stdout:0/588: truncate d17/d20/d4a/d78/faa 858476 0 2024-02-28T23:07:42.295 INFO:tasks.workunit.client.0.smithi057.stdout:8/609: rename d4/d13/d1a/d4a/cb8 to d4/d1e/cdd 0 2024-02-28T23:07:42.300 INFO:tasks.workunit.client.0.smithi057.stdout:8/610: fdatasync d4/d13/d1a/f94 0 2024-02-28T23:07:42.300 INFO:tasks.workunit.client.0.smithi057.stdout:5/526: dwrite d4/d9/d16/f18 [4194304,4194304] 0 2024-02-28T23:07:42.304 INFO:tasks.workunit.client.0.smithi057.stdout:5/527: readlink d4/d9/d14/d3b/l6b 0 2024-02-28T23:07:42.305 INFO:tasks.workunit.client.1.smithi084.stdout:7/715: mknod db/d10/d18/db7/dbb/cd4 0 2024-02-28T23:07:42.305 INFO:tasks.workunit.client.0.smithi057.stdout:9/554: dread dd/de/d14/f3d [0,4194304] 0 2024-02-28T23:07:42.308 INFO:tasks.workunit.client.0.smithi057.stdout:9/555: chown dd/d1b/d40/d44/f62 2263 1 2024-02-28T23:07:42.308 INFO:tasks.workunit.client.1.smithi084.stdout:2/622: truncate d5/d7/d9/f2c 2196644 0 2024-02-28T23:07:42.308 INFO:tasks.workunit.client.0.smithi057.stdout:9/556: readlink dd/de/d4a/d9f/la3 0 2024-02-28T23:07:42.311 INFO:tasks.workunit.client.0.smithi057.stdout:9/557: truncate dd/de/d4a/d60/f7e 1373878 0 2024-02-28T23:07:42.316 INFO:tasks.workunit.client.1.smithi084.stdout:0/668: dread d1/d8/d69/db2/f54 [0,4194304] 0 2024-02-28T23:07:42.317 INFO:tasks.workunit.client.0.smithi057.stdout:7/508: link d0/f85 d0/d5/d36/d4e/d3b/d75/fa5 0 2024-02-28T23:07:42.319 INFO:tasks.workunit.client.1.smithi084.stdout:0/669: fsync d1/d6/db/d37/f96 0 2024-02-28T23:07:42.320 INFO:tasks.workunit.client.0.smithi057.stdout:0/589: creat d17/d20/d26/d48/d7c/d98/da6/fd5 x:0 0 0 2024-02-28T23:07:42.324 INFO:tasks.workunit.client.1.smithi084.stdout:0/670: readlink d1/d8/d18/l67 0 2024-02-28T23:07:42.326 INFO:tasks.workunit.client.0.smithi057.stdout:8/611: creat d4/d13/d2c/db1/fde x:0 0 0 2024-02-28T23:07:42.328 INFO:tasks.workunit.client.1.smithi084.stdout:7/716: dwrite db/d10/d17/d64/fab [0,4194304] 0 2024-02-28T23:07:42.329 INFO:tasks.workunit.client.0.smithi057.stdout:5/528: rmdir d4/d9/d85 39 2024-02-28T23:07:42.330 INFO:tasks.workunit.client.1.smithi084.stdout:7/717: chown db/d5e/dc6/dc7 10578246 1 2024-02-28T23:07:42.332 INFO:tasks.workunit.client.1.smithi084.stdout:3/696: mknod d3/d27/c107 0 2024-02-28T23:07:42.333 INFO:tasks.workunit.client.0.smithi057.stdout:5/529: dread - d4/d7/f8d zero size 2024-02-28T23:07:42.334 INFO:tasks.workunit.client.1.smithi084.stdout:2/623: mknod d5/dd/d69/cb9 0 2024-02-28T23:07:42.336 INFO:tasks.workunit.client.0.smithi057.stdout:5/530: readlink d4/d21/l23 0 2024-02-28T23:07:42.336 INFO:tasks.workunit.client.1.smithi084.stdout:6/648: creat d1/d13/d50/d74/fe4 x:0 0 0 2024-02-28T23:07:42.338 INFO:tasks.workunit.client.1.smithi084.stdout:1/734: link d12/d1b/d2e/d52/d61/lec d12/d84/lf2 0 2024-02-28T23:07:42.339 INFO:tasks.workunit.client.0.smithi057.stdout:3/457: dwrite d3/d4f/f48 [0,4194304] 0 2024-02-28T23:07:42.340 INFO:tasks.workunit.client.1.smithi084.stdout:9/722: rmdir d4/d6/d15/db3/de7 0 2024-02-28T23:07:42.343 INFO:tasks.workunit.client.0.smithi057.stdout:3/458: dread - d3/d8/d1b/f4a zero size 2024-02-28T23:07:42.345 INFO:tasks.workunit.client.1.smithi084.stdout:7/718: dread db/d5e/f6f [0,4194304] 0 2024-02-28T23:07:42.345 INFO:tasks.workunit.client.0.smithi057.stdout:4/534: dwrite f15 [0,4194304] 0 2024-02-28T23:07:42.346 INFO:tasks.workunit.client.1.smithi084.stdout:4/652: mkdir d1/d18/d20/d24/ddb 0 2024-02-28T23:07:42.355 INFO:tasks.workunit.client.1.smithi084.stdout:3/697: creat d3/d27/d7d/db0/d1a/d21/f108 x:0 0 0 2024-02-28T23:07:42.357 INFO:tasks.workunit.client.0.smithi057.stdout:2/631: dwrite d8/d11/d56/f7c [0,4194304] 0 2024-02-28T23:07:42.361 INFO:tasks.workunit.client.0.smithi057.stdout:5/531: creat d4/d9/d14/d3b/d61/fa8 x:0 0 0 2024-02-28T23:07:42.365 INFO:tasks.workunit.client.1.smithi084.stdout:3/698: dread - d3/d27/d7d/db0/d1a/d29/d41/fdf zero size 2024-02-28T23:07:42.367 INFO:tasks.workunit.client.1.smithi084.stdout:2/624: dwrite d5/d7/d2e/f9a [0,4194304] 0 2024-02-28T23:07:42.368 INFO:tasks.workunit.client.0.smithi057.stdout:8/612: dwrite d4/d13/d2c/fb9 [0,4194304] 0 2024-02-28T23:07:42.370 INFO:tasks.workunit.client.1.smithi084.stdout:1/735: dwrite d12/d1b/d2e/d30/f3e [0,4194304] 0 2024-02-28T23:07:42.375 INFO:tasks.workunit.client.0.smithi057.stdout:8/613: dread - d4/d1e/d59/f5f zero size 2024-02-28T23:07:42.377 INFO:tasks.workunit.client.0.smithi057.stdout:6/582: dwrite d0/d17/d94/f9e [0,4194304] 0 2024-02-28T23:07:42.381 INFO:tasks.workunit.client.1.smithi084.stdout:7/719: creat db/d5e/dc6/dc7/fd5 x:0 0 0 2024-02-28T23:07:42.381 INFO:tasks.workunit.client.0.smithi057.stdout:6/583: write d0/d4/d11/f14 [4958952,105462] 0 2024-02-28T23:07:42.385 INFO:tasks.workunit.client.1.smithi084.stdout:0/671: truncate d1/d6/db/d37/f71 251449 0 2024-02-28T23:07:42.388 INFO:tasks.workunit.client.0.smithi057.stdout:4/535: link d1a/d48/d9c/d2e/f7d d1a/d48/d97/d7f/fb8 0 2024-02-28T23:07:42.390 INFO:tasks.workunit.client.0.smithi057.stdout:0/590: dread d17/d20/d26/d30/d6b/d71/f7e [0,4194304] 0 2024-02-28T23:07:42.390 INFO:tasks.workunit.client.0.smithi057.stdout:3/459: creat d3/d4/f94 x:0 0 0 2024-02-28T23:07:42.390 INFO:tasks.workunit.client.1.smithi084.stdout:2/625: mknod d5/d1f/cba 0 2024-02-28T23:07:42.392 INFO:tasks.workunit.client.1.smithi084.stdout:1/736: readlink d12/d1b/d2e/d52/d61/lec 0 2024-02-28T23:07:42.394 INFO:tasks.workunit.client.1.smithi084.stdout:0/672: dread d1/d8/d69/db2/fbc [0,4194304] 0 2024-02-28T23:07:42.394 INFO:tasks.workunit.client.0.smithi057.stdout:9/558: write dd/de/d14/d48/d4b/d41/f70 [2073209,20169] 0 2024-02-28T23:07:42.395 INFO:tasks.workunit.client.1.smithi084.stdout:2/626: write d5/f17 [3270471,38833] 0 2024-02-28T23:07:42.397 INFO:tasks.workunit.client.0.smithi057.stdout:2/632: dread d8/d11/d21/d70/fc1 [0,4194304] 0 2024-02-28T23:07:42.399 INFO:tasks.workunit.client.0.smithi057.stdout:8/614: dwrite d4/d13/d2c/fb9 [0,4194304] 0 2024-02-28T23:07:42.403 INFO:tasks.workunit.client.0.smithi057.stdout:6/584: symlink d0/d1b/d54/lc8 0 2024-02-28T23:07:42.405 INFO:tasks.workunit.client.1.smithi084.stdout:3/699: dread d3/d27/d7d/db0/d1a/d29/d41/f71 [0,4194304] 0 2024-02-28T23:07:42.407 INFO:tasks.workunit.client.1.smithi084.stdout:9/723: getdents d4/d69 0 2024-02-28T23:07:42.409 INFO:tasks.workunit.client.0.smithi057.stdout:3/460: symlink d3/d20/l95 0 2024-02-28T23:07:42.410 INFO:tasks.workunit.client.1.smithi084.stdout:3/700: chown d3/d27/d7d/db0/d1a/d45/d86/f106 4134 1 2024-02-28T23:07:42.411 INFO:tasks.workunit.client.0.smithi057.stdout:4/536: symlink d1a/d48/db0/d4f/lb9 0 2024-02-28T23:07:42.412 INFO:tasks.workunit.client.0.smithi057.stdout:3/461: write d3/f64 [832865,59045] 0 2024-02-28T23:07:42.413 INFO:tasks.workunit.client.1.smithi084.stdout:0/673: creat d1/d8/d69/da3/fd9 x:0 0 0 2024-02-28T23:07:42.417 INFO:tasks.workunit.client.0.smithi057.stdout:7/509: truncate d0/d5/d36/d4e/d15/d29/d3d/f6e 438446 0 2024-02-28T23:07:42.422 INFO:tasks.workunit.client.0.smithi057.stdout:6/585: dwrite d0/d17/d94/da0/fc5 [0,4194304] 0 2024-02-28T23:07:42.424 INFO:tasks.workunit.client.0.smithi057.stdout:2/633: unlink d8/c2c 0 2024-02-28T23:07:42.428 INFO:tasks.workunit.client.0.smithi057.stdout:8/615: rename d4/d13/d7f/fa1 to d4/d1e/d56/d6a/d54/d61/fdf 0 2024-02-28T23:07:42.428 INFO:tasks.workunit.client.1.smithi084.stdout:5/711: sync 2024-02-28T23:07:42.431 INFO:tasks.workunit.client.1.smithi084.stdout:2/627: creat d5/d7/d9/d7c/d5b/da6/fbb x:0 0 0 2024-02-28T23:07:42.435 INFO:tasks.workunit.client.1.smithi084.stdout:4/653: link d1/d18/d20/d14/d30/l7c d1/d18/d20/d24/ldc 0 2024-02-28T23:07:42.435 INFO:tasks.workunit.client.0.smithi057.stdout:3/462: dread d3/d8/d27/f81 [0,4194304] 0 2024-02-28T23:07:42.436 INFO:tasks.workunit.client.1.smithi084.stdout:2/628: write d5/d1f/d42/d74/fa1 [1028000,33568] 0 2024-02-28T23:07:42.438 INFO:tasks.workunit.client.0.smithi057.stdout:0/591: link d17/d19/d4d/d7b/f94 d17/d20/d4a/d78/d9d/fd6 0 2024-02-28T23:07:42.441 INFO:tasks.workunit.client.0.smithi057.stdout:2/634: symlink d8/dae/lc7 0 2024-02-28T23:07:42.443 INFO:tasks.workunit.client.1.smithi084.stdout:8/640: write d0/d24/d4d/f81 [1452442,17963] 0 2024-02-28T23:07:42.445 INFO:tasks.workunit.client.0.smithi057.stdout:7/510: dwrite d0/f50 [0,4194304] 0 2024-02-28T23:07:42.446 INFO:tasks.workunit.client.1.smithi084.stdout:3/701: dwrite f1 [4194304,4194304] 0 2024-02-28T23:07:42.446 INFO:tasks.workunit.client.1.smithi084.stdout:5/712: write dc/de/df/d16/d34/daa/dae/fc5 [1041639,16670] 0 2024-02-28T23:07:42.449 INFO:tasks.workunit.client.0.smithi057.stdout:4/537: dwrite d1a/d48/d9c/d2e/fa2 [0,4194304] 0 2024-02-28T23:07:42.450 INFO:tasks.workunit.client.1.smithi084.stdout:4/654: truncate d1/d18/d20/d24/d2f/d3e/d52/f8c 760037 0 2024-02-28T23:07:42.454 INFO:tasks.workunit.client.0.smithi057.stdout:3/463: truncate d3/d8/d27/f55 515204 0 2024-02-28T23:07:42.455 INFO:tasks.workunit.client.1.smithi084.stdout:4/655: read d1/d4b/fca [2291747,93060] 0 2024-02-28T23:07:42.455 INFO:tasks.workunit.client.1.smithi084.stdout:8/641: mkdir d0/d9/d30/d33/d71/d84/dd9 0 2024-02-28T23:07:42.459 INFO:tasks.workunit.client.0.smithi057.stdout:9/559: dread dd/de/d4a/d60/f7e [0,4194304] 0 2024-02-28T23:07:42.464 INFO:tasks.workunit.client.1.smithi084.stdout:2/629: dwrite d5/f31 [0,4194304] 0 2024-02-28T23:07:42.464 INFO:tasks.workunit.client.1.smithi084.stdout:1/737: mkdir d12/d1b/d2e/df3 0 2024-02-28T23:07:42.466 INFO:tasks.workunit.client.1.smithi084.stdout:5/713: symlink dc/de/df/d16/d34/l107 0 2024-02-28T23:07:42.468 INFO:tasks.workunit.client.0.smithi057.stdout:0/592: mknod d17/d20/d4a/d78/db5/cd7 0 2024-02-28T23:07:42.469 INFO:tasks.workunit.client.0.smithi057.stdout:4/538: dwrite d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:42.472 INFO:tasks.workunit.client.0.smithi057.stdout:6/586: mkdir d0/d4/d25/d60/dc9 0 2024-02-28T23:07:42.473 INFO:tasks.workunit.client.1.smithi084.stdout:3/702: mkdir d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/dce/d109 0 2024-02-28T23:07:42.478 INFO:tasks.workunit.client.0.smithi057.stdout:1/474: sync 2024-02-28T23:07:42.483 INFO:tasks.workunit.client.1.smithi084.stdout:8/642: dwrite d0/d9/d1b/f1d [0,4194304] 0 2024-02-28T23:07:42.486 INFO:tasks.workunit.client.1.smithi084.stdout:6/649: creat d1/d9/d1d/fe5 x:0 0 0 2024-02-28T23:07:42.488 INFO:tasks.workunit.client.1.smithi084.stdout:4/656: dwrite d1/d18/d20/d24/d2f/d3e/d52/f9a [0,4194304] 0 2024-02-28T23:07:42.488 INFO:tasks.workunit.client.1.smithi084.stdout:3/703: creat d3/d27/d7d/db0/f10a x:0 0 0 2024-02-28T23:07:42.491 INFO:tasks.workunit.client.1.smithi084.stdout:8/643: stat d0/d9/d1b/d38/d43/f96 0 2024-02-28T23:07:42.492 INFO:tasks.workunit.client.0.smithi057.stdout:3/464: dread - d3/d4/f66 zero size 2024-02-28T23:07:42.493 INFO:tasks.workunit.client.1.smithi084.stdout:3/704: chown d3/d50/f8c 226912 1 2024-02-28T23:07:42.493 INFO:tasks.workunit.client.1.smithi084.stdout:2/630: dread d5/d7/d12/d24/d2a/d35/f40 [0,4194304] 0 2024-02-28T23:07:42.495 INFO:tasks.workunit.client.1.smithi084.stdout:1/738: mknod d12/d1b/d2e/d30/cf4 0 2024-02-28T23:07:42.498 INFO:tasks.workunit.client.0.smithi057.stdout:0/593: creat d17/d20/d23/d47/fd8 x:0 0 0 2024-02-28T23:07:42.498 INFO:tasks.workunit.client.1.smithi084.stdout:4/657: unlink d1/d18/d20/c2b 0 2024-02-28T23:07:42.499 INFO:tasks.workunit.client.0.smithi057.stdout:1/475: dwrite d2/d7/df/db/d12/d19/f23 [4194304,4194304] 0 2024-02-28T23:07:42.500 INFO:tasks.workunit.client.0.smithi057.stdout:8/616: link d4/d13/d18/c19 d4/d1e/ce0 0 2024-02-28T23:07:42.504 INFO:tasks.workunit.client.0.smithi057.stdout:0/594: read - d17/d20/d23/fcd zero size 2024-02-28T23:07:42.505 INFO:tasks.workunit.client.1.smithi084.stdout:5/714: dwrite dc/de/df/d31/fd7 [0,4194304] 0 2024-02-28T23:07:42.505 INFO:tasks.workunit.client.0.smithi057.stdout:9/560: mknod dd/d46/d6f/cb7 0 2024-02-28T23:07:42.511 INFO:tasks.workunit.client.0.smithi057.stdout:7/511: dread d0/d5/d25/d3e/f54 [0,4194304] 0 2024-02-28T23:07:42.514 INFO:tasks.workunit.client.1.smithi084.stdout:7/720: write db/d10/d17/d3b/f3c [20475,82979] 0 2024-02-28T23:07:42.516 INFO:tasks.workunit.client.1.smithi084.stdout:6/650: rename d1/d88/dbf/fb3 to d1/fe6 0 2024-02-28T23:07:42.519 INFO:tasks.workunit.client.1.smithi084.stdout:6/651: chown d1/d9/l90 0 1 2024-02-28T23:07:42.521 INFO:tasks.workunit.client.1.smithi084.stdout:1/739: truncate d12/d1b/d3b/f7f 3700457 0 2024-02-28T23:07:42.528 INFO:tasks.workunit.client.1.smithi084.stdout:0/674: write d1/d8/d18/f9f [447881,42771] 0 2024-02-28T23:07:42.530 INFO:tasks.workunit.client.1.smithi084.stdout:5/715: symlink dc/ddb/l108 0 2024-02-28T23:07:42.530 INFO:tasks.workunit.client.1.smithi084.stdout:6/652: creat d1/d13/d2e/d41/d4d/d6c/fe7 x:0 0 0 2024-02-28T23:07:42.531 INFO:tasks.workunit.client.0.smithi057.stdout:0/595: dread d17/d20/f66 [0,4194304] 0 2024-02-28T23:07:42.531 INFO:tasks.workunit.client.0.smithi057.stdout:9/561: chown dd/de/l5d 371358 1 2024-02-28T23:07:42.531 INFO:tasks.workunit.client.0.smithi057.stdout:7/512: rename d0/d5/d4a/c68 to d0/d5/ca6 0 2024-02-28T23:07:42.531 INFO:tasks.workunit.client.0.smithi057.stdout:7/513: fsync d0/f23 0 2024-02-28T23:07:42.531 INFO:tasks.workunit.client.0.smithi057.stdout:9/562: write dd/d1b/d40/fb2 [569375,9944] 0 2024-02-28T23:07:42.533 INFO:tasks.workunit.client.0.smithi057.stdout:0/596: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:42.534 INFO:tasks.workunit.client.1.smithi084.stdout:3/705: getdents d3/d27/d7d/db0/d1a/d45 0 2024-02-28T23:07:42.535 INFO:tasks.workunit.client.0.smithi057.stdout:0/597: fsync d17/d20/d23/d37/f74 0 2024-02-28T23:07:42.540 INFO:tasks.workunit.client.1.smithi084.stdout:1/740: creat d12/d1b/d2e/d30/d50/da8/dd9/df0/ff5 x:0 0 0 2024-02-28T23:07:42.540 INFO:tasks.workunit.client.0.smithi057.stdout:0/598: chown d17/d19/d4d/d7b/f94 5312 1 2024-02-28T23:07:42.540 INFO:tasks.workunit.client.0.smithi057.stdout:6/587: dread d0/d17/d20/f31 [0,4194304] 0 2024-02-28T23:07:42.543 INFO:tasks.workunit.client.1.smithi084.stdout:0/675: chown d1/d8/f40 89 1 2024-02-28T23:07:42.543 INFO:tasks.workunit.client.0.smithi057.stdout:8/617: mkdir d4/d13/d18/da7/de1 0 2024-02-28T23:07:42.546 INFO:tasks.workunit.client.1.smithi084.stdout:3/706: chown d3/d27/d7d/db0/d14/f58 3373981 1 2024-02-28T23:07:42.549 INFO:tasks.workunit.client.1.smithi084.stdout:7/721: dwrite db/d10/d17/d9c/fa6 [0,4194304] 0 2024-02-28T23:07:42.549 INFO:tasks.workunit.client.1.smithi084.stdout:2/631: dread d5/f17 [0,4194304] 0 2024-02-28T23:07:42.551 INFO:tasks.workunit.client.1.smithi084.stdout:9/724: dwrite d4/d6/d53/f7d [0,4194304] 0 2024-02-28T23:07:42.552 INFO:tasks.workunit.client.1.smithi084.stdout:6/653: rename d1/d13/d2e/d41/d2c/c6f to d1/d13/d50/dbd/dd1/ce8 0 2024-02-28T23:07:42.555 INFO:tasks.workunit.client.1.smithi084.stdout:4/658: truncate d1/f4 4734337 0 2024-02-28T23:07:42.557 INFO:tasks.workunit.client.0.smithi057.stdout:0/599: dwrite d17/d20/d26/f46 [0,4194304] 0 2024-02-28T23:07:42.558 INFO:tasks.workunit.client.1.smithi084.stdout:7/722: write db/f98 [421375,27659] 0 2024-02-28T23:07:42.561 INFO:tasks.workunit.client.1.smithi084.stdout:9/725: fsync d4/d6/d15/d12/f3d 0 2024-02-28T23:07:42.563 INFO:tasks.workunit.client.0.smithi057.stdout:9/563: dread - dd/de/d24/d72/f7c zero size 2024-02-28T23:07:42.563 INFO:tasks.workunit.client.0.smithi057.stdout:7/514: fdatasync d0/d5/d36/d4e/d3b/f9e 0 2024-02-28T23:07:42.564 INFO:tasks.workunit.client.1.smithi084.stdout:2/632: truncate d5/d7/d12/d24/d2a/d35/f8e 475974 0 2024-02-28T23:07:42.564 INFO:tasks.workunit.client.1.smithi084.stdout:5/716: creat dc/de/df/d16/d34/d3c/d98/f109 x:0 0 0 2024-02-28T23:07:42.564 INFO:tasks.workunit.client.0.smithi057.stdout:2/635: write d8/d11/d21/d28/f82 [2792320,78340] 0 2024-02-28T23:07:42.568 INFO:tasks.workunit.client.1.smithi084.stdout:9/726: chown d4/d6/d15/d75/d9d/la7 0 1 2024-02-28T23:07:42.568 INFO:tasks.workunit.client.1.smithi084.stdout:8/644: dread d0/d9/d30/d33/d71/d84/f8d [0,4194304] 0 2024-02-28T23:07:42.571 INFO:tasks.workunit.client.1.smithi084.stdout:3/707: symlink d3/d27/d7d/db0/d1a/d45/d5e/l10b 0 2024-02-28T23:07:42.580 INFO:tasks.workunit.client.1.smithi084.stdout:1/741: mkdir d12/d1b/d91/d9b/da5/dcb/dd4/df6 0 2024-02-28T23:07:42.582 INFO:tasks.workunit.client.0.smithi057.stdout:0/600: dread d17/d20/d26/f69 [0,4194304] 0 2024-02-28T23:07:42.583 INFO:tasks.workunit.client.1.smithi084.stdout:6/654: mkdir d1/d13/d47/d91/dba/de9 0 2024-02-28T23:07:42.584 INFO:tasks.workunit.client.0.smithi057.stdout:2/636: symlink d8/d11/d39/lc8 0 2024-02-28T23:07:42.585 INFO:tasks.workunit.client.1.smithi084.stdout:9/727: mkdir d4/d6/d15/d12/d41/dea 0 2024-02-28T23:07:42.588 INFO:tasks.workunit.client.1.smithi084.stdout:6/655: chown d1/d13/d2e/d41/d4d/d6c/fe7 20323 1 2024-02-28T23:07:42.589 INFO:tasks.workunit.client.0.smithi057.stdout:3/465: dread d3/d8/d27/f55 [0,4194304] 0 2024-02-28T23:07:42.591 INFO:tasks.workunit.client.1.smithi084.stdout:3/708: symlink d3/d27/d7d/db0/d1a/d21/d3d/l10c 0 2024-02-28T23:07:42.591 INFO:tasks.workunit.client.1.smithi084.stdout:0/676: link d1/d8/d18/f9a d1/d6/db/d35/d82/fda 0 2024-02-28T23:07:42.592 INFO:tasks.workunit.client.1.smithi084.stdout:7/723: symlink db/d5e/ld6 0 2024-02-28T23:07:42.594 INFO:tasks.workunit.client.1.smithi084.stdout:1/742: creat d12/d1b/d2e/d52/d61/d74/ff7 x:0 0 0 2024-02-28T23:07:42.594 INFO:tasks.workunit.client.1.smithi084.stdout:5/717: truncate dc/de/df/d16/d34/de8/fe4 744134 0 2024-02-28T23:07:42.595 INFO:tasks.workunit.client.1.smithi084.stdout:0/677: chown d1/d6/db/d35/d4a 1 1 2024-02-28T23:07:42.598 INFO:tasks.workunit.client.1.smithi084.stdout:8/645: creat d0/d9/d30/d33/d71/d84/dce/fda x:0 0 0 2024-02-28T23:07:42.598 INFO:tasks.workunit.client.0.smithi057.stdout:1/476: write d2/f87 [789770,100834] 0 2024-02-28T23:07:42.598 INFO:tasks.workunit.client.0.smithi057.stdout:3/466: dread - d3/d4/d7/f88 zero size 2024-02-28T23:07:42.599 INFO:tasks.workunit.client.1.smithi084.stdout:6/656: symlink d1/d13/d50/dbd/dd1/lea 0 2024-02-28T23:07:42.602 INFO:tasks.workunit.client.0.smithi057.stdout:0/601: dread d17/d20/d26/d30/d6b/d71/f7e [0,4194304] 0 2024-02-28T23:07:42.603 INFO:tasks.workunit.client.0.smithi057.stdout:1/477: readlink d2/d7/df/db/d12/d3f/l4e 0 2024-02-28T23:07:42.604 INFO:tasks.workunit.client.1.smithi084.stdout:7/724: symlink db/d5e/dc3/ld7 0 2024-02-28T23:07:42.605 INFO:tasks.workunit.client.1.smithi084.stdout:9/728: read d4/f79 [43051,43833] 0 2024-02-28T23:07:42.605 INFO:tasks.workunit.client.0.smithi057.stdout:1/478: readlink d2/d7/df/l58 0 2024-02-28T23:07:42.605 INFO:tasks.workunit.client.0.smithi057.stdout:3/467: unlink d3/d20/d2d/f56 0 2024-02-28T23:07:42.605 INFO:tasks.workunit.client.0.smithi057.stdout:7/515: dread d0/d5/d36/d4e/d15/d29/d3d/f6e [0,4194304] 0 2024-02-28T23:07:42.606 INFO:tasks.workunit.client.1.smithi084.stdout:1/743: creat d12/d1b/d91/d9b/ff8 x:0 0 0 2024-02-28T23:07:42.607 INFO:tasks.workunit.client.1.smithi084.stdout:4/659: write d1/f6 [2547592,6228] 0 2024-02-28T23:07:42.610 INFO:tasks.workunit.client.1.smithi084.stdout:9/729: write d4/d6/d53/fc0 [859715,75292] 0 2024-02-28T23:07:42.614 INFO:tasks.workunit.client.0.smithi057.stdout:1/479: mknod d2/d71/d2d/d39/cac 0 2024-02-28T23:07:42.615 INFO:tasks.workunit.client.0.smithi057.stdout:7/516: mknod d0/d5/d36/d4e/d15/d29/d71/ca7 0 2024-02-28T23:07:42.615 INFO:tasks.workunit.client.0.smithi057.stdout:3/468: rename d3/d20/d2d/f85 to d3/d4/d7/f96 0 2024-02-28T23:07:42.615 INFO:tasks.workunit.client.0.smithi057.stdout:9/564: dwrite dd/de/d14/f21 [4194304,4194304] 0 2024-02-28T23:07:42.616 INFO:tasks.workunit.client.1.smithi084.stdout:0/678: creat d1/d8/d18/d4c/d6c/d74/fdb x:0 0 0 2024-02-28T23:07:42.616 INFO:tasks.workunit.client.1.smithi084.stdout:9/730: read - d4/fdc zero size 2024-02-28T23:07:42.616 INFO:tasks.workunit.client.1.smithi084.stdout:6/657: creat d1/d13/d2e/feb x:0 0 0 2024-02-28T23:07:42.617 INFO:tasks.workunit.client.1.smithi084.stdout:4/660: dread d1/d18/d20/d24/fa2 [0,4194304] 0 2024-02-28T23:07:42.618 INFO:tasks.workunit.client.1.smithi084.stdout:8/646: truncate d0/d11/f99 568756 0 2024-02-28T23:07:42.619 INFO:tasks.workunit.client.0.smithi057.stdout:3/469: fsync d3/d8/d1b/f4a 0 2024-02-28T23:07:42.621 INFO:tasks.workunit.client.1.smithi084.stdout:3/709: dwrite d3/d27/d7d/db0/d14/f58 [4194304,4194304] 0 2024-02-28T23:07:42.621 INFO:tasks.workunit.client.1.smithi084.stdout:7/725: creat db/d10/d18/fd8 x:0 0 0 2024-02-28T23:07:42.621 INFO:tasks.workunit.client.1.smithi084.stdout:8/647: readlink d0/d9/d30/la7 0 2024-02-28T23:07:42.621 INFO:tasks.workunit.client.1.smithi084.stdout:6/658: fdatasync d1/d13/d50/d74/fe4 0 2024-02-28T23:07:42.622 INFO:tasks.workunit.client.1.smithi084.stdout:2/633: link d5/d7/d75/l90 d5/d7/d9/lbc 0 2024-02-28T23:07:42.623 INFO:tasks.workunit.client.0.smithi057.stdout:7/517: creat d0/d5/d36/d4e/d15/d29/d71/fa8 x:0 0 0 2024-02-28T23:07:42.624 INFO:tasks.workunit.client.1.smithi084.stdout:1/744: mkdir d12/d1b/d91/df9 0 2024-02-28T23:07:42.625 INFO:tasks.workunit.client.0.smithi057.stdout:0/602: dwrite d17/d20/d26/d48/d55/fbe [0,4194304] 0 2024-02-28T23:07:42.626 INFO:tasks.workunit.client.0.smithi057.stdout:2/637: dread d8/fb [0,4194304] 0 2024-02-28T23:07:42.627 INFO:tasks.workunit.client.0.smithi057.stdout:3/470: mknod d3/d20/d2d/c97 0 2024-02-28T23:07:42.628 INFO:tasks.workunit.client.1.smithi084.stdout:8/648: truncate d0/d44/d9f/fd1 7525 0 2024-02-28T23:07:42.628 INFO:tasks.workunit.client.1.smithi084.stdout:5/718: rename dc/de/df/d16/d34/d3c/l58 to dc/de/df/d16/d34/daa/dae/l10a 0 2024-02-28T23:07:42.628 INFO:tasks.workunit.client.1.smithi084.stdout:8/649: chown d0/d9/f88 775263 1 2024-02-28T23:07:42.632 INFO:tasks.workunit.client.0.smithi057.stdout:0/603: chown d17/d20/d26/d30/dd0 311 1 2024-02-28T23:07:42.632 INFO:tasks.workunit.client.0.smithi057.stdout:2/638: write d8/d11/d21/f86 [536288,107491] 0 2024-02-28T23:07:42.632 INFO:tasks.workunit.client.0.smithi057.stdout:9/565: mknod dd/d46/cb8 0 2024-02-28T23:07:42.634 INFO:tasks.workunit.client.1.smithi084.stdout:7/726: fsync db/d5e/dc6/f4f 0 2024-02-28T23:07:42.637 INFO:tasks.workunit.client.1.smithi084.stdout:7/727: write db/d10/f56 [1649319,75724] 0 2024-02-28T23:07:42.643 INFO:tasks.workunit.client.0.smithi057.stdout:9/566: dread - dd/de/d89/f96 zero size 2024-02-28T23:07:42.644 INFO:tasks.workunit.client.1.smithi084.stdout:3/710: symlink d3/d27/d4b/deb/l10d 0 2024-02-28T23:07:42.644 INFO:tasks.workunit.client.0.smithi057.stdout:0/604: mknod d17/d20/d4a/d78/db5/cd9 0 2024-02-28T23:07:42.645 INFO:tasks.workunit.client.0.smithi057.stdout:7/518: creat d0/d5/d25/d3e/fa9 x:0 0 0 2024-02-28T23:07:42.648 INFO:tasks.workunit.client.1.smithi084.stdout:3/711: fdatasync d3/d50/d5a/d73/dc8/ff7 0 2024-02-28T23:07:42.648 INFO:tasks.workunit.client.1.smithi084.stdout:6/659: dwrite d1/d13/d50/fc9 [0,4194304] 0 2024-02-28T23:07:42.650 INFO:tasks.workunit.client.0.smithi057.stdout:1/480: dwrite d2/d7/df/db/d3a/f6c [0,4194304] 0 2024-02-28T23:07:42.650 INFO:tasks.workunit.client.0.smithi057.stdout:0/605: chown d17/d20/d23/d4b 197126574 1 2024-02-28T23:07:42.652 INFO:tasks.workunit.client.0.smithi057.stdout:7/519: fsync d0/d5/d36/d4e/d15/d29/d71/fa8 0 2024-02-28T23:07:42.661 INFO:tasks.workunit.client.1.smithi084.stdout:2/634: mkdir d5/d7/d9/d94/dbd 0 2024-02-28T23:07:42.661 INFO:tasks.workunit.client.1.smithi084.stdout:2/635: chown d5/d7/d12/l39 2 1 2024-02-28T23:07:42.664 INFO:tasks.workunit.client.0.smithi057.stdout:1/481: symlink d2/d7/d4f/d5c/d6b/lad 0 2024-02-28T23:07:42.665 INFO:tasks.workunit.client.0.smithi057.stdout:7/520: read d0/d5/d36/d4e/d15/f1a [459121,47451] 0 2024-02-28T23:07:42.667 INFO:tasks.workunit.client.1.smithi084.stdout:5/719: mknod dc/de/c10b 0 2024-02-28T23:07:42.667 INFO:tasks.workunit.client.0.smithi057.stdout:2/639: dwrite d8/d11/d21/d28/f75 [4194304,4194304] 0 2024-02-28T23:07:42.668 INFO:tasks.workunit.client.1.smithi084.stdout:9/731: rename d4/d6/f36 to d4/d6/d15/d12/d41/dea/feb 0 2024-02-28T23:07:42.675 INFO:tasks.workunit.client.1.smithi084.stdout:5/720: fsync dc/de/df/d31/d8e/da9/fce 0 2024-02-28T23:07:42.677 INFO:tasks.workunit.client.1.smithi084.stdout:1/745: rename d12/d84/d9a/fdb to d12/d1b/d2e/d52/d5b/ffa 0 2024-02-28T23:07:42.681 INFO:tasks.workunit.client.0.smithi057.stdout:0/606: dread d17/d20/d26/f41 [0,4194304] 0 2024-02-28T23:07:42.683 INFO:tasks.workunit.client.0.smithi057.stdout:7/521: chown d0/d5/d36/d4e/d15/d29/d3d/d51/f58 106 1 2024-02-28T23:07:42.685 INFO:tasks.workunit.client.1.smithi084.stdout:0/679: dwrite d1/d6/d59/d70/f8d [0,4194304] 0 2024-02-28T23:07:42.686 INFO:tasks.workunit.client.0.smithi057.stdout:0/607: mkdir d17/dda 0 2024-02-28T23:07:42.687 INFO:tasks.workunit.client.1.smithi084.stdout:1/746: unlink l0 0 2024-02-28T23:07:42.688 INFO:tasks.workunit.client.1.smithi084.stdout:2/636: dread d5/f17 [0,4194304] 0 2024-02-28T23:07:42.691 INFO:tasks.workunit.client.1.smithi084.stdout:7/728: getdents db/d10/d17/d9c/d9f/db1 0 2024-02-28T23:07:42.692 INFO:tasks.workunit.client.0.smithi057.stdout:7/522: mknod d0/d5/d36/d4e/d15/d29/d3d/d51/caa 0 2024-02-28T23:07:42.692 INFO:tasks.workunit.client.0.smithi057.stdout:3/471: dread d3/d8/f29 [0,4194304] 0 2024-02-28T23:07:42.694 INFO:tasks.workunit.client.1.smithi084.stdout:2/637: write d5/f31 [4659685,35004] 0 2024-02-28T23:07:42.695 INFO:tasks.workunit.client.0.smithi057.stdout:0/608: mkdir d17/d19/dd2/ddb 0 2024-02-28T23:07:42.696 INFO:tasks.workunit.client.0.smithi057.stdout:1/482: truncate d2/d33/f76 4321783 0 2024-02-28T23:07:42.699 INFO:tasks.workunit.client.0.smithi057.stdout:7/523: symlink d0/d5/d36/d4e/lab 0 2024-02-28T23:07:42.700 INFO:tasks.workunit.client.1.smithi084.stdout:1/747: write d12/d1b/d2e/d32/f47 [6099542,27659] 0 2024-02-28T23:07:42.702 INFO:tasks.workunit.client.1.smithi084.stdout:4/661: dread d1/d18/d20/d14/f39 [0,4194304] 0 2024-02-28T23:07:42.702 INFO:tasks.workunit.client.0.smithi057.stdout:2/640: dwrite d8/d11/d21/d28/f73 [0,4194304] 0 2024-02-28T23:07:42.702 INFO:tasks.workunit.client.0.smithi057.stdout:1/483: creat d2/d33/fae x:0 0 0 2024-02-28T23:07:42.703 INFO:tasks.workunit.client.1.smithi084.stdout:0/680: chown d1/d8/d18/d4c/d6c/d74/dd8 27 1 2024-02-28T23:07:42.703 INFO:tasks.workunit.client.1.smithi084.stdout:8/650: dwrite d0/d9/d30/db6/f80 [0,4194304] 0 2024-02-28T23:07:42.704 INFO:tasks.workunit.client.1.smithi084.stdout:7/729: write db/d10/d17/d64/fb5 [204520,99279] 0 2024-02-28T23:07:42.705 INFO:tasks.workunit.client.0.smithi057.stdout:5/532: sync 2024-02-28T23:07:42.706 INFO:tasks.workunit.client.0.smithi057.stdout:7/524: unlink d0/d5/d36/f7b 0 2024-02-28T23:07:42.706 INFO:tasks.workunit.client.1.smithi084.stdout:9/732: dwrite d4/d6/f29 [0,4194304] 0 2024-02-28T23:07:42.707 INFO:tasks.workunit.client.0.smithi057.stdout:0/609: rename d17/d19/d4d/d7b/l80 to d17/d20/ldc 0 2024-02-28T23:07:42.708 INFO:tasks.workunit.client.1.smithi084.stdout:2/638: write d5/d7/d9/d7c/d5b/f7f [579565,8534] 0 2024-02-28T23:07:42.710 INFO:tasks.workunit.client.1.smithi084.stdout:7/730: readlink db/l4e 0 2024-02-28T23:07:42.717 INFO:tasks.workunit.client.1.smithi084.stdout:8/651: stat d0/d24/l6c 0 2024-02-28T23:07:42.720 INFO:tasks.workunit.client.1.smithi084.stdout:4/662: creat d1/d18/d20/d65/d8e/d55/dc3/fdd x:0 0 0 2024-02-28T23:07:42.725 INFO:tasks.workunit.client.1.smithi084.stdout:9/733: mknod d4/cec 0 2024-02-28T23:07:42.726 INFO:tasks.workunit.client.0.smithi057.stdout:1/484: rename d2/f72 to d2/d71/d2d/d5b/faf 0 2024-02-28T23:07:42.729 INFO:tasks.workunit.client.1.smithi084.stdout:2/639: rmdir d5/d1f/d42/d74 39 2024-02-28T23:07:42.730 INFO:tasks.workunit.client.0.smithi057.stdout:9/567: dread dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:42.730 INFO:tasks.workunit.client.0.smithi057.stdout:1/485: read d2/f21 [541038,115112] 0 2024-02-28T23:07:42.736 INFO:tasks.workunit.client.1.smithi084.stdout:0/681: rename d1/d8/d18/d4c/d6c/d74/f91 to d1/d8/d69/db2/fdc 0 2024-02-28T23:07:42.736 INFO:tasks.workunit.client.0.smithi057.stdout:5/533: rename d4/d9/d14/d3b/l6b to d4/d21/d24/d72/la9 0 2024-02-28T23:07:42.740 INFO:tasks.workunit.client.1.smithi084.stdout:4/663: dread d1/d18/d20/d14/f39 [0,4194304] 0 2024-02-28T23:07:42.745 INFO:tasks.workunit.client.1.smithi084.stdout:6/660: dread d1/d9/d1d/d1e/fcd [0,4194304] 0 2024-02-28T23:07:42.746 INFO:tasks.workunit.client.1.smithi084.stdout:2/640: mkdir d5/d1f/d42/d8f/dbe 0 2024-02-28T23:07:42.746 INFO:tasks.workunit.client.0.smithi057.stdout:5/534: dread - d4/d9/d85/fa3 zero size 2024-02-28T23:07:42.747 INFO:tasks.workunit.client.1.smithi084.stdout:0/682: mknod d1/d8/d5f/d93/cdd 0 2024-02-28T23:07:42.748 INFO:tasks.workunit.client.1.smithi084.stdout:9/734: truncate d4/f90 640631 0 2024-02-28T23:07:42.757 INFO:tasks.workunit.client.0.smithi057.stdout:1/486: rename d2/d7/df/db/d12/l8d to d2/d7/d4f/d5c/d6b/lb0 0 2024-02-28T23:07:42.758 INFO:tasks.workunit.client.0.smithi057.stdout:1/487: chown d2/d7/df/db/d12/d19/d48/c57 1 1 2024-02-28T23:07:42.758 INFO:tasks.workunit.client.1.smithi084.stdout:4/664: creat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/db4/fde x:0 0 0 2024-02-28T23:07:42.758 INFO:tasks.workunit.client.1.smithi084.stdout:2/641: creat d5/d1f/d83/fbf x:0 0 0 2024-02-28T23:07:42.758 INFO:tasks.workunit.client.1.smithi084.stdout:9/735: chown d4/c3a 873655 1 2024-02-28T23:07:42.760 INFO:tasks.workunit.client.0.smithi057.stdout:5/535: rename d4/d9/d14/d3b/d61/d69/f6a to d4/d21/d24/d30/d62/faa 0 2024-02-28T23:07:42.763 INFO:tasks.workunit.client.0.smithi057.stdout:1/488: mkdir d2/d7/d4f/db1 0 2024-02-28T23:07:42.763 INFO:tasks.workunit.client.1.smithi084.stdout:8/652: dwrite d0/d9/d1b/d20/f59 [8388608,4194304] 0 2024-02-28T23:07:42.764 INFO:tasks.workunit.client.1.smithi084.stdout:2/642: chown d5/d7/d12/d24/d2a/d35/f40 0 1 2024-02-28T23:07:42.767 INFO:tasks.workunit.client.0.smithi057.stdout:1/489: symlink d2/d71/d2d/d39/da3/lb2 0 2024-02-28T23:07:42.768 INFO:tasks.workunit.client.1.smithi084.stdout:0/683: rename d1/d8/d18/d4c/d6c/d74/fdb to d1/d6/db/d35/d44/d92/fde 0 2024-02-28T23:07:42.770 INFO:tasks.workunit.client.1.smithi084.stdout:1/748: write d12/d1b/d63/d7d/f95 [646906,57688] 0 2024-02-28T23:07:42.773 INFO:tasks.workunit.client.1.smithi084.stdout:0/684: fdatasync d1/d8/d18/f9a 0 2024-02-28T23:07:42.773 INFO:tasks.workunit.client.1.smithi084.stdout:9/736: symlink d4/d6/led 0 2024-02-28T23:07:42.774 INFO:tasks.workunit.client.0.smithi057.stdout:5/536: rename d4/d9/d16/f19 to d4/d9/d14/fab 0 2024-02-28T23:07:42.774 INFO:tasks.workunit.client.0.smithi057.stdout:0/610: truncate d17/d20/f51 1372030 0 2024-02-28T23:07:42.774 INFO:tasks.workunit.client.0.smithi057.stdout:9/568: truncate dd/d1b/d40/fb2 217254 0 2024-02-28T23:07:42.776 INFO:tasks.workunit.client.1.smithi084.stdout:7/731: dread db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:42.778 INFO:tasks.workunit.client.0.smithi057.stdout:9/569: chown dd/d1b/d40/d44/d5c/daa 4 1 2024-02-28T23:07:42.778 INFO:tasks.workunit.client.1.smithi084.stdout:0/685: chown d1/d8/d69/db2/c95 249896 1 2024-02-28T23:07:42.781 INFO:tasks.workunit.client.0.smithi057.stdout:5/537: creat d4/d9/d16/d4e/fac x:0 0 0 2024-02-28T23:07:42.782 INFO:tasks.workunit.client.1.smithi084.stdout:8/653: rename d0/d9/d30/d33/d71/f92 to d0/fdb 0 2024-02-28T23:07:42.784 INFO:tasks.workunit.client.0.smithi057.stdout:0/611: chown d17/d20/d26/d30/d6b/d71/f7e 0 1 2024-02-28T23:07:42.784 INFO:tasks.workunit.client.0.smithi057.stdout:7/525: dwrite d0/d5/d25/d3e/f5a [0,4194304] 0 2024-02-28T23:07:42.785 INFO:tasks.workunit.client.1.smithi084.stdout:9/737: mkdir d4/dee 0 2024-02-28T23:07:42.785 INFO:tasks.workunit.client.0.smithi057.stdout:2/641: dwrite d8/fb [0,4194304] 0 2024-02-28T23:07:42.785 INFO:tasks.workunit.client.0.smithi057.stdout:5/538: truncate d4/f7d 600592 0 2024-02-28T23:07:42.786 INFO:tasks.workunit.client.1.smithi084.stdout:1/749: mknod d12/d1b/cfb 0 2024-02-28T23:07:42.787 INFO:tasks.workunit.client.1.smithi084.stdout:0/686: symlink d1/d8/da1/ldf 0 2024-02-28T23:07:42.788 INFO:tasks.workunit.client.1.smithi084.stdout:8/654: mknod d0/d9/d30/db6/cdc 0 2024-02-28T23:07:42.791 INFO:tasks.workunit.client.1.smithi084.stdout:8/655: stat d0/d9/d1b/d38/d43/d51/fb3 0 2024-02-28T23:07:42.794 INFO:tasks.workunit.client.1.smithi084.stdout:8/656: stat d0/fdb 0 2024-02-28T23:07:42.799 INFO:tasks.workunit.client.1.smithi084.stdout:0/687: creat d1/d8/d18/d4c/d6c/d74/fe0 x:0 0 0 2024-02-28T23:07:42.800 INFO:tasks.workunit.client.0.smithi057.stdout:7/526: creat d0/d5/d25/d80/d9f/fac x:0 0 0 2024-02-28T23:07:42.802 INFO:tasks.workunit.client.0.smithi057.stdout:5/539: creat d4/d9/d14/d63/fad x:0 0 0 2024-02-28T23:07:42.804 INFO:tasks.workunit.client.0.smithi057.stdout:1/490: link d2/d7/df/db/d12/d19/c51 d2/d33/d54/cb3 0 2024-02-28T23:07:42.805 INFO:tasks.workunit.client.0.smithi057.stdout:0/612: symlink d17/d20/d23/ldd 0 2024-02-28T23:07:42.806 INFO:tasks.workunit.client.0.smithi057.stdout:4/539: sync 2024-02-28T23:07:42.807 INFO:tasks.workunit.client.1.smithi084.stdout:9/738: creat d4/d32/d3f/fef x:0 0 0 2024-02-28T23:07:42.809 INFO:tasks.workunit.client.1.smithi084.stdout:7/732: getdents db/d10/d17/d9c/d9f/db1 0 2024-02-28T23:07:42.809 INFO:tasks.workunit.client.0.smithi057.stdout:9/570: dwrite dd/de/d24/d63/d77/fb4 [0,4194304] 0 2024-02-28T23:07:42.811 INFO:tasks.workunit.client.1.smithi084.stdout:0/688: write d1/d8/d69/d6b/fbe [1334822,74785] 0 2024-02-28T23:07:42.813 INFO:tasks.workunit.client.1.smithi084.stdout:8/657: truncate d0/d24/d6e/f91 421813 0 2024-02-28T23:07:42.815 INFO:tasks.workunit.client.1.smithi084.stdout:6/661: dwrite d1/d13/d47/d91/dba/f9b [0,4194304] 0 2024-02-28T23:07:42.816 INFO:tasks.workunit.client.0.smithi057.stdout:2/642: dread d8/fb [0,4194304] 0 2024-02-28T23:07:42.819 INFO:tasks.workunit.client.0.smithi057.stdout:5/540: fdatasync d4/d9/d14/fab 0 2024-02-28T23:07:42.820 INFO:tasks.workunit.client.0.smithi057.stdout:9/571: mknod dd/de/d89/cb9 0 2024-02-28T23:07:42.820 INFO:tasks.workunit.client.0.smithi057.stdout:0/613: mkdir d17/d19/dd2/ddb/dde 0 2024-02-28T23:07:42.821 INFO:tasks.workunit.client.1.smithi084.stdout:9/739: rename d4/d6/d15/d75/d9d to d4/d6/d15/d75/d9d/dc3/dce/df0 22 2024-02-28T23:07:42.821 INFO:tasks.workunit.client.1.smithi084.stdout:1/750: dwrite d12/d1b/d2e/d52/fe3 [0,4194304] 0 2024-02-28T23:07:42.822 INFO:tasks.workunit.client.0.smithi057.stdout:2/643: stat d8/d11/d21/d28/d49/d85/d92/lab 0 2024-02-28T23:07:42.822 INFO:tasks.workunit.client.1.smithi084.stdout:0/689: chown d1/d6/db/d35/d4a/f66 4253 1 2024-02-28T23:07:42.823 INFO:tasks.workunit.client.1.smithi084.stdout:7/733: chown db/d10/d17/d3b/c78 1995900 1 2024-02-28T23:07:42.826 INFO:tasks.workunit.client.0.smithi057.stdout:9/572: unlink dd/de/d14/c15 0 2024-02-28T23:07:42.826 INFO:tasks.workunit.client.1.smithi084.stdout:9/740: mknod d4/d6/d15/d3c/cf1 0 2024-02-28T23:07:42.826 INFO:tasks.workunit.client.0.smithi057.stdout:5/541: mknod d4/d9/d14/d63/d88/cae 0 2024-02-28T23:07:42.828 INFO:tasks.workunit.client.0.smithi057.stdout:0/614: rmdir d17/d20/d26/d48/d7c/d98/da6 39 2024-02-28T23:07:42.830 INFO:tasks.workunit.client.1.smithi084.stdout:4/665: dread d1/f4 [0,4194304] 0 2024-02-28T23:07:42.830 INFO:tasks.workunit.client.1.smithi084.stdout:1/751: creat d12/d1b/d2e/d52/d5b/ffc x:0 0 0 2024-02-28T23:07:42.830 INFO:tasks.workunit.client.1.smithi084.stdout:0/690: write d1/d8/d69/d6b/f9b [201192,35555] 0 2024-02-28T23:07:42.833 INFO:tasks.workunit.client.1.smithi084.stdout:7/734: creat db/d10/d18/db4/fd9 x:0 0 0 2024-02-28T23:07:42.835 INFO:tasks.workunit.client.0.smithi057.stdout:5/542: write d4/d9/d16/f2b [9518,88011] 0 2024-02-28T23:07:42.841 INFO:tasks.workunit.client.0.smithi057.stdout:7/527: dwrite d0/d5/d36/d4e/d15/d29/d3d/f7a [0,4194304] 0 2024-02-28T23:07:42.844 INFO:tasks.workunit.client.1.smithi084.stdout:8/658: rename d0/d9/d1b/d20/l66 to d0/d9/d30/d33/d71/d84/dc0/ldd 0 2024-02-28T23:07:42.844 INFO:tasks.workunit.client.0.smithi057.stdout:2/644: rename d8/d11/f29 to d8/d11/d48/fc9 0 2024-02-28T23:07:42.845 INFO:tasks.workunit.client.1.smithi084.stdout:9/741: creat d4/d6/d15/d75/d2d/daa/ff2 x:0 0 0 2024-02-28T23:07:42.846 INFO:tasks.workunit.client.0.smithi057.stdout:9/573: dread dd/de/d14/d48/f74 [0,4194304] 0 2024-02-28T23:07:42.847 INFO:tasks.workunit.client.1.smithi084.stdout:9/742: read - d4/fdc zero size 2024-02-28T23:07:42.849 INFO:tasks.workunit.client.0.smithi057.stdout:1/491: write d2/d7/d4f/d5c/d6b/f77 [303951,39233] 0 2024-02-28T23:07:42.849 INFO:tasks.workunit.client.0.smithi057.stdout:9/574: stat dd/de/d14/c71 0 2024-02-28T23:07:42.849 INFO:tasks.workunit.client.0.smithi057.stdout:0/615: creat d17/d20/d26/d48/d7c/d98/da6/fdf x:0 0 0 2024-02-28T23:07:42.850 INFO:tasks.workunit.client.0.smithi057.stdout:5/543: mknod d4/d9/d16/d4e/caf 0 2024-02-28T23:07:42.851 INFO:tasks.workunit.client.1.smithi084.stdout:4/666: dread - d1/d18/d20/d24/d2f/d3e/d52/f9b zero size 2024-02-28T23:07:42.851 INFO:tasks.workunit.client.0.smithi057.stdout:2/645: creat d8/dae/fca x:0 0 0 2024-02-28T23:07:42.855 INFO:tasks.workunit.client.1.smithi084.stdout:8/659: readlink d0/d9/d1b/d38/d43/l4f 0 2024-02-28T23:07:42.855 INFO:tasks.workunit.client.0.smithi057.stdout:1/492: mkdir d2/d7/d4f/db4 0 2024-02-28T23:07:42.855 INFO:tasks.workunit.client.0.smithi057.stdout:9/575: fdatasync dd/de/d14/d48/d4b/f54 0 2024-02-28T23:07:42.856 INFO:tasks.workunit.client.1.smithi084.stdout:0/691: getdents d1/d6/db/d37/d72/d7b 0 2024-02-28T23:07:42.857 INFO:tasks.workunit.client.0.smithi057.stdout:5/544: creat d4/fb0 x:0 0 0 2024-02-28T23:07:42.861 INFO:tasks.workunit.client.0.smithi057.stdout:2/646: write d8/d11/d56/f7c [1173623,46012] 0 2024-02-28T23:07:42.861 INFO:tasks.workunit.client.0.smithi057.stdout:7/528: read d0/d5/d36/d4e/d15/f22 [2619819,32231] 0 2024-02-28T23:07:42.867 INFO:tasks.workunit.client.1.smithi084.stdout:1/752: dwrite d12/d1b/d2e/d30/d50/da8/dd9/df0/ff5 [0,4194304] 0 2024-02-28T23:07:42.867 INFO:tasks.workunit.client.1.smithi084.stdout:0/692: unlink d1/d6/db/d35/d4a/f66 0 2024-02-28T23:07:42.867 INFO:tasks.workunit.client.1.smithi084.stdout:8/660: truncate d0/d24/d9e/fac 572245 0 2024-02-28T23:07:42.867 INFO:tasks.workunit.client.0.smithi057.stdout:4/540: write d1a/d48/d9c/d2e/f7d [4094094,102298] 0 2024-02-28T23:07:42.868 INFO:tasks.workunit.client.1.smithi084.stdout:4/667: rename d1/d18/d20/d24/d2f to d1/d18/d20/d24/d2f/d3e/d52/d45/ddf 22 2024-02-28T23:07:42.868 INFO:tasks.workunit.client.0.smithi057.stdout:4/541: readlink d1a/d2f/db2/l9d 0 2024-02-28T23:07:42.870 INFO:tasks.workunit.client.1.smithi084.stdout:0/693: stat d1/d6/db/d37/d72/lba 0 2024-02-28T23:07:42.872 INFO:tasks.workunit.client.1.smithi084.stdout:7/735: dwrite db/d10/f56 [0,4194304] 0 2024-02-28T23:07:42.876 INFO:tasks.workunit.client.1.smithi084.stdout:8/661: creat d0/d9/d1b/d38/d43/dae/db8/fde x:0 0 0 2024-02-28T23:07:42.883 INFO:tasks.workunit.client.0.smithi057.stdout:3/472: sync 2024-02-28T23:07:42.883 INFO:tasks.workunit.client.0.smithi057.stdout:6/588: sync 2024-02-28T23:07:42.883 INFO:tasks.workunit.client.0.smithi057.stdout:8/618: sync 2024-02-28T23:07:42.884 INFO:tasks.workunit.client.0.smithi057.stdout:0/616: dwrite d17/d20/d26/d48/d7c/d98/da6/fd5 [0,4194304] 0 2024-02-28T23:07:42.884 INFO:tasks.workunit.client.1.smithi084.stdout:6/662: truncate d1/d13/d2e/d41/d2c/f71 1153318 0 2024-02-28T23:07:42.884 INFO:tasks.workunit.client.1.smithi084.stdout:1/753: rename d12/d1b/d2e/d52/fc7 to d12/d1b/d91/d9b/ffd 0 2024-02-28T23:07:42.885 INFO:tasks.workunit.client.1.smithi084.stdout:8/662: mknod d0/d24/d6e/cdf 0 2024-02-28T23:07:42.889 INFO:tasks.workunit.client.1.smithi084.stdout:0/694: write d1/d6/db/d35/d4a/d56/f89 [1507006,119671] 0 2024-02-28T23:07:42.889 INFO:tasks.workunit.client.1.smithi084.stdout:7/736: rename db to db/d5e/dce/dda 22 2024-02-28T23:07:42.897 INFO:tasks.workunit.client.1.smithi084.stdout:1/754: write d12/d1b/d91/d9b/ff8 [903248,51155] 0 2024-02-28T23:07:42.897 INFO:tasks.workunit.client.1.smithi084.stdout:8/663: write d0/d11/f8f [3083638,96348] 0 2024-02-28T23:07:42.898 INFO:tasks.workunit.client.0.smithi057.stdout:5/545: symlink d4/d9/d14/d3b/d61/d71/da1/lb1 0 2024-02-28T23:07:42.903 INFO:tasks.workunit.client.0.smithi057.stdout:1/493: dread d2/d71/d2d/f49 [0,4194304] 0 2024-02-28T23:07:42.903 INFO:tasks.workunit.client.1.smithi084.stdout:9/743: dwrite d4/f37 [0,4194304] 0 2024-02-28T23:07:42.919 INFO:tasks.workunit.client.1.smithi084.stdout:9/744: dread d4/d6/d53/fc0 [0,4194304] 0 2024-02-28T23:07:42.928 INFO:tasks.workunit.client.1.smithi084.stdout:0/695: unlink d1/d6/db/d35/d4a/laa 0 2024-02-28T23:07:42.931 INFO:tasks.workunit.client.1.smithi084.stdout:0/696: dread - d1/d8/d18/d4c/d6c/d74/fe0 zero size 2024-02-28T23:07:42.931 INFO:tasks.workunit.client.1.smithi084.stdout:1/755: creat d12/d1b/d2e/d30/d50/da8/ffe x:0 0 0 2024-02-28T23:07:42.932 INFO:tasks.workunit.client.1.smithi084.stdout:8/664: creat d0/d24/d6e/fe0 x:0 0 0 2024-02-28T23:07:42.933 INFO:tasks.workunit.client.1.smithi084.stdout:1/756: write d12/d84/fcf [320526,104117] 0 2024-02-28T23:07:42.934 INFO:tasks.workunit.client.1.smithi084.stdout:0/697: readlink d1/d8/d18/d4c/l60 0 2024-02-28T23:07:42.940 INFO:tasks.workunit.client.1.smithi084.stdout:7/737: rename db/d10/d17/f8f to db/fdb 0 2024-02-28T23:07:42.945 INFO:tasks.workunit.client.0.smithi057.stdout:7/529: mknod d0/d5/d36/cad 0 2024-02-28T23:07:42.946 INFO:tasks.workunit.client.1.smithi084.stdout:8/665: mknod d0/d11/ce1 0 2024-02-28T23:07:42.946 INFO:tasks.workunit.client.1.smithi084.stdout:6/663: link d1/d9/d1d/d1e/cb4 d1/d9/ddc/cec 0 2024-02-28T23:07:42.946 INFO:tasks.workunit.client.1.smithi084.stdout:0/698: symlink d1/d6/db/d35/d82/daf/le1 0 2024-02-28T23:07:42.946 INFO:tasks.workunit.client.1.smithi084.stdout:6/664: write d1/d9/fd5 [145981,64002] 0 2024-02-28T23:07:42.946 INFO:tasks.workunit.client.1.smithi084.stdout:9/745: getdents d4/d6/d15/de0 0 2024-02-28T23:07:42.947 INFO:tasks.workunit.client.0.smithi057.stdout:3/473: fdatasync d3/d4/d7/f84 0 2024-02-28T23:07:42.956 INFO:tasks.workunit.client.1.smithi084.stdout:0/699: rename d1/d8/d69/da3/fd3 to d1/d8/d5f/d93/fe2 0 2024-02-28T23:07:42.957 INFO:tasks.workunit.client.1.smithi084.stdout:9/746: creat d4/d32/d3f/d89/db8/ff3 x:0 0 0 2024-02-28T23:07:42.958 INFO:tasks.workunit.client.0.smithi057.stdout:1/494: mkdir d2/d71/d2d/d39/da3/db5 0 2024-02-28T23:07:42.960 INFO:tasks.workunit.client.0.smithi057.stdout:1/495: stat d2/d7/df/db/d12/d3f/l4e 0 2024-02-28T23:07:42.962 INFO:tasks.workunit.client.1.smithi084.stdout:6/665: rename d1/d13/d2e/cae to d1/d13/d50/d74/ced 0 2024-02-28T23:07:42.964 INFO:tasks.workunit.client.1.smithi084.stdout:0/700: creat d1/d6/db/d35/d44/fe3 x:0 0 0 2024-02-28T23:07:42.965 INFO:tasks.workunit.client.1.smithi084.stdout:7/738: dwrite db/d10/d18/f1b [0,4194304] 0 2024-02-28T23:07:42.968 INFO:tasks.workunit.client.1.smithi084.stdout:6/666: creat d1/d13/d2e/d41/d4d/fee x:0 0 0 2024-02-28T23:07:42.974 INFO:tasks.workunit.client.1.smithi084.stdout:0/701: creat d1/d8/d69/db2/fe4 x:0 0 0 2024-02-28T23:07:42.975 INFO:tasks.workunit.client.1.smithi084.stdout:6/667: creat d1/d9/d1d/fef x:0 0 0 2024-02-28T23:07:42.976 INFO:tasks.workunit.client.0.smithi057.stdout:0/617: link d17/d20/d23/d47/d60/fd3 d17/d19/dd2/fe0 0 2024-02-28T23:07:42.977 INFO:tasks.workunit.client.0.smithi057.stdout:0/618: stat d17/d20/d26/f69 0 2024-02-28T23:07:42.986 INFO:tasks.workunit.client.0.smithi057.stdout:7/530: getdents d0/d3f/da1 0 2024-02-28T23:07:42.986 INFO:tasks.workunit.client.0.smithi057.stdout:7/531: readlink d0/d5/d4a/l77 0 2024-02-28T23:07:42.986 INFO:tasks.workunit.client.1.smithi084.stdout:7/739: dwrite db/d10/d17/d64/f93 [0,4194304] 0 2024-02-28T23:07:42.991 INFO:tasks.workunit.client.0.smithi057.stdout:6/589: rmdir d0/d17/d20/dbd 0 2024-02-28T23:07:42.993 INFO:tasks.workunit.client.0.smithi057.stdout:0/619: chown d17/d19/l5e 530170 1 2024-02-28T23:07:42.997 INFO:tasks.workunit.client.1.smithi084.stdout:8/666: readlink d0/d11/l26 0 2024-02-28T23:07:42.998 INFO:tasks.workunit.client.1.smithi084.stdout:0/702: dwrite d1/d8/d18/d4c/d6c/d74/fad [0,4194304] 0 2024-02-28T23:07:42.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/747: truncate d4/d6/d15/d12/f48 8045969 0 2024-02-28T23:07:43.001 INFO:tasks.workunit.client.1.smithi084.stdout:9/748: fdatasync d4/f51 0 2024-02-28T23:07:43.002 INFO:tasks.workunit.client.1.smithi084.stdout:6/668: dwrite d1/d13/d47/f7a [0,4194304] 0 2024-02-28T23:07:43.002 INFO:tasks.workunit.client.0.smithi057.stdout:1/496: fdatasync d2/d7/df/db/d12/d3f/d47/f9f 0 2024-02-28T23:07:43.006 INFO:tasks.workunit.client.1.smithi084.stdout:9/749: stat d4/d6/d15/d75/d9d/dc3/dce 0 2024-02-28T23:07:43.006 INFO:tasks.workunit.client.0.smithi057.stdout:3/474: symlink d3/d4f/d4c/d91/l98 0 2024-02-28T23:07:43.009 INFO:tasks.workunit.client.1.smithi084.stdout:8/667: truncate d0/d9/d1b/d38/d43/dae/db8/fc2 67351 0 2024-02-28T23:07:43.010 INFO:tasks.workunit.client.1.smithi084.stdout:9/750: write d4/f51 [2407533,91398] 0 2024-02-28T23:07:43.011 INFO:tasks.workunit.client.0.smithi057.stdout:0/620: dread - d17/d19/f76 zero size 2024-02-28T23:07:43.012 INFO:tasks.workunit.client.0.smithi057.stdout:0/621: fdatasync d17/d19/d4d/f62 0 2024-02-28T23:07:43.013 INFO:tasks.workunit.client.0.smithi057.stdout:5/546: getdents d4/d9/d14/d63 0 2024-02-28T23:07:43.014 INFO:tasks.workunit.client.1.smithi084.stdout:6/669: link d1/d9/d1d/d1e/d33/la2 d1/d13/d2e/d41/dbb/lf0 0 2024-02-28T23:07:43.014 INFO:tasks.workunit.client.1.smithi084.stdout:6/670: chown d1/d13/d47 226 1 2024-02-28T23:07:43.016 INFO:tasks.workunit.client.1.smithi084.stdout:8/668: creat d0/d9/d1b/d38/dd4/fe2 x:0 0 0 2024-02-28T23:07:43.018 INFO:tasks.workunit.client.1.smithi084.stdout:6/671: write d1/d13/d50/f8d [275569,57526] 0 2024-02-28T23:07:43.023 INFO:tasks.workunit.client.1.smithi084.stdout:9/751: rename d4/d6/d15/d75/d9d/dd0/fd2 to d4/d6/d15/d3c/ff4 0 2024-02-28T23:07:43.024 INFO:tasks.workunit.client.0.smithi057.stdout:0/622: truncate d17/d20/d26/f69 1313168 0 2024-02-28T23:07:43.025 INFO:tasks.workunit.client.1.smithi084.stdout:7/740: write db/d10/f56 [5046359,40850] 0 2024-02-28T23:07:43.026 INFO:tasks.workunit.client.1.smithi084.stdout:6/672: dread - d1/d13/d2e/d41/dbb/fb0 zero size 2024-02-28T23:07:43.027 INFO:tasks.workunit.client.0.smithi057.stdout:1/497: symlink d2/d33/daa/lb6 0 2024-02-28T23:07:43.027 INFO:tasks.workunit.client.1.smithi084.stdout:8/669: read d0/d24/d6e/f8b [147887,124053] 0 2024-02-28T23:07:43.028 INFO:tasks.workunit.client.0.smithi057.stdout:0/623: dread - d17/d20/d23/f75 zero size 2024-02-28T23:07:43.029 INFO:tasks.workunit.client.1.smithi084.stdout:6/673: write d1/fe6 [346452,59148] 0 2024-02-28T23:07:43.029 INFO:tasks.workunit.client.1.smithi084.stdout:7/741: symlink db/d5e/d87/dc1/ldc 0 2024-02-28T23:07:43.030 INFO:tasks.workunit.client.1.smithi084.stdout:8/670: write d0/d24/d6e/fc5 [484697,107457] 0 2024-02-28T23:07:43.032 INFO:tasks.workunit.client.0.smithi057.stdout:1/498: write d2/d7/df/db/d3a/f38 [163889,53001] 0 2024-02-28T23:07:43.033 INFO:tasks.workunit.client.1.smithi084.stdout:7/742: read db/d10/d17/d64/f75 [3130282,22525] 0 2024-02-28T23:07:43.033 INFO:tasks.workunit.client.1.smithi084.stdout:6/674: stat d1/d88/dbf/lcc 0 2024-02-28T23:07:43.035 INFO:tasks.workunit.client.0.smithi057.stdout:0/624: read d17/d19/f53 [1457713,3785] 0 2024-02-28T23:07:43.038 INFO:tasks.workunit.client.1.smithi084.stdout:6/675: unlink d1/d9/f4f 0 2024-02-28T23:07:43.039 INFO:tasks.workunit.client.0.smithi057.stdout:3/475: creat d3/d4/f99 x:0 0 0 2024-02-28T23:07:43.041 INFO:tasks.workunit.client.1.smithi084.stdout:7/743: fdatasync db/d10/d17/d3b/f48 0 2024-02-28T23:07:43.042 INFO:tasks.workunit.client.0.smithi057.stdout:3/476: read - d3/d4/d7/f8f zero size 2024-02-28T23:07:43.046 INFO:tasks.workunit.client.1.smithi084.stdout:9/752: dread d4/d6/d15/d12/d41/f64 [0,4194304] 0 2024-02-28T23:07:43.047 INFO:tasks.workunit.client.1.smithi084.stdout:7/744: symlink db/d10/d18/ldd 0 2024-02-28T23:07:43.056 INFO:tasks.workunit.client.1.smithi084.stdout:8/671: dwrite d0/d24/d6e/fc5 [0,4194304] 0 2024-02-28T23:07:43.058 INFO:tasks.workunit.client.1.smithi084.stdout:0/703: dread d1/d6/db/d35/d4a/f88 [0,4194304] 0 2024-02-28T23:07:43.060 INFO:tasks.workunit.client.0.smithi057.stdout:0/625: rmdir d17/d19/dd2/ddb/dde 0 2024-02-28T23:07:43.062 INFO:tasks.workunit.client.1.smithi084.stdout:9/753: rename d4/d6/d15/d12/d41/d55/fa5 to d4/d6/d15/d12/d41/dea/ff5 0 2024-02-28T23:07:43.068 INFO:tasks.workunit.client.1.smithi084.stdout:9/754: dread - d4/d6/d15/d75/d2d/daa/fc6 zero size 2024-02-28T23:07:43.069 INFO:tasks.workunit.client.1.smithi084.stdout:0/704: symlink d1/d6/db/d35/d44/d92/dcf/le5 0 2024-02-28T23:07:43.071 INFO:tasks.workunit.client.1.smithi084.stdout:6/676: dwrite d1/d9/fd5 [0,4194304] 0 2024-02-28T23:07:43.073 INFO:tasks.workunit.client.1.smithi084.stdout:7/745: dread db/d10/d17/d64/f93 [0,4194304] 0 2024-02-28T23:07:43.074 INFO:tasks.workunit.client.1.smithi084.stdout:6/677: chown d1/d13/d50/d74/faa 184546140 1 2024-02-28T23:07:43.074 INFO:tasks.workunit.client.1.smithi084.stdout:9/755: rmdir d4/d69 39 2024-02-28T23:07:43.076 INFO:tasks.workunit.client.1.smithi084.stdout:0/705: symlink d1/d8/d5f/d93/le6 0 2024-02-28T23:07:43.079 INFO:tasks.workunit.client.1.smithi084.stdout:9/756: write d4/d32/d3f/d89/f99 [601963,34783] 0 2024-02-28T23:07:43.079 INFO:tasks.workunit.client.1.smithi084.stdout:6/678: unlink d1/d13/d47/d91/dba/c6b 0 2024-02-28T23:07:43.083 INFO:tasks.workunit.client.0.smithi057.stdout:8/619: truncate d4/d13/d18/f26 206024 0 2024-02-28T23:07:43.083 INFO:tasks.workunit.client.1.smithi084.stdout:7/746: rename db/l4a to db/d5e/dc3/lde 0 2024-02-28T23:07:43.086 INFO:tasks.workunit.client.1.smithi084.stdout:6/679: mknod d1/d13/d47/d91/cf1 0 2024-02-28T23:07:43.086 INFO:tasks.workunit.client.0.smithi057.stdout:8/620: write d4/d13/d18/d62/d66/fbf [477577,43487] 0 2024-02-28T23:07:43.087 INFO:tasks.workunit.client.0.smithi057.stdout:9/576: mknod dd/de/d14/d48/cba 0 2024-02-28T23:07:43.089 INFO:tasks.workunit.client.1.smithi084.stdout:7/747: symlink db/d10/d17/d9c/ldf 0 2024-02-28T23:07:43.089 INFO:tasks.workunit.client.0.smithi057.stdout:6/590: write d0/d4/f19 [2669706,50646] 0 2024-02-28T23:07:43.091 INFO:tasks.workunit.client.0.smithi057.stdout:9/577: dread - dd/de/d24/d72/f91 zero size 2024-02-28T23:07:43.093 INFO:tasks.workunit.client.1.smithi084.stdout:9/757: link d4/d6/d15/d12/f97 d4/d6/d15/d12/d41/ff6 0 2024-02-28T23:07:43.094 INFO:tasks.workunit.client.0.smithi057.stdout:9/578: stat dd/de/d14/c82 0 2024-02-28T23:07:43.095 INFO:tasks.workunit.client.1.smithi084.stdout:7/748: symlink db/d10/d17/d9c/d9f/db1/db2/le0 0 2024-02-28T23:07:43.097 INFO:tasks.workunit.client.1.smithi084.stdout:9/758: fsync d4/d32/d3f/f96 0 2024-02-28T23:07:43.101 INFO:tasks.workunit.client.0.smithi057.stdout:6/591: getdents d0/d4/d25 0 2024-02-28T23:07:43.102 INFO:tasks.workunit.client.1.smithi084.stdout:0/706: dwrite d1/d8/d18/f9a [0,4194304] 0 2024-02-28T23:07:43.102 INFO:tasks.workunit.client.1.smithi084.stdout:7/749: chown db/d10/d17/d3b/f9b 22164922 1 2024-02-28T23:07:43.104 INFO:tasks.workunit.client.0.smithi057.stdout:1/499: write d2/d7/df/db/d3a/f3b [4461876,50084] 0 2024-02-28T23:07:43.105 INFO:tasks.workunit.client.1.smithi084.stdout:8/672: dread d0/d11/f8f [0,4194304] 0 2024-02-28T23:07:43.106 INFO:tasks.workunit.client.0.smithi057.stdout:7/532: dwrite d0/d5/d36/d4e/d15/f46 [0,4194304] 0 2024-02-28T23:07:43.106 INFO:tasks.workunit.client.0.smithi057.stdout:6/592: chown d0/d1b/d3e/l44 29 1 2024-02-28T23:07:43.107 INFO:tasks.workunit.client.1.smithi084.stdout:9/759: creat d4/d6/ff7 x:0 0 0 2024-02-28T23:07:43.114 INFO:tasks.workunit.client.1.smithi084.stdout:7/750: unlink db/d10/d17/l38 0 2024-02-28T23:07:43.116 INFO:tasks.workunit.client.0.smithi057.stdout:2/647: rename d8/d11/d48/d50/f6d to d8/d11/d39/fcb 0 2024-02-28T23:07:43.118 INFO:tasks.workunit.client.1.smithi084.stdout:0/707: mknod d1/d6/d1b/ce7 0 2024-02-28T23:07:43.119 INFO:tasks.workunit.client.1.smithi084.stdout:7/751: chown db/d10/d17/l39 384751 1 2024-02-28T23:07:43.119 INFO:tasks.workunit.client.1.smithi084.stdout:0/708: chown d1/c6e 22 1 2024-02-28T23:07:43.119 INFO:tasks.workunit.client.1.smithi084.stdout:7/752: chown db/d10/d17/d3b/f3f 183 1 2024-02-28T23:07:43.121 INFO:tasks.workunit.client.0.smithi057.stdout:9/579: dwrite dd/de/d4a/d9f/fb3 [0,4194304] 0 2024-02-28T23:07:43.123 INFO:tasks.workunit.client.0.smithi057.stdout:2/648: readlink d8/d11/d48/l4d 0 2024-02-28T23:07:43.126 INFO:tasks.workunit.client.0.smithi057.stdout:7/533: creat d0/d5/d25/d80/fae x:0 0 0 2024-02-28T23:07:43.128 INFO:tasks.workunit.client.0.smithi057.stdout:0/626: write d17/d20/f2b [915517,39776] 0 2024-02-28T23:07:43.129 INFO:tasks.workunit.client.1.smithi084.stdout:6/680: dread d1/d13/d50/fca [0,4194304] 0 2024-02-28T23:07:43.134 INFO:tasks.workunit.client.1.smithi084.stdout:3/712: sync 2024-02-28T23:07:43.135 INFO:tasks.workunit.client.1.smithi084.stdout:6/681: readlink d1/d13/d2e/d41/dbb/l45 0 2024-02-28T23:07:43.138 INFO:tasks.workunit.client.0.smithi057.stdout:3/477: rename d3/d4/d1d/d47/d69 to d3/d4/d7/d9a 0 2024-02-28T23:07:43.139 INFO:tasks.workunit.client.0.smithi057.stdout:4/542: sync 2024-02-28T23:07:43.140 INFO:tasks.workunit.client.1.smithi084.stdout:6/682: dread - d1/d9/ddc/fad zero size 2024-02-28T23:07:43.145 INFO:tasks.workunit.client.1.smithi084.stdout:0/709: dwrite d1/d6/db/d35/d4a/f88 [0,4194304] 0 2024-02-28T23:07:43.146 INFO:tasks.workunit.client.1.smithi084.stdout:6/683: dread - d1/d9/d1d/d1e/fc8 zero size 2024-02-28T23:07:43.146 INFO:tasks.workunit.client.0.smithi057.stdout:5/547: sync 2024-02-28T23:07:43.146 INFO:tasks.workunit.client.0.smithi057.stdout:6/593: dwrite d0/d17/d20/fc3 [0,4194304] 0 2024-02-28T23:07:43.146 INFO:tasks.workunit.client.0.smithi057.stdout:7/534: chown d0/d5/d36/d4e/d3b/f9e 31833612 1 2024-02-28T23:07:43.148 INFO:tasks.workunit.client.1.smithi084.stdout:3/713: link d3/d27/d7d/db0/d14/d31/lc2 d3/d27/d7d/l10e 0 2024-02-28T23:07:43.149 INFO:tasks.workunit.client.1.smithi084.stdout:6/684: stat d1/d13/d2e/fd3 0 2024-02-28T23:07:43.151 INFO:tasks.workunit.client.1.smithi084.stdout:3/714: dread - d3/fc0 zero size 2024-02-28T23:07:43.152 INFO:tasks.workunit.client.0.smithi057.stdout:8/621: dread d4/f7 [12582912,4194304] 0 2024-02-28T23:07:43.153 INFO:tasks.workunit.client.1.smithi084.stdout:7/753: dwrite db/d5e/dc6/f4f [4194304,4194304] 0 2024-02-28T23:07:43.153 INFO:tasks.workunit.client.1.smithi084.stdout:8/673: dread d0/d24/d78/f86 [0,4194304] 0 2024-02-28T23:07:43.154 INFO:tasks.workunit.client.1.smithi084.stdout:3/715: dread - d3/d27/d7d/db0/d1a/d21/d3d/f52 zero size 2024-02-28T23:07:43.158 INFO:tasks.workunit.client.0.smithi057.stdout:8/622: write d4/d13/d1a/f94 [646650,13063] 0 2024-02-28T23:07:43.160 INFO:tasks.workunit.client.1.smithi084.stdout:3/716: write d3/d27/d7d/db0/d1a/d21/d76/fb2 [458685,59237] 0 2024-02-28T23:07:43.160 INFO:tasks.workunit.client.1.smithi084.stdout:7/754: fsync db/d10/d18/f8d 0 2024-02-28T23:07:43.165 INFO:tasks.workunit.client.0.smithi057.stdout:2/649: dwrite d8/d11/d15/f20 [0,4194304] 0 2024-02-28T23:07:43.169 INFO:tasks.workunit.client.1.smithi084.stdout:3/717: fdatasync d3/d27/d7d/db0/d14/d31/d34/fcf 0 2024-02-28T23:07:43.174 INFO:tasks.workunit.client.1.smithi084.stdout:3/718: mknod d3/d27/d7d/db0/d14/c10f 0 2024-02-28T23:07:43.175 INFO:tasks.workunit.client.1.smithi084.stdout:7/755: read db/d10/f94 [1133943,6365] 0 2024-02-28T23:07:43.176 INFO:tasks.workunit.client.1.smithi084.stdout:6/685: dwrite d1/d13/d47/f4b [0,4194304] 0 2024-02-28T23:07:43.177 INFO:tasks.workunit.client.1.smithi084.stdout:7/756: chown db/d10/c91 55 1 2024-02-28T23:07:43.178 INFO:tasks.workunit.client.0.smithi057.stdout:0/627: dwrite d17/d20/db6/fc9 [0,4194304] 0 2024-02-28T23:07:43.178 INFO:tasks.workunit.client.1.smithi084.stdout:7/757: chown db/d10/d17/f42 826 1 2024-02-28T23:07:43.179 INFO:tasks.workunit.client.0.smithi057.stdout:5/548: mkdir d4/d9/d14/d3b/d61/db2 0 2024-02-28T23:07:43.179 INFO:tasks.workunit.client.0.smithi057.stdout:1/500: rename d2/d7/df/f92 to d2/d71/d2d/d39/da3/db5/fb7 0 2024-02-28T23:07:43.183 INFO:tasks.workunit.client.0.smithi057.stdout:0/628: chown d17/d20/d4a/l9e 704159552 1 2024-02-28T23:07:43.184 INFO:tasks.workunit.client.1.smithi084.stdout:7/758: creat db/dbd/fe1 x:0 0 0 2024-02-28T23:07:43.184 INFO:tasks.workunit.client.0.smithi057.stdout:6/594: mkdir d0/d1b/d3e/d3d/d74/dca 0 2024-02-28T23:07:43.187 INFO:tasks.workunit.client.0.smithi057.stdout:8/623: creat d4/d13/d2c/d4e/fe2 x:0 0 0 2024-02-28T23:07:43.189 INFO:tasks.workunit.client.1.smithi084.stdout:8/674: write d0/d9/d1b/d38/f3e [5100740,36505] 0 2024-02-28T23:07:43.192 INFO:tasks.workunit.client.0.smithi057.stdout:2/650: unlink d8/fd 0 2024-02-28T23:07:43.193 INFO:tasks.workunit.client.0.smithi057.stdout:6/595: write d0/d1b/d54/f90 [2027721,36260] 0 2024-02-28T23:07:43.194 INFO:tasks.workunit.client.0.smithi057.stdout:8/624: truncate d4/d1e/d56/d88/fd1 804501 0 2024-02-28T23:07:43.198 INFO:tasks.workunit.client.1.smithi084.stdout:3/719: truncate d3/d27/fb7 2840819 0 2024-02-28T23:07:43.199 INFO:tasks.workunit.client.0.smithi057.stdout:4/543: rename d1a/d48/d9c/l65 to d1a/d48/d9c/d2e/d43/lba 0 2024-02-28T23:07:43.200 INFO:tasks.workunit.client.1.smithi084.stdout:3/720: chown d3/d27 65 1 2024-02-28T23:07:43.202 INFO:tasks.workunit.client.1.smithi084.stdout:6/686: truncate d1/d13/d47/d91/dba/f9b 755620 0 2024-02-28T23:07:43.207 INFO:tasks.workunit.client.1.smithi084.stdout:6/687: fsync d1/d13/d47/f7a 0 2024-02-28T23:07:43.207 INFO:tasks.workunit.client.0.smithi057.stdout:0/629: fdatasync d17/d20/d26/f41 0 2024-02-28T23:07:43.208 INFO:tasks.workunit.client.0.smithi057.stdout:0/630: readlink d17/d20/d4a/l57 0 2024-02-28T23:07:43.209 INFO:tasks.workunit.client.1.smithi084.stdout:6/688: write d1/d13/d2e/d41/f2f [2740451,122289] 0 2024-02-28T23:07:43.209 INFO:tasks.workunit.client.1.smithi084.stdout:8/675: dread d0/d24/d78/f86 [0,4194304] 0 2024-02-28T23:07:43.209 INFO:tasks.workunit.client.0.smithi057.stdout:2/651: truncate d8/fe 3011092 0 2024-02-28T23:07:43.211 INFO:tasks.workunit.client.1.smithi084.stdout:8/676: fsync d0/d24/d9e/fac 0 2024-02-28T23:07:43.212 INFO:tasks.workunit.client.1.smithi084.stdout:6/689: stat d1/d13/d2e/d41/d2c 0 2024-02-28T23:07:43.213 INFO:tasks.workunit.client.0.smithi057.stdout:2/652: dread - d8/d11/d56/f9d zero size 2024-02-28T23:07:43.213 INFO:tasks.workunit.client.1.smithi084.stdout:6/690: fsync d1/d13/d50/d74/faa 0 2024-02-28T23:07:43.217 INFO:tasks.workunit.client.1.smithi084.stdout:8/677: read d0/d9/d30/d33/d71/f75 [2245821,103616] 0 2024-02-28T23:07:43.220 INFO:tasks.workunit.client.0.smithi057.stdout:4/544: dread d1a/d2f/db2/f63 [0,4194304] 0 2024-02-28T23:07:43.221 INFO:tasks.workunit.client.0.smithi057.stdout:7/535: dread d0/d5/d36/d4e/d15/f1a [0,4194304] 0 2024-02-28T23:07:43.222 INFO:tasks.workunit.client.1.smithi084.stdout:7/759: dread db/d10/d18/dc2/fc9 [0,4194304] 0 2024-02-28T23:07:43.222 INFO:tasks.workunit.client.0.smithi057.stdout:3/478: dread d3/d8/d1b/f37 [0,4194304] 0 2024-02-28T23:07:43.222 INFO:tasks.workunit.client.0.smithi057.stdout:9/580: rename dd/de/d14/d48/d76/c81 to dd/de/d24/d72/cbb 0 2024-02-28T23:07:43.223 INFO:tasks.workunit.client.1.smithi084.stdout:3/721: dwrite d3/d27/d7d/db0/d14/d31/d34/fcf [0,4194304] 0 2024-02-28T23:07:43.224 INFO:tasks.workunit.client.0.smithi057.stdout:1/501: dwrite d2/d7/df/db/d3a/f3b [4194304,4194304] 0 2024-02-28T23:07:43.225 INFO:tasks.workunit.client.0.smithi057.stdout:0/631: creat d17/d19/fe1 x:0 0 0 2024-02-28T23:07:43.232 INFO:tasks.workunit.client.1.smithi084.stdout:7/760: creat db/d10/d17/d3b/fe2 x:0 0 0 2024-02-28T23:07:43.233 INFO:tasks.workunit.client.1.smithi084.stdout:8/678: dread d0/d9/d30/d33/d71/d84/f8d [0,4194304] 0 2024-02-28T23:07:43.233 INFO:tasks.workunit.client.0.smithi057.stdout:6/596: unlink d0/d1b/d3e/d3d/d42/d4f/lba 0 2024-02-28T23:07:43.238 INFO:tasks.workunit.client.0.smithi057.stdout:7/536: truncate d0/d5/f39 1897214 0 2024-02-28T23:07:43.239 INFO:tasks.workunit.client.1.smithi084.stdout:7/761: creat db/fe3 x:0 0 0 2024-02-28T23:07:43.241 INFO:tasks.workunit.client.0.smithi057.stdout:0/632: mknod d17/d20/d4a/d78/db5/ce2 0 2024-02-28T23:07:43.242 INFO:tasks.workunit.client.1.smithi084.stdout:8/679: mkdir d0/d9/d30/d33/d71/d84/dd9/de3 0 2024-02-28T23:07:43.245 INFO:tasks.workunit.client.1.smithi084.stdout:7/762: read db/d10/d17/d3b/f4d [1019712,31847] 0 2024-02-28T23:07:43.247 INFO:tasks.workunit.client.0.smithi057.stdout:8/625: link d4/d1e/d59/l85 d4/d1e/d56/d6a/d42/le3 0 2024-02-28T23:07:43.248 INFO:tasks.workunit.client.1.smithi084.stdout:8/680: creat d0/d11/fe4 x:0 0 0 2024-02-28T23:07:43.254 INFO:tasks.workunit.client.0.smithi057.stdout:4/545: mknod d1a/d48/cbb 0 2024-02-28T23:07:43.256 INFO:tasks.workunit.client.1.smithi084.stdout:7/763: dread db/d10/d17/d3b/f3f [0,4194304] 0 2024-02-28T23:07:43.256 INFO:tasks.workunit.client.1.smithi084.stdout:3/722: dwrite d3/d50/d5a/d73/f3f [0,4194304] 0 2024-02-28T23:07:43.256 INFO:tasks.workunit.client.0.smithi057.stdout:3/479: rmdir d3/d20 39 2024-02-28T23:07:43.256 INFO:tasks.workunit.client.1.smithi084.stdout:8/681: link d0/d9/d1b/d38/d43/d51/d54/f9c d0/d9/d1b/d38/d43/dae/db8/fe5 0 2024-02-28T23:07:43.257 INFO:tasks.workunit.client.0.smithi057.stdout:1/502: dread d2/d71/d62/f74 [0,4194304] 0 2024-02-28T23:07:43.258 INFO:tasks.workunit.client.1.smithi084.stdout:7/764: chown db/d10/d17/d64/f81 53924494 1 2024-02-28T23:07:43.259 INFO:tasks.workunit.client.1.smithi084.stdout:7/765: readlink db/dbd/ld2 0 2024-02-28T23:07:43.260 INFO:tasks.workunit.client.1.smithi084.stdout:8/682: chown d0/d24/l37 0 1 2024-02-28T23:07:43.261 INFO:tasks.workunit.client.0.smithi057.stdout:5/549: dwrite d4/d21/d24/d30/d62/faa [0,4194304] 0 2024-02-28T23:07:43.263 INFO:tasks.workunit.client.1.smithi084.stdout:3/723: dread - d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb zero size 2024-02-28T23:07:43.267 INFO:tasks.workunit.client.0.smithi057.stdout:2/653: creat d8/d11/d48/fcc x:0 0 0 2024-02-28T23:07:43.269 INFO:tasks.workunit.client.1.smithi084.stdout:8/683: creat d0/fe6 x:0 0 0 2024-02-28T23:07:43.279 INFO:tasks.workunit.client.0.smithi057.stdout:0/633: creat d17/d20/d23/d47/fe3 x:0 0 0 2024-02-28T23:07:43.282 INFO:tasks.workunit.client.1.smithi084.stdout:3/724: dwrite d3/d50/fd5 [0,4194304] 0 2024-02-28T23:07:43.283 INFO:tasks.workunit.client.0.smithi057.stdout:8/626: mkdir d4/d13/d18/d62/de4 0 2024-02-28T23:07:43.283 INFO:tasks.workunit.client.0.smithi057.stdout:1/503: symlink d2/d71/d2d/d5b/lb8 0 2024-02-28T23:07:43.283 INFO:tasks.workunit.client.1.smithi084.stdout:7/766: dwrite db/d5e/dc6/f95 [0,4194304] 0 2024-02-28T23:07:43.287 INFO:tasks.workunit.client.1.smithi084.stdout:3/725: write d3/d27/d7d/db0/d1a/d21/d7b/fd0 [587531,73839] 0 2024-02-28T23:07:43.289 INFO:tasks.workunit.client.1.smithi084.stdout:8/684: truncate d0/d9/fb 4323410 0 2024-02-28T23:07:43.292 INFO:tasks.workunit.client.0.smithi057.stdout:3/480: dwrite d3/d8/f93 [0,4194304] 0 2024-02-28T23:07:43.292 INFO:tasks.workunit.client.1.smithi084.stdout:3/726: creat d3/d27/d7d/dd9/f110 x:0 0 0 2024-02-28T23:07:43.295 INFO:tasks.workunit.client.0.smithi057.stdout:3/481: fsync d3/d8/d27/f5f 0 2024-02-28T23:07:43.296 INFO:tasks.workunit.client.0.smithi057.stdout:5/550: dwrite d4/d9/d14/f54 [0,4194304] 0 2024-02-28T23:07:43.296 INFO:tasks.workunit.client.1.smithi084.stdout:7/767: link db/l8b db/d10/d18/db7/dbb/le4 0 2024-02-28T23:07:43.300 INFO:tasks.workunit.client.1.smithi084.stdout:3/727: creat d3/d27/f111 x:0 0 0 2024-02-28T23:07:43.301 INFO:tasks.workunit.client.1.smithi084.stdout:8/685: link d0/d24/d9e/laf d0/d24/d78/le7 0 2024-02-28T23:07:43.304 INFO:tasks.workunit.client.0.smithi057.stdout:9/581: dread dd/de/d14/d48/d4b/f54 [0,4194304] 0 2024-02-28T23:07:43.308 INFO:tasks.workunit.client.0.smithi057.stdout:7/537: link d0/d5/d36/d4e/d15/d29/d71/fa8 d0/d5/d36/d4e/d15/d29/d3d/d51/faf 0 2024-02-28T23:07:43.312 INFO:tasks.workunit.client.0.smithi057.stdout:7/538: fdatasync d0/d5/d25/d3e/fa9 0 2024-02-28T23:07:43.312 INFO:tasks.workunit.client.0.smithi057.stdout:8/627: dwrite d4/d1e/d56/fbc [0,4194304] 0 2024-02-28T23:07:43.315 INFO:tasks.workunit.client.0.smithi057.stdout:8/628: write d4/d1e/d56/d6a/d42/fce [995987,123485] 0 2024-02-28T23:07:43.315 INFO:tasks.workunit.client.0.smithi057.stdout:2/654: rename d8/d11/d48/d50/cb4 to d8/d11/d21/d28/d49/d85/d9b/dad/ccd 0 2024-02-28T23:07:43.316 INFO:tasks.workunit.client.1.smithi084.stdout:7/768: dwrite db/d10/d17/d9c/d9f/db1/d36/f4c [0,4194304] 0 2024-02-28T23:07:43.320 INFO:tasks.workunit.client.1.smithi084.stdout:8/686: truncate d0/d24/d78/f86 4038564 0 2024-02-28T23:07:43.320 INFO:tasks.workunit.client.1.smithi084.stdout:7/769: fdatasync db/d10/d17/f42 0 2024-02-28T23:07:43.322 INFO:tasks.workunit.client.1.smithi084.stdout:7/770: fsync db/d5e/dc6/f16 0 2024-02-28T23:07:43.324 INFO:tasks.workunit.client.1.smithi084.stdout:7/771: stat db/d10/d17/d64/f93 0 2024-02-28T23:07:43.325 INFO:tasks.workunit.client.1.smithi084.stdout:8/687: mkdir d0/d9/d1b/d38/d43/d51/de8 0 2024-02-28T23:07:43.326 INFO:tasks.workunit.client.0.smithi057.stdout:3/482: write d3/d4/f12 [7069203,53647] 0 2024-02-28T23:07:43.327 INFO:tasks.workunit.client.1.smithi084.stdout:8/688: readlink d0/d24/d78/le7 0 2024-02-28T23:07:43.330 INFO:tasks.workunit.client.1.smithi084.stdout:7/772: mknod db/d10/d17/d9c/d9f/ce5 0 2024-02-28T23:07:43.332 INFO:tasks.workunit.client.0.smithi057.stdout:9/582: symlink dd/d1b/d40/lbc 0 2024-02-28T23:07:43.332 INFO:tasks.workunit.client.0.smithi057.stdout:3/483: readlink d3/d4/d1d/d47/l92 0 2024-02-28T23:07:43.332 INFO:tasks.workunit.client.1.smithi084.stdout:7/773: write db/d10/d17/f3d [3412812,96296] 0 2024-02-28T23:07:43.335 INFO:tasks.workunit.client.1.smithi084.stdout:7/774: mkdir db/d5e/d87/dc1/de6 0 2024-02-28T23:07:43.336 INFO:tasks.workunit.client.0.smithi057.stdout:7/539: mknod d0/d5/d25/d80/d9f/cb0 0 2024-02-28T23:07:43.336 INFO:tasks.workunit.client.1.smithi084.stdout:7/775: fsync db/d10/f56 0 2024-02-28T23:07:43.338 INFO:tasks.workunit.client.0.smithi057.stdout:2/655: creat d8/d11/d39/d83/fce x:0 0 0 2024-02-28T23:07:43.341 INFO:tasks.workunit.client.1.smithi084.stdout:8/689: write d0/d9/d1b/d38/d43/dae/db8/fd6 [4368573,117055] 0 2024-02-28T23:07:43.341 INFO:tasks.workunit.client.0.smithi057.stdout:8/629: unlink d4/d13/d2c/c33 0 2024-02-28T23:07:43.341 INFO:tasks.workunit.client.0.smithi057.stdout:2/656: stat d8/d11/d21/d28/d49/d85/d92/lab 0 2024-02-28T23:07:43.343 INFO:tasks.workunit.client.1.smithi084.stdout:8/690: creat d0/d9/d30/db6/fe9 x:0 0 0 2024-02-28T23:07:43.348 INFO:tasks.workunit.client.0.smithi057.stdout:0/634: creat d17/d20/d4a/d78/db5/dba/fe4 x:0 0 0 2024-02-28T23:07:43.348 INFO:tasks.workunit.client.0.smithi057.stdout:1/504: truncate d2/f35 2544452 0 2024-02-28T23:07:43.349 INFO:tasks.workunit.client.1.smithi084.stdout:8/691: creat d0/d9/d30/d33/d71/d84/dce/fea x:0 0 0 2024-02-28T23:07:43.349 INFO:tasks.workunit.client.1.smithi084.stdout:7/776: dread db/d10/d17/d64/f93 [0,4194304] 0 2024-02-28T23:07:43.349 INFO:tasks.workunit.client.1.smithi084.stdout:5/721: sync 2024-02-28T23:07:43.350 INFO:tasks.workunit.client.1.smithi084.stdout:7/777: chown db/d10/d17/d3b/f3c 3578094 1 2024-02-28T23:07:43.352 INFO:tasks.workunit.client.0.smithi057.stdout:1/505: write d2/d71/d2d/f46 [565075,8401] 0 2024-02-28T23:07:43.352 INFO:tasks.workunit.client.1.smithi084.stdout:7/778: write db/d10/d17/d3b/f48 [5118456,77996] 0 2024-02-28T23:07:43.355 INFO:tasks.workunit.client.1.smithi084.stdout:5/722: mknod dc/de/df/d16/d34/d3c/d77/c10c 0 2024-02-28T23:07:43.355 INFO:tasks.workunit.client.1.smithi084.stdout:8/692: write d0/d44/d9f/f7e [1231794,54820] 0 2024-02-28T23:07:43.356 INFO:tasks.workunit.client.1.smithi084.stdout:7/779: creat db/d5e/dc3/fe7 x:0 0 0 2024-02-28T23:07:43.358 INFO:tasks.workunit.client.1.smithi084.stdout:5/723: read - dc/de/df/d16/f6a zero size 2024-02-28T23:07:43.360 INFO:tasks.workunit.client.0.smithi057.stdout:9/583: rename dd/de/d24/c2e to dd/de/d24/d72/cbd 0 2024-02-28T23:07:43.364 INFO:tasks.workunit.client.1.smithi084.stdout:7/780: write db/d10/d17/d9c/d9f/db1/f33 [3084349,119076] 0 2024-02-28T23:07:43.366 INFO:tasks.workunit.client.0.smithi057.stdout:0/635: dread d17/d19/f28 [0,4194304] 0 2024-02-28T23:07:43.366 INFO:tasks.workunit.client.1.smithi084.stdout:7/781: creat db/d10/d17/d64/fe8 x:0 0 0 2024-02-28T23:07:43.367 INFO:tasks.workunit.client.0.smithi057.stdout:3/484: mkdir d3/d4/d7/d9a/d9b 0 2024-02-28T23:07:43.374 INFO:tasks.workunit.client.1.smithi084.stdout:7/782: write db/d10/d17/d9c/d9f/fa4 [168292,80755] 0 2024-02-28T23:07:43.377 INFO:tasks.workunit.client.1.smithi084.stdout:8/693: dwrite d0/d24/d4d/f58 [0,4194304] 0 2024-02-28T23:07:43.379 INFO:tasks.workunit.client.1.smithi084.stdout:7/783: chown db/d5e/d80/cd1 24 1 2024-02-28T23:07:43.380 INFO:tasks.workunit.client.0.smithi057.stdout:2/657: unlink d8/d11/d21/d70/fc1 0 2024-02-28T23:07:43.381 INFO:tasks.workunit.client.1.smithi084.stdout:5/724: dwrite dc/de/df/da6/ff7 [0,4194304] 0 2024-02-28T23:07:43.391 INFO:tasks.workunit.client.1.smithi084.stdout:7/784: dread db/d5e/d8a/f9e [0,4194304] 0 2024-02-28T23:07:43.394 INFO:tasks.workunit.client.0.smithi057.stdout:4/546: dread d1a/d48/d97/f21 [0,4194304] 0 2024-02-28T23:07:43.395 INFO:tasks.workunit.client.0.smithi057.stdout:0/636: mknod d17/d20/d23/d4b/ce5 0 2024-02-28T23:07:43.396 INFO:tasks.workunit.client.1.smithi084.stdout:7/785: write db/d5e/dc3/fe7 [814391,28903] 0 2024-02-28T23:07:43.403 INFO:tasks.workunit.client.1.smithi084.stdout:8/694: dwrite d0/d44/d9f/fb0 [0,4194304] 0 2024-02-28T23:07:43.404 INFO:tasks.workunit.client.1.smithi084.stdout:5/725: dwrite dc/de/df/d16/d34/d3c/f5a [0,4194304] 0 2024-02-28T23:07:43.409 INFO:tasks.workunit.client.1.smithi084.stdout:5/726: creat dc/de/df/db0/d104/f10d x:0 0 0 2024-02-28T23:07:43.413 INFO:tasks.workunit.client.1.smithi084.stdout:5/727: fdatasync dc/de/df/d31/d8e/f52 0 2024-02-28T23:07:43.417 INFO:tasks.workunit.client.1.smithi084.stdout:5/728: chown dc/de/df/d16/f45 3 1 2024-02-28T23:07:43.417 INFO:tasks.workunit.client.0.smithi057.stdout:6/597: dwrite d0/d1b/d3e/d3d/d42/d4f/d53/f5b [0,4194304] 0 2024-02-28T23:07:43.421 INFO:tasks.workunit.client.1.smithi084.stdout:2/643: sync 2024-02-28T23:07:43.422 INFO:tasks.workunit.client.0.smithi057.stdout:8/630: link d4/d13/d1a/f58 d4/d13/d18/d62/d66/fe5 0 2024-02-28T23:07:43.422 INFO:tasks.workunit.client.1.smithi084.stdout:7/786: dwrite db/d10/d17/d64/fab [4194304,4194304] 0 2024-02-28T23:07:43.426 INFO:tasks.workunit.client.1.smithi084.stdout:8/695: dwrite d0/d9/d1b/d38/d43/d51/d54/fc3 [0,4194304] 0 2024-02-28T23:07:43.427 INFO:tasks.workunit.client.0.smithi057.stdout:3/485: dread d3/d20/f24 [0,4194304] 0 2024-02-28T23:07:43.428 INFO:tasks.workunit.client.0.smithi057.stdout:0/637: dwrite d17/d20/d26/d48/d55/fbe [0,4194304] 0 2024-02-28T23:07:43.429 INFO:tasks.workunit.client.1.smithi084.stdout:2/644: chown d5/d1f/d42/d74/f76 28 1 2024-02-28T23:07:43.429 INFO:tasks.workunit.client.1.smithi084.stdout:7/787: mkdir db/d10/d18/db7/de9 0 2024-02-28T23:07:43.436 INFO:tasks.workunit.client.0.smithi057.stdout:4/547: mknod d1a/d48/d97/d7f/cbc 0 2024-02-28T23:07:43.440 INFO:tasks.workunit.client.1.smithi084.stdout:2/645: rename d5/d1f/f44 to d5/d7/d2e/fc0 0 2024-02-28T23:07:43.441 INFO:tasks.workunit.client.1.smithi084.stdout:2/646: readlink d5/d1f/l36 0 2024-02-28T23:07:43.444 INFO:tasks.workunit.client.1.smithi084.stdout:7/788: mkdir db/d10/d17/d9c/d9f/db1/d36/dea 0 2024-02-28T23:07:43.445 INFO:tasks.workunit.client.1.smithi084.stdout:8/696: creat d0/d9/d1b/d38/d43/d51/de8/feb x:0 0 0 2024-02-28T23:07:43.446 INFO:tasks.workunit.client.1.smithi084.stdout:7/789: chown db/fd 18610 1 2024-02-28T23:07:43.446 INFO:tasks.workunit.client.0.smithi057.stdout:7/540: creat d0/fb1 x:0 0 0 2024-02-28T23:07:43.447 INFO:tasks.workunit.client.1.smithi084.stdout:7/790: chown db/d10/d17/d9c 54164 1 2024-02-28T23:07:43.448 INFO:tasks.workunit.client.1.smithi084.stdout:2/647: rename d5/d7/d9/c1e to d5/d1f/cc1 0 2024-02-28T23:07:43.449 INFO:tasks.workunit.client.1.smithi084.stdout:8/697: write d0/d9/d1b/d20/f25 [1554285,75370] 0 2024-02-28T23:07:43.451 INFO:tasks.workunit.client.1.smithi084.stdout:7/791: chown db/d10/d17/d9c/d9f/db1/d7a/la3 13513 1 2024-02-28T23:07:43.455 INFO:tasks.workunit.client.1.smithi084.stdout:8/698: creat d0/d9/d1b/d38/d43/dae/fec x:0 0 0 2024-02-28T23:07:43.461 INFO:tasks.workunit.client.0.smithi057.stdout:6/598: mkdir d0/d4/d11/dcb 0 2024-02-28T23:07:43.461 INFO:tasks.workunit.client.1.smithi084.stdout:8/699: stat d0/d9/d30/db6/f89 0 2024-02-28T23:07:43.461 INFO:tasks.workunit.client.1.smithi084.stdout:8/700: chown d0/d24/d4d/f68 4216 1 2024-02-28T23:07:43.461 INFO:tasks.workunit.client.0.smithi057.stdout:8/631: creat d4/d1e/d56/d88/fe6 x:0 0 0 2024-02-28T23:07:43.462 INFO:tasks.workunit.client.1.smithi084.stdout:3/728: dread d3/d27/d7d/db0/d1a/d21/d3d/d43/d61/f65 [0,4194304] 0 2024-02-28T23:07:43.464 INFO:tasks.workunit.client.0.smithi057.stdout:8/632: dread - d4/d13/d2c/d4e/fe2 zero size 2024-02-28T23:07:43.471 INFO:tasks.workunit.client.0.smithi057.stdout:0/638: creat d17/d19/dd2/fe6 x:0 0 0 2024-02-28T23:07:43.473 INFO:tasks.workunit.client.0.smithi057.stdout:7/541: chown d0/d3f/d76/c87 28 1 2024-02-28T23:07:43.474 INFO:tasks.workunit.client.0.smithi057.stdout:0/639: chown d17/d20/d23/d4b/cbc 11 1 2024-02-28T23:07:43.476 INFO:tasks.workunit.client.0.smithi057.stdout:8/633: rename d4/d1e/d56/d6a/d54/d61/fdf to d4/d1e/d56/d6a/d54/d61/fe7 0 2024-02-28T23:07:43.477 INFO:tasks.workunit.client.1.smithi084.stdout:7/792: dwrite db/d10/f60 [4194304,4194304] 0 2024-02-28T23:07:43.477 INFO:tasks.workunit.client.1.smithi084.stdout:8/701: dread d0/d44/d9f/fb0 [0,4194304] 0 2024-02-28T23:07:43.480 INFO:tasks.workunit.client.1.smithi084.stdout:7/793: chown db/d10/d18/db4/fd9 1 1 2024-02-28T23:07:43.489 INFO:tasks.workunit.client.1.smithi084.stdout:3/729: dwrite d3/d27/d7d/db0/d1a/d21/d76/fb2 [0,4194304] 0 2024-02-28T23:07:43.489 INFO:tasks.workunit.client.1.smithi084.stdout:5/729: dread dc/de/df/d31/f7e [0,4194304] 0 2024-02-28T23:07:43.490 INFO:tasks.workunit.client.0.smithi057.stdout:1/506: truncate d2/d7/f41 1454228 0 2024-02-28T23:07:43.494 INFO:tasks.workunit.client.0.smithi057.stdout:5/551: dwrite d4/d21/f2d [0,4194304] 0 2024-02-28T23:07:43.494 INFO:tasks.workunit.client.0.smithi057.stdout:8/634: mkdir d4/d13/d1a/d4a/db3/de8 0 2024-02-28T23:07:43.495 INFO:tasks.workunit.client.0.smithi057.stdout:1/507: dread - d2/d7/df/db/d12/d3f/d47/f8e zero size 2024-02-28T23:07:43.501 INFO:tasks.workunit.client.1.smithi084.stdout:5/730: mkdir dc/de/df/d31/dd1/d5f/d9b/d10e 0 2024-02-28T23:07:43.505 INFO:tasks.workunit.client.1.smithi084.stdout:7/794: dwrite db/fe3 [0,4194304] 0 2024-02-28T23:07:43.508 INFO:tasks.workunit.client.0.smithi057.stdout:3/486: link d3/d8/f3c d3/d8/f9c 0 2024-02-28T23:07:43.509 INFO:tasks.workunit.client.0.smithi057.stdout:4/548: dread d1a/d48/d97/f3e [0,4194304] 0 2024-02-28T23:07:43.509 INFO:tasks.workunit.client.0.smithi057.stdout:3/487: fdatasync d3/d4/f66 0 2024-02-28T23:07:43.511 INFO:tasks.workunit.client.1.smithi084.stdout:5/731: creat dc/de/df/d31/dd1/d5f/d9b/dfe/f10f x:0 0 0 2024-02-28T23:07:43.512 INFO:tasks.workunit.client.1.smithi084.stdout:7/795: creat db/d5e/dc6/dc7/feb x:0 0 0 2024-02-28T23:07:43.516 INFO:tasks.workunit.client.0.smithi057.stdout:7/542: creat d0/d3f/da1/fb2 x:0 0 0 2024-02-28T23:07:43.516 INFO:tasks.workunit.client.1.smithi084.stdout:5/732: fsync dc/de/df/d16/d34/f4d 0 2024-02-28T23:07:43.520 INFO:tasks.workunit.client.1.smithi084.stdout:7/796: rename db/d10/d18/f1b to db/d5e/d80/fec 0 2024-02-28T23:07:43.522 INFO:tasks.workunit.client.1.smithi084.stdout:3/730: truncate d3/fe 2482745 0 2024-02-28T23:07:43.523 INFO:tasks.workunit.client.0.smithi057.stdout:6/599: truncate d0/d4/d25/d60/d6a/f7e 3878103 0 2024-02-28T23:07:43.524 INFO:tasks.workunit.client.0.smithi057.stdout:5/552: creat d4/d9/d14/d63/fb3 x:0 0 0 2024-02-28T23:07:43.525 INFO:tasks.workunit.client.1.smithi084.stdout:7/797: fsync db/d5e/dc6/f44 0 2024-02-28T23:07:43.528 INFO:tasks.workunit.client.0.smithi057.stdout:2/658: dread - d8/d11/d48/d50/f93 zero size 2024-02-28T23:07:43.531 INFO:tasks.workunit.client.1.smithi084.stdout:5/733: link dc/de/df/db0/d104/f10d dc/de/df/f110 0 2024-02-28T23:07:43.532 INFO:tasks.workunit.client.1.smithi084.stdout:3/731: symlink d3/d27/d7d/db0/d14/dee/l112 0 2024-02-28T23:07:43.533 INFO:tasks.workunit.client.1.smithi084.stdout:7/798: creat db/d10/d17/d9c/d9f/db1/db2/fed x:0 0 0 2024-02-28T23:07:43.535 INFO:tasks.workunit.client.1.smithi084.stdout:5/734: mknod dc/de/d83/c111 0 2024-02-28T23:07:43.536 INFO:tasks.workunit.client.0.smithi057.stdout:0/640: creat d17/d20/d26/d48/d7c/d98/fe7 x:0 0 0 2024-02-28T23:07:43.540 INFO:tasks.workunit.client.1.smithi084.stdout:7/799: symlink db/lee 0 2024-02-28T23:07:43.540 INFO:tasks.workunit.client.0.smithi057.stdout:4/549: write f16 [1157487,20054] 0 2024-02-28T23:07:43.545 INFO:tasks.workunit.client.0.smithi057.stdout:5/553: stat d4/d21/d24/d72/la9 0 2024-02-28T23:07:43.547 INFO:tasks.workunit.client.1.smithi084.stdout:8/702: dread d0/d9/d1b/d38/d43/dae/db8/fe5 [0,4194304] 0 2024-02-28T23:07:43.548 INFO:tasks.workunit.client.1.smithi084.stdout:3/732: creat d3/d27/d7d/db0/d1a/d21/d3d/d43/de3/f113 x:0 0 0 2024-02-28T23:07:43.548 INFO:tasks.workunit.client.1.smithi084.stdout:8/703: chown d0/d24/d6e/lbe 1139161092 1 2024-02-28T23:07:43.549 INFO:tasks.workunit.client.0.smithi057.stdout:1/508: write d2/f10 [790194,84174] 0 2024-02-28T23:07:43.549 INFO:tasks.workunit.client.1.smithi084.stdout:5/735: link dc/f25 dc/de/df/d31/dd1/d5f/d9b/d10e/f112 0 2024-02-28T23:07:43.550 INFO:tasks.workunit.client.0.smithi057.stdout:0/641: creat d17/d20/d26/d48/d55/d88/fe8 x:0 0 0 2024-02-28T23:07:43.550 INFO:tasks.workunit.client.0.smithi057.stdout:2/659: write d8/d11/d39/d83/fce [224221,83817] 0 2024-02-28T23:07:43.551 INFO:tasks.workunit.client.1.smithi084.stdout:8/704: chown d0/d9/d30/d33/d71/d84/dc0 423587 1 2024-02-28T23:07:43.552 INFO:tasks.workunit.client.1.smithi084.stdout:0/710: sync 2024-02-28T23:07:43.552 INFO:tasks.workunit.client.1.smithi084.stdout:1/757: sync 2024-02-28T23:07:43.552 INFO:tasks.workunit.client.1.smithi084.stdout:4/668: sync 2024-02-28T23:07:43.553 INFO:tasks.workunit.client.1.smithi084.stdout:9/760: sync 2024-02-28T23:07:43.553 INFO:tasks.workunit.client.1.smithi084.stdout:6/691: sync 2024-02-28T23:07:43.554 INFO:tasks.workunit.client.1.smithi084.stdout:7/800: rename db/d10/d17/d9c/ldf to db/d10/d17/d9c/d9f/lef 0 2024-02-28T23:07:43.555 INFO:tasks.workunit.client.0.smithi057.stdout:2/660: write d8/d11/d48/d50/f66 [5620139,40161] 0 2024-02-28T23:07:43.556 INFO:tasks.workunit.client.0.smithi057.stdout:7/543: rename d0/d5/d36/d4e/d15/d29/f70 to d0/d5/d36/d4e/d15/d29/d71/fb3 0 2024-02-28T23:07:43.556 INFO:tasks.workunit.client.1.smithi084.stdout:4/669: stat d1/d18/d20/d65 0 2024-02-28T23:07:43.557 INFO:tasks.workunit.client.1.smithi084.stdout:0/711: write d1/d8/f3e [2724059,65457] 0 2024-02-28T23:07:43.558 INFO:tasks.workunit.client.0.smithi057.stdout:2/661: readlink d8/d11/d48/l4d 0 2024-02-28T23:07:43.559 INFO:tasks.workunit.client.1.smithi084.stdout:3/733: truncate d3/d27/d7d/db0/d1a/d29/d41/d60/f8b 1344133 0 2024-02-28T23:07:43.559 INFO:tasks.workunit.client.0.smithi057.stdout:8/635: rename d4/d1e to d4/de9 0 2024-02-28T23:07:43.562 INFO:tasks.workunit.client.1.smithi084.stdout:6/692: creat d1/d13/d50/dbd/dc4/ff2 x:0 0 0 2024-02-28T23:07:43.562 INFO:tasks.workunit.client.1.smithi084.stdout:9/761: dread d4/d32/f92 [0,4194304] 0 2024-02-28T23:07:43.563 INFO:tasks.workunit.client.0.smithi057.stdout:5/554: rmdir d4/d7/d58 39 2024-02-28T23:07:43.564 INFO:tasks.workunit.client.0.smithi057.stdout:6/600: dread d0/d17/d20/f5c [4194304,4194304] 0 2024-02-28T23:07:43.564 INFO:tasks.workunit.client.1.smithi084.stdout:5/736: rename dc/de/df/fef to dc/de/f113 0 2024-02-28T23:07:43.565 INFO:tasks.workunit.client.0.smithi057.stdout:5/555: chown d4/f7d 2093074 1 2024-02-28T23:07:43.567 INFO:tasks.workunit.client.0.smithi057.stdout:0/642: mkdir d17/d20/d4a/d78/db5/de9 0 2024-02-28T23:07:43.568 INFO:tasks.workunit.client.1.smithi084.stdout:3/734: read d3/d27/fb7 [2291316,70631] 0 2024-02-28T23:07:43.571 INFO:tasks.workunit.client.1.smithi084.stdout:0/712: fsync d1/d6/db/d35/f8a 0 2024-02-28T23:07:43.572 INFO:tasks.workunit.client.1.smithi084.stdout:8/705: link d0/d24/la5 d0/d9/d1b/d38/d43/dae/led 0 2024-02-28T23:07:43.574 INFO:tasks.workunit.client.0.smithi057.stdout:5/556: read d4/d21/d24/d30/f79 [1658016,73604] 0 2024-02-28T23:07:43.574 INFO:tasks.workunit.client.0.smithi057.stdout:7/544: rmdir d0/d5/d25/d3e 39 2024-02-28T23:07:43.574 INFO:tasks.workunit.client.1.smithi084.stdout:2/648: sync 2024-02-28T23:07:43.575 INFO:tasks.workunit.client.1.smithi084.stdout:5/737: readlink dc/de/df/d31/d8e/l6f 0 2024-02-28T23:07:43.576 INFO:tasks.workunit.client.0.smithi057.stdout:5/557: stat d4/d21/d24/f64 0 2024-02-28T23:07:43.578 INFO:tasks.workunit.client.1.smithi084.stdout:9/762: dread d4/f90 [0,4194304] 0 2024-02-28T23:07:43.579 INFO:tasks.workunit.client.0.smithi057.stdout:1/509: truncate d2/f7e 752560 0 2024-02-28T23:07:43.579 INFO:tasks.workunit.client.1.smithi084.stdout:3/735: unlink d3/d27/d7d/db0/d1a/d45/d5e/fcc 0 2024-02-28T23:07:43.580 INFO:tasks.workunit.client.0.smithi057.stdout:3/488: getdents d3 0 2024-02-28T23:07:43.580 INFO:tasks.workunit.client.0.smithi057.stdout:4/550: dread f14 [0,4194304] 0 2024-02-28T23:07:43.581 INFO:tasks.workunit.client.0.smithi057.stdout:6/601: symlink d0/d1b/d3e/d3d/d7d/lcc 0 2024-02-28T23:07:43.581 INFO:tasks.workunit.client.1.smithi084.stdout:1/758: truncate d12/d1b/d2e/d30/f1c 3129178 0 2024-02-28T23:07:43.582 INFO:tasks.workunit.client.0.smithi057.stdout:3/489: write d3/d4/f94 [46985,102011] 0 2024-02-28T23:07:43.586 INFO:tasks.workunit.client.1.smithi084.stdout:5/738: read dc/de/df/d31/fd7 [3565399,13649] 0 2024-02-28T23:07:43.586 INFO:tasks.workunit.client.1.smithi084.stdout:7/801: dwrite db/d10/d17/fae [0,4194304] 0 2024-02-28T23:07:43.586 INFO:tasks.workunit.client.1.smithi084.stdout:4/670: rename d1/d18/f5f to d1/d18/d20/d24/d2f/d3e/d52/fe0 0 2024-02-28T23:07:43.588 INFO:tasks.workunit.client.0.smithi057.stdout:5/558: stat d4/d9/d14/f43 0 2024-02-28T23:07:43.597 INFO:tasks.workunit.client.0.smithi057.stdout:1/510: creat d2/d7/d4f/d5c/d6b/fb9 x:0 0 0 2024-02-28T23:07:43.598 INFO:tasks.workunit.client.1.smithi084.stdout:6/693: dwrite d1/d13/d50/d74/fe4 [0,4194304] 0 2024-02-28T23:07:43.598 INFO:tasks.workunit.client.0.smithi057.stdout:0/643: dwrite d17/d20/d23/d47/fd8 [0,4194304] 0 2024-02-28T23:07:43.601 INFO:tasks.workunit.client.0.smithi057.stdout:8/636: dwrite d4/de9/d56/d6a/d54/d6b/f6c [0,4194304] 0 2024-02-28T23:07:43.604 INFO:tasks.workunit.client.0.smithi057.stdout:2/662: creat d8/d11/fcf x:0 0 0 2024-02-28T23:07:43.604 INFO:tasks.workunit.client.1.smithi084.stdout:2/649: rename d5/d1f/f41 to d5/d7/d75/fc2 0 2024-02-28T23:07:43.606 INFO:tasks.workunit.client.0.smithi057.stdout:5/559: fdatasync d4/d21/f67 0 2024-02-28T23:07:43.607 INFO:tasks.workunit.client.1.smithi084.stdout:2/650: dread - d5/d1f/d42/fa3 zero size 2024-02-28T23:07:43.608 INFO:tasks.workunit.client.1.smithi084.stdout:8/706: dwrite d0/d44/d9f/fb0 [0,4194304] 0 2024-02-28T23:07:43.610 INFO:tasks.workunit.client.1.smithi084.stdout:8/707: fsync d0/d24/d4d/f68 0 2024-02-28T23:07:43.612 INFO:tasks.workunit.client.0.smithi057.stdout:7/545: symlink d0/d5/d25/d3e/lb4 0 2024-02-28T23:07:43.612 INFO:tasks.workunit.client.0.smithi057.stdout:2/663: dread - d8/d11/d21/d70/fbd zero size 2024-02-28T23:07:43.613 INFO:tasks.workunit.client.0.smithi057.stdout:4/551: mkdir d1a/d48/db0/d4f/db5/dbd 0 2024-02-28T23:07:43.615 INFO:tasks.workunit.client.0.smithi057.stdout:6/602: dwrite d0/d1b/d3e/d3d/d42/fb0 [0,4194304] 0 2024-02-28T23:07:43.616 INFO:tasks.workunit.client.1.smithi084.stdout:0/713: symlink d1/d6/db/d35/d57/le8 0 2024-02-28T23:07:43.617 INFO:tasks.workunit.client.0.smithi057.stdout:1/511: symlink d2/lba 0 2024-02-28T23:07:43.619 INFO:tasks.workunit.client.0.smithi057.stdout:2/664: readlink d8/d11/d15/l4b 0 2024-02-28T23:07:43.619 INFO:tasks.workunit.client.1.smithi084.stdout:5/739: symlink dc/de/df/d16/d34/daa/dae/l114 0 2024-02-28T23:07:43.619 INFO:tasks.workunit.client.1.smithi084.stdout:4/671: truncate d1/d18/f77 204946 0 2024-02-28T23:07:43.630 INFO:tasks.workunit.client.1.smithi084.stdout:7/802: creat db/d10/d17/d64/d77/ff0 x:0 0 0 2024-02-28T23:07:43.631 INFO:tasks.workunit.client.0.smithi057.stdout:8/637: mknod d4/de9/d56/d6a/d54/d61/d8c/dab/cea 0 2024-02-28T23:07:43.632 INFO:tasks.workunit.client.1.smithi084.stdout:1/759: dread d12/d1b/f4a [0,4194304] 0 2024-02-28T23:07:43.632 INFO:tasks.workunit.client.1.smithi084.stdout:7/803: readlink db/d10/d17/l39 0 2024-02-28T23:07:43.633 INFO:tasks.workunit.client.1.smithi084.stdout:1/760: chown d12/f45 1654504 1 2024-02-28T23:07:43.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/665: mknod d8/d11/d39/d83/cd0 0 2024-02-28T23:07:43.637 INFO:tasks.workunit.client.0.smithi057.stdout:2/666: readlink d8/d11/d39/lc8 0 2024-02-28T23:07:43.639 INFO:tasks.workunit.client.0.smithi057.stdout:7/546: truncate d0/d5/d36/d4e/d3b/f5d 692107 0 2024-02-28T23:07:43.643 INFO:tasks.workunit.client.1.smithi084.stdout:8/708: chown d0/d11/fad 7 1 2024-02-28T23:07:43.646 INFO:tasks.workunit.client.1.smithi084.stdout:3/736: creat d3/d27/d7d/db0/d1a/d21/d7b/dbf/f114 x:0 0 0 2024-02-28T23:07:43.646 INFO:tasks.workunit.client.1.smithi084.stdout:0/714: dread - d1/d6/db/d35/d44/f6f zero size 2024-02-28T23:07:43.649 INFO:tasks.workunit.client.0.smithi057.stdout:4/552: rename d1a/d1e/d51/c95 to d1a/d48/db0/d4f/d6e/cbe 0 2024-02-28T23:07:43.651 INFO:tasks.workunit.client.1.smithi084.stdout:0/715: write d1/d8/d69/d6b/f9b [407737,105714] 0 2024-02-28T23:07:43.652 INFO:tasks.workunit.client.0.smithi057.stdout:1/512: mknod d2/cbb 0 2024-02-28T23:07:43.653 INFO:tasks.workunit.client.0.smithi057.stdout:3/490: truncate d3/d8/f93 164856 0 2024-02-28T23:07:43.654 INFO:tasks.workunit.client.1.smithi084.stdout:1/761: mknod d12/d84/d9a/cff 0 2024-02-28T23:07:43.654 INFO:tasks.workunit.client.1.smithi084.stdout:3/737: read d3/d27/d7d/db0/d1a/d45/d5e/d7a/f100 [427470,115528] 0 2024-02-28T23:07:43.656 INFO:tasks.workunit.client.1.smithi084.stdout:3/738: dread - d3/d27/d7d/db0/d1a/d21/f108 zero size 2024-02-28T23:07:43.658 INFO:tasks.workunit.client.0.smithi057.stdout:1/513: write d2/d7/df/db/d12/d3f/d47/f8e [590072,92024] 0 2024-02-28T23:07:43.658 INFO:tasks.workunit.client.1.smithi084.stdout:7/804: symlink db/d5e/d87/dc1/lf1 0 2024-02-28T23:07:43.658 INFO:tasks.workunit.client.1.smithi084.stdout:1/762: write d12/d1b/d91/d9b/da5/dcb/dd4/fe6 [542272,6960] 0 2024-02-28T23:07:43.662 INFO:tasks.workunit.client.0.smithi057.stdout:2/667: creat d8/d11/d39/d83/fd1 x:0 0 0 2024-02-28T23:07:43.663 INFO:tasks.workunit.client.0.smithi057.stdout:0/644: dread d17/d20/d4a/d78/db5/fcb [0,4194304] 0 2024-02-28T23:07:43.663 INFO:tasks.workunit.client.1.smithi084.stdout:7/805: creat db/d5e/d80/ff2 x:0 0 0 2024-02-28T23:07:43.664 INFO:tasks.workunit.client.1.smithi084.stdout:3/739: stat d3/d27/d7d/db0/df/l15 0 2024-02-28T23:07:43.669 INFO:tasks.workunit.client.0.smithi057.stdout:5/560: write d4/d9/d16/d4e/f8b [4702592,39389] 0 2024-02-28T23:07:43.671 INFO:tasks.workunit.client.1.smithi084.stdout:3/740: mknod d3/d50/d97/c115 0 2024-02-28T23:07:43.671 INFO:tasks.workunit.client.0.smithi057.stdout:5/561: stat d4/d9/d16/d52 0 2024-02-28T23:07:43.677 INFO:tasks.workunit.client.0.smithi057.stdout:2/668: mknod d8/d11/d21/da4/cd2 0 2024-02-28T23:07:43.677 INFO:tasks.workunit.client.1.smithi084.stdout:3/741: mkdir d3/d50/d116 0 2024-02-28T23:07:43.677 INFO:tasks.workunit.client.0.smithi057.stdout:3/491: creat d3/d8/d27/d70/f9d x:0 0 0 2024-02-28T23:07:43.678 INFO:tasks.workunit.client.1.smithi084.stdout:0/716: dwrite d1/d8/d18/f9f [4194304,4194304] 0 2024-02-28T23:07:43.679 INFO:tasks.workunit.client.0.smithi057.stdout:0/645: mkdir d17/d20/d23/d47/d60/dea 0 2024-02-28T23:07:43.679 INFO:tasks.workunit.client.0.smithi057.stdout:1/514: creat d2/d7/df/db/d3a/d66/fbc x:0 0 0 2024-02-28T23:07:43.679 INFO:tasks.workunit.client.0.smithi057.stdout:8/638: link d4/d13/c73 d4/de9/d59/ceb 0 2024-02-28T23:07:43.680 INFO:tasks.workunit.client.1.smithi084.stdout:3/742: write d3/f7 [2080344,49738] 0 2024-02-28T23:07:43.684 INFO:tasks.workunit.client.1.smithi084.stdout:1/763: dwrite d12/d1b/d2e/d30/d77/da0/fed [0,4194304] 0 2024-02-28T23:07:43.684 INFO:tasks.workunit.client.1.smithi084.stdout:9/763: write d4/d6/d15/d12/f97 [917502,32545] 0 2024-02-28T23:07:43.685 INFO:tasks.workunit.client.1.smithi084.stdout:6/694: write d1/d13/d2e/f44 [4557674,87149] 0 2024-02-28T23:07:43.688 INFO:tasks.workunit.client.0.smithi057.stdout:0/646: rmdir d17/d20/d26 39 2024-02-28T23:07:43.690 INFO:tasks.workunit.client.1.smithi084.stdout:1/764: readlink d12/d1b/d2e/d32/ldc 0 2024-02-28T23:07:43.690 INFO:tasks.workunit.client.0.smithi057.stdout:7/547: getdents d0/d3f 0 2024-02-28T23:07:43.695 INFO:tasks.workunit.client.0.smithi057.stdout:1/515: rename d2/d7/df/db/d3a/d5a/ca7 to d2/d7/df/db/d3a/d8f/cbd 0 2024-02-28T23:07:43.696 INFO:tasks.workunit.client.0.smithi057.stdout:6/603: dread d0/d4/d25/d60/f97 [0,4194304] 0 2024-02-28T23:07:43.696 INFO:tasks.workunit.client.0.smithi057.stdout:2/669: creat d8/d11/d39/dbc/fd3 x:0 0 0 2024-02-28T23:07:43.699 INFO:tasks.workunit.client.0.smithi057.stdout:9/584: sync 2024-02-28T23:07:43.701 INFO:tasks.workunit.client.1.smithi084.stdout:3/743: rename d3/d27/d7d/db0/d1a/d21 to d3/d50/d117 0 2024-02-28T23:07:43.703 INFO:tasks.workunit.client.0.smithi057.stdout:7/548: fdatasync d0/d5/d36/d4e/d15/f6a 0 2024-02-28T23:07:43.704 INFO:tasks.workunit.client.1.smithi084.stdout:6/695: creat d1/d88/ff3 x:0 0 0 2024-02-28T23:07:43.706 INFO:tasks.workunit.client.0.smithi057.stdout:0/647: rename d17/d20/d4a/l9e to d17/d20/d23/d47/d60/d6f/leb 0 2024-02-28T23:07:43.709 INFO:tasks.workunit.client.1.smithi084.stdout:5/740: dwrite dc/de/df/d31/dd1/fb2 [0,4194304] 0 2024-02-28T23:07:43.709 INFO:tasks.workunit.client.0.smithi057.stdout:5/562: getdents d4/d21 0 2024-02-28T23:07:43.710 INFO:tasks.workunit.client.0.smithi057.stdout:9/585: mknod dd/de/d24/d63/d77/cbe 0 2024-02-28T23:07:43.711 INFO:tasks.workunit.client.1.smithi084.stdout:6/696: write d1/d13/d50/d74/fb7 [130550,52269] 0 2024-02-28T23:07:43.714 INFO:tasks.workunit.client.1.smithi084.stdout:9/764: creat d4/d6/d15/d75/d9d/dd0/ff8 x:0 0 0 2024-02-28T23:07:43.714 INFO:tasks.workunit.client.1.smithi084.stdout:7/806: truncate db/d10/d17/d3b/f48 2319208 0 2024-02-28T23:07:43.714 INFO:tasks.workunit.client.0.smithi057.stdout:1/516: creat d2/d7/df/db/d12/d19/d48/fbe x:0 0 0 2024-02-28T23:07:43.716 INFO:tasks.workunit.client.0.smithi057.stdout:3/492: link d3/d4/d7/c8a d3/d4/d7/c9e 0 2024-02-28T23:07:43.716 INFO:tasks.workunit.client.0.smithi057.stdout:7/549: stat d0/d5/d36/d4e/d15/d29/d71/fb3 0 2024-02-28T23:07:43.718 INFO:tasks.workunit.client.0.smithi057.stdout:9/586: mkdir dd/de/d14/d48/d76/dbf 0 2024-02-28T23:07:43.718 INFO:tasks.workunit.client.1.smithi084.stdout:4/672: dwrite d1/d18/d20/d65/d8e/d55/f69 [0,4194304] 0 2024-02-28T23:07:43.719 INFO:tasks.workunit.client.1.smithi084.stdout:0/717: dwrite d1/d6/db/d35/d82/fda [0,4194304] 0 2024-02-28T23:07:43.719 INFO:tasks.workunit.client.0.smithi057.stdout:6/604: creat d0/d17/fcd x:0 0 0 2024-02-28T23:07:43.720 INFO:tasks.workunit.client.0.smithi057.stdout:1/517: chown d2/d71/d2d/d5b/faf 3 1 2024-02-28T23:07:43.721 INFO:tasks.workunit.client.1.smithi084.stdout:7/807: write db/d10/d17/d3b/f9b [1730644,3500] 0 2024-02-28T23:07:43.721 INFO:tasks.workunit.client.1.smithi084.stdout:1/765: creat d12/d1b/d2e/d52/dd7/f100 x:0 0 0 2024-02-28T23:07:43.723 INFO:tasks.workunit.client.0.smithi057.stdout:9/587: stat dd/de/d14/f1e 0 2024-02-28T23:07:43.723 INFO:tasks.workunit.client.0.smithi057.stdout:7/550: creat d0/d3f/da1/fb5 x:0 0 0 2024-02-28T23:07:43.724 INFO:tasks.workunit.client.0.smithi057.stdout:3/493: write d3/d8/d27/d70/f74 [1009891,125853] 0 2024-02-28T23:07:43.724 INFO:tasks.workunit.client.0.smithi057.stdout:5/563: link d4/d9/d16/d2c/c3c d4/d21/d24/d30/d62/cb4 0 2024-02-28T23:07:43.727 INFO:tasks.workunit.client.1.smithi084.stdout:5/741: creat dc/d51/f115 x:0 0 0 2024-02-28T23:07:43.728 INFO:tasks.workunit.client.0.smithi057.stdout:1/518: write d2/d7/df/db/d3a/d8f/f94 [770348,80356] 0 2024-02-28T23:07:43.729 INFO:tasks.workunit.client.0.smithi057.stdout:4/553: dread d1a/d48/db0/f5d [0,4194304] 0 2024-02-28T23:07:43.734 INFO:tasks.workunit.client.0.smithi057.stdout:3/494: creat d3/d4f/d4c/d91/f9f x:0 0 0 2024-02-28T23:07:43.734 INFO:tasks.workunit.client.1.smithi084.stdout:5/742: stat dc/de/df/d31/dd1/f91 0 2024-02-28T23:07:43.734 INFO:tasks.workunit.client.1.smithi084.stdout:9/765: getdents d4/d6/d15/d3c 0 2024-02-28T23:07:43.735 INFO:tasks.workunit.client.0.smithi057.stdout:7/551: mknod d0/cb6 0 2024-02-28T23:07:43.736 INFO:tasks.workunit.client.0.smithi057.stdout:3/495: chown d3/d4f/f7d 28 1 2024-02-28T23:07:43.736 INFO:tasks.workunit.client.0.smithi057.stdout:6/605: symlink d0/d4/d25/d60/dc9/lce 0 2024-02-28T23:07:43.737 INFO:tasks.workunit.client.0.smithi057.stdout:8/639: write d4/f17 [1015464,12667] 0 2024-02-28T23:07:43.740 INFO:tasks.workunit.client.1.smithi084.stdout:0/718: rename d1/d8/d18/l67 to d1/d6/db/d35/d82/le9 0 2024-02-28T23:07:43.740 INFO:tasks.workunit.client.1.smithi084.stdout:1/766: truncate d12/d1b/d2e/d30/f58 4083285 0 2024-02-28T23:07:43.741 INFO:tasks.workunit.client.0.smithi057.stdout:4/554: mknod d1a/d48/d97/d7f/cbf 0 2024-02-28T23:07:43.743 INFO:tasks.workunit.client.1.smithi084.stdout:1/767: stat d12/d1b/d2e/f57 0 2024-02-28T23:07:43.746 INFO:tasks.workunit.client.0.smithi057.stdout:2/670: truncate d8/d11/d21/d28/f73 2456879 0 2024-02-28T23:07:43.748 INFO:tasks.workunit.client.0.smithi057.stdout:2/671: stat d8/d11/d15/l5f 0 2024-02-28T23:07:43.749 INFO:tasks.workunit.client.1.smithi084.stdout:2/651: sync 2024-02-28T23:07:43.749 INFO:tasks.workunit.client.1.smithi084.stdout:7/808: dwrite db/dbd/fe1 [0,4194304] 0 2024-02-28T23:07:43.749 INFO:tasks.workunit.client.1.smithi084.stdout:1/768: stat d12/d1b/d2e/d30/d50/fac 0 2024-02-28T23:07:43.751 INFO:tasks.workunit.client.0.smithi057.stdout:5/564: dread d4/d21/d24/d30/d62/faa [0,4194304] 0 2024-02-28T23:07:43.752 INFO:tasks.workunit.client.1.smithi084.stdout:0/719: creat d1/d6/db/d35/d44/d92/fea x:0 0 0 2024-02-28T23:07:43.752 INFO:tasks.workunit.client.0.smithi057.stdout:7/552: mknod d0/d5/d4a/cb7 0 2024-02-28T23:07:43.756 INFO:tasks.workunit.client.1.smithi084.stdout:3/744: dread d3/d50/d117/d3d/d43/f48 [0,4194304] 0 2024-02-28T23:07:43.757 INFO:tasks.workunit.client.0.smithi057.stdout:5/565: chown d4/d9/d16/d8f/c94 4333555 1 2024-02-28T23:07:43.757 INFO:tasks.workunit.client.0.smithi057.stdout:7/553: write d0/fb1 [505985,23912] 0 2024-02-28T23:07:43.757 INFO:tasks.workunit.client.0.smithi057.stdout:5/566: dread - d4/f92 zero size 2024-02-28T23:07:43.759 INFO:tasks.workunit.client.1.smithi084.stdout:2/652: fdatasync d5/d7/d12/d24/d2a/d35/fa2 0 2024-02-28T23:07:43.762 INFO:tasks.workunit.client.0.smithi057.stdout:9/588: rename dd/de/d14/d48/d4b/d41/f70 to dd/d46/fc0 0 2024-02-28T23:07:43.767 INFO:tasks.workunit.client.0.smithi057.stdout:2/672: unlink d8/d11/d15/l60 0 2024-02-28T23:07:43.768 INFO:tasks.workunit.client.1.smithi084.stdout:2/653: mknod d5/d7/d12/d24/cc3 0 2024-02-28T23:07:43.771 INFO:tasks.workunit.client.1.smithi084.stdout:3/745: truncate d3/d27/d7d/db0/d1a/d29/d41/d60/f8b 2235547 0 2024-02-28T23:07:43.774 INFO:tasks.workunit.client.1.smithi084.stdout:7/809: dwrite f9 [0,4194304] 0 2024-02-28T23:07:43.774 INFO:tasks.workunit.client.0.smithi057.stdout:0/648: dwrite d17/d20/d23/d47/d60/fd3 [4194304,4194304] 0 2024-02-28T23:07:43.775 INFO:tasks.workunit.client.1.smithi084.stdout:0/720: getdents d1/d6/db/d35/d57 0 2024-02-28T23:07:43.775 INFO:tasks.workunit.client.1.smithi084.stdout:2/654: creat d5/dd/fc4 x:0 0 0 2024-02-28T23:07:43.777 INFO:tasks.workunit.client.0.smithi057.stdout:7/554: unlink d0/d5/d25/d80/d9f/la4 0 2024-02-28T23:07:43.778 INFO:tasks.workunit.client.1.smithi084.stdout:6/697: dread d1/d13/d2e/fd3 [0,4194304] 0 2024-02-28T23:07:43.782 INFO:tasks.workunit.client.0.smithi057.stdout:7/555: read - d0/d5/d36/d4e/d3b/f9d zero size 2024-02-28T23:07:43.783 INFO:tasks.workunit.client.1.smithi084.stdout:0/721: truncate d1/d6/db/d37/f96 4304239 0 2024-02-28T23:07:43.783 INFO:tasks.workunit.client.1.smithi084.stdout:4/673: dread d1/d18/d20/d14/d30/d47/f82 [0,4194304] 0 2024-02-28T23:07:43.784 INFO:tasks.workunit.client.1.smithi084.stdout:7/810: creat db/d10/d17/d3b/ff3 x:0 0 0 2024-02-28T23:07:43.784 INFO:tasks.workunit.client.0.smithi057.stdout:8/640: rename d4/de9/d37/l72 to d4/d13/d7f/lec 0 2024-02-28T23:07:43.785 INFO:tasks.workunit.client.1.smithi084.stdout:1/769: dread d12/d1b/d63/f3a [0,4194304] 0 2024-02-28T23:07:43.786 INFO:tasks.workunit.client.0.smithi057.stdout:9/589: fsync dd/d1b/d40/d44/f62 0 2024-02-28T23:07:43.789 INFO:tasks.workunit.client.1.smithi084.stdout:7/811: fsync db/d10/f5d 0 2024-02-28T23:07:43.789 INFO:tasks.workunit.client.0.smithi057.stdout:4/555: creat d1a/d48/d97/fc0 x:0 0 0 2024-02-28T23:07:43.792 INFO:tasks.workunit.client.0.smithi057.stdout:0/649: stat c13 0 2024-02-28T23:07:43.796 INFO:tasks.workunit.client.0.smithi057.stdout:1/519: dread d2/d7/df/f52 [0,4194304] 0 2024-02-28T23:07:43.796 INFO:tasks.workunit.client.0.smithi057.stdout:7/556: creat d0/d3f/d76/fb8 x:0 0 0 2024-02-28T23:07:43.797 INFO:tasks.workunit.client.1.smithi084.stdout:7/812: write db/d5e/dc6/f4f [2710528,59194] 0 2024-02-28T23:07:43.797 INFO:tasks.workunit.client.1.smithi084.stdout:9/766: dread d4/f60 [4194304,4194304] 0 2024-02-28T23:07:43.801 INFO:tasks.workunit.client.0.smithi057.stdout:7/557: chown d0/d5/d36/d4e/d15/d29/d71/fa8 195743 1 2024-02-28T23:07:43.802 INFO:tasks.workunit.client.0.smithi057.stdout:2/673: mknod d8/d11/d39/d7b/d8b/da2/cd4 0 2024-02-28T23:07:43.802 INFO:tasks.workunit.client.1.smithi084.stdout:4/674: creat d1/d18/d20/d65/d8e/d55/dc3/fe1 x:0 0 0 2024-02-28T23:07:43.803 INFO:tasks.workunit.client.1.smithi084.stdout:7/813: creat db/d10/d17/d64/d77/ff4 x:0 0 0 2024-02-28T23:07:43.804 INFO:tasks.workunit.client.0.smithi057.stdout:8/641: symlink d4/d13/d18/d62/de4/led 0 2024-02-28T23:07:43.805 INFO:tasks.workunit.client.0.smithi057.stdout:4/556: mknod d1a/d48/d9c/d2e/d8f/cc1 0 2024-02-28T23:07:43.805 INFO:tasks.workunit.client.1.smithi084.stdout:9/767: symlink d4/d6/d15/lf9 0 2024-02-28T23:07:43.805 INFO:tasks.workunit.client.0.smithi057.stdout:1/520: creat d2/d71/d2d/fbf x:0 0 0 2024-02-28T23:07:43.807 INFO:tasks.workunit.client.1.smithi084.stdout:1/770: dwrite d12/d1b/d2e/d32/fc9 [4194304,4194304] 0 2024-02-28T23:07:43.809 INFO:tasks.workunit.client.0.smithi057.stdout:6/606: rename d0/d1b/d3e/d3d/d7d/f9b to d0/d4/d25/d60/fcf 0 2024-02-28T23:07:43.811 INFO:tasks.workunit.client.0.smithi057.stdout:8/642: chown d4/d13/d1a/f8f 220 1 2024-02-28T23:07:43.811 INFO:tasks.workunit.client.1.smithi084.stdout:4/675: chown d1/d18/d20/d24/ldc 4 1 2024-02-28T23:07:43.814 INFO:tasks.workunit.client.1.smithi084.stdout:9/768: write d4/d6/d15/d12/d41/fe5 [921413,83809] 0 2024-02-28T23:07:43.816 INFO:tasks.workunit.client.0.smithi057.stdout:2/674: readlink d8/d11/d15/l7e 0 2024-02-28T23:07:43.817 INFO:tasks.workunit.client.1.smithi084.stdout:1/771: readlink d12/d1b/d2e/d52/d61/lec 0 2024-02-28T23:07:43.817 INFO:tasks.workunit.client.0.smithi057.stdout:0/650: unlink d17/d20/d26/d48/d7c/d98/da6/fbb 0 2024-02-28T23:07:43.820 INFO:tasks.workunit.client.1.smithi084.stdout:6/698: dwrite d1/d9/d1d/d1e/fc8 [0,4194304] 0 2024-02-28T23:07:43.820 INFO:tasks.workunit.client.1.smithi084.stdout:7/814: rename db/d10/d17/d64/f75 to db/ff5 0 2024-02-28T23:07:43.821 INFO:tasks.workunit.client.0.smithi057.stdout:0/651: stat d17/d20/d23/c50 0 2024-02-28T23:07:43.821 INFO:tasks.workunit.client.0.smithi057.stdout:1/521: write d2/d71/d2d/f31 [2067232,9507] 0 2024-02-28T23:07:43.821 INFO:tasks.workunit.client.0.smithi057.stdout:3/496: dwrite d3/d4/d7/f10 [4194304,4194304] 0 2024-02-28T23:07:43.826 INFO:tasks.workunit.client.1.smithi084.stdout:1/772: write d12/d1b/d2e/d32/f47 [7258809,45481] 0 2024-02-28T23:07:43.826 INFO:tasks.workunit.client.1.smithi084.stdout:3/746: write d3/d50/d117/d3d/f4e [2331714,98842] 0 2024-02-28T23:07:43.826 INFO:tasks.workunit.client.1.smithi084.stdout:1/773: readlink d12/d1b/lbd 0 2024-02-28T23:07:43.827 INFO:tasks.workunit.client.0.smithi057.stdout:0/652: fdatasync d17/d19/f8f 0 2024-02-28T23:07:43.828 INFO:tasks.workunit.client.0.smithi057.stdout:2/675: stat d8/d11/d21/d28/d49/l55 0 2024-02-28T23:07:43.830 INFO:tasks.workunit.client.1.smithi084.stdout:9/769: symlink d4/d6/lfa 0 2024-02-28T23:07:43.831 INFO:tasks.workunit.client.0.smithi057.stdout:2/676: chown d8/d11/d15/f72 6 1 2024-02-28T23:07:43.834 INFO:tasks.workunit.client.1.smithi084.stdout:7/815: rmdir db/d5e/dc3 39 2024-02-28T23:07:43.834 INFO:tasks.workunit.client.0.smithi057.stdout:6/607: mkdir d0/d1b/d3e/d3d/d42/d4f/dd0 0 2024-02-28T23:07:43.836 INFO:tasks.workunit.client.0.smithi057.stdout:3/497: symlink d3/d20/la0 0 2024-02-28T23:07:43.838 INFO:tasks.workunit.client.0.smithi057.stdout:0/653: mknod d17/d19/cec 0 2024-02-28T23:07:43.839 INFO:tasks.workunit.client.1.smithi084.stdout:4/676: dwrite d1/d18/d20/d24/f2a [0,4194304] 0 2024-02-28T23:07:43.840 INFO:tasks.workunit.client.0.smithi057.stdout:1/522: mkdir d2/d7/df/db/d12/d19/d48/dc0 0 2024-02-28T23:07:43.840 INFO:tasks.workunit.client.0.smithi057.stdout:7/558: link d0/d5/d25/d80/d9f/cb0 d0/d5/d36/d4e/cb9 0 2024-02-28T23:07:43.840 INFO:tasks.workunit.client.1.smithi084.stdout:8/709: sync 2024-02-28T23:07:43.843 INFO:tasks.workunit.client.0.smithi057.stdout:1/523: fdatasync d2/d7/df/db/f28 0 2024-02-28T23:07:43.847 INFO:tasks.workunit.client.1.smithi084.stdout:3/747: mknod d3/d27/d7d/db0/d1a/d29/c118 0 2024-02-28T23:07:43.848 INFO:tasks.workunit.client.0.smithi057.stdout:1/524: truncate d2/d7/df/db/d3a/d2a/f7a 972194 0 2024-02-28T23:07:43.849 INFO:tasks.workunit.client.1.smithi084.stdout:9/770: symlink d4/d6/d15/db3/lfb 0 2024-02-28T23:07:43.850 INFO:tasks.workunit.client.1.smithi084.stdout:7/816: dread db/d10/d17/fae [0,4194304] 0 2024-02-28T23:07:43.850 INFO:tasks.workunit.client.0.smithi057.stdout:1/525: write d2/d71/d2d/d5b/f7c [451116,46462] 0 2024-02-28T23:07:43.851 INFO:tasks.workunit.client.0.smithi057.stdout:2/677: creat d8/d11/d21/d28/d41/dbb/fd5 x:0 0 0 2024-02-28T23:07:43.853 INFO:tasks.workunit.client.1.smithi084.stdout:0/722: dread d1/d6/db/d37/f96 [0,4194304] 0 2024-02-28T23:07:43.853 INFO:tasks.workunit.client.0.smithi057.stdout:1/526: chown d2/d33/daa/lb6 111014 1 2024-02-28T23:07:43.854 INFO:tasks.workunit.client.0.smithi057.stdout:9/590: dread dd/d1b/d40/d44/f7a [0,4194304] 0 2024-02-28T23:07:43.854 INFO:tasks.workunit.client.0.smithi057.stdout:2/678: write d8/d11/d48/fcc [548857,63563] 0 2024-02-28T23:07:43.855 INFO:tasks.workunit.client.0.smithi057.stdout:1/527: dread - d2/d71/d2d/fbf zero size 2024-02-28T23:07:43.858 INFO:tasks.workunit.client.1.smithi084.stdout:0/723: dread - d1/d8/d5f/d93/fe2 zero size 2024-02-28T23:07:43.863 INFO:tasks.workunit.client.0.smithi057.stdout:9/591: mkdir dd/de/d24/d63/dc1 0 2024-02-28T23:07:43.864 INFO:tasks.workunit.client.1.smithi084.stdout:6/699: rename d1/d13/d2e/d41/dbb/l45 to d1/d9/d1d/d1e/lf4 0 2024-02-28T23:07:43.864 INFO:tasks.workunit.client.0.smithi057.stdout:8/643: write d4/de9/d59/f91 [1968898,61360] 0 2024-02-28T23:07:43.866 INFO:tasks.workunit.client.1.smithi084.stdout:3/748: dread d3/d27/d7d/db0/f72 [0,4194304] 0 2024-02-28T23:07:43.869 INFO:tasks.workunit.client.1.smithi084.stdout:9/771: creat d4/d6/d15/d12/d41/dea/ffc x:0 0 0 2024-02-28T23:07:43.870 INFO:tasks.workunit.client.1.smithi084.stdout:2/655: dread d5/d7/d12/d24/d2a/d53/f5f [0,4194304] 0 2024-02-28T23:07:43.871 INFO:tasks.workunit.client.1.smithi084.stdout:0/724: rename d1/d8/d69/d6b/d85/da5/ld1 to d1/d6/db/d35/d4a/leb 0 2024-02-28T23:07:43.872 INFO:tasks.workunit.client.0.smithi057.stdout:2/679: creat d8/d11/d21/d28/d41/fd6 x:0 0 0 2024-02-28T23:07:43.872 INFO:tasks.workunit.client.0.smithi057.stdout:4/557: dwrite d1a/d48/d97/f32 [0,4194304] 0 2024-02-28T23:07:43.872 INFO:tasks.workunit.client.1.smithi084.stdout:4/677: truncate d1/d18/d20/d24/f8b 1344367 0 2024-02-28T23:07:43.874 INFO:tasks.workunit.client.1.smithi084.stdout:5/743: sync 2024-02-28T23:07:43.874 INFO:tasks.workunit.client.0.smithi057.stdout:1/528: unlink d2/d7/df/db/d12/d19/d48/fbe 0 2024-02-28T23:07:43.875 INFO:tasks.workunit.client.0.smithi057.stdout:4/558: fsync f15 0 2024-02-28T23:07:43.875 INFO:tasks.workunit.client.0.smithi057.stdout:0/654: dwrite d17/d20/d26/d48/d7c/d98/da6/fdf [0,4194304] 0 2024-02-28T23:07:43.876 INFO:tasks.workunit.client.1.smithi084.stdout:6/700: symlink d1/d13/d50/d74/lf5 0 2024-02-28T23:07:43.878 INFO:tasks.workunit.client.1.smithi084.stdout:3/749: rename d3/d50/f66 to d3/d27/d7d/db0/d1a/df9/f119 0 2024-02-28T23:07:43.880 INFO:tasks.workunit.client.1.smithi084.stdout:0/725: rmdir d1/d8/d69/d6b 39 2024-02-28T23:07:43.883 INFO:tasks.workunit.client.1.smithi084.stdout:4/678: fsync d1/d18/d20/d24/f84 0 2024-02-28T23:07:43.885 INFO:tasks.workunit.client.1.smithi084.stdout:3/750: write d3/d50/d97/fd2 [129054,80602] 0 2024-02-28T23:07:43.885 INFO:tasks.workunit.client.0.smithi057.stdout:6/608: rename d0/d4/d25/f9c to d0/fd1 0 2024-02-28T23:07:43.885 INFO:tasks.workunit.client.0.smithi057.stdout:9/592: truncate dd/f31 3205007 0 2024-02-28T23:07:43.885 INFO:tasks.workunit.client.0.smithi057.stdout:8/644: dwrite d4/d13/d1a/f89 [0,4194304] 0 2024-02-28T23:07:43.891 INFO:tasks.workunit.client.0.smithi057.stdout:6/609: read d0/fd1 [818526,124453] 0 2024-02-28T23:07:43.893 INFO:tasks.workunit.client.1.smithi084.stdout:9/772: unlink l1 0 2024-02-28T23:07:43.896 INFO:tasks.workunit.client.0.smithi057.stdout:0/655: creat d17/d20/db6/fed x:0 0 0 2024-02-28T23:07:43.898 INFO:tasks.workunit.client.1.smithi084.stdout:6/701: chown d1/d9/d1d/db6/lc5 75 1 2024-02-28T23:07:43.900 INFO:tasks.workunit.client.0.smithi057.stdout:1/529: truncate d2/d71/d2d/f4c 4017386 0 2024-02-28T23:07:43.904 INFO:tasks.workunit.client.0.smithi057.stdout:1/530: chown d2/d7/df/db/d12/d3f/l4e 1815263 1 2024-02-28T23:07:43.907 INFO:tasks.workunit.client.1.smithi084.stdout:8/710: dread d0/fdb [0,4194304] 0 2024-02-28T23:07:43.907 INFO:tasks.workunit.client.0.smithi057.stdout:3/498: dread d3/d4/d7/f84 [0,4194304] 0 2024-02-28T23:07:43.910 INFO:tasks.workunit.client.1.smithi084.stdout:8/711: stat d0/f53 0 2024-02-28T23:07:43.910 INFO:tasks.workunit.client.0.smithi057.stdout:3/499: write d3/d4/d1d/d47/d62/f8c [612323,36956] 0 2024-02-28T23:07:43.911 INFO:tasks.workunit.client.1.smithi084.stdout:1/774: dwrite d12/d84/d9a/fc0 [0,4194304] 0 2024-02-28T23:07:43.911 INFO:tasks.workunit.client.0.smithi057.stdout:9/593: mkdir dd/de/d14/dc2 0 2024-02-28T23:07:43.914 INFO:tasks.workunit.client.1.smithi084.stdout:4/679: dwrite d1/f1d [0,4194304] 0 2024-02-28T23:07:43.914 INFO:tasks.workunit.client.0.smithi057.stdout:9/594: read dd/f12 [828766,11059] 0 2024-02-28T23:07:43.915 INFO:tasks.workunit.client.1.smithi084.stdout:1/775: chown d12/d1b/d2e/d52/f6d 3856144 1 2024-02-28T23:07:43.925 INFO:tasks.workunit.client.0.smithi057.stdout:3/500: dread d3/d8/d27/f81 [0,4194304] 0 2024-02-28T23:07:43.927 INFO:tasks.workunit.client.1.smithi084.stdout:4/680: read d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/f54 [2164664,64256] 0 2024-02-28T23:07:43.930 INFO:tasks.workunit.client.1.smithi084.stdout:2/656: dwrite d5/d7/d12/d24/d3a/d48/f5a [0,4194304] 0 2024-02-28T23:07:43.932 INFO:tasks.workunit.client.0.smithi057.stdout:0/656: creat d17/d19/dd2/ddb/fee x:0 0 0 2024-02-28T23:07:43.934 INFO:tasks.workunit.client.1.smithi084.stdout:2/657: write d5/d7/d2e/d4a/f54 [2666281,101852] 0 2024-02-28T23:07:43.942 INFO:tasks.workunit.client.1.smithi084.stdout:3/751: unlink d3/d27/d7d/db0/d14/d31/d34/cd7 0 2024-02-28T23:07:43.943 INFO:tasks.workunit.client.0.smithi057.stdout:7/559: getdents d0/d5/d36/d4e/d15/d29 0 2024-02-28T23:07:43.943 INFO:tasks.workunit.client.1.smithi084.stdout:6/702: mkdir d1/d13/d2e/d41/d4d/d6c/df6 0 2024-02-28T23:07:43.946 INFO:tasks.workunit.client.1.smithi084.stdout:8/712: mknod d0/d9/d1b/d38/d43/cee 0 2024-02-28T23:07:43.946 INFO:tasks.workunit.client.1.smithi084.stdout:1/776: mknod d12/d1b/d2e/d52/d61/c101 0 2024-02-28T23:07:43.947 INFO:tasks.workunit.client.0.smithi057.stdout:8/645: fdatasync d4/d13/d2c/d4e/f87 0 2024-02-28T23:07:43.952 INFO:tasks.workunit.client.1.smithi084.stdout:1/777: chown d12/d1b/d91/d9b/fde 30792 1 2024-02-28T23:07:43.952 INFO:tasks.workunit.client.1.smithi084.stdout:3/752: mkdir d3/d27/d7d/db0/d1a/d29/d7e/d11a 0 2024-02-28T23:07:43.955 INFO:tasks.workunit.client.1.smithi084.stdout:4/681: mknod d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/ce2 0 2024-02-28T23:07:43.956 INFO:tasks.workunit.client.0.smithi057.stdout:2/680: write d8/d11/d48/d50/f93 [568679,48494] 0 2024-02-28T23:07:43.960 INFO:tasks.workunit.client.1.smithi084.stdout:8/713: symlink d0/d9/d1b/d38/d43/d51/de8/lef 0 2024-02-28T23:07:43.960 INFO:tasks.workunit.client.1.smithi084.stdout:3/753: readlink d3/d27/d7d/db0/d1a/d45/d5e/d7a/l91 0 2024-02-28T23:07:43.961 INFO:tasks.workunit.client.1.smithi084.stdout:0/726: dread d1/d8/d69/d6b/fbe [0,4194304] 0 2024-02-28T23:07:43.961 INFO:tasks.workunit.client.1.smithi084.stdout:8/714: dread - d0/d24/d78/fd5 zero size 2024-02-28T23:07:43.961 INFO:tasks.workunit.client.0.smithi057.stdout:3/501: dread d3/f61 [0,4194304] 0 2024-02-28T23:07:43.961 INFO:tasks.workunit.client.1.smithi084.stdout:1/778: truncate fc 6982101 0 2024-02-28T23:07:43.963 INFO:tasks.workunit.client.1.smithi084.stdout:9/773: getdents d4/d32/d3f/d89 0 2024-02-28T23:07:43.964 INFO:tasks.workunit.client.1.smithi084.stdout:9/774: stat d4/d6/d15/d12/f3d 0 2024-02-28T23:07:43.965 INFO:tasks.workunit.client.0.smithi057.stdout:7/560: symlink d0/d5/d4a/lba 0 2024-02-28T23:07:43.966 INFO:tasks.workunit.client.1.smithi084.stdout:6/703: creat d1/d13/d47/d91/ff7 x:0 0 0 2024-02-28T23:07:43.966 INFO:tasks.workunit.client.1.smithi084.stdout:3/754: rmdir d3/d50/d117/d76 39 2024-02-28T23:07:43.968 INFO:tasks.workunit.client.1.smithi084.stdout:8/715: stat d0/d9/d30/d33/d71/d84/dc0/ldd 0 2024-02-28T23:07:43.969 INFO:tasks.workunit.client.1.smithi084.stdout:3/755: chown d3/d50/d5a/d73/dc8/cf0 91656005 1 2024-02-28T23:07:43.969 INFO:tasks.workunit.client.1.smithi084.stdout:1/779: mknod d12/d1b/d91/c102 0 2024-02-28T23:07:43.971 INFO:tasks.workunit.client.0.smithi057.stdout:7/561: dread d0/d5/d36/d4e/d15/d29/f2a [4194304,4194304] 0 2024-02-28T23:07:43.973 INFO:tasks.workunit.client.1.smithi084.stdout:1/780: write d12/d1b/d2e/d32/f47 [371939,62822] 0 2024-02-28T23:07:43.975 INFO:tasks.workunit.client.1.smithi084.stdout:4/682: symlink d1/d18/d20/d24/le3 0 2024-02-28T23:07:43.983 INFO:tasks.workunit.client.0.smithi057.stdout:7/562: truncate d0/d5/d36/d4e/d15/d29/f90 252445 0 2024-02-28T23:07:43.983 INFO:tasks.workunit.client.0.smithi057.stdout:4/559: truncate d1a/d48/d97/f1c 1053493 0 2024-02-28T23:07:43.984 INFO:tasks.workunit.client.0.smithi057.stdout:0/657: rename d17/d20/d23/d47/d60/d6f/leb to d17/d20/d26/d48/d7c/lef 0 2024-02-28T23:07:43.984 INFO:tasks.workunit.client.0.smithi057.stdout:8/646: dread d4/d13/f2b [0,4194304] 0 2024-02-28T23:07:43.984 INFO:tasks.workunit.client.0.smithi057.stdout:2/681: dwrite d8/d11/d21/f7d [0,4194304] 0 2024-02-28T23:07:43.985 INFO:tasks.workunit.client.1.smithi084.stdout:1/781: creat d12/d1b/d2e/d30/d50/da8/dd9/df0/f103 x:0 0 0 2024-02-28T23:07:43.988 INFO:tasks.workunit.client.1.smithi084.stdout:6/704: dwrite d1/d13/d50/d74/faa [0,4194304] 0 2024-02-28T23:07:43.988 INFO:tasks.workunit.client.1.smithi084.stdout:4/683: mknod d1/d18/d20/d24/ddb/ce4 0 2024-02-28T23:07:43.992 INFO:tasks.workunit.client.0.smithi057.stdout:9/595: link dd/f3c dd/d1b/d40/fc3 0 2024-02-28T23:07:43.994 INFO:tasks.workunit.client.1.smithi084.stdout:6/705: unlink d1/d13/c3c 0 2024-02-28T23:07:43.995 INFO:tasks.workunit.client.1.smithi084.stdout:1/782: fdatasync d12/da2/fe4 0 2024-02-28T23:07:43.997 INFO:tasks.workunit.client.1.smithi084.stdout:5/744: truncate dc/de/df/d31/f7e 1475905 0 2024-02-28T23:07:43.999 INFO:tasks.workunit.client.0.smithi057.stdout:1/531: dwrite d2/d7/d4f/f63 [0,4194304] 0 2024-02-28T23:07:44.002 INFO:tasks.workunit.client.0.smithi057.stdout:8/647: unlink d4/l40 0 2024-02-28T23:07:44.005 INFO:tasks.workunit.client.1.smithi084.stdout:6/706: read - d1/d13/d50/f55 zero size 2024-02-28T23:07:44.007 INFO:tasks.workunit.client.0.smithi057.stdout:4/560: creat d1a/d48/d9c/d2e/fc2 x:0 0 0 2024-02-28T23:07:44.010 INFO:tasks.workunit.client.0.smithi057.stdout:2/682: dwrite d8/d11/d21/f36 [8388608,4194304] 0 2024-02-28T23:07:44.011 INFO:tasks.workunit.client.0.smithi057.stdout:0/658: rename d17/d19/l1d to d17/d20/d26/d30/lf0 0 2024-02-28T23:07:44.011 INFO:tasks.workunit.client.1.smithi084.stdout:9/775: dread d4/d6/f17 [0,4194304] 0 2024-02-28T23:07:44.012 INFO:tasks.workunit.client.0.smithi057.stdout:2/683: fsync d8/d11/d15/f20 0 2024-02-28T23:07:44.015 INFO:tasks.workunit.client.1.smithi084.stdout:5/745: symlink dc/de/df/d31/dd1/d5f/d9b/l116 0 2024-02-28T23:07:44.015 INFO:tasks.workunit.client.0.smithi057.stdout:2/684: write d8/d11/d48/d50/f93 [74737,95710] 0 2024-02-28T23:07:44.016 INFO:tasks.workunit.client.0.smithi057.stdout:8/648: read d4/d13/d18/f23 [1544978,84263] 0 2024-02-28T23:07:44.022 INFO:tasks.workunit.client.0.smithi057.stdout:8/649: truncate d4/de9/d56/d6a/d54/d61/d8c/fa0 127768 0 2024-02-28T23:07:44.025 INFO:tasks.workunit.client.0.smithi057.stdout:8/650: write d4/d13/d18/d62/d66/fc4 [1264770,45287] 0 2024-02-28T23:07:44.029 INFO:tasks.workunit.client.1.smithi084.stdout:6/707: getdents d1/d13/d2e/d41/d4d 0 2024-02-28T23:07:44.032 INFO:tasks.workunit.client.1.smithi084.stdout:3/756: write d3/d50/fbb [655601,68933] 0 2024-02-28T23:07:44.032 INFO:tasks.workunit.client.1.smithi084.stdout:0/727: truncate d1/d8/d69/db2/fdc 707496 0 2024-02-28T23:07:44.036 INFO:tasks.workunit.client.0.smithi057.stdout:3/502: truncate d3/d8/f29 2000252 0 2024-02-28T23:07:44.036 INFO:tasks.workunit.client.1.smithi084.stdout:5/746: link dc/de/df/d31/d8e/c64 dc/de/df/d16/d34/daa/c117 0 2024-02-28T23:07:44.039 INFO:tasks.workunit.client.1.smithi084.stdout:0/728: write d1/d6/d1b/f58 [656700,115278] 0 2024-02-28T23:07:44.041 INFO:tasks.workunit.client.0.smithi057.stdout:1/532: mknod d2/d71/d2d/d39/da3/db5/cc1 0 2024-02-28T23:07:44.042 INFO:tasks.workunit.client.0.smithi057.stdout:8/651: dwrite d4/d13/d1a/f34 [0,4194304] 0 2024-02-28T23:07:44.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/533: read - d2/d71/f9e zero size 2024-02-28T23:07:44.045 INFO:tasks.workunit.client.0.smithi057.stdout:3/503: read d3/d8/d27/f5f [2332670,80862] 0 2024-02-28T23:07:44.047 INFO:tasks.workunit.client.1.smithi084.stdout:5/747: creat dc/de/df/db0/f118 x:0 0 0 2024-02-28T23:07:44.048 INFO:tasks.workunit.client.0.smithi057.stdout:4/561: stat d1a/d48/f79 0 2024-02-28T23:07:44.050 INFO:tasks.workunit.client.1.smithi084.stdout:3/757: rename d3/d27/d7d/db0/d1a/d29/c118 to d3/d27/d7d/db0/d1a/df9/dfe/c11b 0 2024-02-28T23:07:44.053 INFO:tasks.workunit.client.0.smithi057.stdout:7/563: write d0/d5/d36/d4e/d15/d29/d3d/d51/f72 [908725,115165] 0 2024-02-28T23:07:44.053 INFO:tasks.workunit.client.0.smithi057.stdout:2/685: creat d8/d11/d39/d7b/d8b/fd7 x:0 0 0 2024-02-28T23:07:44.056 INFO:tasks.workunit.client.1.smithi084.stdout:8/716: dwrite d0/d9/f4c [0,4194304] 0 2024-02-28T23:07:44.063 INFO:tasks.workunit.client.0.smithi057.stdout:1/534: fdatasync d2/d71/d2d/f49 0 2024-02-28T23:07:44.064 INFO:tasks.workunit.client.1.smithi084.stdout:0/729: rename d1/d6/db/d76/fb0 to d1/d8/da1/dd0/fec 0 2024-02-28T23:07:44.067 INFO:tasks.workunit.client.1.smithi084.stdout:1/783: dwrite d12/da2/fab [0,4194304] 0 2024-02-28T23:07:44.068 INFO:tasks.workunit.client.1.smithi084.stdout:8/717: read - d0/d9/d30/db6/fe9 zero size 2024-02-28T23:07:44.068 INFO:tasks.workunit.client.1.smithi084.stdout:4/684: dwrite d1/d18/d20/d14/d30/d47/f82 [0,4194304] 0 2024-02-28T23:07:44.068 INFO:tasks.workunit.client.0.smithi057.stdout:1/535: chown d2/d7/df/db/d12/d19/d65/f98 179951704 1 2024-02-28T23:07:44.072 INFO:tasks.workunit.client.0.smithi057.stdout:0/659: mknod d17/dda/cf1 0 2024-02-28T23:07:44.073 INFO:tasks.workunit.client.0.smithi057.stdout:7/564: dwrite d0/d5/d36/d4e/f19 [0,4194304] 0 2024-02-28T23:07:44.074 INFO:tasks.workunit.client.1.smithi084.stdout:3/758: mknod d3/d50/d117/c11c 0 2024-02-28T23:07:44.076 INFO:tasks.workunit.client.0.smithi057.stdout:7/565: readlink d0/d5/d4a/lba 0 2024-02-28T23:07:44.077 INFO:tasks.workunit.client.1.smithi084.stdout:8/718: stat d0/d9/d30/d33/d71/d84/dd9 0 2024-02-28T23:07:44.078 INFO:tasks.workunit.client.1.smithi084.stdout:4/685: chown d1/d18/d20/d24/ddb 8122781 1 2024-02-28T23:07:44.080 INFO:tasks.workunit.client.1.smithi084.stdout:6/708: dwrite d1/d9/d1d/d1e/d33/f39 [4194304,4194304] 0 2024-02-28T23:07:44.086 INFO:tasks.workunit.client.0.smithi057.stdout:4/562: unlink d1a/d2f/db2/d49/c84 0 2024-02-28T23:07:44.087 INFO:tasks.workunit.client.1.smithi084.stdout:8/719: stat d0/d9/d30/db6/fd7 0 2024-02-28T23:07:44.087 INFO:tasks.workunit.client.1.smithi084.stdout:1/784: write d12/d1b/d2e/d30/d50/da8/dd9/df0/f103 [276929,94255] 0 2024-02-28T23:07:44.088 INFO:tasks.workunit.client.0.smithi057.stdout:9/596: rename dd/de/d89 to dd/d46/dc4 0 2024-02-28T23:07:44.088 INFO:tasks.workunit.client.0.smithi057.stdout:0/660: dread d17/d20/db6/fc9 [0,4194304] 0 2024-02-28T23:07:44.088 INFO:tasks.workunit.client.0.smithi057.stdout:1/536: mkdir d2/d90/dc2 0 2024-02-28T23:07:44.089 INFO:tasks.workunit.client.1.smithi084.stdout:6/709: mknod d1/d13/d50/d74/d81/cf8 0 2024-02-28T23:07:44.090 INFO:tasks.workunit.client.1.smithi084.stdout:3/759: truncate d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb 173051 0 2024-02-28T23:07:44.092 INFO:tasks.workunit.client.1.smithi084.stdout:8/720: mkdir d0/d24/d78/df0 0 2024-02-28T23:07:44.096 INFO:tasks.workunit.client.1.smithi084.stdout:8/721: stat d0/d9/d1b/d20/l85 0 2024-02-28T23:07:44.097 INFO:tasks.workunit.client.1.smithi084.stdout:4/686: dread d1/d18/d20/f11 [0,4194304] 0 2024-02-28T23:07:44.098 INFO:tasks.workunit.client.1.smithi084.stdout:8/722: readlink d0/l57 0 2024-02-28T23:07:44.099 INFO:tasks.workunit.client.0.smithi057.stdout:4/563: mkdir d1a/d1e/d51/dc3 0 2024-02-28T23:07:44.101 INFO:tasks.workunit.client.1.smithi084.stdout:6/710: mkdir d1/d13/d50/dbd/df9 0 2024-02-28T23:07:44.102 INFO:tasks.workunit.client.0.smithi057.stdout:7/566: write d0/d5/d36/d4e/d15/d29/f2a [4343316,84908] 0 2024-02-28T23:07:44.103 INFO:tasks.workunit.client.0.smithi057.stdout:0/661: symlink d17/d19/d4d/lf2 0 2024-02-28T23:07:44.106 INFO:tasks.workunit.client.0.smithi057.stdout:1/537: creat d2/d7/d4f/db4/fc3 x:0 0 0 2024-02-28T23:07:44.107 INFO:tasks.workunit.client.0.smithi057.stdout:9/597: creat dd/de/d24/d63/dc1/fc5 x:0 0 0 2024-02-28T23:07:44.109 INFO:tasks.workunit.client.1.smithi084.stdout:8/723: symlink d0/d24/d6e/lf1 0 2024-02-28T23:07:44.112 INFO:tasks.workunit.client.1.smithi084.stdout:4/687: creat d1/d18/d20/d24/d2f/d3e/fe5 x:0 0 0 2024-02-28T23:07:44.112 INFO:tasks.workunit.client.1.smithi084.stdout:1/785: dwrite d12/d84/fbb [0,4194304] 0 2024-02-28T23:07:44.114 INFO:tasks.workunit.client.1.smithi084.stdout:6/711: dread - d1/d13/d50/fa6 zero size 2024-02-28T23:07:44.117 INFO:tasks.workunit.client.1.smithi084.stdout:4/688: readlink d1/d18/d20/d24/d2f/d3e/d52/d48/l59 0 2024-02-28T23:07:44.117 INFO:tasks.workunit.client.1.smithi084.stdout:1/786: rmdir d12/d1b/d3b 39 2024-02-28T23:07:44.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/567: sync 2024-02-28T23:07:44.118 INFO:tasks.workunit.client.0.smithi057.stdout:6/610: sync 2024-02-28T23:07:44.119 INFO:tasks.workunit.client.0.smithi057.stdout:0/662: mkdir d17/d20/df3 0 2024-02-28T23:07:44.120 INFO:tasks.workunit.client.1.smithi084.stdout:6/712: mknod d1/d13/d50/d74/cfa 0 2024-02-28T23:07:44.122 INFO:tasks.workunit.client.1.smithi084.stdout:6/713: chown d1/f24 1933831351 1 2024-02-28T23:07:44.123 INFO:tasks.workunit.client.0.smithi057.stdout:3/504: write d3/d8/d1b/f44 [1779455,69676] 0 2024-02-28T23:07:44.123 INFO:tasks.workunit.client.0.smithi057.stdout:7/567: symlink d0/d5/d36/d4e/lbb 0 2024-02-28T23:07:44.124 INFO:tasks.workunit.client.0.smithi057.stdout:5/568: chown d4/d9/d14/d3b/d61/d69/d95/f98 464198 1 2024-02-28T23:07:44.131 INFO:tasks.workunit.client.1.smithi084.stdout:6/714: truncate d1/d9/ddc/fad 16468 0 2024-02-28T23:07:44.132 INFO:tasks.workunit.client.0.smithi057.stdout:2/686: dread d8/d11/d48/d50/f66 [0,4194304] 0 2024-02-28T23:07:44.132 INFO:tasks.workunit.client.1.smithi084.stdout:3/760: write d3/d50/d117/d3d/d43/f48 [1599276,116864] 0 2024-02-28T23:07:44.133 INFO:tasks.workunit.client.0.smithi057.stdout:3/505: mknod d3/d4/d7/d9a/ca1 0 2024-02-28T23:07:44.136 INFO:tasks.workunit.client.1.smithi084.stdout:0/730: dwrite d1/d6/f13 [0,4194304] 0 2024-02-28T23:07:44.137 INFO:tasks.workunit.client.1.smithi084.stdout:3/761: dread - d3/dae/fe7 zero size 2024-02-28T23:07:44.140 INFO:tasks.workunit.client.1.smithi084.stdout:4/689: link d1/c13 d1/d18/d20/d24/d2f/d3e/d52/ce6 0 2024-02-28T23:07:44.143 INFO:tasks.workunit.client.1.smithi084.stdout:3/762: symlink d3/d50/d5a/d73/dc8/l11d 0 2024-02-28T23:07:44.143 INFO:tasks.workunit.client.0.smithi057.stdout:8/652: truncate d4/d13/d18/f31 1555239 0 2024-02-28T23:07:44.144 INFO:tasks.workunit.client.1.smithi084.stdout:1/787: dwrite d12/d1b/d2e/d30/d50/da8/dd9/df0/ff5 [0,4194304] 0 2024-02-28T23:07:44.150 INFO:tasks.workunit.client.0.smithi057.stdout:4/564: dwrite d1a/d48/d97/d7f/fb8 [4194304,4194304] 0 2024-02-28T23:07:44.150 INFO:tasks.workunit.client.1.smithi084.stdout:5/748: dread dc/de/df/d16/f1b [4194304,4194304] 0 2024-02-28T23:07:44.153 INFO:tasks.workunit.client.1.smithi084.stdout:3/763: rmdir d3/d27/d7d/db0/d1a/d29/d41/d60/d98 39 2024-02-28T23:07:44.154 INFO:tasks.workunit.client.1.smithi084.stdout:1/788: chown d12/d1b/d2e/d30/f1c 89249127 1 2024-02-28T23:07:44.155 INFO:tasks.workunit.client.1.smithi084.stdout:4/690: mkdir d1/d18/d20/d24/d2f/d3e/d52/d48/dd7/de7 0 2024-02-28T23:07:44.160 INFO:tasks.workunit.client.0.smithi057.stdout:9/598: dwrite dd/d1b/f6c [0,4194304] 0 2024-02-28T23:07:44.164 INFO:tasks.workunit.client.1.smithi084.stdout:3/764: dread - d3/d27/d4b/deb/ffa zero size 2024-02-28T23:07:44.164 INFO:tasks.workunit.client.1.smithi084.stdout:4/691: mkdir d1/d18/d20/d24/d2f/d3e/d52/d48/d73/de8 0 2024-02-28T23:07:44.164 INFO:tasks.workunit.client.1.smithi084.stdout:1/789: write d12/d1b/d91/fe0 [369152,115860] 0 2024-02-28T23:07:44.167 INFO:tasks.workunit.client.1.smithi084.stdout:8/724: dread d0/d9/d1b/dc1/fc7 [0,4194304] 0 2024-02-28T23:07:44.171 INFO:tasks.workunit.client.1.smithi084.stdout:5/749: creat dc/de/df/d16/f119 x:0 0 0 2024-02-28T23:07:44.173 INFO:tasks.workunit.client.0.smithi057.stdout:1/538: write d2/f7e [960817,62746] 0 2024-02-28T23:07:44.174 INFO:tasks.workunit.client.0.smithi057.stdout:6/611: write d0/d4/f50 [788091,125482] 0 2024-02-28T23:07:44.176 INFO:tasks.workunit.client.1.smithi084.stdout:4/692: read d1/d18/d20/d14/d30/f81 [3534921,54092] 0 2024-02-28T23:07:44.177 INFO:tasks.workunit.client.1.smithi084.stdout:1/790: fsync d12/d1b/d2e/d52/d5b/f68 0 2024-02-28T23:07:44.178 INFO:tasks.workunit.client.1.smithi084.stdout:0/731: dwrite d1/d6/db/d35/d4a/d56/fc1 [0,4194304] 0 2024-02-28T23:07:44.181 INFO:tasks.workunit.client.1.smithi084.stdout:0/732: stat d1/d8/f75 0 2024-02-28T23:07:44.187 INFO:tasks.workunit.client.1.smithi084.stdout:8/725: mknod d0/d9/d1b/d38/d43/d51/d54/cf2 0 2024-02-28T23:07:44.190 INFO:tasks.workunit.client.1.smithi084.stdout:8/726: fsync d0/d44/d9f/fb0 0 2024-02-28T23:07:44.192 INFO:tasks.workunit.client.1.smithi084.stdout:5/750: dwrite dc/de/df/da6/ff7 [0,4194304] 0 2024-02-28T23:07:44.194 INFO:tasks.workunit.client.1.smithi084.stdout:3/765: mknod d3/d27/d7d/db0/d1a/d29/d41/d60/d98/c11e 0 2024-02-28T23:07:44.195 INFO:tasks.workunit.client.1.smithi084.stdout:0/733: fdatasync d1/d8/f3e 0 2024-02-28T23:07:44.203 INFO:tasks.workunit.client.0.smithi057.stdout:3/506: symlink d3/d4/d7/la2 0 2024-02-28T23:07:44.205 INFO:tasks.workunit.client.0.smithi057.stdout:3/507: write d3/d8/d1b/f4a [849383,5966] 0 2024-02-28T23:07:44.206 INFO:tasks.workunit.client.1.smithi084.stdout:5/751: rename dc/de/df/d31/dd1/f69 to dc/de/df/db0/f11a 0 2024-02-28T23:07:44.206 INFO:tasks.workunit.client.1.smithi084.stdout:0/734: creat d1/d6/d59/d70/fed x:0 0 0 2024-02-28T23:07:44.206 INFO:tasks.workunit.client.1.smithi084.stdout:8/727: rmdir d0/d24/d83 39 2024-02-28T23:07:44.209 INFO:tasks.workunit.client.0.smithi057.stdout:3/508: truncate d3/d4f/d4c/d91/f9f 179026 0 2024-02-28T23:07:44.209 INFO:tasks.workunit.client.0.smithi057.stdout:0/663: truncate d17/d20/d26/f41 980092 0 2024-02-28T23:07:44.210 INFO:tasks.workunit.client.1.smithi084.stdout:0/735: dread - d1/d6/db/d35/d44/f6f zero size 2024-02-28T23:07:44.210 INFO:tasks.workunit.client.0.smithi057.stdout:8/653: unlink d4/d13/d1a/d4a/db3/lcf 0 2024-02-28T23:07:44.211 INFO:tasks.workunit.client.0.smithi057.stdout:3/509: chown d3/d4/d1d/d47/l87 747849 1 2024-02-28T23:07:44.212 INFO:tasks.workunit.client.1.smithi084.stdout:3/766: symlink d3/d27/d7d/db0/d14/l11f 0 2024-02-28T23:07:44.214 INFO:tasks.workunit.client.0.smithi057.stdout:3/510: fdatasync d3/d4f/f83 0 2024-02-28T23:07:44.215 INFO:tasks.workunit.client.0.smithi057.stdout:5/569: rename d4/d21/d24/d30/d62/cb4 to d4/d9/d14/d3b/cb5 0 2024-02-28T23:07:44.216 INFO:tasks.workunit.client.0.smithi057.stdout:4/565: mknod d1a/d48/d9c/d2e/d8f/cc4 0 2024-02-28T23:07:44.216 INFO:tasks.workunit.client.1.smithi084.stdout:8/728: symlink d0/d9/d1b/dc1/lf3 0 2024-02-28T23:07:44.217 INFO:tasks.workunit.client.0.smithi057.stdout:5/570: stat d4/d32/l99 0 2024-02-28T23:07:44.218 INFO:tasks.workunit.client.0.smithi057.stdout:6/612: symlink d0/d1b/d3e/d40/ld2 0 2024-02-28T23:07:44.219 INFO:tasks.workunit.client.0.smithi057.stdout:0/664: symlink d17/d19/dd2/ddb/lf4 0 2024-02-28T23:07:44.221 INFO:tasks.workunit.client.0.smithi057.stdout:9/599: symlink dd/d1b/d40/d44/da5/lc6 0 2024-02-28T23:07:44.223 INFO:tasks.workunit.client.0.smithi057.stdout:0/665: dread - d17/d20/d23/fb8 zero size 2024-02-28T23:07:44.226 INFO:tasks.workunit.client.1.smithi084.stdout:4/693: rmdir d1/d18/d20/d24/d2f/d3e/d52 39 2024-02-28T23:07:44.227 INFO:tasks.workunit.client.1.smithi084.stdout:5/752: dwrite dc/de/df/d31/fcb [0,4194304] 0 2024-02-28T23:07:44.228 INFO:tasks.workunit.client.0.smithi057.stdout:0/666: write d17/d19/d4d/f62 [2189608,95697] 0 2024-02-28T23:07:44.233 INFO:tasks.workunit.client.0.smithi057.stdout:7/568: dwrite d0/d5/d25/d3e/f54 [4194304,4194304] 0 2024-02-28T23:07:44.233 INFO:tasks.workunit.client.0.smithi057.stdout:0/667: chown d17/d19/d4d/lf2 0 1 2024-02-28T23:07:44.234 INFO:tasks.workunit.client.1.smithi084.stdout:3/767: rename d3/d27/d7d/db0/d14/c10f to d3/d27/d7d/db0/d1a/d29/c120 0 2024-02-28T23:07:44.237 INFO:tasks.workunit.client.1.smithi084.stdout:5/753: unlink dc/de/df/d6e/ff3 0 2024-02-28T23:07:44.239 INFO:tasks.workunit.client.0.smithi057.stdout:1/539: rename d2/d71/d2d/d5b/c5d to d2/d7/df/db/d12/d3f/d47/cc4 0 2024-02-28T23:07:44.242 INFO:tasks.workunit.client.1.smithi084.stdout:4/694: chown d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/db4 2880 1 2024-02-28T23:07:44.243 INFO:tasks.workunit.client.0.smithi057.stdout:8/654: mkdir d4/d13/d18/da7/de1/dee 0 2024-02-28T23:07:44.246 INFO:tasks.workunit.client.0.smithi057.stdout:4/566: mkdir d1a/d48/d9c/d68/dc5 0 2024-02-28T23:07:44.246 INFO:tasks.workunit.client.1.smithi084.stdout:5/754: mknod dc/de/df/d16/d34/d3c/d98/c11b 0 2024-02-28T23:07:44.248 INFO:tasks.workunit.client.1.smithi084.stdout:8/729: creat d0/d9/d1b/d38/ff4 x:0 0 0 2024-02-28T23:07:44.250 INFO:tasks.workunit.client.1.smithi084.stdout:4/695: unlink d1/d18/d20/d24/d2f/d3e/d52/f9b 0 2024-02-28T23:07:44.253 INFO:tasks.workunit.client.1.smithi084.stdout:5/755: read dc/de/df/d31/dd1/d5f/fe2 [1339245,110557] 0 2024-02-28T23:07:44.256 INFO:tasks.workunit.client.1.smithi084.stdout:1/791: dwrite d12/f23 [0,4194304] 0 2024-02-28T23:07:44.259 INFO:tasks.workunit.client.1.smithi084.stdout:3/768: dwrite d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb [0,4194304] 0 2024-02-28T23:07:44.265 INFO:tasks.workunit.client.1.smithi084.stdout:4/696: creat d1/d18/d20/d14/d30/fe9 x:0 0 0 2024-02-28T23:07:44.268 INFO:tasks.workunit.client.1.smithi084.stdout:5/756: rename dc/de/df/d31/dd1/da4/dee to dc/de/d83/d11c 0 2024-02-28T23:07:44.270 INFO:tasks.workunit.client.1.smithi084.stdout:3/769: fsync d3/d50/d117/d3d/faa 0 2024-02-28T23:07:44.273 INFO:tasks.workunit.client.0.smithi057.stdout:1/540: rmdir d2/d33/d54 39 2024-02-28T23:07:44.274 INFO:tasks.workunit.client.1.smithi084.stdout:7/817: sync 2024-02-28T23:07:44.274 INFO:tasks.workunit.client.1.smithi084.stdout:0/736: dwrite d1/d8/da1/dd0/fec [0,4194304] 0 2024-02-28T23:07:44.278 INFO:tasks.workunit.client.1.smithi084.stdout:1/792: mknod d12/d1b/d2e/d30/d50/da8/dd9/def/c104 0 2024-02-28T23:07:44.278 INFO:tasks.workunit.client.1.smithi084.stdout:7/818: readlink db/l4e 0 2024-02-28T23:07:44.278 INFO:tasks.workunit.client.0.smithi057.stdout:8/655: chown d4/d13/d2c/c64 2696782 1 2024-02-28T23:07:44.279 INFO:tasks.workunit.client.0.smithi057.stdout:5/571: dread d4/d9/f1c [0,4194304] 0 2024-02-28T23:07:44.280 INFO:tasks.workunit.client.1.smithi084.stdout:4/697: unlink d1/d18/d20/d14/d30/d47/c80 0 2024-02-28T23:07:44.280 INFO:tasks.workunit.client.0.smithi057.stdout:4/567: creat d1a/d48/db0/db1/fc6 x:0 0 0 2024-02-28T23:07:44.283 INFO:tasks.workunit.client.1.smithi084.stdout:1/793: stat d12/d1b/d2e/d30/d77/feb 0 2024-02-28T23:07:44.284 INFO:tasks.workunit.client.1.smithi084.stdout:8/730: dwrite d0/d24/d78/fbd [0,4194304] 0 2024-02-28T23:07:44.284 INFO:tasks.workunit.client.1.smithi084.stdout:3/770: truncate d3/d27/d7d/dd9/f110 354783 0 2024-02-28T23:07:44.285 INFO:tasks.workunit.client.1.smithi084.stdout:0/737: mkdir d1/d6/db/d76/dee 0 2024-02-28T23:07:44.291 INFO:tasks.workunit.client.1.smithi084.stdout:4/698: write d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/fd6 [788529,89795] 0 2024-02-28T23:07:44.291 INFO:tasks.workunit.client.0.smithi057.stdout:3/511: dread d3/d8/f73 [0,4194304] 0 2024-02-28T23:07:44.295 INFO:tasks.workunit.client.1.smithi084.stdout:8/731: read d0/f53 [616593,38898] 0 2024-02-28T23:07:44.297 INFO:tasks.workunit.client.1.smithi084.stdout:5/757: link dc/de/df/d31/dd1/f91 dc/d51/d7c/f11d 0 2024-02-28T23:07:44.300 INFO:tasks.workunit.client.0.smithi057.stdout:7/569: link d0/f10 d0/d5/d36/d4e/fbc 0 2024-02-28T23:07:44.300 INFO:tasks.workunit.client.1.smithi084.stdout:8/732: read - d0/d24/d78/fd2 zero size 2024-02-28T23:07:44.302 INFO:tasks.workunit.client.1.smithi084.stdout:3/771: mknod d3/d27/d7d/db0/d1a/df9/c121 0 2024-02-28T23:07:44.303 INFO:tasks.workunit.client.1.smithi084.stdout:1/794: symlink d12/d1b/d2e/d52/l105 0 2024-02-28T23:07:44.306 INFO:tasks.workunit.client.0.smithi057.stdout:4/568: dwrite f16 [0,4194304] 0 2024-02-28T23:07:44.308 INFO:tasks.workunit.client.1.smithi084.stdout:8/733: read d0/d24/d78/f86 [542005,92503] 0 2024-02-28T23:07:44.309 INFO:tasks.workunit.client.0.smithi057.stdout:3/512: mknod d3/d4f/d4c/d91/ca3 0 2024-02-28T23:07:44.310 INFO:tasks.workunit.client.0.smithi057.stdout:5/572: mknod d4/d9/cb6 0 2024-02-28T23:07:44.312 INFO:tasks.workunit.client.1.smithi084.stdout:6/715: sync 2024-02-28T23:07:44.312 INFO:tasks.workunit.client.1.smithi084.stdout:9/776: sync 2024-02-28T23:07:44.312 INFO:tasks.workunit.client.1.smithi084.stdout:2/658: sync 2024-02-28T23:07:44.315 INFO:tasks.workunit.client.0.smithi057.stdout:0/668: link d17/d20/d4a/d78/fc7 d17/d20/d23/d47/d60/dea/ff5 0 2024-02-28T23:07:44.316 INFO:tasks.workunit.client.0.smithi057.stdout:0/669: stat d17/d20/d4a/d78/db5/ce2 0 2024-02-28T23:07:44.317 INFO:tasks.workunit.client.1.smithi084.stdout:1/795: read f1 [3788284,90857] 0 2024-02-28T23:07:44.317 INFO:tasks.workunit.client.0.smithi057.stdout:2/687: write d8/d11/d15/f95 [49485,67195] 0 2024-02-28T23:07:44.318 INFO:tasks.workunit.client.0.smithi057.stdout:0/670: chown d17/d20/d26/d48/d7c/d98 1290 1 2024-02-28T23:07:44.319 INFO:tasks.workunit.client.1.smithi084.stdout:0/738: dwrite d1/d8/d18/d4c/d6c/d74/fad [0,4194304] 0 2024-02-28T23:07:44.326 INFO:tasks.workunit.client.1.smithi084.stdout:8/734: read d0/d9/d30/db6/f80 [36937,100299] 0 2024-02-28T23:07:44.326 INFO:tasks.workunit.client.1.smithi084.stdout:6/716: write d1/d13/d50/d74/faa [2833889,54030] 0 2024-02-28T23:07:44.327 INFO:tasks.workunit.client.0.smithi057.stdout:7/570: creat d0/d5/d36/d4e/d15/da0/fbd x:0 0 0 2024-02-28T23:07:44.334 INFO:tasks.workunit.client.1.smithi084.stdout:2/659: symlink d5/d1f/d42/d8f/dbe/lc5 0 2024-02-28T23:07:44.335 INFO:tasks.workunit.client.0.smithi057.stdout:2/688: rename d8/d11/d48/d50/l76 to d8/d11/d39/dbc/ld8 0 2024-02-28T23:07:44.337 INFO:tasks.workunit.client.1.smithi084.stdout:9/777: fsync d4/d6/d15/d75/d2d/fc4 0 2024-02-28T23:07:44.338 INFO:tasks.workunit.client.0.smithi057.stdout:0/671: mknod d17/d20/d26/d48/d55/d88/cf6 0 2024-02-28T23:07:44.341 INFO:tasks.workunit.client.0.smithi057.stdout:7/571: creat d0/d5/d36/d4e/d3b/fbe x:0 0 0 2024-02-28T23:07:44.345 INFO:tasks.workunit.client.0.smithi057.stdout:0/672: chown d17/d20/d26/d30/lf0 149476 1 2024-02-28T23:07:44.347 INFO:tasks.workunit.client.0.smithi057.stdout:2/689: truncate d8/d11/d39/f62 599687 0 2024-02-28T23:07:44.348 INFO:tasks.workunit.client.1.smithi084.stdout:7/819: dread db/d10/d17/f42 [0,4194304] 0 2024-02-28T23:07:44.349 INFO:tasks.workunit.client.0.smithi057.stdout:1/541: dread d2/f21 [0,4194304] 0 2024-02-28T23:07:44.350 INFO:tasks.workunit.client.1.smithi084.stdout:1/796: dread d12/d1b/d63/d7d/f85 [0,4194304] 0 2024-02-28T23:07:44.351 INFO:tasks.workunit.client.0.smithi057.stdout:2/690: chown d8/d11/d48/f52 76934957 1 2024-02-28T23:07:44.351 INFO:tasks.workunit.client.0.smithi057.stdout:0/673: unlink d17/d20/d23/d37/f43 0 2024-02-28T23:07:44.352 INFO:tasks.workunit.client.1.smithi084.stdout:9/778: unlink d4/d6/d53/l4d 0 2024-02-28T23:07:44.356 INFO:tasks.workunit.client.1.smithi084.stdout:4/699: write d1/d18/d20/d24/f8b [160762,74253] 0 2024-02-28T23:07:44.356 INFO:tasks.workunit.client.0.smithi057.stdout:0/674: chown d17/d20/d26 29 1 2024-02-28T23:07:44.356 INFO:tasks.workunit.client.1.smithi084.stdout:0/739: creat d1/fef x:0 0 0 2024-02-28T23:07:44.358 INFO:tasks.workunit.client.1.smithi084.stdout:7/820: creat db/d5e/d8a/ff6 x:0 0 0 2024-02-28T23:07:44.359 INFO:tasks.workunit.client.0.smithi057.stdout:6/613: dwrite d0/d1b/d54/f65 [0,4194304] 0 2024-02-28T23:07:44.359 INFO:tasks.workunit.client.0.smithi057.stdout:0/675: write d17/d19/dd2/fe0 [7021917,127651] 0 2024-02-28T23:07:44.361 INFO:tasks.workunit.client.0.smithi057.stdout:1/542: rmdir d2/d7/df 39 2024-02-28T23:07:44.362 INFO:tasks.workunit.client.1.smithi084.stdout:9/779: write d4/f60 [3720898,55346] 0 2024-02-28T23:07:44.367 INFO:tasks.workunit.client.1.smithi084.stdout:6/717: write d1/d13/d50/d74/d9e/fb9 [5003529,28517] 0 2024-02-28T23:07:44.367 INFO:tasks.workunit.client.1.smithi084.stdout:8/735: link d0/d24/d9e/fac d0/d9/d1b/d38/d43/d51/ff5 0 2024-02-28T23:07:44.372 INFO:tasks.workunit.client.1.smithi084.stdout:4/700: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea x:0 0 0 2024-02-28T23:07:44.373 INFO:tasks.workunit.client.1.smithi084.stdout:5/758: dwrite dc/de/df/d31/dd1/f5d [0,4194304] 0 2024-02-28T23:07:44.373 INFO:tasks.workunit.client.1.smithi084.stdout:8/736: readlink d0/d24/d9e/laf 0 2024-02-28T23:07:44.373 INFO:tasks.workunit.client.0.smithi057.stdout:0/676: chown d17/d20/d4a/d78/db5/cd9 337897 1 2024-02-28T23:07:44.373 INFO:tasks.workunit.client.0.smithi057.stdout:8/656: dwrite d4/de9/f36 [0,4194304] 0 2024-02-28T23:07:44.373 INFO:tasks.workunit.client.0.smithi057.stdout:3/513: truncate d3/d8/f5e 3933351 0 2024-02-28T23:07:44.374 INFO:tasks.workunit.client.0.smithi057.stdout:0/677: write d17/d20/d23/d4b/fb9 [2239576,43146] 0 2024-02-28T23:07:44.375 INFO:tasks.workunit.client.0.smithi057.stdout:8/657: readlink d4/d13/d2c/d4e/ld0 0 2024-02-28T23:07:44.376 INFO:tasks.workunit.client.1.smithi084.stdout:9/780: symlink d4/d6/d15/d12/lfd 0 2024-02-28T23:07:44.379 INFO:tasks.workunit.client.1.smithi084.stdout:6/718: write d1/d9/d1d/d1e/d33/f89 [21796,32260] 0 2024-02-28T23:07:44.380 INFO:tasks.workunit.client.1.smithi084.stdout:3/772: dwrite d3/d27/d7d/db0/df/f12 [0,4194304] 0 2024-02-28T23:07:44.381 INFO:tasks.workunit.client.1.smithi084.stdout:1/797: getdents d12/d1b/d2e/d32 0 2024-02-28T23:07:44.384 INFO:tasks.workunit.client.0.smithi057.stdout:2/691: unlink d8/f1a 0 2024-02-28T23:07:44.387 INFO:tasks.workunit.client.1.smithi084.stdout:0/740: rmdir d1/d6/d59/d5b 0 2024-02-28T23:07:44.388 INFO:tasks.workunit.client.0.smithi057.stdout:0/678: mkdir d17/d20/d4a/d78/df7 0 2024-02-28T23:07:44.389 INFO:tasks.workunit.client.0.smithi057.stdout:2/692: write d8/d11/d21/f7d [1173417,93395] 0 2024-02-28T23:07:44.390 INFO:tasks.workunit.client.1.smithi084.stdout:8/737: symlink d0/lf6 0 2024-02-28T23:07:44.391 INFO:tasks.workunit.client.1.smithi084.stdout:2/660: truncate d5/d7/d2e/f9a 3144589 0 2024-02-28T23:07:44.392 INFO:tasks.workunit.client.1.smithi084.stdout:4/701: mknod d1/d18/d20/d24/d2f/ceb 0 2024-02-28T23:07:44.393 INFO:tasks.workunit.client.1.smithi084.stdout:5/759: mknod dc/de/df/d16/d34/d3c/c11e 0 2024-02-28T23:07:44.393 INFO:tasks.workunit.client.0.smithi057.stdout:5/573: dread d4/d21/d24/f51 [0,4194304] 0 2024-02-28T23:07:44.394 INFO:tasks.workunit.client.0.smithi057.stdout:6/614: dwrite d0/d4/d25/f2a [0,4194304] 0 2024-02-28T23:07:44.395 INFO:tasks.workunit.client.1.smithi084.stdout:1/798: creat d12/da2/f106 x:0 0 0 2024-02-28T23:07:44.395 INFO:tasks.workunit.client.1.smithi084.stdout:4/702: fdatasync d1/d18/d20/d24/f84 0 2024-02-28T23:07:44.395 INFO:tasks.workunit.client.1.smithi084.stdout:7/821: getdents db/d5e/d87/dc1 0 2024-02-28T23:07:44.396 INFO:tasks.workunit.client.1.smithi084.stdout:9/781: fsync d4/d6/d15/d12/f48 0 2024-02-28T23:07:44.397 INFO:tasks.workunit.client.0.smithi057.stdout:1/543: mkdir d2/d7/df/db/d12/d19/da4/dc5 0 2024-02-28T23:07:44.398 INFO:tasks.workunit.client.1.smithi084.stdout:6/719: rename d1/d9/d1d/d1e/d33/d38 to d1/d13/d2e/d41/d2c/dfb 0 2024-02-28T23:07:44.400 INFO:tasks.workunit.client.0.smithi057.stdout:1/544: fdatasync d2/d71/d2d/d5b/f7c 0 2024-02-28T23:07:44.401 INFO:tasks.workunit.client.0.smithi057.stdout:8/658: symlink d4/de9/d56/d6a/d54/lef 0 2024-02-28T23:07:44.401 INFO:tasks.workunit.client.0.smithi057.stdout:1/545: readlink d2/d7/d4f/d5c/d6b/lad 0 2024-02-28T23:07:44.404 INFO:tasks.workunit.client.0.smithi057.stdout:0/679: creat d17/d20/d26/d48/d7c/d98/da6/ff8 x:0 0 0 2024-02-28T23:07:44.404 INFO:tasks.workunit.client.1.smithi084.stdout:7/822: dread - db/d10/d17/d64/d77/ff0 zero size 2024-02-28T23:07:44.406 INFO:tasks.workunit.client.0.smithi057.stdout:1/546: truncate d2/d71/d2d/d5b/f7d 676788 0 2024-02-28T23:07:44.409 INFO:tasks.workunit.client.1.smithi084.stdout:3/773: mkdir d3/d27/d7d/db0/d14/d122 0 2024-02-28T23:07:44.411 INFO:tasks.workunit.client.0.smithi057.stdout:6/615: rename d0/d4/d11/l87 to d0/d17/d94/da0/ld3 0 2024-02-28T23:07:44.411 INFO:tasks.workunit.client.1.smithi084.stdout:4/703: fdatasync d1/d18/d20/f11 0 2024-02-28T23:07:44.413 INFO:tasks.workunit.client.1.smithi084.stdout:6/720: symlink d1/d13/d50/dbd/lfc 0 2024-02-28T23:07:44.414 INFO:tasks.workunit.client.0.smithi057.stdout:7/572: write d0/d5/d36/d4e/d15/f60 [152641,121845] 0 2024-02-28T23:07:44.414 INFO:tasks.workunit.client.0.smithi057.stdout:6/616: chown d0/d1b/d3e/d3d/d42/d4f 68 1 2024-02-28T23:07:44.417 INFO:tasks.workunit.client.0.smithi057.stdout:1/547: mknod d2/d7/df/db/d3a/d66/cc6 0 2024-02-28T23:07:44.420 INFO:tasks.workunit.client.1.smithi084.stdout:1/799: creat d12/d1b/d2e/d52/dd7/f107 x:0 0 0 2024-02-28T23:07:44.420 INFO:tasks.workunit.client.1.smithi084.stdout:7/823: chown db/d10/d17/d9c/d9f/db1/f53 13526907 1 2024-02-28T23:07:44.423 INFO:tasks.workunit.client.1.smithi084.stdout:4/704: truncate d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/f5e 8590106 0 2024-02-28T23:07:44.425 INFO:tasks.workunit.client.1.smithi084.stdout:1/800: readlink d12/d1b/d2e/d52/d5b/lb6 0 2024-02-28T23:07:44.425 INFO:tasks.workunit.client.0.smithi057.stdout:6/617: read d0/d4/d11/fa1 [1146359,45285] 0 2024-02-28T23:07:44.428 INFO:tasks.workunit.client.0.smithi057.stdout:0/680: mknod d17/d20/d23/cf9 0 2024-02-28T23:07:44.429 INFO:tasks.workunit.client.1.smithi084.stdout:4/705: write d1/d18/d20/d14/d30/fe9 [555686,10109] 0 2024-02-28T23:07:44.429 INFO:tasks.workunit.client.1.smithi084.stdout:8/738: dwrite d0/d24/d78/fd5 [0,4194304] 0 2024-02-28T23:07:44.430 INFO:tasks.workunit.client.0.smithi057.stdout:7/573: mkdir d0/d5/d36/d4e/d3b/d75/dbf 0 2024-02-28T23:07:44.431 INFO:tasks.workunit.client.1.smithi084.stdout:5/760: link dc/de/df/d31/dd1/d5f/d9b/dfe/f10f dc/de/df/da6/f11f 0 2024-02-28T23:07:44.431 INFO:tasks.workunit.client.0.smithi057.stdout:3/514: dwrite d3/d8/d27/f55 [0,4194304] 0 2024-02-28T23:07:44.433 INFO:tasks.workunit.client.0.smithi057.stdout:7/574: write d0/d5/d36/d4e/d3b/f9d [458371,113217] 0 2024-02-28T23:07:44.447 INFO:tasks.workunit.client.1.smithi084.stdout:3/774: creat d3/d27/d7d/db0/d14/de1/f123 x:0 0 0 2024-02-28T23:07:44.450 INFO:tasks.workunit.client.1.smithi084.stdout:3/775: readlink d3/d27/l82 0 2024-02-28T23:07:44.452 INFO:tasks.workunit.client.1.smithi084.stdout:1/801: dread d12/d1b/d2e/d52/f6d [0,4194304] 0 2024-02-28T23:07:44.452 INFO:tasks.workunit.client.1.smithi084.stdout:2/661: write d5/dd/f58 [1761430,100746] 0 2024-02-28T23:07:44.456 INFO:tasks.workunit.client.1.smithi084.stdout:8/739: mkdir d0/d9/d30/d33/d71/d84/dd9/de3/df7 0 2024-02-28T23:07:44.457 INFO:tasks.workunit.client.1.smithi084.stdout:5/761: dread - dc/de/df/d31/dd1/d5f/d9b/fb9 zero size 2024-02-28T23:07:44.459 INFO:tasks.workunit.client.1.smithi084.stdout:1/802: fdatasync d12/f45 0 2024-02-28T23:07:44.461 INFO:tasks.workunit.client.1.smithi084.stdout:2/662: write d5/d7/d75/fac [114906,92346] 0 2024-02-28T23:07:44.463 INFO:tasks.workunit.client.1.smithi084.stdout:9/782: truncate d4/d6/d15/f71 1747956 0 2024-02-28T23:07:44.468 INFO:tasks.workunit.client.1.smithi084.stdout:5/762: symlink dc/d51/d7c/l120 0 2024-02-28T23:07:44.468 INFO:tasks.workunit.client.1.smithi084.stdout:3/776: unlink d3/d27/d7d/db0/d1a/d45/d5e/d7a/l91 0 2024-02-28T23:07:44.469 INFO:tasks.workunit.client.1.smithi084.stdout:0/741: dwrite d1/d8/f3c [0,4194304] 0 2024-02-28T23:07:44.471 INFO:tasks.workunit.client.1.smithi084.stdout:4/706: dread d1/d18/d20/d14/d30/d47/f82 [0,4194304] 0 2024-02-28T23:07:44.471 INFO:tasks.workunit.client.1.smithi084.stdout:1/803: read d12/d1b/d63/f3a [772497,88459] 0 2024-02-28T23:07:44.474 INFO:tasks.workunit.client.1.smithi084.stdout:4/707: chown d1/c3 7 1 2024-02-28T23:07:44.475 INFO:tasks.workunit.client.1.smithi084.stdout:7/824: dread db/d5e/dc6/f4f [0,4194304] 0 2024-02-28T23:07:44.476 INFO:tasks.workunit.client.1.smithi084.stdout:9/783: mknod d4/d6/d53/cfe 0 2024-02-28T23:07:44.477 INFO:tasks.workunit.client.1.smithi084.stdout:3/777: write d3/d27/d7d/db0/d14/d31/d34/f96 [1191299,62396] 0 2024-02-28T23:07:44.479 INFO:tasks.workunit.client.0.smithi057.stdout:9/600: sync 2024-02-28T23:07:44.480 INFO:tasks.workunit.client.1.smithi084.stdout:8/740: rename d0/d9/f4c to d0/d24/d9e/ff8 0 2024-02-28T23:07:44.481 INFO:tasks.workunit.client.1.smithi084.stdout:2/663: dread d5/d7/d12/d24/d3a/d48/f8d [0,4194304] 0 2024-02-28T23:07:44.483 INFO:tasks.workunit.client.1.smithi084.stdout:1/804: write d12/d1b/d2e/d52/dd7/f107 [238735,92045] 0 2024-02-28T23:07:44.485 INFO:tasks.workunit.client.1.smithi084.stdout:5/763: mknod dc/de/c121 0 2024-02-28T23:07:44.488 INFO:tasks.workunit.client.1.smithi084.stdout:5/764: chown dc/de/df/d6e/ff1 2266933 1 2024-02-28T23:07:44.490 INFO:tasks.workunit.client.1.smithi084.stdout:7/825: truncate db/d10/f3e 947606 0 2024-02-28T23:07:44.491 INFO:tasks.workunit.client.1.smithi084.stdout:9/784: mknod d4/d69/cff 0 2024-02-28T23:07:44.493 INFO:tasks.workunit.client.1.smithi084.stdout:7/826: dread - db/d10/d17/d64/fe8 zero size 2024-02-28T23:07:44.494 INFO:tasks.workunit.client.0.smithi057.stdout:8/659: truncate d4/de9/d56/fbc 1744510 0 2024-02-28T23:07:44.498 INFO:tasks.workunit.client.0.smithi057.stdout:7/575: fdatasync d0/d5/f2e 0 2024-02-28T23:07:44.500 INFO:tasks.workunit.client.0.smithi057.stdout:3/515: creat d3/d8/d1b/fa4 x:0 0 0 2024-02-28T23:07:44.500 INFO:tasks.workunit.client.1.smithi084.stdout:7/827: creat db/dbd/ff7 x:0 0 0 2024-02-28T23:07:44.500 INFO:tasks.workunit.client.1.smithi084.stdout:4/708: dwrite d1/d18/fcf [0,4194304] 0 2024-02-28T23:07:44.502 INFO:tasks.workunit.client.0.smithi057.stdout:1/548: rename d2/d7/d4f/db4/fc3 to d2/d33/d54/fc7 0 2024-02-28T23:07:44.502 INFO:tasks.workunit.client.1.smithi084.stdout:0/742: link d1/d6/d59/lb1 d1/d6/db/d37/dae/lf0 0 2024-02-28T23:07:44.506 INFO:tasks.workunit.client.1.smithi084.stdout:5/765: truncate dc/de/df/d16/d34/fea 912292 0 2024-02-28T23:07:44.507 INFO:tasks.workunit.client.1.smithi084.stdout:2/664: dwrite d5/d7/d2e/d4a/d5c/f89 [0,4194304] 0 2024-02-28T23:07:44.509 INFO:tasks.workunit.client.0.smithi057.stdout:3/516: write d3/d8/d1b/f37 [3229695,41739] 0 2024-02-28T23:07:44.509 INFO:tasks.workunit.client.1.smithi084.stdout:9/785: getdents d4/d32/d3f 0 2024-02-28T23:07:44.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/743: fdatasync d1/d6/db/d35/d44/f79 0 2024-02-28T23:07:44.510 INFO:tasks.workunit.client.1.smithi084.stdout:7/828: mknod db/d5e/dc3/cf8 0 2024-02-28T23:07:44.512 INFO:tasks.workunit.client.1.smithi084.stdout:5/766: write dc/f27 [442918,112707] 0 2024-02-28T23:07:44.515 INFO:tasks.workunit.client.0.smithi057.stdout:6/618: dwrite d0/d17/fa7 [0,4194304] 0 2024-02-28T23:07:44.515 INFO:tasks.workunit.client.1.smithi084.stdout:2/665: creat d5/d7/d12/d24/d3a/fc6 x:0 0 0 2024-02-28T23:07:44.517 INFO:tasks.workunit.client.0.smithi057.stdout:0/681: creat d17/d20/d26/d48/ffa x:0 0 0 2024-02-28T23:07:44.517 INFO:tasks.workunit.client.0.smithi057.stdout:9/601: mknod dd/de/d14/d48/cc7 0 2024-02-28T23:07:44.520 INFO:tasks.workunit.client.1.smithi084.stdout:5/767: mknod dc/de/df/da6/c122 0 2024-02-28T23:07:44.522 INFO:tasks.workunit.client.1.smithi084.stdout:9/786: truncate d4/d6/d15/d12/d41/f65 859100 0 2024-02-28T23:07:44.523 INFO:tasks.workunit.client.1.smithi084.stdout:8/741: getdents d0/d24/d9e 0 2024-02-28T23:07:44.526 INFO:tasks.workunit.client.1.smithi084.stdout:7/829: getdents db/d10/d17/d9c/d9f 0 2024-02-28T23:07:44.527 INFO:tasks.workunit.client.0.smithi057.stdout:7/576: creat d0/fc0 x:0 0 0 2024-02-28T23:07:44.527 INFO:tasks.workunit.client.1.smithi084.stdout:2/666: dread - d5/d7/d9/d7c/d5b/da6/fbb zero size 2024-02-28T23:07:44.528 INFO:tasks.workunit.client.0.smithi057.stdout:7/577: readlink d0/l33 0 2024-02-28T23:07:44.528 INFO:tasks.workunit.client.0.smithi057.stdout:9/602: mknod dd/d1b/d40/d44/da5/cc8 0 2024-02-28T23:07:44.529 INFO:tasks.workunit.client.1.smithi084.stdout:8/742: write d0/d9/d1b/d38/f3e [4281691,42007] 0 2024-02-28T23:07:44.530 INFO:tasks.workunit.client.0.smithi057.stdout:6/619: truncate d0/d1b/d3e/f7b 549685 0 2024-02-28T23:07:44.532 INFO:tasks.workunit.client.0.smithi057.stdout:3/517: dwrite d3/d4f/f48 [4194304,4194304] 0 2024-02-28T23:07:44.537 INFO:tasks.workunit.client.0.smithi057.stdout:3/518: chown d3/c19 56129 1 2024-02-28T23:07:44.538 INFO:tasks.workunit.client.1.smithi084.stdout:5/768: truncate dc/d51/d7c/f97 660572 0 2024-02-28T23:07:44.539 INFO:tasks.workunit.client.1.smithi084.stdout:1/805: truncate d12/d1b/d2e/d32/fc9 1429905 0 2024-02-28T23:07:44.541 INFO:tasks.workunit.client.0.smithi057.stdout:9/603: mkdir dd/d46/d4d/dc9 0 2024-02-28T23:07:44.541 INFO:tasks.workunit.client.0.smithi057.stdout:7/578: dread - d0/d5/d36/d4e/d15/d29/d3d/d51/faf zero size 2024-02-28T23:07:44.542 INFO:tasks.workunit.client.1.smithi084.stdout:9/787: unlink d4/d6/d15/d3c/l9a 0 2024-02-28T23:07:44.543 INFO:tasks.workunit.client.1.smithi084.stdout:7/830: creat db/d10/d18/db4/ff9 x:0 0 0 2024-02-28T23:07:44.544 INFO:tasks.workunit.client.0.smithi057.stdout:0/682: getdents d17/d20/d23/d47 0 2024-02-28T23:07:44.544 INFO:tasks.workunit.client.1.smithi084.stdout:0/744: dwrite d1/d8/d18/d4c/d6c/d74/fad [0,4194304] 0 2024-02-28T23:07:44.545 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:44 smithi057 ceph-mon[29096]: pgmap v28: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 230 MiB/s wr, 425 op/s 2024-02-28T23:07:44.546 INFO:tasks.workunit.client.0.smithi057.stdout:9/604: dread - dd/de/d4a/d9f/fab zero size 2024-02-28T23:07:44.547 INFO:tasks.workunit.client.1.smithi084.stdout:2/667: mknod d5/d1f/d42/d8f/cc7 0 2024-02-28T23:07:44.549 INFO:tasks.workunit.client.0.smithi057.stdout:8/660: write d4/de9/d56/d6a/d42/f5b [910660,103644] 0 2024-02-28T23:07:44.551 INFO:tasks.workunit.client.0.smithi057.stdout:7/579: write d0/d5/d36/d4e/d3b/d75/f94 [927756,33952] 0 2024-02-28T23:07:44.552 INFO:tasks.workunit.client.1.smithi084.stdout:1/806: mkdir d12/d1b/d2e/d52/d61/d74/d108 0 2024-02-28T23:07:44.553 INFO:tasks.workunit.client.1.smithi084.stdout:4/709: truncate d1/d18/d20/d14/d30/d47/f82 751772 0 2024-02-28T23:07:44.554 INFO:tasks.workunit.client.0.smithi057.stdout:0/683: truncate d17/d20/d26/d30/fad 166951 0 2024-02-28T23:07:44.556 INFO:tasks.workunit.client.1.smithi084.stdout:7/831: rename db/d5e/dc6 to db/d10/d18/db7/de9/dfa 0 2024-02-28T23:07:44.562 INFO:tasks.workunit.client.0.smithi057.stdout:7/580: mkdir d0/d3f/d76/dc1 0 2024-02-28T23:07:44.563 INFO:tasks.workunit.client.1.smithi084.stdout:8/743: dread d0/d44/d9f/f7e [0,4194304] 0 2024-02-28T23:07:44.565 INFO:tasks.workunit.client.1.smithi084.stdout:0/745: symlink d1/d8/d18/d4c/lf1 0 2024-02-28T23:07:44.566 INFO:tasks.workunit.client.1.smithi084.stdout:8/744: readlink d0/lf6 0 2024-02-28T23:07:44.567 INFO:tasks.workunit.client.1.smithi084.stdout:2/668: fdatasync d5/d7/d2e/f9a 0 2024-02-28T23:07:44.569 INFO:tasks.workunit.client.0.smithi057.stdout:8/661: unlink d4/d13/d18/d62/d66/d74/c78 0 2024-02-28T23:07:44.571 INFO:tasks.workunit.client.1.smithi084.stdout:0/746: getdents d1/dc0 0 2024-02-28T23:07:44.572 INFO:tasks.workunit.client.1.smithi084.stdout:5/769: dwrite dc/de/df/d31/fcb [0,4194304] 0 2024-02-28T23:07:44.576 INFO:tasks.workunit.client.0.smithi057.stdout:0/684: rename d17/d20/d26/d48/d55/f93 to d17/d20/db6/ffb 0 2024-02-28T23:07:44.576 INFO:tasks.workunit.client.0.smithi057.stdout:1/549: dwrite d2/d7/df/db/d12/d19/f9d [0,4194304] 0 2024-02-28T23:07:44.577 INFO:tasks.workunit.client.1.smithi084.stdout:9/788: read d4/d6/d15/d12/d41/ff6 [134698,112426] 0 2024-02-28T23:07:44.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:44 smithi084 ceph-mon[34826]: pgmap v28: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 230 MiB/s wr, 425 op/s 2024-02-28T23:07:44.580 INFO:tasks.workunit.client.0.smithi057.stdout:8/662: mkdir d4/d13/df0 0 2024-02-28T23:07:44.584 INFO:tasks.workunit.client.1.smithi084.stdout:4/710: dwrite d1/d18/d20/d24/d2f/d3e/fe5 [0,4194304] 0 2024-02-28T23:07:44.586 INFO:tasks.workunit.client.0.smithi057.stdout:9/605: dwrite dd/d46/d6f/fa2 [0,4194304] 0 2024-02-28T23:07:44.589 INFO:tasks.workunit.client.1.smithi084.stdout:1/807: dwrite d12/d1b/d3b/fa1 [0,4194304] 0 2024-02-28T23:07:44.594 INFO:tasks.workunit.client.1.smithi084.stdout:0/747: rename d1/d8/d69/da3/dd7 to d1/d6/db/d76/dee/df2 0 2024-02-28T23:07:44.600 INFO:tasks.workunit.client.1.smithi084.stdout:8/745: creat d0/d9/d30/d33/d71/ff9 x:0 0 0 2024-02-28T23:07:44.601 INFO:tasks.workunit.client.0.smithi057.stdout:3/519: dread d3/d8/d1b/f44 [0,4194304] 0 2024-02-28T23:07:44.601 INFO:tasks.workunit.client.0.smithi057.stdout:1/550: dwrite d2/d7/df/db/f28 [4194304,4194304] 0 2024-02-28T23:07:44.605 INFO:tasks.workunit.client.1.smithi084.stdout:1/808: chown d12/d1b/d2e/d30/d77/cb8 125361 1 2024-02-28T23:07:44.606 INFO:tasks.workunit.client.0.smithi057.stdout:0/685: dwrite d17/d20/d26/d30/fad [0,4194304] 0 2024-02-28T23:07:44.610 INFO:tasks.workunit.client.1.smithi084.stdout:1/809: chown d12/d1b/d91/d9b/da5/dcb/dd4 15 1 2024-02-28T23:07:44.612 INFO:tasks.workunit.client.1.smithi084.stdout:6/721: sync 2024-02-28T23:07:44.614 INFO:tasks.workunit.client.0.smithi057.stdout:3/520: unlink d3/d4/d7/f88 0 2024-02-28T23:07:44.615 INFO:tasks.workunit.client.1.smithi084.stdout:3/778: sync 2024-02-28T23:07:44.616 INFO:tasks.workunit.client.0.smithi057.stdout:0/686: write d17/d20/f66 [1359784,22419] 0 2024-02-28T23:07:44.616 INFO:tasks.workunit.client.1.smithi084.stdout:4/711: unlink d1/d18/f77 0 2024-02-28T23:07:44.619 INFO:tasks.workunit.client.1.smithi084.stdout:6/722: chown d1/d13/d2e/d41/d4d/f63 1339903 1 2024-02-28T23:07:44.623 INFO:tasks.workunit.client.0.smithi057.stdout:6/620: dread d0/d1b/d3e/d40/f4d [0,4194304] 0 2024-02-28T23:07:44.623 INFO:tasks.workunit.client.0.smithi057.stdout:1/551: getdents d2/d7/df/db/d12/d19/da4/dc5 0 2024-02-28T23:07:44.627 INFO:tasks.workunit.client.0.smithi057.stdout:6/621: write d0/d1b/d3e/d3d/d42/d4f/fbb [7687,60226] 0 2024-02-28T23:07:44.630 INFO:tasks.workunit.client.0.smithi057.stdout:0/687: rename c13 to d17/d20/d26/d30/d6b/cfc 0 2024-02-28T23:07:44.631 INFO:tasks.workunit.client.1.smithi084.stdout:8/746: creat d0/d9/d30/d33/d71/d84/dd9/de3/df7/ffa x:0 0 0 2024-02-28T23:07:44.632 INFO:tasks.workunit.client.0.smithi057.stdout:1/552: mkdir d2/d33/d54/dc8 0 2024-02-28T23:07:44.632 INFO:tasks.workunit.client.0.smithi057.stdout:0/688: chown d17/d20/d4a/d78/dd4 7601827 1 2024-02-28T23:07:44.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/622: chown d0/d1b/d3e/lad 6377 1 2024-02-28T23:07:44.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/693: sync 2024-02-28T23:07:44.636 INFO:tasks.workunit.client.0.smithi057.stdout:5/574: sync 2024-02-28T23:07:44.636 INFO:tasks.workunit.client.0.smithi057.stdout:0/689: stat d17/d20/d26/c3d 0 2024-02-28T23:07:44.636 INFO:tasks.workunit.client.1.smithi084.stdout:8/747: write d0/d9/fbf [201725,34524] 0 2024-02-28T23:07:44.637 INFO:tasks.workunit.client.1.smithi084.stdout:3/779: mknod d3/d27/d7d/db0/d1a/d29/d41/c124 0 2024-02-28T23:07:44.639 INFO:tasks.workunit.client.0.smithi057.stdout:1/553: creat d2/d7/df/db/d3a/fc9 x:0 0 0 2024-02-28T23:07:44.640 INFO:tasks.workunit.client.0.smithi057.stdout:2/694: chown d8/d11/d21/d70/d99/l9a 36398322 1 2024-02-28T23:07:44.642 INFO:tasks.workunit.client.1.smithi084.stdout:7/832: write db/d10/d17/d64/d77/f85 [587558,38439] 0 2024-02-28T23:07:44.642 INFO:tasks.workunit.client.0.smithi057.stdout:6/623: symlink d0/d4/d11/ld4 0 2024-02-28T23:07:44.642 INFO:tasks.workunit.client.1.smithi084.stdout:1/810: link d12/d1b/d2e/d30/c2b d12/d1b/d2e/d52/d61/d74/c109 0 2024-02-28T23:07:44.643 INFO:tasks.workunit.client.0.smithi057.stdout:7/581: write d0/d5/d36/d4e/d15/d29/d3d/f9c [578595,2435] 0 2024-02-28T23:07:44.644 INFO:tasks.workunit.client.1.smithi084.stdout:8/748: read d0/d9/f5f [2472035,5903] 0 2024-02-28T23:07:44.645 INFO:tasks.workunit.client.0.smithi057.stdout:4/569: sync 2024-02-28T23:07:44.646 INFO:tasks.workunit.client.0.smithi057.stdout:2/695: write d8/d11/d21/d28/d49/d85/d92/fba [198028,66098] 0 2024-02-28T23:07:44.648 INFO:tasks.workunit.client.0.smithi057.stdout:8/663: write d4/f3a [412459,8991] 0 2024-02-28T23:07:44.649 INFO:tasks.workunit.client.0.smithi057.stdout:4/570: chown d1a/d48/d6c 1 1 2024-02-28T23:07:44.650 INFO:tasks.workunit.client.0.smithi057.stdout:7/582: write d0/d5/d25/d3e/f54 [4755533,115974] 0 2024-02-28T23:07:44.650 INFO:tasks.workunit.client.1.smithi084.stdout:6/723: getdents d1/d13/d47/d91 0 2024-02-28T23:07:44.651 INFO:tasks.workunit.client.0.smithi057.stdout:6/624: creat d0/d1b/fd5 x:0 0 0 2024-02-28T23:07:44.651 INFO:tasks.workunit.client.0.smithi057.stdout:4/571: fsync d1a/d1e/f5b 0 2024-02-28T23:07:44.652 INFO:tasks.workunit.client.1.smithi084.stdout:7/833: write db/d10/d17/d9c/d9f/db1/f33 [3860829,128748] 0 2024-02-28T23:07:44.653 INFO:tasks.workunit.client.0.smithi057.stdout:0/690: mkdir d17/d20/d26/d48/d7c/d98/dfd 0 2024-02-28T23:07:44.654 INFO:tasks.workunit.client.1.smithi084.stdout:8/749: unlink d0/d24/d4d/f68 0 2024-02-28T23:07:44.654 INFO:tasks.workunit.client.1.smithi084.stdout:5/770: dread dc/de/df/d31/dd1/d5f/d9b/f9c [0,4194304] 0 2024-02-28T23:07:44.655 INFO:tasks.workunit.client.0.smithi057.stdout:9/606: write dd/de/d14/f61 [3898220,54549] 0 2024-02-28T23:07:44.655 INFO:tasks.workunit.client.1.smithi084.stdout:6/724: fdatasync d1/d13/d2e/d41/d4d/d6c/fe7 0 2024-02-28T23:07:44.658 INFO:tasks.workunit.client.0.smithi057.stdout:2/696: creat d8/d11/d21/d28/d41/fd9 x:0 0 0 2024-02-28T23:07:44.658 INFO:tasks.workunit.client.0.smithi057.stdout:6/625: write d0/d1b/d3e/d3d/d42/d4f/d53/f5b [2536379,5979] 0 2024-02-28T23:07:44.663 INFO:tasks.workunit.client.0.smithi057.stdout:8/664: symlink d4/d13/d18/d62/d66/lf1 0 2024-02-28T23:07:44.665 INFO:tasks.workunit.client.0.smithi057.stdout:7/583: dread d0/fb1 [0,4194304] 0 2024-02-28T23:07:44.667 INFO:tasks.workunit.client.1.smithi084.stdout:3/780: rename d3/d27/d7d/db0/l4a to d3/d50/d5a/d73/l125 0 2024-02-28T23:07:44.667 INFO:tasks.workunit.client.1.smithi084.stdout:7/834: rmdir db/d5e/d87/dc1 39 2024-02-28T23:07:44.667 INFO:tasks.workunit.client.1.smithi084.stdout:8/750: mkdir d0/d44/dfb 0 2024-02-28T23:07:44.668 INFO:tasks.workunit.client.1.smithi084.stdout:5/771: symlink dc/de/d83/l123 0 2024-02-28T23:07:44.670 INFO:tasks.workunit.client.0.smithi057.stdout:9/607: symlink dd/de/d24/d63/lca 0 2024-02-28T23:07:44.670 INFO:tasks.workunit.client.1.smithi084.stdout:1/811: dwrite d12/f23 [0,4194304] 0 2024-02-28T23:07:44.670 INFO:tasks.workunit.client.0.smithi057.stdout:4/572: symlink d1a/d48/d9c/d68/dc5/lc7 0 2024-02-28T23:07:44.671 INFO:tasks.workunit.client.0.smithi057.stdout:3/521: rmdir d3/d4 39 2024-02-28T23:07:44.674 INFO:tasks.workunit.client.0.smithi057.stdout:7/584: rmdir d0/d5/d25/d80 39 2024-02-28T23:07:44.675 INFO:tasks.workunit.client.1.smithi084.stdout:8/751: chown d0/d24/l65 0 1 2024-02-28T23:07:44.675 INFO:tasks.workunit.client.1.smithi084.stdout:5/772: rename dc/de/d83 to dc/de/d83/dd9/d124 22 2024-02-28T23:07:44.676 INFO:tasks.workunit.client.0.smithi057.stdout:5/575: dread d4/d9/d14/f54 [0,4194304] 0 2024-02-28T23:07:44.677 INFO:tasks.workunit.client.0.smithi057.stdout:8/665: creat d4/de9/d56/d6a/d42/ff2 x:0 0 0 2024-02-28T23:07:44.678 INFO:tasks.workunit.client.1.smithi084.stdout:1/812: rmdir d12/d1b/d2e/d32 39 2024-02-28T23:07:44.680 INFO:tasks.workunit.client.1.smithi084.stdout:8/752: fsync d0/d9/f15 0 2024-02-28T23:07:44.681 INFO:tasks.workunit.client.0.smithi057.stdout:1/554: write d2/d7/df/db/d12/d3f/d47/f9f [1291134,94755] 0 2024-02-28T23:07:44.683 INFO:tasks.workunit.client.0.smithi057.stdout:8/666: chown d4/d13/d18/d62/d66/fc4 234349 1 2024-02-28T23:07:44.684 INFO:tasks.workunit.client.1.smithi084.stdout:2/669: truncate d5/d1f/d42/d74/f76 2139870 0 2024-02-28T23:07:44.684 INFO:tasks.workunit.client.0.smithi057.stdout:9/608: truncate dd/de/d14/f8e 902791 0 2024-02-28T23:07:44.686 INFO:tasks.workunit.client.0.smithi057.stdout:6/626: creat d0/d4/fd6 x:0 0 0 2024-02-28T23:07:44.687 INFO:tasks.workunit.client.1.smithi084.stdout:6/725: dwrite d1/d13/d47/f4b [0,4194304] 0 2024-02-28T23:07:44.689 INFO:tasks.workunit.client.0.smithi057.stdout:1/555: chown d2/d7/df/db/d12/d19/f23 1172 1 2024-02-28T23:07:44.693 INFO:tasks.workunit.client.0.smithi057.stdout:6/627: write d0/d1b/d3e/d3d/d42/d4f/fac [579394,78175] 0 2024-02-28T23:07:44.695 INFO:tasks.workunit.client.1.smithi084.stdout:2/670: dread - d5/d7/d9/d94/f98 zero size 2024-02-28T23:07:44.698 INFO:tasks.workunit.client.0.smithi057.stdout:5/576: chown d4/d21/d24/d30/l8c 271688 1 2024-02-28T23:07:44.700 INFO:tasks.workunit.client.1.smithi084.stdout:1/813: creat d12/d1b/d2e/df3/f10a x:0 0 0 2024-02-28T23:07:44.705 INFO:tasks.workunit.client.1.smithi084.stdout:6/726: fdatasync d1/d9/d1d/f61 0 2024-02-28T23:07:44.705 INFO:tasks.workunit.client.1.smithi084.stdout:9/789: dwrite d4/d6/f7 [4194304,4194304] 0 2024-02-28T23:07:44.705 INFO:tasks.workunit.client.1.smithi084.stdout:3/781: link d3/d27/d7d/db0/d14/d31/fbc d3/d50/d117/d3d/d43/d61/dce/d109/f126 0 2024-02-28T23:07:44.706 INFO:tasks.workunit.client.0.smithi057.stdout:6/628: stat d0/d4/d25/d60/d6a/l81 0 2024-02-28T23:07:44.706 INFO:tasks.workunit.client.0.smithi057.stdout:2/697: getdents d8/d11/d21/d28/d49 0 2024-02-28T23:07:44.706 INFO:tasks.workunit.client.0.smithi057.stdout:3/522: dwrite d3/d8/d1b/f37 [0,4194304] 0 2024-02-28T23:07:44.706 INFO:tasks.workunit.client.1.smithi084.stdout:7/835: dwrite db/d10/d17/d9c/fa6 [0,4194304] 0 2024-02-28T23:07:44.710 INFO:tasks.workunit.client.1.smithi084.stdout:2/671: readlink d5/d7/d12/l21 0 2024-02-28T23:07:44.711 INFO:tasks.workunit.client.0.smithi057.stdout:1/556: fsync d2/d33/f76 0 2024-02-28T23:07:44.716 INFO:tasks.workunit.client.1.smithi084.stdout:0/748: dwrite d1/d8/d69/f84 [0,4194304] 0 2024-02-28T23:07:44.718 INFO:tasks.workunit.client.1.smithi084.stdout:6/727: chown d1/d13/l29 24249 1 2024-02-28T23:07:44.718 INFO:tasks.workunit.client.1.smithi084.stdout:7/836: read db/d10/d17/f3d [1508136,19663] 0 2024-02-28T23:07:44.719 INFO:tasks.workunit.client.1.smithi084.stdout:2/672: readlink d5/dd/d69/l82 0 2024-02-28T23:07:44.724 INFO:tasks.workunit.client.0.smithi057.stdout:0/691: dwrite d17/d20/d23/f2c [0,4194304] 0 2024-02-28T23:07:44.727 INFO:tasks.workunit.client.0.smithi057.stdout:5/577: creat d4/d9/d14/d3b/d61/d69/d95/fb7 x:0 0 0 2024-02-28T23:07:44.727 INFO:tasks.workunit.client.1.smithi084.stdout:1/814: fdatasync d12/d1b/d3b/f42 0 2024-02-28T23:07:44.728 INFO:tasks.workunit.client.1.smithi084.stdout:3/782: mkdir d3/d50/d127 0 2024-02-28T23:07:44.730 INFO:tasks.workunit.client.1.smithi084.stdout:3/783: chown d3 625373 1 2024-02-28T23:07:44.731 INFO:tasks.workunit.client.0.smithi057.stdout:3/523: unlink d3/d8/d1b/l71 0 2024-02-28T23:07:44.731 INFO:tasks.workunit.client.1.smithi084.stdout:8/753: dwrite d0/d9/d1b/fc4 [0,4194304] 0 2024-02-28T23:07:44.732 INFO:tasks.workunit.client.0.smithi057.stdout:4/573: dwrite d1a/d2f/db2/d49/d4d/f88 [0,4194304] 0 2024-02-28T23:07:44.733 INFO:tasks.workunit.client.1.smithi084.stdout:4/712: dwrite d1/d18/d20/d24/d2f/d3e/f60 [0,4194304] 0 2024-02-28T23:07:44.734 INFO:tasks.workunit.client.0.smithi057.stdout:1/557: symlink d2/d33/daa/lca 0 2024-02-28T23:07:44.734 INFO:tasks.workunit.client.1.smithi084.stdout:0/749: creat d1/d6/d59/ff3 x:0 0 0 2024-02-28T23:07:44.735 INFO:tasks.workunit.client.0.smithi057.stdout:8/667: dwrite d4/de9/d56/d6a/d54/f70 [0,4194304] 0 2024-02-28T23:07:44.736 INFO:tasks.workunit.client.0.smithi057.stdout:6/629: mknod d0/d1b/d3e/d3d/d42/d4f/d53/cd7 0 2024-02-28T23:07:44.740 INFO:tasks.workunit.client.1.smithi084.stdout:8/754: stat d0/d9/d30/d33/l40 0 2024-02-28T23:07:44.743 INFO:tasks.workunit.client.0.smithi057.stdout:2/698: mkdir d8/d11/d39/dda 0 2024-02-28T23:07:44.744 INFO:tasks.workunit.client.1.smithi084.stdout:5/773: link dc/de/df/d16/d34/daa/dae/l10a dc/de/df/d31/d8e/l125 0 2024-02-28T23:07:44.749 INFO:tasks.workunit.client.0.smithi057.stdout:3/524: chown d3/d4/d7/f10 944605 1 2024-02-28T23:07:44.750 INFO:tasks.workunit.client.0.smithi057.stdout:0/692: fdatasync d17/d20/d4a/d78/d9d/fd1 0 2024-02-28T23:07:44.753 INFO:tasks.workunit.client.0.smithi057.stdout:6/630: stat d0/d1b/d3e/d3d/d42 0 2024-02-28T23:07:44.754 INFO:tasks.workunit.client.1.smithi084.stdout:4/713: mkdir d1/d18/d20/d65/d8e/d55/dc3/dec 0 2024-02-28T23:07:44.756 INFO:tasks.workunit.client.0.smithi057.stdout:0/693: fdatasync d17/d19/f2f 0 2024-02-28T23:07:44.757 INFO:tasks.workunit.client.1.smithi084.stdout:8/755: mknod d0/d9/d1b/cfc 0 2024-02-28T23:07:44.762 INFO:tasks.workunit.client.1.smithi084.stdout:3/784: mknod d3/d50/d5a/d73/c128 0 2024-02-28T23:07:44.762 INFO:tasks.workunit.client.1.smithi084.stdout:2/673: creat d5/d7/fc8 x:0 0 0 2024-02-28T23:07:44.763 INFO:tasks.workunit.client.0.smithi057.stdout:8/668: creat d4/d13/d18/da7/ff3 x:0 0 0 2024-02-28T23:07:44.763 INFO:tasks.workunit.client.0.smithi057.stdout:9/609: dwrite dd/d1b/d40/f59 [0,4194304] 0 2024-02-28T23:07:44.765 INFO:tasks.workunit.client.1.smithi084.stdout:5/774: read dc/de/df/d31/f43 [1913684,36914] 0 2024-02-28T23:07:44.766 INFO:tasks.workunit.client.0.smithi057.stdout:3/525: creat d3/d4/d1d/d47/d62/fa5 x:0 0 0 2024-02-28T23:07:44.766 INFO:tasks.workunit.client.0.smithi057.stdout:8/669: write d4/d13/d2c/db1/fb2 [2516293,112308] 0 2024-02-28T23:07:44.769 INFO:tasks.workunit.client.0.smithi057.stdout:3/526: chown d3/c18 2 1 2024-02-28T23:07:44.769 INFO:tasks.workunit.client.0.smithi057.stdout:9/610: truncate dd/d46/dc4/f96 741992 0 2024-02-28T23:07:44.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/694: rename d17/d20/d4a/cb2 to d17/d20/d4a/d78/db5/cfe 0 2024-02-28T23:07:44.776 INFO:tasks.workunit.client.0.smithi057.stdout:0/695: read - d17/d20/d4a/d78/d9d/fd6 zero size 2024-02-28T23:07:44.776 INFO:tasks.workunit.client.1.smithi084.stdout:0/750: mknod d1/d6/db/d37/d72/d7b/cf4 0 2024-02-28T23:07:44.777 INFO:tasks.workunit.client.1.smithi084.stdout:5/775: fdatasync dc/de/df/d31/dd1/d5f/d9b/f9c 0 2024-02-28T23:07:44.778 INFO:tasks.workunit.client.1.smithi084.stdout:1/815: getdents d12/d1b/d2e/d30/d77/da0 0 2024-02-28T23:07:44.779 INFO:tasks.workunit.client.0.smithi057.stdout:0/696: chown d17/d19/f76 714 1 2024-02-28T23:07:44.779 INFO:tasks.workunit.client.0.smithi057.stdout:2/699: read d8/fe [966071,11332] 0 2024-02-28T23:07:44.784 INFO:tasks.workunit.client.1.smithi084.stdout:4/714: creat d1/d18/d20/d24/d2f/d3e/d52/d45/fed x:0 0 0 2024-02-28T23:07:44.785 INFO:tasks.workunit.client.1.smithi084.stdout:5/776: read - dc/de/df/d6e/ff1 zero size 2024-02-28T23:07:44.785 INFO:tasks.workunit.client.0.smithi057.stdout:0/697: chown d17/f9c 293252920 1 2024-02-28T23:07:44.785 INFO:tasks.workunit.client.0.smithi057.stdout:3/527: chown d3/d4/d7/c8a 27341 1 2024-02-28T23:07:44.787 INFO:tasks.workunit.client.1.smithi084.stdout:9/790: dwrite d4/d6/d15/d12/f48 [0,4194304] 0 2024-02-28T23:07:44.788 INFO:tasks.workunit.client.0.smithi057.stdout:7/585: dread d0/d5/d25/d3e/f5a [0,4194304] 0 2024-02-28T23:07:44.789 INFO:tasks.workunit.client.0.smithi057.stdout:4/574: rename d1a/d48/d9c/d2e/fc2 to d1a/d2f/db2/d49/fc8 0 2024-02-28T23:07:44.796 INFO:tasks.workunit.client.0.smithi057.stdout:7/586: chown d0/d3f/f6c 680 1 2024-02-28T23:07:44.796 INFO:tasks.workunit.client.0.smithi057.stdout:0/698: truncate d17/d20/d23/d47/d60/fd3 8894698 0 2024-02-28T23:07:44.797 INFO:tasks.workunit.client.1.smithi084.stdout:4/715: symlink d1/d18/d20/lee 0 2024-02-28T23:07:44.798 INFO:tasks.workunit.client.1.smithi084.stdout:0/751: dread d1/d6/d1b/f58 [0,4194304] 0 2024-02-28T23:07:44.800 INFO:tasks.workunit.client.1.smithi084.stdout:6/728: dwrite d1/d9/d1d/f32 [0,4194304] 0 2024-02-28T23:07:44.805 INFO:tasks.workunit.client.0.smithi057.stdout:2/700: readlink d8/d11/d39/dbc/ld8 0 2024-02-28T23:07:44.808 INFO:tasks.workunit.client.1.smithi084.stdout:5/777: mkdir dc/de/df/d31/dd1/d126 0 2024-02-28T23:07:44.809 INFO:tasks.workunit.client.1.smithi084.stdout:7/837: dread db/d10/d17/f3d [0,4194304] 0 2024-02-28T23:07:44.809 INFO:tasks.workunit.client.1.smithi084.stdout:0/752: write d1/d8/d78/f9d [943643,43681] 0 2024-02-28T23:07:44.810 INFO:tasks.workunit.client.0.smithi057.stdout:3/528: read d3/d8/f9c [215443,114581] 0 2024-02-28T23:07:44.811 INFO:tasks.workunit.client.0.smithi057.stdout:6/631: getdents d0/d4/d25/d60 0 2024-02-28T23:07:44.812 INFO:tasks.workunit.client.0.smithi057.stdout:8/670: creat d4/de9/d56/d6a/ff4 x:0 0 0 2024-02-28T23:07:44.812 INFO:tasks.workunit.client.0.smithi057.stdout:9/611: rename dd/d46/d6f/fa2 to dd/de/d14/d48/d4b/d41/fcb 0 2024-02-28T23:07:44.813 INFO:tasks.workunit.client.0.smithi057.stdout:3/529: chown d3/d4/d7/la2 2904647 1 2024-02-28T23:07:44.814 INFO:tasks.workunit.client.0.smithi057.stdout:4/575: truncate d1a/d48/d9c/f89 893922 0 2024-02-28T23:07:44.815 INFO:tasks.workunit.client.1.smithi084.stdout:8/756: dwrite d0/d9/d1b/d38/d43/d51/d54/fab [0,4194304] 0 2024-02-28T23:07:44.816 INFO:tasks.workunit.client.1.smithi084.stdout:4/716: creat d1/dd9/fef x:0 0 0 2024-02-28T23:07:44.816 INFO:tasks.workunit.client.0.smithi057.stdout:3/530: fsync d3/d20/d2d/f82 0 2024-02-28T23:07:44.818 INFO:tasks.workunit.client.1.smithi084.stdout:3/785: dwrite d3/fb [0,4194304] 0 2024-02-28T23:07:44.819 INFO:tasks.workunit.client.0.smithi057.stdout:9/612: write dd/de/d14/f1e [4181133,117041] 0 2024-02-28T23:07:44.821 INFO:tasks.workunit.client.1.smithi084.stdout:2/674: dwrite d5/d1f/d42/d8f/fa8 [0,4194304] 0 2024-02-28T23:07:44.823 INFO:tasks.workunit.client.0.smithi057.stdout:9/613: chown dd/de/d4a/d60/f75 2679754 1 2024-02-28T23:07:44.824 INFO:tasks.workunit.client.0.smithi057.stdout:8/671: write d4/d13/d1a/fdb [2656660,264] 0 2024-02-28T23:07:44.826 INFO:tasks.workunit.client.1.smithi084.stdout:9/791: mknod d4/d6/d15/d75/d9d/c100 0 2024-02-28T23:07:44.830 INFO:tasks.workunit.client.0.smithi057.stdout:7/587: rename d0/d5/d36/d4e/d3b/d75/c91 to d0/d3f/da1/cc2 0 2024-02-28T23:07:44.830 INFO:tasks.workunit.client.1.smithi084.stdout:4/717: read - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea zero size 2024-02-28T23:07:44.830 INFO:tasks.workunit.client.1.smithi084.stdout:0/753: rmdir d1/d6/db/d37 39 2024-02-28T23:07:44.830 INFO:tasks.workunit.client.1.smithi084.stdout:5/778: creat dc/ddb/f127 x:0 0 0 2024-02-28T23:07:44.831 INFO:tasks.workunit.client.0.smithi057.stdout:7/588: dread - d0/d98/fa2 zero size 2024-02-28T23:07:44.834 INFO:tasks.workunit.client.1.smithi084.stdout:2/675: chown d5/d7/d75/fc2 13816325 1 2024-02-28T23:07:44.835 INFO:tasks.workunit.client.0.smithi057.stdout:4/576: read d1a/d48/d9c/d68/fa7 [610894,120289] 0 2024-02-28T23:07:44.836 INFO:tasks.workunit.client.1.smithi084.stdout:1/816: dwrite d12/d1b/d91/d9b/da5/dcb/dd4/fe6 [0,4194304] 0 2024-02-28T23:07:44.839 INFO:tasks.workunit.client.0.smithi057.stdout:6/632: creat d0/d4/d25/d60/fd8 x:0 0 0 2024-02-28T23:07:44.839 INFO:tasks.workunit.client.0.smithi057.stdout:7/589: write d0/d5/d36/d4e/d15/d29/d3d/d51/f72 [1518344,114177] 0 2024-02-28T23:07:44.841 INFO:tasks.workunit.client.1.smithi084.stdout:3/786: creat d3/d27/d7d/db0/d1a/d45/dde/f129 x:0 0 0 2024-02-28T23:07:44.842 INFO:tasks.workunit.client.0.smithi057.stdout:5/578: write d4/d9/d14/fab [855651,119283] 0 2024-02-28T23:07:44.846 INFO:tasks.workunit.client.0.smithi057.stdout:6/633: chown d0/d17/d20/f5c 32469710 1 2024-02-28T23:07:44.847 INFO:tasks.workunit.client.1.smithi084.stdout:3/787: dread - d3/d50/d117/f85 zero size 2024-02-28T23:07:44.847 INFO:tasks.workunit.client.1.smithi084.stdout:0/754: chown d1/d8/d5f/d93/le6 447182 1 2024-02-28T23:07:44.849 INFO:tasks.workunit.client.0.smithi057.stdout:8/672: dread - d4/d13/fa8 zero size 2024-02-28T23:07:44.849 INFO:tasks.workunit.client.1.smithi084.stdout:9/792: creat d4/d6/d15/d12/ddd/f101 x:0 0 0 2024-02-28T23:07:44.851 INFO:tasks.workunit.client.1.smithi084.stdout:2/676: creat d5/d7/d9/d94/dbd/fc9 x:0 0 0 2024-02-28T23:07:44.851 INFO:tasks.workunit.client.0.smithi057.stdout:9/614: truncate dd/d1b/d40/d44/d5c/f8a 294976 0 2024-02-28T23:07:44.852 INFO:tasks.workunit.client.1.smithi084.stdout:1/817: stat d12/d1b/d2e/d30/d77/da0/lc5 0 2024-02-28T23:07:44.852 INFO:tasks.workunit.client.1.smithi084.stdout:8/757: truncate d0/d9/d30/d33/d71/d84/dd9/de3/df7/ffa 765453 0 2024-02-28T23:07:44.853 INFO:tasks.workunit.client.1.smithi084.stdout:4/718: creat d1/d18/d20/d24/d2f/d3e/d52/d48/ff0 x:0 0 0 2024-02-28T23:07:44.853 INFO:tasks.workunit.client.1.smithi084.stdout:0/755: dread - d1/d6/db/d35/d44/d92/fea zero size 2024-02-28T23:07:44.854 INFO:tasks.workunit.client.0.smithi057.stdout:0/699: rename d17/d20/d4a/d78/d9d/fd1 to d17/d20/d4a/d78/fff 0 2024-02-28T23:07:44.855 INFO:tasks.workunit.client.1.smithi084.stdout:8/758: stat d0/d9/d1b/d38/d43/l4f 0 2024-02-28T23:07:44.857 INFO:tasks.workunit.client.1.smithi084.stdout:9/793: fsync d4/d32/f5f 0 2024-02-28T23:07:44.859 INFO:tasks.workunit.client.0.smithi057.stdout:2/701: getdents d8/d11/d21/d28/d49/d85/d92 0 2024-02-28T23:07:44.859 INFO:tasks.workunit.client.1.smithi084.stdout:3/788: creat d3/d27/d7d/db0/d1a/d45/dde/f12a x:0 0 0 2024-02-28T23:07:44.862 INFO:tasks.workunit.client.1.smithi084.stdout:9/794: write d4/f85 [778646,64343] 0 2024-02-28T23:07:44.863 INFO:tasks.workunit.client.0.smithi057.stdout:4/577: mkdir d1a/d2f/db2/d49/d4d/dc9 0 2024-02-28T23:07:44.863 INFO:tasks.workunit.client.1.smithi084.stdout:4/719: read - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea zero size 2024-02-28T23:07:44.866 INFO:tasks.workunit.client.1.smithi084.stdout:2/677: symlink d5/lca 0 2024-02-28T23:07:44.866 INFO:tasks.workunit.client.0.smithi057.stdout:9/615: symlink dd/d1b/d40/d44/lcc 0 2024-02-28T23:07:44.871 INFO:tasks.workunit.client.0.smithi057.stdout:2/702: chown d8/d11/d15/c2e 614393789 1 2024-02-28T23:07:44.873 INFO:tasks.workunit.client.0.smithi057.stdout:5/579: symlink d4/d9/d14/d3b/d61/db2/lb8 0 2024-02-28T23:07:44.874 INFO:tasks.workunit.client.1.smithi084.stdout:0/756: creat d1/d6/db/d37/d72/ff5 x:0 0 0 2024-02-28T23:07:44.875 INFO:tasks.workunit.client.0.smithi057.stdout:9/616: mknod dd/d46/d6f/ccd 0 2024-02-28T23:07:44.877 INFO:tasks.workunit.client.0.smithi057.stdout:3/531: getdents d3/d20/d2d 0 2024-02-28T23:07:44.877 INFO:tasks.workunit.client.1.smithi084.stdout:4/720: truncate d1/d18/d20/d24/d2f/d3e/fe5 5033708 0 2024-02-28T23:07:44.878 INFO:tasks.workunit.client.1.smithi084.stdout:8/759: rename d0/d24/d4d/f81 to d0/ffd 0 2024-02-28T23:07:44.879 INFO:tasks.workunit.client.1.smithi084.stdout:3/789: mkdir d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/d12b 0 2024-02-28T23:07:44.879 INFO:tasks.workunit.client.1.smithi084.stdout:5/779: dwrite dc/de/df/d31/dd1/f61 [0,4194304] 0 2024-02-28T23:07:44.882 INFO:tasks.workunit.client.0.smithi057.stdout:5/580: read d4/d21/f2d [1304444,95739] 0 2024-02-28T23:07:44.882 INFO:tasks.workunit.client.0.smithi057.stdout:3/532: symlink d3/d4f/la6 0 2024-02-28T23:07:44.883 INFO:tasks.workunit.client.1.smithi084.stdout:2/678: symlink d5/d7/d12/d24/d3a/lcb 0 2024-02-28T23:07:44.885 INFO:tasks.workunit.client.1.smithi084.stdout:9/795: truncate d4/d6/d15/f71 2739693 0 2024-02-28T23:07:44.886 INFO:tasks.workunit.client.0.smithi057.stdout:6/634: getdents d0/d4/d11 0 2024-02-28T23:07:44.886 INFO:tasks.workunit.client.0.smithi057.stdout:9/617: symlink dd/de/lce 0 2024-02-28T23:07:44.886 INFO:tasks.workunit.client.0.smithi057.stdout:5/581: creat d4/fb9 x:0 0 0 2024-02-28T23:07:44.888 INFO:tasks.workunit.client.1.smithi084.stdout:8/760: creat d0/d24/d4d/ffe x:0 0 0 2024-02-28T23:07:44.888 INFO:tasks.workunit.client.0.smithi057.stdout:3/533: symlink d3/la7 0 2024-02-28T23:07:44.889 INFO:tasks.workunit.client.1.smithi084.stdout:9/796: fsync d4/d6/d15/fd4 0 2024-02-28T23:07:44.891 INFO:tasks.workunit.client.1.smithi084.stdout:4/721: unlink d1/d18/d20/d24/d2f/d3e/d52/d48/ff0 0 2024-02-28T23:07:44.891 INFO:tasks.workunit.client.0.smithi057.stdout:9/618: unlink dd/c2a 0 2024-02-28T23:07:44.893 INFO:tasks.workunit.client.0.smithi057.stdout:4/578: dwrite d1a/d48/d97/d7f/fb8 [4194304,4194304] 0 2024-02-28T23:07:44.894 INFO:tasks.workunit.client.0.smithi057.stdout:6/635: creat d0/d1b/d3e/d3d/d42/d4f/fd9 x:0 0 0 2024-02-28T23:07:44.897 INFO:tasks.workunit.client.0.smithi057.stdout:7/590: write d0/d5/d25/d3e/f7d [624948,40500] 0 2024-02-28T23:07:44.900 INFO:tasks.workunit.client.0.smithi057.stdout:0/700: write d17/d20/d26/f4c [824710,20288] 0 2024-02-28T23:07:44.902 INFO:tasks.workunit.client.0.smithi057.stdout:3/534: mknod d3/d8/ca8 0 2024-02-28T23:07:44.903 INFO:tasks.workunit.client.0.smithi057.stdout:4/579: write d1a/d2f/db2/d49/d4d/f55 [4458414,71583] 0 2024-02-28T23:07:44.903 INFO:tasks.workunit.client.0.smithi057.stdout:2/703: write d8/d11/d21/f31 [8166807,14562] 0 2024-02-28T23:07:44.904 INFO:tasks.workunit.client.1.smithi084.stdout:7/838: truncate db/d10/d17/d9c/d9f/db1/d7a/fac 2568233 0 2024-02-28T23:07:44.906 INFO:tasks.workunit.client.1.smithi084.stdout:6/729: dwrite d1/d9/ddc/fad [0,4194304] 0 2024-02-28T23:07:44.907 INFO:tasks.workunit.client.1.smithi084.stdout:5/780: mkdir dc/de/df/d16/d34/d3c/d128 0 2024-02-28T23:07:44.908 INFO:tasks.workunit.client.0.smithi057.stdout:9/619: rename dd/d1b to dd/de/d24/d63/d77/dcf 0 2024-02-28T23:07:44.908 INFO:tasks.workunit.client.1.smithi084.stdout:2/679: unlink d5/d7/d12/d24/d2a/d53/f5f 0 2024-02-28T23:07:44.909 INFO:tasks.workunit.client.0.smithi057.stdout:6/636: mknod d0/d1b/d3e/d3d/d42/d4f/d53/cda 0 2024-02-28T23:07:44.911 INFO:tasks.workunit.client.1.smithi084.stdout:0/757: dread d1/d8/f40 [0,4194304] 0 2024-02-28T23:07:44.916 INFO:tasks.workunit.client.0.smithi057.stdout:8/673: dread d4/de9/d59/f98 [0,4194304] 0 2024-02-28T23:07:44.917 INFO:tasks.workunit.client.1.smithi084.stdout:6/730: read - d1/d13/d50/fdf zero size 2024-02-28T23:07:44.918 INFO:tasks.workunit.client.1.smithi084.stdout:2/680: chown d5/d1f/d42/d74 52588 1 2024-02-28T23:07:44.919 INFO:tasks.workunit.client.1.smithi084.stdout:0/758: stat d1/d6/db/d37/d72 0 2024-02-28T23:07:44.920 INFO:tasks.workunit.client.0.smithi057.stdout:4/580: creat d1a/d1e/d81/fca x:0 0 0 2024-02-28T23:07:44.920 INFO:tasks.workunit.client.0.smithi057.stdout:7/591: dread d0/d5/d36/d4e/d3b/d75/f94 [0,4194304] 0 2024-02-28T23:07:44.921 INFO:tasks.workunit.client.0.smithi057.stdout:2/704: write d8/d11/d21/d28/d49/d85/d92/fba [1051051,4083] 0 2024-02-28T23:07:44.924 INFO:tasks.workunit.client.0.smithi057.stdout:8/674: dread - d4/d13/fa8 zero size 2024-02-28T23:07:44.926 INFO:tasks.workunit.client.1.smithi084.stdout:2/681: write d5/d7/d9/d7c/d5b/d9f/fa0 [2065310,82056] 0 2024-02-28T23:07:44.929 INFO:tasks.workunit.client.0.smithi057.stdout:4/581: fsync d1a/d1e/f5b 0 2024-02-28T23:07:44.929 INFO:tasks.workunit.client.1.smithi084.stdout:8/761: dwrite d0/d9/d1b/d20/f59 [8388608,4194304] 0 2024-02-28T23:07:44.930 INFO:tasks.workunit.client.1.smithi084.stdout:1/818: dwrite d12/d1b/d63/d7d/f85 [0,4194304] 0 2024-02-28T23:07:44.931 INFO:tasks.workunit.client.1.smithi084.stdout:1/819: readlink d12/d1b/d2e/d52/d5b/lb6 0 2024-02-28T23:07:44.939 INFO:tasks.workunit.client.1.smithi084.stdout:2/682: dread d5/d7/d9/d94/f9e [0,4194304] 0 2024-02-28T23:07:44.951 INFO:tasks.workunit.client.0.smithi057.stdout:8/675: fsync d4/f7 0 2024-02-28T23:07:44.954 INFO:tasks.workunit.client.0.smithi057.stdout:5/582: dread d4/d32/f6e [0,4194304] 0 2024-02-28T23:07:44.954 INFO:tasks.workunit.client.0.smithi057.stdout:0/701: getdents d17/d19/dd2/ddb 0 2024-02-28T23:07:44.955 INFO:tasks.workunit.client.1.smithi084.stdout:9/797: symlink d4/d6/d15/d12/d41/l102 0 2024-02-28T23:07:44.956 INFO:tasks.workunit.client.0.smithi057.stdout:8/676: chown d4/d13/d2c/d4e/l55 3450961 1 2024-02-28T23:07:44.956 INFO:tasks.workunit.client.1.smithi084.stdout:3/790: link d3/d50/d117/d3d/faa d3/d50/d5a/f12c 0 2024-02-28T23:07:44.957 INFO:tasks.workunit.client.1.smithi084.stdout:9/798: chown d4/d32/d3f/d89/fd3 6318178 1 2024-02-28T23:07:44.958 INFO:tasks.workunit.client.0.smithi057.stdout:8/677: chown d4/de9/d59 325 1 2024-02-28T23:07:44.959 INFO:tasks.workunit.client.0.smithi057.stdout:7/592: truncate d0/d5/d36/d4e/d3b/f9e 404382 0 2024-02-28T23:07:44.960 INFO:tasks.workunit.client.0.smithi057.stdout:5/583: symlink d4/d21/d24/lba 0 2024-02-28T23:07:44.961 INFO:tasks.workunit.client.0.smithi057.stdout:8/678: readlink d4/l12 0 2024-02-28T23:07:44.963 INFO:tasks.workunit.client.1.smithi084.stdout:7/839: symlink db/d10/d18/db7/de9/lfb 0 2024-02-28T23:07:44.963 INFO:tasks.workunit.client.1.smithi084.stdout:2/683: dwrite d5/d7/d2e/d4a/f54 [0,4194304] 0 2024-02-28T23:07:44.965 INFO:tasks.workunit.client.0.smithi057.stdout:5/584: chown d4/d9/d14/d3b/l74 3 1 2024-02-28T23:07:44.965 INFO:tasks.workunit.client.0.smithi057.stdout:9/620: dwrite dd/de/d24/d63/d77/dcf/d40/f59 [4194304,4194304] 0 2024-02-28T23:07:44.966 INFO:tasks.workunit.client.1.smithi084.stdout:1/820: dread - d12/d1b/d63/f56 zero size 2024-02-28T23:07:44.967 INFO:tasks.workunit.client.0.smithi057.stdout:5/585: stat d4/d9/d14/f54 0 2024-02-28T23:07:44.967 INFO:tasks.workunit.client.1.smithi084.stdout:7/840: readlink db/d10/d17/d9c/db9/lc4 0 2024-02-28T23:07:44.969 INFO:tasks.workunit.client.0.smithi057.stdout:7/593: getdents d0/d3f/d76 0 2024-02-28T23:07:44.971 INFO:tasks.workunit.client.0.smithi057.stdout:9/621: write dd/de/d4a/d9f/fb3 [2645103,87102] 0 2024-02-28T23:07:44.972 INFO:tasks.workunit.client.1.smithi084.stdout:9/799: creat d4/d6/d15/d75/d9d/f103 x:0 0 0 2024-02-28T23:07:44.972 INFO:tasks.workunit.client.1.smithi084.stdout:0/759: mkdir d1/d6/df6 0 2024-02-28T23:07:44.972 INFO:tasks.workunit.client.1.smithi084.stdout:3/791: truncate d3/d50/d117/d7b/dbf/fc7 359542 0 2024-02-28T23:07:44.975 INFO:tasks.workunit.client.0.smithi057.stdout:2/705: dread d8/d11/d21/d28/f73 [0,4194304] 0 2024-02-28T23:07:44.978 INFO:tasks.workunit.client.0.smithi057.stdout:9/622: chown dd/d46/c7b 7 1 2024-02-28T23:07:44.979 INFO:tasks.workunit.client.1.smithi084.stdout:1/821: symlink d12/d1b/d91/d9b/l10b 0 2024-02-28T23:07:44.980 INFO:tasks.workunit.client.0.smithi057.stdout:9/623: readlink dd/de/d14/d48/d4b/l66 0 2024-02-28T23:07:44.984 INFO:tasks.workunit.client.1.smithi084.stdout:4/722: dwrite d1/d18/d20/d65/d8e/d55/fb8 [0,4194304] 0 2024-02-28T23:07:44.988 INFO:tasks.workunit.client.1.smithi084.stdout:0/760: truncate d1/d6/db/d35/d4a/d56/f89 2396265 0 2024-02-28T23:07:44.989 INFO:tasks.workunit.client.0.smithi057.stdout:9/624: truncate dd/de/d24/d63/d77/dcf/d40/d44/fa4 720874 0 2024-02-28T23:07:44.989 INFO:tasks.workunit.client.0.smithi057.stdout:7/594: fsync d0/d5/d36/d4e/d3b/f9e 0 2024-02-28T23:07:44.989 INFO:tasks.workunit.client.0.smithi057.stdout:2/706: symlink d8/d11/d15/d97/ldb 0 2024-02-28T23:07:44.991 INFO:tasks.workunit.client.1.smithi084.stdout:7/841: creat db/d10/d18/db7/de9/ffc x:0 0 0 2024-02-28T23:07:44.991 INFO:tasks.workunit.client.1.smithi084.stdout:3/792: symlink d3/d27/d7d/db0/d1a/d29/d41/d60/d98/l12d 0 2024-02-28T23:07:44.992 INFO:tasks.workunit.client.0.smithi057.stdout:2/707: fdatasync d8/d11/d21/d28/d49/d85/d92/fba 0 2024-02-28T23:07:44.995 INFO:tasks.workunit.client.1.smithi084.stdout:3/793: write d3/d27/d7d/db0/d14/d31/d34/f96 [2223507,45436] 0 2024-02-28T23:07:44.996 INFO:tasks.workunit.client.0.smithi057.stdout:3/535: dread d3/d20/f24 [0,4194304] 0 2024-02-28T23:07:44.998 INFO:tasks.workunit.client.1.smithi084.stdout:1/822: symlink d12/d1b/d2e/d52/dd7/l10c 0 2024-02-28T23:07:44.998 INFO:tasks.workunit.client.1.smithi084.stdout:0/761: rmdir d1/d8/d69/da3 39 2024-02-28T23:07:45.003 INFO:tasks.workunit.client.0.smithi057.stdout:2/708: mkdir d8/d11/d21/d28/d49/d85/d9b/ddc 0 2024-02-28T23:07:45.004 INFO:tasks.workunit.client.1.smithi084.stdout:0/762: chown d1/d6/db/d35/d4a/d56/fc1 848949 1 2024-02-28T23:07:45.004 INFO:tasks.workunit.client.0.smithi057.stdout:3/536: chown d3/d8/c6d 157 1 2024-02-28T23:07:45.005 INFO:tasks.workunit.client.1.smithi084.stdout:2/684: rename d5/f3b to d5/d7/d9/d7c/fcc 0 2024-02-28T23:07:45.006 INFO:tasks.workunit.client.1.smithi084.stdout:7/842: rename db/d10/d17 to db/d10/d17/d9c/d9f/db1/dfd 22 2024-02-28T23:07:45.007 INFO:tasks.workunit.client.1.smithi084.stdout:4/723: getdents d1/d18/d20/d65/d8e/d55/dc3/dec 0 2024-02-28T23:07:45.008 INFO:tasks.workunit.client.0.smithi057.stdout:5/586: dread d4/d7/d8/f26 [4194304,4194304] 0 2024-02-28T23:07:45.009 INFO:tasks.workunit.client.1.smithi084.stdout:3/794: write d3/d27/d7d/db0/d1a/df9/f119 [2210012,126097] 0 2024-02-28T23:07:45.013 INFO:tasks.workunit.client.0.smithi057.stdout:2/709: symlink d8/d11/ldd 0 2024-02-28T23:07:45.015 INFO:tasks.workunit.client.1.smithi084.stdout:6/731: dread d1/d13/d2e/d41/fa9 [0,4194304] 0 2024-02-28T23:07:45.015 INFO:tasks.workunit.client.0.smithi057.stdout:0/702: dread d17/d20/d26/d30/f3f [0,4194304] 0 2024-02-28T23:07:45.016 INFO:tasks.workunit.client.0.smithi057.stdout:5/587: fsync d4/d21/f2f 0 2024-02-28T23:07:45.018 INFO:tasks.workunit.client.0.smithi057.stdout:2/710: write d8/d11/d48/d50/f93 [800996,104022] 0 2024-02-28T23:07:45.018 INFO:tasks.workunit.client.1.smithi084.stdout:0/763: unlink d1/d8/d18/d4c/cbf 0 2024-02-28T23:07:45.019 INFO:tasks.workunit.client.0.smithi057.stdout:3/537: link d3/d4f/d4c/d91/f9f d3/d4/d7/fa9 0 2024-02-28T23:07:45.019 INFO:tasks.workunit.client.1.smithi084.stdout:2/685: mkdir d5/d7/d75/dcd 0 2024-02-28T23:07:45.021 INFO:tasks.workunit.client.1.smithi084.stdout:1/823: creat d12/d1b/d2e/d52/d61/d74/d108/f10d x:0 0 0 2024-02-28T23:07:45.022 INFO:tasks.workunit.client.1.smithi084.stdout:4/724: rmdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5 39 2024-02-28T23:07:45.023 INFO:tasks.workunit.client.0.smithi057.stdout:4/582: dread f15 [0,4194304] 0 2024-02-28T23:07:45.023 INFO:tasks.workunit.client.1.smithi084.stdout:2/686: chown d5/d7/d9/d7c/d5b/caa 0 1 2024-02-28T23:07:45.024 INFO:tasks.workunit.client.1.smithi084.stdout:3/795: creat d3/d27/d7d/db0/d14/de1/f12e x:0 0 0 2024-02-28T23:07:45.032 INFO:tasks.workunit.client.0.smithi057.stdout:9/625: dwrite dd/f5b [0,4194304] 0 2024-02-28T23:07:45.033 INFO:tasks.workunit.client.1.smithi084.stdout:2/687: symlink d5/d7/d75/lce 0 2024-02-28T23:07:45.034 INFO:tasks.workunit.client.0.smithi057.stdout:0/703: creat d17/d20/d23/d47/d60/dea/f100 x:0 0 0 2024-02-28T23:07:45.035 INFO:tasks.workunit.client.1.smithi084.stdout:9/800: write d4/d6/f72 [676580,10369] 0 2024-02-28T23:07:45.038 INFO:tasks.workunit.client.1.smithi084.stdout:3/796: rename d3/d27/d7d/db0/d1a/d29/c40 to d3/d50/c12f 0 2024-02-28T23:07:45.040 INFO:tasks.workunit.client.1.smithi084.stdout:7/843: dwrite db/fdb [0,4194304] 0 2024-02-28T23:07:45.041 INFO:tasks.workunit.client.0.smithi057.stdout:3/538: mknod d3/caa 0 2024-02-28T23:07:45.041 INFO:tasks.workunit.client.0.smithi057.stdout:7/595: dwrite d0/d3f/f4c [4194304,4194304] 0 2024-02-28T23:07:45.047 INFO:tasks.workunit.client.0.smithi057.stdout:0/704: mknod d17/d20/d26/d48/d55/c101 0 2024-02-28T23:07:45.049 INFO:tasks.workunit.client.1.smithi084.stdout:4/725: dread - d1/d18/d20/d24/d2f/d3e/d52/d57/fa0 zero size 2024-02-28T23:07:45.049 INFO:tasks.workunit.client.0.smithi057.stdout:0/705: readlink d17/d20/d23/ldd 0 2024-02-28T23:07:45.049 INFO:tasks.workunit.client.0.smithi057.stdout:9/626: unlink dd/de/d24/d63/d77/fb4 0 2024-02-28T23:07:45.052 INFO:tasks.workunit.client.1.smithi084.stdout:6/732: getdents d1/d9/d1d/d1e/d33 0 2024-02-28T23:07:45.056 INFO:tasks.workunit.client.0.smithi057.stdout:3/539: write d3/d4f/d4c/f54 [2135155,85471] 0 2024-02-28T23:07:45.056 INFO:tasks.workunit.client.0.smithi057.stdout:7/596: truncate d0/d5/f1e 3366529 0 2024-02-28T23:07:45.057 INFO:tasks.workunit.client.0.smithi057.stdout:5/588: write d4/d7/d58/f89 [809186,6159] 0 2024-02-28T23:07:45.058 INFO:tasks.workunit.client.1.smithi084.stdout:3/797: unlink d3/d27/d7d/db0/d1a/d29/d41/d60/d98/c11e 0 2024-02-28T23:07:45.059 INFO:tasks.workunit.client.1.smithi084.stdout:0/764: write d1/d8/fb4 [1016375,36193] 0 2024-02-28T23:07:45.060 INFO:tasks.workunit.client.0.smithi057.stdout:9/627: mkdir dd/de/d24/d63/d77/dcf/d40/dd0 0 2024-02-28T23:07:45.060 INFO:tasks.workunit.client.0.smithi057.stdout:2/711: dread d8/d11/d21/f36 [4194304,4194304] 0 2024-02-28T23:07:45.063 INFO:tasks.workunit.client.1.smithi084.stdout:6/733: rmdir d1/d9 39 2024-02-28T23:07:45.066 INFO:tasks.workunit.client.0.smithi057.stdout:0/706: mknod d17/d20/d4a/d78/dd4/c102 0 2024-02-28T23:07:45.067 INFO:tasks.workunit.client.0.smithi057.stdout:5/589: write d4/d9/d16/d4e/fac [362580,89574] 0 2024-02-28T23:07:45.068 INFO:tasks.workunit.client.0.smithi057.stdout:7/597: symlink d0/d5/d36/d4e/d15/d29/d3d/d51/lc3 0 2024-02-28T23:07:45.069 INFO:tasks.workunit.client.1.smithi084.stdout:7/844: dwrite db/d10/d18/db7/de9/dfa/dc7/feb [0,4194304] 0 2024-02-28T23:07:45.069 INFO:tasks.workunit.client.0.smithi057.stdout:3/540: truncate d3/d8/f3c 262793 0 2024-02-28T23:07:45.071 INFO:tasks.workunit.client.0.smithi057.stdout:9/628: unlink dd/de/d24/d63/d77/dcf/d40/d44/f9b 0 2024-02-28T23:07:45.072 INFO:tasks.workunit.client.1.smithi084.stdout:0/765: creat d1/d8/ff7 x:0 0 0 2024-02-28T23:07:45.073 INFO:tasks.workunit.client.0.smithi057.stdout:0/707: mknod d17/c103 0 2024-02-28T23:07:45.073 INFO:tasks.workunit.client.0.smithi057.stdout:4/583: dwrite d1a/d48/d97/d7f/fa5 [0,4194304] 0 2024-02-28T23:07:45.075 INFO:tasks.workunit.client.0.smithi057.stdout:5/590: unlink d4/d32/l3e 0 2024-02-28T23:07:45.077 INFO:tasks.workunit.client.0.smithi057.stdout:2/712: rename d8/d11/d48 to d8/d11/d39/d83/dde 0 2024-02-28T23:07:45.078 INFO:tasks.workunit.client.1.smithi084.stdout:7/845: unlink db/d10/d17/d3b/f3f 0 2024-02-28T23:07:45.079 INFO:tasks.workunit.client.1.smithi084.stdout:1/824: dwrite d12/d1b/d63/f3a [0,4194304] 0 2024-02-28T23:07:45.079 INFO:tasks.workunit.client.1.smithi084.stdout:2/688: link d5/d7/d12/d24/d3a/c7a d5/d7/d12/d24/d2a/d35/ccf 0 2024-02-28T23:07:45.080 INFO:tasks.workunit.client.0.smithi057.stdout:3/541: truncate d3/d8/d27/f81 1910828 0 2024-02-28T23:07:45.083 INFO:tasks.workunit.client.0.smithi057.stdout:0/708: read d17/d20/d26/f67 [1173519,107110] 0 2024-02-28T23:07:45.085 INFO:tasks.workunit.client.1.smithi084.stdout:9/801: write d4/d6/d53/fc0 [1408146,10687] 0 2024-02-28T23:07:45.086 INFO:tasks.workunit.client.0.smithi057.stdout:2/713: creat d8/d11/d39/d7b/d8b/fdf x:0 0 0 2024-02-28T23:07:45.088 INFO:tasks.workunit.client.0.smithi057.stdout:0/709: dread - d17/d19/d4d/d7b/f94 zero size 2024-02-28T23:07:45.089 INFO:tasks.workunit.client.1.smithi084.stdout:0/766: rename d1/d6/d2d/c8c to d1/d8/d18/d4c/d6c/cf8 0 2024-02-28T23:07:45.091 INFO:tasks.workunit.client.0.smithi057.stdout:7/598: dread d0/d5/d36/d4e/d15/d29/d3d/d51/f58 [0,4194304] 0 2024-02-28T23:07:45.094 INFO:tasks.workunit.client.1.smithi084.stdout:3/798: getdents d3/d27/d7d/db0/d1a/d29/d41/d5b 0 2024-02-28T23:07:45.097 INFO:tasks.workunit.client.0.smithi057.stdout:0/710: mknod d17/d20/d26/d30/d6b/d71/c104 0 2024-02-28T23:07:45.098 INFO:tasks.workunit.client.1.smithi084.stdout:2/689: rename d5/d7/d9/d7c/d5b/f87 to d5/d1f/d42/d74/fd0 0 2024-02-28T23:07:45.099 INFO:tasks.workunit.client.1.smithi084.stdout:7/846: dread db/d10/d17/f9a [0,4194304] 0 2024-02-28T23:07:45.099 INFO:tasks.workunit.client.1.smithi084.stdout:3/799: dread - d3/d50/d117/d3d/faa zero size 2024-02-28T23:07:45.100 INFO:tasks.workunit.client.0.smithi057.stdout:0/711: fsync d17/d20/d23/d47/d60/fd3 0 2024-02-28T23:07:45.103 INFO:tasks.workunit.client.1.smithi084.stdout:3/800: chown d3/d50/d116 1038195 1 2024-02-28T23:07:45.108 INFO:tasks.workunit.client.1.smithi084.stdout:7/847: write db/d10/d17/d3b/f3c [964586,129681] 0 2024-02-28T23:07:45.110 INFO:tasks.workunit.client.1.smithi084.stdout:1/825: dread d12/d1b/d3b/fa1 [0,4194304] 0 2024-02-28T23:07:45.112 INFO:tasks.workunit.client.0.smithi057.stdout:7/599: link d0/d5/d36/d4e/d15/f22 d0/d5/d25/d80/d9f/fc4 0 2024-02-28T23:07:45.113 INFO:tasks.workunit.client.1.smithi084.stdout:0/767: truncate d1/d8/f55 2601499 0 2024-02-28T23:07:45.117 INFO:tasks.workunit.client.1.smithi084.stdout:4/726: dread d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf [0,4194304] 0 2024-02-28T23:07:45.122 INFO:tasks.workunit.client.1.smithi084.stdout:7/848: creat db/dbd/ffe x:0 0 0 2024-02-28T23:07:45.122 INFO:tasks.workunit.client.1.smithi084.stdout:3/801: creat d3/d27/d7d/db0/df/ded/f130 x:0 0 0 2024-02-28T23:07:45.123 INFO:tasks.workunit.client.0.smithi057.stdout:9/629: truncate dd/f2d 1506497 0 2024-02-28T23:07:45.124 INFO:tasks.workunit.client.0.smithi057.stdout:0/712: link d17/d20/d26/d48/d55/d88/cf6 d17/d20/d26/d48/d7c/c105 0 2024-02-28T23:07:45.134 INFO:tasks.workunit.client.1.smithi084.stdout:9/802: write d4/d6/f8 [2569622,66215] 0 2024-02-28T23:07:45.136 INFO:tasks.workunit.client.0.smithi057.stdout:0/713: readlink d17/d20/ldc 0 2024-02-28T23:07:45.136 INFO:tasks.workunit.client.1.smithi084.stdout:3/802: fdatasync d3/f6 0 2024-02-28T23:07:45.136 INFO:tasks.workunit.client.0.smithi057.stdout:7/600: dwrite d0/d5/d36/d4e/d15/f22 [4194304,4194304] 0 2024-02-28T23:07:45.141 INFO:tasks.workunit.client.1.smithi084.stdout:2/690: dread d5/d7/d12/d24/d2a/d35/f5d [0,4194304] 0 2024-02-28T23:07:45.144 INFO:tasks.workunit.client.1.smithi084.stdout:1/826: mknod d12/d1b/d63/d7d/dd3/c10e 0 2024-02-28T23:07:45.147 INFO:tasks.workunit.client.1.smithi084.stdout:0/768: dwrite d1/d6/db/d35/d4a/fac [0,4194304] 0 2024-02-28T23:07:45.149 INFO:tasks.workunit.client.1.smithi084.stdout:2/691: read d5/dd/f16 [924578,57491] 0 2024-02-28T23:07:45.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/601: readlink d0/d5/d36/d4e/d3b/l84 0 2024-02-28T23:07:45.155 INFO:tasks.workunit.client.0.smithi057.stdout:4/584: dread d1a/d2f/f45 [0,4194304] 0 2024-02-28T23:07:45.155 INFO:tasks.workunit.client.1.smithi084.stdout:0/769: write d1/d6/d59/d70/f8d [633717,58888] 0 2024-02-28T23:07:45.158 INFO:tasks.workunit.client.0.smithi057.stdout:5/591: dread d4/d21/f2d [0,4194304] 0 2024-02-28T23:07:45.160 INFO:tasks.workunit.client.0.smithi057.stdout:0/714: write d17/d20/d4a/d78/f90 [1023177,54040] 0 2024-02-28T23:07:45.164 INFO:tasks.workunit.client.0.smithi057.stdout:2/714: dwrite d8/f84 [0,4194304] 0 2024-02-28T23:07:45.164 INFO:tasks.workunit.client.1.smithi084.stdout:3/803: dread d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb [0,4194304] 0 2024-02-28T23:07:45.165 INFO:tasks.workunit.client.0.smithi057.stdout:5/592: chown d4/d7/f83 915852 1 2024-02-28T23:07:45.166 INFO:tasks.workunit.client.0.smithi057.stdout:7/602: chown d0/d5/d36/d4e/d3b/l84 1948302 1 2024-02-28T23:07:45.167 INFO:tasks.workunit.client.1.smithi084.stdout:3/804: chown d3/d50/d117/d3d/d43/de3 14 1 2024-02-28T23:07:45.168 INFO:tasks.workunit.client.0.smithi057.stdout:3/542: dwrite d3/d8/f9c [0,4194304] 0 2024-02-28T23:07:45.172 INFO:tasks.workunit.client.1.smithi084.stdout:7/849: write db/d5e/d87/fa7 [808151,117502] 0 2024-02-28T23:07:45.172 INFO:tasks.workunit.client.1.smithi084.stdout:3/805: chown d3/d27/d7d/db0/d1a/l9f 14135385 1 2024-02-28T23:07:45.173 INFO:tasks.workunit.client.0.smithi057.stdout:3/543: truncate d3/d4f/d4c/f54 5084121 0 2024-02-28T23:07:45.173 INFO:tasks.workunit.client.1.smithi084.stdout:0/770: rename d1/d8/f75 to d1/d8/da1/ff9 0 2024-02-28T23:07:45.175 INFO:tasks.workunit.client.0.smithi057.stdout:9/630: dread dd/de/d14/d48/d4b/f54 [0,4194304] 0 2024-02-28T23:07:45.182 INFO:tasks.workunit.client.1.smithi084.stdout:2/692: symlink d5/d7/ld1 0 2024-02-28T23:07:45.184 INFO:tasks.workunit.client.0.smithi057.stdout:1/558: sync 2024-02-28T23:07:45.186 INFO:tasks.workunit.client.1.smithi084.stdout:3/806: rmdir d3/d27/d7d/db0/d1a/d45/d5e/d7a 39 2024-02-28T23:07:45.191 INFO:tasks.workunit.client.1.smithi084.stdout:7/850: getdents db/d5e/d8a 0 2024-02-28T23:07:45.191 INFO:tasks.workunit.client.0.smithi057.stdout:7/603: symlink d0/d5/d25/d80/lc5 0 2024-02-28T23:07:45.192 INFO:tasks.workunit.client.0.smithi057.stdout:4/585: dread - d1a/d48/d97/d7f/faa zero size 2024-02-28T23:07:45.192 INFO:tasks.workunit.client.0.smithi057.stdout:9/631: mkdir dd/de/d24/d63/dc1/dd1 0 2024-02-28T23:07:45.192 INFO:tasks.workunit.client.0.smithi057.stdout:5/593: dread - d4/d21/d24/f86 zero size 2024-02-28T23:07:45.196 INFO:tasks.workunit.client.1.smithi084.stdout:3/807: chown d3/d27/d7d/db0/d1a/d45/d86/f106 293317508 1 2024-02-28T23:07:45.196 INFO:tasks.workunit.client.0.smithi057.stdout:3/544: mkdir d3/d4/d7/d9a/d9b/dab 0 2024-02-28T23:07:45.196 INFO:tasks.workunit.client.0.smithi057.stdout:9/632: fdatasync dd/de/f6b 0 2024-02-28T23:07:45.196 INFO:tasks.workunit.client.0.smithi057.stdout:1/559: dread d2/d7/df/db/d12/f5e [0,4194304] 0 2024-02-28T23:07:45.197 INFO:tasks.workunit.client.1.smithi084.stdout:2/693: fsync d5/d7/d12/d24/d3a/d48/f5a 0 2024-02-28T23:07:45.197 INFO:tasks.workunit.client.1.smithi084.stdout:9/803: dread d4/d6/d15/d75/d2d/f34 [0,4194304] 0 2024-02-28T23:07:45.197 INFO:tasks.workunit.client.0.smithi057.stdout:2/715: getdents d8/d11/d39/d83/dde/d50 0 2024-02-28T23:07:45.198 INFO:tasks.workunit.client.0.smithi057.stdout:4/586: mkdir d1a/d2f/db2/dcb 0 2024-02-28T23:07:45.199 INFO:tasks.workunit.client.0.smithi057.stdout:7/604: symlink d0/d5/d36/d4e/d15/d29/lc6 0 2024-02-28T23:07:45.199 INFO:tasks.workunit.client.0.smithi057.stdout:3/545: read d3/d20/d2d/f3b [598189,6196] 0 2024-02-28T23:07:45.200 INFO:tasks.workunit.client.0.smithi057.stdout:2/716: dread - d8/d11/d39/d7b/d8b/fd7 zero size 2024-02-28T23:07:45.202 INFO:tasks.workunit.client.0.smithi057.stdout:3/546: write d3/d4/d7/f6c [797209,58250] 0 2024-02-28T23:07:45.202 INFO:tasks.workunit.client.1.smithi084.stdout:0/771: dwrite d1/d8/d18/d4c/d6c/d74/fe0 [0,4194304] 0 2024-02-28T23:07:45.204 INFO:tasks.workunit.client.0.smithi057.stdout:3/547: fdatasync d3/d4/d7/f10 0 2024-02-28T23:07:45.205 INFO:tasks.workunit.client.0.smithi057.stdout:7/605: fsync d0/f67 0 2024-02-28T23:07:45.207 INFO:tasks.workunit.client.1.smithi084.stdout:2/694: unlink d5/dd/d69/c99 0 2024-02-28T23:07:45.211 INFO:tasks.workunit.client.1.smithi084.stdout:0/772: unlink d1/d8/d69/db2/fe4 0 2024-02-28T23:07:45.215 INFO:tasks.workunit.client.1.smithi084.stdout:0/773: chown d1/d6/db/d76/f9c 11785387 1 2024-02-28T23:07:45.217 INFO:tasks.workunit.client.1.smithi084.stdout:4/727: truncate d1/d18/d20/d14/d30/d47/f49 274621 0 2024-02-28T23:07:45.220 INFO:tasks.workunit.client.1.smithi084.stdout:3/808: write d3/f6 [12708941,53494] 0 2024-02-28T23:07:45.227 INFO:tasks.workunit.client.1.smithi084.stdout:7/851: dwrite db/d10/d18/db7/de9/dfa/f95 [4194304,4194304] 0 2024-02-28T23:07:45.229 INFO:tasks.workunit.client.1.smithi084.stdout:9/804: dwrite d4/d6/d15/d75/d9d/dd0/ff8 [0,4194304] 0 2024-02-28T23:07:45.236 INFO:tasks.workunit.client.1.smithi084.stdout:1/827: dwrite d12/d1b/d3b/f7f [0,4194304] 0 2024-02-28T23:07:45.237 INFO:tasks.workunit.client.1.smithi084.stdout:9/805: stat d4/d6/d15/d12/d41/l76 0 2024-02-28T23:07:45.240 INFO:tasks.workunit.client.1.smithi084.stdout:7/852: unlink db/d10/faa 0 2024-02-28T23:07:45.240 INFO:tasks.workunit.client.1.smithi084.stdout:9/806: readlink d4/d6/le 0 2024-02-28T23:07:45.244 INFO:tasks.workunit.client.1.smithi084.stdout:5/781: sync 2024-02-28T23:07:45.245 INFO:tasks.workunit.client.1.smithi084.stdout:8/762: sync 2024-02-28T23:07:45.249 INFO:tasks.workunit.client.1.smithi084.stdout:1/828: unlink d12/d84/d9a/cff 0 2024-02-28T23:07:45.252 INFO:tasks.workunit.client.1.smithi084.stdout:2/695: dwrite d5/d7/d9/d7c/f95 [0,4194304] 0 2024-02-28T23:07:45.252 INFO:tasks.workunit.client.1.smithi084.stdout:0/774: rename d1/d6/db/d35/d57/d81/lb3 to d1/d8/d69/da3/lfa 0 2024-02-28T23:07:45.255 INFO:tasks.workunit.client.1.smithi084.stdout:7/853: rename db/d10/d17/d9c to db/d10/d17/d9c/dff 22 2024-02-28T23:07:45.258 INFO:tasks.workunit.client.1.smithi084.stdout:5/782: symlink dc/de/d83/dd9/l129 0 2024-02-28T23:07:45.259 INFO:tasks.workunit.client.1.smithi084.stdout:5/783: fsync dc/f27 0 2024-02-28T23:07:45.262 INFO:tasks.workunit.client.1.smithi084.stdout:1/829: mknod d12/d1b/d2e/d30/d50/c10f 0 2024-02-28T23:07:45.266 INFO:tasks.workunit.client.1.smithi084.stdout:0/775: dread - d1/d6/db/d35/d44/f6f zero size 2024-02-28T23:07:45.266 INFO:tasks.workunit.client.1.smithi084.stdout:1/830: fsync d12/da2/fab 0 2024-02-28T23:07:45.267 INFO:tasks.workunit.client.1.smithi084.stdout:5/784: creat dc/d51/f12a x:0 0 0 2024-02-28T23:07:45.270 INFO:tasks.workunit.client.1.smithi084.stdout:7/854: mkdir db/d10/d18/db7/de9/dfa/dc7/d100 0 2024-02-28T23:07:45.271 INFO:tasks.workunit.client.1.smithi084.stdout:4/728: dread d1/f9c [0,4194304] 0 2024-02-28T23:07:45.272 INFO:tasks.workunit.client.1.smithi084.stdout:3/809: dwrite d3/d27/d7d/dd9/fca [0,4194304] 0 2024-02-28T23:07:45.272 INFO:tasks.workunit.client.1.smithi084.stdout:8/763: link d0/d9/d30/d33/d71/f93 d0/d44/fff 0 2024-02-28T23:07:45.277 INFO:tasks.workunit.client.1.smithi084.stdout:7/855: symlink db/d5e/d80/l101 0 2024-02-28T23:07:45.277 INFO:tasks.workunit.client.1.smithi084.stdout:9/807: dwrite d4/d32/d3f/d89/db8/fda [0,4194304] 0 2024-02-28T23:07:45.278 INFO:tasks.workunit.client.1.smithi084.stdout:5/785: creat dc/d51/f12b x:0 0 0 2024-02-28T23:07:45.278 INFO:tasks.workunit.client.1.smithi084.stdout:4/729: chown d1/d18/d20/d24/d2f/d3e/d52/l5b 17 1 2024-02-28T23:07:45.280 INFO:tasks.workunit.client.1.smithi084.stdout:7/856: fsync db/d10/d17/d64/d77/fbf 0 2024-02-28T23:07:45.282 INFO:tasks.workunit.client.1.smithi084.stdout:2/696: rename d5/d7/d12/d24/d3a/cb7 to d5/d7/d12/d24/cd2 0 2024-02-28T23:07:45.286 INFO:tasks.workunit.client.1.smithi084.stdout:1/831: mknod d12/d1b/d63/c110 0 2024-02-28T23:07:45.287 INFO:tasks.workunit.client.1.smithi084.stdout:5/786: dread - dc/de/fe3 zero size 2024-02-28T23:07:45.293 INFO:tasks.workunit.client.1.smithi084.stdout:7/857: creat db/d10/d17/d3b/f102 x:0 0 0 2024-02-28T23:07:45.295 INFO:tasks.workunit.client.1.smithi084.stdout:7/858: chown db/d10/d18/db7 46312023 1 2024-02-28T23:07:45.296 INFO:tasks.workunit.client.1.smithi084.stdout:1/832: write d12/d1b/d2e/d52/dd7/fe2 [910225,22561] 0 2024-02-28T23:07:45.297 INFO:tasks.workunit.client.1.smithi084.stdout:4/730: write d1/d18/d20/d14/fab [1542034,81425] 0 2024-02-28T23:07:45.297 INFO:tasks.workunit.client.1.smithi084.stdout:2/697: creat d5/d1f/d42/d8f/dbe/fd3 x:0 0 0 2024-02-28T23:07:45.301 INFO:tasks.workunit.client.1.smithi084.stdout:9/808: unlink d4/d6/d15/d75/c35 0 2024-02-28T23:07:45.303 INFO:tasks.workunit.client.1.smithi084.stdout:6/734: sync 2024-02-28T23:07:45.303 INFO:tasks.workunit.client.1.smithi084.stdout:2/698: chown d5/d7/d1d/f4d 63128 1 2024-02-28T23:07:45.305 INFO:tasks.workunit.client.1.smithi084.stdout:4/731: write d1/d18/d20/d65/d8e/d55/fcd [46113,89272] 0 2024-02-28T23:07:45.306 INFO:tasks.workunit.client.1.smithi084.stdout:5/787: dread dc/f28 [0,4194304] 0 2024-02-28T23:07:45.306 INFO:tasks.workunit.client.0.smithi057.stdout:2/717: write d8/d11/d39/d7b/d8b/fdf [916629,13025] 0 2024-02-28T23:07:45.313 INFO:tasks.workunit.client.1.smithi084.stdout:2/699: rmdir d5/dd/d69 39 2024-02-28T23:07:45.313 INFO:tasks.workunit.client.1.smithi084.stdout:7/859: write db/d10/d17/d9c/d9f/db1/f53 [6516917,114416] 0 2024-02-28T23:07:45.315 INFO:tasks.workunit.client.1.smithi084.stdout:8/764: dread d0/f1a [0,4194304] 0 2024-02-28T23:07:45.316 INFO:tasks.workunit.client.1.smithi084.stdout:3/810: rename d3/d27/d4b/c75 to d3/d27/d7d/db0/c131 0 2024-02-28T23:07:45.316 INFO:tasks.workunit.client.1.smithi084.stdout:0/776: getdents d1/d6/db/d37/d72/d7b 0 2024-02-28T23:07:45.317 INFO:tasks.workunit.client.0.smithi057.stdout:7/606: truncate d0/d5/d36/f3c 707109 0 2024-02-28T23:07:45.320 INFO:tasks.workunit.client.1.smithi084.stdout:5/788: fsync dc/de/df/d16/f6a 0 2024-02-28T23:07:45.323 INFO:tasks.workunit.client.0.smithi057.stdout:5/594: dwrite d4/d9/d14/d63/fb3 [0,4194304] 0 2024-02-28T23:07:45.326 INFO:tasks.workunit.client.1.smithi084.stdout:1/833: rename d12/d1b/d2e/d30/d50/da8/dd9/df0/ff5 to d12/d1b/d91/f111 0 2024-02-28T23:07:45.327 INFO:tasks.workunit.client.0.smithi057.stdout:7/607: creat d0/d5/d36/d4e/d15/d29/fc7 x:0 0 0 2024-02-28T23:07:45.328 INFO:tasks.workunit.client.0.smithi057.stdout:9/633: dwrite dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:45.328 INFO:tasks.workunit.client.1.smithi084.stdout:4/732: mknod d1/d18/d20/d14/d30/d47/cf1 0 2024-02-28T23:07:45.330 INFO:tasks.workunit.client.1.smithi084.stdout:6/735: dwrite d1/d13/d50/fc9 [0,4194304] 0 2024-02-28T23:07:45.332 INFO:tasks.workunit.client.0.smithi057.stdout:9/634: truncate dd/de/d14/d48/d4b/d41/fad 556867 0 2024-02-28T23:07:45.334 INFO:tasks.workunit.client.0.smithi057.stdout:5/595: rename d4/d9/d14/d3b/d61/d69 to d4/d21/d24/da2/dbb 0 2024-02-28T23:07:45.335 INFO:tasks.workunit.client.1.smithi084.stdout:9/809: rename d4/l95 to d4/d6/d15/d75/d2d/daa/l104 0 2024-02-28T23:07:45.338 INFO:tasks.workunit.client.1.smithi084.stdout:2/700: getdents d5/d1f/d42/d74 0 2024-02-28T23:07:45.339 INFO:tasks.workunit.client.0.smithi057.stdout:4/587: dread d1a/d1e/d51/f90 [0,4194304] 0 2024-02-28T23:07:45.339 INFO:tasks.workunit.client.1.smithi084.stdout:9/810: write d4/d6/f72 [542477,69792] 0 2024-02-28T23:07:45.341 INFO:tasks.workunit.client.1.smithi084.stdout:2/701: write d5/dd/fc4 [301890,108745] 0 2024-02-28T23:07:45.346 INFO:tasks.workunit.client.0.smithi057.stdout:2/718: dwrite d8/d11/d15/f71 [0,4194304] 0 2024-02-28T23:07:45.347 INFO:tasks.workunit.client.0.smithi057.stdout:4/588: creat d1a/d48/d97/d7f/fcc x:0 0 0 2024-02-28T23:07:45.348 INFO:tasks.workunit.client.1.smithi084.stdout:7/860: link db/d5e/f6a db/d10/d17/d64/f103 0 2024-02-28T23:07:45.350 INFO:tasks.workunit.client.0.smithi057.stdout:4/589: write d1a/d48/d9c/d2e/fa3 [190563,90485] 0 2024-02-28T23:07:45.351 INFO:tasks.workunit.client.0.smithi057.stdout:4/590: chown f15 13316 1 2024-02-28T23:07:45.354 INFO:tasks.workunit.client.1.smithi084.stdout:3/811: write d3/d50/d5a/d73/db8/fc1 [1544586,46034] 0 2024-02-28T23:07:45.354 INFO:tasks.workunit.client.0.smithi057.stdout:4/591: dread - d1a/d2f/db2/d49/d4d/f78 zero size 2024-02-28T23:07:45.358 INFO:tasks.workunit.client.0.smithi057.stdout:4/592: creat d1a/d48/db0/d4f/db5/fcd x:0 0 0 2024-02-28T23:07:45.359 INFO:tasks.workunit.client.1.smithi084.stdout:4/733: dwrite d1/d18/d20/d24/d2f/d3e/d52/f8c [0,4194304] 0 2024-02-28T23:07:45.360 INFO:tasks.workunit.client.1.smithi084.stdout:9/811: symlink d4/d6/d15/d75/d9d/dd0/l105 0 2024-02-28T23:07:45.363 INFO:tasks.workunit.client.0.smithi057.stdout:7/608: dread d0/d5/d36/d4e/d15/f46 [0,4194304] 0 2024-02-28T23:07:45.364 INFO:tasks.workunit.client.1.smithi084.stdout:1/834: link d12/d1b/d2e/d30/f26 d12/da2/f112 0 2024-02-28T23:07:45.364 INFO:tasks.workunit.client.1.smithi084.stdout:0/777: write d1/d6/db/d35/d44/f98 [905662,50360] 0 2024-02-28T23:07:45.365 INFO:tasks.workunit.client.0.smithi057.stdout:4/593: rename d1a/d48/d9c/f52 to d1a/d1e/d51/dc3/fce 0 2024-02-28T23:07:45.366 INFO:tasks.workunit.client.0.smithi057.stdout:2/719: dwrite d8/d11/d39/dbc/fd3 [0,4194304] 0 2024-02-28T23:07:45.367 INFO:tasks.workunit.client.1.smithi084.stdout:7/861: mkdir db/d10/d17/d9c/d9f/db1/d104 0 2024-02-28T23:07:45.370 INFO:tasks.workunit.client.0.smithi057.stdout:7/609: unlink d0/d5/d4a/cb7 0 2024-02-28T23:07:45.370 INFO:tasks.workunit.client.0.smithi057.stdout:6/637: sync 2024-02-28T23:07:45.370 INFO:tasks.workunit.client.0.smithi057.stdout:4/594: readlink d1a/d48/db0/l8e 0 2024-02-28T23:07:45.370 INFO:tasks.workunit.client.1.smithi084.stdout:3/812: read d3/d50/d117/d3d/f4e [424238,110818] 0 2024-02-28T23:07:45.373 INFO:tasks.workunit.client.0.smithi057.stdout:7/610: mknod d0/d3f/cc8 0 2024-02-28T23:07:45.373 INFO:tasks.workunit.client.0.smithi057.stdout:4/595: read d1a/d2f/f45 [262665,85187] 0 2024-02-28T23:07:45.375 INFO:tasks.workunit.client.1.smithi084.stdout:2/702: creat d5/fd4 x:0 0 0 2024-02-28T23:07:45.376 INFO:tasks.workunit.client.1.smithi084.stdout:3/813: chown d3/d27/d7d/db0/d1a/d45/d5e 43902769 1 2024-02-28T23:07:45.378 INFO:tasks.workunit.client.1.smithi084.stdout:8/765: rename d0/d11/c5a to d0/d9/d1b/d38/d43/d51/c100 0 2024-02-28T23:07:45.381 INFO:tasks.workunit.client.0.smithi057.stdout:7/611: mknod d0/d5/d36/d4e/d3b/d75/dbf/cc9 0 2024-02-28T23:07:45.383 INFO:tasks.workunit.client.1.smithi084.stdout:6/736: dwrite d1/d9/ddc/fad [0,4194304] 0 2024-02-28T23:07:45.384 INFO:tasks.workunit.client.1.smithi084.stdout:4/734: creat d1/d18/d20/d65/d8e/d55/dc3/dec/ff2 x:0 0 0 2024-02-28T23:07:45.387 INFO:tasks.workunit.client.1.smithi084.stdout:2/703: mkdir d5/d7/d12/d24/d2a/d53/dd5 0 2024-02-28T23:07:45.390 INFO:tasks.workunit.client.0.smithi057.stdout:2/720: dwrite d8/d11/fcf [0,4194304] 0 2024-02-28T23:07:45.390 INFO:tasks.workunit.client.1.smithi084.stdout:3/814: chown d3/d27/d7d/db0/d1a/d29/d41/cda 7 1 2024-02-28T23:07:45.392 INFO:tasks.workunit.client.1.smithi084.stdout:8/766: write d0/d9/d1b/f46 [2759336,81267] 0 2024-02-28T23:07:45.393 INFO:tasks.workunit.client.1.smithi084.stdout:2/704: chown d5/d7/d12/d24/f66 23 1 2024-02-28T23:07:45.394 INFO:tasks.workunit.client.1.smithi084.stdout:5/789: rename dc/f25 to dc/de/df/da6/f12c 0 2024-02-28T23:07:45.400 INFO:tasks.workunit.client.0.smithi057.stdout:2/721: getdents d8/d11/d39/d83/dde 0 2024-02-28T23:07:45.401 INFO:tasks.workunit.client.1.smithi084.stdout:1/835: getdents d12/d1b/d2e/d30/d77 0 2024-02-28T23:07:45.401 INFO:tasks.workunit.client.1.smithi084.stdout:8/767: fsync d0/d44/d9f/f7e 0 2024-02-28T23:07:45.401 INFO:tasks.workunit.client.1.smithi084.stdout:3/815: chown d3/d27/d4b/deb 54107 1 2024-02-28T23:07:45.404 INFO:tasks.workunit.client.0.smithi057.stdout:2/722: creat d8/d11/d21/d28/d41/dbb/fe0 x:0 0 0 2024-02-28T23:07:45.410 INFO:tasks.workunit.client.1.smithi084.stdout:9/812: dread d4/d6/d15/f71 [0,4194304] 0 2024-02-28T23:07:45.410 INFO:tasks.workunit.client.0.smithi057.stdout:5/596: dwrite d4/d9/f2a [0,4194304] 0 2024-02-28T23:07:45.413 INFO:tasks.workunit.client.0.smithi057.stdout:5/597: symlink d4/d21/d24/d72/lbc 0 2024-02-28T23:07:45.418 INFO:tasks.workunit.client.1.smithi084.stdout:5/790: dread dc/de/df/d16/d34/d3c/d77/fbd [0,4194304] 0 2024-02-28T23:07:45.419 INFO:tasks.workunit.client.0.smithi057.stdout:4/596: write d1a/d48/d97/f93 [1010170,111001] 0 2024-02-28T23:07:45.420 INFO:tasks.workunit.client.0.smithi057.stdout:5/598: truncate d4/d21/d24/f86 444748 0 2024-02-28T23:07:45.421 INFO:tasks.workunit.client.1.smithi084.stdout:4/735: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/fd6 [0,4194304] 0 2024-02-28T23:07:45.423 INFO:tasks.workunit.client.0.smithi057.stdout:5/599: truncate d4/d7/d58/f89 1762673 0 2024-02-28T23:07:45.425 INFO:tasks.workunit.client.0.smithi057.stdout:5/600: truncate d4/d21/d24/d72/f8e 5519928 0 2024-02-28T23:07:45.426 INFO:tasks.workunit.client.1.smithi084.stdout:7/862: link db/d10/d18/c28 db/d5e/dce/c105 0 2024-02-28T23:07:45.427 INFO:tasks.workunit.client.1.smithi084.stdout:0/778: rename d1/d8/d69/db2/f54 to d1/d6/d59/ffb 0 2024-02-28T23:07:45.431 INFO:tasks.workunit.client.1.smithi084.stdout:9/813: creat d4/d6/d15/d12/ddd/f106 x:0 0 0 2024-02-28T23:07:45.440 INFO:tasks.workunit.client.0.smithi057.stdout:4/597: dwrite d1a/d2f/db2/f35 [0,4194304] 0 2024-02-28T23:07:45.443 INFO:tasks.workunit.client.1.smithi084.stdout:9/814: dread d4/f56 [0,4194304] 0 2024-02-28T23:07:45.445 INFO:tasks.workunit.client.0.smithi057.stdout:4/598: mknod d1a/d48/db0/d4f/d6e/ccf 0 2024-02-28T23:07:45.446 INFO:tasks.workunit.client.1.smithi084.stdout:5/791: mknod dc/de/df/d16/d34/daa/df4/c12d 0 2024-02-28T23:07:45.450 INFO:tasks.workunit.client.1.smithi084.stdout:5/792: fsync dc/de/df/d31/d8e/da9/fcc 0 2024-02-28T23:07:45.451 INFO:tasks.workunit.client.0.smithi057.stdout:4/599: fsync d1a/d48/d9c/d68/fa7 0 2024-02-28T23:07:45.453 INFO:tasks.workunit.client.1.smithi084.stdout:2/705: getdents d5/d1f/d42 0 2024-02-28T23:07:45.455 INFO:tasks.workunit.client.0.smithi057.stdout:5/601: dwrite d4/d21/d24/f82 [0,4194304] 0 2024-02-28T23:07:45.455 INFO:tasks.workunit.client.0.smithi057.stdout:4/600: mknod d1a/d48/d9c/cd0 0 2024-02-28T23:07:45.457 INFO:tasks.workunit.client.1.smithi084.stdout:6/737: truncate d1/d13/d50/fc9 174653 0 2024-02-28T23:07:45.459 INFO:tasks.workunit.client.0.smithi057.stdout:4/601: write d1a/d1e/f5b [3998892,68589] 0 2024-02-28T23:07:45.459 INFO:tasks.workunit.client.0.smithi057.stdout:7/612: dread d0/d5/d25/d80/d9f/fc4 [0,4194304] 0 2024-02-28T23:07:45.462 INFO:tasks.workunit.client.1.smithi084.stdout:3/816: write d3/d50/d5a/f12c [978624,109435] 0 2024-02-28T23:07:45.462 INFO:tasks.workunit.client.0.smithi057.stdout:4/602: dread - d1a/d48/db0/db1/fc6 zero size 2024-02-28T23:07:45.463 INFO:tasks.workunit.client.0.smithi057.stdout:5/602: mknod d4/d7/d58/d78/cbd 0 2024-02-28T23:07:45.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/603: fdatasync d1a/d48/d97/fc0 0 2024-02-28T23:07:45.464 INFO:tasks.workunit.client.0.smithi057.stdout:2/723: dread d8/dae/fb8 [0,4194304] 0 2024-02-28T23:07:45.465 INFO:tasks.workunit.client.0.smithi057.stdout:5/603: dread - d4/f92 zero size 2024-02-28T23:07:45.467 INFO:tasks.workunit.client.1.smithi084.stdout:6/738: write d1/fdb [113913,22363] 0 2024-02-28T23:07:45.469 INFO:tasks.workunit.client.1.smithi084.stdout:0/779: dwrite d1/d8/d78/fa4 [0,4194304] 0 2024-02-28T23:07:45.473 INFO:tasks.workunit.client.0.smithi057.stdout:4/604: rename d1a/d2f/db2/l9d to d1a/d48/d9c/d2e/d8f/db7/ld1 0 2024-02-28T23:07:45.473 INFO:tasks.workunit.client.0.smithi057.stdout:5/604: truncate d4/d9/d14/f43 6635628 0 2024-02-28T23:07:45.474 INFO:tasks.workunit.client.1.smithi084.stdout:3/817: read - d3/d27/d7d/db0/f10a zero size 2024-02-28T23:07:45.474 INFO:tasks.workunit.client.0.smithi057.stdout:4/605: chown d1a/d1e/f5b 57328465 1 2024-02-28T23:07:45.475 INFO:tasks.workunit.client.1.smithi084.stdout:9/815: rename d4/d6/d15/d12/d41/dea/ff5 to d4/d32/f107 0 2024-02-28T23:07:45.476 INFO:tasks.workunit.client.1.smithi084.stdout:1/836: dread fc [0,4194304] 0 2024-02-28T23:07:45.477 INFO:tasks.workunit.client.1.smithi084.stdout:0/780: fdatasync d1/d8/d5f/d93/fe2 0 2024-02-28T23:07:45.478 INFO:tasks.workunit.client.0.smithi057.stdout:4/606: symlink d1a/d48/d9c/d68/dc5/ld2 0 2024-02-28T23:07:45.482 INFO:tasks.workunit.client.0.smithi057.stdout:4/607: read f16 [4024779,84527] 0 2024-02-28T23:07:45.485 INFO:tasks.workunit.client.0.smithi057.stdout:5/605: dread d4/d9/d14/d63/fb3 [0,4194304] 0 2024-02-28T23:07:45.486 INFO:tasks.workunit.client.1.smithi084.stdout:2/706: rename d5/d7/d2e/d4a/f54 to d5/d1f/d42/d8f/fd6 0 2024-02-28T23:07:45.490 INFO:tasks.workunit.client.1.smithi084.stdout:7/863: dwrite db/d10/d17/d64/f8c [0,4194304] 0 2024-02-28T23:07:45.492 INFO:tasks.workunit.client.0.smithi057.stdout:5/606: symlink d4/d21/d24/d30/lbe 0 2024-02-28T23:07:45.494 INFO:tasks.workunit.client.0.smithi057.stdout:8/679: sync 2024-02-28T23:07:45.495 INFO:tasks.workunit.client.1.smithi084.stdout:1/837: fsync d12/d1b/d2e/d52/f5a 0 2024-02-28T23:07:45.496 INFO:tasks.workunit.client.1.smithi084.stdout:4/736: dwrite d1/d18/d20/d14/f36 [0,4194304] 0 2024-02-28T23:07:45.496 INFO:tasks.workunit.client.1.smithi084.stdout:9/816: creat d4/d6/d15/d75/d2d/dbd/f108 x:0 0 0 2024-02-28T23:07:45.496 INFO:tasks.workunit.client.0.smithi057.stdout:8/680: chown d4/de9/d56/d88 130318 1 2024-02-28T23:07:45.497 INFO:tasks.workunit.client.0.smithi057.stdout:7/613: dwrite d0/d5/d36/d4e/d3b/f5d [0,4194304] 0 2024-02-28T23:07:45.498 INFO:tasks.workunit.client.1.smithi084.stdout:7/864: stat db/d10/d17/c21 0 2024-02-28T23:07:45.502 INFO:tasks.workunit.client.1.smithi084.stdout:6/739: rename d1/d88/ff3 to d1/d9/d1d/db6/ffd 0 2024-02-28T23:07:45.504 INFO:tasks.workunit.client.0.smithi057.stdout:4/608: dwrite d1a/d2f/db2/f35 [0,4194304] 0 2024-02-28T23:07:45.508 INFO:tasks.workunit.client.1.smithi084.stdout:3/818: mkdir d3/d27/d7d/db0/d1a/d45/d5e/d7a/d132 0 2024-02-28T23:07:45.509 INFO:tasks.workunit.client.1.smithi084.stdout:9/817: creat d4/d6/d15/d12/f109 x:0 0 0 2024-02-28T23:07:45.512 INFO:tasks.workunit.client.1.smithi084.stdout:1/838: read d12/d1b/d91/d9b/da5/dcb/dd4/fe9 [560175,103506] 0 2024-02-28T23:07:45.515 INFO:tasks.workunit.client.1.smithi084.stdout:9/818: write d4/d6/d15/d75/d2d/daa/fe1 [650059,35690] 0 2024-02-28T23:07:45.515 INFO:tasks.workunit.client.0.smithi057.stdout:7/614: creat d0/d5/d36/d4e/d15/fca x:0 0 0 2024-02-28T23:07:45.518 INFO:tasks.workunit.client.1.smithi084.stdout:4/737: creat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/ff3 x:0 0 0 2024-02-28T23:07:45.519 INFO:tasks.workunit.client.0.smithi057.stdout:7/615: creat d0/d98/fcb x:0 0 0 2024-02-28T23:07:45.520 INFO:tasks.workunit.client.1.smithi084.stdout:3/819: dread d3/d50/d5a/fab [0,4194304] 0 2024-02-28T23:07:45.522 INFO:tasks.workunit.client.1.smithi084.stdout:7/865: rmdir db/d10/d17/d9c/d9f 39 2024-02-28T23:07:45.523 INFO:tasks.workunit.client.1.smithi084.stdout:6/740: rename d1/d9/d1d/d1e/d33/lb5 to d1/d13/d2e/d41/dbb/lfe 0 2024-02-28T23:07:45.524 INFO:tasks.workunit.client.0.smithi057.stdout:7/616: chown d0/d5/ca6 30474 1 2024-02-28T23:07:45.525 INFO:tasks.workunit.client.0.smithi057.stdout:8/681: dwrite d4/d13/d1a/f4b [0,4194304] 0 2024-02-28T23:07:45.525 INFO:tasks.workunit.client.1.smithi084.stdout:5/793: dread dc/de/df/d16/f57 [0,4194304] 0 2024-02-28T23:07:45.526 INFO:tasks.workunit.client.1.smithi084.stdout:4/738: write d1/d18/d20/d65/d8e/f5a [2852739,18662] 0 2024-02-28T23:07:45.527 INFO:tasks.workunit.client.0.smithi057.stdout:7/617: chown d0/d3f/d76/fb8 38053 1 2024-02-28T23:07:45.528 INFO:tasks.workunit.client.0.smithi057.stdout:8/682: stat d4/de9/d56/d88/lc2 0 2024-02-28T23:07:45.530 INFO:tasks.workunit.client.1.smithi084.stdout:3/820: write d3/d27/d7d/db0/d14/de1/f123 [379641,5243] 0 2024-02-28T23:07:45.533 INFO:tasks.workunit.client.0.smithi057.stdout:7/618: creat d0/d5/d25/fcc x:0 0 0 2024-02-28T23:07:45.533 INFO:tasks.workunit.client.0.smithi057.stdout:8/683: getdents d4/d13/d18/da7/de1/dee 0 2024-02-28T23:07:45.536 INFO:tasks.workunit.client.0.smithi057.stdout:5/607: dread d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:45.536 INFO:tasks.workunit.client.0.smithi057.stdout:7/619: chown d0/d5/d36/d4e/d3b/d75 10159 1 2024-02-28T23:07:45.538 INFO:tasks.workunit.client.0.smithi057.stdout:8/684: write d4/d13/d1a/d4a/fdc [1838086,50387] 0 2024-02-28T23:07:45.541 INFO:tasks.workunit.client.0.smithi057.stdout:5/608: creat d4/d21/d24/fbf x:0 0 0 2024-02-28T23:07:45.544 INFO:tasks.workunit.client.1.smithi084.stdout:6/741: creat d1/d13/d47/d91/fff x:0 0 0 2024-02-28T23:07:45.546 INFO:tasks.workunit.client.1.smithi084.stdout:3/821: symlink d3/d27/d7d/db0/d1a/d29/d7e/d11a/l133 0 2024-02-28T23:07:45.549 INFO:tasks.workunit.client.0.smithi057.stdout:5/609: dread d4/d9/d16/f2b [0,4194304] 0 2024-02-28T23:07:45.550 INFO:tasks.workunit.client.1.smithi084.stdout:6/742: dread - d1/d13/d50/fa6 zero size 2024-02-28T23:07:45.551 INFO:tasks.workunit.client.0.smithi057.stdout:5/610: stat d4/d9/d14/d63/fad 0 2024-02-28T23:07:45.554 INFO:tasks.workunit.client.0.smithi057.stdout:5/611: getdents d4/d9 0 2024-02-28T23:07:45.557 INFO:tasks.workunit.client.1.smithi084.stdout:6/743: write d1/d13/d2e/d41/f37 [2746041,9182] 0 2024-02-28T23:07:45.558 INFO:tasks.workunit.client.0.smithi057.stdout:5/612: symlink d4/d9/d14/d3b/d61/d71/da1/lc0 0 2024-02-28T23:07:45.561 INFO:tasks.workunit.client.0.smithi057.stdout:4/609: dread d1a/d48/db0/f5d [0,4194304] 0 2024-02-28T23:07:45.562 INFO:tasks.workunit.client.1.smithi084.stdout:0/781: dread d1/d8/d18/f65 [0,4194304] 0 2024-02-28T23:07:45.563 INFO:tasks.workunit.client.1.smithi084.stdout:7/866: dwrite db/d10/d17/d9c/d9f/db1/f33 [4194304,4194304] 0 2024-02-28T23:07:45.563 INFO:tasks.workunit.client.1.smithi084.stdout:6/744: fdatasync d1/d13/d2e/fd3 0 2024-02-28T23:07:45.567 INFO:tasks.workunit.client.0.smithi057.stdout:4/610: write d1a/d2f/db2/d49/d4d/f8d [431164,23551] 0 2024-02-28T23:07:45.570 INFO:tasks.workunit.client.1.smithi084.stdout:0/782: dread - d1/d8/d69/d6b/d85/fab zero size 2024-02-28T23:07:45.574 INFO:tasks.workunit.client.0.smithi057.stdout:4/611: mknod d1a/d48/db0/db1/cd3 0 2024-02-28T23:07:45.575 INFO:tasks.workunit.client.1.smithi084.stdout:7/867: mkdir db/d10/d17/d9c/d9f/d106 0 2024-02-28T23:07:45.578 INFO:tasks.workunit.client.0.smithi057.stdout:8/685: write d4/d13/d18/d62/d66/fe5 [1204230,116986] 0 2024-02-28T23:07:45.582 INFO:tasks.workunit.client.1.smithi084.stdout:9/819: write d4/d6/d15/d12/d41/f7e [3502900,95039] 0 2024-02-28T23:07:45.582 INFO:tasks.workunit.client.0.smithi057.stdout:4/612: mknod d1a/d2f/db2/d49/cd4 0 2024-02-28T23:07:45.583 INFO:tasks.workunit.client.0.smithi057.stdout:5/613: dwrite d4/f92 [0,4194304] 0 2024-02-28T23:07:45.584 INFO:tasks.workunit.client.0.smithi057.stdout:4/613: truncate f0 4628597 0 2024-02-28T23:07:45.585 INFO:tasks.workunit.client.0.smithi057.stdout:7/620: dwrite d0/d5/d36/f8c [0,4194304] 0 2024-02-28T23:07:45.586 INFO:tasks.workunit.client.0.smithi057.stdout:4/614: readlink d1a/d48/l7e 0 2024-02-28T23:07:45.589 INFO:tasks.workunit.client.0.smithi057.stdout:8/686: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:45.589 INFO:tasks.workunit.client.0.smithi057.stdout:4/615: chown d1a/d1e/d81 120976 1 2024-02-28T23:07:45.589 INFO:tasks.workunit.client.1.smithi084.stdout:3/822: dwrite d3/d50/d117/d3d/f67 [0,4194304] 0 2024-02-28T23:07:45.591 INFO:tasks.workunit.client.0.smithi057.stdout:5/614: mkdir d4/d7/d58/d78/dc1 0 2024-02-28T23:07:45.591 INFO:tasks.workunit.client.1.smithi084.stdout:2/707: dwrite d5/d7/d9/d7c/d5b/f7e [0,4194304] 0 2024-02-28T23:07:45.593 INFO:tasks.workunit.client.0.smithi057.stdout:4/616: chown d1a/d48/d9c/d68/fa7 4 1 2024-02-28T23:07:45.596 INFO:tasks.workunit.client.1.smithi084.stdout:8/768: sync 2024-02-28T23:07:45.597 INFO:tasks.workunit.client.0.smithi057.stdout:0/715: sync 2024-02-28T23:07:45.598 INFO:tasks.workunit.client.0.smithi057.stdout:1/560: sync 2024-02-28T23:07:45.598 INFO:tasks.workunit.client.1.smithi084.stdout:6/745: truncate d1/d13/d2e/d41/d2c/f71 1780327 0 2024-02-28T23:07:45.598 INFO:tasks.workunit.client.1.smithi084.stdout:3/823: read - d3/d50/f87 zero size 2024-02-28T23:07:45.600 INFO:tasks.workunit.client.0.smithi057.stdout:4/617: write d1a/d48/d97/d7f/faa [426720,69848] 0 2024-02-28T23:07:45.600 INFO:tasks.workunit.client.1.smithi084.stdout:1/839: dwrite d12/d1b/d63/f56 [0,4194304] 0 2024-02-28T23:07:45.603 INFO:tasks.workunit.client.0.smithi057.stdout:4/618: chown d1a/d2f/db2/dcb 333 1 2024-02-28T23:07:45.603 INFO:tasks.workunit.client.0.smithi057.stdout:1/561: write d2/d7/df/db/f28 [5992577,27809] 0 2024-02-28T23:07:45.604 INFO:tasks.workunit.client.0.smithi057.stdout:8/687: mkdir d4/de9/d56/dc9/df5 0 2024-02-28T23:07:45.608 INFO:tasks.workunit.client.0.smithi057.stdout:8/688: chown d4/d13/d2c/f84 121859 1 2024-02-28T23:07:45.608 INFO:tasks.workunit.client.1.smithi084.stdout:5/794: dread dc/de/df/f20 [0,4194304] 0 2024-02-28T23:07:45.611 INFO:tasks.workunit.client.0.smithi057.stdout:0/716: creat d17/d19/d4d/f106 x:0 0 0 2024-02-28T23:07:45.616 INFO:tasks.workunit.client.1.smithi084.stdout:0/783: dread d1/d8/d69/d6b/fbe [0,4194304] 0 2024-02-28T23:07:45.616 INFO:tasks.workunit.client.1.smithi084.stdout:8/769: creat d0/d9/d30/f101 x:0 0 0 2024-02-28T23:07:45.618 INFO:tasks.workunit.client.1.smithi084.stdout:2/708: symlink d5/d7/d12/d24/d2a/d35/ld7 0 2024-02-28T23:07:45.618 INFO:tasks.workunit.client.1.smithi084.stdout:4/739: dwrite d1/d18/d20/d14/d30/d47/f82 [0,4194304] 0 2024-02-28T23:07:45.619 INFO:tasks.workunit.client.0.smithi057.stdout:5/615: dwrite d4/d21/d24/f82 [0,4194304] 0 2024-02-28T23:07:45.619 INFO:tasks.workunit.client.0.smithi057.stdout:4/619: dread d1a/d1e/f5b [0,4194304] 0 2024-02-28T23:07:45.624 INFO:tasks.workunit.client.1.smithi084.stdout:9/820: write d4/d32/f67 [311270,14130] 0 2024-02-28T23:07:45.625 INFO:tasks.workunit.client.1.smithi084.stdout:0/784: fsync d1/d8/f40 0 2024-02-28T23:07:45.626 INFO:tasks.workunit.client.1.smithi084.stdout:8/770: write d0/d9/d1b/d38/d43/d51/de8/feb [579545,1788] 0 2024-02-28T23:07:45.628 INFO:tasks.workunit.client.0.smithi057.stdout:0/717: symlink d17/d20/d4a/d78/d9d/l107 0 2024-02-28T23:07:45.629 INFO:tasks.workunit.client.0.smithi057.stdout:7/621: link d0/d5/d36/d4e/d15/d29/d3d/l4f d0/d5/d36/d4e/d15/lcd 0 2024-02-28T23:07:45.629 INFO:tasks.workunit.client.1.smithi084.stdout:8/771: readlink d0/l57 0 2024-02-28T23:07:45.630 INFO:tasks.workunit.client.1.smithi084.stdout:2/709: truncate d5/d7/d12/f51 4933964 0 2024-02-28T23:07:45.633 INFO:tasks.workunit.client.1.smithi084.stdout:3/824: truncate d3/d27/d7d/db0/d14/f6d 4851106 0 2024-02-28T23:07:45.633 INFO:tasks.workunit.client.1.smithi084.stdout:0/785: write d1/fef [208086,104894] 0 2024-02-28T23:07:45.634 INFO:tasks.workunit.client.0.smithi057.stdout:1/562: fdatasync d2/d71/d2d/f4c 0 2024-02-28T23:07:45.638 INFO:tasks.workunit.client.0.smithi057.stdout:0/718: unlink d17/d20/f51 0 2024-02-28T23:07:45.640 INFO:tasks.workunit.client.0.smithi057.stdout:7/622: symlink d0/d5/d36/d4e/d15/d29/d3d/lce 0 2024-02-28T23:07:45.640 INFO:tasks.workunit.client.0.smithi057.stdout:0/719: chown d17/c21 42763 1 2024-02-28T23:07:45.642 INFO:tasks.workunit.client.1.smithi084.stdout:7/868: rename db/d10/d17/d9c/db9 to db/d10/d17/d9c/d9f/d107 0 2024-02-28T23:07:45.643 INFO:tasks.workunit.client.1.smithi084.stdout:5/795: unlink dc/de/df/d31/dd1/da4/lfd 0 2024-02-28T23:07:45.643 INFO:tasks.workunit.client.0.smithi057.stdout:0/720: chown d17/d20/d26/l40 6 1 2024-02-28T23:07:45.647 INFO:tasks.workunit.client.0.smithi057.stdout:1/563: truncate d2/d33/d54/f89 1426149 0 2024-02-28T23:07:45.648 INFO:tasks.workunit.client.0.smithi057.stdout:0/721: write d17/d20/d23/d47/d60/fd3 [3891922,18788] 0 2024-02-28T23:07:45.651 INFO:tasks.workunit.client.1.smithi084.stdout:4/740: dwrite d1/d18/d20/d65/d8e/f5a [4194304,4194304] 0 2024-02-28T23:07:45.652 INFO:tasks.workunit.client.0.smithi057.stdout:7/623: mknod d0/d5/d25/ccf 0 2024-02-28T23:07:45.656 INFO:tasks.workunit.client.0.smithi057.stdout:0/722: fdatasync d17/d20/d4a/d78/d9d/fd6 0 2024-02-28T23:07:45.656 INFO:tasks.workunit.client.1.smithi084.stdout:6/746: dread d1/fe6 [0,4194304] 0 2024-02-28T23:07:45.657 INFO:tasks.workunit.client.1.smithi084.stdout:1/840: rename d12/d1b/d3b/f42 to d12/d1b/d91/d9b/da5/dcb/dd4/f113 0 2024-02-28T23:07:45.658 INFO:tasks.workunit.client.0.smithi057.stdout:8/689: write d4/de9/d56/d6a/f4d [5938453,110092] 0 2024-02-28T23:07:45.658 INFO:tasks.workunit.client.0.smithi057.stdout:1/564: creat d2/d7/df/db/d3a/d5a/fcb x:0 0 0 2024-02-28T23:07:45.661 INFO:tasks.workunit.client.0.smithi057.stdout:0/723: chown d17/d20/d26/d48/l5f 1 1 2024-02-28T23:07:45.662 INFO:tasks.workunit.client.1.smithi084.stdout:4/741: dread - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea zero size 2024-02-28T23:07:45.662 INFO:tasks.workunit.client.1.smithi084.stdout:7/869: symlink db/d10/d17/d3b/l108 0 2024-02-28T23:07:45.663 INFO:tasks.workunit.client.0.smithi057.stdout:8/690: fsync d4/de9/d56/d6a/d42/f5b 0 2024-02-28T23:07:45.664 INFO:tasks.workunit.client.1.smithi084.stdout:5/796: symlink dc/de/df/d31/d8e/da9/l12e 0 2024-02-28T23:07:45.664 INFO:tasks.workunit.client.0.smithi057.stdout:1/565: write d2/f44 [1422586,110428] 0 2024-02-28T23:07:45.666 INFO:tasks.workunit.client.0.smithi057.stdout:0/724: chown d17/d19/dd2/fe0 443 1 2024-02-28T23:07:45.667 INFO:tasks.workunit.client.0.smithi057.stdout:1/566: stat d2/d71/d2d 0 2024-02-28T23:07:45.667 INFO:tasks.workunit.client.1.smithi084.stdout:0/786: symlink d1/dc0/lfc 0 2024-02-28T23:07:45.669 INFO:tasks.workunit.client.1.smithi084.stdout:2/710: rename d5/d1f/d83/fbf to d5/dd/d69/fd8 0 2024-02-28T23:07:45.670 INFO:tasks.workunit.client.0.smithi057.stdout:8/691: dread - d4/de9/d56/d6a/ff4 zero size 2024-02-28T23:07:45.673 INFO:tasks.workunit.client.0.smithi057.stdout:1/567: read d2/d7/f41 [1264354,74266] 0 2024-02-28T23:07:45.674 INFO:tasks.workunit.client.1.smithi084.stdout:3/825: dwrite d3/d50/d117/d7b/dbf/f114 [0,4194304] 0 2024-02-28T23:07:45.674 INFO:tasks.workunit.client.1.smithi084.stdout:6/747: unlink d1/d13/d2e/d41/f92 0 2024-02-28T23:07:45.674 INFO:tasks.workunit.client.1.smithi084.stdout:1/841: write d12/d1b/d91/d9b/da5/dcb/dd4/fe6 [2964054,34731] 0 2024-02-28T23:07:45.675 INFO:tasks.workunit.client.0.smithi057.stdout:4/620: dwrite d1a/d2f/f45 [0,4194304] 0 2024-02-28T23:07:45.677 INFO:tasks.workunit.client.1.smithi084.stdout:2/711: chown d5/d1f/d42/d8f/fd6 8 1 2024-02-28T23:07:45.678 INFO:tasks.workunit.client.1.smithi084.stdout:8/772: dread d0/d9/d1b/dc1/fc7 [0,4194304] 0 2024-02-28T23:07:45.683 INFO:tasks.workunit.client.1.smithi084.stdout:5/797: creat dc/de/df/d31/dd1/d5f/d9b/f12f x:0 0 0 2024-02-28T23:07:45.685 INFO:tasks.workunit.client.1.smithi084.stdout:5/798: chown dc 11701085 1 2024-02-28T23:07:45.685 INFO:tasks.workunit.client.1.smithi084.stdout:8/773: truncate d0/d9/d30/db6/fe9 537577 0 2024-02-28T23:07:45.686 INFO:tasks.workunit.client.0.smithi057.stdout:4/621: dread - d1a/d48/d97/d7f/fcc zero size 2024-02-28T23:07:45.688 INFO:tasks.workunit.client.1.smithi084.stdout:7/870: getdents db/d10/d18/dc2 0 2024-02-28T23:07:45.689 INFO:tasks.workunit.client.0.smithi057.stdout:8/692: getdents d4/de9/d56/d6a/d54/d6b/dca 0 2024-02-28T23:07:45.690 INFO:tasks.workunit.client.0.smithi057.stdout:7/624: truncate d0/d5/d36/d4e/d3b/f5d 1889078 0 2024-02-28T23:07:45.691 INFO:tasks.workunit.client.1.smithi084.stdout:6/748: unlink d1/d13/d2e/d41/d4d/l9f 0 2024-02-28T23:07:45.692 INFO:tasks.workunit.client.1.smithi084.stdout:1/842: fdatasync d12/f45 0 2024-02-28T23:07:45.695 INFO:tasks.workunit.client.0.smithi057.stdout:4/622: link f16 d1a/d2f/db2/d49/d4d/dc9/fd5 0 2024-02-28T23:07:45.697 INFO:tasks.workunit.client.0.smithi057.stdout:1/568: dwrite d2/d33/fae [0,4194304] 0 2024-02-28T23:07:45.700 INFO:tasks.workunit.client.0.smithi057.stdout:1/569: dread - d2/d71/d62/fa6 zero size 2024-02-28T23:07:45.701 INFO:tasks.workunit.client.1.smithi084.stdout:9/821: write d4/d6/d15/d75/d2d/dbd/fd7 [26037,124851] 0 2024-02-28T23:07:45.701 INFO:tasks.workunit.client.1.smithi084.stdout:3/826: mkdir d3/d50/d117/d134 0 2024-02-28T23:07:45.701 INFO:tasks.workunit.client.1.smithi084.stdout:4/742: creat d1/d18/d20/d24/d2f/d3e/d52/d45/db5/ff4 x:0 0 0 2024-02-28T23:07:45.701 INFO:tasks.workunit.client.1.smithi084.stdout:9/822: readlink d4/la1 0 2024-02-28T23:07:45.703 INFO:tasks.workunit.client.1.smithi084.stdout:7/871: truncate db/d10/d17/f9a 1222213 0 2024-02-28T23:07:45.704 INFO:tasks.workunit.client.0.smithi057.stdout:1/570: dread - d2/d7/df/db/d3a/d5a/fcb zero size 2024-02-28T23:07:45.706 INFO:tasks.workunit.client.0.smithi057.stdout:8/693: truncate d4/de9/d56/d6a/d54/d6b/dca/fda 2179078 0 2024-02-28T23:07:45.706 INFO:tasks.workunit.client.0.smithi057.stdout:1/571: fdatasync d2/d71/d2d/f49 0 2024-02-28T23:07:45.706 INFO:tasks.workunit.client.0.smithi057.stdout:7/625: getdents d0/d5/d36/d4e/d15/da0 0 2024-02-28T23:07:45.708 INFO:tasks.workunit.client.0.smithi057.stdout:7/626: readlink d0/d5/d25/d3e/lb4 0 2024-02-28T23:07:45.710 INFO:tasks.workunit.client.0.smithi057.stdout:8/694: write d4/de9/d56/d6a/d54/f70 [5049848,34998] 0 2024-02-28T23:07:45.714 INFO:tasks.workunit.client.1.smithi084.stdout:8/774: rename d0/d9/d1b/d20/f59 to d0/d44/f102 0 2024-02-28T23:07:45.716 INFO:tasks.workunit.client.1.smithi084.stdout:0/787: creat d1/d6/db/d37/ffd x:0 0 0 2024-02-28T23:07:45.719 INFO:tasks.workunit.client.1.smithi084.stdout:0/788: readlink d1/d8/d5f/d93/le6 0 2024-02-28T23:07:45.722 INFO:tasks.workunit.client.1.smithi084.stdout:3/827: creat d3/dae/f135 x:0 0 0 2024-02-28T23:07:45.723 INFO:tasks.workunit.client.0.smithi057.stdout:8/695: rename d4/d13/d18/f26 to d4/de9/d7c/ff6 0 2024-02-28T23:07:45.723 INFO:tasks.workunit.client.0.smithi057.stdout:0/725: dread d17/d19/f2f [0,4194304] 0 2024-02-28T23:07:45.725 INFO:tasks.workunit.client.0.smithi057.stdout:4/623: fsync f16 0 2024-02-28T23:07:45.726 INFO:tasks.workunit.client.1.smithi084.stdout:1/843: dwrite d12/d1b/d2e/d30/d50/da8/dd9/df0/f103 [0,4194304] 0 2024-02-28T23:07:45.726 INFO:tasks.workunit.client.0.smithi057.stdout:0/726: fdatasync d17/d19/fe1 0 2024-02-28T23:07:45.729 INFO:tasks.workunit.client.1.smithi084.stdout:3/828: read d3/d27/d7d/db0/d1a/df9/f119 [923179,60433] 0 2024-02-28T23:07:45.729 INFO:tasks.workunit.client.0.smithi057.stdout:7/627: creat d0/fd0 x:0 0 0 2024-02-28T23:07:45.730 INFO:tasks.workunit.client.1.smithi084.stdout:7/872: creat db/d10/d17/d9c/d9f/d107/f109 x:0 0 0 2024-02-28T23:07:45.730 INFO:tasks.workunit.client.0.smithi057.stdout:0/727: mknod d17/d20/d26/c108 0 2024-02-28T23:07:45.731 INFO:tasks.workunit.client.0.smithi057.stdout:1/572: dwrite d2/d7/df/db/d3a/f99 [0,4194304] 0 2024-02-28T23:07:45.731 INFO:tasks.workunit.client.1.smithi084.stdout:4/743: mknod d1/d18/d20/d24/d2f/d3e/d52/d45/db5/cf5 0 2024-02-28T23:07:45.733 INFO:tasks.workunit.client.1.smithi084.stdout:3/829: read d3/fbd [4217744,73734] 0 2024-02-28T23:07:45.733 INFO:tasks.workunit.client.0.smithi057.stdout:0/728: fsync d17/d19/dd2/fe6 0 2024-02-28T23:07:45.734 INFO:tasks.workunit.client.0.smithi057.stdout:4/624: mkdir d1a/d2f/db2/d49/d4d/dc9/dd6 0 2024-02-28T23:07:45.734 INFO:tasks.workunit.client.0.smithi057.stdout:8/696: getdents d4/de9/d56/d6a/dbd 0 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.1.smithi084.stdout:4/744: chown d1/d18/d20/d24/d2f/d3e/d52/d45/f93 127 1 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.0.smithi057.stdout:3/548: sync 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.0.smithi057.stdout:6/638: sync 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.0.smithi057.stdout:9/635: sync 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.0.smithi057.stdout:2/724: sync 2024-02-28T23:07:45.735 INFO:tasks.workunit.client.1.smithi084.stdout:3/830: write d3/d27/d7d/db0/d1a/d29/d41/fff [446604,71947] 0 2024-02-28T23:07:45.737 INFO:tasks.workunit.client.1.smithi084.stdout:4/745: write d1/f97 [2850698,120587] 0 2024-02-28T23:07:45.737 INFO:tasks.workunit.client.1.smithi084.stdout:8/775: symlink d0/d9/d30/db6/l103 0 2024-02-28T23:07:45.744 INFO:tasks.workunit.client.1.smithi084.stdout:6/749: chown d1/d9/d1d/d1e/d33/f6d 13156644 1 2024-02-28T23:07:45.744 INFO:tasks.workunit.client.1.smithi084.stdout:8/776: stat d0/d9/d1b/l7a 0 2024-02-28T23:07:45.745 INFO:tasks.workunit.client.0.smithi057.stdout:4/625: creat d1a/d48/d9c/d68/fd7 x:0 0 0 2024-02-28T23:07:45.745 INFO:tasks.workunit.client.0.smithi057.stdout:9/636: readlink dd/d46/l68 0 2024-02-28T23:07:45.746 INFO:tasks.workunit.client.0.smithi057.stdout:3/549: mknod d3/d4f/d4c/d91/cac 0 2024-02-28T23:07:45.747 INFO:tasks.workunit.client.0.smithi057.stdout:4/626: fdatasync d1a/d1e/d81/fca 0 2024-02-28T23:07:45.749 INFO:tasks.workunit.client.0.smithi057.stdout:9/637: chown c0 64567 1 2024-02-28T23:07:45.749 INFO:tasks.workunit.client.0.smithi057.stdout:3/550: write d3/d4f/f83 [711050,3963] 0 2024-02-28T23:07:45.752 INFO:tasks.workunit.client.1.smithi084.stdout:1/844: write d12/f45 [1691613,30973] 0 2024-02-28T23:07:45.752 INFO:tasks.workunit.client.1.smithi084.stdout:6/750: write d1/d13/d2e/d41/d4d/d6c/f7c [817854,125954] 0 2024-02-28T23:07:45.753 INFO:tasks.workunit.client.1.smithi084.stdout:2/712: getdents d5/d7/d12/d24/d2a/d35 0 2024-02-28T23:07:45.754 INFO:tasks.workunit.client.1.smithi084.stdout:7/873: dwrite db/d10/d17/d64/d77/f85 [0,4194304] 0 2024-02-28T23:07:45.754 INFO:tasks.workunit.client.0.smithi057.stdout:9/638: chown dd/de/d14/d48/d4b/d41/fad 834489477 1 2024-02-28T23:07:45.755 INFO:tasks.workunit.client.0.smithi057.stdout:4/627: symlink d1a/d48/d9c/d2e/ld8 0 2024-02-28T23:07:45.755 INFO:tasks.workunit.client.1.smithi084.stdout:5/799: dwrite dc/d51/d7c/fc3 [0,4194304] 0 2024-02-28T23:07:45.757 INFO:tasks.workunit.client.0.smithi057.stdout:7/628: dwrite d0/d5/d36/d4e/d15/d29/d3d/f88 [0,4194304] 0 2024-02-28T23:07:45.759 INFO:tasks.workunit.client.1.smithi084.stdout:7/874: chown db/d10/d17/d64/d77/l79 33 1 2024-02-28T23:07:45.761 INFO:tasks.workunit.client.0.smithi057.stdout:9/639: fsync dd/de/d24/f85 0 2024-02-28T23:07:45.763 INFO:tasks.workunit.client.1.smithi084.stdout:3/831: symlink d3/d50/d117/d3d/d43/d61/l136 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:2/725: dread d8/d11/d15/f35 [0,4194304] 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:4/628: creat d1a/d2f/db2/dcb/fd9 x:0 0 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:7/629: mknod d0/d5/d25/d3e/cd1 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:0/729: dwrite d17/d20/d23/d4b/fb9 [4194304,4194304] 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:1/573: read d2/d71/d2d/f46 [534350,83271] 0 2024-02-28T23:07:45.770 INFO:tasks.workunit.client.0.smithi057.stdout:9/640: write dd/de/d24/d63/dc1/fc5 [875886,30802] 0 2024-02-28T23:07:45.771 INFO:tasks.workunit.client.1.smithi084.stdout:4/746: unlink d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/c9e 0 2024-02-28T23:07:45.772 INFO:tasks.workunit.client.0.smithi057.stdout:2/726: mkdir d8/dae/de1 0 2024-02-28T23:07:45.780 INFO:tasks.workunit.client.1.smithi084.stdout:1/845: creat d12/d1b/d2e/d30/f114 x:0 0 0 2024-02-28T23:07:45.780 INFO:tasks.workunit.client.1.smithi084.stdout:2/713: mknod d5/d7/d9/d7c/d5b/cd9 0 2024-02-28T23:07:45.781 INFO:tasks.workunit.client.0.smithi057.stdout:1/574: fdatasync d2/d7/df/db/d3a/d8f/f94 0 2024-02-28T23:07:45.782 INFO:tasks.workunit.client.0.smithi057.stdout:4/629: truncate d1a/d48/d9c/d2e/f98 478220 0 2024-02-28T23:07:45.784 INFO:tasks.workunit.client.0.smithi057.stdout:0/730: read f10 [3547342,130858] 0 2024-02-28T23:07:45.786 INFO:tasks.workunit.client.1.smithi084.stdout:5/800: mkdir dc/de/df/d31/dd1/d5f/d130 0 2024-02-28T23:07:45.787 INFO:tasks.workunit.client.1.smithi084.stdout:7/875: stat db/d10/d17/d9c/d9f/db1/d36/c4b 0 2024-02-28T23:07:45.787 INFO:tasks.workunit.client.0.smithi057.stdout:9/641: creat dd/de/d14/d48/d76/dbf/fd2 x:0 0 0 2024-02-28T23:07:45.788 INFO:tasks.workunit.client.1.smithi084.stdout:2/714: dread d5/d7/d12/d24/d2a/d35/f8e [0,4194304] 0 2024-02-28T23:07:45.791 INFO:tasks.workunit.client.1.smithi084.stdout:9/823: getdents d4/d6/d53 0 2024-02-28T23:07:45.791 INFO:tasks.workunit.client.0.smithi057.stdout:7/630: link d0/f7e d0/d5/d36/d4e/fd2 0 2024-02-28T23:07:45.792 INFO:tasks.workunit.client.1.smithi084.stdout:8/777: fsync d0/d24/d78/fbd 0 2024-02-28T23:07:45.795 INFO:tasks.workunit.client.0.smithi057.stdout:4/630: creat d1a/d48/d9c/fda x:0 0 0 2024-02-28T23:07:45.795 INFO:tasks.workunit.client.0.smithi057.stdout:2/727: dread d8/d11/d21/d28/d49/d85/f9f [0,4194304] 0 2024-02-28T23:07:45.796 INFO:tasks.workunit.client.0.smithi057.stdout:1/575: mknod d2/d7/d4f/db1/ccc 0 2024-02-28T23:07:45.796 INFO:tasks.workunit.client.0.smithi057.stdout:7/631: truncate d0/d5/f55 2679580 0 2024-02-28T23:07:45.797 INFO:tasks.workunit.client.1.smithi084.stdout:4/747: dwrite d1/d18/d20/d65/d8e/d55/f16 [4194304,4194304] 0 2024-02-28T23:07:45.799 INFO:tasks.workunit.client.0.smithi057.stdout:0/731: getdents d17/d20/df3 0 2024-02-28T23:07:45.801 INFO:tasks.workunit.client.1.smithi084.stdout:9/824: mknod d4/d6/d15/d3c/c10a 0 2024-02-28T23:07:45.802 INFO:tasks.workunit.client.0.smithi057.stdout:4/631: symlink d1a/d48/d9c/d68/dc5/ldb 0 2024-02-28T23:07:45.804 INFO:tasks.workunit.client.1.smithi084.stdout:0/789: write d1/d8/d18/d4c/d6c/d74/f97 [53017,57738] 0 2024-02-28T23:07:45.804 INFO:tasks.workunit.client.0.smithi057.stdout:3/551: dwrite d3/d4/d7/f6a [0,4194304] 0 2024-02-28T23:07:45.805 INFO:tasks.workunit.client.0.smithi057.stdout:7/632: chown d0/d5/d25/d3e/lb4 3918554 1 2024-02-28T23:07:45.806 INFO:tasks.workunit.client.1.smithi084.stdout:8/778: unlink d0/l57 0 2024-02-28T23:07:45.806 INFO:tasks.workunit.client.0.smithi057.stdout:2/728: rmdir d8/d11/d21/d28/d41 39 2024-02-28T23:07:45.810 INFO:tasks.workunit.client.1.smithi084.stdout:2/715: read d5/d7/d9/d7c/fcc [296593,31732] 0 2024-02-28T23:07:45.813 INFO:tasks.workunit.client.1.smithi084.stdout:9/825: creat d4/d32/d3f/d89/f10b x:0 0 0 2024-02-28T23:07:45.813 INFO:tasks.workunit.client.1.smithi084.stdout:4/748: mkdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/df6 0 2024-02-28T23:07:45.819 INFO:tasks.workunit.client.0.smithi057.stdout:1/576: truncate d2/d7/df/db/d3a/d2a/f7a 1548345 0 2024-02-28T23:07:45.820 INFO:tasks.workunit.client.0.smithi057.stdout:7/633: symlink d0/d3f/d76/ld3 0 2024-02-28T23:07:45.820 INFO:tasks.workunit.client.1.smithi084.stdout:2/716: write d5/d1f/d42/d74/fd0 [556463,69859] 0 2024-02-28T23:07:45.820 INFO:tasks.workunit.client.1.smithi084.stdout:6/751: truncate d1/fdb 110081 0 2024-02-28T23:07:45.821 INFO:tasks.workunit.client.1.smithi084.stdout:5/801: write dc/de/df/d31/d8e/da9/dab/fe6 [255315,52175] 0 2024-02-28T23:07:45.823 INFO:tasks.workunit.client.1.smithi084.stdout:5/802: readlink dc/lfc 0 2024-02-28T23:07:45.825 INFO:tasks.workunit.client.0.smithi057.stdout:2/729: symlink d8/d11/d21/d28/d49/d85/d9b/ddc/le2 0 2024-02-28T23:07:45.825 INFO:tasks.workunit.client.1.smithi084.stdout:0/790: mknod d1/d8/d18/d4c/cfe 0 2024-02-28T23:07:45.826 INFO:tasks.workunit.client.1.smithi084.stdout:3/832: dwrite d3/d27/d7d/db0/f10 [0,4194304] 0 2024-02-28T23:07:45.830 INFO:tasks.workunit.client.0.smithi057.stdout:0/732: dwrite d17/d20/d26/d48/f5c [0,4194304] 0 2024-02-28T23:07:45.831 INFO:tasks.workunit.client.1.smithi084.stdout:5/803: write dc/d51/f115 [656482,6907] 0 2024-02-28T23:07:45.833 INFO:tasks.workunit.client.1.smithi084.stdout:3/833: stat d3/d50/d117/d3d/d43/f70 0 2024-02-28T23:07:45.834 INFO:tasks.workunit.client.0.smithi057.stdout:2/730: chown d8/d11/d21/d28/d41/f5c 11 1 2024-02-28T23:07:45.835 INFO:tasks.workunit.client.1.smithi084.stdout:2/717: mknod d5/d7/d12/d24/d2a/cda 0 2024-02-28T23:07:45.835 INFO:tasks.workunit.client.1.smithi084.stdout:0/791: write d1/d8/f3c [847396,78050] 0 2024-02-28T23:07:45.836 INFO:tasks.workunit.client.0.smithi057.stdout:4/632: link d1a/d2f/db2/d49/c57 d1a/d48/d9c/d2e/d8f/db7/cdc 0 2024-02-28T23:07:45.838 INFO:tasks.workunit.client.1.smithi084.stdout:1/846: dread d12/f23 [0,4194304] 0 2024-02-28T23:07:45.840 INFO:tasks.workunit.client.0.smithi057.stdout:0/733: symlink d17/d20/d26/d48/d55/l109 0 2024-02-28T23:07:45.841 INFO:tasks.workunit.client.1.smithi084.stdout:7/876: dread db/d10/f56 [0,4194304] 0 2024-02-28T23:07:45.841 INFO:tasks.workunit.client.0.smithi057.stdout:2/731: fdatasync d8/d11/d39/d83/fa3 0 2024-02-28T23:07:45.843 INFO:tasks.workunit.client.0.smithi057.stdout:2/732: stat d8/d11/d56/f9d 0 2024-02-28T23:07:45.844 INFO:tasks.workunit.client.0.smithi057.stdout:0/734: write d17/d20/d23/d47/f5b [2920409,47729] 0 2024-02-28T23:07:45.845 INFO:tasks.workunit.client.1.smithi084.stdout:6/752: creat d1/d13/d50/dbd/df9/f100 x:0 0 0 2024-02-28T23:07:45.847 INFO:tasks.workunit.client.0.smithi057.stdout:0/735: write d17/d20/d23/d47/d60/fd3 [9393246,55930] 0 2024-02-28T23:07:45.848 INFO:tasks.workunit.client.1.smithi084.stdout:1/847: read d12/d1b/d2e/d52/d5b/f62 [273796,34107] 0 2024-02-28T23:07:45.848 INFO:tasks.workunit.client.1.smithi084.stdout:2/718: write d5/d7/d12/d24/d2a/d35/f8e [705812,9797] 0 2024-02-28T23:07:45.849 INFO:tasks.workunit.client.0.smithi057.stdout:2/733: mkdir d8/d11/d39/d83/dde/de3 0 2024-02-28T23:07:45.852 INFO:tasks.workunit.client.0.smithi057.stdout:4/633: read d1a/d48/d9c/d2e/d43/f47 [578317,20065] 0 2024-02-28T23:07:45.856 INFO:tasks.workunit.client.0.smithi057.stdout:0/736: creat d17/d20/d26/d30/f10a x:0 0 0 2024-02-28T23:07:45.858 INFO:tasks.workunit.client.1.smithi084.stdout:3/834: symlink d3/d50/d117/d7b/l137 0 2024-02-28T23:07:45.860 INFO:tasks.workunit.client.0.smithi057.stdout:0/737: creat d17/d20/db6/f10b x:0 0 0 2024-02-28T23:07:45.862 INFO:tasks.workunit.client.1.smithi084.stdout:7/877: mkdir db/d5e/d8a/d10a 0 2024-02-28T23:07:45.862 INFO:tasks.workunit.client.1.smithi084.stdout:0/792: mkdir d1/d6/db/d37/dae/dff 0 2024-02-28T23:07:45.864 INFO:tasks.workunit.client.0.smithi057.stdout:0/738: read d17/f9c [485207,124455] 0 2024-02-28T23:07:45.867 INFO:tasks.workunit.client.0.smithi057.stdout:7/634: dwrite d0/d5/d36/d4e/d15/d29/d71/f8b [0,4194304] 0 2024-02-28T23:07:45.868 INFO:tasks.workunit.client.0.smithi057.stdout:5/616: sync 2024-02-28T23:07:45.871 INFO:tasks.workunit.client.1.smithi084.stdout:1/848: chown d12/d1b/d2e/d52/d5b/ca4 1 1 2024-02-28T23:07:45.872 INFO:tasks.workunit.client.0.smithi057.stdout:5/617: chown d4/d21/d24/d72/f8e 301048246 1 2024-02-28T23:07:45.872 INFO:tasks.workunit.client.0.smithi057.stdout:7/635: stat d0/d5/d25/d80/d9f/fac 0 2024-02-28T23:07:45.872 INFO:tasks.workunit.client.1.smithi084.stdout:9/826: truncate d4/f13 2204830 0 2024-02-28T23:07:45.872 INFO:tasks.workunit.client.1.smithi084.stdout:7/878: mknod db/d10/d18/db7/dbb/c10b 0 2024-02-28T23:07:45.874 INFO:tasks.workunit.client.1.smithi084.stdout:3/835: symlink d3/d50/d117/l138 0 2024-02-28T23:07:45.876 INFO:tasks.workunit.client.0.smithi057.stdout:5/618: symlink d4/d9/d14/d3b/d61/d71/lc2 0 2024-02-28T23:07:45.876 INFO:tasks.workunit.client.1.smithi084.stdout:4/749: dwrite d1/d18/d20/d24/d2f/d3e/f60 [0,4194304] 0 2024-02-28T23:07:45.879 INFO:tasks.workunit.client.1.smithi084.stdout:6/753: dread d1/d88/dbf/fc7 [0,4194304] 0 2024-02-28T23:07:45.879 INFO:tasks.workunit.client.0.smithi057.stdout:7/636: write d0/d5/d36/d4e/d15/d29/d3d/f7a [2424950,57674] 0 2024-02-28T23:07:45.881 INFO:tasks.workunit.client.1.smithi084.stdout:9/827: stat d4/d6/d15/d75/d2d/daa/ccc 0 2024-02-28T23:07:45.881 INFO:tasks.workunit.client.1.smithi084.stdout:8/779: dread d0/d9/f15 [0,4194304] 0 2024-02-28T23:07:45.881 INFO:tasks.workunit.client.1.smithi084.stdout:4/750: fsync d1/dd9/fef 0 2024-02-28T23:07:45.882 INFO:tasks.workunit.client.0.smithi057.stdout:3/552: dread d3/d4f/f48 [4194304,4194304] 0 2024-02-28T23:07:45.882 INFO:tasks.workunit.client.1.smithi084.stdout:5/804: rename dc/de/df/d16/d34/fea to dc/de/df/d16/d34/daa/f131 0 2024-02-28T23:07:45.882 INFO:tasks.workunit.client.1.smithi084.stdout:0/793: creat d1/d6/db/d35/d82/f100 x:0 0 0 2024-02-28T23:07:45.888 INFO:tasks.workunit.client.0.smithi057.stdout:4/634: truncate d1a/d48/d9c/d2e/f7d 6395397 0 2024-02-28T23:07:45.888 INFO:tasks.workunit.client.1.smithi084.stdout:7/879: creat db/d10/d17/d9c/d9f/d106/f10c x:0 0 0 2024-02-28T23:07:45.888 INFO:tasks.workunit.client.1.smithi084.stdout:1/849: link d12/d1b/d2e/fe5 d12/d1b/d2e/f115 0 2024-02-28T23:07:45.888 INFO:tasks.workunit.client.1.smithi084.stdout:4/751: write d1/d18/d20/d24/d2f/d3e/d52/f9a [2965915,21525] 0 2024-02-28T23:07:45.889 INFO:tasks.workunit.client.1.smithi084.stdout:8/780: write d0/d24/f5b [3281309,69230] 0 2024-02-28T23:07:45.891 INFO:tasks.workunit.client.1.smithi084.stdout:5/805: fsync dc/de/df/d16/f95 0 2024-02-28T23:07:45.891 INFO:tasks.workunit.client.0.smithi057.stdout:5/619: creat d4/d9/d16/d52/d7a/fc3 x:0 0 0 2024-02-28T23:07:45.892 INFO:tasks.workunit.client.0.smithi057.stdout:0/739: write d17/d20/d23/d4b/fae [255821,79092] 0 2024-02-28T23:07:45.894 INFO:tasks.workunit.client.1.smithi084.stdout:3/836: creat d3/d27/d7d/db0/d14/d31/f139 x:0 0 0 2024-02-28T23:07:45.896 INFO:tasks.workunit.client.1.smithi084.stdout:5/806: dread - dc/de/d83/d11c/fff zero size 2024-02-28T23:07:45.896 INFO:tasks.workunit.client.1.smithi084.stdout:8/781: chown d0/d44/d9f/f7e 88046527 1 2024-02-28T23:07:45.896 INFO:tasks.workunit.client.1.smithi084.stdout:6/754: write d1/d13/d50/d74/d9e/fb9 [853838,57268] 0 2024-02-28T23:07:45.897 INFO:tasks.workunit.client.0.smithi057.stdout:1/577: dread d2/d7/d4f/f63 [0,4194304] 0 2024-02-28T23:07:45.898 INFO:tasks.workunit.client.0.smithi057.stdout:4/635: rename c6 to d1a/d2f/db2/d49/cdd 0 2024-02-28T23:07:45.901 INFO:tasks.workunit.client.1.smithi084.stdout:0/794: creat d1/d8/d69/d6b/d85/da5/f101 x:0 0 0 2024-02-28T23:07:45.902 INFO:tasks.workunit.client.0.smithi057.stdout:0/740: write d17/d19/fe1 [359863,17150] 0 2024-02-28T23:07:45.906 INFO:tasks.workunit.client.0.smithi057.stdout:4/636: symlink d1a/d1e/d51/dc3/lde 0 2024-02-28T23:07:45.907 INFO:tasks.workunit.client.0.smithi057.stdout:1/578: write d2/d7/df/db/d12/d19/f9d [3771305,22948] 0 2024-02-28T23:07:45.910 INFO:tasks.workunit.client.0.smithi057.stdout:3/553: dwrite d3/d4/d7/f7f [0,4194304] 0 2024-02-28T23:07:45.910 INFO:tasks.workunit.client.0.smithi057.stdout:0/741: chown d17/d20/d4a/d78/db5 86955 1 2024-02-28T23:07:45.914 INFO:tasks.workunit.client.1.smithi084.stdout:5/807: unlink dc/d51/f103 0 2024-02-28T23:07:45.915 INFO:tasks.workunit.client.0.smithi057.stdout:1/579: mknod d2/d33/daa/ccd 0 2024-02-28T23:07:45.915 INFO:tasks.workunit.client.0.smithi057.stdout:3/554: symlink d3/d8/d1b/lad 0 2024-02-28T23:07:45.915 INFO:tasks.workunit.client.1.smithi084.stdout:6/755: read - d1/d13/d50/d74/d81/fab zero size 2024-02-28T23:07:45.916 INFO:tasks.workunit.client.1.smithi084.stdout:8/782: mkdir d0/d9/d1b/d38/d43/dae/d104 0 2024-02-28T23:07:45.917 INFO:tasks.workunit.client.0.smithi057.stdout:4/637: truncate d1a/d48/db0/db1/fb6 583348 0 2024-02-28T23:07:45.917 INFO:tasks.workunit.client.0.smithi057.stdout:3/555: stat d3/c4b 0 2024-02-28T23:07:45.917 INFO:tasks.workunit.client.0.smithi057.stdout:7/637: dwrite d0/d5/d36/f8c [0,4194304] 0 2024-02-28T23:07:45.919 INFO:tasks.workunit.client.1.smithi084.stdout:7/880: write db/d10/d17/d9c/d9f/fa5 [4448400,118105] 0 2024-02-28T23:07:45.921 INFO:tasks.workunit.client.0.smithi057.stdout:5/620: dread d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:45.923 INFO:tasks.workunit.client.1.smithi084.stdout:4/752: dwrite d1/d18/d20/d24/f2a [4194304,4194304] 0 2024-02-28T23:07:45.924 INFO:tasks.workunit.client.0.smithi057.stdout:1/580: mkdir d2/d7/d4f/db1/dce 0 2024-02-28T23:07:45.925 INFO:tasks.workunit.client.0.smithi057.stdout:4/638: rmdir d1a/d48/d9c/d2e 39 2024-02-28T23:07:45.927 INFO:tasks.workunit.client.0.smithi057.stdout:5/621: creat d4/d21/d24/d72/fc4 x:0 0 0 2024-02-28T23:07:45.927 INFO:tasks.workunit.client.0.smithi057.stdout:7/638: symlink d0/ld4 0 2024-02-28T23:07:45.936 INFO:tasks.workunit.client.0.smithi057.stdout:4/639: creat d1a/d48/d97/d7f/fdf x:0 0 0 2024-02-28T23:07:45.939 INFO:tasks.workunit.client.0.smithi057.stdout:7/639: read d0/d5/d25/d3e/f7d [197766,24605] 0 2024-02-28T23:07:45.940 INFO:tasks.workunit.client.1.smithi084.stdout:0/795: dwrite d1/d6/db/d37/dae/fd4 [0,4194304] 0 2024-02-28T23:07:45.940 INFO:tasks.workunit.client.1.smithi084.stdout:9/828: dwrite d4/f3b [0,4194304] 0 2024-02-28T23:07:45.941 INFO:tasks.workunit.client.0.smithi057.stdout:0/742: dwrite d17/d20/d23/f2c [4194304,4194304] 0 2024-02-28T23:07:45.943 INFO:tasks.workunit.client.0.smithi057.stdout:3/556: rmdir d3/d8 39 2024-02-28T23:07:45.944 INFO:tasks.workunit.client.0.smithi057.stdout:5/622: creat d4/d9/d14/d63/d88/fc5 x:0 0 0 2024-02-28T23:07:45.953 INFO:tasks.workunit.client.0.smithi057.stdout:4/640: symlink d1a/d48/d9c/le0 0 2024-02-28T23:07:45.955 INFO:tasks.workunit.client.0.smithi057.stdout:5/623: fdatasync d4/d9/d14/d63/fb3 0 2024-02-28T23:07:45.958 INFO:tasks.workunit.client.0.smithi057.stdout:0/743: truncate d17/d19/f2f 3359256 0 2024-02-28T23:07:45.959 INFO:tasks.workunit.client.0.smithi057.stdout:1/581: dwrite d2/d33/d54/fc7 [0,4194304] 0 2024-02-28T23:07:45.962 INFO:tasks.workunit.client.1.smithi084.stdout:6/756: mkdir d1/d13/d50/d74/d9e/d101 0 2024-02-28T23:07:45.964 INFO:tasks.workunit.client.0.smithi057.stdout:4/641: symlink d1a/d1e/d81/le1 0 2024-02-28T23:07:45.966 INFO:tasks.workunit.client.1.smithi084.stdout:7/881: symlink db/d10/d18/db7/de9/dfa/dc7/l10d 0 2024-02-28T23:07:45.967 INFO:tasks.workunit.client.0.smithi057.stdout:3/557: link d3/d4/d7/cf d3/d4f/d4c/d91/cae 0 2024-02-28T23:07:45.970 INFO:tasks.workunit.client.0.smithi057.stdout:5/624: fdatasync d4/d9/d16/d4e/f87 0 2024-02-28T23:07:45.971 INFO:tasks.workunit.client.1.smithi084.stdout:0/796: creat d1/d8/d69/d6b/d85/da5/f102 x:0 0 0 2024-02-28T23:07:45.972 INFO:tasks.workunit.client.1.smithi084.stdout:5/808: stat dc/de/df/d16/d34/d3c/d77/df8 0 2024-02-28T23:07:45.973 INFO:tasks.workunit.client.1.smithi084.stdout:4/753: mkdir d1/d18/d20/d24/d2f/df7 0 2024-02-28T23:07:45.974 INFO:tasks.workunit.client.0.smithi057.stdout:0/744: creat d17/d20/d4a/d78/db5/dba/f10c x:0 0 0 2024-02-28T23:07:45.975 INFO:tasks.workunit.client.0.smithi057.stdout:7/640: dread d0/d5/d36/d4e/d15/d29/d3d/f88 [0,4194304] 0 2024-02-28T23:07:45.975 INFO:tasks.workunit.client.1.smithi084.stdout:5/809: write dc/d51/f12a [391108,128782] 0 2024-02-28T23:07:45.977 INFO:tasks.workunit.client.0.smithi057.stdout:5/625: write d4/d9/d16/d4e/f8b [4646375,110967] 0 2024-02-28T23:07:45.979 INFO:tasks.workunit.client.0.smithi057.stdout:4/642: rename d1a/d48/db0/d4f to d1a/d2f/db2/d49/d4d/de2 0 2024-02-28T23:07:45.982 INFO:tasks.workunit.client.1.smithi084.stdout:5/810: symlink dc/de/df/da6/l132 0 2024-02-28T23:07:45.984 INFO:tasks.workunit.client.0.smithi057.stdout:3/558: creat d3/d4/d1d/d47/d62/faf x:0 0 0 2024-02-28T23:07:45.984 INFO:tasks.workunit.client.1.smithi084.stdout:9/829: getdents d4/d6/d15/d3c 0 2024-02-28T23:07:45.993 INFO:tasks.workunit.client.0.smithi057.stdout:5/626: mknod d4/d9/d16/d52/cc6 0 2024-02-28T23:07:45.995 INFO:tasks.workunit.client.0.smithi057.stdout:0/745: creat d17/d20/d4a/d78/df7/f10d x:0 0 0 2024-02-28T23:07:45.996 INFO:tasks.workunit.client.0.smithi057.stdout:4/643: dread d1a/d48/d97/f93 [0,4194304] 0 2024-02-28T23:07:45.997 INFO:tasks.workunit.client.0.smithi057.stdout:5/627: fdatasync d4/d9/d14/d63/d88/fc5 0 2024-02-28T23:07:46.002 INFO:tasks.workunit.client.0.smithi057.stdout:4/644: write d1a/d2f/db2/d49/d4d/de2/db5/fcd [681163,2201] 0 2024-02-28T23:07:46.004 INFO:tasks.workunit.client.0.smithi057.stdout:5/628: truncate d4/d21/f2f 4460589 0 2024-02-28T23:07:46.005 INFO:tasks.workunit.client.0.smithi057.stdout:4/645: stat d1a/d2f/db2/d49/d4d/de2/f64 0 2024-02-28T23:07:46.009 INFO:tasks.workunit.client.0.smithi057.stdout:5/629: mknod d4/d21/d24/d30/d62/cc7 0 2024-02-28T23:07:46.009 INFO:tasks.workunit.client.0.smithi057.stdout:1/582: dread d2/d33/d54/f89 [0,4194304] 0 2024-02-28T23:07:46.016 INFO:tasks.workunit.client.0.smithi057.stdout:5/630: write d4/d9/d16/d4e/f8b [710691,50273] 0 2024-02-28T23:07:46.016 INFO:tasks.workunit.client.0.smithi057.stdout:1/583: creat d2/d71/d62/fcf x:0 0 0 2024-02-28T23:07:46.021 INFO:tasks.workunit.client.0.smithi057.stdout:7/641: dread d0/d5/d36/d4e/d15/f49 [0,4194304] 0 2024-02-28T23:07:46.022 INFO:tasks.workunit.client.1.smithi084.stdout:5/811: symlink dc/de/df/d16/d34/d3c/l133 0 2024-02-28T23:07:46.023 INFO:tasks.workunit.client.1.smithi084.stdout:9/830: mkdir d4/d32/d3f/d89/d10c 0 2024-02-28T23:07:46.025 INFO:tasks.workunit.client.1.smithi084.stdout:1/850: truncate f11 3176780 0 2024-02-28T23:07:46.026 INFO:tasks.workunit.client.0.smithi057.stdout:4/646: dread d1a/d2f/f45 [0,4194304] 0 2024-02-28T23:07:46.026 INFO:tasks.workunit.client.1.smithi084.stdout:8/783: write d0/d9/d1b/f39 [2416848,120149] 0 2024-02-28T23:07:46.028 INFO:tasks.workunit.client.0.smithi057.stdout:7/642: fdatasync d0/d5/d36/d4e/d15/d29/d3d/d51/f72 0 2024-02-28T23:07:46.030 INFO:tasks.workunit.client.1.smithi084.stdout:9/831: mknod d4/d32/d3f/d89/c10d 0 2024-02-28T23:07:46.030 INFO:tasks.workunit.client.1.smithi084.stdout:5/812: write dc/de/df/d16/f119 [202706,109699] 0 2024-02-28T23:07:46.035 INFO:tasks.workunit.client.0.smithi057.stdout:0/746: dread d17/d20/f66 [0,4194304] 0 2024-02-28T23:07:46.037 INFO:tasks.workunit.client.1.smithi084.stdout:1/851: fdatasync d12/d1b/d2e/d52/d5b/fb3 0 2024-02-28T23:07:46.039 INFO:tasks.workunit.client.1.smithi084.stdout:7/882: dread db/d10/d18/db7/de9/dfa/f68 [0,4194304] 0 2024-02-28T23:07:46.041 INFO:tasks.workunit.client.1.smithi084.stdout:8/784: unlink d0/d9/d1b/d38/f3e 0 2024-02-28T23:07:46.045 INFO:tasks.workunit.client.1.smithi084.stdout:4/754: dwrite d1/fbd [0,4194304] 0 2024-02-28T23:07:46.050 INFO:tasks.workunit.client.0.smithi057.stdout:5/631: rename d4/d21/f2f to d4/d9/d16/d4e/fc8 0 2024-02-28T23:07:46.051 INFO:tasks.workunit.client.0.smithi057.stdout:4/647: dwrite d1a/d48/d97/d7f/fdf [0,4194304] 0 2024-02-28T23:07:46.054 INFO:tasks.workunit.client.0.smithi057.stdout:4/648: readlink d1a/d2f/db2/d49/d4d/l96 0 2024-02-28T23:07:46.054 INFO:tasks.workunit.client.1.smithi084.stdout:3/837: dwrite d3/d27/d7d/db0/d14/d31/fbc [0,4194304] 0 2024-02-28T23:07:46.055 INFO:tasks.workunit.client.0.smithi057.stdout:1/584: dread d2/d7/df/f4d [0,4194304] 0 2024-02-28T23:07:46.055 INFO:tasks.workunit.client.0.smithi057.stdout:5/632: mknod d4/d9/d14/d63/d88/cc9 0 2024-02-28T23:07:46.058 INFO:tasks.workunit.client.0.smithi057.stdout:7/643: dread d0/d5/d36/d4e/d15/d29/f32 [0,4194304] 0 2024-02-28T23:07:46.059 INFO:tasks.workunit.client.1.smithi084.stdout:6/757: dread d1/d9/fd5 [0,4194304] 0 2024-02-28T23:07:46.060 INFO:tasks.workunit.client.0.smithi057.stdout:4/649: truncate d1a/d2f/db2/dcb/fd9 914957 0 2024-02-28T23:07:46.060 INFO:tasks.workunit.client.0.smithi057.stdout:1/585: fsync d2/d71/d2d/d5b/faf 0 2024-02-28T23:07:46.064 INFO:tasks.workunit.client.0.smithi057.stdout:1/586: chown d2/d7/df/db/d3a/d5a/fcb 8489343 1 2024-02-28T23:07:46.067 INFO:tasks.workunit.client.0.smithi057.stdout:7/644: mknod d0/d5/d36/d4e/cd5 0 2024-02-28T23:07:46.070 INFO:tasks.workunit.client.1.smithi084.stdout:3/838: creat d3/d27/d7d/db0/d1a/df9/f13a x:0 0 0 2024-02-28T23:07:46.072 INFO:tasks.workunit.client.0.smithi057.stdout:5/633: rename d4/d9/d16/l33 to d4/d9/d14/d3b/d61/d71/lca 0 2024-02-28T23:07:46.073 INFO:tasks.workunit.client.0.smithi057.stdout:1/587: mknod d2/d7/df/db/d12/d19/cd0 0 2024-02-28T23:07:46.075 INFO:tasks.workunit.client.1.smithi084.stdout:7/883: creat db/f10e x:0 0 0 2024-02-28T23:07:46.076 INFO:tasks.workunit.client.0.smithi057.stdout:5/634: chown d4/d9/d14/d3b/d61/d71/da1 1577758 1 2024-02-28T23:07:46.076 INFO:tasks.workunit.client.1.smithi084.stdout:8/785: rename d0/d9/c16 to d0/d9/d30/db6/c105 0 2024-02-28T23:07:46.078 INFO:tasks.workunit.client.1.smithi084.stdout:3/839: write d3/d27/d7d/dd9/fca [3846814,89576] 0 2024-02-28T23:07:46.078 INFO:tasks.workunit.client.0.smithi057.stdout:1/588: readlink d2/d33/daa/lb6 0 2024-02-28T23:07:46.080 INFO:tasks.workunit.client.0.smithi057.stdout:5/635: fdatasync d4/d9/d16/f1a 0 2024-02-28T23:07:46.082 INFO:tasks.workunit.client.0.smithi057.stdout:5/636: readlink d4/d9/d14/d3b/d61/db2/lb8 0 2024-02-28T23:07:46.082 INFO:tasks.workunit.client.1.smithi084.stdout:8/786: rename d0/d9/d1b to d0/d9/d1b/d38/d43/dae/d104/d106 22 2024-02-28T23:07:46.083 INFO:tasks.workunit.client.0.smithi057.stdout:0/747: dread f5 [8388608,4194304] 0 2024-02-28T23:07:46.083 INFO:tasks.workunit.client.0.smithi057.stdout:1/589: creat d2/d71/d2d/d39/fd1 x:0 0 0 2024-02-28T23:07:46.083 INFO:tasks.workunit.client.0.smithi057.stdout:7/645: link d0/d5/d4a/c95 d0/cd6 0 2024-02-28T23:07:46.084 INFO:tasks.workunit.client.1.smithi084.stdout:6/758: truncate d1/d9/d1d/d1e/f5b 3102199 0 2024-02-28T23:07:46.089 INFO:tasks.workunit.client.1.smithi084.stdout:6/759: mknod d1/d88/c102 0 2024-02-28T23:07:46.091 INFO:tasks.workunit.client.1.smithi084.stdout:1/852: dread d12/d1b/d2e/d30/f26 [0,4194304] 0 2024-02-28T23:07:46.092 INFO:tasks.workunit.client.0.smithi057.stdout:1/590: mkdir d2/d33/d54/dc8/dd2 0 2024-02-28T23:07:46.092 INFO:tasks.workunit.client.1.smithi084.stdout:7/884: getdents db/d10/d18/db7/de9 0 2024-02-28T23:07:46.096 INFO:tasks.workunit.client.1.smithi084.stdout:9/832: truncate d4/d6/d15/d12/d41/d55/fbc 2557762 0 2024-02-28T23:07:46.097 INFO:tasks.workunit.client.0.smithi057.stdout:1/591: mkdir d2/d7/df/db/d3a/d5a/dd3 0 2024-02-28T23:07:46.100 INFO:tasks.workunit.client.1.smithi084.stdout:9/833: truncate d4/d6/d15/d12/f109 247388 0 2024-02-28T23:07:46.101 INFO:tasks.workunit.client.0.smithi057.stdout:1/592: mkdir d2/dd4 0 2024-02-28T23:07:46.101 INFO:tasks.workunit.client.0.smithi057.stdout:5/637: dwrite d4/d9/d14/d63/fb3 [0,4194304] 0 2024-02-28T23:07:46.102 INFO:tasks.workunit.client.1.smithi084.stdout:7/885: creat db/d10/d18/db7/de9/dfa/dc7/f10f x:0 0 0 2024-02-28T23:07:46.102 INFO:tasks.workunit.client.1.smithi084.stdout:4/755: dread d1/d18/d20/d65/d8e/d55/f69 [0,4194304] 0 2024-02-28T23:07:46.104 INFO:tasks.workunit.client.1.smithi084.stdout:1/853: mknod d12/d1b/d91/d9b/da5/dcb/dd4/df6/c116 0 2024-02-28T23:07:46.106 INFO:tasks.workunit.client.1.smithi084.stdout:7/886: chown db/d10/d18/fd8 141 1 2024-02-28T23:07:46.106 INFO:tasks.workunit.client.1.smithi084.stdout:9/834: write d4/d32/d3f/f96 [2674045,126676] 0 2024-02-28T23:07:46.109 INFO:tasks.workunit.client.1.smithi084.stdout:6/760: dread d1/d13/d2e/d41/d2c/f71 [0,4194304] 0 2024-02-28T23:07:46.112 INFO:tasks.workunit.client.0.smithi057.stdout:0/748: dwrite d17/d7a/fb7 [0,4194304] 0 2024-02-28T23:07:46.112 INFO:tasks.workunit.client.0.smithi057.stdout:7/646: dwrite d0/d5/d36/d4e/d15/f60 [0,4194304] 0 2024-02-28T23:07:46.113 INFO:tasks.workunit.client.1.smithi084.stdout:4/756: read d1/d18/d20/d14/d30/f3d [217459,99985] 0 2024-02-28T23:07:46.117 INFO:tasks.workunit.client.1.smithi084.stdout:9/835: unlink d4/d6/d15/d12/d41/f65 0 2024-02-28T23:07:46.117 INFO:tasks.workunit.client.0.smithi057.stdout:0/749: chown d17/d20/d23/d4b/lc0 1608024 1 2024-02-28T23:07:46.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/638: link d4/d9/d14/d3b/l74 d4/d7/d58/d78/lcb 0 2024-02-28T23:07:46.123 INFO:tasks.workunit.client.0.smithi057.stdout:0/750: write d17/d20/d26/f46 [3416197,52475] 0 2024-02-28T23:07:46.124 INFO:tasks.workunit.client.1.smithi084.stdout:3/840: dread d3/d50/fd5 [0,4194304] 0 2024-02-28T23:07:46.127 INFO:tasks.workunit.client.1.smithi084.stdout:7/887: creat db/d10/d17/d9c/d9f/d107/f110 x:0 0 0 2024-02-28T23:07:46.127 INFO:tasks.workunit.client.1.smithi084.stdout:6/761: mknod d1/d13/d2e/d41/c103 0 2024-02-28T23:07:46.128 INFO:tasks.workunit.client.0.smithi057.stdout:0/751: chown d17/d20/d23/d47/d60/dea 0 1 2024-02-28T23:07:46.128 INFO:tasks.workunit.client.0.smithi057.stdout:1/593: dwrite d2/d71/d2d/f3d [0,4194304] 0 2024-02-28T23:07:46.128 INFO:tasks.workunit.client.0.smithi057.stdout:5/639: chown d4/d9/d14/d3b/d61/d71/lca 69 1 2024-02-28T23:07:46.129 INFO:tasks.workunit.client.1.smithi084.stdout:9/836: mknod d4/d32/d3f/d89/db8/c10e 0 2024-02-28T23:07:46.130 INFO:tasks.workunit.client.1.smithi084.stdout:3/841: symlink d3/l13b 0 2024-02-28T23:07:46.131 INFO:tasks.workunit.client.0.smithi057.stdout:5/640: write d4/d9/d16/f1a [2833811,123469] 0 2024-02-28T23:07:46.131 INFO:tasks.workunit.client.0.smithi057.stdout:0/752: chown d17/d20/d23/d47/d60/d6f 1437022326 1 2024-02-28T23:07:46.134 INFO:tasks.workunit.client.0.smithi057.stdout:0/753: write d17/d20/d26/d48/d7c/d98/da6/ff8 [878779,101537] 0 2024-02-28T23:07:46.135 INFO:tasks.workunit.client.1.smithi084.stdout:6/762: readlink d1/d13/d50/d74/d9e/dc2/ld6 0 2024-02-28T23:07:46.136 INFO:tasks.workunit.client.1.smithi084.stdout:4/757: mkdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/df6/df8 0 2024-02-28T23:07:46.136 INFO:tasks.workunit.client.0.smithi057.stdout:0/754: readlink d17/d19/l56 0 2024-02-28T23:07:46.137 INFO:tasks.workunit.client.0.smithi057.stdout:5/641: rename d4/d9/d14/f36 to d4/d9/d14/fcc 0 2024-02-28T23:07:46.137 INFO:tasks.workunit.client.1.smithi084.stdout:7/888: read - db/d10/d17/d9c/d9f/db1/db2/fed zero size 2024-02-28T23:07:46.141 INFO:tasks.workunit.client.1.smithi084.stdout:6/763: chown d1/d13/d2e/d41/d2c/dfb/lda 11675907 1 2024-02-28T23:07:46.146 INFO:tasks.workunit.client.1.smithi084.stdout:1/854: dwrite d12/d1b/d2e/d30/d77/da0/fed [0,4194304] 0 2024-02-28T23:07:46.147 INFO:tasks.workunit.client.0.smithi057.stdout:7/647: dread d0/d5/d36/d4e/d15/f3a [0,4194304] 0 2024-02-28T23:07:46.148 INFO:tasks.workunit.client.1.smithi084.stdout:7/889: truncate db/d10/f71 3667311 0 2024-02-28T23:07:46.149 INFO:tasks.workunit.client.0.smithi057.stdout:7/648: readlink d0/d5/d36/d4e/la 0 2024-02-28T23:07:46.151 INFO:tasks.workunit.client.1.smithi084.stdout:1/855: fsync d12/d1b/d2e/d30/f46 0 2024-02-28T23:07:46.152 INFO:tasks.workunit.client.1.smithi084.stdout:4/758: truncate d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/ff3 513973 0 2024-02-28T23:07:46.154 INFO:tasks.workunit.client.1.smithi084.stdout:7/890: write db/d10/d18/dc2/fc9 [3432404,108847] 0 2024-02-28T23:07:46.157 INFO:tasks.workunit.client.0.smithi057.stdout:7/649: link d0/d5/d36/d4e/d15/f60 d0/fd7 0 2024-02-28T23:07:46.160 INFO:tasks.workunit.client.1.smithi084.stdout:1/856: readlink d12/d1b/d2e/d30/d77/lcd 0 2024-02-28T23:07:46.160 INFO:tasks.workunit.client.1.smithi084.stdout:4/759: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/ff9 x:0 0 0 2024-02-28T23:07:46.163 INFO:tasks.workunit.client.1.smithi084.stdout:7/891: write db/d10/d17/d9c/d9f/fa5 [4968456,37562] 0 2024-02-28T23:07:46.169 INFO:tasks.workunit.client.0.smithi057.stdout:0/755: truncate d17/d20/d26/f69 159261 0 2024-02-28T23:07:46.178 INFO:tasks.workunit.client.1.smithi084.stdout:9/837: write d4/d6/d15/d12/d41/d55/fbb [498566,120031] 0 2024-02-28T23:07:46.178 INFO:tasks.workunit.client.1.smithi084.stdout:2/719: sync 2024-02-28T23:07:46.178 INFO:tasks.workunit.client.0.smithi057.stdout:5/642: dwrite d4/d9/d14/d3b/fa0 [0,4194304] 0 2024-02-28T23:07:46.182 INFO:tasks.workunit.client.1.smithi084.stdout:3/842: readlink d3/d27/d7d/db0/d14/lb6 0 2024-02-28T23:07:46.184 INFO:tasks.workunit.client.1.smithi084.stdout:4/760: dread d1/d18/d20/d24/f2a [4194304,4194304] 0 2024-02-28T23:07:46.185 INFO:tasks.workunit.client.0.smithi057.stdout:5/643: write d4/d21/d24/d30/d62/faa [4428529,14491] 0 2024-02-28T23:07:46.186 INFO:tasks.workunit.client.0.smithi057.stdout:0/756: fdatasync d17/d20/d4a/d78/fc7 0 2024-02-28T23:07:46.189 INFO:tasks.workunit.client.0.smithi057.stdout:0/757: symlink d17/d20/d23/d37/l10e 0 2024-02-28T23:07:46.189 INFO:tasks.workunit.client.0.smithi057.stdout:7/650: dread d0/d5/d36/d4e/d3b/f5d [0,4194304] 0 2024-02-28T23:07:46.189 INFO:tasks.workunit.client.1.smithi084.stdout:9/838: mkdir d4/d32/d3f/d89/d10c/d10f 0 2024-02-28T23:07:46.189 INFO:tasks.workunit.client.1.smithi084.stdout:7/892: dread db/d10/d17/d9c/d9f/db1/d36/f8e [0,4194304] 0 2024-02-28T23:07:46.190 INFO:tasks.workunit.client.0.smithi057.stdout:5/644: fsync d4/d9/d16/d52/d7a/f7f 0 2024-02-28T23:07:46.192 INFO:tasks.workunit.client.0.smithi057.stdout:0/758: fdatasync d17/d20/d23/d47/d60/dea/f100 0 2024-02-28T23:07:46.194 INFO:tasks.workunit.client.0.smithi057.stdout:0/759: dread - d17/d20/d23/f75 zero size 2024-02-28T23:07:46.194 INFO:tasks.workunit.client.0.smithi057.stdout:7/651: unlink d0/d5/d36/f63 0 2024-02-28T23:07:46.195 INFO:tasks.workunit.client.1.smithi084.stdout:7/893: stat db/d10/d17/ca0 0 2024-02-28T23:07:46.195 INFO:tasks.workunit.client.0.smithi057.stdout:5/645: creat d4/d9/fcd x:0 0 0 2024-02-28T23:07:46.196 INFO:tasks.workunit.client.1.smithi084.stdout:4/761: creat d1/d18/ffa x:0 0 0 2024-02-28T23:07:46.197 INFO:tasks.workunit.client.1.smithi084.stdout:3/843: symlink d3/d50/d117/d3d/d43/l13c 0 2024-02-28T23:07:46.199 INFO:tasks.workunit.client.1.smithi084.stdout:8/787: dread d0/d9/d1b/d38/d43/d51/d54/fab [0,4194304] 0 2024-02-28T23:07:46.202 INFO:tasks.workunit.client.1.smithi084.stdout:1/857: truncate d12/d1b/d2e/d52/d5b/f6f 3707101 0 2024-02-28T23:07:46.205 INFO:tasks.workunit.client.1.smithi084.stdout:6/764: dwrite d1/d13/d47/d91/dba/f95 [4194304,4194304] 0 2024-02-28T23:07:46.205 INFO:tasks.workunit.client.1.smithi084.stdout:9/839: symlink d4/d6/d15/d12/d41/dea/l110 0 2024-02-28T23:07:46.206 INFO:tasks.workunit.client.1.smithi084.stdout:4/762: mknod d1/d18/d20/d24/ddb/cfb 0 2024-02-28T23:07:46.209 INFO:tasks.workunit.client.1.smithi084.stdout:9/840: chown d4/d6/d15/d75 60 1 2024-02-28T23:07:46.212 INFO:tasks.workunit.client.1.smithi084.stdout:0/797: sync 2024-02-28T23:07:46.212 INFO:tasks.workunit.client.1.smithi084.stdout:5/813: sync 2024-02-28T23:07:46.213 INFO:tasks.workunit.client.1.smithi084.stdout:7/894: creat db/d10/d17/d9c/f111 x:0 0 0 2024-02-28T23:07:46.214 INFO:tasks.workunit.client.1.smithi084.stdout:2/720: dwrite d5/d7/d75/fac [0,4194304] 0 2024-02-28T23:07:46.214 INFO:tasks.workunit.client.1.smithi084.stdout:8/788: write d0/d9/d1b/f74 [3703029,61860] 0 2024-02-28T23:07:46.215 INFO:tasks.workunit.client.0.smithi057.stdout:5/646: truncate d4/d21/f2d 1095349 0 2024-02-28T23:07:46.215 INFO:tasks.workunit.client.1.smithi084.stdout:6/765: write d1/f24 [6653693,81522] 0 2024-02-28T23:07:46.217 INFO:tasks.workunit.client.0.smithi057.stdout:0/760: truncate d17/d20/d26/d48/d7c/d98/da6/ff8 7969 0 2024-02-28T23:07:46.218 INFO:tasks.workunit.client.0.smithi057.stdout:0/761: chown d17/c103 5 1 2024-02-28T23:07:46.219 INFO:tasks.workunit.client.1.smithi084.stdout:3/844: dread d3/d50/d117/d7b/dbf/fc7 [0,4194304] 0 2024-02-28T23:07:46.222 INFO:tasks.workunit.client.0.smithi057.stdout:5/647: unlink d4/d21/d24/f82 0 2024-02-28T23:07:46.222 INFO:tasks.workunit.client.1.smithi084.stdout:5/814: mkdir dc/d134 0 2024-02-28T23:07:46.223 INFO:tasks.workunit.client.1.smithi084.stdout:9/841: mknod d4/d6/d15/d12/d41/dea/c111 0 2024-02-28T23:07:46.224 INFO:tasks.workunit.client.1.smithi084.stdout:8/789: unlink d0/d9/d30/db6/ccf 0 2024-02-28T23:07:46.225 INFO:tasks.workunit.client.1.smithi084.stdout:0/798: write d1/d6/db/d37/ffd [434799,2245] 0 2024-02-28T23:07:46.225 INFO:tasks.workunit.client.1.smithi084.stdout:6/766: chown d1/f1b 36429875 1 2024-02-28T23:07:46.225 INFO:tasks.workunit.client.0.smithi057.stdout:0/762: creat d17/dda/f10f x:0 0 0 2024-02-28T23:07:46.226 INFO:tasks.workunit.client.1.smithi084.stdout:3/845: readlink d3/d50/d117/l36 0 2024-02-28T23:07:46.227 INFO:tasks.workunit.client.0.smithi057.stdout:0/763: chown d17/d20/d26/c64 31952228 1 2024-02-28T23:07:46.228 INFO:tasks.workunit.client.1.smithi084.stdout:7/895: symlink db/d10/d17/d64/d77/l112 0 2024-02-28T23:07:46.228 INFO:tasks.workunit.client.1.smithi084.stdout:2/721: fsync d5/d7/d12/d24/d2a/d35/f8e 0 2024-02-28T23:07:46.232 INFO:tasks.workunit.client.0.smithi057.stdout:5/648: rename d4/d21/d24/f64 to d4/d9/d85/fce 0 2024-02-28T23:07:46.233 INFO:tasks.workunit.client.1.smithi084.stdout:4/763: chown d1/d18/d20/d24/d2f/d3e/d52/d45/db5/cf5 27077 1 2024-02-28T23:07:46.233 INFO:tasks.workunit.client.1.smithi084.stdout:9/842: mknod d4/d6/d15/d75/d2d/daa/c112 0 2024-02-28T23:07:46.234 INFO:tasks.workunit.client.0.smithi057.stdout:0/764: creat d17/d20/d4a/d78/f110 x:0 0 0 2024-02-28T23:07:46.236 INFO:tasks.workunit.client.0.smithi057.stdout:0/765: chown d17/d20/d26/d30/fad 1494341424 1 2024-02-28T23:07:46.239 INFO:tasks.workunit.client.1.smithi084.stdout:0/799: unlink d1/d6/db/d37/dae/fd6 0 2024-02-28T23:07:46.239 INFO:tasks.workunit.client.1.smithi084.stdout:6/767: chown d1/d13/d2e/d41/dbb/lf0 241 1 2024-02-28T23:07:46.239 INFO:tasks.workunit.client.1.smithi084.stdout:8/790: read d0/d9/d30/db6/f80 [1439555,130187] 0 2024-02-28T23:07:46.239 INFO:tasks.workunit.client.0.smithi057.stdout:5/649: read d4/d7/f1e [913527,98153] 0 2024-02-28T23:07:46.241 INFO:tasks.workunit.client.1.smithi084.stdout:2/722: chown d5/d7/d2e/c65 4111 1 2024-02-28T23:07:46.241 INFO:tasks.workunit.client.1.smithi084.stdout:6/768: fdatasync d1/d9/d1d/fe5 0 2024-02-28T23:07:46.241 INFO:tasks.workunit.client.1.smithi084.stdout:7/896: mknod db/d10/d18/dc2/c113 0 2024-02-28T23:07:46.242 INFO:tasks.workunit.client.0.smithi057.stdout:0/766: getdents d17/d20/d26/d48/d7c/d98/da6 0 2024-02-28T23:07:46.243 INFO:tasks.workunit.client.0.smithi057.stdout:5/650: write d4/d9/d14/d63/fb3 [3511185,110752] 0 2024-02-28T23:07:46.243 INFO:tasks.workunit.client.1.smithi084.stdout:9/843: creat d4/d6/d15/d75/d2d/dbd/dde/f113 x:0 0 0 2024-02-28T23:07:46.245 INFO:tasks.workunit.client.0.smithi057.stdout:5/651: write d4/d9/fcd [401176,74109] 0 2024-02-28T23:07:46.245 INFO:tasks.workunit.client.1.smithi084.stdout:6/769: chown d1/d9/f28 15247894 1 2024-02-28T23:07:46.247 INFO:tasks.workunit.client.0.smithi057.stdout:0/767: unlink d17/d20/d4a/d78/f90 0 2024-02-28T23:07:46.251 INFO:tasks.workunit.client.0.smithi057.stdout:0/768: stat d17/d20/d26/d48/l6a 0 2024-02-28T23:07:46.252 INFO:tasks.workunit.client.1.smithi084.stdout:7/897: creat db/d10/d18/db4/f114 x:0 0 0 2024-02-28T23:07:46.253 INFO:tasks.workunit.client.0.smithi057.stdout:0/769: write d17/d20/d4a/d78/f110 [348118,112579] 0 2024-02-28T23:07:46.255 INFO:tasks.workunit.client.0.smithi057.stdout:5/652: write d4/d21/d24/d30/f79 [4362555,21160] 0 2024-02-28T23:07:46.258 INFO:tasks.workunit.client.0.smithi057.stdout:5/653: write d4/d9/f2a [5546945,80994] 0 2024-02-28T23:07:46.262 INFO:tasks.workunit.client.1.smithi084.stdout:3/846: dwrite d3/d27/d7d/db0/d1a/df9/f13a [0,4194304] 0 2024-02-28T23:07:46.262 INFO:tasks.workunit.client.1.smithi084.stdout:0/800: rename d1/d8/d69/da3/fd9 to d1/d6/db/d76/dee/f103 0 2024-02-28T23:07:46.264 INFO:tasks.workunit.client.0.smithi057.stdout:5/654: link d4/d7/d58/d78/lcb d4/d9/d14/d3b/d61/lcf 0 2024-02-28T23:07:46.267 INFO:tasks.workunit.client.1.smithi084.stdout:0/801: fdatasync d1/d6/db/d35/d44/fe3 0 2024-02-28T23:07:46.269 INFO:tasks.workunit.client.1.smithi084.stdout:4/764: dwrite d1/d18/d20/d14/d30/f3d [0,4194304] 0 2024-02-28T23:07:46.277 INFO:tasks.workunit.client.1.smithi084.stdout:5/815: write dc/fbf [147769,21306] 0 2024-02-28T23:07:46.287 INFO:tasks.workunit.client.1.smithi084.stdout:6/770: dwrite d1/d13/d47/d91/dba/f95 [4194304,4194304] 0 2024-02-28T23:07:46.290 INFO:tasks.workunit.client.0.smithi057.stdout:5/655: truncate d4/d21/d24/d30/f79 3900209 0 2024-02-28T23:07:46.290 INFO:tasks.workunit.client.1.smithi084.stdout:6/771: stat d1/d13/d50/d74/d81 0 2024-02-28T23:07:46.293 INFO:tasks.workunit.client.0.smithi057.stdout:5/656: symlink d4/d9/d16/d2c/ld0 0 2024-02-28T23:07:46.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/791: truncate d0/d9/d1b/f39 2742576 0 2024-02-28T23:07:46.297 INFO:tasks.workunit.client.0.smithi057.stdout:5/657: link d4/d21/d24/d30/lbe d4/d7/d8/ld1 0 2024-02-28T23:07:46.298 INFO:tasks.workunit.client.1.smithi084.stdout:0/802: getdents d1/d8/d69/d6b 0 2024-02-28T23:07:46.300 INFO:tasks.workunit.client.1.smithi084.stdout:3/847: dread d3/d50/f8c [0,4194304] 0 2024-02-28T23:07:46.302 INFO:tasks.workunit.client.1.smithi084.stdout:5/816: link dc/de/df/d16/ffa dc/de/df/d31/f135 0 2024-02-28T23:07:46.304 INFO:tasks.workunit.client.1.smithi084.stdout:0/803: symlink d1/d8/da1/l104 0 2024-02-28T23:07:46.305 INFO:tasks.workunit.client.1.smithi084.stdout:5/817: fdatasync dc/f6b 0 2024-02-28T23:07:46.307 INFO:tasks.workunit.client.1.smithi084.stdout:6/772: creat d1/d9/f104 x:0 0 0 2024-02-28T23:07:46.310 INFO:tasks.workunit.client.1.smithi084.stdout:5/818: rmdir dc/de/df/d16/d34 39 2024-02-28T23:07:46.313 INFO:tasks.workunit.client.1.smithi084.stdout:3/848: mknod d3/d50/d117/d3d/d43/d61/dce/d109/c13d 0 2024-02-28T23:07:46.316 INFO:tasks.workunit.client.1.smithi084.stdout:8/792: link d0/l5 d0/d9/d30/d33/d71/d84/dd9/de3/l107 0 2024-02-28T23:07:46.319 INFO:tasks.workunit.client.1.smithi084.stdout:1/858: truncate d12/d1b/d2e/d52/d5b/f6f 2222892 0 2024-02-28T23:07:46.319 INFO:tasks.workunit.client.0.smithi057.stdout:5/658: dread d4/d9/d14/f28 [0,4194304] 0 2024-02-28T23:07:46.320 INFO:tasks.workunit.client.1.smithi084.stdout:4/765: write d1/d18/d20/f11 [1587642,36788] 0 2024-02-28T23:07:46.322 INFO:tasks.workunit.client.1.smithi084.stdout:0/804: creat d1/d6/db/d35/f105 x:0 0 0 2024-02-28T23:07:46.327 INFO:tasks.workunit.client.1.smithi084.stdout:5/819: truncate dc/de/df/d31/d8e/da9/dab/fb4 208288 0 2024-02-28T23:07:46.329 INFO:tasks.workunit.client.1.smithi084.stdout:7/898: dwrite db/d10/d18/db7/de9/dfa/f5b [0,4194304] 0 2024-02-28T23:07:46.334 INFO:tasks.workunit.client.0.smithi057.stdout:5/659: dwrite d4/d9/d14/f54 [0,4194304] 0 2024-02-28T23:07:46.335 INFO:tasks.workunit.client.1.smithi084.stdout:2/723: dread d5/d7/d9/d7c/d5b/d9f/fa0 [0,4194304] 0 2024-02-28T23:07:46.336 INFO:tasks.workunit.client.0.smithi057.stdout:5/660: readlink d4/d9/d16/d2c/ld0 0 2024-02-28T23:07:46.337 INFO:tasks.workunit.client.1.smithi084.stdout:3/849: symlink d3/d27/d7d/db0/d14/l13e 0 2024-02-28T23:07:46.341 INFO:tasks.workunit.client.1.smithi084.stdout:9/844: dwrite d4/d6/d15/d12/d41/d55/fbc [0,4194304] 0 2024-02-28T23:07:46.343 INFO:tasks.workunit.client.1.smithi084.stdout:8/793: readlink d0/d9/d30/d33/d71/d84/dc0/ldd 0 2024-02-28T23:07:46.344 INFO:tasks.workunit.client.1.smithi084.stdout:1/859: mkdir d12/d1b/d2e/d30/d77/da0/d117 0 2024-02-28T23:07:46.344 INFO:tasks.workunit.client.1.smithi084.stdout:0/805: write d1/d6/db/d76/f9c [3912069,55561] 0 2024-02-28T23:07:46.346 INFO:tasks.workunit.client.1.smithi084.stdout:3/850: symlink d3/d27/d7d/db0/d14/d31/l13f 0 2024-02-28T23:07:46.353 INFO:tasks.workunit.client.1.smithi084.stdout:4/766: rename d1/d18/d20/d24/d2f/d3e/d52/d45/fd5 to d1/d4b/ffc 0 2024-02-28T23:07:46.353 INFO:tasks.workunit.client.1.smithi084.stdout:0/806: chown d1/d8 2424927 1 2024-02-28T23:07:46.353 INFO:tasks.workunit.client.1.smithi084.stdout:6/773: dwrite d1/d9/d1d/d1e/fc8 [4194304,4194304] 0 2024-02-28T23:07:46.354 INFO:tasks.workunit.client.1.smithi084.stdout:9/845: stat d4/d6/d15/d3c/l86 0 2024-02-28T23:07:46.356 INFO:tasks.workunit.client.1.smithi084.stdout:7/899: mkdir db/d10/d17/d9c/d9f/db1/d115 0 2024-02-28T23:07:46.358 INFO:tasks.workunit.client.1.smithi084.stdout:1/860: fsync d12/d84/d9a/fce 0 2024-02-28T23:07:46.359 INFO:tasks.workunit.client.1.smithi084.stdout:2/724: creat d5/d7/d9/d94/dbd/fdb x:0 0 0 2024-02-28T23:07:46.360 INFO:tasks.workunit.client.1.smithi084.stdout:8/794: mknod d0/d24/c108 0 2024-02-28T23:07:46.363 INFO:tasks.workunit.client.0.smithi057.stdout:5/661: dwrite d4/d9/d16/d4e/f87 [0,4194304] 0 2024-02-28T23:07:46.364 INFO:tasks.workunit.client.1.smithi084.stdout:8/795: fsync d0/d24/f5b 0 2024-02-28T23:07:46.366 INFO:tasks.workunit.client.0.smithi057.stdout:5/662: rmdir d4/d9/d85 39 2024-02-28T23:07:46.374 INFO:tasks.workunit.client.0.smithi057.stdout:5/663: getdents d4/d9/d14/d3b/d61/d71 0 2024-02-28T23:07:46.376 INFO:tasks.workunit.client.0.smithi057.stdout:5/664: creat d4/d21/fd2 x:0 0 0 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:5/665: creat d4/d9/d14/d63/d88/fd3 x:0 0 0 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:3/559: sync 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:8/697: sync 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:4/650: sync 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:6/639: sync 2024-02-28T23:07:46.379 INFO:tasks.workunit.client.0.smithi057.stdout:2/734: sync 2024-02-28T23:07:46.380 INFO:tasks.workunit.client.0.smithi057.stdout:9/642: sync 2024-02-28T23:07:46.382 INFO:tasks.workunit.client.0.smithi057.stdout:9/643: readlink dd/de/d4a/d9f/la3 0 2024-02-28T23:07:46.382 INFO:tasks.workunit.client.0.smithi057.stdout:3/560: dread - d3/d4f/d4c/f89 zero size 2024-02-28T23:07:46.383 INFO:tasks.workunit.client.0.smithi057.stdout:9/644: stat dd/de/c9c 0 2024-02-28T23:07:46.384 INFO:tasks.workunit.client.0.smithi057.stdout:6/640: unlink d0/d17/fbc 0 2024-02-28T23:07:46.385 INFO:tasks.workunit.client.0.smithi057.stdout:5/666: read - d4/d21/d24/da2/dbb/fa4 zero size 2024-02-28T23:07:46.386 INFO:tasks.workunit.client.0.smithi057.stdout:8/698: truncate d4/d13/d18/da7/ff3 759020 0 2024-02-28T23:07:46.386 INFO:tasks.workunit.client.0.smithi057.stdout:3/561: write d3/d20/f23 [1494440,34473] 0 2024-02-28T23:07:46.388 INFO:tasks.workunit.client.1.smithi084.stdout:9/846: rename d4/d6/d15/d75/l88 to d4/d69/l114 0 2024-02-28T23:07:46.389 INFO:tasks.workunit.client.0.smithi057.stdout:2/735: creat d8/d11/d21/d28/d49/d85/d9b/ddc/fe4 x:0 0 0 2024-02-28T23:07:46.389 INFO:tasks.workunit.client.0.smithi057.stdout:9/645: unlink dd/l67 0 2024-02-28T23:07:46.390 INFO:tasks.workunit.client.1.smithi084.stdout:6/774: dwrite d1/d9/d1d/d1e/d33/f89 [12582912,4194304] 0 2024-02-28T23:07:46.393 INFO:tasks.workunit.client.0.smithi057.stdout:2/736: write d8/d11/d21/f36 [407872,46981] 0 2024-02-28T23:07:46.393 INFO:tasks.workunit.client.0.smithi057.stdout:9/646: chown dd/de/d24/d63/d77/dcf/d40/d44/fa4 0 1 2024-02-28T23:07:46.394 INFO:tasks.workunit.client.0.smithi057.stdout:8/699: mkdir d4/df7 0 2024-02-28T23:07:46.395 INFO:tasks.workunit.client.1.smithi084.stdout:2/725: dread d5/d7/d75/fac [0,4194304] 0 2024-02-28T23:07:46.396 INFO:tasks.workunit.client.1.smithi084.stdout:9/847: read - d4/d32/f5f zero size 2024-02-28T23:07:46.396 INFO:tasks.workunit.client.1.smithi084.stdout:2/726: chown d5/d64 88 1 2024-02-28T23:07:46.397 INFO:tasks.workunit.client.0.smithi057.stdout:6/641: creat d0/d1b/d3e/fdb x:0 0 0 2024-02-28T23:07:46.400 INFO:tasks.workunit.client.0.smithi057.stdout:8/700: dread - d4/de9/d56/d6a/d42/fcd zero size 2024-02-28T23:07:46.402 INFO:tasks.workunit.client.0.smithi057.stdout:2/737: unlink d8/d11/d39/d7b/l98 0 2024-02-28T23:07:46.403 INFO:tasks.workunit.client.0.smithi057.stdout:5/667: getdents d4/d32 0 2024-02-28T23:07:46.403 INFO:tasks.workunit.client.0.smithi057.stdout:3/562: link d3/d4/l6f d3/d20/lb0 0 2024-02-28T23:07:46.403 INFO:tasks.workunit.client.1.smithi084.stdout:6/775: write d1/d13/d50/d74/fb7 [294475,54336] 0 2024-02-28T23:07:46.404 INFO:tasks.workunit.client.1.smithi084.stdout:7/900: dwrite db/d10/d17/d64/d77/fbf [0,4194304] 0 2024-02-28T23:07:46.406 INFO:tasks.workunit.client.1.smithi084.stdout:9/848: write d4/d6/d15/d75/d2d/dbd/f108 [271322,116368] 0 2024-02-28T23:07:46.406 INFO:tasks.workunit.client.0.smithi057.stdout:5/668: readlink d4/d21/d24/d30/l8c 0 2024-02-28T23:07:46.409 INFO:tasks.workunit.client.1.smithi084.stdout:6/776: dread - d1/d13/d47/d91/dba/fe3 zero size 2024-02-28T23:07:46.412 INFO:tasks.workunit.client.0.smithi057.stdout:3/563: unlink d3/d4f/f83 0 2024-02-28T23:07:46.414 INFO:tasks.workunit.client.0.smithi057.stdout:5/669: chown d4/d9/d14/d63/ca5 15361 1 2024-02-28T23:07:46.415 INFO:tasks.workunit.client.1.smithi084.stdout:2/727: getdents d5/d7/d12/d24/d2a/d53 0 2024-02-28T23:07:46.416 INFO:tasks.workunit.client.0.smithi057.stdout:2/738: fsync d8/fb 0 2024-02-28T23:07:46.418 INFO:tasks.workunit.client.1.smithi084.stdout:0/807: dread d1/d6/db/d37/f96 [0,4194304] 0 2024-02-28T23:07:46.419 INFO:tasks.workunit.client.1.smithi084.stdout:9/849: rename d4/d32/d3f/f96 to d4/d32/d3f/d89/f115 0 2024-02-28T23:07:46.423 INFO:tasks.workunit.client.0.smithi057.stdout:6/642: rename d0/d1b/d3e/d40/ld2 to d0/d4/d25/d60/d6a/ldc 0 2024-02-28T23:07:46.424 INFO:tasks.workunit.client.1.smithi084.stdout:0/808: readlink d1/d6/db/d35/d4a/d56/l64 0 2024-02-28T23:07:46.424 INFO:tasks.workunit.client.1.smithi084.stdout:5/820: dwrite dc/de/df/da6/f11f [0,4194304] 0 2024-02-28T23:07:46.425 INFO:tasks.workunit.client.1.smithi084.stdout:6/777: mknod d1/d9/d1d/db6/c105 0 2024-02-28T23:07:46.426 INFO:tasks.workunit.client.0.smithi057.stdout:9/647: dwrite dd/de/d14/f21 [4194304,4194304] 0 2024-02-28T23:07:46.427 INFO:tasks.workunit.client.0.smithi057.stdout:6/643: write d0/d1b/d3e/d3d/d42/d4f/fac [1410259,94089] 0 2024-02-28T23:07:46.427 INFO:tasks.workunit.client.1.smithi084.stdout:3/851: write d3/d27/d7d/db0/df/f19 [2022785,18482] 0 2024-02-28T23:07:46.429 INFO:tasks.workunit.client.0.smithi057.stdout:5/670: mknod d4/d7/d58/d78/dc1/cd4 0 2024-02-28T23:07:46.431 INFO:tasks.workunit.client.0.smithi057.stdout:2/739: creat d8/d11/d21/d28/d49/d85/fe5 x:0 0 0 2024-02-28T23:07:46.434 INFO:tasks.workunit.client.0.smithi057.stdout:6/644: truncate d0/d1b/d3e/d40/fc6 994032 0 2024-02-28T23:07:46.434 INFO:tasks.workunit.client.1.smithi084.stdout:7/901: mknod db/d10/d17/d9c/d9f/db1/c116 0 2024-02-28T23:07:46.435 INFO:tasks.workunit.client.0.smithi057.stdout:2/740: mkdir d8/d11/d21/de6 0 2024-02-28T23:07:46.437 INFO:tasks.workunit.client.0.smithi057.stdout:6/645: chown d0/d1b/d3e/d3d/f55 31945 1 2024-02-28T23:07:46.438 INFO:tasks.workunit.client.1.smithi084.stdout:1/861: write d12/d1b/d2e/d30/f58 [3837660,68614] 0 2024-02-28T23:07:46.438 INFO:tasks.workunit.client.1.smithi084.stdout:4/767: dwrite d1/d18/d20/d24/d2f/d3e/d52/d57/fa0 [0,4194304] 0 2024-02-28T23:07:46.440 INFO:tasks.workunit.client.0.smithi057.stdout:1/594: sync 2024-02-28T23:07:46.440 INFO:tasks.workunit.client.1.smithi084.stdout:4/768: dread - d1/d18/ffa zero size 2024-02-28T23:07:46.442 INFO:tasks.workunit.client.1.smithi084.stdout:2/728: truncate d5/d1f/d83/f97 245566 0 2024-02-28T23:07:46.442 INFO:tasks.workunit.client.1.smithi084.stdout:4/769: fsync d1/dd9/fef 0 2024-02-28T23:07:46.444 INFO:tasks.workunit.client.1.smithi084.stdout:2/729: readlink d5/d7/d12/l39 0 2024-02-28T23:07:46.445 INFO:tasks.workunit.client.1.smithi084.stdout:5/821: unlink l4 0 2024-02-28T23:07:46.448 INFO:tasks.workunit.client.1.smithi084.stdout:6/778: creat d1/d13/d47/f106 x:0 0 0 2024-02-28T23:07:46.451 INFO:tasks.workunit.client.0.smithi057.stdout:4/651: dread d1a/d2f/db2/d49/d4d/f8d [0,4194304] 0 2024-02-28T23:07:46.454 INFO:tasks.workunit.client.1.smithi084.stdout:0/809: read d1/d8/d69/d6b/f77 [3878414,73244] 0 2024-02-28T23:07:46.454 INFO:tasks.workunit.client.1.smithi084.stdout:1/862: creat d12/d1b/d91/d9b/f118 x:0 0 0 2024-02-28T23:07:46.458 INFO:tasks.workunit.client.1.smithi084.stdout:9/850: fdatasync d4/d6/d15/f2c 0 2024-02-28T23:07:46.458 INFO:tasks.workunit.client.0.smithi057.stdout:6/646: getdents d0/d1b/d3e/d3d/d42 0 2024-02-28T23:07:46.460 INFO:tasks.workunit.client.1.smithi084.stdout:8/796: dread d0/d11/f3c [0,4194304] 0 2024-02-28T23:07:46.461 INFO:tasks.workunit.client.1.smithi084.stdout:5/822: unlink dc/de/df/d16/f9a 0 2024-02-28T23:07:46.463 INFO:tasks.workunit.client.1.smithi084.stdout:1/863: mkdir d12/d1b/d2e/d30/d50/d119 0 2024-02-28T23:07:46.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/652: fsync d1a/d2f/db2/d49/d4d/f78 0 2024-02-28T23:07:46.467 INFO:tasks.workunit.client.0.smithi057.stdout:6/647: rmdir d0/d4/d25/d60 39 2024-02-28T23:07:46.467 INFO:tasks.workunit.client.1.smithi084.stdout:8/797: stat d0/d9/d1b/d38/d43/d51/d54/cb1 0 2024-02-28T23:07:46.469 INFO:tasks.workunit.client.1.smithi084.stdout:4/770: symlink d1/d18/d20/d65/lfd 0 2024-02-28T23:07:46.472 INFO:tasks.workunit.client.1.smithi084.stdout:1/864: write d12/d1b/d63/d7d/f85 [1737515,71819] 0 2024-02-28T23:07:46.474 INFO:tasks.workunit.client.1.smithi084.stdout:7/902: mkdir db/d5e/d87/dc1/de6/d117 0 2024-02-28T23:07:46.475 INFO:tasks.workunit.client.1.smithi084.stdout:9/851: creat d4/d6/d15/d75/d2d/dbd/dde/f116 x:0 0 0 2024-02-28T23:07:46.475 INFO:tasks.workunit.client.1.smithi084.stdout:2/730: dwrite d5/d7/d9/d94/dbd/fdb [0,4194304] 0 2024-02-28T23:07:46.477 INFO:tasks.workunit.client.0.smithi057.stdout:5/671: dwrite d4/d21/d24/d30/d62/faa [0,4194304] 0 2024-02-28T23:07:46.477 INFO:tasks.workunit.client.0.smithi057.stdout:4/653: rmdir d1a/d48/d9c/d68 39 2024-02-28T23:07:46.480 INFO:tasks.workunit.client.1.smithi084.stdout:4/771: chown d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5 953727 1 2024-02-28T23:07:46.480 INFO:tasks.workunit.client.1.smithi084.stdout:1/865: chown d12/d1b/f48 13987643 1 2024-02-28T23:07:46.481 INFO:tasks.workunit.client.0.smithi057.stdout:2/741: dwrite d8/d11/d39/d7b/d8b/fdf [0,4194304] 0 2024-02-28T23:07:46.484 INFO:tasks.workunit.client.0.smithi057.stdout:9/648: dwrite dd/de/d14/f17 [4194304,4194304] 0 2024-02-28T23:07:46.485 INFO:tasks.workunit.client.0.smithi057.stdout:3/564: dwrite d3/d4/d1d/f2e [0,4194304] 0 2024-02-28T23:07:46.488 INFO:tasks.workunit.client.1.smithi084.stdout:0/810: dwrite d1/d6/d2d/fc8 [0,4194304] 0 2024-02-28T23:07:46.488 INFO:tasks.workunit.client.1.smithi084.stdout:8/798: creat d0/d9/d30/d33/d71/d84/dce/f109 x:0 0 0 2024-02-28T23:07:46.488 INFO:tasks.workunit.client.1.smithi084.stdout:5/823: chown dc/de/df/d16/d34/lcd 2873055 1 2024-02-28T23:07:46.489 INFO:tasks.workunit.client.1.smithi084.stdout:9/852: rmdir d4/d6/d53 39 2024-02-28T23:07:46.490 INFO:tasks.workunit.client.1.smithi084.stdout:5/824: stat dc/ddb/l108 0 2024-02-28T23:07:46.491 INFO:tasks.workunit.client.0.smithi057.stdout:8/701: dwrite d4/de9/f2e [0,4194304] 0 2024-02-28T23:07:46.493 INFO:tasks.workunit.client.1.smithi084.stdout:6/779: dwrite d1/d9/d1d/db6/ffd [0,4194304] 0 2024-02-28T23:07:46.494 INFO:tasks.workunit.client.0.smithi057.stdout:6/648: creat d0/d4/d25/fdd x:0 0 0 2024-02-28T23:07:46.496 INFO:tasks.workunit.client.1.smithi084.stdout:9/853: fsync d4/d6/f72 0 2024-02-28T23:07:46.497 INFO:tasks.workunit.client.1.smithi084.stdout:2/731: creat d5/d7/d9/d7c/fdc x:0 0 0 2024-02-28T23:07:46.499 INFO:tasks.workunit.client.0.smithi057.stdout:4/654: fsync d1a/d2f/db2/d49/f86 0 2024-02-28T23:07:46.502 INFO:tasks.workunit.client.0.smithi057.stdout:3/565: symlink d3/d4f/lb1 0 2024-02-28T23:07:46.503 INFO:tasks.workunit.client.1.smithi084.stdout:7/903: creat db/d5e/d8a/d10a/f118 x:0 0 0 2024-02-28T23:07:46.508 INFO:tasks.workunit.client.0.smithi057.stdout:7/652: sync 2024-02-28T23:07:46.509 INFO:tasks.workunit.client.0.smithi057.stdout:0/770: sync 2024-02-28T23:07:46.509 INFO:tasks.workunit.client.0.smithi057.stdout:9/649: mkdir dd/d46/dc4/dd3 0 2024-02-28T23:07:46.511 INFO:tasks.workunit.client.0.smithi057.stdout:4/655: mknod d1a/d1e/d81/ce3 0 2024-02-28T23:07:46.513 INFO:tasks.workunit.client.1.smithi084.stdout:6/780: symlink d1/d88/dbf/de2/l107 0 2024-02-28T23:07:46.514 INFO:tasks.workunit.client.1.smithi084.stdout:7/904: mknod db/d5e/d8a/c119 0 2024-02-28T23:07:46.516 INFO:tasks.workunit.client.0.smithi057.stdout:0/771: creat d17/d20/d26/d48/d55/d88/f111 x:0 0 0 2024-02-28T23:07:46.517 INFO:tasks.workunit.client.1.smithi084.stdout:9/854: dwrite d4/f3b [0,4194304] 0 2024-02-28T23:07:46.517 INFO:tasks.workunit.client.1.smithi084.stdout:3/852: truncate d3/d27/d7d/db0/df/f12 3966528 0 2024-02-28T23:07:46.519 INFO:tasks.workunit.client.1.smithi084.stdout:4/772: creat d1/d18/d20/d24/d2f/d3e/d52/d45/ffe x:0 0 0 2024-02-28T23:07:46.521 INFO:tasks.workunit.client.0.smithi057.stdout:7/653: creat d0/d5/d36/fd8 x:0 0 0 2024-02-28T23:07:46.521 INFO:tasks.workunit.client.0.smithi057.stdout:9/650: fsync dd/de/d14/d48/d76/dbf/fd2 0 2024-02-28T23:07:46.521 INFO:tasks.workunit.client.0.smithi057.stdout:0/772: symlink d17/d20/d4a/d78/l112 0 2024-02-28T23:07:46.524 INFO:tasks.workunit.client.0.smithi057.stdout:7/654: chown d0/d5/d25/f28 10435313 1 2024-02-28T23:07:46.525 INFO:tasks.workunit.client.0.smithi057.stdout:9/651: fdatasync dd/de/d14/d48/d4b/f54 0 2024-02-28T23:07:46.525 INFO:tasks.workunit.client.0.smithi057.stdout:7/655: chown d0/d3f 1427612756 1 2024-02-28T23:07:46.526 INFO:tasks.workunit.client.1.smithi084.stdout:6/781: unlink d1/d9/d1d/f61 0 2024-02-28T23:07:46.528 INFO:tasks.workunit.client.1.smithi084.stdout:6/782: fdatasync d1/d13/d50/f8d 0 2024-02-28T23:07:46.529 INFO:tasks.workunit.client.0.smithi057.stdout:1/595: truncate d2/d7/df/db/d12/d19/d65/f96 3428187 0 2024-02-28T23:07:46.529 INFO:tasks.workunit.client.0.smithi057.stdout:8/702: read d4/de9/d56/fbc [235428,92287] 0 2024-02-28T23:07:46.531 INFO:tasks.workunit.client.1.smithi084.stdout:0/811: getdents d1/d6/db/d35/d82 0 2024-02-28T23:07:46.532 INFO:tasks.workunit.client.0.smithi057.stdout:0/773: mkdir d17/d19/d4d/d7b/d113 0 2024-02-28T23:07:46.533 INFO:tasks.workunit.client.1.smithi084.stdout:9/855: symlink d4/l117 0 2024-02-28T23:07:46.534 INFO:tasks.workunit.client.0.smithi057.stdout:2/742: getdents d8/d11/d21/d70/d91 0 2024-02-28T23:07:46.536 INFO:tasks.workunit.client.1.smithi084.stdout:3/853: mkdir d3/d27/d7d/db0/d1a/d45/d5e/d140 0 2024-02-28T23:07:46.537 INFO:tasks.workunit.client.1.smithi084.stdout:6/783: symlink d1/d13/d47/l108 0 2024-02-28T23:07:46.538 INFO:tasks.workunit.client.1.smithi084.stdout:0/812: write d1/d6/d59/ff3 [1005416,126328] 0 2024-02-28T23:07:46.538 INFO:tasks.workunit.client.0.smithi057.stdout:7/656: creat d0/d5/d36/d4e/d3b/d75/dbf/fd9 x:0 0 0 2024-02-28T23:07:46.542 INFO:tasks.workunit.client.1.smithi084.stdout:6/784: chown d1/d13/d2e/d41/d2c/dfb/la5 2974995 1 2024-02-28T23:07:46.544 INFO:tasks.workunit.client.0.smithi057.stdout:1/596: mkdir d2/dd4/dd5 0 2024-02-28T23:07:46.544 INFO:tasks.workunit.client.0.smithi057.stdout:2/743: write d8/d11/d21/d70/fbd [472254,62871] 0 2024-02-28T23:07:46.544 INFO:tasks.workunit.client.0.smithi057.stdout:3/566: link d3/d4/d1d/d47/l52 d3/d4/lb2 0 2024-02-28T23:07:46.544 INFO:tasks.workunit.client.1.smithi084.stdout:0/813: dread - d1/d8/d69/d6b/d85/da5/f102 zero size 2024-02-28T23:07:46.544 INFO:tasks.workunit.client.1.smithi084.stdout:3/854: read - d3/d27/d7d/db0/d1a/d45/d5e/d7a/f9b zero size 2024-02-28T23:07:46.545 INFO:tasks.workunit.client.1.smithi084.stdout:9/856: mkdir d4/d32/d3f/d89/db8/d118 0 2024-02-28T23:07:46.546 INFO:tasks.workunit.client.0.smithi057.stdout:9/652: getdents dd/de/d4a 0 2024-02-28T23:07:46.547 INFO:tasks.workunit.client.1.smithi084.stdout:4/773: dread d1/f6e [0,4194304] 0 2024-02-28T23:07:46.548 INFO:tasks.workunit.client.1.smithi084.stdout:3/855: write d3/d27/d7d/db0/d14/d31/f139 [804625,47792] 0 2024-02-28T23:07:46.548 INFO:tasks.workunit.client.1.smithi084.stdout:0/814: symlink d1/d6/db/d35/d44/da9/l106 0 2024-02-28T23:07:46.550 INFO:tasks.workunit.client.0.smithi057.stdout:7/657: chown d0/d5/d36/d4e/d15/c86 683 1 2024-02-28T23:07:46.550 INFO:tasks.workunit.client.1.smithi084.stdout:3/856: chown d3/d27/d7d 2682 1 2024-02-28T23:07:46.550 INFO:tasks.workunit.client.1.smithi084.stdout:0/815: chown d1/d6/c8b 0 1 2024-02-28T23:07:46.551 INFO:tasks.workunit.client.0.smithi057.stdout:2/744: chown d8/d11/d21/d70/fbd 7155875 1 2024-02-28T23:07:46.552 INFO:tasks.workunit.client.0.smithi057.stdout:6/649: dwrite d0/d1b/d3e/d40/fc6 [0,4194304] 0 2024-02-28T23:07:46.555 INFO:tasks.workunit.client.0.smithi057.stdout:3/567: read d3/d4/d1d/f49 [3293195,116252] 0 2024-02-28T23:07:46.559 INFO:tasks.workunit.client.0.smithi057.stdout:6/650: chown d0/d4/d11/dcb 50 1 2024-02-28T23:07:46.562 INFO:tasks.workunit.client.1.smithi084.stdout:6/785: rename d1/d9/d1d/d1e/lf4 to d1/d13/d50/d74/d9e/l109 0 2024-02-28T23:07:46.563 INFO:tasks.workunit.client.1.smithi084.stdout:7/905: dread f9 [0,4194304] 0 2024-02-28T23:07:46.564 INFO:tasks.workunit.client.1.smithi084.stdout:4/774: dread d1/fbd [0,4194304] 0 2024-02-28T23:07:46.565 INFO:tasks.workunit.client.0.smithi057.stdout:7/658: link d0/d5/d36/d4e/d15/f46 d0/d5/fda 0 2024-02-28T23:07:46.565 INFO:tasks.workunit.client.0.smithi057.stdout:6/651: chown d0/d17/d94/da0 566049060 1 2024-02-28T23:07:46.565 INFO:tasks.workunit.client.0.smithi057.stdout:3/568: truncate d3/d4/f12 8552294 0 2024-02-28T23:07:46.566 INFO:tasks.workunit.client.1.smithi084.stdout:7/906: mkdir db/d5e/d80/d11a 0 2024-02-28T23:07:46.568 INFO:tasks.workunit.client.0.smithi057.stdout:1/597: getdents d2/d71/d2d 0 2024-02-28T23:07:46.568 INFO:tasks.workunit.client.0.smithi057.stdout:9/653: truncate dd/f2f 22656 0 2024-02-28T23:07:46.568 INFO:tasks.workunit.client.1.smithi084.stdout:6/786: creat d1/d13/d2e/d41/d4d/d6c/f10a x:0 0 0 2024-02-28T23:07:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:46 smithi057 ceph-mon[29096]: pgmap v29: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 116 MiB/s rd, 340 MiB/s wr, 663 op/s 2024-02-28T23:07:46.571 INFO:tasks.workunit.client.1.smithi084.stdout:3/857: link d3/d27/d7d/db0/d1a/d45/dde/le8 d3/d50/d117/d3d/d43/d61/dce/l141 0 2024-02-28T23:07:46.572 INFO:tasks.workunit.client.0.smithi057.stdout:5/672: dwrite d4/d9/fa6 [0,4194304] 0 2024-02-28T23:07:46.573 INFO:tasks.workunit.client.1.smithi084.stdout:4/775: unlink d1/d18/d20/d14/d30/d47/lb0 0 2024-02-28T23:07:46.574 INFO:tasks.workunit.client.1.smithi084.stdout:3/858: dread - d3/d27/d7d/db0/f10a zero size 2024-02-28T23:07:46.575 INFO:tasks.workunit.client.1.smithi084.stdout:4/776: fsync d1/d18/ffa 0 2024-02-28T23:07:46.575 INFO:tasks.workunit.client.1.smithi084.stdout:3/859: fsync d3/d50/fbb 0 2024-02-28T23:07:46.577 INFO:tasks.workunit.client.0.smithi057.stdout:5/673: chown d4/d7/l4d 53897 1 2024-02-28T23:07:46.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:46 smithi084 ceph-mon[34826]: pgmap v29: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 116 MiB/s rd, 340 MiB/s wr, 663 op/s 2024-02-28T23:07:46.580 INFO:tasks.workunit.client.0.smithi057.stdout:8/703: dwrite d4/d13/d18/da7/ff3 [0,4194304] 0 2024-02-28T23:07:46.580 INFO:tasks.workunit.client.0.smithi057.stdout:2/745: write d8/d11/d39/d83/dde/d50/fb1 [496854,107421] 0 2024-02-28T23:07:46.583 INFO:tasks.workunit.client.1.smithi084.stdout:7/907: dwrite db/d10/d17/d64/d77/fbf [0,4194304] 0 2024-02-28T23:07:46.584 INFO:tasks.workunit.client.0.smithi057.stdout:7/659: rename d0/fb1 to d0/d5/d25/d80/d9f/fdb 0 2024-02-28T23:07:46.584 INFO:tasks.workunit.client.0.smithi057.stdout:6/652: rmdir d0/d4/d11/dcb 0 2024-02-28T23:07:46.585 INFO:tasks.workunit.client.1.smithi084.stdout:7/908: write db/d10/d17/d64/fe8 [566975,129282] 0 2024-02-28T23:07:46.586 INFO:tasks.workunit.client.1.smithi084.stdout:4/777: dread d1/d18/d20/d65/d8e/d55/fcd [0,4194304] 0 2024-02-28T23:07:46.586 INFO:tasks.workunit.client.0.smithi057.stdout:1/598: mkdir d2/d7/df/db/d12/dd6 0 2024-02-28T23:07:46.586 INFO:tasks.workunit.client.0.smithi057.stdout:8/704: truncate d4/de9/d59/f5f 23524 0 2024-02-28T23:07:46.586 INFO:tasks.workunit.client.0.smithi057.stdout:7/660: readlink d0/l34 0 2024-02-28T23:07:46.587 INFO:tasks.workunit.client.1.smithi084.stdout:7/909: chown db/d10/d17/d3b/lb8 407 1 2024-02-28T23:07:46.590 INFO:tasks.workunit.client.0.smithi057.stdout:5/674: unlink d4/d7/l4d 0 2024-02-28T23:07:46.591 INFO:tasks.workunit.client.1.smithi084.stdout:7/910: chown db/d10/d17/d9c/d9f/lef 7 1 2024-02-28T23:07:46.594 INFO:tasks.workunit.client.1.smithi084.stdout:4/778: link d1/d18/d20/d65/d8e/d55/c3a d1/d18/d20/d24/da8/cff 0 2024-02-28T23:07:46.597 INFO:tasks.workunit.client.0.smithi057.stdout:9/654: dwrite dd/de/d4a/d9f/fab [0,4194304] 0 2024-02-28T23:07:46.598 INFO:tasks.workunit.client.0.smithi057.stdout:2/746: symlink d8/d11/d39/le7 0 2024-02-28T23:07:46.598 INFO:tasks.workunit.client.1.smithi084.stdout:4/779: write d1/d18/d20/d24/d2f/d3e/d52/d45/f92 [741701,73255] 0 2024-02-28T23:07:46.598 INFO:tasks.workunit.client.0.smithi057.stdout:3/569: truncate d3/d20/d2d/f82 847225 0 2024-02-28T23:07:46.603 INFO:tasks.workunit.client.0.smithi057.stdout:8/705: mknod d4/d13/d18/cf8 0 2024-02-28T23:07:46.603 INFO:tasks.workunit.client.0.smithi057.stdout:9/655: dread - dd/de/d24/d72/f91 zero size 2024-02-28T23:07:46.604 INFO:tasks.workunit.client.0.smithi057.stdout:1/599: mknod d2/d7/df/db/d3a/d5a/cd7 0 2024-02-28T23:07:46.605 INFO:tasks.workunit.client.0.smithi057.stdout:3/570: stat d3/d4/d7/f10 0 2024-02-28T23:07:46.607 INFO:tasks.workunit.client.0.smithi057.stdout:2/747: readlink d8/d11/d21/d28/d41/la6 0 2024-02-28T23:07:46.608 INFO:tasks.workunit.client.0.smithi057.stdout:6/653: getdents d0/d1b/d3e/d3d/d42 0 2024-02-28T23:07:46.610 INFO:tasks.workunit.client.1.smithi084.stdout:7/911: dwrite db/d10/d17/f3d [4194304,4194304] 0 2024-02-28T23:07:46.611 INFO:tasks.workunit.client.0.smithi057.stdout:9/656: mknod dd/de/d24/d63/d77/dcf/d40/d44/d5c/cd4 0 2024-02-28T23:07:46.613 INFO:tasks.workunit.client.0.smithi057.stdout:8/706: rename d4/d13/d2c/f69 to d4/de9/d7c/ff9 0 2024-02-28T23:07:46.614 INFO:tasks.workunit.client.0.smithi057.stdout:9/657: write dd/de/d24/d63/d77/dcf/f6c [4837486,99388] 0 2024-02-28T23:07:46.615 INFO:tasks.workunit.client.0.smithi057.stdout:2/748: read d8/d11/d15/f95 [22265,26913] 0 2024-02-28T23:07:46.617 INFO:tasks.workunit.client.0.smithi057.stdout:3/571: mknod d3/d4/d1d/cb3 0 2024-02-28T23:07:46.619 INFO:tasks.workunit.client.0.smithi057.stdout:3/572: read - d3/d4/d7/f96 zero size 2024-02-28T23:07:46.620 INFO:tasks.workunit.client.0.smithi057.stdout:2/749: chown d8/d11/d39/d83 32630711 1 2024-02-28T23:07:46.620 INFO:tasks.workunit.client.0.smithi057.stdout:9/658: write dd/de/d14/d48/d76/dbf/fd2 [799811,54155] 0 2024-02-28T23:07:46.620 INFO:tasks.workunit.client.0.smithi057.stdout:8/707: creat d4/d13/d18/d62/d66/d74/ffa x:0 0 0 2024-02-28T23:07:46.623 INFO:tasks.workunit.client.0.smithi057.stdout:1/600: getdents d2/d7/df/db/d12/d3f 0 2024-02-28T23:07:46.627 INFO:tasks.workunit.client.0.smithi057.stdout:6/654: truncate d0/d4/d25/d60/fcf 539413 0 2024-02-28T23:07:46.628 INFO:tasks.workunit.client.0.smithi057.stdout:8/708: creat d4/de9/d56/d6a/d54/d6b/dca/ffb x:0 0 0 2024-02-28T23:07:46.629 INFO:tasks.workunit.client.0.smithi057.stdout:1/601: symlink d2/d7/d4f/d5c/d6b/ld8 0 2024-02-28T23:07:46.629 INFO:tasks.workunit.client.0.smithi057.stdout:2/750: truncate d8/d11/d21/d70/fbd 828004 0 2024-02-28T23:07:46.631 INFO:tasks.workunit.client.0.smithi057.stdout:8/709: chown d4/de9/d56/d88/lc2 137571732 1 2024-02-28T23:07:46.636 INFO:tasks.workunit.client.0.smithi057.stdout:1/602: unlink d2/d7/df/db/d12/d3f/c5f 0 2024-02-28T23:07:46.690 INFO:tasks.workunit.client.1.smithi084.stdout:5/825: write dc/de/df/d31/dd1/d5f/d9b/fb9 [638105,119881] 0 2024-02-28T23:07:46.692 INFO:tasks.workunit.client.1.smithi084.stdout:8/799: truncate d0/d9/d1b/f46 961668 0 2024-02-28T23:07:46.693 INFO:tasks.workunit.client.1.smithi084.stdout:8/800: chown d0/d44 13918 1 2024-02-28T23:07:46.694 INFO:tasks.workunit.client.1.smithi084.stdout:2/732: write d5/d1f/d83/f97 [1126416,18587] 0 2024-02-28T23:07:46.697 INFO:tasks.workunit.client.1.smithi084.stdout:2/733: readlink d5/d1f/d42/d8f/dbe/lc5 0 2024-02-28T23:07:46.698 INFO:tasks.workunit.client.1.smithi084.stdout:8/801: dread - d0/d9/d1b/d38/ff4 zero size 2024-02-28T23:07:46.700 INFO:tasks.workunit.client.1.smithi084.stdout:5/826: fsync dc/de/df/f71 0 2024-02-28T23:07:46.702 INFO:tasks.workunit.client.1.smithi084.stdout:8/802: write d0/d24/d4d/f58 [2783001,75752] 0 2024-02-28T23:07:46.703 INFO:tasks.workunit.client.0.smithi057.stdout:2/751: getdents d8/d11/d21/d28/d49/d85/d9b/dad 0 2024-02-28T23:07:46.705 INFO:tasks.workunit.client.1.smithi084.stdout:2/734: symlink d5/dd/ldd 0 2024-02-28T23:07:46.706 INFO:tasks.workunit.client.1.smithi084.stdout:5/827: creat dc/d51/f136 x:0 0 0 2024-02-28T23:07:46.707 INFO:tasks.workunit.client.1.smithi084.stdout:0/816: rmdir d1/d6/db/d35/d44 39 2024-02-28T23:07:46.708 INFO:tasks.workunit.client.0.smithi057.stdout:2/752: dread - d8/d11/d21/d28/d41/fd9 zero size 2024-02-28T23:07:46.709 INFO:tasks.workunit.client.1.smithi084.stdout:2/735: chown d5/d7/d9/f2c 5 1 2024-02-28T23:07:46.712 INFO:tasks.workunit.client.1.smithi084.stdout:2/736: write d5/d1f/d42/fa3 [544537,44780] 0 2024-02-28T23:07:46.714 INFO:tasks.workunit.client.1.smithi084.stdout:5/828: mknod dc/c137 0 2024-02-28T23:07:46.718 INFO:tasks.workunit.client.1.smithi084.stdout:3/860: truncate d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb 3677543 0 2024-02-28T23:07:46.720 INFO:tasks.workunit.client.0.smithi057.stdout:2/753: dread d8/d11/d15/f95 [0,4194304] 0 2024-02-28T23:07:46.721 INFO:tasks.workunit.client.1.smithi084.stdout:2/737: mknod d5/d7/d12/d24/d3a/d48/cde 0 2024-02-28T23:07:46.721 INFO:tasks.workunit.client.1.smithi084.stdout:4/780: write d1/d18/d20/d24/fa2 [753779,95788] 0 2024-02-28T23:07:46.723 INFO:tasks.workunit.client.1.smithi084.stdout:5/829: mknod dc/de/df/d31/dd1/c138 0 2024-02-28T23:07:46.725 INFO:tasks.workunit.client.0.smithi057.stdout:8/710: dwrite d4/de9/d56/d6a/d42/ff2 [0,4194304] 0 2024-02-28T23:07:46.727 INFO:tasks.workunit.client.1.smithi084.stdout:9/857: dread d4/d6/d15/d75/fa4 [0,4194304] 0 2024-02-28T23:07:46.730 INFO:tasks.workunit.client.1.smithi084.stdout:3/861: chown d3/d50/d5a/d73/db8/fc1 25 1 2024-02-28T23:07:46.730 INFO:tasks.workunit.client.0.smithi057.stdout:9/659: dwrite dd/de/d24/d63/d77/dcf/d40/d44/d92/fa8 [0,4194304] 0 2024-02-28T23:07:46.730 INFO:tasks.workunit.client.0.smithi057.stdout:5/675: dwrite d4/d7/f1e [0,4194304] 0 2024-02-28T23:07:46.733 INFO:tasks.workunit.client.1.smithi084.stdout:5/830: fsync dc/de/df/da6/fdd 0 2024-02-28T23:07:46.733 INFO:tasks.workunit.client.1.smithi084.stdout:4/781: dread - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea zero size 2024-02-28T23:07:46.735 INFO:tasks.workunit.client.0.smithi057.stdout:3/573: dwrite d3/d4/f30 [0,4194304] 0 2024-02-28T23:07:46.736 INFO:tasks.workunit.client.0.smithi057.stdout:9/660: dread - dd/de/d24/d72/f91 zero size 2024-02-28T23:07:46.740 INFO:tasks.workunit.client.1.smithi084.stdout:0/817: dwrite d1/d8/d18/d4c/d6c/d74/fad [0,4194304] 0 2024-02-28T23:07:46.742 INFO:tasks.workunit.client.0.smithi057.stdout:2/754: dread d8/d11/d21/d70/fbd [0,4194304] 0 2024-02-28T23:07:46.747 INFO:tasks.workunit.client.0.smithi057.stdout:9/661: chown dd/de/f43 138 1 2024-02-28T23:07:46.748 INFO:tasks.workunit.client.1.smithi084.stdout:4/782: write d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/f5e [7707969,32311] 0 2024-02-28T23:07:46.748 INFO:tasks.workunit.client.1.smithi084.stdout:9/858: mknod d4/d32/d3f/d89/c119 0 2024-02-28T23:07:46.748 INFO:tasks.workunit.client.1.smithi084.stdout:3/862: write d3/fbd [7735080,80527] 0 2024-02-28T23:07:46.751 INFO:tasks.workunit.client.0.smithi057.stdout:3/574: truncate d3/d4/f17 3614671 0 2024-02-28T23:07:46.752 INFO:tasks.workunit.client.1.smithi084.stdout:8/803: dread d0/d9/d30/db6/f80 [0,4194304] 0 2024-02-28T23:07:46.753 INFO:tasks.workunit.client.0.smithi057.stdout:1/603: dread d2/d7/df/db/d3a/f3b [0,4194304] 0 2024-02-28T23:07:46.757 INFO:tasks.workunit.client.1.smithi084.stdout:2/738: dread d5/d1f/d42/d74/fa1 [0,4194304] 0 2024-02-28T23:07:46.757 INFO:tasks.workunit.client.1.smithi084.stdout:0/818: truncate d1/d6/f53 658083 0 2024-02-28T23:07:46.758 INFO:tasks.workunit.client.1.smithi084.stdout:2/739: readlink d5/dd/ldd 0 2024-02-28T23:07:46.759 INFO:tasks.workunit.client.0.smithi057.stdout:8/711: dread d4/d13/d18/f31 [0,4194304] 0 2024-02-28T23:07:46.761 INFO:tasks.workunit.client.1.smithi084.stdout:8/804: symlink d0/d9/d30/l10a 0 2024-02-28T23:07:46.762 INFO:tasks.workunit.client.0.smithi057.stdout:3/575: mkdir d3/d4/d1d/d47/d62/db4 0 2024-02-28T23:07:46.767 INFO:tasks.workunit.client.0.smithi057.stdout:8/712: readlink d4/d13/d18/d62/d66/lf1 0 2024-02-28T23:07:46.767 INFO:tasks.workunit.client.0.smithi057.stdout:3/576: dread - d3/d4f/d4c/f89 zero size 2024-02-28T23:07:46.767 INFO:tasks.workunit.client.0.smithi057.stdout:7/661: dread d0/f10 [0,4194304] 0 2024-02-28T23:07:46.767 INFO:tasks.workunit.client.0.smithi057.stdout:5/676: dwrite d4/d21/d24/da2/dbb/d95/fb7 [0,4194304] 0 2024-02-28T23:07:46.768 INFO:tasks.workunit.client.0.smithi057.stdout:2/755: write d8/fe [2605215,10578] 0 2024-02-28T23:07:46.775 INFO:tasks.workunit.client.1.smithi084.stdout:5/831: dwrite dc/de/df/d31/dd1/d5f/d9b/dfe/f10f [4194304,4194304] 0 2024-02-28T23:07:46.777 INFO:tasks.workunit.client.1.smithi084.stdout:9/859: getdents d4/d6/d15/d12 0 2024-02-28T23:07:46.778 INFO:tasks.workunit.client.0.smithi057.stdout:7/662: dread d0/d5/d36/d4e/d3b/f9e [0,4194304] 0 2024-02-28T23:07:46.780 INFO:tasks.workunit.client.1.smithi084.stdout:8/805: creat d0/f10b x:0 0 0 2024-02-28T23:07:46.780 INFO:tasks.workunit.client.0.smithi057.stdout:3/577: rmdir d3/d4/d7/d9a 39 2024-02-28T23:07:46.781 INFO:tasks.workunit.client.0.smithi057.stdout:8/713: dread - d4/de9/d56/d6a/f95 zero size 2024-02-28T23:07:46.783 INFO:tasks.workunit.client.0.smithi057.stdout:7/663: creat d0/d3f/da1/fdc x:0 0 0 2024-02-28T23:07:46.783 INFO:tasks.workunit.client.0.smithi057.stdout:3/578: chown d3/la7 43825929 1 2024-02-28T23:07:46.785 INFO:tasks.workunit.client.1.smithi084.stdout:2/740: dwrite d5/fd4 [0,4194304] 0 2024-02-28T23:07:46.787 INFO:tasks.workunit.client.1.smithi084.stdout:2/741: chown d5/d1f/d42/l57 131976780 1 2024-02-28T23:07:46.790 INFO:tasks.workunit.client.0.smithi057.stdout:3/579: fdatasync d3/d8/d1b/f4a 0 2024-02-28T23:07:46.792 INFO:tasks.workunit.client.0.smithi057.stdout:2/756: dread d8/d11/d15/f72 [0,4194304] 0 2024-02-28T23:07:46.793 INFO:tasks.workunit.client.1.smithi084.stdout:5/832: mkdir dc/de/df/db0/d104/d139 0 2024-02-28T23:07:46.793 INFO:tasks.workunit.client.0.smithi057.stdout:9/662: dread dd/de/d24/fa6 [0,4194304] 0 2024-02-28T23:07:46.796 INFO:tasks.workunit.client.0.smithi057.stdout:7/664: symlink d0/d98/ldd 0 2024-02-28T23:07:46.798 INFO:tasks.workunit.client.1.smithi084.stdout:8/806: unlink d0/d9/f15 0 2024-02-28T23:07:46.802 INFO:tasks.workunit.client.1.smithi084.stdout:1/866: sync 2024-02-28T23:07:46.803 INFO:tasks.workunit.client.1.smithi084.stdout:3/863: dread d3/d50/d5a/f12c [0,4194304] 0 2024-02-28T23:07:46.804 INFO:tasks.workunit.client.0.smithi057.stdout:8/714: rename d4/de9/d56/d6a/d54/d61/d8c to d4/d13/d1a/d4a/db3/dfc 0 2024-02-28T23:07:46.811 INFO:tasks.workunit.client.0.smithi057.stdout:7/665: chown d0/d5/d36/d4e/d15/c24 96 1 2024-02-28T23:07:46.813 INFO:tasks.workunit.client.1.smithi084.stdout:8/807: mkdir d0/d9/d1b/d38/d43/dae/d10c 0 2024-02-28T23:07:46.814 INFO:tasks.workunit.client.1.smithi084.stdout:2/742: link d5/d7/d9/d7c/d5b/d9f/fa0 d5/d7/d9/d94/fdf 0 2024-02-28T23:07:46.816 INFO:tasks.workunit.client.1.smithi084.stdout:1/867: creat d12/d1b/d2e/d30/d50/da8/f11a x:0 0 0 2024-02-28T23:07:46.816 INFO:tasks.workunit.client.0.smithi057.stdout:8/715: mkdir d4/de9/d56/dfd 0 2024-02-28T23:07:46.816 INFO:tasks.workunit.client.0.smithi057.stdout:3/580: dwrite d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:46.816 INFO:tasks.workunit.client.1.smithi084.stdout:5/833: truncate dc/de/df/d16/d34/f4d 2591706 0 2024-02-28T23:07:46.821 INFO:tasks.workunit.client.0.smithi057.stdout:2/757: rename d8/d11/d39/d83/dde/f52 to d8/d11/d21/d70/fe8 0 2024-02-28T23:07:46.822 INFO:tasks.workunit.client.0.smithi057.stdout:7/666: mknod d0/d5/d36/d4e/d15/cde 0 2024-02-28T23:07:46.827 INFO:tasks.workunit.client.0.smithi057.stdout:3/581: symlink d3/d4/d7/d9a/d9b/lb5 0 2024-02-28T23:07:46.828 INFO:tasks.workunit.client.1.smithi084.stdout:2/743: symlink d5/d7/d9/d7c/d5b/d9f/le0 0 2024-02-28T23:07:46.830 INFO:tasks.workunit.client.0.smithi057.stdout:2/758: mkdir d8/d11/db3/de9 0 2024-02-28T23:07:46.831 INFO:tasks.workunit.client.0.smithi057.stdout:8/716: rmdir d4/d13/d1a/d4a/db3/de8 0 2024-02-28T23:07:46.833 INFO:tasks.workunit.client.1.smithi084.stdout:1/868: rmdir d12/d1b/d2e/d52/d61/d74 39 2024-02-28T23:07:46.834 INFO:tasks.workunit.client.0.smithi057.stdout:8/717: fsync d4/de9/d56/d6a/f4d 0 2024-02-28T23:07:46.835 INFO:tasks.workunit.client.0.smithi057.stdout:3/582: creat d3/d4/d1d/d47/d62/db4/fb6 x:0 0 0 2024-02-28T23:07:46.837 INFO:tasks.workunit.client.0.smithi057.stdout:8/718: chown d4/de9/d59/l68 9615462 1 2024-02-28T23:07:46.837 INFO:tasks.workunit.client.0.smithi057.stdout:2/759: getdents d8/d11/d21/d28/d49/d85/d92 0 2024-02-28T23:07:46.838 INFO:tasks.workunit.client.1.smithi084.stdout:5/834: truncate dc/de/df/fa7 494919 0 2024-02-28T23:07:46.838 INFO:tasks.workunit.client.1.smithi084.stdout:1/869: chown d12/d1b/d2e/d52/f67 34 1 2024-02-28T23:07:46.839 INFO:tasks.workunit.client.1.smithi084.stdout:9/860: getdents d4/d32/d3f/d89/db8 0 2024-02-28T23:07:46.841 INFO:tasks.workunit.client.0.smithi057.stdout:3/583: creat d3/d8/d1b/fb7 x:0 0 0 2024-02-28T23:07:46.842 INFO:tasks.workunit.client.0.smithi057.stdout:7/667: dread d0/d5/d36/d4e/d15/d29/f40 [0,4194304] 0 2024-02-28T23:07:46.842 INFO:tasks.workunit.client.0.smithi057.stdout:5/677: dread f2 [0,4194304] 0 2024-02-28T23:07:46.842 INFO:tasks.workunit.client.1.smithi084.stdout:4/783: dwrite d1/d18/d20/d14/f6f [0,4194304] 0 2024-02-28T23:07:46.843 INFO:tasks.workunit.client.0.smithi057.stdout:2/760: fsync d8/d11/d21/d28/f3d 0 2024-02-28T23:07:46.846 INFO:tasks.workunit.client.0.smithi057.stdout:2/761: readlink d8/d11/d15/l5f 0 2024-02-28T23:07:46.849 INFO:tasks.workunit.client.0.smithi057.stdout:5/678: dread - d4/d9/d14/f5f zero size 2024-02-28T23:07:46.853 INFO:tasks.workunit.client.0.smithi057.stdout:5/679: fdatasync d4/d21/f67 0 2024-02-28T23:07:46.855 INFO:tasks.workunit.client.1.smithi084.stdout:0/819: dwrite d1/d8/d69/db2/fbc [0,4194304] 0 2024-02-28T23:07:46.855 INFO:tasks.workunit.client.0.smithi057.stdout:7/668: rename d0/f10 to d0/d3f/d76/dc1/fdf 0 2024-02-28T23:07:46.860 INFO:tasks.workunit.client.0.smithi057.stdout:5/680: fsync d4/d9/d14/d3b/f47 0 2024-02-28T23:07:46.866 INFO:tasks.workunit.client.0.smithi057.stdout:5/681: write d4/d9/fa6 [4150632,105815] 0 2024-02-28T23:07:46.866 INFO:tasks.workunit.client.1.smithi084.stdout:9/861: truncate d4/d32/f107 3937310 0 2024-02-28T23:07:46.866 INFO:tasks.workunit.client.1.smithi084.stdout:3/864: dread d3/d50/d5a/d73/f3f [0,4194304] 0 2024-02-28T23:07:46.868 INFO:tasks.workunit.client.0.smithi057.stdout:3/584: dwrite d3/d8/d27/d70/f74 [0,4194304] 0 2024-02-28T23:07:46.869 INFO:tasks.workunit.client.1.smithi084.stdout:8/808: getdents d0/d9/d30/d33/d71/d84/dc0 0 2024-02-28T23:07:46.869 INFO:tasks.workunit.client.0.smithi057.stdout:8/719: write d4/d13/d1a/f7a [1357898,65505] 0 2024-02-28T23:07:46.869 INFO:tasks.workunit.client.0.smithi057.stdout:5/682: write d4/fb9 [588481,63598] 0 2024-02-28T23:07:46.874 INFO:tasks.workunit.client.1.smithi084.stdout:8/809: chown d0/d9/d30/d33/d71/f75 3 1 2024-02-28T23:07:46.876 INFO:tasks.workunit.client.0.smithi057.stdout:5/683: rmdir d4/d9/d85 39 2024-02-28T23:07:46.878 INFO:tasks.workunit.client.0.smithi057.stdout:2/762: write d8/d11/d39/f87 [542788,94530] 0 2024-02-28T23:07:46.879 INFO:tasks.workunit.client.1.smithi084.stdout:1/870: symlink d12/d1b/d2e/d30/d77/da0/d117/l11b 0 2024-02-28T23:07:46.880 INFO:tasks.workunit.client.0.smithi057.stdout:7/669: dread d0/d5/d36/d4e/fbc [0,4194304] 0 2024-02-28T23:07:46.882 INFO:tasks.workunit.client.0.smithi057.stdout:2/763: chown d8/d11/d21/d28/d49/d85/d9b/fac 20736 1 2024-02-28T23:07:46.883 INFO:tasks.workunit.client.0.smithi057.stdout:5/684: chown d4/d21/d24/d30/lbe 44945 1 2024-02-28T23:07:46.885 INFO:tasks.workunit.client.0.smithi057.stdout:5/685: fdatasync d4/d9/d14/f54 0 2024-02-28T23:07:46.887 INFO:tasks.workunit.client.0.smithi057.stdout:2/764: mknod d8/d11/d21/cea 0 2024-02-28T23:07:46.890 INFO:tasks.workunit.client.1.smithi084.stdout:0/820: dwrite d1/d6/d59/d70/f73 [0,4194304] 0 2024-02-28T23:07:46.890 INFO:tasks.workunit.client.1.smithi084.stdout:3/865: mknod d3/d27/d7d/db0/d14/de1/c142 0 2024-02-28T23:07:46.892 INFO:tasks.workunit.client.0.smithi057.stdout:2/765: rmdir d8/d11/d15/d97 39 2024-02-28T23:07:46.892 INFO:tasks.workunit.client.0.smithi057.stdout:3/585: dwrite d3/d4/d7/f8f [0,4194304] 0 2024-02-28T23:07:46.894 INFO:tasks.workunit.client.1.smithi084.stdout:8/810: unlink d0/d24/f5b 0 2024-02-28T23:07:46.896 INFO:tasks.workunit.client.0.smithi057.stdout:8/720: dread d4/d13/d18/d62/d66/fe5 [0,4194304] 0 2024-02-28T23:07:46.897 INFO:tasks.workunit.client.1.smithi084.stdout:9/862: dwrite d4/d6/d15/f8e [0,4194304] 0 2024-02-28T23:07:46.897 INFO:tasks.workunit.client.1.smithi084.stdout:3/866: fdatasync d3/d27/d7d/db0/d1a/d45/d86/f106 0 2024-02-28T23:07:46.897 INFO:tasks.workunit.client.1.smithi084.stdout:1/871: fsync d12/d84/f8e 0 2024-02-28T23:07:46.898 INFO:tasks.workunit.client.1.smithi084.stdout:8/811: fdatasync d0/d24/d78/fd5 0 2024-02-28T23:07:46.899 INFO:tasks.workunit.client.0.smithi057.stdout:3/586: creat d3/d20/d2d/fb8 x:0 0 0 2024-02-28T23:07:46.902 INFO:tasks.workunit.client.1.smithi084.stdout:4/784: getdents d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0 0 2024-02-28T23:07:46.903 INFO:tasks.workunit.client.1.smithi084.stdout:5/835: dwrite dc/de/df/d31/f89 [0,4194304] 0 2024-02-28T23:07:46.903 INFO:tasks.workunit.client.0.smithi057.stdout:7/670: write d0/d5/d36/d4e/d15/d29/d71/fa8 [734849,63062] 0 2024-02-28T23:07:46.907 INFO:tasks.workunit.client.1.smithi084.stdout:1/872: fdatasync d12/d1b/d2e/d30/d50/da8/f11a 0 2024-02-28T23:07:46.908 INFO:tasks.workunit.client.1.smithi084.stdout:0/821: fdatasync d1/d8/d69/d6b/f77 0 2024-02-28T23:07:46.909 INFO:tasks.workunit.client.1.smithi084.stdout:3/867: write d3/d27/d7d/db0/d14/d31/d34/f96 [1092687,23334] 0 2024-02-28T23:07:46.912 INFO:tasks.workunit.client.0.smithi057.stdout:8/721: getdents d4/d13/d18/d62 0 2024-02-28T23:07:46.914 INFO:tasks.workunit.client.0.smithi057.stdout:3/587: rename d3/d4/d7/f2b to d3/d4/d7/fb9 0 2024-02-28T23:07:46.916 INFO:tasks.workunit.client.0.smithi057.stdout:7/671: write d0/d5/d36/d4e/d3b/d75/f94 [35037,109711] 0 2024-02-28T23:07:46.918 INFO:tasks.workunit.client.0.smithi057.stdout:8/722: creat d4/de9/d56/da6/ffe x:0 0 0 2024-02-28T23:07:46.922 INFO:tasks.workunit.client.0.smithi057.stdout:7/672: mknod d0/d5/ce0 0 2024-02-28T23:07:46.930 INFO:tasks.workunit.client.0.smithi057.stdout:7/673: rename d0/d5/ce0 to d0/d5/d36/d4e/d15/d29/d71/ce1 0 2024-02-28T23:07:46.931 INFO:tasks.workunit.client.0.smithi057.stdout:2/766: write d8/d11/d21/d70/fe8 [895160,94859] 0 2024-02-28T23:07:46.931 INFO:tasks.workunit.client.1.smithi084.stdout:0/822: write d1/d8/d69/f84 [2599149,28612] 0 2024-02-28T23:07:46.932 INFO:tasks.workunit.client.1.smithi084.stdout:5/836: truncate dc/de/df/d16/f26 1508690 0 2024-02-28T23:07:46.934 INFO:tasks.workunit.client.1.smithi084.stdout:3/868: chown d3/d50/d117/d3d/faa 30860 1 2024-02-28T23:07:46.935 INFO:tasks.workunit.client.0.smithi057.stdout:2/767: creat d8/d11/d15/d97/feb x:0 0 0 2024-02-28T23:07:46.936 INFO:tasks.workunit.client.0.smithi057.stdout:2/768: write d8/d11/f32 [581725,92244] 0 2024-02-28T23:07:46.938 INFO:tasks.workunit.client.0.smithi057.stdout:8/723: dread d4/d13/d2c/d4e/f60 [0,4194304] 0 2024-02-28T23:07:46.941 INFO:tasks.workunit.client.0.smithi057.stdout:2/769: mkdir d8/d11/d21/d28/dec 0 2024-02-28T23:07:46.942 INFO:tasks.workunit.client.1.smithi084.stdout:9/863: write d4/f56 [947397,94276] 0 2024-02-28T23:07:46.946 INFO:tasks.workunit.client.0.smithi057.stdout:2/770: truncate d8/dae/fb8 3122439 0 2024-02-28T23:07:46.948 INFO:tasks.workunit.client.0.smithi057.stdout:7/674: dwrite d0/d98/fa2 [0,4194304] 0 2024-02-28T23:07:46.950 INFO:tasks.workunit.client.0.smithi057.stdout:2/771: symlink d8/d11/d21/d70/d99/led 0 2024-02-28T23:07:46.951 INFO:tasks.workunit.client.1.smithi084.stdout:4/785: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fd2 [0,4194304] 0 2024-02-28T23:07:46.956 INFO:tasks.workunit.client.0.smithi057.stdout:8/724: write d4/d13/d18/d62/d66/d74/fc7 [794199,36964] 0 2024-02-28T23:07:46.956 INFO:tasks.workunit.client.0.smithi057.stdout:2/772: fdatasync d8/d11/d21/d28/f2f 0 2024-02-28T23:07:46.959 INFO:tasks.workunit.client.1.smithi084.stdout:8/812: dread d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:46.960 INFO:tasks.workunit.client.1.smithi084.stdout:6/787: sync 2024-02-28T23:07:46.960 INFO:tasks.workunit.client.1.smithi084.stdout:7/912: sync 2024-02-28T23:07:46.960 INFO:tasks.workunit.client.0.smithi057.stdout:7/675: link d0/d5/fda d0/d5/d25/fe2 0 2024-02-28T23:07:46.963 INFO:tasks.workunit.client.0.smithi057.stdout:7/676: creat d0/d3f/fe3 x:0 0 0 2024-02-28T23:07:46.967 INFO:tasks.workunit.client.0.smithi057.stdout:7/677: symlink d0/d5/d36/d4e/d15/da0/le4 0 2024-02-28T23:07:46.968 INFO:tasks.workunit.client.0.smithi057.stdout:2/773: read d8/d11/d15/daf/fb7 [7399265,55951] 0 2024-02-28T23:07:46.970 INFO:tasks.workunit.client.0.smithi057.stdout:7/678: mknod d0/d5/d25/ce5 0 2024-02-28T23:07:46.971 INFO:tasks.workunit.client.0.smithi057.stdout:7/679: dread - d0/d5/d36/fd8 zero size 2024-02-28T23:07:46.973 INFO:tasks.workunit.client.1.smithi084.stdout:9/864: dread d4/d6/d15/f98 [0,4194304] 0 2024-02-28T23:07:46.974 INFO:tasks.workunit.client.0.smithi057.stdout:7/680: creat d0/d5/d36/d4e/d15/d29/d71/fe6 x:0 0 0 2024-02-28T23:07:46.976 INFO:tasks.workunit.client.1.smithi084.stdout:5/837: rmdir dc/de/d83/dd9 39 2024-02-28T23:07:46.978 INFO:tasks.workunit.client.1.smithi084.stdout:2/744: sync 2024-02-28T23:07:46.980 INFO:tasks.workunit.client.1.smithi084.stdout:6/788: rmdir d1/d9/d1d/d1e 39 2024-02-28T23:07:46.982 INFO:tasks.workunit.client.1.smithi084.stdout:0/823: fdatasync d1/d8/da1/ff9 0 2024-02-28T23:07:46.982 INFO:tasks.workunit.client.0.smithi057.stdout:2/774: dwrite d8/d11/d21/f31 [0,4194304] 0 2024-02-28T23:07:46.984 INFO:tasks.workunit.client.1.smithi084.stdout:2/745: write d5/d1f/d42/d74/fd0 [1615101,60582] 0 2024-02-28T23:07:46.985 INFO:tasks.workunit.client.1.smithi084.stdout:4/786: dread d1/d18/d20/d14/d30/f81 [0,4194304] 0 2024-02-28T23:07:46.986 INFO:tasks.workunit.client.0.smithi057.stdout:7/681: dread d0/d5/d36/d4e/fbc [0,4194304] 0 2024-02-28T23:07:46.986 INFO:tasks.workunit.client.1.smithi084.stdout:6/789: write d1/d13/d50/d74/fb7 [1380205,44243] 0 2024-02-28T23:07:46.987 INFO:tasks.workunit.client.0.smithi057.stdout:2/775: mkdir d8/d11/d21/d28/d49/d85/d9b/dad/dee 0 2024-02-28T23:07:46.988 INFO:tasks.workunit.client.1.smithi084.stdout:7/913: mkdir db/d10/d17/d9c/d9f/d11b 0 2024-02-28T23:07:46.990 INFO:tasks.workunit.client.1.smithi084.stdout:5/838: dread dc/de/df/d31/d8e/da9/dab/fe6 [0,4194304] 0 2024-02-28T23:07:46.990 INFO:tasks.workunit.client.0.smithi057.stdout:2/776: stat d8/d11/d39/f3c 0 2024-02-28T23:07:46.991 INFO:tasks.workunit.client.1.smithi084.stdout:0/824: symlink d1/d6/db/d76/dee/df2/l107 0 2024-02-28T23:07:46.991 INFO:tasks.workunit.client.1.smithi084.stdout:6/790: dread - d1/d13/d50/dbd/dc4/fdd zero size 2024-02-28T23:07:46.992 INFO:tasks.workunit.client.1.smithi084.stdout:1/873: link d12/f70 d12/f11c 0 2024-02-28T23:07:46.993 INFO:tasks.workunit.client.0.smithi057.stdout:2/777: mkdir d8/d11/d15/def 0 2024-02-28T23:07:46.994 INFO:tasks.workunit.client.1.smithi084.stdout:9/865: rmdir d4/dee 0 2024-02-28T23:07:46.996 INFO:tasks.workunit.client.1.smithi084.stdout:4/787: creat d1/d18/d20/d24/ddb/f100 x:0 0 0 2024-02-28T23:07:46.998 INFO:tasks.workunit.client.0.smithi057.stdout:2/778: dread d8/d11/d39/f87 [0,4194304] 0 2024-02-28T23:07:46.999 INFO:tasks.workunit.client.1.smithi084.stdout:2/746: symlink d5/d7/d12/d24/d2a/d53/le1 0 2024-02-28T23:07:47.001 INFO:tasks.workunit.client.1.smithi084.stdout:7/914: creat db/d10/d17/d64/d77/f11c x:0 0 0 2024-02-28T23:07:47.002 INFO:tasks.workunit.client.1.smithi084.stdout:9/866: dread d4/d6/f8 [4194304,4194304] 0 2024-02-28T23:07:47.004 INFO:tasks.workunit.client.1.smithi084.stdout:7/915: dread - db/d5e/d80/ff2 zero size 2024-02-28T23:07:47.005 INFO:tasks.workunit.client.1.smithi084.stdout:9/867: write d4/d6/f72 [200294,47580] 0 2024-02-28T23:07:47.008 INFO:tasks.workunit.client.0.smithi057.stdout:7/682: dwrite d0/d5/d36/d4e/d15/d29/f40 [0,4194304] 0 2024-02-28T23:07:47.009 INFO:tasks.workunit.client.1.smithi084.stdout:1/874: rename d12/d1b/d91/c102 to d12/d1b/d2e/d30/d77/da0/c11d 0 2024-02-28T23:07:47.011 INFO:tasks.workunit.client.1.smithi084.stdout:4/788: mkdir d1/d18/d20/d14/d101 0 2024-02-28T23:07:47.012 INFO:tasks.workunit.client.1.smithi084.stdout:2/747: fdatasync d5/f15 0 2024-02-28T23:07:47.013 INFO:tasks.workunit.client.0.smithi057.stdout:7/683: mkdir d0/d5/d36/d4e/d15/d29/d3d/de7 0 2024-02-28T23:07:47.023 INFO:tasks.workunit.client.1.smithi084.stdout:5/839: fdatasync dc/de/df/d16/d34/daa/f131 0 2024-02-28T23:07:47.024 INFO:tasks.workunit.client.1.smithi084.stdout:0/825: creat d1/d8/d18/d4c/f108 x:0 0 0 2024-02-28T23:07:47.032 INFO:tasks.workunit.client.0.smithi057.stdout:7/684: write d0/d5/f30 [1321792,68435] 0 2024-02-28T23:07:47.032 INFO:tasks.workunit.client.1.smithi084.stdout:9/868: mkdir d4/d32/d3f/d89/d10c/d10f/d11a 0 2024-02-28T23:07:47.042 INFO:tasks.workunit.client.1.smithi084.stdout:4/789: creat d1/f102 x:0 0 0 2024-02-28T23:07:47.042 INFO:tasks.workunit.client.1.smithi084.stdout:2/748: link d5/d1f/d83/faf d5/d7/d12/d24/fe2 0 2024-02-28T23:07:47.047 INFO:tasks.workunit.client.0.smithi057.stdout:4/656: sync 2024-02-28T23:07:47.048 INFO:tasks.workunit.client.1.smithi084.stdout:9/869: truncate d4/d32/d3f/fef 269158 0 2024-02-28T23:07:47.048 INFO:tasks.workunit.client.1.smithi084.stdout:0/826: dwrite d1/d8/d18/d4c/d6c/d74/f97 [0,4194304] 0 2024-02-28T23:07:47.048 INFO:tasks.workunit.client.1.smithi084.stdout:2/749: mknod d5/d1f/d42/d8f/ce3 0 2024-02-28T23:07:47.051 INFO:tasks.workunit.client.1.smithi084.stdout:3/869: write d3/d27/d7d/db0/d1a/d45/d5e/fe2 [729639,34658] 0 2024-02-28T23:07:47.052 INFO:tasks.workunit.client.1.smithi084.stdout:8/813: write d0/d9/d30/d33/d71/f6b [423528,123729] 0 2024-02-28T23:07:47.055 INFO:tasks.workunit.client.1.smithi084.stdout:3/870: write d3/d27/d4b/fd6 [1305229,95213] 0 2024-02-28T23:07:47.057 INFO:tasks.workunit.client.1.smithi084.stdout:1/875: dread d12/d84/f8e [0,4194304] 0 2024-02-28T23:07:47.057 INFO:tasks.workunit.client.1.smithi084.stdout:2/750: write d5/f17 [1349807,13915] 0 2024-02-28T23:07:47.059 INFO:tasks.workunit.client.1.smithi084.stdout:8/814: readlink d0/d9/d1b/d38/d43/d51/de8/lef 0 2024-02-28T23:07:47.061 INFO:tasks.workunit.client.1.smithi084.stdout:6/791: write d1/fbc [249917,79524] 0 2024-02-28T23:07:47.063 INFO:tasks.workunit.client.1.smithi084.stdout:9/870: rename d4/d32/d3f/fef to d4/d6/d15/d75/d9d/f11b 0 2024-02-28T23:07:47.064 INFO:tasks.workunit.client.1.smithi084.stdout:0/827: truncate d1/d6/f53 294055 0 2024-02-28T23:07:47.065 INFO:tasks.workunit.client.1.smithi084.stdout:6/792: dread d1/d88/dbf/fc7 [0,4194304] 0 2024-02-28T23:07:47.071 INFO:tasks.workunit.client.1.smithi084.stdout:8/815: read d0/d9/d30/d33/f5e [664261,71274] 0 2024-02-28T23:07:47.073 INFO:tasks.workunit.client.1.smithi084.stdout:1/876: mkdir d12/d1b/d2e/d11e 0 2024-02-28T23:07:47.075 INFO:tasks.workunit.client.1.smithi084.stdout:0/828: mkdir d1/d8/da1/d109 0 2024-02-28T23:07:47.083 INFO:tasks.workunit.client.0.smithi057.stdout:7/685: mkdir d0/d5/d36/d4e/d3b/d75/de8 0 2024-02-28T23:07:47.084 INFO:tasks.workunit.client.1.smithi084.stdout:6/793: creat d1/d9/f10b x:0 0 0 2024-02-28T23:07:47.086 INFO:tasks.workunit.client.1.smithi084.stdout:7/916: dwrite db/d10/d18/db7/de9/dfa/f44 [0,4194304] 0 2024-02-28T23:07:47.086 INFO:tasks.workunit.client.1.smithi084.stdout:8/816: mknod d0/d9/d30/d33/d71/d84/dce/c10d 0 2024-02-28T23:07:47.090 INFO:tasks.workunit.client.0.smithi057.stdout:7/686: creat d0/d5/d36/d4e/d15/fe9 x:0 0 0 2024-02-28T23:07:47.092 INFO:tasks.workunit.client.0.smithi057.stdout:4/657: write d1a/d48/d97/f2c [638553,4173] 0 2024-02-28T23:07:47.092 INFO:tasks.workunit.client.1.smithi084.stdout:3/871: dwrite d3/d27/f111 [0,4194304] 0 2024-02-28T23:07:47.092 INFO:tasks.workunit.client.1.smithi084.stdout:2/751: write d5/d7/d9/f3c [2692567,77914] 0 2024-02-28T23:07:47.094 INFO:tasks.workunit.client.0.smithi057.stdout:2/779: dread d8/d11/d39/d83/dde/d50/f66 [4194304,4194304] 0 2024-02-28T23:07:47.095 INFO:tasks.workunit.client.1.smithi084.stdout:6/794: dread - d1/d9/f104 zero size 2024-02-28T23:07:47.095 INFO:tasks.workunit.client.0.smithi057.stdout:7/687: write d0/d5/d25/d3e/f7d [1566466,91971] 0 2024-02-28T23:07:47.099 INFO:tasks.workunit.client.1.smithi084.stdout:5/840: dwrite dc/de/df/da6/fd3 [0,4194304] 0 2024-02-28T23:07:47.103 INFO:tasks.workunit.client.1.smithi084.stdout:9/871: dread d4/f13 [0,4194304] 0 2024-02-28T23:07:47.103 INFO:tasks.workunit.client.1.smithi084.stdout:5/841: chown dc/de/df/d31/dd1/d5f 381789484 1 2024-02-28T23:07:47.109 INFO:tasks.workunit.client.1.smithi084.stdout:8/817: mknod d0/d9/d30/d33/d71/d84/dc0/c10e 0 2024-02-28T23:07:47.112 INFO:tasks.workunit.client.1.smithi084.stdout:9/872: readlink d4/d6/le 0 2024-02-28T23:07:47.114 INFO:tasks.workunit.client.1.smithi084.stdout:3/872: fdatasync d3/d50/d117/d3d/d43/f6c 0 2024-02-28T23:07:47.116 INFO:tasks.workunit.client.1.smithi084.stdout:0/829: dread d1/d8/f3e [0,4194304] 0 2024-02-28T23:07:47.117 INFO:tasks.workunit.client.1.smithi084.stdout:6/795: mknod d1/d13/d2e/d41/d4d/d6c/c10c 0 2024-02-28T23:07:47.120 INFO:tasks.workunit.client.0.smithi057.stdout:4/658: creat d1a/fe4 x:0 0 0 2024-02-28T23:07:47.121 INFO:tasks.workunit.client.1.smithi084.stdout:8/818: mkdir d0/d9/d30/d33/d71/d84/dce/d10f 0 2024-02-28T23:07:47.123 INFO:tasks.workunit.client.0.smithi057.stdout:4/659: stat d1a/d2f/db2/d49/d4d/de2/d6e/cac 0 2024-02-28T23:07:47.124 INFO:tasks.workunit.client.1.smithi084.stdout:5/842: symlink dc/de/df/d16/d34/l13a 0 2024-02-28T23:07:47.124 INFO:tasks.workunit.client.0.smithi057.stdout:2/780: rmdir d8/d11/d39/d83/dde/de3 0 2024-02-28T23:07:47.126 INFO:tasks.workunit.client.0.smithi057.stdout:7/688: getdents d0 0 2024-02-28T23:07:47.127 INFO:tasks.workunit.client.1.smithi084.stdout:3/873: write d3/d50/d117/d7b/fa9 [2894596,75980] 0 2024-02-28T23:07:47.128 INFO:tasks.workunit.client.1.smithi084.stdout:9/873: creat d4/d6/d15/d75/d2d/daa/f11c x:0 0 0 2024-02-28T23:07:47.128 INFO:tasks.workunit.client.0.smithi057.stdout:2/781: unlink d8/d11/d15/f72 0 2024-02-28T23:07:47.130 INFO:tasks.workunit.client.1.smithi084.stdout:2/752: dwrite d5/fd4 [4194304,4194304] 0 2024-02-28T23:07:47.130 INFO:tasks.workunit.client.0.smithi057.stdout:7/689: fsync d0/d5/d25/d80/d9f/fc4 0 2024-02-28T23:07:47.131 INFO:tasks.workunit.client.1.smithi084.stdout:6/796: read d1/d13/d50/fc9 [61261,98586] 0 2024-02-28T23:07:47.133 INFO:tasks.workunit.client.1.smithi084.stdout:8/819: rmdir d0/d24/d78 39 2024-02-28T23:07:47.140 INFO:tasks.workunit.client.1.smithi084.stdout:0/830: unlink d1/d6/db/c43 0 2024-02-28T23:07:47.140 INFO:tasks.workunit.client.1.smithi084.stdout:5/843: creat dc/de/df/d31/dd1/d5f/d9b/dfe/f13b x:0 0 0 2024-02-28T23:07:47.141 INFO:tasks.workunit.client.1.smithi084.stdout:7/917: link db/d10/d18/db7/dbb/le4 db/d10/d18/l11d 0 2024-02-28T23:07:47.143 INFO:tasks.workunit.client.1.smithi084.stdout:3/874: mkdir d3/d27/d7d/db0/d1a/d29/d143 0 2024-02-28T23:07:47.147 INFO:tasks.workunit.client.0.smithi057.stdout:4/660: dwrite d1a/d48/d9c/d68/fa6 [0,4194304] 0 2024-02-28T23:07:47.150 INFO:tasks.workunit.client.1.smithi084.stdout:6/797: rename d1/d13/d2e/d41/d4d/d6c/c83 to d1/d13/d2e/d41/d2c/dfb/c10d 0 2024-02-28T23:07:47.152 INFO:tasks.workunit.client.1.smithi084.stdout:6/798: readlink d1/d13/d2e/d41/d2c/dfb/l57 0 2024-02-28T23:07:47.155 INFO:tasks.workunit.client.0.smithi057.stdout:4/661: rename d1a/d48/d9c/d2e/d43/f47 to d1a/d48/d9c/d2e/d8f/fe5 0 2024-02-28T23:07:47.160 INFO:tasks.workunit.client.1.smithi084.stdout:3/875: rmdir d3/d50/d97 39 2024-02-28T23:07:47.161 INFO:tasks.workunit.client.1.smithi084.stdout:9/874: write d4/d6/d15/d12/fae [2690761,30278] 0 2024-02-28T23:07:47.163 INFO:tasks.workunit.client.1.smithi084.stdout:7/918: mknod db/d10/d17/c11e 0 2024-02-28T23:07:47.167 INFO:tasks.workunit.client.0.smithi057.stdout:0/774: sync 2024-02-28T23:07:47.167 INFO:tasks.workunit.client.1.smithi084.stdout:6/799: fdatasync d1/d13/d2e/d41/fa9 0 2024-02-28T23:07:47.168 INFO:tasks.workunit.client.1.smithi084.stdout:0/831: dwrite d1/d6/d59/ff3 [0,4194304] 0 2024-02-28T23:07:47.174 INFO:tasks.workunit.client.1.smithi084.stdout:5/844: link dc/de/df/d16/l35 dc/de/df/d16/d34/daa/df4/l13c 0 2024-02-28T23:07:47.177 INFO:tasks.workunit.client.1.smithi084.stdout:7/919: creat db/d10/d18/dc2/f11f x:0 0 0 2024-02-28T23:07:47.177 INFO:tasks.workunit.client.1.smithi084.stdout:8/820: dwrite d0/d9/d1b/f28 [0,4194304] 0 2024-02-28T23:07:47.188 INFO:tasks.workunit.client.1.smithi084.stdout:2/753: dwrite d5/d1f/d42/d74/f76 [0,4194304] 0 2024-02-28T23:07:47.191 INFO:tasks.workunit.client.1.smithi084.stdout:2/754: chown d5/d7/d12/d24/d2a/c30 327200436 1 2024-02-28T23:07:47.193 INFO:tasks.workunit.client.1.smithi084.stdout:2/755: chown d5/d1f/d42/d8f/dbe/lc5 138 1 2024-02-28T23:07:47.194 INFO:tasks.workunit.client.1.smithi084.stdout:2/756: chown d5/d7/d2e 17497 1 2024-02-28T23:07:47.195 INFO:tasks.workunit.client.1.smithi084.stdout:4/790: sync 2024-02-28T23:07:47.203 INFO:tasks.workunit.client.0.smithi057.stdout:4/662: rename d1a/d48/d97/f5c to d1a/d1e/fe6 0 2024-02-28T23:07:47.205 INFO:tasks.workunit.client.0.smithi057.stdout:4/663: fdatasync d1a/d48/d97/d7f/fdf 0 2024-02-28T23:07:47.206 INFO:tasks.workunit.client.0.smithi057.stdout:0/775: dread d17/d20/d23/d47/fa2 [0,4194304] 0 2024-02-28T23:07:47.211 INFO:tasks.workunit.client.0.smithi057.stdout:0/776: rename d17/d19/c54 to d17/d20/db6/c114 0 2024-02-28T23:07:47.212 INFO:tasks.workunit.client.1.smithi084.stdout:2/757: dwrite d5/d1f/d42/d74/fd0 [0,4194304] 0 2024-02-28T23:07:47.214 INFO:tasks.workunit.client.0.smithi057.stdout:0/777: creat d17/d19/d4d/f115 x:0 0 0 2024-02-28T23:07:47.216 INFO:tasks.workunit.client.0.smithi057.stdout:0/778: read - d17/d20/d4a/d78/db5/dba/f10c zero size 2024-02-28T23:07:47.219 INFO:tasks.workunit.client.0.smithi057.stdout:0/779: write d17/d19/d4d/f106 [747797,7503] 0 2024-02-28T23:07:47.227 INFO:tasks.workunit.client.0.smithi057.stdout:0/780: symlink d17/d20/d23/d47/d60/l116 0 2024-02-28T23:07:47.230 INFO:tasks.workunit.client.0.smithi057.stdout:0/781: mknod d17/d20/d23/c117 0 2024-02-28T23:07:47.235 INFO:tasks.workunit.client.0.smithi057.stdout:0/782: rename d17/d19/d4d/d7b/d113 to d17/d20/d4a/d78/db5/de9/d118 0 2024-02-28T23:07:47.237 INFO:tasks.workunit.client.0.smithi057.stdout:0/783: creat d17/d20/d4a/f119 x:0 0 0 2024-02-28T23:07:47.245 INFO:tasks.workunit.client.1.smithi084.stdout:1/877: sync 2024-02-28T23:07:47.254 INFO:tasks.workunit.client.0.smithi057.stdout:0/784: dwrite d17/d19/d4d/d7b/f94 [0,4194304] 0 2024-02-28T23:07:47.258 INFO:tasks.workunit.client.0.smithi057.stdout:0/785: stat d17/d20/d26/d48/d7c/d98/dfd 0 2024-02-28T23:07:47.277 INFO:tasks.workunit.client.1.smithi084.stdout:8/821: write d0/d11/f3c [1852329,114435] 0 2024-02-28T23:07:47.280 INFO:tasks.workunit.client.1.smithi084.stdout:6/800: write d1/d9/fac [543468,54511] 0 2024-02-28T23:07:47.287 INFO:tasks.workunit.client.1.smithi084.stdout:9/875: dwrite d4/d6/d15/d12/fad [0,4194304] 0 2024-02-28T23:07:47.292 INFO:tasks.workunit.client.1.smithi084.stdout:0/832: fsync d1/d8/d18/d4c/d6c/d74/fad 0 2024-02-28T23:07:47.293 INFO:tasks.workunit.client.0.smithi057.stdout:1/604: sync 2024-02-28T23:07:47.293 INFO:tasks.workunit.client.0.smithi057.stdout:6/655: sync 2024-02-28T23:07:47.295 INFO:tasks.workunit.client.0.smithi057.stdout:6/656: readlink d0/d1b/d3e/d3d/d7d/l95 0 2024-02-28T23:07:47.297 INFO:tasks.workunit.client.0.smithi057.stdout:6/657: chown d0/d17/d94/cbf 5716034 1 2024-02-28T23:07:47.304 INFO:tasks.workunit.client.1.smithi084.stdout:6/801: mknod d1/d9/ddc/c10e 0 2024-02-28T23:07:47.305 INFO:tasks.workunit.client.1.smithi084.stdout:6/802: write d1/d13/d2e/f44 [5101776,1706] 0 2024-02-28T23:07:47.311 INFO:tasks.workunit.client.0.smithi057.stdout:0/786: symlink d17/d20/d26/d48/d7c/d98/l11a 0 2024-02-28T23:07:47.313 INFO:tasks.workunit.client.0.smithi057.stdout:1/605: creat d2/d7/d4f/fd9 x:0 0 0 2024-02-28T23:07:47.314 INFO:tasks.workunit.client.0.smithi057.stdout:0/787: write d17/d20/db6/f10b [837054,50843] 0 2024-02-28T23:07:47.314 INFO:tasks.workunit.client.1.smithi084.stdout:0/833: chown d1/d8/d18/d4c/d6c/d74/fe0 8092 1 2024-02-28T23:07:47.317 INFO:tasks.workunit.client.1.smithi084.stdout:5/845: creat dc/de/df/d16/f13d x:0 0 0 2024-02-28T23:07:47.317 INFO:tasks.workunit.client.1.smithi084.stdout:0/834: stat d1/d6/d59/d70 0 2024-02-28T23:07:47.317 INFO:tasks.workunit.client.0.smithi057.stdout:0/788: truncate f4 1530371 0 2024-02-28T23:07:47.321 INFO:tasks.workunit.client.1.smithi084.stdout:5/846: chown dc/de/df/d31/f41 676437 1 2024-02-28T23:07:47.331 INFO:tasks.workunit.client.1.smithi084.stdout:2/758: dread d5/d7/d12/d24/f66 [0,4194304] 0 2024-02-28T23:07:47.332 INFO:tasks.workunit.client.1.smithi084.stdout:7/920: getdents db/d5e/d80 0 2024-02-28T23:07:47.333 INFO:tasks.workunit.client.0.smithi057.stdout:0/789: dwrite d17/f9c [0,4194304] 0 2024-02-28T23:07:47.334 INFO:tasks.workunit.client.1.smithi084.stdout:2/759: chown d5/dd/cb2 0 1 2024-02-28T23:07:47.336 INFO:tasks.workunit.client.1.smithi084.stdout:8/822: creat d0/d44/d9f/f110 x:0 0 0 2024-02-28T23:07:47.336 INFO:tasks.workunit.client.1.smithi084.stdout:0/835: truncate d1/d6/db/d37/d72/fb6 235792 0 2024-02-28T23:07:47.338 INFO:tasks.workunit.client.1.smithi084.stdout:4/791: link d1/d18/d20/d24/d2f/d3e/d52/ce6 d1/d18/d20/d24/da8/c103 0 2024-02-28T23:07:47.339 INFO:tasks.workunit.client.1.smithi084.stdout:0/836: chown d1/d8/d69/d6b/f9b 0 1 2024-02-28T23:07:47.340 INFO:tasks.workunit.client.0.smithi057.stdout:0/790: rename d17/d20/d26/d48/d55/d88/f111 to d17/d20/db6/f11b 0 2024-02-28T23:07:47.344 INFO:tasks.workunit.client.1.smithi084.stdout:5/847: rename dc/de/df/d16/d34/d3c/d98/f109 to dc/de/df/db0/f13e 0 2024-02-28T23:07:47.347 INFO:tasks.workunit.client.1.smithi084.stdout:8/823: rmdir d0/d9/d30 39 2024-02-28T23:07:47.347 INFO:tasks.workunit.client.1.smithi084.stdout:8/824: chown d0/d9 39 1 2024-02-28T23:07:47.351 INFO:tasks.workunit.client.1.smithi084.stdout:8/825: truncate d0/d44/d9f/f110 93908 0 2024-02-28T23:07:47.355 INFO:tasks.workunit.client.0.smithi057.stdout:0/791: dwrite d17/d19/dd2/fe0 [0,4194304] 0 2024-02-28T23:07:47.356 INFO:tasks.workunit.client.1.smithi084.stdout:4/792: fsync d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f9f 0 2024-02-28T23:07:47.356 INFO:tasks.workunit.client.1.smithi084.stdout:4/793: readlink d1/d18/l83 0 2024-02-28T23:07:47.359 INFO:tasks.workunit.client.1.smithi084.stdout:3/876: link d3/c74 d3/d27/d7d/db0/df/c144 0 2024-02-28T23:07:47.362 INFO:tasks.workunit.client.1.smithi084.stdout:4/794: chown d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/ce2 177 1 2024-02-28T23:07:47.365 INFO:tasks.workunit.client.1.smithi084.stdout:4/795: write d1/d18/d20/d65/d8e/d55/dc3/dec/ff2 [67918,96259] 0 2024-02-28T23:07:47.372 INFO:tasks.workunit.client.0.smithi057.stdout:0/792: dwrite d17/d20/d23/fb8 [0,4194304] 0 2024-02-28T23:07:47.377 INFO:tasks.workunit.client.1.smithi084.stdout:0/837: getdents d1/d8 0 2024-02-28T23:07:47.387 INFO:tasks.workunit.client.1.smithi084.stdout:0/838: creat d1/d8/d18/d4c/d6c/d74/f10a x:0 0 0 2024-02-28T23:07:47.399 INFO:tasks.workunit.client.1.smithi084.stdout:2/760: mkdir d5/d7/d9/d7c/d5b/de4 0 2024-02-28T23:07:47.400 INFO:tasks.workunit.client.1.smithi084.stdout:2/761: write d5/d7/d9/d7c/d5b/da6/fbb [325751,4357] 0 2024-02-28T23:07:47.404 INFO:tasks.workunit.client.1.smithi084.stdout:2/762: write d5/d7/d9/d7c/d5b/f7f [211696,2783] 0 2024-02-28T23:07:47.411 INFO:tasks.workunit.client.0.smithi057.stdout:0/793: dread f10 [0,4194304] 0 2024-02-28T23:07:47.413 INFO:tasks.workunit.client.0.smithi057.stdout:0/794: symlink d17/l11c 0 2024-02-28T23:07:47.416 INFO:tasks.workunit.client.0.smithi057.stdout:0/795: creat d17/d20/d23/d47/d60/dea/f11d x:0 0 0 2024-02-28T23:07:47.416 INFO:tasks.workunit.client.0.smithi057.stdout:0/796: chown cf 9 1 2024-02-28T23:07:47.419 INFO:tasks.workunit.client.1.smithi084.stdout:2/763: dwrite d5/d1f/d42/d8f/dbe/fd3 [0,4194304] 0 2024-02-28T23:07:47.422 INFO:tasks.workunit.client.1.smithi084.stdout:2/764: write d5/d1f/d42/d8f/dbe/fd3 [45091,41202] 0 2024-02-28T23:07:47.426 INFO:tasks.workunit.client.1.smithi084.stdout:0/839: unlink d1/d6/db/d37/d72/fb6 0 2024-02-28T23:07:47.428 INFO:tasks.workunit.client.1.smithi084.stdout:0/840: write d1/d8/d78/f9d [1224814,123709] 0 2024-02-28T23:07:47.430 INFO:tasks.workunit.client.1.smithi084.stdout:0/841: dread - d1/d8/d69/d6b/d85/fd5 zero size 2024-02-28T23:07:47.432 INFO:tasks.workunit.client.0.smithi057.stdout:7/690: sync 2024-02-28T23:07:47.432 INFO:tasks.workunit.client.0.smithi057.stdout:9/663: sync 2024-02-28T23:07:47.432 INFO:tasks.workunit.client.0.smithi057.stdout:5/686: sync 2024-02-28T23:07:47.434 INFO:tasks.workunit.client.0.smithi057.stdout:7/691: rmdir d0 39 2024-02-28T23:07:47.437 INFO:tasks.workunit.client.1.smithi084.stdout:6/803: truncate d1/d9/d1d/d1e/d33/f39 1705017 0 2024-02-28T23:07:47.437 INFO:tasks.workunit.client.1.smithi084.stdout:0/842: mknod d1/d6/db/d35/d44/da9/c10b 0 2024-02-28T23:07:47.439 INFO:tasks.workunit.client.1.smithi084.stdout:8/826: write d0/d9/d30/db6/fc8 [355342,56980] 0 2024-02-28T23:07:47.444 INFO:tasks.workunit.client.1.smithi084.stdout:1/878: mkdir d12/d1b/d2e/d52/d11f 0 2024-02-28T23:07:47.444 INFO:tasks.workunit.client.0.smithi057.stdout:0/797: write d17/d19/f76 [470271,124559] 0 2024-02-28T23:07:47.445 INFO:tasks.workunit.client.1.smithi084.stdout:4/796: truncate d1/d18/d20/d14/fab 1917138 0 2024-02-28T23:07:47.449 INFO:tasks.workunit.client.0.smithi057.stdout:7/692: rename d0/d5/d36/d4e/d3b to d0/d3f/d76/dc1/dea 0 2024-02-28T23:07:47.449 INFO:tasks.workunit.client.1.smithi084.stdout:2/765: dwrite d5/d7/d12/d24/d3a/d48/f8d [0,4194304] 0 2024-02-28T23:07:47.451 INFO:tasks.workunit.client.1.smithi084.stdout:1/879: symlink d12/d1b/d2e/d30/d77/l120 0 2024-02-28T23:07:47.452 INFO:tasks.workunit.client.0.smithi057.stdout:0/798: dread - d17/d20/d23/d47/f7f zero size 2024-02-28T23:07:47.452 INFO:tasks.workunit.client.0.smithi057.stdout:9/664: link cb dd/de/d24/d63/dc1/dd1/cd5 0 2024-02-28T23:07:47.454 INFO:tasks.workunit.client.0.smithi057.stdout:7/693: read d0/d5/d25/d80/d9f/fdb [499997,23528] 0 2024-02-28T23:07:47.455 INFO:tasks.workunit.client.1.smithi084.stdout:8/827: truncate d0/d24/d78/fbd 5019218 0 2024-02-28T23:07:47.456 INFO:tasks.workunit.client.1.smithi084.stdout:3/877: creat d3/d27/d7d/db0/d1a/d45/d5e/f145 x:0 0 0 2024-02-28T23:07:47.457 INFO:tasks.workunit.client.1.smithi084.stdout:4/797: fsync d1/d18/d20/d14/d30/f81 0 2024-02-28T23:07:47.458 INFO:tasks.workunit.client.0.smithi057.stdout:9/665: creat dd/fd6 x:0 0 0 2024-02-28T23:07:47.460 INFO:tasks.workunit.client.0.smithi057.stdout:5/687: dwrite d4/d21/d24/f86 [0,4194304] 0 2024-02-28T23:07:47.461 INFO:tasks.workunit.client.1.smithi084.stdout:9/876: rmdir d4/d6/d15/d12/d41 39 2024-02-28T23:07:47.464 INFO:tasks.workunit.client.0.smithi057.stdout:5/688: fdatasync d4/d21/d24/d30/d62/faa 0 2024-02-28T23:07:47.465 INFO:tasks.workunit.client.1.smithi084.stdout:6/804: getdents d1/d13/d50/d74/d9e/dc2 0 2024-02-28T23:07:47.465 INFO:tasks.workunit.client.1.smithi084.stdout:9/877: chown d4/d6/d15/de0/le2 1 1 2024-02-28T23:07:47.465 INFO:tasks.workunit.client.1.smithi084.stdout:4/798: stat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fa1 0 2024-02-28T23:07:47.466 INFO:tasks.workunit.client.1.smithi084.stdout:7/921: rename db/d10/d18/dc2 to db/d10/d17/d9c/d9f/db1/d104/d120 0 2024-02-28T23:07:47.469 INFO:tasks.workunit.client.1.smithi084.stdout:1/880: symlink d12/d1b/d2e/d52/d61/l121 0 2024-02-28T23:07:47.469 INFO:tasks.workunit.client.0.smithi057.stdout:9/666: creat dd/fd7 x:0 0 0 2024-02-28T23:07:47.471 INFO:tasks.workunit.client.0.smithi057.stdout:7/694: write d0/d5/f2e [1452248,8114] 0 2024-02-28T23:07:47.472 INFO:tasks.workunit.client.0.smithi057.stdout:9/667: write dd/de/d14/f61 [3501839,50840] 0 2024-02-28T23:07:47.472 INFO:tasks.workunit.client.1.smithi084.stdout:7/922: dread - db/d10/d17/d3b/ff3 zero size 2024-02-28T23:07:47.474 INFO:tasks.workunit.client.0.smithi057.stdout:0/799: dwrite d17/d20/d26/d48/d55/fbe [0,4194304] 0 2024-02-28T23:07:47.475 INFO:tasks.workunit.client.1.smithi084.stdout:2/766: link d5/d7/d9/d7c/d5b/d9f/lb5 d5/d1f/d42/d8f/dbe/le5 0 2024-02-28T23:07:47.481 INFO:tasks.workunit.client.1.smithi084.stdout:0/843: dread d1/d6/d59/f5a [0,4194304] 0 2024-02-28T23:07:47.481 INFO:tasks.workunit.client.0.smithi057.stdout:0/800: mknod d17/d20/d23/d47/d60/c11e 0 2024-02-28T23:07:47.482 INFO:tasks.workunit.client.1.smithi084.stdout:3/878: rename d3/d27/d7d/db0/d1a/d45/d5e/d140 to d3/d27/d7d/db0/d1a/d29/d41/d5b/d146 0 2024-02-28T23:07:47.490 INFO:tasks.workunit.client.0.smithi057.stdout:0/801: rename d17/d20/d26/d48/d55/d88/fe8 to d17/d20/d26/d48/d7c/d98/da6/f11f 0 2024-02-28T23:07:47.490 INFO:tasks.workunit.client.1.smithi084.stdout:9/878: rename d4/d6/d15/d12/d41/fc5 to d4/d32/d3f/d89/d10c/d10f/d11a/f11d 0 2024-02-28T23:07:47.491 INFO:tasks.workunit.client.1.smithi084.stdout:6/805: dwrite d1/d9/f104 [0,4194304] 0 2024-02-28T23:07:47.491 INFO:tasks.workunit.client.0.smithi057.stdout:0/802: readlink d17/d19/dd2/ddb/lf4 0 2024-02-28T23:07:47.492 INFO:tasks.workunit.client.0.smithi057.stdout:9/668: truncate dd/de/d4a/d9f/fab 3679146 0 2024-02-28T23:07:47.496 INFO:tasks.workunit.client.1.smithi084.stdout:7/923: truncate db/d5e/d87/fa2 244872 0 2024-02-28T23:07:47.500 INFO:tasks.workunit.client.0.smithi057.stdout:9/669: rename dd/de/d24/f28 to dd/de/d4a/d60/fd8 0 2024-02-28T23:07:47.502 INFO:tasks.workunit.client.0.smithi057.stdout:9/670: readlink dd/d46/l68 0 2024-02-28T23:07:47.503 INFO:tasks.workunit.client.1.smithi084.stdout:0/844: dread d1/d6/db/d37/f96 [0,4194304] 0 2024-02-28T23:07:47.505 INFO:tasks.workunit.client.1.smithi084.stdout:2/767: rename d5/dd/cb2 to d5/d7/d9/d7c/d5b/de4/ce6 0 2024-02-28T23:07:47.507 INFO:tasks.workunit.client.1.smithi084.stdout:6/806: mkdir d1/d13/d50/dbd/d10f 0 2024-02-28T23:07:47.507 INFO:tasks.workunit.client.0.smithi057.stdout:9/671: stat dd/de/d24/d63/d77/dcf/l99 0 2024-02-28T23:07:47.511 INFO:tasks.workunit.client.1.smithi084.stdout:7/924: symlink db/d5e/d87/dc1/de6/l121 0 2024-02-28T23:07:47.516 INFO:tasks.workunit.client.0.smithi057.stdout:0/803: dwrite d17/d20/d26/d30/fad [0,4194304] 0 2024-02-28T23:07:47.517 INFO:tasks.workunit.client.1.smithi084.stdout:9/879: dwrite d4/d6/d15/d12/f109 [0,4194304] 0 2024-02-28T23:07:47.517 INFO:tasks.workunit.client.1.smithi084.stdout:0/845: symlink d1/d8/d69/db2/l10c 0 2024-02-28T23:07:47.519 INFO:tasks.workunit.client.0.smithi057.stdout:5/689: dread d4/d9/f1c [0,4194304] 0 2024-02-28T23:07:47.519 INFO:tasks.workunit.client.0.smithi057.stdout:7/695: dread d0/d5/d36/d4e/d15/f49 [0,4194304] 0 2024-02-28T23:07:47.521 INFO:tasks.workunit.client.1.smithi084.stdout:2/768: read d5/dd/f1c [1265716,60843] 0 2024-02-28T23:07:47.521 INFO:tasks.workunit.client.0.smithi057.stdout:9/672: mknod dd/de/d24/d63/d77/dcf/d40/d44/da5/cd9 0 2024-02-28T23:07:47.525 INFO:tasks.workunit.client.0.smithi057.stdout:5/690: creat d4/d21/d24/d30/d62/fd5 x:0 0 0 2024-02-28T23:07:47.525 INFO:tasks.workunit.client.1.smithi084.stdout:0/846: symlink d1/d8/d5f/l10d 0 2024-02-28T23:07:47.530 INFO:tasks.workunit.client.1.smithi084.stdout:7/925: mkdir db/d10/d18/db7/de9/dfa/dc7/d100/d122 0 2024-02-28T23:07:47.530 INFO:tasks.workunit.client.0.smithi057.stdout:9/673: dread dd/d46/dc4/f96 [0,4194304] 0 2024-02-28T23:07:47.536 INFO:tasks.workunit.client.1.smithi084.stdout:1/881: dwrite d12/d1b/d2e/d30/f26 [0,4194304] 0 2024-02-28T23:07:47.536 INFO:tasks.workunit.client.1.smithi084.stdout:7/926: read db/d10/d17/f42 [941015,120113] 0 2024-02-28T23:07:47.538 INFO:tasks.workunit.client.1.smithi084.stdout:3/879: dread d3/d27/d7d/db0/d14/d31/d34/fcf [0,4194304] 0 2024-02-28T23:07:47.540 INFO:tasks.workunit.client.1.smithi084.stdout:4/799: truncate d1/d18/d20/d14/d30/fe9 69101 0 2024-02-28T23:07:47.541 INFO:tasks.workunit.client.0.smithi057.stdout:7/696: write d0/d3f/d76/dc1/dea/f5b [758301,23657] 0 2024-02-28T23:07:47.541 INFO:tasks.workunit.client.1.smithi084.stdout:9/880: mkdir d4/d32/d3f/d89/d11e 0 2024-02-28T23:07:47.542 INFO:tasks.workunit.client.1.smithi084.stdout:0/847: dread - d1/d6/db/d35/d82/daf/fc6 zero size 2024-02-28T23:07:47.544 INFO:tasks.workunit.client.1.smithi084.stdout:9/881: chown d4/d6/d15/d75/d9d/dd0 235891 1 2024-02-28T23:07:47.545 INFO:tasks.workunit.client.0.smithi057.stdout:6/658: sync 2024-02-28T23:07:47.545 INFO:tasks.workunit.client.0.smithi057.stdout:4/664: sync 2024-02-28T23:07:47.545 INFO:tasks.workunit.client.0.smithi057.stdout:2/782: sync 2024-02-28T23:07:47.545 INFO:tasks.workunit.client.0.smithi057.stdout:1/606: sync 2024-02-28T23:07:47.546 INFO:tasks.workunit.client.0.smithi057.stdout:3/588: sync 2024-02-28T23:07:47.546 INFO:tasks.workunit.client.0.smithi057.stdout:8/725: sync 2024-02-28T23:07:47.547 INFO:tasks.workunit.client.1.smithi084.stdout:3/880: fsync d3/d27/d7d/db0/f24 0 2024-02-28T23:07:47.550 INFO:tasks.workunit.client.1.smithi084.stdout:1/882: dread - d12/d1b/d2e/d30/d50/fae zero size 2024-02-28T23:07:47.554 INFO:tasks.workunit.client.1.smithi084.stdout:7/927: mknod db/d10/d17/d9c/d9f/d11b/c123 0 2024-02-28T23:07:47.554 INFO:tasks.workunit.client.0.smithi057.stdout:4/665: fsync d1a/d48/db0/f5d 0 2024-02-28T23:07:47.555 INFO:tasks.workunit.client.0.smithi057.stdout:7/697: truncate d0/d5/d36/f3c 162580 0 2024-02-28T23:07:47.557 INFO:tasks.workunit.client.0.smithi057.stdout:3/589: chown d3/d8/d27/d70/f74 26 1 2024-02-28T23:07:47.559 INFO:tasks.workunit.client.0.smithi057.stdout:2/783: creat d8/d11/d15/daf/ff0 x:0 0 0 2024-02-28T23:07:47.560 INFO:tasks.workunit.client.1.smithi084.stdout:1/883: dread d12/d1b/d2e/d52/dd7/f107 [0,4194304] 0 2024-02-28T23:07:47.560 INFO:tasks.workunit.client.1.smithi084.stdout:4/800: rename d1/d18/d20/d14/d30/d47/cbc to d1/d18/d20/d14/d101/c104 0 2024-02-28T23:07:47.562 INFO:tasks.workunit.client.0.smithi057.stdout:9/674: dwrite dd/de/d14/f18 [0,4194304] 0 2024-02-28T23:07:47.564 INFO:tasks.workunit.client.0.smithi057.stdout:4/666: creat d1a/d48/d9c/d68/dc5/fe7 x:0 0 0 2024-02-28T23:07:47.565 INFO:tasks.workunit.client.1.smithi084.stdout:0/848: dwrite d1/d8/ff7 [0,4194304] 0 2024-02-28T23:07:47.568 INFO:tasks.workunit.client.0.smithi057.stdout:5/691: dwrite d4/d21/f35 [0,4194304] 0 2024-02-28T23:07:47.569 INFO:tasks.workunit.client.0.smithi057.stdout:0/804: dread d17/d20/d26/d48/d7c/d98/da6/fdf [0,4194304] 0 2024-02-28T23:07:47.570 INFO:tasks.workunit.client.1.smithi084.stdout:1/884: truncate d12/da2/fa9 696442 0 2024-02-28T23:07:47.571 INFO:tasks.workunit.client.0.smithi057.stdout:1/607: rename d2/d33/l85 to d2/d7/lda 0 2024-02-28T23:07:47.578 INFO:tasks.workunit.client.1.smithi084.stdout:1/885: creat d12/da2/f122 x:0 0 0 2024-02-28T23:07:47.578 INFO:tasks.workunit.client.1.smithi084.stdout:4/801: creat d1/d18/d20/d24/d2f/d3e/d52/f105 x:0 0 0 2024-02-28T23:07:47.580 INFO:tasks.workunit.client.1.smithi084.stdout:7/928: dwrite db/d5e/d8a/ff6 [0,4194304] 0 2024-02-28T23:07:47.580 INFO:tasks.workunit.client.1.smithi084.stdout:0/849: symlink d1/d8/d69/l10e 0 2024-02-28T23:07:47.581 INFO:tasks.workunit.client.0.smithi057.stdout:5/692: write d4/d21/d24/f86 [2341905,82781] 0 2024-02-28T23:07:47.581 INFO:tasks.workunit.client.1.smithi084.stdout:9/882: write d4/d32/f5f [390634,95395] 0 2024-02-28T23:07:47.583 INFO:tasks.workunit.client.0.smithi057.stdout:3/590: link d3/c19 d3/d20/cba 0 2024-02-28T23:07:47.583 INFO:tasks.workunit.client.0.smithi057.stdout:8/726: rename d4/d13/d2c/d4e/fe2 to d4/d13/d1a/d4a/db3/dfc/fff 0 2024-02-28T23:07:47.584 INFO:tasks.workunit.client.0.smithi057.stdout:1/608: rmdir d2/d71 39 2024-02-28T23:07:47.586 INFO:tasks.workunit.client.1.smithi084.stdout:0/850: stat d1/d6/db/d76/dee/df2 0 2024-02-28T23:07:47.587 INFO:tasks.workunit.client.1.smithi084.stdout:9/883: fdatasync d4/d6/d15/d12/d41/d55/fbb 0 2024-02-28T23:07:47.587 INFO:tasks.workunit.client.1.smithi084.stdout:1/886: write d12/d1b/d2e/d52/d61/d74/ff7 [30758,95226] 0 2024-02-28T23:07:47.588 INFO:tasks.workunit.client.1.smithi084.stdout:7/929: chown db/d10/d17/d9c/d9f/db1/d36/f8e 1634866 1 2024-02-28T23:07:47.591 INFO:tasks.workunit.client.0.smithi057.stdout:9/675: creat dd/de/d4a/d60/d8c/fda x:0 0 0 2024-02-28T23:07:47.593 INFO:tasks.workunit.client.0.smithi057.stdout:5/693: dread - d4/d9/d14/d3b/d61/d71/fa7 zero size 2024-02-28T23:07:47.593 INFO:tasks.workunit.client.1.smithi084.stdout:9/884: write d4/f51 [1029109,117105] 0 2024-02-28T23:07:47.595 INFO:tasks.workunit.client.1.smithi084.stdout:4/802: symlink d1/d18/d20/d24/d2f/d3e/d52/l106 0 2024-02-28T23:07:47.597 INFO:tasks.workunit.client.0.smithi057.stdout:0/805: rename d17/d20/d23/l36 to d17/d20/d4a/d78/l120 0 2024-02-28T23:07:47.599 INFO:tasks.workunit.client.1.smithi084.stdout:0/851: rename d1/d8/d18/l46 to d1/d6/db/d35/d82/l10f 0 2024-02-28T23:07:47.600 INFO:tasks.workunit.client.0.smithi057.stdout:8/727: symlink d4/d13/d1a/d4a/db3/l100 0 2024-02-28T23:07:47.602 INFO:tasks.workunit.client.1.smithi084.stdout:6/807: write d1/d9/d1d/d1e/d33/f39 [2407731,68922] 0 2024-02-28T23:07:47.603 INFO:tasks.workunit.client.0.smithi057.stdout:1/609: creat d2/d7/df/db/fdb x:0 0 0 2024-02-28T23:07:47.605 INFO:tasks.workunit.client.0.smithi057.stdout:4/667: link d1a/d48/cbb d1a/ce8 0 2024-02-28T23:07:47.607 INFO:tasks.workunit.client.1.smithi084.stdout:3/881: dread d3/d50/d117/d3d/f3c [0,4194304] 0 2024-02-28T23:07:47.608 INFO:tasks.workunit.client.1.smithi084.stdout:6/808: chown d1/d13/d2e/d41/d2c/dfb/cc3 77101 1 2024-02-28T23:07:47.610 INFO:tasks.workunit.client.1.smithi084.stdout:3/882: chown d3/d50/d117/d7b/l8d 3777 1 2024-02-28T23:07:47.611 INFO:tasks.workunit.client.0.smithi057.stdout:7/698: dwrite d0/d5/d36/d4e/d15/d29/d3d/f88 [0,4194304] 0 2024-02-28T23:07:47.613 INFO:tasks.workunit.client.1.smithi084.stdout:4/803: creat d1/d18/d20/d24/f107 x:0 0 0 2024-02-28T23:07:47.614 INFO:tasks.workunit.client.0.smithi057.stdout:2/784: dwrite d8/d11/d21/f36 [8388608,4194304] 0 2024-02-28T23:07:47.616 INFO:tasks.workunit.client.0.smithi057.stdout:1/610: mkdir d2/dd4/dd5/ddc 0 2024-02-28T23:07:47.617 INFO:tasks.workunit.client.1.smithi084.stdout:7/930: dread db/d10/d18/db7/de9/dfa/f68 [0,4194304] 0 2024-02-28T23:07:47.618 INFO:tasks.workunit.client.1.smithi084.stdout:0/852: rename d1/d6/d59/d70/fed to d1/d8/d69/db2/f110 0 2024-02-28T23:07:47.618 INFO:tasks.workunit.client.0.smithi057.stdout:7/699: chown d0/d5/d36/d4e/d15/d29/d71/fa8 8036 1 2024-02-28T23:07:47.622 INFO:tasks.workunit.client.0.smithi057.stdout:3/591: write d3/d8/f93 [990644,75178] 0 2024-02-28T23:07:47.623 INFO:tasks.workunit.client.1.smithi084.stdout:4/804: mknod d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/c108 0 2024-02-28T23:07:47.626 INFO:tasks.workunit.client.0.smithi057.stdout:5/694: write d4/f6 [771647,115021] 0 2024-02-28T23:07:47.626 INFO:tasks.workunit.client.1.smithi084.stdout:3/883: creat d3/d27/d7d/db0/d1a/d29/d143/f147 x:0 0 0 2024-02-28T23:07:47.629 INFO:tasks.workunit.client.1.smithi084.stdout:0/853: mkdir d1/d8/d78/d111 0 2024-02-28T23:07:47.629 INFO:tasks.workunit.client.0.smithi057.stdout:2/785: rename d8/d11/d21/d28/d49/d85/d9b/dad/dee to d8/d11/d21/d28/d49/d85/d92/dc5/df1 0 2024-02-28T23:07:47.629 INFO:tasks.workunit.client.0.smithi057.stdout:3/592: creat d3/d8/d27/fbb x:0 0 0 2024-02-28T23:07:47.630 INFO:tasks.workunit.client.1.smithi084.stdout:9/885: dwrite d4/d6/d15/d75/d2d/dbd/fd7 [4194304,4194304] 0 2024-02-28T23:07:47.631 INFO:tasks.workunit.client.0.smithi057.stdout:0/806: write d17/d20/f5d [808444,125953] 0 2024-02-28T23:07:47.634 INFO:tasks.workunit.client.1.smithi084.stdout:6/809: dwrite d1/d13/d47/d91/dba/fe3 [0,4194304] 0 2024-02-28T23:07:47.634 INFO:tasks.workunit.client.1.smithi084.stdout:9/886: fsync d4/f85 0 2024-02-28T23:07:47.635 INFO:tasks.workunit.client.0.smithi057.stdout:1/611: fdatasync d2/d71/d62/f64 0 2024-02-28T23:07:47.636 INFO:tasks.workunit.client.0.smithi057.stdout:8/728: truncate d4/d13/d1a/f34 455863 0 2024-02-28T23:07:47.636 INFO:tasks.workunit.client.1.smithi084.stdout:0/854: write d1/d8/fb4 [2054400,102263] 0 2024-02-28T23:07:47.637 INFO:tasks.workunit.client.1.smithi084.stdout:1/887: write d12/d1b/d2e/d52/f67 [2666925,83687] 0 2024-02-28T23:07:47.638 INFO:tasks.workunit.client.0.smithi057.stdout:9/676: dread dd/de/d4a/d60/f7e [0,4194304] 0 2024-02-28T23:07:47.639 INFO:tasks.workunit.client.1.smithi084.stdout:6/810: read - d1/d9/d1d/fef zero size 2024-02-28T23:07:47.640 INFO:tasks.workunit.client.0.smithi057.stdout:7/700: link d0/d5/d36/d4e/d15/d29/f40 d0/d3f/d76/dc1/feb 0 2024-02-28T23:07:47.641 INFO:tasks.workunit.client.0.smithi057.stdout:3/593: chown d3/d20/cba 152971 1 2024-02-28T23:07:47.644 INFO:tasks.workunit.client.0.smithi057.stdout:0/807: symlink d17/d19/d4d/l121 0 2024-02-28T23:07:47.645 INFO:tasks.workunit.client.1.smithi084.stdout:3/884: unlink d3/d27/d7d/db0/d14/d31/d34/c9d 0 2024-02-28T23:07:47.647 INFO:tasks.workunit.client.1.smithi084.stdout:6/811: chown d1/d13/d2e/d41/fa9 9559 1 2024-02-28T23:07:47.648 INFO:tasks.workunit.client.0.smithi057.stdout:5/695: creat d4/fd6 x:0 0 0 2024-02-28T23:07:47.652 INFO:tasks.workunit.client.1.smithi084.stdout:0/855: mkdir d1/d6/d59/d112 0 2024-02-28T23:07:47.653 INFO:tasks.workunit.client.1.smithi084.stdout:1/888: truncate d12/d1b/d2e/f57 2629946 0 2024-02-28T23:07:47.654 INFO:tasks.workunit.client.0.smithi057.stdout:5/696: stat d4/d9/d85/fa3 0 2024-02-28T23:07:47.656 INFO:tasks.workunit.client.1.smithi084.stdout:1/889: truncate d12/d1b/d2e/fe5 614612 0 2024-02-28T23:07:47.657 INFO:tasks.workunit.client.0.smithi057.stdout:8/729: dread d4/de9/d56/d88/fd1 [0,4194304] 0 2024-02-28T23:07:47.657 INFO:tasks.workunit.client.0.smithi057.stdout:0/808: write d17/d20/d23/fb8 [4437610,57214] 0 2024-02-28T23:07:47.658 INFO:tasks.workunit.client.1.smithi084.stdout:3/885: symlink d3/d27/d7d/db0/d1a/d29/d7e/l148 0 2024-02-28T23:07:47.659 INFO:tasks.workunit.client.0.smithi057.stdout:4/668: dwrite d1a/d1e/fa8 [0,4194304] 0 2024-02-28T23:07:47.659 INFO:tasks.workunit.client.1.smithi084.stdout:6/812: dread d1/fbc [0,4194304] 0 2024-02-28T23:07:47.660 INFO:tasks.workunit.client.0.smithi057.stdout:7/701: getdents d0/d5/d36 0 2024-02-28T23:07:47.662 INFO:tasks.workunit.client.1.smithi084.stdout:9/887: creat d4/d6/d15/d75/d9d/dc3/dce/f11f x:0 0 0 2024-02-28T23:07:47.666 INFO:tasks.workunit.client.1.smithi084.stdout:0/856: mkdir d1/d6/db/d35/d44/d113 0 2024-02-28T23:07:47.667 INFO:tasks.workunit.client.1.smithi084.stdout:6/813: read d1/d9/fac [218890,68426] 0 2024-02-28T23:07:47.668 INFO:tasks.workunit.client.0.smithi057.stdout:5/697: readlink d4/d9/d14/d63/d88/l91 0 2024-02-28T23:07:47.669 INFO:tasks.workunit.client.1.smithi084.stdout:1/890: unlink d12/d1b/d2e/d52/d5b/f62 0 2024-02-28T23:07:47.673 INFO:tasks.workunit.client.0.smithi057.stdout:8/730: creat d4/d13/d18/da7/f101 x:0 0 0 2024-02-28T23:07:47.673 INFO:tasks.workunit.client.1.smithi084.stdout:7/931: dread db/d10/d17/d9c/d9f/db1/f35 [0,4194304] 0 2024-02-28T23:07:47.675 INFO:tasks.workunit.client.0.smithi057.stdout:5/698: chown d4/d7/d58/d78/cbd 4505435 1 2024-02-28T23:07:47.678 INFO:tasks.workunit.client.0.smithi057.stdout:8/731: stat d4/d13/d18/d62/d66/d74 0 2024-02-28T23:07:47.678 INFO:tasks.workunit.client.1.smithi084.stdout:5/848: sync 2024-02-28T23:07:47.678 INFO:tasks.workunit.client.1.smithi084.stdout:0/857: symlink d1/d6/db/l114 0 2024-02-28T23:07:47.680 INFO:tasks.workunit.client.1.smithi084.stdout:5/849: dread - dc/d51/f136 zero size 2024-02-28T23:07:47.680 INFO:tasks.workunit.client.1.smithi084.stdout:7/932: symlink db/d10/d17/d9c/d9f/db1/d36/dea/l124 0 2024-02-28T23:07:47.688 INFO:tasks.workunit.client.0.smithi057.stdout:1/612: dwrite d2/d7/df/db/d3a/fc9 [0,4194304] 0 2024-02-28T23:07:47.689 INFO:tasks.workunit.client.0.smithi057.stdout:2/786: dwrite d8/d11/d15/fc4 [0,4194304] 0 2024-02-28T23:07:47.689 INFO:tasks.workunit.client.1.smithi084.stdout:9/888: link d4/d32/d3f/d89/d10c/d10f/d11a/f11d d4/d6/d15/d75/f120 0 2024-02-28T23:07:47.689 INFO:tasks.workunit.client.1.smithi084.stdout:5/850: mknod dc/de/df/d16/d34/daa/df4/c13f 0 2024-02-28T23:07:47.697 INFO:tasks.workunit.client.0.smithi057.stdout:3/594: dwrite d3/d4f/f7d [0,4194304] 0 2024-02-28T23:07:47.697 INFO:tasks.workunit.client.1.smithi084.stdout:9/889: mknod d4/d32/d3f/d89/db8/d118/c121 0 2024-02-28T23:07:47.699 INFO:tasks.workunit.client.0.smithi057.stdout:5/699: creat d4/d9/d14/d3b/d61/db2/fd7 x:0 0 0 2024-02-28T23:07:47.700 INFO:tasks.workunit.client.0.smithi057.stdout:9/677: truncate dd/de/d24/d63/d77/dcf/d40/d44/f7a 3269123 0 2024-02-28T23:07:47.703 INFO:tasks.workunit.client.1.smithi084.stdout:9/890: creat d4/d69/f122 x:0 0 0 2024-02-28T23:07:47.705 INFO:tasks.workunit.client.0.smithi057.stdout:3/595: write d3/fd [802076,2633] 0 2024-02-28T23:07:47.705 INFO:tasks.workunit.client.1.smithi084.stdout:3/886: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:47.708 INFO:tasks.workunit.client.0.smithi057.stdout:2/787: truncate d8/d11/d15/db9/fbf 1234406 0 2024-02-28T23:07:47.711 INFO:tasks.workunit.client.1.smithi084.stdout:4/805: truncate d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/fa1 1173542 0 2024-02-28T23:07:47.714 INFO:tasks.workunit.client.0.smithi057.stdout:7/702: dread d0/d5/d36/d4e/d15/d29/d3d/d51/f5e [4194304,4194304] 0 2024-02-28T23:07:47.715 INFO:tasks.workunit.client.1.smithi084.stdout:8/828: sync 2024-02-28T23:07:47.715 INFO:tasks.workunit.client.0.smithi057.stdout:8/732: truncate d4/d13/d1a/f34 611347 0 2024-02-28T23:07:47.717 INFO:tasks.workunit.client.0.smithi057.stdout:3/596: mkdir d3/d4/d1d/d47/d62/dbc 0 2024-02-28T23:07:47.717 INFO:tasks.workunit.client.0.smithi057.stdout:5/700: write d4/d9/d14/d3b/fa0 [1828067,101115] 0 2024-02-28T23:07:47.718 INFO:tasks.workunit.client.0.smithi057.stdout:2/788: mknod d8/d11/db3/cf2 0 2024-02-28T23:07:47.719 INFO:tasks.workunit.client.1.smithi084.stdout:4/806: chown d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf 9 1 2024-02-28T23:07:47.720 INFO:tasks.workunit.client.0.smithi057.stdout:8/733: chown d4/de9/d56/da6 345892 1 2024-02-28T23:07:47.720 INFO:tasks.workunit.client.0.smithi057.stdout:7/703: mknod d0/d3f/da1/cec 0 2024-02-28T23:07:47.721 INFO:tasks.workunit.client.1.smithi084.stdout:1/891: write d12/faf [538193,60303] 0 2024-02-28T23:07:47.723 INFO:tasks.workunit.client.1.smithi084.stdout:8/829: creat d0/d9/d30/f111 x:0 0 0 2024-02-28T23:07:47.728 INFO:tasks.workunit.client.1.smithi084.stdout:1/892: chown d12/d1b/l78 596 1 2024-02-28T23:07:47.732 INFO:tasks.workunit.client.0.smithi057.stdout:8/734: symlink d4/d13/d18/da7/de1/l102 0 2024-02-28T23:07:47.733 INFO:tasks.workunit.client.0.smithi057.stdout:2/789: creat d8/d11/d21/d70/ff3 x:0 0 0 2024-02-28T23:07:47.737 INFO:tasks.workunit.client.1.smithi084.stdout:8/830: mknod d0/d9/d1b/dc1/c112 0 2024-02-28T23:07:47.738 INFO:tasks.workunit.client.1.smithi084.stdout:3/887: write d3/d27/d7d/db0/d1a/d29/d41/d60/d98/fa8 [4079197,66703] 0 2024-02-28T23:07:47.739 INFO:tasks.workunit.client.0.smithi057.stdout:1/613: dwrite d2/d71/d2d/d39/da3/db5/fb7 [0,4194304] 0 2024-02-28T23:07:47.741 INFO:tasks.workunit.client.1.smithi084.stdout:7/933: dread db/d10/d17/d9c/d9f/db1/f3a [0,4194304] 0 2024-02-28T23:07:47.743 INFO:tasks.workunit.client.1.smithi084.stdout:1/893: symlink d12/d1b/d91/l123 0 2024-02-28T23:07:47.745 INFO:tasks.workunit.client.1.smithi084.stdout:6/814: dwrite d1/d9/d1d/d1e/d33/f75 [0,4194304] 0 2024-02-28T23:07:47.747 INFO:tasks.workunit.client.1.smithi084.stdout:0/858: dwrite d1/d8/f55 [0,4194304] 0 2024-02-28T23:07:47.748 INFO:tasks.workunit.client.1.smithi084.stdout:9/891: dwrite d4/d6/f29 [0,4194304] 0 2024-02-28T23:07:47.749 INFO:tasks.workunit.client.1.smithi084.stdout:8/831: write d0/d24/d9e/ff8 [5238798,93425] 0 2024-02-28T23:07:47.751 INFO:tasks.workunit.client.1.smithi084.stdout:3/888: creat d3/d50/d117/d3d/d43/de3/f149 x:0 0 0 2024-02-28T23:07:47.756 INFO:tasks.workunit.client.1.smithi084.stdout:5/851: dread dc/de/df/da6/ff7 [0,4194304] 0 2024-02-28T23:07:47.756 INFO:tasks.workunit.client.1.smithi084.stdout:1/894: symlink d12/d1b/d2e/d52/dd7/l124 0 2024-02-28T23:07:47.757 INFO:tasks.workunit.client.1.smithi084.stdout:6/815: chown d1/d13/d2e/d41/d2c/f71 15 1 2024-02-28T23:07:47.758 INFO:tasks.workunit.client.0.smithi057.stdout:0/809: dwrite d17/d20/d4a/d78/f79 [0,4194304] 0 2024-02-28T23:07:47.759 INFO:tasks.workunit.client.0.smithi057.stdout:1/614: rmdir d2/d7/df/db/d12/d19/d65 39 2024-02-28T23:07:47.760 INFO:tasks.workunit.client.1.smithi084.stdout:9/892: fdatasync d4/d6/d53/f7d 0 2024-02-28T23:07:47.761 INFO:tasks.workunit.client.1.smithi084.stdout:7/934: mknod db/d10/d17/d9c/d9f/db1/d104/d120/c125 0 2024-02-28T23:07:47.763 INFO:tasks.workunit.client.1.smithi084.stdout:8/832: truncate d0/d9/d1b/dc1/fc7 4500669 0 2024-02-28T23:07:47.763 INFO:tasks.workunit.client.0.smithi057.stdout:8/735: mknod d4/de9/d56/d6a/d54/d6b/dca/c103 0 2024-02-28T23:07:47.766 INFO:tasks.workunit.client.1.smithi084.stdout:3/889: dread - d3/d27/d7d/db0/d1a/d45/dde/f12a zero size 2024-02-28T23:07:47.766 INFO:tasks.workunit.client.1.smithi084.stdout:5/852: symlink dc/de/df/d31/dd1/d5f/d66/l140 0 2024-02-28T23:07:47.767 INFO:tasks.workunit.client.0.smithi057.stdout:9/678: dwrite dd/fd7 [0,4194304] 0 2024-02-28T23:07:47.768 INFO:tasks.workunit.client.1.smithi084.stdout:1/895: chown d12/d1b/d63/d7d/lc2 14411 1 2024-02-28T23:07:47.768 INFO:tasks.workunit.client.1.smithi084.stdout:6/816: chown d1/d13/d50/fa6 7 1 2024-02-28T23:07:47.768 INFO:tasks.workunit.client.1.smithi084.stdout:9/893: write d4/d32/f92 [130483,116712] 0 2024-02-28T23:07:47.769 INFO:tasks.workunit.client.0.smithi057.stdout:3/597: dwrite d3/d4f/f48 [0,4194304] 0 2024-02-28T23:07:47.773 INFO:tasks.workunit.client.1.smithi084.stdout:3/890: rmdir d3/d27/d7d/db0/d1a/df9/dfe 39 2024-02-28T23:07:47.774 INFO:tasks.workunit.client.1.smithi084.stdout:6/817: rmdir d1/d9/ddc 39 2024-02-28T23:07:47.775 INFO:tasks.workunit.client.0.smithi057.stdout:4/669: dread d1a/d2f/db2/d49/d4d/dc9/fd5 [0,4194304] 0 2024-02-28T23:07:47.776 INFO:tasks.workunit.client.1.smithi084.stdout:9/894: rmdir d4/d6/d15 39 2024-02-28T23:07:47.778 INFO:tasks.workunit.client.0.smithi057.stdout:0/810: dread - d17/d20/d4a/d78/f7d zero size 2024-02-28T23:07:47.779 INFO:tasks.workunit.client.1.smithi084.stdout:7/935: creat db/f126 x:0 0 0 2024-02-28T23:07:47.780 INFO:tasks.workunit.client.1.smithi084.stdout:8/833: mknod d0/c113 0 2024-02-28T23:07:47.780 INFO:tasks.workunit.client.1.smithi084.stdout:5/853: dread dc/de/f46 [0,4194304] 0 2024-02-28T23:07:47.780 INFO:tasks.workunit.client.1.smithi084.stdout:9/895: write d4/d6/f7 [4262411,95853] 0 2024-02-28T23:07:47.780 INFO:tasks.workunit.client.1.smithi084.stdout:4/807: dread d1/f12 [0,4194304] 0 2024-02-28T23:07:47.783 INFO:tasks.workunit.client.0.smithi057.stdout:0/811: chown d17/f18 481591204 1 2024-02-28T23:07:47.783 INFO:tasks.workunit.client.1.smithi084.stdout:1/896: symlink d12/d1b/d2e/d30/d50/da8/dd9/df0/l125 0 2024-02-28T23:07:47.784 INFO:tasks.workunit.client.0.smithi057.stdout:3/598: rename d3/d8/d27/l7c to d3/d8/d27/lbd 0 2024-02-28T23:07:47.785 INFO:tasks.workunit.client.1.smithi084.stdout:0/859: dread d1/d8/d69/d6b/f9b [0,4194304] 0 2024-02-28T23:07:47.786 INFO:tasks.workunit.client.1.smithi084.stdout:2/769: sync 2024-02-28T23:07:47.786 INFO:tasks.workunit.client.0.smithi057.stdout:7/704: dwrite d0/d5/d36/d4e/d15/d29/d71/fa8 [0,4194304] 0 2024-02-28T23:07:47.786 INFO:tasks.workunit.client.1.smithi084.stdout:6/818: creat d1/d88/dbf/f110 x:0 0 0 2024-02-28T23:07:47.789 INFO:tasks.workunit.client.1.smithi084.stdout:7/936: truncate db/dbd/ff7 310585 0 2024-02-28T23:07:47.790 INFO:tasks.workunit.client.1.smithi084.stdout:8/834: chown d0/d11/f23 345 1 2024-02-28T23:07:47.791 INFO:tasks.workunit.client.0.smithi057.stdout:2/790: write d8/d11/d21/d28/d41/f5c [790504,89793] 0 2024-02-28T23:07:47.793 INFO:tasks.workunit.client.1.smithi084.stdout:1/897: unlink d12/d1b/lbd 0 2024-02-28T23:07:47.793 INFO:tasks.workunit.client.0.smithi057.stdout:9/679: mknod dd/de/d24/d63/d77/dcf/d40/d44/d92/cdb 0 2024-02-28T23:07:47.795 INFO:tasks.workunit.client.0.smithi057.stdout:2/791: fdatasync d8/d11/d56/f9d 0 2024-02-28T23:07:47.795 INFO:tasks.workunit.client.1.smithi084.stdout:4/808: mkdir d1/d18/d20/d14/d109 0 2024-02-28T23:07:47.797 INFO:tasks.workunit.client.1.smithi084.stdout:9/896: write d4/d6/d15/d12/d41/fe5 [560737,87162] 0 2024-02-28T23:07:47.798 INFO:tasks.workunit.client.0.smithi057.stdout:5/701: dwrite d4/d9/d14/f5e [0,4194304] 0 2024-02-28T23:07:47.798 INFO:tasks.workunit.client.0.smithi057.stdout:1/615: getdents d2/d33 0 2024-02-28T23:07:47.798 INFO:tasks.workunit.client.0.smithi057.stdout:8/736: dwrite d4/d13/d18/da7/ff3 [0,4194304] 0 2024-02-28T23:07:47.800 INFO:tasks.workunit.client.1.smithi084.stdout:0/860: truncate d1/d8/f27 40201 0 2024-02-28T23:07:47.802 INFO:tasks.workunit.client.0.smithi057.stdout:9/680: chown dd/de/d24/cac 1292581 1 2024-02-28T23:07:47.802 INFO:tasks.workunit.client.1.smithi084.stdout:1/898: read d12/f23 [2670047,44227] 0 2024-02-28T23:07:47.803 INFO:tasks.workunit.client.0.smithi057.stdout:8/737: chown d4/l12 2678 1 2024-02-28T23:07:47.804 INFO:tasks.workunit.client.1.smithi084.stdout:2/770: creat d5/d7/d2e/d4a/d5c/fe7 x:0 0 0 2024-02-28T23:07:47.804 INFO:tasks.workunit.client.0.smithi057.stdout:6/659: sync 2024-02-28T23:07:47.808 INFO:tasks.workunit.client.1.smithi084.stdout:3/891: write d3/d50/d5a/fab [442317,65096] 0 2024-02-28T23:07:47.809 INFO:tasks.workunit.client.1.smithi084.stdout:9/897: creat d4/d69/f123 x:0 0 0 2024-02-28T23:07:47.813 INFO:tasks.workunit.client.1.smithi084.stdout:1/899: creat d12/da2/f126 x:0 0 0 2024-02-28T23:07:47.814 INFO:tasks.workunit.client.0.smithi057.stdout:5/702: read d4/d9/d14/f28 [9951369,13985] 0 2024-02-28T23:07:47.814 INFO:tasks.workunit.client.0.smithi057.stdout:3/599: dwrite d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:47.818 INFO:tasks.workunit.client.0.smithi057.stdout:3/600: chown d3/d8/d1b/f37 1 1 2024-02-28T23:07:47.819 INFO:tasks.workunit.client.1.smithi084.stdout:2/771: creat d5/d7/d12/d24/d2a/fe8 x:0 0 0 2024-02-28T23:07:47.820 INFO:tasks.workunit.client.1.smithi084.stdout:4/809: symlink d1/d18/d20/d24/d2f/l10a 0 2024-02-28T23:07:47.821 INFO:tasks.workunit.client.1.smithi084.stdout:3/892: rmdir d3 39 2024-02-28T23:07:47.822 INFO:tasks.workunit.client.0.smithi057.stdout:7/705: mknod d0/d5/d36/d4e/d15/ced 0 2024-02-28T23:07:47.823 INFO:tasks.workunit.client.1.smithi084.stdout:9/898: creat d4/d6/d15/d75/d9d/dd0/f124 x:0 0 0 2024-02-28T23:07:47.824 INFO:tasks.workunit.client.1.smithi084.stdout:6/819: link d1/d13/d2e/d41/fa9 d1/d13/d50/d74/d9e/d101/f111 0 2024-02-28T23:07:47.824 INFO:tasks.workunit.client.0.smithi057.stdout:0/812: getdents d17/d20/d23/d47 0 2024-02-28T23:07:47.826 INFO:tasks.workunit.client.1.smithi084.stdout:2/772: mkdir d5/d7/d75/de9 0 2024-02-28T23:07:47.828 INFO:tasks.workunit.client.1.smithi084.stdout:8/835: getdents d0/d9/d1b/d38/d43/d51/d54 0 2024-02-28T23:07:47.831 INFO:tasks.workunit.client.0.smithi057.stdout:8/738: readlink d4/de9/d56/d6a/d42/le3 0 2024-02-28T23:07:47.831 INFO:tasks.workunit.client.1.smithi084.stdout:6/820: unlink d1/d88/dbf/fc7 0 2024-02-28T23:07:47.832 INFO:tasks.workunit.client.1.smithi084.stdout:0/861: dwrite d1/d6/db/d35/d4a/d56/fc1 [0,4194304] 0 2024-02-28T23:07:47.836 INFO:tasks.workunit.client.1.smithi084.stdout:3/893: mknod d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/c14a 0 2024-02-28T23:07:47.838 INFO:tasks.workunit.client.0.smithi057.stdout:1/616: dwrite d2/d7/df/f4d [0,4194304] 0 2024-02-28T23:07:47.840 INFO:tasks.workunit.client.1.smithi084.stdout:4/810: rmdir d1/d18/d20/d24/d2f/d3e/d52/d48/dd7/de7 0 2024-02-28T23:07:47.844 INFO:tasks.workunit.client.0.smithi057.stdout:3/601: creat d3/d4/d1d/d47/d62/fbe x:0 0 0 2024-02-28T23:07:47.845 INFO:tasks.workunit.client.0.smithi057.stdout:5/703: creat d4/d9/d14/d63/d88/fd8 x:0 0 0 2024-02-28T23:07:47.849 INFO:tasks.workunit.client.0.smithi057.stdout:6/660: mkdir d0/d17/dde 0 2024-02-28T23:07:47.852 INFO:tasks.workunit.client.0.smithi057.stdout:6/661: chown d0/d1b/d3e/d3d/d42 133375411 1 2024-02-28T23:07:47.855 INFO:tasks.workunit.client.0.smithi057.stdout:7/706: fdatasync d0/d3f/d76/dc1/dea/d75/fa5 0 2024-02-28T23:07:47.856 INFO:tasks.workunit.client.1.smithi084.stdout:9/899: getdents d4/d6 0 2024-02-28T23:07:47.859 INFO:tasks.workunit.client.0.smithi057.stdout:7/707: write d0/fc0 [168375,51944] 0 2024-02-28T23:07:47.861 INFO:tasks.workunit.client.0.smithi057.stdout:8/739: rename d4/de9/d59/dc3 to d4/d13/df0/d104 0 2024-02-28T23:07:47.862 INFO:tasks.workunit.client.1.smithi084.stdout:8/836: link d0/d9/d30/d33/d71/f93 d0/d9/d1b/d38/d43/dae/d104/f114 0 2024-02-28T23:07:47.869 INFO:tasks.workunit.client.0.smithi057.stdout:9/681: creat dd/de/d14/d48/fdc x:0 0 0 2024-02-28T23:07:47.870 INFO:tasks.workunit.client.0.smithi057.stdout:6/662: rename d0/d17/d20/f84 to d0/d1b/d54/fdf 0 2024-02-28T23:07:47.871 INFO:tasks.workunit.client.0.smithi057.stdout:4/670: dwrite d1a/d48/d6c/f8b [4194304,4194304] 0 2024-02-28T23:07:47.871 INFO:tasks.workunit.client.1.smithi084.stdout:7/937: write db/d10/d17/d64/d77/fa8 [638534,105634] 0 2024-02-28T23:07:47.872 INFO:tasks.workunit.client.0.smithi057.stdout:7/708: symlink d0/d5/d25/d80/lee 0 2024-02-28T23:07:47.872 INFO:tasks.workunit.client.0.smithi057.stdout:2/792: link d8/d11/d39/l6e d8/d11/lf4 0 2024-02-28T23:07:47.876 INFO:tasks.workunit.client.0.smithi057.stdout:8/740: mknod d4/d13/c105 0 2024-02-28T23:07:47.877 INFO:tasks.workunit.client.1.smithi084.stdout:4/811: dwrite d1/d18/d20/d65/d8e/d55/dc3/dec/ff2 [0,4194304] 0 2024-02-28T23:07:47.877 INFO:tasks.workunit.client.1.smithi084.stdout:8/837: mknod d0/d24/d9e/c115 0 2024-02-28T23:07:47.880 INFO:tasks.workunit.client.1.smithi084.stdout:5/854: write dc/de/df/d16/d34/de8/f102 [1720828,76554] 0 2024-02-28T23:07:47.881 INFO:tasks.workunit.client.0.smithi057.stdout:6/663: creat d0/d1b/d3e/d3d/d42/fe0 x:0 0 0 2024-02-28T23:07:47.882 INFO:tasks.workunit.client.1.smithi084.stdout:5/855: readlink dc/d51/d7c/le7 0 2024-02-28T23:07:47.895 INFO:tasks.workunit.client.1.smithi084.stdout:4/812: mknod d1/d18/d20/d14/c10b 0 2024-02-28T23:07:47.899 INFO:tasks.workunit.client.1.smithi084.stdout:8/838: creat d0/d9/d1b/dc1/f116 x:0 0 0 2024-02-28T23:07:47.904 INFO:tasks.workunit.client.0.smithi057.stdout:4/671: dread d1a/d48/f71 [0,4194304] 0 2024-02-28T23:07:47.909 INFO:tasks.workunit.client.0.smithi057.stdout:4/672: write d1a/d48/db0/db1/fc6 [532538,300] 0 2024-02-28T23:07:47.910 INFO:tasks.workunit.client.0.smithi057.stdout:4/673: chown f0 0 1 2024-02-28T23:07:47.913 INFO:tasks.workunit.client.1.smithi084.stdout:6/821: link d1/d9/ddc/cec d1/d13/d50/dbd/c112 0 2024-02-28T23:07:47.914 INFO:tasks.workunit.client.1.smithi084.stdout:4/813: creat d1/d18/d20/d24/d2f/d3e/d52/d45/db5/f10c x:0 0 0 2024-02-28T23:07:47.923 INFO:tasks.workunit.client.1.smithi084.stdout:7/938: creat db/d10/d17/d9c/d9f/f127 x:0 0 0 2024-02-28T23:07:47.924 INFO:tasks.workunit.client.1.smithi084.stdout:9/900: write d4/d69/fb7 [411313,68560] 0 2024-02-28T23:07:47.929 INFO:tasks.workunit.client.1.smithi084.stdout:3/894: dwrite d3/d50/ff6 [0,4194304] 0 2024-02-28T23:07:47.933 INFO:tasks.workunit.client.1.smithi084.stdout:9/901: chown d4/d6/d15/d75/d2d/daa/c112 17706 1 2024-02-28T23:07:47.938 INFO:tasks.workunit.client.1.smithi084.stdout:9/902: readlink d4/d6/d15/d12/la0 0 2024-02-28T23:07:47.938 INFO:tasks.workunit.client.0.smithi057.stdout:3/602: dread d3/d8/f5e [0,4194304] 0 2024-02-28T23:07:47.944 INFO:tasks.workunit.client.0.smithi057.stdout:4/674: dread d1a/d48/d97/d7f/fa5 [4194304,4194304] 0 2024-02-28T23:07:47.945 INFO:tasks.workunit.client.1.smithi084.stdout:3/895: write d3/d50/d117/d3d/d43/de3/f149 [341404,68772] 0 2024-02-28T23:07:47.950 INFO:tasks.workunit.client.1.smithi084.stdout:0/862: dwrite d1/d6/db/d35/d57/d81/fc2 [0,4194304] 0 2024-02-28T23:07:47.960 INFO:tasks.workunit.client.1.smithi084.stdout:9/903: dwrite d4/d6/d15/d12/d41/d55/fbc [0,4194304] 0 2024-02-28T23:07:47.960 INFO:tasks.workunit.client.1.smithi084.stdout:8/839: write d0/d11/f99 [611492,98967] 0 2024-02-28T23:07:47.965 INFO:tasks.workunit.client.0.smithi057.stdout:6/664: fsync d0/d1b/f27 0 2024-02-28T23:07:47.966 INFO:tasks.workunit.client.0.smithi057.stdout:2/793: mkdir d8/d11/d21/d28/d49/d85/d92/df5 0 2024-02-28T23:07:47.971 INFO:tasks.workunit.client.1.smithi084.stdout:2/773: sync 2024-02-28T23:07:47.974 INFO:tasks.workunit.client.0.smithi057.stdout:0/813: dwrite d17/d20/d26/d30/f3f [0,4194304] 0 2024-02-28T23:07:47.974 INFO:tasks.workunit.client.1.smithi084.stdout:1/900: sync 2024-02-28T23:07:47.974 INFO:tasks.workunit.client.1.smithi084.stdout:5/856: dread dc/de/df/d16/d34/f53 [0,4194304] 0 2024-02-28T23:07:47.976 INFO:tasks.workunit.client.0.smithi057.stdout:0/814: truncate d17/d19/dd2/fe6 203343 0 2024-02-28T23:07:47.977 INFO:tasks.workunit.client.0.smithi057.stdout:4/675: rename d1a/d2f/db2/d49/d4d/de2/db5/dbd to d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9 0 2024-02-28T23:07:47.980 INFO:tasks.workunit.client.1.smithi084.stdout:1/901: dread d12/d1b/d2e/fe5 [0,4194304] 0 2024-02-28T23:07:47.984 INFO:tasks.workunit.client.1.smithi084.stdout:1/902: chown d12/d1b/d91/d9b/da5/dcb/dd4 12699 1 2024-02-28T23:07:47.985 INFO:tasks.workunit.client.1.smithi084.stdout:9/904: dwrite d4/d6/d15/d75/d2d/daa/fc6 [0,4194304] 0 2024-02-28T23:07:47.986 INFO:tasks.workunit.client.0.smithi057.stdout:2/794: symlink d8/d11/d39/d83/dde/d50/lf6 0 2024-02-28T23:07:47.988 INFO:tasks.workunit.client.0.smithi057.stdout:1/617: truncate d2/f44 1010027 0 2024-02-28T23:07:47.990 INFO:tasks.workunit.client.0.smithi057.stdout:0/815: dread d17/d19/d4d/f106 [0,4194304] 0 2024-02-28T23:07:47.993 INFO:tasks.workunit.client.0.smithi057.stdout:5/704: write d4/d9/d85/fce [1010062,64636] 0 2024-02-28T23:07:47.993 INFO:tasks.workunit.client.0.smithi057.stdout:2/795: dread d8/d11/d15/f95 [0,4194304] 0 2024-02-28T23:07:47.997 INFO:tasks.workunit.client.0.smithi057.stdout:4/676: fdatasync d1a/d2f/db2/d49/d4d/de2/f53 0 2024-02-28T23:07:47.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/905: truncate d4/d6/d15/d75/d9d/dc3/dce/f11f 687024 0 2024-02-28T23:07:47.998 INFO:tasks.workunit.client.1.smithi084.stdout:9/906: dread - d4/d6/d15/fd4 zero size 2024-02-28T23:07:47.998 INFO:tasks.workunit.client.1.smithi084.stdout:6/822: creat d1/d13/d47/d91/dba/de9/f113 x:0 0 0 2024-02-28T23:07:48.000 INFO:tasks.workunit.client.1.smithi084.stdout:6/823: write d1/d13/d47/f7a [977307,1421] 0 2024-02-28T23:07:48.001 INFO:tasks.workunit.client.1.smithi084.stdout:9/907: chown d4/d6/d15/d75/d2d/dbd/dde/f116 6656 1 2024-02-28T23:07:48.004 INFO:tasks.workunit.client.1.smithi084.stdout:8/840: unlink d0/d9/d1b/d38/d43/dae/db8/fc2 0 2024-02-28T23:07:48.007 INFO:tasks.workunit.client.0.smithi057.stdout:5/705: dread d4/d21/d24/f86 [0,4194304] 0 2024-02-28T23:07:48.010 INFO:tasks.workunit.client.0.smithi057.stdout:0/816: rename d17/d19/d4d/f62 to d17/d20/d26/d30/d6b/f122 0 2024-02-28T23:07:48.013 INFO:tasks.workunit.client.1.smithi084.stdout:4/814: creat d1/f10d x:0 0 0 2024-02-28T23:07:48.020 INFO:tasks.workunit.client.0.smithi057.stdout:6/665: link d0/d1b/d3e/d3d/d42/d4f/fac d0/fe1 0 2024-02-28T23:07:48.021 INFO:tasks.workunit.client.0.smithi057.stdout:4/677: symlink d1a/d1e/d51/dc3/lea 0 2024-02-28T23:07:48.021 INFO:tasks.workunit.client.1.smithi084.stdout:7/939: link db/d10/c91 db/d10/d17/d9c/d9f/db1/db2/c128 0 2024-02-28T23:07:48.021 INFO:tasks.workunit.client.1.smithi084.stdout:3/896: creat d3/d50/d5a/d73/f14b x:0 0 0 2024-02-28T23:07:48.022 INFO:tasks.workunit.client.1.smithi084.stdout:4/815: truncate d1/d18/d20/d65/d8e/f95 1484621 0 2024-02-28T23:07:48.026 INFO:tasks.workunit.client.1.smithi084.stdout:4/816: rmdir d1/d18/d20/d65/d8e/d55/dc3 39 2024-02-28T23:07:48.028 INFO:tasks.workunit.client.1.smithi084.stdout:3/897: mkdir d3/d50/d127/d14c 0 2024-02-28T23:07:48.028 INFO:tasks.workunit.client.0.smithi057.stdout:3/603: dread d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:48.029 INFO:tasks.workunit.client.1.smithi084.stdout:7/940: rename db/d5e/d87/dc1/ldc to db/d10/d17/d9c/d9f/d11b/l129 0 2024-02-28T23:07:48.032 INFO:tasks.workunit.client.0.smithi057.stdout:2/796: getdents d8/d11/d21/d70/d91 0 2024-02-28T23:07:48.034 INFO:tasks.workunit.client.0.smithi057.stdout:1/618: rmdir d2/d90/dc2 0 2024-02-28T23:07:48.035 INFO:tasks.workunit.client.1.smithi084.stdout:7/941: mkdir db/d10/d18/db7/de9/dfa/dc7/d100/d12a 0 2024-02-28T23:07:48.039 INFO:tasks.workunit.client.0.smithi057.stdout:0/817: dwrite d17/d20/d26/d48/d7c/d98/fe7 [0,4194304] 0 2024-02-28T23:07:48.045 INFO:tasks.workunit.client.1.smithi084.stdout:6/824: dread d1/d9/d1d/d1e/f5b [0,4194304] 0 2024-02-28T23:07:48.045 INFO:tasks.workunit.client.0.smithi057.stdout:2/797: write d8/d11/d39/d83/dde/d50/f93 [1722415,90709] 0 2024-02-28T23:07:48.046 INFO:tasks.workunit.client.0.smithi057.stdout:5/706: rename d4/d9/d14/d3b/l74 to d4/d9/d14/d9e/ld9 0 2024-02-28T23:07:48.048 INFO:tasks.workunit.client.1.smithi084.stdout:7/942: rename db/f10e to db/d10/d17/d9c/d9f/d106/f12b 0 2024-02-28T23:07:48.050 INFO:tasks.workunit.client.0.smithi057.stdout:2/798: chown d8/d11/d15/c2e 0 1 2024-02-28T23:07:48.050 INFO:tasks.workunit.client.1.smithi084.stdout:6/825: chown d1/d13/d47/d91/dba/fe3 308 1 2024-02-28T23:07:48.051 INFO:tasks.workunit.client.1.smithi084.stdout:7/943: readlink db/d5e/d87/dc1/lf1 0 2024-02-28T23:07:48.052 INFO:tasks.workunit.client.0.smithi057.stdout:5/707: chown d4/d9/d14/d3b/d61/d71/da1/lb1 24476 1 2024-02-28T23:07:48.052 INFO:tasks.workunit.client.0.smithi057.stdout:4/678: dwrite d1a/d48/d97/fc0 [0,4194304] 0 2024-02-28T23:07:48.053 INFO:tasks.workunit.client.0.smithi057.stdout:0/818: mkdir d17/d20/d4a/d78/dd4/d123 0 2024-02-28T23:07:48.053 INFO:tasks.workunit.client.1.smithi084.stdout:3/898: dread d3/d27/d7d/db0/d14/d31/d34/fcf [0,4194304] 0 2024-02-28T23:07:48.060 INFO:tasks.workunit.client.0.smithi057.stdout:1/619: dwrite d2/d7/df/db/d3a/f99 [0,4194304] 0 2024-02-28T23:07:48.064 INFO:tasks.workunit.client.0.smithi057.stdout:1/620: readlink d2/d7/df/db/d12/l59 0 2024-02-28T23:07:48.065 INFO:tasks.workunit.client.0.smithi057.stdout:4/679: creat d1a/d1e/d51/dc3/feb x:0 0 0 2024-02-28T23:07:48.068 INFO:tasks.workunit.client.0.smithi057.stdout:3/604: rename d3/d4/l13 to d3/d4/d1d/d47/lbf 0 2024-02-28T23:07:48.069 INFO:tasks.workunit.client.0.smithi057.stdout:1/621: write d2/d71/d2d/d5b/faf [2053983,41347] 0 2024-02-28T23:07:48.069 INFO:tasks.workunit.client.0.smithi057.stdout:7/709: sync 2024-02-28T23:07:48.071 INFO:tasks.workunit.client.1.smithi084.stdout:8/841: dread d0/d9/d30/d33/d71/f75 [0,4194304] 0 2024-02-28T23:07:48.074 INFO:tasks.workunit.client.1.smithi084.stdout:6/826: unlink d1/d13/d2e/feb 0 2024-02-28T23:07:48.074 INFO:tasks.workunit.client.0.smithi057.stdout:5/708: mknod d4/d9/d16/d52/d7a/cda 0 2024-02-28T23:07:48.075 INFO:tasks.workunit.client.1.smithi084.stdout:4/817: dread d1/f9 [0,4194304] 0 2024-02-28T23:07:48.075 INFO:tasks.workunit.client.0.smithi057.stdout:4/680: mkdir d1a/d2f/db2/d49/d4d/dc9/dec 0 2024-02-28T23:07:48.076 INFO:tasks.workunit.client.0.smithi057.stdout:5/709: readlink d4/d21/d24/d30/l8c 0 2024-02-28T23:07:48.078 INFO:tasks.workunit.client.1.smithi084.stdout:4/818: read d1/f12 [3482805,129433] 0 2024-02-28T23:07:48.080 INFO:tasks.workunit.client.0.smithi057.stdout:5/710: readlink d4/d21/d24/d30/lbe 0 2024-02-28T23:07:48.081 INFO:tasks.workunit.client.1.smithi084.stdout:3/899: dwrite d3/d50/d5a/d73/db8/fc1 [0,4194304] 0 2024-02-28T23:07:48.085 INFO:tasks.workunit.client.0.smithi057.stdout:1/622: symlink d2/d7/df/db/d12/d19/da4/dc5/ldd 0 2024-02-28T23:07:48.085 INFO:tasks.workunit.client.0.smithi057.stdout:5/711: symlink d4/d9/d14/d3b/d61/db2/ldb 0 2024-02-28T23:07:48.087 INFO:tasks.workunit.client.1.smithi084.stdout:6/827: unlink d1/d13/d2e/f44 0 2024-02-28T23:07:48.089 INFO:tasks.workunit.client.0.smithi057.stdout:9/682: truncate dd/fd7 3354211 0 2024-02-28T23:07:48.094 INFO:tasks.workunit.client.1.smithi084.stdout:3/900: symlink d3/d50/d117/d3d/d43/d61/dce/l14d 0 2024-02-28T23:07:48.098 INFO:tasks.workunit.client.0.smithi057.stdout:9/683: read dd/de/d14/d48/d4b/d41/fcb [2926608,59332] 0 2024-02-28T23:07:48.101 INFO:tasks.workunit.client.0.smithi057.stdout:8/741: dwrite d4/d13/fa8 [0,4194304] 0 2024-02-28T23:07:48.101 INFO:tasks.workunit.client.1.smithi084.stdout:3/901: creat d3/d50/d117/d3d/d43/d61/dce/d109/f14e x:0 0 0 2024-02-28T23:07:48.103 INFO:tasks.workunit.client.1.smithi084.stdout:4/819: link d1/d18/d20/d65/d8e/d55/dc3/fe1 d1/d18/d20/d65/d8e/d55/dc3/f10e 0 2024-02-28T23:07:48.103 INFO:tasks.workunit.client.0.smithi057.stdout:1/623: getdents d2/d7/df/db/d3a/d5a 0 2024-02-28T23:07:48.105 INFO:tasks.workunit.client.1.smithi084.stdout:0/863: write d1/d8/d18/f61 [1056223,112608] 0 2024-02-28T23:07:48.105 INFO:tasks.workunit.client.1.smithi084.stdout:3/902: readlink d3/d27/d7d/db0/d1a/d29/d41/l101 0 2024-02-28T23:07:48.107 INFO:tasks.workunit.client.0.smithi057.stdout:9/684: unlink dd/de/d4a/d9f/fb3 0 2024-02-28T23:07:48.109 INFO:tasks.workunit.client.1.smithi084.stdout:2/774: write d5/d7/d12/d24/fa7 [348102,94206] 0 2024-02-28T23:07:48.109 INFO:tasks.workunit.client.1.smithi084.stdout:4/820: write d1/d18/d20/d24/d2f/d3e/fe5 [1080953,48593] 0 2024-02-28T23:07:48.109 INFO:tasks.workunit.client.0.smithi057.stdout:2/799: dread d8/d11/d21/d28/d41/f9e [0,4194304] 0 2024-02-28T23:07:48.112 INFO:tasks.workunit.client.0.smithi057.stdout:5/712: dwrite d4/d9/d14/f28 [0,4194304] 0 2024-02-28T23:07:48.113 INFO:tasks.workunit.client.0.smithi057.stdout:8/742: symlink d4/d13/df0/l106 0 2024-02-28T23:07:48.114 INFO:tasks.workunit.client.1.smithi084.stdout:5/857: write dc/de/d83/dd9/ffb [583789,28613] 0 2024-02-28T23:07:48.117 INFO:tasks.workunit.client.1.smithi084.stdout:5/858: stat dc/fc8 0 2024-02-28T23:07:48.119 INFO:tasks.workunit.client.0.smithi057.stdout:1/624: symlink d2/d7/df/db/d3a/d8f/lde 0 2024-02-28T23:07:48.121 INFO:tasks.workunit.client.0.smithi057.stdout:9/685: creat dd/de/d4a/d60/fdd x:0 0 0 2024-02-28T23:07:48.124 INFO:tasks.workunit.client.1.smithi084.stdout:0/864: truncate d1/d8/d5f/d93/fe2 436375 0 2024-02-28T23:07:48.125 INFO:tasks.workunit.client.1.smithi084.stdout:5/859: fsync dc/d51/f12b 0 2024-02-28T23:07:48.125 INFO:tasks.workunit.client.0.smithi057.stdout:5/713: mkdir d4/d9/d14/d3b/d61/ddc 0 2024-02-28T23:07:48.125 INFO:tasks.workunit.client.0.smithi057.stdout:1/625: chown d2/d7/df/db/d3a/d2a/l75 1316839 1 2024-02-28T23:07:48.127 INFO:tasks.workunit.client.0.smithi057.stdout:2/800: creat d8/dae/de1/ff7 x:0 0 0 2024-02-28T23:07:48.127 INFO:tasks.workunit.client.1.smithi084.stdout:3/903: creat d3/d27/d7d/db0/d1a/f14f x:0 0 0 2024-02-28T23:07:48.129 INFO:tasks.workunit.client.0.smithi057.stdout:9/686: mkdir dd/de/d14/d48/d4b/d41/dde 0 2024-02-28T23:07:48.129 INFO:tasks.workunit.client.0.smithi057.stdout:5/714: dread - d4/d7/d58/f96 zero size 2024-02-28T23:07:48.131 INFO:tasks.workunit.client.1.smithi084.stdout:9/908: dwrite d4/d6/d15/d12/d41/f64 [0,4194304] 0 2024-02-28T23:07:48.133 INFO:tasks.workunit.client.1.smithi084.stdout:8/842: write d0/d9/d30/d33/f94 [1914733,47299] 0 2024-02-28T23:07:48.133 INFO:tasks.workunit.client.0.smithi057.stdout:5/715: truncate d4/d7/f73 1553700 0 2024-02-28T23:07:48.136 INFO:tasks.workunit.client.1.smithi084.stdout:0/865: creat d1/d6/db/d35/d82/f115 x:0 0 0 2024-02-28T23:07:48.137 INFO:tasks.workunit.client.1.smithi084.stdout:1/903: dwrite d12/d1b/d2e/d30/d77/f90 [0,4194304] 0 2024-02-28T23:07:48.138 INFO:tasks.workunit.client.1.smithi084.stdout:6/828: write d1/d13/d47/f56 [7176456,121158] 0 2024-02-28T23:07:48.139 INFO:tasks.workunit.client.1.smithi084.stdout:3/904: fsync d3/d50/d117/d7b/dbf/f114 0 2024-02-28T23:07:48.140 INFO:tasks.workunit.client.1.smithi084.stdout:8/843: chown d0/d9/f21 174 1 2024-02-28T23:07:48.144 INFO:tasks.workunit.client.0.smithi057.stdout:9/687: truncate dd/de/d14/d48/d4b/d41/f93 603002 0 2024-02-28T23:07:48.145 INFO:tasks.workunit.client.0.smithi057.stdout:9/688: readlink dd/de/d24/l36 0 2024-02-28T23:07:48.149 INFO:tasks.workunit.client.1.smithi084.stdout:7/944: dwrite db/d10/d17/fae [0,4194304] 0 2024-02-28T23:07:48.151 INFO:tasks.workunit.client.0.smithi057.stdout:9/689: stat dd/de/d24/d63/d77/dcf/d40/d44/lcc 0 2024-02-28T23:07:48.153 INFO:tasks.workunit.client.1.smithi084.stdout:0/866: truncate d1/d6/d2d/fc8 4289344 0 2024-02-28T23:07:48.154 INFO:tasks.workunit.client.1.smithi084.stdout:4/821: getdents d1/d18/d20/d65/d8e/d55 0 2024-02-28T23:07:48.159 INFO:tasks.workunit.client.0.smithi057.stdout:3/605: dread d3/d8/d27/f90 [0,4194304] 0 2024-02-28T23:07:48.161 INFO:tasks.workunit.client.1.smithi084.stdout:3/905: write d3/d50/d117/d3d/d43/d61/dce/d109/f14e [196346,126763] 0 2024-02-28T23:07:48.163 INFO:tasks.workunit.client.0.smithi057.stdout:6/666: write d0/d4/d25/f71 [4152620,114614] 0 2024-02-28T23:07:48.163 INFO:tasks.workunit.client.1.smithi084.stdout:7/945: mknod db/d10/d17/d9c/d9f/db1/c12c 0 2024-02-28T23:07:48.163 INFO:tasks.workunit.client.1.smithi084.stdout:0/867: creat d1/d6/d59/f116 x:0 0 0 2024-02-28T23:07:48.164 INFO:tasks.workunit.client.1.smithi084.stdout:1/904: mknod d12/d1b/d2e/d52/d11f/c127 0 2024-02-28T23:07:48.165 INFO:tasks.workunit.client.0.smithi057.stdout:4/681: rmdir d1a/d48/d97/d7f 39 2024-02-28T23:07:48.171 INFO:tasks.workunit.client.0.smithi057.stdout:9/690: truncate dd/de/d24/d63/d77/dcf/d40/d44/d5c/f8a 559513 0 2024-02-28T23:07:48.173 INFO:tasks.workunit.client.1.smithi084.stdout:2/775: truncate d5/d1f/d42/d74/f76 4039476 0 2024-02-28T23:07:48.176 INFO:tasks.workunit.client.1.smithi084.stdout:6/829: getdents d1/d9/d1d/db6 0 2024-02-28T23:07:48.176 INFO:tasks.workunit.client.0.smithi057.stdout:7/710: write d0/d3f/f7f [2764240,87585] 0 2024-02-28T23:07:48.177 INFO:tasks.workunit.client.1.smithi084.stdout:4/822: mkdir d1/d18/d20/d14/d30/d47/da4/d10f 0 2024-02-28T23:07:48.178 INFO:tasks.workunit.client.1.smithi084.stdout:7/946: rmdir db/d10/d17/d9c/d9f/db1/d104 39 2024-02-28T23:07:48.179 INFO:tasks.workunit.client.0.smithi057.stdout:7/711: write d0/fc0 [395608,74244] 0 2024-02-28T23:07:48.180 INFO:tasks.workunit.client.1.smithi084.stdout:0/868: creat d1/d6/db/d35/d4a/f117 x:0 0 0 2024-02-28T23:07:48.180 INFO:tasks.workunit.client.1.smithi084.stdout:1/905: creat d12/d1b/d91/d9b/da5/dcb/dd4/f128 x:0 0 0 2024-02-28T23:07:48.183 INFO:tasks.workunit.client.1.smithi084.stdout:9/909: dwrite d4/d6/d15/d75/d2d/f34 [0,4194304] 0 2024-02-28T23:07:48.184 INFO:tasks.workunit.client.1.smithi084.stdout:2/776: creat d5/d7/d12/fea x:0 0 0 2024-02-28T23:07:48.189 INFO:tasks.workunit.client.1.smithi084.stdout:1/906: chown d12/d1b/c6a 205895 1 2024-02-28T23:07:48.190 INFO:tasks.workunit.client.1.smithi084.stdout:6/830: symlink d1/d13/d47/l114 0 2024-02-28T23:07:48.192 INFO:tasks.workunit.client.0.smithi057.stdout:2/801: dread d8/d11/d21/d28/f82 [0,4194304] 0 2024-02-28T23:07:48.194 INFO:tasks.workunit.client.1.smithi084.stdout:0/869: creat d1/d6/d59/d70/f118 x:0 0 0 2024-02-28T23:07:48.196 INFO:tasks.workunit.client.0.smithi057.stdout:2/802: write d8/d11/d39/d83/fd1 [570406,29287] 0 2024-02-28T23:07:48.197 INFO:tasks.workunit.client.0.smithi057.stdout:7/712: dwrite d0/d3f/d76/dc1/dea/fbe [0,4194304] 0 2024-02-28T23:07:48.197 INFO:tasks.workunit.client.1.smithi084.stdout:4/823: creat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/f110 x:0 0 0 2024-02-28T23:07:48.197 INFO:tasks.workunit.client.1.smithi084.stdout:3/906: dread d3/d27/d7d/fdd [0,4194304] 0 2024-02-28T23:07:48.198 INFO:tasks.workunit.client.1.smithi084.stdout:2/777: rename d5/d7/d1d/f4d to d5/d7/d12/d24/feb 0 2024-02-28T23:07:48.199 INFO:tasks.workunit.client.1.smithi084.stdout:8/844: dwrite d0/d9/d30/d33/d71/f75 [0,4194304] 0 2024-02-28T23:07:48.199 INFO:tasks.workunit.client.1.smithi084.stdout:5/860: dread dc/de/df/da6/f12c [0,4194304] 0 2024-02-28T23:07:48.200 INFO:tasks.workunit.client.1.smithi084.stdout:4/824: chown d1/d18/d20/d24/ddb 21 1 2024-02-28T23:07:48.203 INFO:tasks.workunit.client.1.smithi084.stdout:9/910: creat d4/d6/d15/d75/d9d/dd0/f125 x:0 0 0 2024-02-28T23:07:48.203 INFO:tasks.workunit.client.1.smithi084.stdout:7/947: creat db/d10/d17/d9c/d9f/db1/d104/f12d x:0 0 0 2024-02-28T23:07:48.203 INFO:tasks.workunit.client.0.smithi057.stdout:8/743: unlink d4/c1c 0 2024-02-28T23:07:48.204 INFO:tasks.workunit.client.1.smithi084.stdout:1/907: write d12/d1b/d2e/d52/d5b/ffa [37097,72952] 0 2024-02-28T23:07:48.211 INFO:tasks.workunit.client.1.smithi084.stdout:8/845: fsync d0/d11/f2a 0 2024-02-28T23:07:48.212 INFO:tasks.workunit.client.1.smithi084.stdout:2/778: creat d5/d7/d75/db4/fec x:0 0 0 2024-02-28T23:07:48.213 INFO:tasks.workunit.client.1.smithi084.stdout:7/948: creat db/d10/d18/db7/de9/f12e x:0 0 0 2024-02-28T23:07:48.214 INFO:tasks.workunit.client.1.smithi084.stdout:9/911: creat d4/d6/f126 x:0 0 0 2024-02-28T23:07:48.220 INFO:tasks.workunit.client.1.smithi084.stdout:5/861: mknod dc/de/df/d31/dd1/d5f/d66/c141 0 2024-02-28T23:07:48.220 INFO:tasks.workunit.client.0.smithi057.stdout:5/716: mkdir d4/d9/ddd 0 2024-02-28T23:07:48.221 INFO:tasks.workunit.client.0.smithi057.stdout:5/717: dread - d4/d9/d14/f5f zero size 2024-02-28T23:07:48.229 INFO:tasks.workunit.client.0.smithi057.stdout:0/819: link d17/d20/d23/d47/d60/dea/f100 d17/d20/f124 0 2024-02-28T23:07:48.229 INFO:tasks.workunit.client.0.smithi057.stdout:6/667: mknod d0/d1b/d3e/d3d/d42/d4f/d92/ce2 0 2024-02-28T23:07:48.229 INFO:tasks.workunit.client.1.smithi084.stdout:4/825: rename d1/d18/d20/d14/f36 to d1/d18/d20/d24/d2f/d3e/d52/d48/dd7/f111 0 2024-02-28T23:07:48.229 INFO:tasks.workunit.client.1.smithi084.stdout:1/908: link le d12/d1b/d91/l129 0 2024-02-28T23:07:48.229 INFO:tasks.workunit.client.1.smithi084.stdout:5/862: read dc/de/df/d31/d8e/da9/fcc [3588755,74662] 0 2024-02-28T23:07:48.230 INFO:tasks.workunit.client.0.smithi057.stdout:9/691: symlink dd/de/d24/d63/d77/dcf/d40/d44/d92/ldf 0 2024-02-28T23:07:48.230 INFO:tasks.workunit.client.0.smithi057.stdout:7/713: creat d0/d98/fef x:0 0 0 2024-02-28T23:07:48.230 INFO:tasks.workunit.client.1.smithi084.stdout:4/826: write d1/f10d [552976,77701] 0 2024-02-28T23:07:48.231 INFO:tasks.workunit.client.0.smithi057.stdout:2/803: truncate d8/d11/d15/f78 600603 0 2024-02-28T23:07:48.232 INFO:tasks.workunit.client.1.smithi084.stdout:8/846: read - d0/d9/d1b/d38/d43/dae/db8/fd8 zero size 2024-02-28T23:07:48.233 INFO:tasks.workunit.client.1.smithi084.stdout:7/949: mknod db/d10/d17/d9c/c12f 0 2024-02-28T23:07:48.235 INFO:tasks.workunit.client.0.smithi057.stdout:5/718: rename d4/d21/f35 to d4/d7/d8/fde 0 2024-02-28T23:07:48.236 INFO:tasks.workunit.client.0.smithi057.stdout:2/804: fsync d8/d11/d21/d28/d41/fd9 0 2024-02-28T23:07:48.237 INFO:tasks.workunit.client.1.smithi084.stdout:8/847: truncate d0/f10b 1033056 0 2024-02-28T23:07:48.238 INFO:tasks.workunit.client.1.smithi084.stdout:3/907: link d3/d27/d7d/db0/d1a/d45/dde/le8 d3/d27/d7d/db0/d1a/l150 0 2024-02-28T23:07:48.238 INFO:tasks.workunit.client.1.smithi084.stdout:1/909: read d12/d1b/d2e/d52/d61/d74/ff7 [77789,69306] 0 2024-02-28T23:07:48.238 INFO:tasks.workunit.client.0.smithi057.stdout:0/820: mkdir d17/d20/d23/d47/d60/d125 0 2024-02-28T23:07:48.239 INFO:tasks.workunit.client.0.smithi057.stdout:8/744: mknod d4/d13/c107 0 2024-02-28T23:07:48.241 INFO:tasks.workunit.client.0.smithi057.stdout:0/821: chown d17/d20/d23/d4b 0 1 2024-02-28T23:07:48.242 INFO:tasks.workunit.client.0.smithi057.stdout:7/714: creat d0/d5/d36/d4e/d15/d29/d71/ff0 x:0 0 0 2024-02-28T23:07:48.242 INFO:tasks.workunit.client.0.smithi057.stdout:6/668: mknod d0/d1b/d3e/ce3 0 2024-02-28T23:07:48.242 INFO:tasks.workunit.client.1.smithi084.stdout:5/863: truncate dc/de/df/d31/dd1/f80 5012353 0 2024-02-28T23:07:48.242 INFO:tasks.workunit.client.1.smithi084.stdout:3/908: readlink d3/d27/d7d/db0/d14/lf1 0 2024-02-28T23:07:48.243 INFO:tasks.workunit.client.1.smithi084.stdout:1/910: chown d12/d1b/d91/d9b/da5 9 1 2024-02-28T23:07:48.243 INFO:tasks.workunit.client.0.smithi057.stdout:4/682: dread d1a/d1e/fe6 [0,4194304] 0 2024-02-28T23:07:48.244 INFO:tasks.workunit.client.1.smithi084.stdout:7/950: mkdir db/d5e/d8a/d130 0 2024-02-28T23:07:48.244 INFO:tasks.workunit.client.1.smithi084.stdout:2/779: dwrite d5/d7/d2e/d4a/d5c/f89 [4194304,4194304] 0 2024-02-28T23:07:48.244 INFO:tasks.workunit.client.0.smithi057.stdout:5/719: symlink d4/d9/d16/d2c/ldf 0 2024-02-28T23:07:48.246 INFO:tasks.workunit.client.0.smithi057.stdout:2/805: symlink d8/d11/d21/d28/lf8 0 2024-02-28T23:07:48.247 INFO:tasks.workunit.client.1.smithi084.stdout:6/831: dread d1/d9/fd5 [0,4194304] 0 2024-02-28T23:07:48.247 INFO:tasks.workunit.client.1.smithi084.stdout:7/951: chown db/d10/d18/db7/de9/dfa/f4f 702 1 2024-02-28T23:07:48.248 INFO:tasks.workunit.client.0.smithi057.stdout:4/683: chown d1a/d1e/d81/le1 19587 1 2024-02-28T23:07:48.251 INFO:tasks.workunit.client.0.smithi057.stdout:6/669: unlink d0/d4/d11/c59 0 2024-02-28T23:07:48.254 INFO:tasks.workunit.client.1.smithi084.stdout:8/848: rmdir d0/d9/d1b/d38/d43/dae 39 2024-02-28T23:07:48.255 INFO:tasks.workunit.client.0.smithi057.stdout:8/745: rename d4/le to d4/d13/d18/da7/de1/l108 0 2024-02-28T23:07:48.257 INFO:tasks.workunit.client.0.smithi057.stdout:9/692: dwrite dd/de/d4a/d60/fdd [0,4194304] 0 2024-02-28T23:07:48.257 INFO:tasks.workunit.client.1.smithi084.stdout:8/849: stat d0/d9/d30/f101 0 2024-02-28T23:07:48.257 INFO:tasks.workunit.client.1.smithi084.stdout:4/827: mknod d1/d18/d20/d24/d2f/d3e/d52/d57/c112 0 2024-02-28T23:07:48.259 INFO:tasks.workunit.client.0.smithi057.stdout:4/684: unlink d1a/fe4 0 2024-02-28T23:07:48.264 INFO:tasks.workunit.client.0.smithi057.stdout:7/715: truncate d0/f7e 3999303 0 2024-02-28T23:07:48.264 INFO:tasks.workunit.client.1.smithi084.stdout:3/909: unlink d3/d27/d7d/db0/d1a/d45/d5e/d7a/f9b 0 2024-02-28T23:07:48.265 INFO:tasks.workunit.client.1.smithi084.stdout:5/864: creat dc/de/df/d16/d34/daa/dae/f142 x:0 0 0 2024-02-28T23:07:48.265 INFO:tasks.workunit.client.0.smithi057.stdout:7/716: stat d0/d5/d25/c65 0 2024-02-28T23:07:48.265 INFO:tasks.workunit.client.0.smithi057.stdout:6/670: symlink d0/d1b/d3e/d3d/d42/d4f/d53/le4 0 2024-02-28T23:07:48.265 INFO:tasks.workunit.client.1.smithi084.stdout:1/911: truncate d12/d1b/d3b/f44 3749301 0 2024-02-28T23:07:48.269 INFO:tasks.workunit.client.0.smithi057.stdout:2/806: rename d8/d11/d39/f3c to d8/d11/db3/de9/ff9 0 2024-02-28T23:07:48.270 INFO:tasks.workunit.client.1.smithi084.stdout:0/870: dread d1/d8/d69/d6b/f7a [0,4194304] 0 2024-02-28T23:07:48.270 INFO:tasks.workunit.client.1.smithi084.stdout:4/828: truncate d1/f6 1713309 0 2024-02-28T23:07:48.273 INFO:tasks.workunit.client.1.smithi084.stdout:0/871: dread - d1/d8/d69/db2/f110 zero size 2024-02-28T23:07:48.274 INFO:tasks.workunit.client.0.smithi057.stdout:7/717: fsync d0/d5/d25/f28 0 2024-02-28T23:07:48.278 INFO:tasks.workunit.client.0.smithi057.stdout:7/718: chown d0/d5/d36/d4e/d15/da0/fbd 7736 1 2024-02-28T23:07:48.279 INFO:tasks.workunit.client.0.smithi057.stdout:5/720: link d4/d9/d14/d63/f7b d4/d7/d8/fe0 0 2024-02-28T23:07:48.282 INFO:tasks.workunit.client.1.smithi084.stdout:8/850: symlink d0/d9/d1b/d38/d43/dae/db8/l117 0 2024-02-28T23:07:48.283 INFO:tasks.workunit.client.1.smithi084.stdout:7/952: creat db/d10/d17/d9c/d9f/f131 x:0 0 0 2024-02-28T23:07:48.283 INFO:tasks.workunit.client.1.smithi084.stdout:4/829: chown d1/d18/d20/d14/d30/fe9 1637074 1 2024-02-28T23:07:48.284 INFO:tasks.workunit.client.0.smithi057.stdout:4/685: rename d1a/d2f/db2/c9a to d1a/d2f/db2/dcb/ced 0 2024-02-28T23:07:48.286 INFO:tasks.workunit.client.0.smithi057.stdout:8/746: rmdir d4/de9/d56/dc9/df5 0 2024-02-28T23:07:48.287 INFO:tasks.workunit.client.1.smithi084.stdout:3/910: creat d3/d27/d7d/db0/d1a/d29/d41/d60/f151 x:0 0 0 2024-02-28T23:07:48.288 INFO:tasks.workunit.client.0.smithi057.stdout:2/807: dwrite d8/d11/d21/d28/d49/d85/d92/fba [0,4194304] 0 2024-02-28T23:07:48.288 INFO:tasks.workunit.client.1.smithi084.stdout:9/912: rmdir d4/d6 39 2024-02-28T23:07:48.290 INFO:tasks.workunit.client.1.smithi084.stdout:0/872: mkdir d1/d6/db/d76/dee/df2/d119 0 2024-02-28T23:07:48.291 INFO:tasks.workunit.client.0.smithi057.stdout:7/719: mknod d0/d5/d36/d4e/d15/d29/d71/cf1 0 2024-02-28T23:07:48.293 INFO:tasks.workunit.client.1.smithi084.stdout:5/865: dwrite dc/d51/d7c/fc3 [4194304,4194304] 0 2024-02-28T23:07:48.293 INFO:tasks.workunit.client.1.smithi084.stdout:4/830: write d1/d18/d20/d24/d2f/d3e/d52/f9a [3961126,74066] 0 2024-02-28T23:07:48.293 INFO:tasks.workunit.client.1.smithi084.stdout:8/851: mknod d0/d24/d78/c118 0 2024-02-28T23:07:48.293 INFO:tasks.workunit.client.0.smithi057.stdout:8/747: rmdir d4/de9/d7c 39 2024-02-28T23:07:48.294 INFO:tasks.workunit.client.0.smithi057.stdout:7/720: chown d0/d3f/f7f 226080 1 2024-02-28T23:07:48.295 INFO:tasks.workunit.client.0.smithi057.stdout:4/686: symlink d1a/d2f/db2/d49/d4d/dc9/lee 0 2024-02-28T23:07:48.298 INFO:tasks.workunit.client.0.smithi057.stdout:7/721: write d0/d98/fa2 [4803714,34981] 0 2024-02-28T23:07:48.300 INFO:tasks.workunit.client.0.smithi057.stdout:1/626: write d2/f44 [1081252,121915] 0 2024-02-28T23:07:48.303 INFO:tasks.workunit.client.1.smithi084.stdout:3/911: creat d3/d50/d117/d134/f152 x:0 0 0 2024-02-28T23:07:48.307 INFO:tasks.workunit.client.1.smithi084.stdout:5/866: creat dc/de/df/d31/dd1/f143 x:0 0 0 2024-02-28T23:07:48.309 INFO:tasks.workunit.client.1.smithi084.stdout:3/912: chown d3/d50/d117/d3d/d43/d61/dce 1044283 1 2024-02-28T23:07:48.311 INFO:tasks.workunit.client.1.smithi084.stdout:9/913: symlink d4/d6/d15/db3/l127 0 2024-02-28T23:07:48.311 INFO:tasks.workunit.client.0.smithi057.stdout:2/808: link d8/d11/d15/f71 d8/d11/ffa 0 2024-02-28T23:07:48.314 INFO:tasks.workunit.client.1.smithi084.stdout:6/832: dread d1/d13/d2e/d41/f37 [0,4194304] 0 2024-02-28T23:07:48.317 INFO:tasks.workunit.client.0.smithi057.stdout:1/627: symlink d2/d7/df/db/d12/dd6/ldf 0 2024-02-28T23:07:48.318 INFO:tasks.workunit.client.0.smithi057.stdout:2/809: dread d8/d11/d39/f87 [0,4194304] 0 2024-02-28T23:07:48.320 INFO:tasks.workunit.client.0.smithi057.stdout:2/810: chown d8/d11/d39 25 1 2024-02-28T23:07:48.320 INFO:tasks.workunit.client.1.smithi084.stdout:5/867: mkdir dc/de/d144 0 2024-02-28T23:07:48.323 INFO:tasks.workunit.client.1.smithi084.stdout:3/913: mkdir d3/d27/d153 0 2024-02-28T23:07:48.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/722: fsync d0/f7e 0 2024-02-28T23:07:48.325 INFO:tasks.workunit.client.1.smithi084.stdout:9/914: mknod d4/d6/d15/d75/d9d/dc3/c128 0 2024-02-28T23:07:48.325 INFO:tasks.workunit.client.0.smithi057.stdout:2/811: read d8/d11/d39/d83/fce [225056,33336] 0 2024-02-28T23:07:48.326 INFO:tasks.workunit.client.1.smithi084.stdout:8/852: getdents d0/d9/d30/d33/d71/d84/dd9/de3/df7 0 2024-02-28T23:07:48.329 INFO:tasks.workunit.client.1.smithi084.stdout:3/914: readlink d3/d27/d7d/db0/d1a/le9 0 2024-02-28T23:07:48.330 INFO:tasks.workunit.client.0.smithi057.stdout:0/822: dwrite d17/d20/d4a/d78/fc7 [0,4194304] 0 2024-02-28T23:07:48.331 INFO:tasks.workunit.client.0.smithi057.stdout:1/628: creat d2/d71/d62/fe0 x:0 0 0 2024-02-28T23:07:48.331 INFO:tasks.workunit.client.0.smithi057.stdout:6/671: write d0/fe1 [1653761,45670] 0 2024-02-28T23:07:48.333 INFO:tasks.workunit.client.1.smithi084.stdout:5/868: creat dc/de/df/d16/d34/daa/dae/f145 x:0 0 0 2024-02-28T23:07:48.333 INFO:tasks.workunit.client.1.smithi084.stdout:1/912: write d12/d1b/d3b/f44 [71307,8186] 0 2024-02-28T23:07:48.336 INFO:tasks.workunit.client.1.smithi084.stdout:9/915: unlink d4/d6/d15/d75/d2d/daa/fc6 0 2024-02-28T23:07:48.336 INFO:tasks.workunit.client.1.smithi084.stdout:3/915: write d3/fbd [8304608,45491] 0 2024-02-28T23:07:48.337 INFO:tasks.workunit.client.1.smithi084.stdout:6/833: symlink d1/d13/d2e/d41/d4d/d6c/df6/l115 0 2024-02-28T23:07:48.340 INFO:tasks.workunit.client.0.smithi057.stdout:7/723: unlink d0/d5/d36/d4e/lc 0 2024-02-28T23:07:48.341 INFO:tasks.workunit.client.1.smithi084.stdout:3/916: chown d3/d50/d117/d3d/f67 40224 1 2024-02-28T23:07:48.342 INFO:tasks.workunit.client.1.smithi084.stdout:1/913: chown d12/d1b/d2e/d52/d5b/f68 304247436 1 2024-02-28T23:07:48.344 INFO:tasks.workunit.client.0.smithi057.stdout:5/721: truncate d4/d9/d14/f5e 304089 0 2024-02-28T23:07:48.346 INFO:tasks.workunit.client.0.smithi057.stdout:0/823: creat d17/d20/d26/d48/d55/f126 x:0 0 0 2024-02-28T23:07:48.347 INFO:tasks.workunit.client.1.smithi084.stdout:0/873: dread d1/d6/db/d35/d44/f79 [0,4194304] 0 2024-02-28T23:07:48.348 INFO:tasks.workunit.client.1.smithi084.stdout:5/869: mkdir dc/de/df/d16/d34/de8/d146 0 2024-02-28T23:07:48.350 INFO:tasks.workunit.client.0.smithi057.stdout:7/724: symlink d0/d5/d36/lf2 0 2024-02-28T23:07:48.351 INFO:tasks.workunit.client.1.smithi084.stdout:9/916: symlink d4/d6/d15/d75/d2d/l129 0 2024-02-28T23:07:48.352 INFO:tasks.workunit.client.0.smithi057.stdout:1/629: dwrite d2/d71/d62/fcf [0,4194304] 0 2024-02-28T23:07:48.354 INFO:tasks.workunit.client.1.smithi084.stdout:9/917: chown d4/d6/d15/d75/d2d 150 1 2024-02-28T23:07:48.354 INFO:tasks.workunit.client.1.smithi084.stdout:3/917: dread d3/f7 [0,4194304] 0 2024-02-28T23:07:48.355 INFO:tasks.workunit.client.1.smithi084.stdout:5/870: mkdir dc/de/df/d16/d34/d3c/d77/df8/d147 0 2024-02-28T23:07:48.356 INFO:tasks.workunit.client.1.smithi084.stdout:0/874: truncate d1/d6/db/d35/d82/f83 1565020 0 2024-02-28T23:07:48.358 INFO:tasks.workunit.client.0.smithi057.stdout:5/722: truncate d4/d21/d24/da2/dbb/d95/f98 730431 0 2024-02-28T23:07:48.361 INFO:tasks.workunit.client.1.smithi084.stdout:5/871: stat dc/de/df/d6e/faf 0 2024-02-28T23:07:48.362 INFO:tasks.workunit.client.1.smithi084.stdout:9/918: creat d4/d32/d3f/f12a x:0 0 0 2024-02-28T23:07:48.362 INFO:tasks.workunit.client.0.smithi057.stdout:0/824: symlink d17/d20/d26/d48/l127 0 2024-02-28T23:07:48.362 INFO:tasks.workunit.client.0.smithi057.stdout:8/748: dread d4/de9/f2e [0,4194304] 0 2024-02-28T23:07:48.363 INFO:tasks.workunit.client.0.smithi057.stdout:1/630: truncate d2/d33/f76 4218636 0 2024-02-28T23:07:48.364 INFO:tasks.workunit.client.0.smithi057.stdout:0/825: chown d17/d20/d23/f2c 598 1 2024-02-28T23:07:48.365 INFO:tasks.workunit.client.1.smithi084.stdout:5/872: chown dc/de/df/d31/dd1/d5f/d66 2 1 2024-02-28T23:07:48.366 INFO:tasks.workunit.client.1.smithi084.stdout:0/875: creat d1/d6/db/d35/d57/d81/f11a x:0 0 0 2024-02-28T23:07:48.366 INFO:tasks.workunit.client.0.smithi057.stdout:2/812: write d8/d11/d21/d70/fbd [890088,85962] 0 2024-02-28T23:07:48.367 INFO:tasks.workunit.client.1.smithi084.stdout:5/873: readlink dc/d51/d7c/l92 0 2024-02-28T23:07:48.367 INFO:tasks.workunit.client.0.smithi057.stdout:8/749: rmdir d4/d13/df0/d104 39 2024-02-28T23:07:48.369 INFO:tasks.workunit.client.1.smithi084.stdout:8/853: write d0/d9/d1b/d38/d43/f96 [960496,118878] 0 2024-02-28T23:07:48.371 INFO:tasks.workunit.client.1.smithi084.stdout:0/876: fdatasync d1/d8/f29 0 2024-02-28T23:07:48.372 INFO:tasks.workunit.client.0.smithi057.stdout:2/813: symlink d8/d11/db3/lfb 0 2024-02-28T23:07:48.372 INFO:tasks.workunit.client.0.smithi057.stdout:6/672: truncate d0/fe1 549168 0 2024-02-28T23:07:48.374 INFO:tasks.workunit.client.1.smithi084.stdout:5/874: mknod dc/de/df/d16/d34/d3c/d77/dd2/c148 0 2024-02-28T23:07:48.374 INFO:tasks.workunit.client.0.smithi057.stdout:4/687: dread d1a/d48/d97/d7f/fdf [0,4194304] 0 2024-02-28T23:07:48.375 INFO:tasks.workunit.client.0.smithi057.stdout:7/725: rmdir d0/d5/d36 39 2024-02-28T23:07:48.376 INFO:tasks.workunit.client.1.smithi084.stdout:8/854: chown d0/d24/d4d/fcb 923 1 2024-02-28T23:07:48.378 INFO:tasks.workunit.client.1.smithi084.stdout:6/834: truncate d1/d13/d2e/d41/d4d/d6c/f7c 393904 0 2024-02-28T23:07:48.380 INFO:tasks.workunit.client.1.smithi084.stdout:3/918: write d3/d50/d117/d3d/d43/f6c [4793892,130831] 0 2024-02-28T23:07:48.382 INFO:tasks.workunit.client.0.smithi057.stdout:5/723: write d4/d9/d14/f5f [28875,64821] 0 2024-02-28T23:07:48.382 INFO:tasks.workunit.client.1.smithi084.stdout:5/875: dread - dc/de/df/db0/d104/f10d zero size 2024-02-28T23:07:48.383 INFO:tasks.workunit.client.0.smithi057.stdout:7/726: mkdir d0/df3 0 2024-02-28T23:07:48.384 INFO:tasks.workunit.client.1.smithi084.stdout:6/835: stat d1/d13/d47/d91/dba/de9/f113 0 2024-02-28T23:07:48.387 INFO:tasks.workunit.client.1.smithi084.stdout:0/877: link d1/d8/d69/d6b/d85/fab d1/f11b 0 2024-02-28T23:07:48.388 INFO:tasks.workunit.client.1.smithi084.stdout:6/836: dread - d1/d13/d50/dbd/df9/f100 zero size 2024-02-28T23:07:48.391 INFO:tasks.workunit.client.0.smithi057.stdout:8/750: read d4/de9/d7c/ff9 [58176,15702] 0 2024-02-28T23:07:48.391 INFO:tasks.workunit.client.1.smithi084.stdout:6/837: chown d1/d88/dbf/lcc 12829 1 2024-02-28T23:07:48.393 INFO:tasks.workunit.client.1.smithi084.stdout:0/878: read - d1/d6/db/d35/d82/f115 zero size 2024-02-28T23:07:48.393 INFO:tasks.workunit.client.0.smithi057.stdout:1/631: link d2/d7/cd d2/d7/df/db/d12/d19/ce1 0 2024-02-28T23:07:48.394 INFO:tasks.workunit.client.1.smithi084.stdout:5/876: creat dc/de/df/db0/f149 x:0 0 0 2024-02-28T23:07:48.397 INFO:tasks.workunit.client.1.smithi084.stdout:9/919: dwrite d4/d6/d15/d75/d9d/f11b [0,4194304] 0 2024-02-28T23:07:48.397 INFO:tasks.workunit.client.0.smithi057.stdout:0/826: dwrite d17/d19/fe1 [0,4194304] 0 2024-02-28T23:07:48.399 INFO:tasks.workunit.client.0.smithi057.stdout:1/632: write d2/d7/df/db/d12/d3f/d47/f8e [923820,97122] 0 2024-02-28T23:07:48.400 INFO:tasks.workunit.client.0.smithi057.stdout:5/724: rename d4/d9/d14/f28 to d4/d9/d16/d52/fe1 0 2024-02-28T23:07:48.401 INFO:tasks.workunit.client.1.smithi084.stdout:6/838: truncate d1/d13/d2e/d41/d4d/f63 906487 0 2024-02-28T23:07:48.403 INFO:tasks.workunit.client.1.smithi084.stdout:9/920: write d4/d6/d15/d75/d2d/fc4 [861357,26008] 0 2024-02-28T23:07:48.404 INFO:tasks.workunit.client.1.smithi084.stdout:6/839: fdatasync d1/d13/d50/fdf 0 2024-02-28T23:07:48.407 INFO:tasks.workunit.client.1.smithi084.stdout:5/877: creat dc/de/df/d31/d8e/da9/dab/f14a x:0 0 0 2024-02-28T23:07:48.409 INFO:tasks.workunit.client.1.smithi084.stdout:8/855: write d0/d9/d30/d33/d71/f93 [577494,96703] 0 2024-02-28T23:07:48.411 INFO:tasks.workunit.client.0.smithi057.stdout:0/827: mknod d17/d20/d4a/d78/d9d/c128 0 2024-02-28T23:07:48.412 INFO:tasks.workunit.client.0.smithi057.stdout:5/725: fdatasync d4/d32/f6e 0 2024-02-28T23:07:48.413 INFO:tasks.workunit.client.1.smithi084.stdout:6/840: read d1/d13/d50/fca [499145,34430] 0 2024-02-28T23:07:48.414 INFO:tasks.workunit.client.0.smithi057.stdout:1/633: creat d2/d33/fe2 x:0 0 0 2024-02-28T23:07:48.418 INFO:tasks.workunit.client.1.smithi084.stdout:5/878: rmdir dc/de/df/d31 39 2024-02-28T23:07:48.419 INFO:tasks.workunit.client.1.smithi084.stdout:0/879: link d1/d6/db/d35/d82/le9 d1/d6/db/d35/d57/l11c 0 2024-02-28T23:07:48.420 INFO:tasks.workunit.client.0.smithi057.stdout:7/727: truncate d0/d5/d36/d4e/d15/d29/d71/ff0 898340 0 2024-02-28T23:07:48.421 INFO:tasks.workunit.client.1.smithi084.stdout:6/841: stat d1/d13/d2e/d41/dbb/lc1 0 2024-02-28T23:07:48.422 INFO:tasks.workunit.client.1.smithi084.stdout:0/880: dread - d1/d8/d18/d4c/f108 zero size 2024-02-28T23:07:48.423 INFO:tasks.workunit.client.0.smithi057.stdout:4/688: dwrite d1a/d1e/d51/f90 [0,4194304] 0 2024-02-28T23:07:48.424 INFO:tasks.workunit.client.0.smithi057.stdout:2/814: dwrite d8/dae/fb8 [0,4194304] 0 2024-02-28T23:07:48.425 INFO:tasks.workunit.client.1.smithi084.stdout:9/921: rename d4/d6/c78 to d4/d6/d15/d12/d41/c12b 0 2024-02-28T23:07:48.428 INFO:tasks.workunit.client.1.smithi084.stdout:9/922: chown d4/d6/d15/d12/f9e 13197282 1 2024-02-28T23:07:48.429 INFO:tasks.workunit.client.1.smithi084.stdout:3/919: dwrite d3/dae/fea [0,4194304] 0 2024-02-28T23:07:48.429 INFO:tasks.workunit.client.1.smithi084.stdout:0/881: dread - d1/d6/db/d37/d72/ff5 zero size 2024-02-28T23:07:48.429 INFO:tasks.workunit.client.0.smithi057.stdout:8/751: dwrite d4/f5 [0,4194304] 0 2024-02-28T23:07:48.434 INFO:tasks.workunit.client.1.smithi084.stdout:5/879: mknod dc/de/d83/c14b 0 2024-02-28T23:07:48.436 INFO:tasks.workunit.client.0.smithi057.stdout:7/728: creat d0/d5/d36/d4e/d15/d29/ff4 x:0 0 0 2024-02-28T23:07:48.436 INFO:tasks.workunit.client.0.smithi057.stdout:6/673: dread d0/d17/d94/f9e [0,4194304] 0 2024-02-28T23:07:48.439 INFO:tasks.workunit.client.1.smithi084.stdout:0/882: mkdir d1/d6/db/d35/d4a/d11d 0 2024-02-28T23:07:48.441 INFO:tasks.workunit.client.1.smithi084.stdout:9/923: unlink d4/d6/d15/d75/d2d/f34 0 2024-02-28T23:07:48.441 INFO:tasks.workunit.client.0.smithi057.stdout:1/634: link d2/d71/f34 d2/d7/df/db/d3a/d2a/d82/fe3 0 2024-02-28T23:07:48.444 INFO:tasks.workunit.client.0.smithi057.stdout:2/815: stat d8/d11/d21/d28/d41/c8c 0 2024-02-28T23:07:48.446 INFO:tasks.workunit.client.1.smithi084.stdout:0/883: chown d1/d6/db/d37/d72/d7b 15 1 2024-02-28T23:07:48.446 INFO:tasks.workunit.client.0.smithi057.stdout:2/816: stat d8/l2b 0 2024-02-28T23:07:48.447 INFO:tasks.workunit.client.0.smithi057.stdout:6/674: rmdir d0/d17/dde 0 2024-02-28T23:07:48.448 INFO:tasks.workunit.client.1.smithi084.stdout:0/884: chown d1/d6/d1b 7473 1 2024-02-28T23:07:48.453 INFO:tasks.workunit.client.0.smithi057.stdout:0/828: dread d17/d20/d4a/d78/d9d/fd6 [0,4194304] 0 2024-02-28T23:07:48.454 INFO:tasks.workunit.client.1.smithi084.stdout:0/885: mkdir d1/d8/d18/d4c/d6c/d74/d11e 0 2024-02-28T23:07:48.457 INFO:tasks.workunit.client.0.smithi057.stdout:6/675: creat d0/d17/d94/fe5 x:0 0 0 2024-02-28T23:07:48.457 INFO:tasks.workunit.client.1.smithi084.stdout:8/856: truncate d0/d9/d30/d33/f94 3948360 0 2024-02-28T23:07:48.459 INFO:tasks.workunit.client.0.smithi057.stdout:0/829: unlink d17/f9c 0 2024-02-28T23:07:48.459 INFO:tasks.workunit.client.1.smithi084.stdout:0/886: truncate d1/d6/d59/ffb 5283973 0 2024-02-28T23:07:48.459 INFO:tasks.workunit.client.1.smithi084.stdout:8/857: rmdir d0/d24/d9e 39 2024-02-28T23:07:48.459 INFO:tasks.workunit.client.0.smithi057.stdout:1/635: getdents d2/d90 0 2024-02-28T23:07:48.468 INFO:tasks.workunit.client.0.smithi057.stdout:1/636: write d2/d7/d4f/f63 [876852,61809] 0 2024-02-28T23:07:48.468 INFO:tasks.workunit.client.1.smithi084.stdout:0/887: unlink d1/d8/d69/d6b/f9b 0 2024-02-28T23:07:48.471 INFO:tasks.workunit.client.0.smithi057.stdout:0/830: rmdir d17/d20/d4a/d78/db5/dba 39 2024-02-28T23:07:48.474 INFO:tasks.workunit.client.1.smithi084.stdout:9/924: write d4/d6/d15/d3c/f47 [7984853,110887] 0 2024-02-28T23:07:48.474 INFO:tasks.workunit.client.0.smithi057.stdout:8/752: truncate d4/de9/f36 2693437 0 2024-02-28T23:07:48.476 INFO:tasks.workunit.client.0.smithi057.stdout:0/831: rmdir d17/d20/d23/d47/d60 39 2024-02-28T23:07:48.478 INFO:tasks.workunit.client.1.smithi084.stdout:3/920: dread d3/d50/fd5 [0,4194304] 0 2024-02-28T23:07:48.478 INFO:tasks.workunit.client.1.smithi084.stdout:0/888: rmdir d1/d8/d18/d4c/d6c/d74/dd8 0 2024-02-28T23:07:48.481 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:48 smithi057 ceph-mon[29096]: pgmap v30: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 89 MiB/s rd, 224 MiB/s wr, 464 op/s 2024-02-28T23:07:48.483 INFO:tasks.workunit.client.0.smithi057.stdout:7/729: truncate d0/d5/d36/d4e/d15/f60 3492046 0 2024-02-28T23:07:48.483 INFO:tasks.workunit.client.1.smithi084.stdout:0/889: chown d1/d8/d18/d4c/d6c/d74 12 1 2024-02-28T23:07:48.484 INFO:tasks.workunit.client.1.smithi084.stdout:6/842: dwrite d1/d13/d2e/d41/d4d/f63 [0,4194304] 0 2024-02-28T23:07:48.484 INFO:tasks.workunit.client.1.smithi084.stdout:3/921: write d3/d50/d117/d134/f152 [13209,29055] 0 2024-02-28T23:07:48.485 INFO:tasks.workunit.client.1.smithi084.stdout:5/880: dwrite dc/de/df/d31/d8e/da9/fb3 [0,4194304] 0 2024-02-28T23:07:48.490 INFO:tasks.workunit.client.0.smithi057.stdout:2/817: dwrite d8/d11/d15/fc4 [0,4194304] 0 2024-02-28T23:07:48.493 INFO:tasks.workunit.client.1.smithi084.stdout:9/925: write d4/f9c [51808,59047] 0 2024-02-28T23:07:48.494 INFO:tasks.workunit.client.0.smithi057.stdout:4/689: dwrite d1a/d1e/f5b [0,4194304] 0 2024-02-28T23:07:48.495 INFO:tasks.workunit.client.0.smithi057.stdout:3/606: sync 2024-02-28T23:07:48.496 INFO:tasks.workunit.client.1.smithi084.stdout:3/922: creat d3/d50/d5a/d73/dd8/f154 x:0 0 0 2024-02-28T23:07:48.496 INFO:tasks.workunit.client.0.smithi057.stdout:1/637: truncate d2/d7/df/db/d12/d3f/d47/f8e 1280985 0 2024-02-28T23:07:48.497 INFO:tasks.workunit.client.1.smithi084.stdout:9/926: readlink d4/d32/d3f/lc7 0 2024-02-28T23:07:48.505 INFO:tasks.workunit.client.0.smithi057.stdout:0/832: write d17/d20/d26/d48/ffa [53585,123595] 0 2024-02-28T23:07:48.506 INFO:tasks.workunit.client.1.smithi084.stdout:3/923: read d3/d27/d7d/db0/d1a/d29/d41/d60/fc5 [24672,11916] 0 2024-02-28T23:07:48.507 INFO:tasks.workunit.client.0.smithi057.stdout:5/726: dwrite d4/d9/d14/f5e [0,4194304] 0 2024-02-28T23:07:48.507 INFO:tasks.workunit.client.0.smithi057.stdout:8/753: link d4/f7 d4/de9/d56/f109 0 2024-02-28T23:07:48.512 INFO:tasks.workunit.client.1.smithi084.stdout:3/924: creat d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/f155 x:0 0 0 2024-02-28T23:07:48.517 INFO:tasks.workunit.client.1.smithi084.stdout:0/890: dread d1/d8/f36 [0,4194304] 0 2024-02-28T23:07:48.517 INFO:tasks.workunit.client.0.smithi057.stdout:7/730: read d0/d3f/d76/dc1/fdf [3165333,117317] 0 2024-02-28T23:07:48.523 INFO:tasks.workunit.client.1.smithi084.stdout:0/891: truncate d1/d6/db/d35/d44/d92/fea 814802 0 2024-02-28T23:07:48.524 INFO:tasks.workunit.client.0.smithi057.stdout:5/727: chown d4/d9/d14/d63/d88/l91 493725 1 2024-02-28T23:07:48.525 INFO:tasks.workunit.client.1.smithi084.stdout:0/892: chown d1/d8/d69/db2/fdc 3183073 1 2024-02-28T23:07:48.526 INFO:tasks.workunit.client.0.smithi057.stdout:6/676: dwrite d0/d1b/d3e/d40/f8b [0,4194304] 0 2024-02-28T23:07:48.526 INFO:tasks.workunit.client.0.smithi057.stdout:7/731: write d0/d5/d36/d4e/f19 [5701582,128387] 0 2024-02-28T23:07:48.529 INFO:tasks.workunit.client.1.smithi084.stdout:0/893: unlink d1/dc0/lfc 0 2024-02-28T23:07:48.532 INFO:tasks.workunit.client.0.smithi057.stdout:8/754: creat d4/de9/d56/d6a/f10a x:0 0 0 2024-02-28T23:07:48.533 INFO:tasks.workunit.client.0.smithi057.stdout:0/833: symlink d17/d20/d26/d48/d7c/d98/da6/l129 0 2024-02-28T23:07:48.535 INFO:tasks.workunit.client.0.smithi057.stdout:2/818: creat d8/d11/d39/d83/dde/ffc x:0 0 0 2024-02-28T23:07:48.536 INFO:tasks.workunit.client.0.smithi057.stdout:6/677: rename d0/d1b/d3e/d3d/d7d to d0/d17/d94/da0/de6 0 2024-02-28T23:07:48.538 INFO:tasks.workunit.client.1.smithi084.stdout:0/894: creat d1/d6/df6/f11f x:0 0 0 2024-02-28T23:07:48.538 INFO:tasks.workunit.client.1.smithi084.stdout:6/843: dread d1/d9/ddc/fad [0,4194304] 0 2024-02-28T23:07:48.539 INFO:tasks.workunit.client.0.smithi057.stdout:5/728: fdatasync d4/d9/d16/d2c/f80 0 2024-02-28T23:07:48.540 INFO:tasks.workunit.client.1.smithi084.stdout:3/925: dread d3/d27/d7d/db0/d14/d31/f5d [0,4194304] 0 2024-02-28T23:07:48.543 INFO:tasks.workunit.client.1.smithi084.stdout:6/844: symlink d1/d13/d50/l116 0 2024-02-28T23:07:48.544 INFO:tasks.workunit.client.1.smithi084.stdout:3/926: fdatasync d3/d27/d7d/db0/d1a/d45/d5e/fe2 0 2024-02-28T23:07:48.544 INFO:tasks.workunit.client.1.smithi084.stdout:6/845: chown d1/d13/d47/d91 662202 1 2024-02-28T23:07:48.548 INFO:tasks.workunit.client.1.smithi084.stdout:6/846: symlink d1/d13/d47/l117 0 2024-02-28T23:07:48.549 INFO:tasks.workunit.client.1.smithi084.stdout:6/847: chown d1/f24 5567 1 2024-02-28T23:07:48.553 INFO:tasks.workunit.client.0.smithi057.stdout:8/755: mknod d4/d13/d18/d62/de4/c10b 0 2024-02-28T23:07:48.553 INFO:tasks.workunit.client.0.smithi057.stdout:0/834: dread d17/d20/db6/f10b [0,4194304] 0 2024-02-28T23:07:48.569 INFO:tasks.workunit.client.0.smithi057.stdout:0/835: creat d17/d19/dd2/f12a x:0 0 0 2024-02-28T23:07:48.574 INFO:tasks.workunit.client.0.smithi057.stdout:8/756: dread - d4/d13/d18/d62/d66/fd8 zero size 2024-02-28T23:07:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:48 smithi084 ceph-mon[34826]: pgmap v30: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 89 MiB/s rd, 224 MiB/s wr, 464 op/s 2024-02-28T23:07:48.580 INFO:tasks.workunit.client.0.smithi057.stdout:5/729: rename d4/d7/f73 to d4/d9/d14/d63/fe2 0 2024-02-28T23:07:48.581 INFO:tasks.workunit.client.0.smithi057.stdout:9/693: sync 2024-02-28T23:07:48.586 INFO:tasks.workunit.client.1.smithi084.stdout:6/848: dread d1/d9/d1d/d1e/fc8 [0,4194304] 0 2024-02-28T23:07:48.590 INFO:tasks.workunit.client.0.smithi057.stdout:9/694: fdatasync dd/de/d14/f18 0 2024-02-28T23:07:48.593 INFO:tasks.workunit.client.1.smithi084.stdout:0/895: dwrite d1/d6/db/d35/d82/daf/fc6 [0,4194304] 0 2024-02-28T23:07:48.593 INFO:tasks.workunit.client.0.smithi057.stdout:0/836: mknod d17/d20/d26/d48/d7c/d98/da6/c12b 0 2024-02-28T23:07:48.594 INFO:tasks.workunit.client.1.smithi084.stdout:0/896: readlink d1/d6/db/d37/d72/lba 0 2024-02-28T23:07:48.595 INFO:tasks.workunit.client.0.smithi057.stdout:6/678: dread d0/d4/d11/f2d [0,4194304] 0 2024-02-28T23:07:48.595 INFO:tasks.workunit.client.0.smithi057.stdout:1/638: write d2/f21 [1204153,27298] 0 2024-02-28T23:07:48.596 INFO:tasks.workunit.client.1.smithi084.stdout:0/897: read - d1/d6/d59/d70/f118 zero size 2024-02-28T23:07:48.596 INFO:tasks.workunit.client.0.smithi057.stdout:3/607: dwrite d3/d20/d2d/f3b [0,4194304] 0 2024-02-28T23:07:48.597 INFO:tasks.workunit.client.0.smithi057.stdout:6/679: stat d0/d4/d25/fa9 0 2024-02-28T23:07:48.597 INFO:tasks.workunit.client.1.smithi084.stdout:0/898: fsync d1/d8/f40 0 2024-02-28T23:07:48.599 INFO:tasks.workunit.client.1.smithi084.stdout:0/899: chown d1/d6/db/d37/dae/dff 3 1 2024-02-28T23:07:48.601 INFO:tasks.workunit.client.1.smithi084.stdout:0/900: chown d1/d8/d18/d4c/d6c/d74/fad 196542 1 2024-02-28T23:07:48.603 INFO:tasks.workunit.client.1.smithi084.stdout:0/901: symlink d1/d6/db/d35/d4a/d56/l120 0 2024-02-28T23:07:48.604 INFO:tasks.workunit.client.1.smithi084.stdout:6/849: dread d1/d13/d2e/fd3 [4194304,4194304] 0 2024-02-28T23:07:48.606 INFO:tasks.workunit.client.1.smithi084.stdout:6/850: dread - d1/d13/d50/fdf zero size 2024-02-28T23:07:48.608 INFO:tasks.workunit.client.1.smithi084.stdout:6/851: mknod d1/d13/d50/dbd/dc4/c118 0 2024-02-28T23:07:48.609 INFO:tasks.workunit.client.1.smithi084.stdout:6/852: readlink d1/d88/dbf/lc0 0 2024-02-28T23:07:48.610 INFO:tasks.workunit.client.0.smithi057.stdout:8/757: mkdir d4/d13/df0/d104/d10c 0 2024-02-28T23:07:48.615 INFO:tasks.workunit.client.0.smithi057.stdout:9/695: chown dd/de/d14/d48/d4b/d41/f93 676902 1 2024-02-28T23:07:48.615 INFO:tasks.workunit.client.1.smithi084.stdout:6/853: link d1/d9/d1d/d1e/d33/f6d d1/d13/d2e/f119 0 2024-02-28T23:07:48.617 INFO:tasks.workunit.client.0.smithi057.stdout:5/730: dwrite d4/d9/d14/f5e [0,4194304] 0 2024-02-28T23:07:48.618 INFO:tasks.workunit.client.1.smithi084.stdout:0/902: dwrite d1/d8/d78/f9d [0,4194304] 0 2024-02-28T23:07:48.618 INFO:tasks.workunit.client.1.smithi084.stdout:6/854: mknod d1/d13/d50/dbd/d10f/c11a 0 2024-02-28T23:07:48.622 INFO:tasks.workunit.client.1.smithi084.stdout:6/855: creat d1/d13/d50/d74/d9e/f11b x:0 0 0 2024-02-28T23:07:48.627 INFO:tasks.workunit.client.1.smithi084.stdout:0/903: creat d1/d6/db/d35/f121 x:0 0 0 2024-02-28T23:07:48.628 INFO:tasks.workunit.client.0.smithi057.stdout:7/732: dwrite d0/d3f/d76/dc1/dea/f5d [0,4194304] 0 2024-02-28T23:07:48.628 INFO:tasks.workunit.client.0.smithi057.stdout:8/758: mkdir d4/d13/d18/d10d 0 2024-02-28T23:07:48.629 INFO:tasks.workunit.client.1.smithi084.stdout:0/904: fsync d1/d6/db/d35/d82/fda 0 2024-02-28T23:07:48.630 INFO:tasks.workunit.client.1.smithi084.stdout:6/856: creat d1/d9/d1d/d1e/d33/f11c x:0 0 0 2024-02-28T23:07:48.631 INFO:tasks.workunit.client.1.smithi084.stdout:6/857: rename d1 to d1/d13/d2e/d11d 22 2024-02-28T23:07:48.633 INFO:tasks.workunit.client.1.smithi084.stdout:6/858: readlink d1/d88/dbf/de2/l107 0 2024-02-28T23:07:48.633 INFO:tasks.workunit.client.1.smithi084.stdout:6/859: readlink d1/l25 0 2024-02-28T23:07:48.635 INFO:tasks.workunit.client.1.smithi084.stdout:0/905: mknod d1/d6/db/d35/d57/c122 0 2024-02-28T23:07:48.635 INFO:tasks.workunit.client.0.smithi057.stdout:8/759: rmdir d4/de9/d59 39 2024-02-28T23:07:48.641 INFO:tasks.workunit.client.0.smithi057.stdout:7/733: symlink d0/d5/d25/d80/lf5 0 2024-02-28T23:07:48.641 INFO:tasks.workunit.client.0.smithi057.stdout:8/760: write d4/f5 [344235,10704] 0 2024-02-28T23:07:48.643 INFO:tasks.workunit.client.0.smithi057.stdout:3/608: link d3/d4/d1d/f1e d3/d4f/d4c/d91/fc0 0 2024-02-28T23:07:48.644 INFO:tasks.workunit.client.0.smithi057.stdout:6/680: creat d0/fe7 x:0 0 0 2024-02-28T23:07:48.653 INFO:tasks.workunit.client.0.smithi057.stdout:3/609: rename d3/d4/d7/f6c to d3/d4/d7/d9a/d9b/dab/fc1 0 2024-02-28T23:07:48.653 INFO:tasks.workunit.client.1.smithi084.stdout:6/860: getdents d1/d13/d50/d74/d9e 0 2024-02-28T23:07:48.655 INFO:tasks.workunit.client.0.smithi057.stdout:1/639: write d2/d7/df/db/d12/f26 [4026187,94846] 0 2024-02-28T23:07:48.656 INFO:tasks.workunit.client.0.smithi057.stdout:8/761: creat d4/d13/df0/f10e x:0 0 0 2024-02-28T23:07:48.657 INFO:tasks.workunit.client.1.smithi084.stdout:6/861: creat d1/d13/d50/dbd/d10f/f11e x:0 0 0 2024-02-28T23:07:48.659 INFO:tasks.workunit.client.1.smithi084.stdout:6/862: fdatasync d1/d13/d2e/d41/d4d/fee 0 2024-02-28T23:07:48.698 INFO:tasks.workunit.client.1.smithi084.stdout:1/914: sync 2024-02-28T23:07:48.698 INFO:tasks.workunit.client.1.smithi084.stdout:2/780: sync 2024-02-28T23:07:48.698 INFO:tasks.workunit.client.1.smithi084.stdout:7/953: sync 2024-02-28T23:07:48.698 INFO:tasks.workunit.client.1.smithi084.stdout:4/831: sync 2024-02-28T23:07:48.703 INFO:tasks.workunit.client.1.smithi084.stdout:2/781: unlink d5/d1f/d42/l57 0 2024-02-28T23:07:48.704 INFO:tasks.workunit.client.1.smithi084.stdout:7/954: symlink db/d10/d17/d9c/l132 0 2024-02-28T23:07:48.707 INFO:tasks.workunit.client.1.smithi084.stdout:4/832: creat d1/d18/d20/d14/d30/f113 x:0 0 0 2024-02-28T23:07:48.707 INFO:tasks.workunit.client.1.smithi084.stdout:7/955: fsync db/d10/d17/d64/fe8 0 2024-02-28T23:07:48.709 INFO:tasks.workunit.client.1.smithi084.stdout:2/782: symlink d5/d7/d9/d7c/d5b/led 0 2024-02-28T23:07:48.710 INFO:tasks.workunit.client.1.smithi084.stdout:4/833: chown d1/d18/d20/d14/d30/f3d 15 1 2024-02-28T23:07:48.712 INFO:tasks.workunit.client.1.smithi084.stdout:7/956: write db/d10/d18/db7/de9/dfa/f4f [4624499,112879] 0 2024-02-28T23:07:48.713 INFO:tasks.workunit.client.1.smithi084.stdout:2/783: symlink d5/d7/lee 0 2024-02-28T23:07:48.722 INFO:tasks.workunit.client.1.smithi084.stdout:7/957: link db/d10/d17/f9a db/d10/d17/d9c/d9f/db1/d104/d120/f133 0 2024-02-28T23:07:48.724 INFO:tasks.workunit.client.1.smithi084.stdout:7/958: dread - db/dbd/ffe zero size 2024-02-28T23:07:48.730 INFO:tasks.workunit.client.0.smithi057.stdout:5/731: truncate d4/d9/d16/d4e/f87 461782 0 2024-02-28T23:07:48.735 INFO:tasks.workunit.client.1.smithi084.stdout:7/959: link db/d10/d18/db7/dbb/c10b db/c134 0 2024-02-28T23:07:48.735 INFO:tasks.workunit.client.0.smithi057.stdout:3/610: rename d3/d8/d27/l63 to d3/d4/d1d/d47/lc2 0 2024-02-28T23:07:48.736 INFO:tasks.workunit.client.1.smithi084.stdout:7/960: rmdir db/d5e/d87 39 2024-02-28T23:07:48.737 INFO:tasks.workunit.client.1.smithi084.stdout:2/784: dwrite d5/d7/d2e/f3d [0,4194304] 0 2024-02-28T23:07:48.738 INFO:tasks.workunit.client.0.smithi057.stdout:7/734: dwrite d0/d5/f30 [0,4194304] 0 2024-02-28T23:07:48.738 INFO:tasks.workunit.client.0.smithi057.stdout:6/681: dwrite d0/d4/fd6 [0,4194304] 0 2024-02-28T23:07:48.743 INFO:tasks.workunit.client.1.smithi084.stdout:2/785: symlink d5/d1f/d83/lef 0 2024-02-28T23:07:48.743 INFO:tasks.workunit.client.1.smithi084.stdout:7/961: unlink db/d10/d18/db7/de9/dfa/dc7/l10d 0 2024-02-28T23:07:48.746 INFO:tasks.workunit.client.0.smithi057.stdout:0/837: dread d17/d19/f3c [0,4194304] 0 2024-02-28T23:07:48.747 INFO:tasks.workunit.client.0.smithi057.stdout:5/732: rmdir d4/d9/d16/d8f 39 2024-02-28T23:07:48.748 INFO:tasks.workunit.client.1.smithi084.stdout:4/834: dread d1/d4b/fca [0,4194304] 0 2024-02-28T23:07:48.749 INFO:tasks.workunit.client.1.smithi084.stdout:2/786: fsync d5/d7/d9/d94/f9e 0 2024-02-28T23:07:48.750 INFO:tasks.workunit.client.0.smithi057.stdout:9/696: dwrite dd/de/d24/d63/d77/dcf/d40/fc3 [0,4194304] 0 2024-02-28T23:07:48.751 INFO:tasks.workunit.client.1.smithi084.stdout:2/787: chown d5/d7/d2e/f3d 835 1 2024-02-28T23:07:48.753 INFO:tasks.workunit.client.0.smithi057.stdout:0/838: fdatasync d17/d20/d26/d48/ffa 0 2024-02-28T23:07:48.753 INFO:tasks.workunit.client.1.smithi084.stdout:2/788: write d5/fd4 [6613092,91111] 0 2024-02-28T23:07:48.753 INFO:tasks.workunit.client.1.smithi084.stdout:4/835: creat d1/d18/d20/d65/d8e/d55/dc3/f114 x:0 0 0 2024-02-28T23:07:48.764 INFO:tasks.workunit.client.0.smithi057.stdout:5/733: rename d4/d21/d24/da2/dbb/d95/fb7 to d4/d21/d24/d30/d62/fe3 0 2024-02-28T23:07:48.770 INFO:tasks.workunit.client.0.smithi057.stdout:9/697: mkdir dd/de/d24/d63/dc1/de0 0 2024-02-28T23:07:48.772 INFO:tasks.workunit.client.1.smithi084.stdout:4/836: dwrite d1/d4b/ffc [0,4194304] 0 2024-02-28T23:07:48.774 INFO:tasks.workunit.client.0.smithi057.stdout:3/611: dwrite d3/d8/d1b/fb7 [0,4194304] 0 2024-02-28T23:07:48.775 INFO:tasks.workunit.client.0.smithi057.stdout:9/698: readlink dd/de/d24/l36 0 2024-02-28T23:07:48.776 INFO:tasks.workunit.client.0.smithi057.stdout:5/734: mknod d4/d9/d16/d52/d7a/ce4 0 2024-02-28T23:07:48.777 INFO:tasks.workunit.client.1.smithi084.stdout:4/837: mkdir d1/d18/d20/d14/d30/d47/da4/d10f/d115 0 2024-02-28T23:07:48.777 INFO:tasks.workunit.client.0.smithi057.stdout:8/762: dread d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:48.780 INFO:tasks.workunit.client.0.smithi057.stdout:0/839: truncate d17/d20/d26/d48/d7c/d98/da6/fd5 673019 0 2024-02-28T23:07:48.783 INFO:tasks.workunit.client.0.smithi057.stdout:5/735: readlink d4/d21/d24/da2/dbb/d95/l9f 0 2024-02-28T23:07:48.786 INFO:tasks.workunit.client.1.smithi084.stdout:4/838: rename d1/d18/d20/d24/ddb/ce4 to d1/d18/d20/d24/d2f/d3e/d52/d57/c116 0 2024-02-28T23:07:48.788 INFO:tasks.workunit.client.1.smithi084.stdout:4/839: chown d1/d18/d20/d14/l78 0 1 2024-02-28T23:07:48.791 INFO:tasks.workunit.client.0.smithi057.stdout:3/612: creat d3/d20/fc3 x:0 0 0 2024-02-28T23:07:48.793 INFO:tasks.workunit.client.0.smithi057.stdout:9/699: mknod dd/de/d14/d48/d4b/ce1 0 2024-02-28T23:07:48.795 INFO:tasks.workunit.client.1.smithi084.stdout:7/962: dread db/d10/d18/db7/fbe [0,4194304] 0 2024-02-28T23:07:48.798 INFO:tasks.workunit.client.1.smithi084.stdout:7/963: stat db/d10/d17/d9c/d9f/db1/c12c 0 2024-02-28T23:07:48.798 INFO:tasks.workunit.client.0.smithi057.stdout:7/735: write d0/d5/d25/f28 [1398439,35044] 0 2024-02-28T23:07:48.798 INFO:tasks.workunit.client.0.smithi057.stdout:0/840: creat d17/d20/db6/f12c x:0 0 0 2024-02-28T23:07:48.799 INFO:tasks.workunit.client.0.smithi057.stdout:5/736: dread - d4/fb0 zero size 2024-02-28T23:07:48.800 INFO:tasks.workunit.client.0.smithi057.stdout:3/613: symlink d3/d4/d1d/d47/d62/db4/lc4 0 2024-02-28T23:07:48.800 INFO:tasks.workunit.client.1.smithi084.stdout:7/964: chown db/d10/d17/d9c/d9f/d107/fc0 10937356 1 2024-02-28T23:07:48.802 INFO:tasks.workunit.client.0.smithi057.stdout:7/736: dread - d0/d3f/fe3 zero size 2024-02-28T23:07:48.804 INFO:tasks.workunit.client.1.smithi084.stdout:4/840: dwrite d1/dd9/fef [0,4194304] 0 2024-02-28T23:07:48.805 INFO:tasks.workunit.client.1.smithi084.stdout:7/965: creat db/d10/d18/db7/de9/f135 x:0 0 0 2024-02-28T23:07:48.806 INFO:tasks.workunit.client.0.smithi057.stdout:3/614: write d3/d4/f30 [2360838,73193] 0 2024-02-28T23:07:48.808 INFO:tasks.workunit.client.1.smithi084.stdout:7/966: fdatasync db/d10/d17/d64/d77/ff0 0 2024-02-28T23:07:48.813 INFO:tasks.workunit.client.1.smithi084.stdout:2/789: dread d5/d1f/d42/d74/fd0 [0,4194304] 0 2024-02-28T23:07:48.814 INFO:tasks.workunit.client.1.smithi084.stdout:4/841: symlink d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/df6/l117 0 2024-02-28T23:07:48.814 INFO:tasks.workunit.client.0.smithi057.stdout:0/841: mkdir d17/d19/dd2/d12d 0 2024-02-28T23:07:48.814 INFO:tasks.workunit.client.0.smithi057.stdout:5/737: symlink d4/d21/d24/d72/le5 0 2024-02-28T23:07:48.814 INFO:tasks.workunit.client.0.smithi057.stdout:1/640: dwrite d2/d7/df/db/d12/d19/d65/f96 [0,4194304] 0 2024-02-28T23:07:48.822 INFO:tasks.workunit.client.0.smithi057.stdout:7/737: creat d0/d5/d36/d4e/d15/da0/ff6 x:0 0 0 2024-02-28T23:07:48.829 INFO:tasks.workunit.client.1.smithi084.stdout:2/790: write d5/d7/d2e/f46 [1379651,123939] 0 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.1.smithi084.stdout:3/927: sync 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.1.smithi084.stdout:9/927: sync 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.1.smithi084.stdout:5/881: sync 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.1.smithi084.stdout:8/858: sync 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.0.smithi057.stdout:0/842: truncate d17/d20/d23/d47/f7f 1014891 0 2024-02-28T23:07:48.830 INFO:tasks.workunit.client.1.smithi084.stdout:4/842: write d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fcc [841596,100527] 0 2024-02-28T23:07:48.831 INFO:tasks.workunit.client.0.smithi057.stdout:1/641: rename d2/d71/f34 to d2/d7/df/db/d3a/d2a/d82/fe4 0 2024-02-28T23:07:48.833 INFO:tasks.workunit.client.1.smithi084.stdout:5/882: chown dc/de/df/d16/d34/de8/cd0 404763959 1 2024-02-28T23:07:48.844 INFO:tasks.workunit.client.0.smithi057.stdout:1/642: read d2/d71/d2d/d5b/f7d [354537,112835] 0 2024-02-28T23:07:48.844 INFO:tasks.workunit.client.0.smithi057.stdout:4/690: sync 2024-02-28T23:07:48.844 INFO:tasks.workunit.client.0.smithi057.stdout:9/700: dwrite dd/de/d14/d48/d4b/d41/fcb [0,4194304] 0 2024-02-28T23:07:48.844 INFO:tasks.workunit.client.1.smithi084.stdout:8/859: truncate d0/d24/d78/fd5 1057342 0 2024-02-28T23:07:48.845 INFO:tasks.workunit.client.1.smithi084.stdout:8/860: readlink d0/d44/lbb 0 2024-02-28T23:07:48.847 INFO:tasks.workunit.client.1.smithi084.stdout:7/967: dread db/d10/d17/d3b/f4d [4194304,4194304] 0 2024-02-28T23:07:48.848 INFO:tasks.workunit.client.1.smithi084.stdout:4/843: rename d1/d18/d20/d24/l26 to d1/d18/d20/d24/d2f/d3e/l118 0 2024-02-28T23:07:48.849 INFO:tasks.workunit.client.0.smithi057.stdout:8/763: dread d4/de9/d56/d6a/d42/ff2 [0,4194304] 0 2024-02-28T23:07:48.851 INFO:tasks.workunit.client.1.smithi084.stdout:8/861: dread - d0/d9/d30/db6/fd7 zero size 2024-02-28T23:07:48.854 INFO:tasks.workunit.client.1.smithi084.stdout:4/844: chown d1/d18/d20/d24/d2f/d3e/d52/d45/f93 125990 1 2024-02-28T23:07:48.854 INFO:tasks.workunit.client.1.smithi084.stdout:3/928: write d3/f11 [1789798,21866] 0 2024-02-28T23:07:48.855 INFO:tasks.workunit.client.1.smithi084.stdout:5/883: write dc/de/df/d16/d34/d3c/fb8 [1052897,12675] 0 2024-02-28T23:07:48.855 INFO:tasks.workunit.client.0.smithi057.stdout:4/691: symlink d1a/d48/d97/lef 0 2024-02-28T23:07:48.856 INFO:tasks.workunit.client.0.smithi057.stdout:8/764: readlink d4/l16 0 2024-02-28T23:07:48.857 INFO:tasks.workunit.client.1.smithi084.stdout:8/862: symlink d0/d9/l119 0 2024-02-28T23:07:48.857 INFO:tasks.workunit.client.0.smithi057.stdout:3/615: dwrite d3/d4/d7/f8f [0,4194304] 0 2024-02-28T23:07:48.862 INFO:tasks.workunit.client.1.smithi084.stdout:8/863: write d0/d9/d30/f111 [66173,85070] 0 2024-02-28T23:07:48.862 INFO:tasks.workunit.client.1.smithi084.stdout:4/845: fdatasync d1/d18/d20/d14/d30/f81 0 2024-02-28T23:07:48.865 INFO:tasks.workunit.client.1.smithi084.stdout:3/929: unlink d3/d50/d117/d3d/f4e 0 2024-02-28T23:07:48.865 INFO:tasks.workunit.client.1.smithi084.stdout:7/968: dread db/d10/d17/fae [0,4194304] 0 2024-02-28T23:07:48.866 INFO:tasks.workunit.client.0.smithi057.stdout:5/738: dwrite d4/d9/d14/f54 [4194304,4194304] 0 2024-02-28T23:07:48.868 INFO:tasks.workunit.client.0.smithi057.stdout:4/692: chown d1a/d48/d97/f21 2224479 1 2024-02-28T23:07:48.869 INFO:tasks.workunit.client.1.smithi084.stdout:7/969: dread - db/d10/d18/db7/de9/f135 zero size 2024-02-28T23:07:48.876 INFO:tasks.workunit.client.1.smithi084.stdout:9/928: dread d4/d32/d3f/d89/db8/fda [0,4194304] 0 2024-02-28T23:07:48.876 INFO:tasks.workunit.client.1.smithi084.stdout:2/791: dread d5/f15 [0,4194304] 0 2024-02-28T23:07:48.877 INFO:tasks.workunit.client.1.smithi084.stdout:7/970: mkdir db/d5e/d80/d136 0 2024-02-28T23:07:48.878 INFO:tasks.workunit.client.1.smithi084.stdout:3/930: rename d3/d50/d117/d3d/d43/f48 to d3/d50/d117/d7b/dbf/f156 0 2024-02-28T23:07:48.878 INFO:tasks.workunit.client.0.smithi057.stdout:7/738: dwrite d0/d5/d36/d4e/d15/fca [0,4194304] 0 2024-02-28T23:07:48.879 INFO:tasks.workunit.client.1.smithi084.stdout:8/864: truncate d0/d9/d1b/d38/d43/d51/ff5 963933 0 2024-02-28T23:07:48.883 INFO:tasks.workunit.client.0.smithi057.stdout:4/693: mkdir d1a/d48/d9c/d68/df0 0 2024-02-28T23:07:48.884 INFO:tasks.workunit.client.0.smithi057.stdout:1/643: dwrite d2/d71/d62/f74 [0,4194304] 0 2024-02-28T23:07:48.886 INFO:tasks.workunit.client.0.smithi057.stdout:3/616: rename d3/d4/d1d/f2e to d3/d8/d27/d70/fc5 0 2024-02-28T23:07:48.887 INFO:tasks.workunit.client.1.smithi084.stdout:3/931: creat d3/d27/d7d/db0/d1a/d45/dde/f157 x:0 0 0 2024-02-28T23:07:48.888 INFO:tasks.workunit.client.1.smithi084.stdout:8/865: creat d0/d9/d30/f11a x:0 0 0 2024-02-28T23:07:48.889 INFO:tasks.workunit.client.1.smithi084.stdout:6/863: sync 2024-02-28T23:07:48.889 INFO:tasks.workunit.client.1.smithi084.stdout:0/906: sync 2024-02-28T23:07:48.891 INFO:tasks.workunit.client.0.smithi057.stdout:4/694: write d1a/d48/d97/fc0 [1794746,17430] 0 2024-02-28T23:07:48.894 INFO:tasks.workunit.client.0.smithi057.stdout:4/695: dread - d1a/d2f/db2/d49/fc8 zero size 2024-02-28T23:07:48.898 INFO:tasks.workunit.client.0.smithi057.stdout:4/696: write d1a/d1e/f5b [3918913,980] 0 2024-02-28T23:07:48.899 INFO:tasks.workunit.client.1.smithi084.stdout:8/866: truncate d0/d9/f5f 2882346 0 2024-02-28T23:07:48.904 INFO:tasks.workunit.client.1.smithi084.stdout:8/867: fdatasync d0/d9/d1b/f28 0 2024-02-28T23:07:48.908 INFO:tasks.workunit.client.0.smithi057.stdout:7/739: rename d0/d3f/da1 to d0/d5/d36/d4e/d15/d29/d3d/de7/df7 0 2024-02-28T23:07:48.909 INFO:tasks.workunit.client.1.smithi084.stdout:9/929: dread d4/d6/d15/d12/d41/f64 [0,4194304] 0 2024-02-28T23:07:48.909 INFO:tasks.workunit.client.1.smithi084.stdout:7/971: dwrite db/d10/d17/d9c/d9f/db1/d104/f12d [0,4194304] 0 2024-02-28T23:07:48.911 INFO:tasks.workunit.client.0.smithi057.stdout:3/617: symlink d3/d8/d1b/lc6 0 2024-02-28T23:07:48.912 INFO:tasks.workunit.client.1.smithi084.stdout:3/932: creat d3/d27/d7d/db0/d1a/d29/d41/d60/d98/f158 x:0 0 0 2024-02-28T23:07:48.919 INFO:tasks.workunit.client.0.smithi057.stdout:3/618: chown d3/d8/d1b/l28 146242 1 2024-02-28T23:07:48.920 INFO:tasks.workunit.client.0.smithi057.stdout:9/701: write dd/de/d24/d63/d77/dcf/f50 [436995,79863] 0 2024-02-28T23:07:48.923 INFO:tasks.workunit.client.1.smithi084.stdout:5/884: dwrite dc/d51/d7c/f84 [0,4194304] 0 2024-02-28T23:07:48.924 INFO:tasks.workunit.client.0.smithi057.stdout:3/619: symlink d3/lc7 0 2024-02-28T23:07:48.930 INFO:tasks.workunit.client.1.smithi084.stdout:2/792: write d5/f31 [2520369,19488] 0 2024-02-28T23:07:48.931 INFO:tasks.workunit.client.1.smithi084.stdout:7/972: creat db/d10/d18/db7/dbb/f137 x:0 0 0 2024-02-28T23:07:48.931 INFO:tasks.workunit.client.1.smithi084.stdout:5/885: fsync dc/de/df/d16/d34/f53 0 2024-02-28T23:07:48.943 INFO:tasks.workunit.client.0.smithi057.stdout:8/765: dwrite d4/d13/d2c/d4e/f87 [0,4194304] 0 2024-02-28T23:07:48.944 INFO:tasks.workunit.client.0.smithi057.stdout:1/644: dread d2/d33/d54/f8b [0,4194304] 0 2024-02-28T23:07:48.946 INFO:tasks.workunit.client.0.smithi057.stdout:4/697: dwrite d1a/d2f/db2/f63 [0,4194304] 0 2024-02-28T23:07:48.946 INFO:tasks.workunit.client.1.smithi084.stdout:6/864: rename d1/d13/d50/dbd/d10f/f11e to d1/d13/d47/d91/f11f 0 2024-02-28T23:07:48.946 INFO:tasks.workunit.client.1.smithi084.stdout:4/846: dwrite d1/f9c [0,4194304] 0 2024-02-28T23:07:48.953 INFO:tasks.workunit.client.1.smithi084.stdout:6/865: write d1/d13/d50/f8d [1991656,8353] 0 2024-02-28T23:07:48.955 INFO:tasks.workunit.client.0.smithi057.stdout:9/702: dwrite dd/de/d24/d63/d77/dcf/f50 [0,4194304] 0 2024-02-28T23:07:48.961 INFO:tasks.workunit.client.1.smithi084.stdout:8/868: write d0/d9/d30/d33/d71/d84/dc0/fc6 [1037906,107014] 0 2024-02-28T23:07:48.964 INFO:tasks.workunit.client.0.smithi057.stdout:3/620: dwrite d3/d4/d7/f96 [0,4194304] 0 2024-02-28T23:07:48.966 INFO:tasks.workunit.client.0.smithi057.stdout:3/621: dread - d3/d20/fc3 zero size 2024-02-28T23:07:48.967 INFO:tasks.workunit.client.0.smithi057.stdout:1/645: mkdir d2/dd4/de5 0 2024-02-28T23:07:48.973 INFO:tasks.workunit.client.0.smithi057.stdout:8/766: rename d4/d13/d2c/d4e/l53 to d4/de9/d56/da6/dae/l10f 0 2024-02-28T23:07:48.975 INFO:tasks.workunit.client.0.smithi057.stdout:3/622: write d3/d4/d7/f6a [4423816,127449] 0 2024-02-28T23:07:48.978 INFO:tasks.workunit.client.0.smithi057.stdout:1/646: write d2/d7/df/db/d3a/d8f/f94 [1035574,63542] 0 2024-02-28T23:07:48.981 INFO:tasks.workunit.client.0.smithi057.stdout:8/767: truncate d4/d13/fa8 5052423 0 2024-02-28T23:07:48.982 INFO:tasks.workunit.client.1.smithi084.stdout:2/793: dwrite d5/d7/d12/d24/fa7 [0,4194304] 0 2024-02-28T23:07:48.982 INFO:tasks.workunit.client.0.smithi057.stdout:4/698: write d1a/d1e/f42 [3892478,1490] 0 2024-02-28T23:07:48.983 INFO:tasks.workunit.client.0.smithi057.stdout:9/703: dread dd/de/d24/d63/d77/dcf/d40/d44/d5c/f8a [0,4194304] 0 2024-02-28T23:07:48.986 INFO:tasks.workunit.client.0.smithi057.stdout:3/623: mknod d3/d8/cc8 0 2024-02-28T23:07:48.987 INFO:tasks.workunit.client.0.smithi057.stdout:1/647: symlink d2/d7/df/db/d12/dd6/le6 0 2024-02-28T23:07:48.988 INFO:tasks.workunit.client.0.smithi057.stdout:9/704: chown dd/de/l5e 15692 1 2024-02-28T23:07:48.988 INFO:tasks.workunit.client.0.smithi057.stdout:4/699: mknod d1a/d2f/db2/cf1 0 2024-02-28T23:07:48.988 INFO:tasks.workunit.client.1.smithi084.stdout:3/933: creat d3/f159 x:0 0 0 2024-02-28T23:07:48.993 INFO:tasks.workunit.client.1.smithi084.stdout:9/930: rename d4/d6/d15/d12/f97 to d4/d6/d15/d75/d9d/dd0/f12c 0 2024-02-28T23:07:48.996 INFO:tasks.workunit.client.0.smithi057.stdout:9/705: chown dd/de/d24/d63/d77/dcf/d40/d44/d5c/f8a 189920 1 2024-02-28T23:07:48.997 INFO:tasks.workunit.client.0.smithi057.stdout:9/706: chown dd/de/d4a/d9f 7466 1 2024-02-28T23:07:48.998 INFO:tasks.workunit.client.1.smithi084.stdout:6/866: creat d1/d88/dbf/f120 x:0 0 0 2024-02-28T23:07:49.003 INFO:tasks.workunit.client.1.smithi084.stdout:5/886: creat dc/de/df/d16/d34/d3c/d77/df8/f14c x:0 0 0 2024-02-28T23:07:49.005 INFO:tasks.workunit.client.1.smithi084.stdout:8/869: fdatasync d0/d44/f67 0 2024-02-28T23:07:49.006 INFO:tasks.workunit.client.1.smithi084.stdout:5/887: read - dc/de/df/da6/ff5 zero size 2024-02-28T23:07:49.007 INFO:tasks.workunit.client.0.smithi057.stdout:9/707: symlink dd/d46/dc4/dd3/le2 0 2024-02-28T23:07:49.010 INFO:tasks.workunit.client.1.smithi084.stdout:9/931: creat d4/d6/d15/d12/d41/dea/f12d x:0 0 0 2024-02-28T23:07:49.011 INFO:tasks.workunit.client.1.smithi084.stdout:7/973: dread db/d10/d17/d64/f8c [0,4194304] 0 2024-02-28T23:07:49.011 INFO:tasks.workunit.client.0.smithi057.stdout:9/708: stat dd/de/d24/l30 0 2024-02-28T23:07:49.012 INFO:tasks.workunit.client.1.smithi084.stdout:6/867: creat d1/d13/d50/dbd/dd1/f121 x:0 0 0 2024-02-28T23:07:49.012 INFO:tasks.workunit.client.0.smithi057.stdout:2/819: sync 2024-02-28T23:07:49.014 INFO:tasks.workunit.client.1.smithi084.stdout:8/870: fdatasync d0/ffd 0 2024-02-28T23:07:49.016 INFO:tasks.workunit.client.1.smithi084.stdout:8/871: readlink d0/d9/d30/d33/l72 0 2024-02-28T23:07:49.016 INFO:tasks.workunit.client.1.smithi084.stdout:9/932: rmdir d4/d32/d3f/d89/db8 39 2024-02-28T23:07:49.018 INFO:tasks.workunit.client.1.smithi084.stdout:3/934: creat d3/d27/d7d/db0/d1a/f15a x:0 0 0 2024-02-28T23:07:49.019 INFO:tasks.workunit.client.1.smithi084.stdout:7/974: fsync db/d10/d17/d9c/d9f/d106/f12b 0 2024-02-28T23:07:49.019 INFO:tasks.workunit.client.1.smithi084.stdout:4/847: getdents d1/d18/d20/d24/d2f/d3e/d52/d45/dd4 0 2024-02-28T23:07:49.022 INFO:tasks.workunit.client.1.smithi084.stdout:5/888: mknod dc/de/c14d 0 2024-02-28T23:07:49.022 INFO:tasks.workunit.client.1.smithi084.stdout:2/794: dread d5/d7/d75/fac [0,4194304] 0 2024-02-28T23:07:49.022 INFO:tasks.workunit.client.0.smithi057.stdout:8/768: dwrite d4/de9/d56/d6a/d54/d6b/f6c [0,4194304] 0 2024-02-28T23:07:49.023 INFO:tasks.workunit.client.0.smithi057.stdout:9/709: fsync dd/d46/d6f/f79 0 2024-02-28T23:07:49.024 INFO:tasks.workunit.client.0.smithi057.stdout:3/624: dwrite d3/d4/f12 [0,4194304] 0 2024-02-28T23:07:49.024 INFO:tasks.workunit.client.1.smithi084.stdout:8/872: write d0/d24/d6e/fc5 [3745083,43692] 0 2024-02-28T23:07:49.026 INFO:tasks.workunit.client.1.smithi084.stdout:4/848: readlink d1/d18/d20/d24/d2f/d3e/lc5 0 2024-02-28T23:07:49.027 INFO:tasks.workunit.client.0.smithi057.stdout:4/700: dwrite d1a/d48/d97/f93 [0,4194304] 0 2024-02-28T23:07:49.028 INFO:tasks.workunit.client.1.smithi084.stdout:6/868: rename d1/d13/d2e/d41/d2c/f71 to d1/d13/d47/f122 0 2024-02-28T23:07:49.029 INFO:tasks.workunit.client.0.smithi057.stdout:4/701: chown d1a/c66 1765044 1 2024-02-28T23:07:49.030 INFO:tasks.workunit.client.1.smithi084.stdout:1/915: sync 2024-02-28T23:07:49.033 INFO:tasks.workunit.client.0.smithi057.stdout:3/625: rmdir d3/d4/d7/d9a 39 2024-02-28T23:07:49.033 INFO:tasks.workunit.client.1.smithi084.stdout:5/889: creat dc/de/df/db0/f14e x:0 0 0 2024-02-28T23:07:49.035 INFO:tasks.workunit.client.0.smithi057.stdout:3/626: write d3/d4/d1d/d47/d62/fa5 [823280,15046] 0 2024-02-28T23:07:49.037 INFO:tasks.workunit.client.1.smithi084.stdout:6/869: write d1/d9/d1d/fe5 [819666,48103] 0 2024-02-28T23:07:49.037 INFO:tasks.workunit.client.0.smithi057.stdout:8/769: creat d4/d13/d18/d62/d66/d74/f110 x:0 0 0 2024-02-28T23:07:49.039 INFO:tasks.workunit.client.1.smithi084.stdout:3/935: dread d3/d27/d7d/db0/d1a/d45/d5e/fe2 [0,4194304] 0 2024-02-28T23:07:49.043 INFO:tasks.workunit.client.1.smithi084.stdout:9/933: rename d4/d6/d15/d75/d9d/caf to d4/d32/d3f/d89/d10c/c12e 0 2024-02-28T23:07:49.045 INFO:tasks.workunit.client.0.smithi057.stdout:3/627: mknod d3/d4/d1d/d47/cc9 0 2024-02-28T23:07:49.046 INFO:tasks.workunit.client.1.smithi084.stdout:9/934: chown d4/d6/d15/d75/d9d/dd0/ff8 1067709 1 2024-02-28T23:07:49.051 INFO:tasks.workunit.client.1.smithi084.stdout:7/975: rmdir db/d5e/d8a/d130 0 2024-02-28T23:07:49.054 INFO:tasks.workunit.client.1.smithi084.stdout:6/870: fdatasync d1/d13/d50/dbd/dc4/ff2 0 2024-02-28T23:07:49.058 INFO:tasks.workunit.client.1.smithi084.stdout:4/849: creat d1/d18/d20/d14/d30/d47/da4/d10f/d115/f119 x:0 0 0 2024-02-28T23:07:49.059 INFO:tasks.workunit.client.1.smithi084.stdout:9/935: read d4/d6/d15/d12/d41/d55/f87 [1074596,107009] 0 2024-02-28T23:07:49.067 INFO:tasks.workunit.client.1.smithi084.stdout:3/936: symlink d3/d27/d7d/db0/d1a/df9/dfe/l15b 0 2024-02-28T23:07:49.073 INFO:tasks.workunit.client.1.smithi084.stdout:2/795: dread d5/d7/d9/d94/f9e [0,4194304] 0 2024-02-28T23:07:49.081 INFO:tasks.workunit.client.1.smithi084.stdout:6/871: symlink d1/d13/d47/d91/dba/de9/l123 0 2024-02-28T23:07:49.084 INFO:tasks.workunit.client.1.smithi084.stdout:4/850: dwrite d1/f9c [0,4194304] 0 2024-02-28T23:07:49.088 INFO:tasks.workunit.client.1.smithi084.stdout:0/907: sync 2024-02-28T23:07:49.088 INFO:tasks.workunit.client.1.smithi084.stdout:7/976: creat db/d10/d17/d9c/d9f/db1/d115/f138 x:0 0 0 2024-02-28T23:07:49.088 INFO:tasks.workunit.client.1.smithi084.stdout:2/796: creat d5/d7/d12/d24/ff0 x:0 0 0 2024-02-28T23:07:49.088 INFO:tasks.workunit.client.1.smithi084.stdout:4/851: stat d1/d18/d20/d24/le3 0 2024-02-28T23:07:49.090 INFO:tasks.workunit.client.1.smithi084.stdout:3/937: mkdir d3/d50/d127/d14c/d15c 0 2024-02-28T23:07:49.090 INFO:tasks.workunit.client.0.smithi057.stdout:1/648: dread d2/d71/d2d/f49 [4194304,4194304] 0 2024-02-28T23:07:49.091 INFO:tasks.workunit.client.1.smithi084.stdout:6/872: truncate d1/d13/d2e/d41/d4d/d6c/fe7 852331 0 2024-02-28T23:07:49.092 INFO:tasks.workunit.client.1.smithi084.stdout:9/936: mkdir d4/d12f 0 2024-02-28T23:07:49.094 INFO:tasks.workunit.client.1.smithi084.stdout:3/938: write d3/d50/d117/d3d/d43/d61/dce/d109/f126 [3886469,18202] 0 2024-02-28T23:07:49.096 INFO:tasks.workunit.client.1.smithi084.stdout:0/908: rmdir d1/d6/df6 39 2024-02-28T23:07:49.097 INFO:tasks.workunit.client.0.smithi057.stdout:1/649: link d2/d71/d62/fcf d2/d71/d2d/d5b/fe7 0 2024-02-28T23:07:49.100 INFO:tasks.workunit.client.1.smithi084.stdout:7/977: mkdir db/d10/d17/d64/d139 0 2024-02-28T23:07:49.103 INFO:tasks.workunit.client.1.smithi084.stdout:4/852: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f11a x:0 0 0 2024-02-28T23:07:49.104 INFO:tasks.workunit.client.0.smithi057.stdout:1/650: creat d2/d7/df/db/d3a/d2a/d82/fe8 x:0 0 0 2024-02-28T23:07:49.106 INFO:tasks.workunit.client.0.smithi057.stdout:2/820: truncate d8/d11/fcf 2387324 0 2024-02-28T23:07:49.108 INFO:tasks.workunit.client.0.smithi057.stdout:9/710: fsync dd/d46/d6f/fa0 0 2024-02-28T23:07:49.109 INFO:tasks.workunit.client.1.smithi084.stdout:2/797: mknod d5/dd/cf1 0 2024-02-28T23:07:49.110 INFO:tasks.workunit.client.0.smithi057.stdout:8/770: write d4/d13/d18/d62/d66/fd8 [541902,47307] 0 2024-02-28T23:07:49.112 INFO:tasks.workunit.client.0.smithi057.stdout:2/821: write d8/d11/d21/d28/d49/d85/fe5 [749658,28028] 0 2024-02-28T23:07:49.112 INFO:tasks.workunit.client.0.smithi057.stdout:6/682: sync 2024-02-28T23:07:49.112 INFO:tasks.workunit.client.0.smithi057.stdout:0/843: sync 2024-02-28T23:07:49.113 INFO:tasks.workunit.client.1.smithi084.stdout:0/909: creat d1/d6/db/d35/d44/da9/f123 x:0 0 0 2024-02-28T23:07:49.116 INFO:tasks.workunit.client.1.smithi084.stdout:3/939: mkdir d3/d27/d7d/db0/d1a/d45/d86/d15d 0 2024-02-28T23:07:49.116 INFO:tasks.workunit.client.1.smithi084.stdout:0/910: fsync d1/d6/d59/d70/f73 0 2024-02-28T23:07:49.117 INFO:tasks.workunit.client.0.smithi057.stdout:9/711: creat dd/de/d4a/d60/fe3 x:0 0 0 2024-02-28T23:07:49.119 INFO:tasks.workunit.client.1.smithi084.stdout:7/978: creat db/d10/d18/db7/de9/dfa/dc7/d100/f13a x:0 0 0 2024-02-28T23:07:49.123 INFO:tasks.workunit.client.0.smithi057.stdout:4/702: dwrite d1a/d48/f79 [0,4194304] 0 2024-02-28T23:07:49.124 INFO:tasks.workunit.client.0.smithi057.stdout:6/683: symlink d0/d1b/d3e/d40/le8 0 2024-02-28T23:07:49.124 INFO:tasks.workunit.client.0.smithi057.stdout:8/771: creat d4/df7/f111 x:0 0 0 2024-02-28T23:07:49.128 INFO:tasks.workunit.client.1.smithi084.stdout:3/940: mkdir d3/d50/d117/d3d/d43/d61/dce/d15e 0 2024-02-28T23:07:49.133 INFO:tasks.workunit.client.0.smithi057.stdout:4/703: chown d1a/d1e/f25 250698 1 2024-02-28T23:07:49.134 INFO:tasks.workunit.client.1.smithi084.stdout:6/873: rename d1/d13/d47/d91/cf1 to d1/d9/d1d/c124 0 2024-02-28T23:07:49.135 INFO:tasks.workunit.client.0.smithi057.stdout:9/712: mkdir dd/de/d24/d63/d77/dcf/d40/d44/da5/de4 0 2024-02-28T23:07:49.136 INFO:tasks.workunit.client.0.smithi057.stdout:1/651: truncate d2/d7/df/db/f28 6393157 0 2024-02-28T23:07:49.136 INFO:tasks.workunit.client.0.smithi057.stdout:9/713: stat dd/d46/dc4 0 2024-02-28T23:07:49.137 INFO:tasks.workunit.client.0.smithi057.stdout:6/684: creat d0/d17/d20/fe9 x:0 0 0 2024-02-28T23:07:49.139 INFO:tasks.workunit.client.0.smithi057.stdout:7/740: sync 2024-02-28T23:07:49.139 INFO:tasks.workunit.client.0.smithi057.stdout:5/739: sync 2024-02-28T23:07:49.140 INFO:tasks.workunit.client.0.smithi057.stdout:4/704: mknod d1a/d2f/db2/d49/d4d/cf2 0 2024-02-28T23:07:49.141 INFO:tasks.workunit.client.1.smithi084.stdout:0/911: dwrite d1/d6/db/d35/d44/fe3 [0,4194304] 0 2024-02-28T23:07:49.142 INFO:tasks.workunit.client.0.smithi057.stdout:3/628: dread d3/d4/d1d/f36 [0,4194304] 0 2024-02-28T23:07:49.142 INFO:tasks.workunit.client.0.smithi057.stdout:6/685: chown d0/d4/d25/d60/fd8 390400470 1 2024-02-28T23:07:49.142 INFO:tasks.workunit.client.0.smithi057.stdout:4/705: chown d1a/d1e/d81 25249609 1 2024-02-28T23:07:49.143 INFO:tasks.workunit.client.0.smithi057.stdout:0/844: link d17/d20/d26/d48/l5f d17/d20/db6/l12e 0 2024-02-28T23:07:49.144 INFO:tasks.workunit.client.0.smithi057.stdout:5/740: dread - d4/d21/d24/d30/d62/fd5 zero size 2024-02-28T23:07:49.146 INFO:tasks.workunit.client.1.smithi084.stdout:3/941: creat d3/d27/d7d/db0/d14/dee/f15f x:0 0 0 2024-02-28T23:07:49.147 INFO:tasks.workunit.client.1.smithi084.stdout:6/874: fsync d1/d9/f28 0 2024-02-28T23:07:49.147 INFO:tasks.workunit.client.1.smithi084.stdout:7/979: mkdir db/d10/d18/db7/de9/dfa/dc7/d100/d12a/d13b 0 2024-02-28T23:07:49.148 INFO:tasks.workunit.client.0.smithi057.stdout:5/741: chown d4/d7/d8/ff 1983 1 2024-02-28T23:07:49.151 INFO:tasks.workunit.client.0.smithi057.stdout:9/714: readlink dd/d46/d4d/l58 0 2024-02-28T23:07:49.152 INFO:tasks.workunit.client.0.smithi057.stdout:1/652: readlink d2/d33/daa/lb6 0 2024-02-28T23:07:49.152 INFO:tasks.workunit.client.0.smithi057.stdout:6/686: symlink d0/d4/d11/lea 0 2024-02-28T23:07:49.152 INFO:tasks.workunit.client.1.smithi084.stdout:7/980: readlink db/d5e/dc3/ld0 0 2024-02-28T23:07:49.153 INFO:tasks.workunit.client.1.smithi084.stdout:0/912: dread - d1/d6/db/d35/d82/f100 zero size 2024-02-28T23:07:49.153 INFO:tasks.workunit.client.0.smithi057.stdout:5/742: chown d4/d7/d8/ld1 4757233 1 2024-02-28T23:07:49.158 INFO:tasks.workunit.client.0.smithi057.stdout:4/706: link d1a/d2f/db2/d49/fc8 d1a/d2f/db2/d49/d4d/dc9/dec/ff3 0 2024-02-28T23:07:49.160 INFO:tasks.workunit.client.0.smithi057.stdout:9/715: mkdir dd/d46/dc4/de5 0 2024-02-28T23:07:49.161 INFO:tasks.workunit.client.0.smithi057.stdout:3/629: creat d3/d4/fca x:0 0 0 2024-02-28T23:07:49.161 INFO:tasks.workunit.client.1.smithi084.stdout:9/937: dread d4/d6/d15/d12/f3d [0,4194304] 0 2024-02-28T23:07:49.163 INFO:tasks.workunit.client.1.smithi084.stdout:7/981: mknod db/d5e/dce/c13c 0 2024-02-28T23:07:49.165 INFO:tasks.workunit.client.0.smithi057.stdout:3/630: dread - d3/d8/d27/d70/f9d zero size 2024-02-28T23:07:49.166 INFO:tasks.workunit.client.1.smithi084.stdout:7/982: chown db/d5e/d87/c92 0 1 2024-02-28T23:07:49.170 INFO:tasks.workunit.client.1.smithi084.stdout:0/913: rename d1/d6/db/d35/d82/le9 to d1/d6/db/d35/d82/l124 0 2024-02-28T23:07:49.172 INFO:tasks.workunit.client.1.smithi084.stdout:9/938: creat d4/d32/d3f/d89/db8/f130 x:0 0 0 2024-02-28T23:07:49.173 INFO:tasks.workunit.client.1.smithi084.stdout:0/914: write d1/d8/f3e [3620924,46938] 0 2024-02-28T23:07:49.175 INFO:tasks.workunit.client.0.smithi057.stdout:4/707: creat d1a/d48/db0/db1/ff4 x:0 0 0 2024-02-28T23:07:49.177 INFO:tasks.workunit.client.0.smithi057.stdout:3/631: dread - d3/d8/d1b/fa4 zero size 2024-02-28T23:07:49.177 INFO:tasks.workunit.client.0.smithi057.stdout:1/653: rmdir d2/d7/df/db/d12/d19/d48/dc0 0 2024-02-28T23:07:49.177 INFO:tasks.workunit.client.0.smithi057.stdout:6/687: creat d0/d4/feb x:0 0 0 2024-02-28T23:07:49.178 INFO:tasks.workunit.client.1.smithi084.stdout:9/939: rmdir d4/d6/d15/d12 39 2024-02-28T23:07:49.179 INFO:tasks.workunit.client.0.smithi057.stdout:0/845: dread d17/d20/f2b [0,4194304] 0 2024-02-28T23:07:49.179 INFO:tasks.workunit.client.0.smithi057.stdout:8/772: write d4/d13/d1a/d4a/db3/dfc/fa0 [789895,117484] 0 2024-02-28T23:07:49.180 INFO:tasks.workunit.client.0.smithi057.stdout:3/632: chown d3/d4f/d4c 22595377 1 2024-02-28T23:07:49.180 INFO:tasks.workunit.client.0.smithi057.stdout:1/654: stat d2/d7/d4f/db1/ccc 0 2024-02-28T23:07:49.181 INFO:tasks.workunit.client.1.smithi084.stdout:0/915: symlink d1/d8/d69/l125 0 2024-02-28T23:07:49.182 INFO:tasks.workunit.client.1.smithi084.stdout:9/940: read d4/f85 [1410593,6630] 0 2024-02-28T23:07:49.183 INFO:tasks.workunit.client.1.smithi084.stdout:8/873: truncate d0/d24/d6e/fc5 1816396 0 2024-02-28T23:07:49.191 INFO:tasks.workunit.client.1.smithi084.stdout:7/983: dread db/d10/d18/db7/fbe [0,4194304] 0 2024-02-28T23:07:49.193 INFO:tasks.workunit.client.1.smithi084.stdout:9/941: mkdir d4/d6/d15/d75/d131 0 2024-02-28T23:07:49.193 INFO:tasks.workunit.client.1.smithi084.stdout:5/890: dwrite dc/de/df/d16/fa0 [0,4194304] 0 2024-02-28T23:07:49.195 INFO:tasks.workunit.client.0.smithi057.stdout:2/822: dread d8/d11/d21/d28/d41/f5c [0,4194304] 0 2024-02-28T23:07:49.199 INFO:tasks.workunit.client.1.smithi084.stdout:1/916: dwrite d12/d1b/d2e/d32/f5e [0,4194304] 0 2024-02-28T23:07:49.204 INFO:tasks.workunit.client.1.smithi084.stdout:0/916: rename d1/d6/db/d35/d44/f6f to d1/d6/db/d35/d4a/d56/f126 0 2024-02-28T23:07:49.207 INFO:tasks.workunit.client.1.smithi084.stdout:3/942: dread d3/d27/d7d/db0/f72 [0,4194304] 0 2024-02-28T23:07:49.207 INFO:tasks.workunit.client.1.smithi084.stdout:6/875: dread d1/d9/d1d/db6/ffd [0,4194304] 0 2024-02-28T23:07:49.209 INFO:tasks.workunit.client.0.smithi057.stdout:7/741: dread d0/d5/d36/d4e/d15/f2b [0,4194304] 0 2024-02-28T23:07:49.499 INFO:tasks.workunit.client.0.smithi057.stdout:2/823: dwrite d8/d11/d15/fc4 [0,4194304] 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:6/688: dread d0/d4/d25/f2a [0,4194304] 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:0/846: unlink d17/d20/d4a/l57 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:3/633: fsync d3/d8/d27/f3e 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:7/742: symlink d0/d5/d36/d4e/d15/d29/d71/lf8 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:9/716: link dd/de/d24/fa6 dd/de/d24/d63/d77/dcf/fe6 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:4/708: mknod d1a/cf5 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:8/773: mknod d4/c112 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:7/743: chown d0/d5/d36/d4e/d15/d29/d3d/d51/lc3 54685 1 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:7/744: stat d0/d5/d25/d80/d9f/fdb 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:4/709: fsync d1a/d48/d97/f93 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:1/655: creat d2/fe9 x:0 0 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:9/717: rename dd/de/d24/d63/d77/dcf/d40/fc3 to dd/de/d24/d63/d77/dcf/d40/d44/d5c/fe7 0 2024-02-28T23:07:49.500 INFO:tasks.workunit.client.0.smithi057.stdout:8/774: mkdir d4/de9/d56/d6a/d54/d61/d113 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:7/745: creat d0/d5/d25/d80/d9f/ff9 x:0 0 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:4/710: truncate f14 4108441 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/847: getdents d17/d20/d26/d48 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:3/634: dwrite d3/d20/d2d/f3b [0,4194304] 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/848: write d17/d20/d26/d30/fad [3700960,109630] 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:9/718: creat dd/de/d14/dc2/fe8 x:0 0 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/849: chown d17/d20/d26/d48 801015 1 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:4/711: rename d1a/d2f/db2/d49/fc8 to d1a/d48/db0/ff6 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:8/775: symlink d4/l114 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:4/712: chown d1a/d2f/db2/f35 710306 1 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:0/850: truncate d17/d20/d4a/d78/f110 815780 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:9/719: truncate dd/de/d14/f55 3224031 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:7/746: dwrite d0/d5/d25/d3e/fa9 [0,4194304] 0 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:7/747: chown d0/d3f/d76/dc1/dea/f9d 132 1 2024-02-28T23:07:49.501 INFO:tasks.workunit.client.0.smithi057.stdout:3/635: creat d3/d4/d1d/fcb x:0 0 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:8/776: dwrite d4/de9/d59/f98 [0,4194304] 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:0/851: creat d17/d20/d4a/d78/dd4/d123/f12f x:0 0 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:7/748: dwrite d0/fd0 [0,4194304] 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:9/720: truncate dd/de/d24/d63/d77/dcf/d40/d44/d5c/f8a 705846 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:5/743: sync 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:4/713: link d1a/d48/d9c/l28 d1a/d48/d9c/d2e/lf7 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:0/852: symlink d17/d20/d26/l130 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:9/721: rename dd/de/d14/d48/cc7 to dd/de/d24/d63/d77/dcf/d40/d44/da5/ce9 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:3/636: creat d3/d4/d7/d9a/fcc x:0 0 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:8/777: mknod d4/d13/d18/d10d/c115 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:9/722: chown dd/de/d4a/d60/d8c/fda 4821109 1 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:4/714: mkdir d1a/d48/d9c/d2e/df8 0 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:9/723: chown dd/de/d24/d63/d77/dcf/d40/d44/d5c/cd4 893141282 1 2024-02-28T23:07:49.502 INFO:tasks.workunit.client.0.smithi057.stdout:7/749: mknod d0/df3/cfa 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/824: write d8/f19 [1321942,3250] 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:3/637: dwrite d3/d4/f30 [4194304,4194304] 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:5/744: getdents d4/d9/d14/d3b 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:8/778: dwrite d4/f3a [0,4194304] 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:7/750: creat d0/d5/d25/d3e/ffb x:0 0 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/825: symlink d8/d11/d21/de6/lfd 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:5/745: rename d4/d9/d14/f5f to d4/fe6 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/826: truncate d8/d11/d15/daf/fb7 3098093 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:1/656: sync 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:6/689: sync 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:3/638: dwrite d3/d20/f24 [0,4194304] 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/827: write d8/d11/d21/d28/d49/d85/d9b/ddc/fe4 [39981,88308] 0 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/828: chown d8/d11/d56/c57 2381 1 2024-02-28T23:07:49.503 INFO:tasks.workunit.client.0.smithi057.stdout:6/690: dread - d0/d17/d20/fe9 zero size 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:7/751: getdents d0/d3f 0 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:2/829: chown d8/l2b 1 1 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:7/752: chown d0/d5/f30 143526105 1 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:9/724: truncate dd/de/d14/d48/d4b/d41/fcb 3699103 0 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:1/657: creat d2/d7/df/db/d12/d3f/fea x:0 0 0 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:6/691: creat d0/d4/d11/fec x:0 0 0 2024-02-28T23:07:49.504 INFO:tasks.workunit.client.0.smithi057.stdout:5/746: dread d4/d9/d16/f2b [0,4194304] 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:8/779: dread d4/d13/d1a/f20 [0,4194304] 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:8/780: dread - d4/de9/d56/d6a/ff4 zero size 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:1/658: truncate d2/d7/df/db/d12/d3f/fea 931097 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:3/639: write d3/d8/d27/f90 [2693955,7404] 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:2/830: truncate d8/d11/ffa 3520099 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:8/781: mknod d4/d13/d18/d10d/c116 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:5/747: mknod d4/d9/d16/d8f/ce7 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:3/640: readlink d3/d4/l6f 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:6/692: dread d0/d4/fa [0,4194304] 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:0/853: dread d17/d20/db6/fc9 [0,4194304] 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:9/725: getdents dd/de/d24/d63/d77/dcf 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:2/831: symlink d8/d11/d56/lfe 0 2024-02-28T23:07:49.505 INFO:tasks.workunit.client.0.smithi057.stdout:3/641: creat d3/d4f/d4c/fcd x:0 0 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:6/693: creat d0/d1b/fed x:0 0 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:4/715: dread d1a/d48/d97/fc0 [0,4194304] 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:4/716: chown d1a/d1e/d51/l73 7473158 1 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:1/659: link d2/d7/df/db/d3a/fc9 d2/d7/d4f/db1/feb 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:0/854: fdatasync d17/d20/d23/d47/f9f 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:6/694: rmdir d0/d17/d20 39 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:3/642: creat d3/d4f/d4c/fce x:0 0 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:6/695: stat d0/d17/d94/da0/de6/l95 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:2/832: creat d8/d11/d21/d28/d49/d85/d92/dc5/fff x:0 0 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:9/726: creat dd/de/d24/d63/d77/dcf/d40/d44/da5/fea x:0 0 0 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:9/727: chown dd/de/d24 474445 1 2024-02-28T23:07:49.506 INFO:tasks.workunit.client.0.smithi057.stdout:4/717: creat d1a/d48/d9c/d2e/d8f/db7/ff9 x:0 0 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:1/917: mkdir d12/d1b/d2e/d52/d61/d74/d12a 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:8/874: dread d0/d11/f95 [0,4194304] 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:4/853: truncate d1/d18/d20/d24/d2f/d3e/d52/f8c 1850558 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:2/798: truncate d5/f31 3484122 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:4/854: chown d1/d18/d20/d14/d30/d47/da4/d10f/d115 70355 1 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:4/855: readlink d1/d18/d20/d24/le3 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:9/942: symlink d4/d32/d3f/l132 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:6/876: fsync d1/d88/dbf/fde 0 2024-02-28T23:07:49.508 INFO:tasks.workunit.client.1.smithi084.stdout:4/856: write d1/f97 [1414022,59426] 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/917: creat d1/d6/db/d35/d44/da9/f127 x:0 0 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/918: write d1/d8/d18/d4c/d6c/d74/fe0 [1908953,106173] 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:8/875: rmdir d0/d9/d30/db6 39 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:5/891: write dc/de/df/d31/f7e [416090,61224] 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:1/918: rmdir d12/d1b/d2e/d11e 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:2/799: creat d5/d1f/d42/d8f/ff2 x:0 0 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:4/857: unlink d1/d18/d20/d14/d101/c104 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:7/984: dwrite db/d10/d17/d9c/d9f/db1/d36/f86 [0,4194304] 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/919: mknod d1/d8/d18/d4c/c128 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:9/943: creat d4/d32/d3f/f133 x:0 0 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:8/876: mknod d0/d9/d30/db6/c11b 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:4/858: mkdir d1/d18/d20/d65/d8e/d55/dc3/dec/d11b 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:0/920: symlink d1/d6/db/d76/dee/df2/l129 0 2024-02-28T23:07:49.509 INFO:tasks.workunit.client.1.smithi084.stdout:6/877: dwrite d1/d13/d50/d74/d9e/fb9 [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:3/943: dwrite d3/fc0 [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:0/921: read d1/d6/db/d35/d44/f4e [319274,4089] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:4/859: read d1/f7 [1943112,116260] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:9/944: creat d4/d6/d53/f134 x:0 0 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:6/878: chown d1/d13/d2e/d41/d4d/d6c/fe7 0 1 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:8/877: getdents d0 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:2/800: dwrite d5/d1f/d42/d74/fd0 [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:5/892: dwrite dc/de/f14 [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:1/919: dwrite d12/d84/fcf [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:6/879: creat d1/d13/d47/d91/dba/de9/f125 x:0 0 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:8/878: link d0/d24/d4d/ffe d0/d9/d1b/dc1/f11c 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:2/801: creat d5/d7/d75/dcd/ff3 x:0 0 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:9/945: truncate d4/d32/d3f/d89/d10c/d10f/d11a/f11d 774090 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:7/985: dread db/d5e/d80/fec [0,4194304] 0 2024-02-28T23:07:49.510 INFO:tasks.workunit.client.1.smithi084.stdout:9/946: chown d4/d6/d15/d75/d2d/dbd/f108 3601588 1 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:2/802: symlink d5/d7/d2e/lf4 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:1/920: rename d12/d1b/d2e/d52/d61/l121 to d12/d1b/d63/l12b 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:9/947: truncate d4/d6/d15/d75/d9d/dc3/dce/f11f 870204 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:2/803: mknod d5/d7/d9/d7c/d5b/d9f/cf5 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:2/804: readlink d5/d1f/l36 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:9/948: creat d4/d32/d3f/d89/d10c/d10f/f135 x:0 0 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:1/921: truncate d12/d1b/d2e/d52/f5a 633798 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:0/922: dwrite d1/d8/d18/d4c/d6c/fbd [0,4194304] 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:9/949: read d4/f9c [4574,18434] 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:2/805: symlink d5/dd/d69/lf6 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:1/922: fsync d12/d1b/f4a 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:1/923: readlink d12/d1b/d63/d7d/lca 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:6/880: write d1/fdb [170153,28137] 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:4/860: dread d1/d18/d20/d65/d8e/f5a [0,4194304] 0 2024-02-28T23:07:49.511 INFO:tasks.workunit.client.1.smithi084.stdout:8/879: write d0/d9/d30/d33/d71/fa4 [1712766,6506] 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:7/986: write db/d10/d17/d9c/d9f/db1/d36/faf [3278896,74531] 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:5/893: truncate dc/de/df/d31/f89 1220255 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:6/881: readlink d1/d13/d2e/d41/l97 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:3/944: dread d3/d27/d7d/db0/d1a/d45/f78 [0,4194304] 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:1/924: chown d12/d1b/d2e/d52/cd1 63533 1 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:0/923: link d1/d8/d5f/d93/fe2 d1/d8/f12a 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:0/924: dread - d1/f11b zero size 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:8/880: creat d0/d24/d4d/f11d x:0 0 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:6/882: symlink d1/d88/l126 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:8/881: fdatasync d0/d9/d30/d33/d71/d84/dd9/de3/df7/ffa 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:9/950: dwrite d4/f85 [4194304,4194304] 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:4/861: creat d1/d18/d20/d14/d30/d47/da4/d10f/d115/f11c x:0 0 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:3/945: write d3/d27/d7d/dd9/fca [4095566,39712] 0 2024-02-28T23:07:49.512 INFO:tasks.workunit.client.1.smithi084.stdout:9/951: chown d4/d6/d15/ca3 7 1 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:8/882: write d0/d9/d30/f11a [31283,77914] 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:1/925: creat d12/d1b/d2e/d30/d50/f12c x:0 0 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:3/946: chown d3/d27/d7d/db0/df/ded 45176488 1 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:8/883: read d0/d9/d30/db6/fc8 [251510,27455] 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:3/947: chown d3/d27/d7d/db0/d1a/d45 7 1 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:6/883: chown d1/d13/d2e/d41/d2c/dfb/c10d 11876038 1 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:5/894: creat dc/de/df/d16/f14f x:0 0 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:1/926: rename d12/d1b/d2e/d32 to d12/d1b/d2e/d30/d12d 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:1/927: readlink d12/d1b/d2e/d30/l2d 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:8/884: truncate d0/d9/d1b/d38/d43/d51/ff5 1727973 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:0/925: dwrite d1/d8/d69/db2/f110 [0,4194304] 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:0/926: chown d1/d8/d18/d4c/d6c/d74 336886 1 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:6/884: link d1/d13/d2e/d41/dbb/fb0 d1/d13/d2e/d41/dbb/f127 0 2024-02-28T23:07:49.513 INFO:tasks.workunit.client.1.smithi084.stdout:6/885: stat d1/d13/d2e/d41/d4d/d6c 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:6/886: readlink d1/d13/d47/l114 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:8/885: dread d0/d9/d30/f111 [0,4194304] 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:6/887: write d1/d13/d50/f8d [5050822,71461] 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:5/895: link dc/de/df/d31/dd1/c138 dc/de/df/d31/d8e/da9/dab/c150 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:8/886: rmdir d0/d9/d1b/d20 39 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:0/927: symlink d1/d6/db/d35/d44/d113/l12b 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:6/888: stat d1/d9/d1d/d1e/d33/f75 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:0/928: chown d1/d8/d18/f9a 414 1 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:0/929: fsync d1/d8/d69/d6b/f7a 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:8/887: mknod d0/d9/d1b/c11e 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:7/987: write db/d10/d17/d9c/d9f/db1/d7a/fac [3608343,31088] 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:0/930: chown d1/d6/db 489872 1 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:6/889: mkdir d1/d9/d1d/d1e/d33/d128 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:3/948: write d3/d27/d7d/db0/df/f12 [1733335,107864] 0 2024-02-28T23:07:49.514 INFO:tasks.workunit.client.1.smithi084.stdout:0/931: stat d1/d6/db/d37/ffd 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:8/888: unlink d0/fdb 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:9/952: dread d4/d6/d15/d12/d41/d55/fbc [0,4194304] 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:1/928: write d12/d1b/d2e/d52/fe3 [3808086,44194] 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:6/890: creat d1/d13/d47/f129 x:0 0 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:3/949: creat d3/d27/d7d/db0/d1a/d29/f160 x:0 0 0 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:6/891: dread - d1/d13/d50/dbd/dc4/ff2 zero size 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:9/953: rmdir d4/d6/d15/d12/d41/d55 39 2024-02-28T23:07:49.515 INFO:tasks.workunit.client.1.smithi084.stdout:3/950: dread - d3/d50/d117/d7b/f105 zero size 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.0.smithi057.stdout:1/660: fsync d2/d7/df/db/d12/f5e 0 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.0.smithi057.stdout:5/748: getdents d4/d9/d14 0 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.0.smithi057.stdout:3/643: truncate d3/f61 230343 0 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.0.smithi057.stdout:7/753: dwrite d0/d5/d36/d4e/d15/f1a [0,4194304] 0 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.1.smithi084.stdout:6/892: rmdir d1/d13/d50/d74/d81 39 2024-02-28T23:07:49.516 INFO:tasks.workunit.client.1.smithi084.stdout:5/896: dwrite dc/de/df/d16/d34/f4d [0,4194304] 0 2024-02-28T23:07:49.517 INFO:tasks.workunit.client.1.smithi084.stdout:7/988: dwrite db/d10/d17/fae [0,4194304] 0 2024-02-28T23:07:49.518 INFO:tasks.workunit.client.1.smithi084.stdout:1/929: link d12/d84/f8e d12/d1b/d63/f12e 0 2024-02-28T23:07:49.518 INFO:tasks.workunit.client.1.smithi084.stdout:3/951: chown d3/d27/d7d/db0/d1a/d29/d41/d60/ca4 250 1 2024-02-28T23:07:49.519 INFO:tasks.workunit.client.1.smithi084.stdout:0/932: dwrite d1/d6/d59/d70/f118 [0,4194304] 0 2024-02-28T23:07:49.521 INFO:tasks.workunit.client.0.smithi057.stdout:6/696: rename d0/d1b/d3e/l44 to d0/d1b/d3e/d3d/d42/lee 0 2024-02-28T23:07:49.524 INFO:tasks.workunit.client.0.smithi057.stdout:0/855: chown d17/d20/d23/d47/d60/d6f/lb3 403830355 1 2024-02-28T23:07:49.525 INFO:tasks.workunit.client.0.smithi057.stdout:1/661: rmdir d2/d7/df/db/d12/d19/da4 39 2024-02-28T23:07:49.526 INFO:tasks.workunit.client.1.smithi084.stdout:8/889: chown d0/d9/d1b/d38/l36 14394665 1 2024-02-28T23:07:49.527 INFO:tasks.workunit.client.1.smithi084.stdout:1/930: dread d12/da2/fe4 [0,4194304] 0 2024-02-28T23:07:49.530 INFO:tasks.workunit.client.1.smithi084.stdout:7/989: truncate db/fe3 3550601 0 2024-02-28T23:07:49.533 INFO:tasks.workunit.client.1.smithi084.stdout:0/933: mknod d1/d6/db/d35/d44/d92/c12c 0 2024-02-28T23:07:49.533 INFO:tasks.workunit.client.0.smithi057.stdout:7/754: mknod d0/d5/d36/d4e/d15/d29/d3d/de7/cfc 0 2024-02-28T23:07:49.534 INFO:tasks.workunit.client.0.smithi057.stdout:1/662: creat d2/d7/d4f/d5c/fec x:0 0 0 2024-02-28T23:07:49.535 INFO:tasks.workunit.client.1.smithi084.stdout:3/952: dread d3/d50/d117/d3d/d43/de3/f149 [0,4194304] 0 2024-02-28T23:07:49.538 INFO:tasks.workunit.client.1.smithi084.stdout:9/954: dwrite d4/d6/d15/d12/d41/ff6 [0,4194304] 0 2024-02-28T23:07:49.539 INFO:tasks.workunit.client.0.smithi057.stdout:6/697: link d0/d1b/d3e/c8c d0/d4/d25/d60/d6a/cef 0 2024-02-28T23:07:49.543 INFO:tasks.workunit.client.0.smithi057.stdout:1/663: symlink d2/d71/d2d/d39/da3/db5/led 0 2024-02-28T23:07:49.544 INFO:tasks.workunit.client.0.smithi057.stdout:5/749: rename d4/d21/d24/da2/dbb/d95/f98 to d4/d9/d16/d4e/fe8 0 2024-02-28T23:07:49.546 INFO:tasks.workunit.client.1.smithi084.stdout:0/934: unlink d1/d8/f3f 0 2024-02-28T23:07:49.550 INFO:tasks.workunit.client.1.smithi084.stdout:8/890: mkdir d0/d24/d78/df0/d11f 0 2024-02-28T23:07:49.550 INFO:tasks.workunit.client.1.smithi084.stdout:9/955: symlink d4/d6/d15/d12/d41/dea/l136 0 2024-02-28T23:07:49.550 INFO:tasks.workunit.client.1.smithi084.stdout:8/891: fdatasync d0/d9/d1b/dc1/fc7 0 2024-02-28T23:07:49.554 INFO:tasks.workunit.client.0.smithi057.stdout:5/750: creat d4/d9/d14/d3b/d61/db2/fe9 x:0 0 0 2024-02-28T23:07:49.555 INFO:tasks.workunit.client.0.smithi057.stdout:2/833: dwrite d8/d11/d15/f35 [0,4194304] 0 2024-02-28T23:07:49.557 INFO:tasks.workunit.client.1.smithi084.stdout:8/892: rename d0/d9/l3d to d0/d24/d78/l120 0 2024-02-28T23:07:49.561 INFO:tasks.workunit.client.1.smithi084.stdout:9/956: write d4/d6/d15/d75/d9d/f11b [2584108,90985] 0 2024-02-28T23:07:49.563 INFO:tasks.workunit.client.0.smithi057.stdout:5/751: creat d4/d9/ddd/fea x:0 0 0 2024-02-28T23:07:49.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:49 smithi057 ceph-mon[29096]: pgmap v31: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 103 MiB/s rd, 253 MiB/s wr, 512 op/s 2024-02-28T23:07:49.569 INFO:tasks.workunit.client.1.smithi084.stdout:9/957: dread d4/d6/d15/d75/d9d/dc3/dce/f11f [0,4194304] 0 2024-02-28T23:07:49.570 INFO:tasks.workunit.client.1.smithi084.stdout:6/893: dread d1/d9/f3b [0,4194304] 0 2024-02-28T23:07:49.573 INFO:tasks.workunit.client.0.smithi057.stdout:5/752: fsync d4/d9/d14/f43 0 2024-02-28T23:07:49.576 INFO:tasks.workunit.client.1.smithi084.stdout:6/894: write d1/d9/fd5 [3792694,44156] 0 2024-02-28T23:07:49.580 INFO:tasks.workunit.client.1.smithi084.stdout:0/935: dwrite d1/d8/d18/d4c/d6c/d74/f97 [0,4194304] 0 2024-02-28T23:07:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:49 smithi084 ceph-mon[34826]: pgmap v31: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 103 MiB/s rd, 253 MiB/s wr, 512 op/s 2024-02-28T23:07:49.581 INFO:tasks.workunit.client.0.smithi057.stdout:3/644: write d3/d4/d1d/f36 [3730880,84507] 0 2024-02-28T23:07:49.583 INFO:tasks.workunit.client.1.smithi084.stdout:0/936: stat d1/d6/d59/d70/f118 0 2024-02-28T23:07:49.585 INFO:tasks.workunit.client.1.smithi084.stdout:5/897: write dc/de/df/d16/f26 [61518,122825] 0 2024-02-28T23:07:49.585 INFO:tasks.workunit.client.1.smithi084.stdout:1/931: write d12/d1b/d63/d7d/f8c [389045,39125] 0 2024-02-28T23:07:49.587 INFO:tasks.workunit.client.1.smithi084.stdout:6/895: creat d1/d13/d2e/d41/d2c/f12a x:0 0 0 2024-02-28T23:07:49.590 INFO:tasks.workunit.client.1.smithi084.stdout:5/898: stat dc/de/df/d6e/ff1 0 2024-02-28T23:07:49.590 INFO:tasks.workunit.client.1.smithi084.stdout:0/937: mkdir d1/d6/db/d35/d4a/d12d 0 2024-02-28T23:07:49.590 INFO:tasks.workunit.client.0.smithi057.stdout:5/753: creat d4/d9/d16/feb x:0 0 0 2024-02-28T23:07:49.592 INFO:tasks.workunit.client.0.smithi057.stdout:3/645: truncate d3/d4/d7/f10 8827366 0 2024-02-28T23:07:49.593 INFO:tasks.workunit.client.0.smithi057.stdout:1/664: dread d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:49.594 INFO:tasks.workunit.client.1.smithi084.stdout:6/896: mkdir d1/d9/ddc/d12b 0 2024-02-28T23:07:49.596 INFO:tasks.workunit.client.0.smithi057.stdout:4/718: dwrite d1a/d48/d97/f74 [0,4194304] 0 2024-02-28T23:07:49.596 INFO:tasks.workunit.client.0.smithi057.stdout:5/754: creat d4/d9/d14/fec x:0 0 0 2024-02-28T23:07:49.602 INFO:tasks.workunit.client.1.smithi084.stdout:0/938: symlink d1/d6/db/d35/d44/d113/l12e 0 2024-02-28T23:07:49.603 INFO:tasks.workunit.client.1.smithi084.stdout:5/899: rmdir dc/d134 0 2024-02-28T23:07:49.608 INFO:tasks.workunit.client.1.smithi084.stdout:0/939: mknod d1/d6/db/d76/dee/df2/c12f 0 2024-02-28T23:07:49.612 INFO:tasks.workunit.client.0.smithi057.stdout:6/698: dwrite d0/d1b/d3e/d40/fb6 [0,4194304] 0 2024-02-28T23:07:49.613 INFO:tasks.workunit.client.0.smithi057.stdout:5/755: fsync d4/d7/d8/fe0 0 2024-02-28T23:07:49.613 INFO:tasks.workunit.client.1.smithi084.stdout:0/940: dread - d1/d6/db/d35/d82/f100 zero size 2024-02-28T23:07:49.614 INFO:tasks.workunit.client.0.smithi057.stdout:1/665: dread d2/d7/df/db/d12/d3f/fea [0,4194304] 0 2024-02-28T23:07:49.615 INFO:tasks.workunit.client.0.smithi057.stdout:2/834: dwrite d8/d11/d39/f62 [0,4194304] 0 2024-02-28T23:07:49.615 INFO:tasks.workunit.client.1.smithi084.stdout:8/893: dread d0/d44/f102 [4194304,4194304] 0 2024-02-28T23:07:49.615 INFO:tasks.workunit.client.1.smithi084.stdout:5/900: rename dc/de/df/d16/f26 to dc/de/df/db0/d104/f151 0 2024-02-28T23:07:49.616 INFO:tasks.workunit.client.1.smithi084.stdout:9/958: getdents d4/d6/d15/d12/d41/dea 0 2024-02-28T23:07:49.617 INFO:tasks.workunit.client.0.smithi057.stdout:5/756: stat d4/d9/d14/d9e 0 2024-02-28T23:07:49.618 INFO:tasks.workunit.client.1.smithi084.stdout:3/953: truncate d3/d27/d7d/db0/f72 193316 0 2024-02-28T23:07:49.624 INFO:tasks.workunit.client.1.smithi084.stdout:9/959: fdatasync d4/d6/d15/d75/d2d/dbd/f108 0 2024-02-28T23:07:49.626 INFO:tasks.workunit.client.0.smithi057.stdout:2/835: creat d8/d11/db3/de9/f100 x:0 0 0 2024-02-28T23:07:49.628 INFO:tasks.workunit.client.0.smithi057.stdout:1/666: rmdir d2/dd4/dd5/ddc 0 2024-02-28T23:07:49.629 INFO:tasks.workunit.client.1.smithi084.stdout:3/954: stat d3/d27/d7d/db0/d1a/d29/d41/l101 0 2024-02-28T23:07:49.629 INFO:tasks.workunit.client.1.smithi084.stdout:7/990: dwrite db/d10/f2c [0,4194304] 0 2024-02-28T23:07:49.629 INFO:tasks.workunit.client.0.smithi057.stdout:8/782: sync 2024-02-28T23:07:49.630 INFO:tasks.workunit.client.1.smithi084.stdout:8/894: symlink d0/d9/d30/d33/d71/d84/dd9/l121 0 2024-02-28T23:07:49.632 INFO:tasks.workunit.client.1.smithi084.stdout:9/960: stat d4/fb6 0 2024-02-28T23:07:49.635 INFO:tasks.workunit.client.1.smithi084.stdout:0/941: link d1/d6/d1b/cc3 d1/d8/d18/d4c/d6c/d74/c130 0 2024-02-28T23:07:49.636 INFO:tasks.workunit.client.0.smithi057.stdout:8/783: truncate d4/d13/d18/da7/f101 558348 0 2024-02-28T23:07:49.637 INFO:tasks.workunit.client.1.smithi084.stdout:8/895: chown d0/d9/d1b/d38/d43/d51/d54/cd3 0 1 2024-02-28T23:07:49.638 INFO:tasks.workunit.client.0.smithi057.stdout:1/667: rename d2/d90/f9c to d2/d90/fee 0 2024-02-28T23:07:49.639 INFO:tasks.workunit.client.0.smithi057.stdout:7/755: dread d0/d5/d36/d4e/d15/f6a [0,4194304] 0 2024-02-28T23:07:49.640 INFO:tasks.workunit.client.1.smithi084.stdout:0/942: mknod d1/d8/d78/c131 0 2024-02-28T23:07:49.641 INFO:tasks.workunit.client.0.smithi057.stdout:2/836: rename d8/d11/d21/d28/d41/f5c to d8/d11/d21/d28/d49/d85/d92/dc5/f101 0 2024-02-28T23:07:49.641 INFO:tasks.workunit.client.1.smithi084.stdout:6/897: dread d1/d9/f28 [0,4194304] 0 2024-02-28T23:07:49.642 INFO:tasks.workunit.client.1.smithi084.stdout:7/991: mkdir db/d5e/d87/dc1/de6/d117/d13d 0 2024-02-28T23:07:49.643 INFO:tasks.workunit.client.0.smithi057.stdout:6/699: read d0/d1b/d3e/f7b [186515,100656] 0 2024-02-28T23:07:49.643 INFO:tasks.workunit.client.1.smithi084.stdout:9/961: unlink d4/d69/l114 0 2024-02-28T23:07:49.646 INFO:tasks.workunit.client.0.smithi057.stdout:5/757: dwrite d4/f92 [4194304,4194304] 0 2024-02-28T23:07:49.646 INFO:tasks.workunit.client.0.smithi057.stdout:8/784: symlink d4/d13/d2c/d4e/l117 0 2024-02-28T23:07:49.647 INFO:tasks.workunit.client.1.smithi084.stdout:0/943: fdatasync d1/d6/db/d35/d44/f4b 0 2024-02-28T23:07:49.647 INFO:tasks.workunit.client.1.smithi084.stdout:9/962: readlink d4/d6/d15/d12/la0 0 2024-02-28T23:07:49.650 INFO:tasks.workunit.client.1.smithi084.stdout:9/963: dread - d4/d69/f122 zero size 2024-02-28T23:07:49.655 INFO:tasks.workunit.client.1.smithi084.stdout:0/944: write d1/d8/f3c [1515696,8210] 0 2024-02-28T23:07:49.655 INFO:tasks.workunit.client.0.smithi057.stdout:1/668: mkdir d2/d71/def 0 2024-02-28T23:07:49.655 INFO:tasks.workunit.client.0.smithi057.stdout:3/646: truncate d3/d8/d27/f81 94334 0 2024-02-28T23:07:49.655 INFO:tasks.workunit.client.0.smithi057.stdout:2/837: fdatasync d8/d11/fcf 0 2024-02-28T23:07:49.658 INFO:tasks.workunit.client.0.smithi057.stdout:3/647: readlink d3/d4/d7/l3d 0 2024-02-28T23:07:49.661 INFO:tasks.workunit.client.1.smithi084.stdout:8/896: dread d0/d9/d1b/d38/d43/d51/de8/feb [0,4194304] 0 2024-02-28T23:07:49.662 INFO:tasks.workunit.client.1.smithi084.stdout:1/932: dwrite d12/d1b/d2e/d52/d5b/f8a [0,4194304] 0 2024-02-28T23:07:49.662 INFO:tasks.workunit.client.0.smithi057.stdout:5/758: creat d4/d9/d16/d4e/fed x:0 0 0 2024-02-28T23:07:49.662 INFO:tasks.workunit.client.1.smithi084.stdout:7/992: creat db/f13e x:0 0 0 2024-02-28T23:07:49.662 INFO:tasks.workunit.client.1.smithi084.stdout:5/901: write dc/f6b [7975291,111421] 0 2024-02-28T23:07:49.663 INFO:tasks.workunit.client.0.smithi057.stdout:3/648: readlink d3/d4/d1d/l76 0 2024-02-28T23:07:49.667 INFO:tasks.workunit.client.1.smithi084.stdout:6/898: truncate d1/d13/d2e/d41/fa9 156363 0 2024-02-28T23:07:49.667 INFO:tasks.workunit.client.0.smithi057.stdout:1/669: symlink d2/d7/df/db/d3a/d5a/lf0 0 2024-02-28T23:07:49.668 INFO:tasks.workunit.client.1.smithi084.stdout:8/897: read d0/d9/fbf [2091,85980] 0 2024-02-28T23:07:49.668 INFO:tasks.workunit.client.1.smithi084.stdout:1/933: unlink d12/c4f 0 2024-02-28T23:07:49.674 INFO:tasks.workunit.client.1.smithi084.stdout:5/902: creat dc/ddb/f152 x:0 0 0 2024-02-28T23:07:49.678 INFO:tasks.workunit.client.0.smithi057.stdout:8/785: dread d4/de9/d56/d6a/d42/f5b [0,4194304] 0 2024-02-28T23:07:49.679 INFO:tasks.workunit.client.1.smithi084.stdout:8/898: fsync d0/d9/d30/db6/f89 0 2024-02-28T23:07:49.679 INFO:tasks.workunit.client.1.smithi084.stdout:6/899: symlink d1/d13/d47/d91/l12c 0 2024-02-28T23:07:49.680 INFO:tasks.workunit.client.0.smithi057.stdout:2/838: dread d8/d11/d39/d83/fce [0,4194304] 0 2024-02-28T23:07:49.682 INFO:tasks.workunit.client.1.smithi084.stdout:5/903: write dc/de/df/d31/dd1/d5f/d9b/dfe/f13b [359049,39681] 0 2024-02-28T23:07:49.682 INFO:tasks.workunit.client.0.smithi057.stdout:3/649: getdents d3/d4 0 2024-02-28T23:07:49.685 INFO:tasks.workunit.client.0.smithi057.stdout:6/700: dwrite d0/d1b/d3e/d3d/d42/fe0 [0,4194304] 0 2024-02-28T23:07:49.687 INFO:tasks.workunit.client.1.smithi084.stdout:6/900: write d1/d13/d2e/d41/d4d/f63 [492762,87316] 0 2024-02-28T23:07:49.687 INFO:tasks.workunit.client.1.smithi084.stdout:8/899: creat d0/d24/d78/f122 x:0 0 0 2024-02-28T23:07:49.687 INFO:tasks.workunit.client.1.smithi084.stdout:1/934: dread - d12/d1b/d2e/d30/d50/da8/ffe zero size 2024-02-28T23:07:49.690 INFO:tasks.workunit.client.1.smithi084.stdout:3/955: dwrite f1 [8388608,4194304] 0 2024-02-28T23:07:49.690 INFO:tasks.workunit.client.0.smithi057.stdout:5/759: getdents d4/d9/d14/d63 0 2024-02-28T23:07:49.691 INFO:tasks.workunit.client.0.smithi057.stdout:1/670: symlink d2/d7/df/db/d12/d19/lf1 0 2024-02-28T23:07:49.692 INFO:tasks.workunit.client.1.smithi084.stdout:6/901: write d1/d13/d2e/d41/d4d/d6c/fe7 [1094314,128309] 0 2024-02-28T23:07:49.693 INFO:tasks.workunit.client.0.smithi057.stdout:7/756: dwrite d0/d5/d25/d3e/fa9 [0,4194304] 0 2024-02-28T23:07:49.695 INFO:tasks.workunit.client.0.smithi057.stdout:2/839: rename d8/d11/d21/d70/ff3 to d8/d11/d56/f102 0 2024-02-28T23:07:49.696 INFO:tasks.workunit.client.1.smithi084.stdout:5/904: write dc/de/df/d31/dd1/d5f/d9b/f12f [301481,116803] 0 2024-02-28T23:07:49.702 INFO:tasks.workunit.client.1.smithi084.stdout:6/902: write d1/d9/d1d/d1e/d33/f89 [5382482,87650] 0 2024-02-28T23:07:49.703 INFO:tasks.workunit.client.1.smithi084.stdout:1/935: creat d12/d1b/d91/d9b/da5/f12f x:0 0 0 2024-02-28T23:07:49.704 INFO:tasks.workunit.client.0.smithi057.stdout:6/701: dread d0/d1b/d3e/d3d/d42/d4f/fbb [0,4194304] 0 2024-02-28T23:07:49.704 INFO:tasks.workunit.client.1.smithi084.stdout:3/956: mkdir d3/d50/d117/d7b/d161 0 2024-02-28T23:07:49.704 INFO:tasks.workunit.client.0.smithi057.stdout:8/786: mkdir d4/d13/d18/da7/de1/dee/d118 0 2024-02-28T23:07:49.705 INFO:tasks.workunit.client.0.smithi057.stdout:5/760: creat d4/d7/d58/fee x:0 0 0 2024-02-28T23:07:49.705 INFO:tasks.workunit.client.0.smithi057.stdout:2/840: symlink d8/d11/d39/dbc/l103 0 2024-02-28T23:07:49.706 INFO:tasks.workunit.client.1.smithi084.stdout:6/903: rmdir d1/d13/d47/d91 39 2024-02-28T23:07:49.707 INFO:tasks.workunit.client.1.smithi084.stdout:9/964: write d4/f51 [1301559,119935] 0 2024-02-28T23:07:49.712 INFO:tasks.workunit.client.0.smithi057.stdout:5/761: dread - d4/d21/d24/d72/fc4 zero size 2024-02-28T23:07:49.712 INFO:tasks.workunit.client.1.smithi084.stdout:9/965: fdatasync d4/d6/d15/f8e 0 2024-02-28T23:07:49.712 INFO:tasks.workunit.client.0.smithi057.stdout:7/757: mknod d0/d5/d25/cfd 0 2024-02-28T23:07:49.714 INFO:tasks.workunit.client.1.smithi084.stdout:6/904: creat d1/d13/d2e/d41/f12d x:0 0 0 2024-02-28T23:07:49.718 INFO:tasks.workunit.client.1.smithi084.stdout:6/905: fsync d1/d13/d50/fdf 0 2024-02-28T23:07:49.718 INFO:tasks.workunit.client.1.smithi084.stdout:1/936: creat d12/d1b/d63/f130 x:0 0 0 2024-02-28T23:07:49.719 INFO:tasks.workunit.client.0.smithi057.stdout:8/787: readlink d4/de9/d56/da6/dae/l10f 0 2024-02-28T23:07:49.721 INFO:tasks.workunit.client.1.smithi084.stdout:8/900: dread d0/d9/d1b/d38/d43/d51/ff5 [0,4194304] 0 2024-02-28T23:07:49.721 INFO:tasks.workunit.client.1.smithi084.stdout:9/966: mknod d4/d6/d15/c137 0 2024-02-28T23:07:49.722 INFO:tasks.workunit.client.1.smithi084.stdout:1/937: chown d12/d1b/d91/d9b/f118 29484262 1 2024-02-28T23:07:49.727 INFO:tasks.workunit.client.0.smithi057.stdout:8/788: dread - d4/df7/f111 zero size 2024-02-28T23:07:49.728 INFO:tasks.workunit.client.1.smithi084.stdout:9/967: fsync d4/f90 0 2024-02-28T23:07:49.728 INFO:tasks.workunit.client.0.smithi057.stdout:1/671: dwrite d2/d71/d62/fe0 [0,4194304] 0 2024-02-28T23:07:49.729 INFO:tasks.workunit.client.1.smithi084.stdout:1/938: creat d12/d1b/d3b/f131 x:0 0 0 2024-02-28T23:07:49.731 INFO:tasks.workunit.client.0.smithi057.stdout:3/650: dwrite d3/d4/d7/f6a [0,4194304] 0 2024-02-28T23:07:49.735 INFO:tasks.workunit.client.1.smithi084.stdout:6/906: dread d1/d13/d50/fc9 [0,4194304] 0 2024-02-28T23:07:49.739 INFO:tasks.workunit.client.1.smithi084.stdout:6/907: chown d1/d13/c82 397835 1 2024-02-28T23:07:49.740 INFO:tasks.workunit.client.1.smithi084.stdout:9/968: mkdir d4/d32/d3f/d89/d138 0 2024-02-28T23:07:49.742 INFO:tasks.workunit.client.1.smithi084.stdout:3/957: dwrite d3/d50/d117/d76/fb2 [0,4194304] 0 2024-02-28T23:07:49.743 INFO:tasks.workunit.client.1.smithi084.stdout:5/905: write dc/de/df/d16/d34/d3c/f5a [2237630,77135] 0 2024-02-28T23:07:49.743 INFO:tasks.workunit.client.0.smithi057.stdout:6/702: dwrite d0/d1b/fd5 [0,4194304] 0 2024-02-28T23:07:49.747 INFO:tasks.workunit.client.0.smithi057.stdout:5/762: dwrite d4/d9/d14/fec [0,4194304] 0 2024-02-28T23:07:49.749 INFO:tasks.workunit.client.0.smithi057.stdout:8/789: symlink d4/de9/d56/dc9/l119 0 2024-02-28T23:07:49.749 INFO:tasks.workunit.client.0.smithi057.stdout:7/758: dwrite d0/d3f/d76/dc1/dea/f9d [0,4194304] 0 2024-02-28T23:07:49.752 INFO:tasks.workunit.client.0.smithi057.stdout:5/763: read - d4/d21/d24/da2/dbb/fa4 zero size 2024-02-28T23:07:49.757 INFO:tasks.workunit.client.0.smithi057.stdout:5/764: write d4/d9/d14/d63/d88/fc5 [646348,54911] 0 2024-02-28T23:07:49.758 INFO:tasks.workunit.client.1.smithi084.stdout:5/906: truncate dc/de/df/d16/ffa 521558 0 2024-02-28T23:07:49.758 INFO:tasks.workunit.client.1.smithi084.stdout:7/993: dread db/d10/d17/d9c/d9f/db1/d104/d120/fc9 [0,4194304] 0 2024-02-28T23:07:49.763 INFO:tasks.workunit.client.1.smithi084.stdout:5/907: fsync dc/de/df/d31/f7e 0 2024-02-28T23:07:49.766 INFO:tasks.workunit.client.0.smithi057.stdout:1/672: rename d2/d33/d54/f89 to d2/d71/def/ff2 0 2024-02-28T23:07:49.766 INFO:tasks.workunit.client.1.smithi084.stdout:7/994: chown db/d10/f56 585416 1 2024-02-28T23:07:49.766 INFO:tasks.workunit.client.0.smithi057.stdout:8/790: rmdir d4/de9 39 2024-02-28T23:07:49.771 INFO:tasks.workunit.client.0.smithi057.stdout:5/765: creat d4/d21/d24/da2/dbb/fef x:0 0 0 2024-02-28T23:07:49.771 INFO:tasks.workunit.client.1.smithi084.stdout:5/908: chown dc/de/f2d 0 1 2024-02-28T23:07:49.771 INFO:tasks.workunit.client.1.smithi084.stdout:7/995: dread - db/d10/d17/d3b/fe2 zero size 2024-02-28T23:07:49.774 INFO:tasks.workunit.client.0.smithi057.stdout:1/673: mknod d2/d7/cf3 0 2024-02-28T23:07:49.774 INFO:tasks.workunit.client.1.smithi084.stdout:6/908: dwrite d1/d88/dbf/f120 [0,4194304] 0 2024-02-28T23:07:49.774 INFO:tasks.workunit.client.1.smithi084.stdout:9/969: dwrite d4/d6/d15/d12/fae [0,4194304] 0 2024-02-28T23:07:49.774 INFO:tasks.workunit.client.1.smithi084.stdout:7/996: chown db/d5e/d80/ff2 1 1 2024-02-28T23:07:49.776 INFO:tasks.workunit.client.0.smithi057.stdout:8/791: rename d4/d13/d18/d10d to d4/d13/d1a/d4a/d11a 0 2024-02-28T23:07:49.782 INFO:tasks.workunit.client.1.smithi084.stdout:7/997: mknod db/d10/d17/d64/c13f 0 2024-02-28T23:07:49.783 INFO:tasks.workunit.client.1.smithi084.stdout:5/909: symlink dc/de/df/db0/d104/d139/l153 0 2024-02-28T23:07:49.786 INFO:tasks.workunit.client.0.smithi057.stdout:1/674: dread d2/d71/d2d/d5b/f7c [0,4194304] 0 2024-02-28T23:07:49.789 INFO:tasks.workunit.client.0.smithi057.stdout:8/792: dread d4/de9/d56/d6a/d42/f5b [0,4194304] 0 2024-02-28T23:07:49.789 INFO:tasks.workunit.client.1.smithi084.stdout:2/806: sync 2024-02-28T23:07:49.790 INFO:tasks.workunit.client.1.smithi084.stdout:9/970: symlink d4/d6/d15/d75/d2d/dbd/dde/l139 0 2024-02-28T23:07:49.790 INFO:tasks.workunit.client.1.smithi084.stdout:7/998: write db/d10/d17/fae [3074575,65532] 0 2024-02-28T23:07:49.791 INFO:tasks.workunit.client.0.smithi057.stdout:1/675: stat d2/d7/df/db/d3a/d5a/dd3 0 2024-02-28T23:07:49.794 INFO:tasks.workunit.client.0.smithi057.stdout:3/651: truncate d3/d4f/f48 3184893 0 2024-02-28T23:07:49.795 INFO:tasks.workunit.client.1.smithi084.stdout:2/807: readlink d5/d7/d12/l55 0 2024-02-28T23:07:49.796 INFO:tasks.workunit.client.0.smithi057.stdout:6/703: dwrite d0/d17/f2c [0,4194304] 0 2024-02-28T23:07:49.797 INFO:tasks.workunit.client.0.smithi057.stdout:5/766: truncate d4/d9/d16/d52/fe1 4126683 0 2024-02-28T23:07:49.800 INFO:tasks.workunit.client.1.smithi084.stdout:2/808: symlink d5/d1f/d42/lf7 0 2024-02-28T23:07:49.802 INFO:tasks.workunit.client.0.smithi057.stdout:7/759: dwrite d0/f23 [0,4194304] 0 2024-02-28T23:07:49.803 INFO:tasks.workunit.client.1.smithi084.stdout:7/999: creat db/d10/d18/db7/de9/f140 x:0 0 0 2024-02-28T23:07:49.807 INFO:tasks.workunit.client.0.smithi057.stdout:5/767: write d4/d21/d24/f86 [3656750,3799] 0 2024-02-28T23:07:49.808 INFO:tasks.workunit.client.0.smithi057.stdout:3/652: chown d3/d8/d1b/c41 9043700 1 2024-02-28T23:07:49.810 INFO:tasks.workunit.client.0.smithi057.stdout:8/793: rename d4/de9/d56/d6a/f95 to d4/d13/d2c/d4e/f11b 0 2024-02-28T23:07:49.812 INFO:tasks.workunit.client.0.smithi057.stdout:3/653: truncate d3/d8/f3c 5175197 0 2024-02-28T23:07:49.814 INFO:tasks.workunit.client.0.smithi057.stdout:0/856: sync 2024-02-28T23:07:49.814 INFO:tasks.workunit.client.0.smithi057.stdout:9/728: sync 2024-02-28T23:07:49.814 INFO:tasks.workunit.client.1.smithi084.stdout:5/910: dwrite dc/de/f113 [0,4194304] 0 2024-02-28T23:07:49.818 INFO:tasks.workunit.client.0.smithi057.stdout:5/768: truncate d4/d9/d16/d52/f65 5208059 0 2024-02-28T23:07:49.822 INFO:tasks.workunit.client.1.smithi084.stdout:8/901: dwrite d0/d9/d1b/d38/d43/dae/db8/fe5 [0,4194304] 0 2024-02-28T23:07:49.823 INFO:tasks.workunit.client.0.smithi057.stdout:3/654: fdatasync d3/d4/d1d/f39 0 2024-02-28T23:07:49.826 INFO:tasks.workunit.client.0.smithi057.stdout:5/769: mknod d4/d7/d58/d78/cf0 0 2024-02-28T23:07:49.830 INFO:tasks.workunit.client.0.smithi057.stdout:0/857: truncate d17/d20/d26/d48/d7c/d98/da6/f11f 26028 0 2024-02-28T23:07:49.833 INFO:tasks.workunit.client.0.smithi057.stdout:7/760: dread d0/f23 [0,4194304] 0 2024-02-28T23:07:49.834 INFO:tasks.workunit.client.0.smithi057.stdout:5/770: read - d4/d9/d16/d52/d7a/f7f zero size 2024-02-28T23:07:49.835 INFO:tasks.workunit.client.1.smithi084.stdout:0/945: sync 2024-02-28T23:07:49.835 INFO:tasks.workunit.client.1.smithi084.stdout:4/862: sync 2024-02-28T23:07:49.836 INFO:tasks.workunit.client.1.smithi084.stdout:5/911: chown dc/de/df/d31/dd1/d5f/d66/l140 1066160 1 2024-02-28T23:07:49.836 INFO:tasks.workunit.client.0.smithi057.stdout:8/794: link d4/c112 d4/de9/d56/da6/dae/c11c 0 2024-02-28T23:07:49.837 INFO:tasks.workunit.client.0.smithi057.stdout:1/676: dwrite d2/d71/d62/fa6 [0,4194304] 0 2024-02-28T23:07:49.837 INFO:tasks.workunit.client.0.smithi057.stdout:7/761: chown d0/d5/d36/d4e/d15/c9a 12581146 1 2024-02-28T23:07:49.846 INFO:tasks.workunit.client.1.smithi084.stdout:3/958: dwrite d3/d27/d7d/db0/d1a/d29/d41/d5b/f83 [0,4194304] 0 2024-02-28T23:07:49.846 INFO:tasks.workunit.client.1.smithi084.stdout:0/946: creat d1/d8/da1/f132 x:0 0 0 2024-02-28T23:07:49.847 INFO:tasks.workunit.client.0.smithi057.stdout:7/762: rmdir d0/d3f 39 2024-02-28T23:07:49.847 INFO:tasks.workunit.client.0.smithi057.stdout:1/677: write d2/d7/df/db/d12/d19/f9d [2351868,20617] 0 2024-02-28T23:07:49.849 INFO:tasks.workunit.client.1.smithi084.stdout:6/909: dwrite d1/d13/d2e/d41/d2c/dfb/f9a [0,4194304] 0 2024-02-28T23:07:49.849 INFO:tasks.workunit.client.1.smithi084.stdout:1/939: dread d12/d1b/d91/d9b/fde [0,4194304] 0 2024-02-28T23:07:49.851 INFO:tasks.workunit.client.0.smithi057.stdout:8/795: creat d4/d13/d2c/db1/f11d x:0 0 0 2024-02-28T23:07:49.851 INFO:tasks.workunit.client.0.smithi057.stdout:1/678: chown d2/d71/d2d 4366962 1 2024-02-28T23:07:49.851 INFO:tasks.workunit.client.0.smithi057.stdout:9/729: write dd/de/f4f [1310771,3068] 0 2024-02-28T23:07:49.853 INFO:tasks.workunit.client.0.smithi057.stdout:8/796: dread - d4/d13/df0/f10e zero size 2024-02-28T23:07:49.855 INFO:tasks.workunit.client.1.smithi084.stdout:6/910: symlink d1/d88/dbf/l12e 0 2024-02-28T23:07:49.858 INFO:tasks.workunit.client.0.smithi057.stdout:7/763: rename d0/l34 to d0/d5/d36/d4e/d15/da0/lfe 0 2024-02-28T23:07:49.858 INFO:tasks.workunit.client.1.smithi084.stdout:5/912: truncate dc/de/df/d31/dd1/f143 442539 0 2024-02-28T23:07:49.861 INFO:tasks.workunit.client.0.smithi057.stdout:1/679: mknod d2/d71/d2d/d39/da3/cf4 0 2024-02-28T23:07:49.861 INFO:tasks.workunit.client.0.smithi057.stdout:5/771: getdents d4 0 2024-02-28T23:07:49.862 INFO:tasks.workunit.client.1.smithi084.stdout:3/959: chown d3/d27/d7d/db0/d1a/d29/d41/d60/d98/l12d 246494942 1 2024-02-28T23:07:49.863 INFO:tasks.workunit.client.1.smithi084.stdout:2/809: dwrite d5/d7/d9/f88 [0,4194304] 0 2024-02-28T23:07:49.865 INFO:tasks.workunit.client.0.smithi057.stdout:3/655: write d3/d4/d1d/d47/d62/f8c [995043,103571] 0 2024-02-28T23:07:49.867 INFO:tasks.workunit.client.0.smithi057.stdout:7/764: chown d0/d5/d36/d4e/d15/d29/d3d/c97 950642 1 2024-02-28T23:07:49.867 INFO:tasks.workunit.client.1.smithi084.stdout:6/911: read d1/d13/d50/d74/fb7 [907752,8931] 0 2024-02-28T23:07:49.867 INFO:tasks.workunit.client.1.smithi084.stdout:2/810: stat d5/d7/d75/fc2 0 2024-02-28T23:07:49.868 INFO:tasks.workunit.client.0.smithi057.stdout:9/730: creat dd/de/d14/d48/feb x:0 0 0 2024-02-28T23:07:49.868 INFO:tasks.workunit.client.0.smithi057.stdout:6/704: dread d0/d1b/d3e/d40/f4d [0,4194304] 0 2024-02-28T23:07:49.870 INFO:tasks.workunit.client.1.smithi084.stdout:9/971: dread d4/d32/f31 [0,4194304] 0 2024-02-28T23:07:49.872 INFO:tasks.workunit.client.0.smithi057.stdout:5/772: truncate d4/d21/d24/fbf 484180 0 2024-02-28T23:07:49.877 INFO:tasks.workunit.client.1.smithi084.stdout:4/863: rename d1/d18/d20/d24/d2f/d3e/d52/d45/db5/ff4 to d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/df6/f11d 0 2024-02-28T23:07:49.877 INFO:tasks.workunit.client.1.smithi084.stdout:9/972: chown d4/d32/d3f/d89/db8 148178 1 2024-02-28T23:07:49.877 INFO:tasks.workunit.client.1.smithi084.stdout:1/940: truncate d12/d1b/d2e/f4e 1625916 0 2024-02-28T23:07:49.882 INFO:tasks.workunit.client.0.smithi057.stdout:4/719: sync 2024-02-28T23:07:49.889 INFO:tasks.workunit.client.0.smithi057.stdout:5/773: rename d4/d21 to d4/d32/df1 0 2024-02-28T23:07:49.892 INFO:tasks.workunit.client.0.smithi057.stdout:4/720: unlink d1a/d1e/d81/le1 0 2024-02-28T23:07:49.893 INFO:tasks.workunit.client.0.smithi057.stdout:0/858: dread d17/d20/d23/d4b/fae [0,4194304] 0 2024-02-28T23:07:49.894 INFO:tasks.workunit.client.1.smithi084.stdout:6/912: symlink d1/d9/ddc/d12b/l12f 0 2024-02-28T23:07:49.897 INFO:tasks.workunit.client.1.smithi084.stdout:9/973: symlink d4/d6/d15/d75/d9d/dd0/l13a 0 2024-02-28T23:07:49.897 INFO:tasks.workunit.client.0.smithi057.stdout:8/797: dwrite d4/de9/d56/d6a/f4d [4194304,4194304] 0 2024-02-28T23:07:49.897 INFO:tasks.workunit.client.0.smithi057.stdout:1/680: dread d2/d7/df/db/d12/d3f/d47/f9f [0,4194304] 0 2024-02-28T23:07:49.897 INFO:tasks.workunit.client.1.smithi084.stdout:2/811: rename d5/d7/d12/d24/l81 to d5/d7/d12/d24/d2a/lf8 0 2024-02-28T23:07:49.900 INFO:tasks.workunit.client.0.smithi057.stdout:3/656: dwrite d3/d8/d27/f55 [0,4194304] 0 2024-02-28T23:07:49.901 INFO:tasks.workunit.client.0.smithi057.stdout:0/859: fdatasync d17/d20/d26/f46 0 2024-02-28T23:07:49.902 INFO:tasks.workunit.client.1.smithi084.stdout:9/974: chown d4/d6/d15/d75/d9d/dd0/ff8 51889230 1 2024-02-28T23:07:49.908 INFO:tasks.workunit.client.0.smithi057.stdout:4/721: rename d1a/d2f/db2/d49/c56 to d1a/d2f/db2/dcb/cfa 0 2024-02-28T23:07:49.909 INFO:tasks.workunit.client.1.smithi084.stdout:9/975: fdatasync d4/d32/f67 0 2024-02-28T23:07:49.909 INFO:tasks.workunit.client.1.smithi084.stdout:5/913: link dc/de/df/d16/d34/d3c/d77/dd2/c148 dc/de/c154 0 2024-02-28T23:07:49.909 INFO:tasks.workunit.client.1.smithi084.stdout:4/864: getdents d1/d18/d20/d24/ddb 0 2024-02-28T23:07:49.910 INFO:tasks.workunit.client.1.smithi084.stdout:8/902: dread d0/d9/d1b/d38/f5d [0,4194304] 0 2024-02-28T23:07:49.911 INFO:tasks.workunit.client.1.smithi084.stdout:1/941: dwrite d12/d1b/d91/d9b/fc1 [0,4194304] 0 2024-02-28T23:07:49.912 INFO:tasks.workunit.client.0.smithi057.stdout:3/657: stat d3/d8/ca8 0 2024-02-28T23:07:49.915 INFO:tasks.workunit.client.1.smithi084.stdout:2/812: write d5/dd/d69/f85 [6205000,103589] 0 2024-02-28T23:07:49.915 INFO:tasks.workunit.client.1.smithi084.stdout:6/913: read d1/d9/d1d/d1e/fcd [2010525,129213] 0 2024-02-28T23:07:49.916 INFO:tasks.workunit.client.0.smithi057.stdout:6/705: dwrite d0/fe7 [0,4194304] 0 2024-02-28T23:07:49.917 INFO:tasks.workunit.client.0.smithi057.stdout:8/798: rename d4/de9/d56/d6a/d42/f44 to d4/df7/f11e 0 2024-02-28T23:07:49.918 INFO:tasks.workunit.client.1.smithi084.stdout:3/960: truncate d3/d27/d7d/db0/d14/f49 2302401 0 2024-02-28T23:07:49.923 INFO:tasks.workunit.client.1.smithi084.stdout:8/903: stat d0/d24/d78/l120 0 2024-02-28T23:07:49.923 INFO:tasks.workunit.client.1.smithi084.stdout:5/914: dread - dc/de/f79 zero size 2024-02-28T23:07:49.924 INFO:tasks.workunit.client.1.smithi084.stdout:1/942: truncate f1 4644736 0 2024-02-28T23:07:49.924 INFO:tasks.workunit.client.1.smithi084.stdout:2/813: write d5/d7/d9/d7c/d5b/da6/fbb [444607,31016] 0 2024-02-28T23:07:49.925 INFO:tasks.workunit.client.0.smithi057.stdout:8/799: creat d4/d13/d1a/f11f x:0 0 0 2024-02-28T23:07:49.933 INFO:tasks.workunit.client.0.smithi057.stdout:3/658: creat d3/d4/d1d/d47/d62/fcf x:0 0 0 2024-02-28T23:07:49.935 INFO:tasks.workunit.client.0.smithi057.stdout:8/800: stat d4/d13/d18/d62/d66/d74/d8b 0 2024-02-28T23:07:49.935 INFO:tasks.workunit.client.1.smithi084.stdout:2/814: write d5/d7/d9/d7c/d5b/f7f [1114305,60433] 0 2024-02-28T23:07:49.937 INFO:tasks.workunit.client.0.smithi057.stdout:5/774: stat d4/d9/d16/d52/d7a/f7f 0 2024-02-28T23:07:49.939 INFO:tasks.workunit.client.1.smithi084.stdout:6/914: unlink d1/d13/d2e/d41/d4d/d6c/fb8 0 2024-02-28T23:07:49.942 INFO:tasks.workunit.client.1.smithi084.stdout:4/865: creat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/db4/f11e x:0 0 0 2024-02-28T23:07:49.942 INFO:tasks.workunit.client.0.smithi057.stdout:6/706: link d0/d17/d20/f5c d0/d17/d94/da0/ff0 0 2024-02-28T23:07:49.945 INFO:tasks.workunit.client.0.smithi057.stdout:5/775: chown d4/d7/f1e 0 1 2024-02-28T23:07:49.945 INFO:tasks.workunit.client.0.smithi057.stdout:3/659: mkdir d3/d4/d7/d9a/d9b/dab/dd0 0 2024-02-28T23:07:49.947 INFO:tasks.workunit.client.0.smithi057.stdout:2/841: sync 2024-02-28T23:07:49.948 INFO:tasks.workunit.client.0.smithi057.stdout:8/801: fsync d4/de9/d56/d88/fd1 0 2024-02-28T23:07:49.951 INFO:tasks.workunit.client.0.smithi057.stdout:3/660: chown d3/d20/f24 9926734 1 2024-02-28T23:07:49.951 INFO:tasks.workunit.client.0.smithi057.stdout:5/776: mkdir d4/d9/d14/df2 0 2024-02-28T23:07:49.954 INFO:tasks.workunit.client.0.smithi057.stdout:8/802: dread - d4/de9/d56/d6a/d42/fcd zero size 2024-02-28T23:07:49.956 INFO:tasks.workunit.client.0.smithi057.stdout:5/777: readlink d4/d7/d8/ld1 0 2024-02-28T23:07:49.959 INFO:tasks.workunit.client.0.smithi057.stdout:3/661: symlink d3/d4/d1d/d47/d62/ld1 0 2024-02-28T23:07:49.962 INFO:tasks.workunit.client.1.smithi084.stdout:1/943: read d12/d1b/d2e/d30/d12d/f4b [1878464,104656] 0 2024-02-28T23:07:49.963 INFO:tasks.workunit.client.0.smithi057.stdout:9/731: dread dd/f31 [0,4194304] 0 2024-02-28T23:07:49.963 INFO:tasks.workunit.client.0.smithi057.stdout:2/842: fsync d8/d11/db3/de9/ff9 0 2024-02-28T23:07:49.966 INFO:tasks.workunit.client.1.smithi084.stdout:0/947: dread d1/d6/db/d35/d82/fda [0,4194304] 0 2024-02-28T23:07:49.968 INFO:tasks.workunit.client.0.smithi057.stdout:0/860: dread d17/d20/f27 [0,4194304] 0 2024-02-28T23:07:49.968 INFO:tasks.workunit.client.1.smithi084.stdout:3/961: dwrite d3/d27/d7d/db0/df/ded/f130 [0,4194304] 0 2024-02-28T23:07:49.969 INFO:tasks.workunit.client.1.smithi084.stdout:5/915: dwrite dc/de/df/d16/f95 [0,4194304] 0 2024-02-28T23:07:49.969 INFO:tasks.workunit.client.0.smithi057.stdout:4/722: dread d1a/d2f/db2/d49/d4d/f88 [0,4194304] 0 2024-02-28T23:07:49.970 INFO:tasks.workunit.client.0.smithi057.stdout:8/803: rename d4/d13/d18/d62/de4/led to d4/d13/d18/l120 0 2024-02-28T23:07:49.970 INFO:tasks.workunit.client.0.smithi057.stdout:7/765: dread d0/d3f/d76/dc1/dea/fbe [0,4194304] 0 2024-02-28T23:07:49.971 INFO:tasks.workunit.client.1.smithi084.stdout:0/948: chown d1/d6/db/d76/dee/df2 59958 1 2024-02-28T23:07:49.974 INFO:tasks.workunit.client.1.smithi084.stdout:9/976: dwrite d4/d6/d15/d12/f3d [0,4194304] 0 2024-02-28T23:07:49.976 INFO:tasks.workunit.client.0.smithi057.stdout:9/732: creat dd/de/d14/d48/d4b/fec x:0 0 0 2024-02-28T23:07:49.977 INFO:tasks.workunit.client.0.smithi057.stdout:3/662: creat d3/d4/d7/d9a/d9b/fd2 x:0 0 0 2024-02-28T23:07:49.979 INFO:tasks.workunit.client.1.smithi084.stdout:3/962: stat d3/d27/d7d/db0/d14/c42 0 2024-02-28T23:07:49.981 INFO:tasks.workunit.client.0.smithi057.stdout:1/681: dread d2/d7/df/db/d3a/d2a/d82/fe3 [0,4194304] 0 2024-02-28T23:07:49.982 INFO:tasks.workunit.client.0.smithi057.stdout:7/766: symlink d0/d5/d36/d4e/d15/d29/d3d/de7/lff 0 2024-02-28T23:07:49.985 INFO:tasks.workunit.client.0.smithi057.stdout:2/843: creat d8/d11/d39/dda/f104 x:0 0 0 2024-02-28T23:07:49.987 INFO:tasks.workunit.client.1.smithi084.stdout:8/904: mknod d0/d9/d1b/d20/c123 0 2024-02-28T23:07:49.989 INFO:tasks.workunit.client.0.smithi057.stdout:3/663: write d3/d20/d2d/fb8 [58996,129856] 0 2024-02-28T23:07:49.989 INFO:tasks.workunit.client.0.smithi057.stdout:0/861: symlink d17/d20/d4a/d78/db5/l131 0 2024-02-28T23:07:49.997 INFO:tasks.workunit.client.1.smithi084.stdout:6/915: rmdir d1/d13/d47/d91/dba/de9 39 2024-02-28T23:07:49.998 INFO:tasks.workunit.client.1.smithi084.stdout:5/916: symlink dc/de/df/d16/d34/daa/df4/l155 0 2024-02-28T23:07:49.999 INFO:tasks.workunit.client.1.smithi084.stdout:9/977: mkdir d4/d6/d15/de0/d13b 0 2024-02-28T23:07:50.000 INFO:tasks.workunit.client.0.smithi057.stdout:7/767: creat d0/d3f/d76/dc1/dea/f100 x:0 0 0 2024-02-28T23:07:50.000 INFO:tasks.workunit.client.0.smithi057.stdout:2/844: rmdir d8/d11/d21/d28/d49/d85/d9b/dad 39 2024-02-28T23:07:50.001 INFO:tasks.workunit.client.0.smithi057.stdout:8/804: link d4/d13/d7f/lec d4/de9/d59/l121 0 2024-02-28T23:07:50.001 INFO:tasks.workunit.client.1.smithi084.stdout:6/916: write d1/d13/d50/d74/d9e/f11b [383110,3162] 0 2024-02-28T23:07:50.002 INFO:tasks.workunit.client.1.smithi084.stdout:2/815: rmdir d5/d7/d12/d24/d2a/d53/dd5 0 2024-02-28T23:07:50.006 INFO:tasks.workunit.client.0.smithi057.stdout:1/682: write d2/d7/df/db/d3a/f3b [2464627,117983] 0 2024-02-28T23:07:50.006 INFO:tasks.workunit.client.0.smithi057.stdout:7/768: chown d0/d3f/c93 152 1 2024-02-28T23:07:50.006 INFO:tasks.workunit.client.1.smithi084.stdout:4/866: mknod d1/d18/d20/d24/d2f/d3e/d52/c11f 0 2024-02-28T23:07:50.007 INFO:tasks.workunit.client.0.smithi057.stdout:3/664: rename d3/d4/d7/d9a/l78 to d3/d4/d7/d9a/ld3 0 2024-02-28T23:07:50.008 INFO:tasks.workunit.client.0.smithi057.stdout:7/769: readlink d0/l33 0 2024-02-28T23:07:50.011 INFO:tasks.workunit.client.1.smithi084.stdout:2/816: creat d5/d7/d75/db4/ff9 x:0 0 0 2024-02-28T23:07:50.012 INFO:tasks.workunit.client.0.smithi057.stdout:2/845: mkdir d8/d11/d15/daf/d105 0 2024-02-28T23:07:50.015 INFO:tasks.workunit.client.1.smithi084.stdout:6/917: write d1/d13/d47/f4b [2634635,32618] 0 2024-02-28T23:07:50.015 INFO:tasks.workunit.client.0.smithi057.stdout:8/805: readlink d4/d13/d1a/l24 0 2024-02-28T23:07:50.016 INFO:tasks.workunit.client.1.smithi084.stdout:2/817: write d5/d7/d9/d7c/d5b/f7f [781963,33188] 0 2024-02-28T23:07:50.016 INFO:tasks.workunit.client.1.smithi084.stdout:5/917: creat dc/de/df/d31/dd1/d126/f156 x:0 0 0 2024-02-28T23:07:50.017 INFO:tasks.workunit.client.0.smithi057.stdout:7/770: dread - d0/d5/d25/d3e/ffb zero size 2024-02-28T23:07:50.019 INFO:tasks.workunit.client.0.smithi057.stdout:3/665: mknod d3/d4/cd4 0 2024-02-28T23:07:50.019 INFO:tasks.workunit.client.1.smithi084.stdout:8/905: creat d0/d9/d30/d33/d71/f124 x:0 0 0 2024-02-28T23:07:50.020 INFO:tasks.workunit.client.1.smithi084.stdout:2/818: chown d5/d7/d9/d7c/d5b/d9f/fa0 122 1 2024-02-28T23:07:50.021 INFO:tasks.workunit.client.0.smithi057.stdout:2/846: rename d8/d11/d21/d28/f3d to d8/d11/d15/d97/f106 0 2024-02-28T23:07:50.022 INFO:tasks.workunit.client.0.smithi057.stdout:5/778: write d4/d7/d58/f96 [32085,32896] 0 2024-02-28T23:07:50.024 INFO:tasks.workunit.client.1.smithi084.stdout:4/867: mkdir d1/d18/d20/d14/dd3/d120 0 2024-02-28T23:07:50.026 INFO:tasks.workunit.client.1.smithi084.stdout:5/918: mknod dc/de/df/d31/d8e/da9/dab/c157 0 2024-02-28T23:07:50.028 INFO:tasks.workunit.client.1.smithi084.stdout:9/978: truncate d4/d6/d53/f7d 1122514 0 2024-02-28T23:07:50.028 INFO:tasks.workunit.client.0.smithi057.stdout:7/771: mknod d0/d5/d36/d4e/c101 0 2024-02-28T23:07:50.029 INFO:tasks.workunit.client.1.smithi084.stdout:0/949: dread d1/d6/db/d76/f9c [0,4194304] 0 2024-02-28T23:07:50.030 INFO:tasks.workunit.client.1.smithi084.stdout:8/906: creat d0/d9/d30/d33/d71/d84/dd9/f125 x:0 0 0 2024-02-28T23:07:50.031 INFO:tasks.workunit.client.0.smithi057.stdout:6/707: dread d0/d1b/d54/fdf [0,4194304] 0 2024-02-28T23:07:50.032 INFO:tasks.workunit.client.1.smithi084.stdout:2/819: rename d5/d7/d12/d24/d2a/d35/f40 to d5/d7/d2e/ffa 0 2024-02-28T23:07:50.034 INFO:tasks.workunit.client.0.smithi057.stdout:1/683: link d2/d7/df/db/d12/l91 d2/d7/df/db/d3a/d5a/dd3/lf5 0 2024-02-28T23:07:50.035 INFO:tasks.workunit.client.0.smithi057.stdout:3/666: dread d3/d8/d1b/f4a [0,4194304] 0 2024-02-28T23:07:50.035 INFO:tasks.workunit.client.0.smithi057.stdout:8/806: dread d4/d13/d18/d62/d66/fbf [0,4194304] 0 2024-02-28T23:07:50.035 INFO:tasks.workunit.client.1.smithi084.stdout:4/868: mkdir d1/d18/d20/d14/d121 0 2024-02-28T23:07:50.036 INFO:tasks.workunit.client.0.smithi057.stdout:0/862: dread d17/d20/d26/d30/d6b/fc3 [4194304,4194304] 0 2024-02-28T23:07:50.037 INFO:tasks.workunit.client.1.smithi084.stdout:6/918: symlink d1/d9/d1d/d1e/l130 0 2024-02-28T23:07:50.037 INFO:tasks.workunit.client.1.smithi084.stdout:5/919: dread - dc/ddb/f127 zero size 2024-02-28T23:07:50.038 INFO:tasks.workunit.client.0.smithi057.stdout:7/772: write d0/d5/d36/f8c [2843500,76255] 0 2024-02-28T23:07:50.041 INFO:tasks.workunit.client.0.smithi057.stdout:8/807: chown d4/d13/d18/da7/de1/dee 1703494 1 2024-02-28T23:07:50.041 INFO:tasks.workunit.client.0.smithi057.stdout:0/863: mknod d17/d19/c132 0 2024-02-28T23:07:50.042 INFO:tasks.workunit.client.1.smithi084.stdout:0/950: creat d1/d6/db/d35/d44/d92/f133 x:0 0 0 2024-02-28T23:07:50.042 INFO:tasks.workunit.client.0.smithi057.stdout:8/808: stat d4/l75 0 2024-02-28T23:07:50.044 INFO:tasks.workunit.client.0.smithi057.stdout:0/864: chown d17/d20/d26/d48/d55/d88/cce 107180534 1 2024-02-28T23:07:50.045 INFO:tasks.workunit.client.0.smithi057.stdout:7/773: write d0/d5/f30 [4477241,46332] 0 2024-02-28T23:07:50.045 INFO:tasks.workunit.client.1.smithi084.stdout:2/820: mknod d5/d7/d75/db4/cfb 0 2024-02-28T23:07:50.047 INFO:tasks.workunit.client.0.smithi057.stdout:8/809: stat d4/de9/ce0 0 2024-02-28T23:07:50.048 INFO:tasks.workunit.client.1.smithi084.stdout:8/907: truncate d0/d9/fb 2879504 0 2024-02-28T23:07:50.049 INFO:tasks.workunit.client.0.smithi057.stdout:0/865: write d17/d20/d26/f46 [203380,115823] 0 2024-02-28T23:07:50.050 INFO:tasks.workunit.client.1.smithi084.stdout:4/869: mkdir d1/d18/d20/d14/d121/d122 0 2024-02-28T23:07:50.052 INFO:tasks.workunit.client.1.smithi084.stdout:0/951: rename d1/c6e to d1/d6/db/d35/d44/d92/dcf/c134 0 2024-02-28T23:07:50.053 INFO:tasks.workunit.client.0.smithi057.stdout:7/774: write d0/d5/d36/d4e/d15/da0/fbd [971569,18977] 0 2024-02-28T23:07:50.054 INFO:tasks.workunit.client.0.smithi057.stdout:8/810: dread - d4/d13/d18/d62/d66/d74/ffa zero size 2024-02-28T23:07:50.054 INFO:tasks.workunit.client.0.smithi057.stdout:1/684: creat d2/d71/d2d/d39/da3/ff6 x:0 0 0 2024-02-28T23:07:50.054 INFO:tasks.workunit.client.1.smithi084.stdout:5/920: mknod dc/de/df/d16/d34/d3c/d128/c158 0 2024-02-28T23:07:50.055 INFO:tasks.workunit.client.1.smithi084.stdout:0/952: chown d1/d6/db/d35/d82/f100 7003191 1 2024-02-28T23:07:50.056 INFO:tasks.workunit.client.1.smithi084.stdout:2/821: fsync d5/d7/d12/d24/d2a/d35/fa2 0 2024-02-28T23:07:50.057 INFO:tasks.workunit.client.1.smithi084.stdout:6/919: truncate d1/d13/d2e/d41/d4d/d6c/f7c 19577 0 2024-02-28T23:07:50.058 INFO:tasks.workunit.client.0.smithi057.stdout:6/708: truncate d0/d4/d25/d60/fcf 156400 0 2024-02-28T23:07:50.058 INFO:tasks.workunit.client.0.smithi057.stdout:7/775: fsync d0/d5/d36/d4e/d15/fe9 0 2024-02-28T23:07:50.059 INFO:tasks.workunit.client.1.smithi084.stdout:8/908: fsync d0/d44/d9f/f98 0 2024-02-28T23:07:50.059 INFO:tasks.workunit.client.1.smithi084.stdout:0/953: write d1/d6/d59/d70/f118 [917023,116894] 0 2024-02-28T23:07:50.061 INFO:tasks.workunit.client.0.smithi057.stdout:0/866: mknod d17/d20/d26/d48/d7c/c133 0 2024-02-28T23:07:50.061 INFO:tasks.workunit.client.1.smithi084.stdout:4/870: stat d1/d18/d20/d24/d2f/d3e/d52/d57/cb1 0 2024-02-28T23:07:50.062 INFO:tasks.workunit.client.0.smithi057.stdout:1/685: mkdir d2/d71/d2d/d39/da3/db5/df7 0 2024-02-28T23:07:50.062 INFO:tasks.workunit.client.1.smithi084.stdout:5/921: truncate dc/de/df/d16/f13d 904992 0 2024-02-28T23:07:50.063 INFO:tasks.workunit.client.1.smithi084.stdout:4/871: readlink d1/d18/d20/d65/lce 0 2024-02-28T23:07:50.065 INFO:tasks.workunit.client.0.smithi057.stdout:6/709: chown d0/d1b/d3e/d3d/d42/d4f/l96 1995 1 2024-02-28T23:07:50.067 INFO:tasks.workunit.client.0.smithi057.stdout:9/733: dwrite dd/de/d14/f8e [0,4194304] 0 2024-02-28T23:07:50.068 INFO:tasks.workunit.client.1.smithi084.stdout:0/954: read d1/d8/f3e [49384,42096] 0 2024-02-28T23:07:50.068 INFO:tasks.workunit.client.1.smithi084.stdout:2/822: dread d5/dd/fc4 [0,4194304] 0 2024-02-28T23:07:50.070 INFO:tasks.workunit.client.1.smithi084.stdout:8/909: fsync d0/d9/d30/db6/fd7 0 2024-02-28T23:07:50.072 INFO:tasks.workunit.client.0.smithi057.stdout:7/776: write d0/d3f/d76/dc1/dea/d75/f94 [696065,72250] 0 2024-02-28T23:07:50.073 INFO:tasks.workunit.client.1.smithi084.stdout:0/955: write d1/d8/d69/d6b/d85/da5/f101 [630637,89570] 0 2024-02-28T23:07:50.073 INFO:tasks.workunit.client.1.smithi084.stdout:8/910: dread d0/d9/fbf [0,4194304] 0 2024-02-28T23:07:50.074 INFO:tasks.workunit.client.1.smithi084.stdout:5/922: creat dc/de/df/d16/d34/d3c/d128/f159 x:0 0 0 2024-02-28T23:07:50.074 INFO:tasks.workunit.client.0.smithi057.stdout:0/867: mknod d17/d20/d4a/d78/dd4/d123/c134 0 2024-02-28T23:07:50.078 INFO:tasks.workunit.client.1.smithi084.stdout:4/872: rename d1/c6c to d1/d18/d20/d65/d8e/d55/dc3/dec/d11b/c123 0 2024-02-28T23:07:50.078 INFO:tasks.workunit.client.0.smithi057.stdout:5/779: dwrite d4/d32/df1/d24/da2/dbb/fa4 [0,4194304] 0 2024-02-28T23:07:50.080 INFO:tasks.workunit.client.0.smithi057.stdout:9/734: mkdir dd/de/d24/d63/d77/dcf/d40/d44/ded 0 2024-02-28T23:07:50.081 INFO:tasks.workunit.client.0.smithi057.stdout:6/710: mkdir d0/d4/d25/d60/df1 0 2024-02-28T23:07:50.082 INFO:tasks.workunit.client.0.smithi057.stdout:3/667: dwrite d3/d8/d27/d70/f74 [0,4194304] 0 2024-02-28T23:07:50.082 INFO:tasks.workunit.client.0.smithi057.stdout:7/777: chown d0/d5/d36/d4e/d15/d29/d3d/de7/df7/fdc 0 1 2024-02-28T23:07:50.087 INFO:tasks.workunit.client.0.smithi057.stdout:6/711: stat d0/d1b/c33 0 2024-02-28T23:07:50.087 INFO:tasks.workunit.client.0.smithi057.stdout:0/868: rename d17/d20/d4a/d78/dd4/c102 to d17/d20/d23/d4b/c135 0 2024-02-28T23:07:50.088 INFO:tasks.workunit.client.1.smithi084.stdout:1/944: truncate d12/d1b/d3b/f5d 3131790 0 2024-02-28T23:07:50.089 INFO:tasks.workunit.client.0.smithi057.stdout:5/780: mkdir d4/d9/d14/d3b/df3 0 2024-02-28T23:07:50.090 INFO:tasks.workunit.client.1.smithi084.stdout:2/823: unlink d5/d7/d12/d24/d3a/l6e 0 2024-02-28T23:07:50.091 INFO:tasks.workunit.client.0.smithi057.stdout:6/712: fdatasync d0/d17/d20/f5e 0 2024-02-28T23:07:50.092 INFO:tasks.workunit.client.1.smithi084.stdout:3/963: write d3/d27/d7d/db0/d14/f6d [90053,13276] 0 2024-02-28T23:07:50.092 INFO:tasks.workunit.client.0.smithi057.stdout:3/668: rmdir d3/d8/d27/d70 39 2024-02-28T23:07:50.096 INFO:tasks.workunit.client.0.smithi057.stdout:1/686: dwrite d2/d7/df/db/d12/d19/f9d [0,4194304] 0 2024-02-28T23:07:50.100 INFO:tasks.workunit.client.0.smithi057.stdout:0/869: write d17/d19/d4d/d7b/f94 [3972423,127334] 0 2024-02-28T23:07:50.101 INFO:tasks.workunit.client.1.smithi084.stdout:4/873: readlink d1/d18/d20/d24/d2f/d3e/l118 0 2024-02-28T23:07:50.103 INFO:tasks.workunit.client.1.smithi084.stdout:8/911: dwrite d0/d44/d9f/f110 [0,4194304] 0 2024-02-28T23:07:50.104 INFO:tasks.workunit.client.1.smithi084.stdout:2/824: readlink d5/d1f/d42/d8f/dbe/lc5 0 2024-02-28T23:07:50.108 INFO:tasks.workunit.client.0.smithi057.stdout:6/713: rmdir d0/d4/d25/d60/df1 0 2024-02-28T23:07:50.108 INFO:tasks.workunit.client.0.smithi057.stdout:0/870: symlink d17/d20/d4a/d78/d9d/l136 0 2024-02-28T23:07:50.112 INFO:tasks.workunit.client.0.smithi057.stdout:1/687: link d2/d33/d54/f8b d2/d71/d62/ff8 0 2024-02-28T23:07:50.113 INFO:tasks.workunit.client.0.smithi057.stdout:6/714: mknod d0/d1b/d3e/d40/cf2 0 2024-02-28T23:07:50.114 INFO:tasks.workunit.client.1.smithi084.stdout:4/874: rename d1/c85 to d1/d18/d20/d24/ddb/c124 0 2024-02-28T23:07:50.114 INFO:tasks.workunit.client.1.smithi084.stdout:8/912: fsync d0/d9/d1b/dc1/f116 0 2024-02-28T23:07:50.118 INFO:tasks.workunit.client.0.smithi057.stdout:7/778: dwrite d0/d5/d36/d4e/d15/d29/d71/f8b [0,4194304] 0 2024-02-28T23:07:50.119 INFO:tasks.workunit.client.0.smithi057.stdout:1/688: symlink d2/d7/df/db/d12/d19/lf9 0 2024-02-28T23:07:50.127 INFO:tasks.workunit.client.0.smithi057.stdout:1/689: mkdir d2/d33/d54/dc8/dfa 0 2024-02-28T23:07:50.128 INFO:tasks.workunit.client.1.smithi084.stdout:4/875: symlink d1/d18/d20/d65/d8e/d55/l125 0 2024-02-28T23:07:50.129 INFO:tasks.workunit.client.0.smithi057.stdout:7/779: unlink d0/d5/d25/d3e/lb4 0 2024-02-28T23:07:50.138 INFO:tasks.workunit.client.1.smithi084.stdout:9/979: dwrite d4/d6/d15/fe8 [4194304,4194304] 0 2024-02-28T23:07:50.138 INFO:tasks.workunit.client.1.smithi084.stdout:3/964: truncate d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb 2524999 0 2024-02-28T23:07:50.142 INFO:tasks.workunit.client.0.smithi057.stdout:5/781: write d4/d32/df1/d24/d30/f79 [983599,12127] 0 2024-02-28T23:07:50.143 INFO:tasks.workunit.client.1.smithi084.stdout:2/825: link d5/dd/cf1 d5/d7/d12/cfc 0 2024-02-28T23:07:50.145 INFO:tasks.workunit.client.0.smithi057.stdout:1/690: dread d2/d7/df/db/d3a/d2a/f7a [0,4194304] 0 2024-02-28T23:07:50.147 INFO:tasks.workunit.client.0.smithi057.stdout:5/782: write d4/d32/df1/fd2 [540904,92481] 0 2024-02-28T23:07:50.149 INFO:tasks.workunit.client.0.smithi057.stdout:5/783: readlink d4/d9/d16/d2c/l34 0 2024-02-28T23:07:50.150 INFO:tasks.workunit.client.0.smithi057.stdout:0/871: write d17/d20/d4a/d78/db5/dba/fe4 [389,95642] 0 2024-02-28T23:07:50.152 INFO:tasks.workunit.client.1.smithi084.stdout:1/945: dread d12/d1b/f4a [0,4194304] 0 2024-02-28T23:07:50.152 INFO:tasks.workunit.client.1.smithi084.stdout:6/920: dwrite d1/d13/d50/d74/d81/f94 [0,4194304] 0 2024-02-28T23:07:50.157 INFO:tasks.workunit.client.1.smithi084.stdout:4/876: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/de8/f126 x:0 0 0 2024-02-28T23:07:50.158 INFO:tasks.workunit.client.1.smithi084.stdout:2/826: creat d5/d7/d75/ffd x:0 0 0 2024-02-28T23:07:50.160 INFO:tasks.workunit.client.0.smithi057.stdout:1/691: rename d2/d90/fee to d2/d7/df/db/d12/dd6/ffb 0 2024-02-28T23:07:50.160 INFO:tasks.workunit.client.1.smithi084.stdout:1/946: unlink d12/d1b/d63/f3a 0 2024-02-28T23:07:50.164 INFO:tasks.workunit.client.1.smithi084.stdout:6/921: unlink d1/d13/d50/dbd/dc4/c118 0 2024-02-28T23:07:50.165 INFO:tasks.workunit.client.0.smithi057.stdout:6/715: dwrite d0/d4/fc1 [0,4194304] 0 2024-02-28T23:07:50.165 INFO:tasks.workunit.client.0.smithi057.stdout:0/872: link d17/d20/d26/f4c d17/d20/d4a/d78/db5/dba/f137 0 2024-02-28T23:07:50.173 INFO:tasks.workunit.client.0.smithi057.stdout:0/873: dread - d17/d20/d26/d30/f10a zero size 2024-02-28T23:07:50.174 INFO:tasks.workunit.client.1.smithi084.stdout:0/956: dread d1/d8/d69/d6b/f77 [0,4194304] 0 2024-02-28T23:07:50.174 INFO:tasks.workunit.client.1.smithi084.stdout:5/923: dwrite dc/de/df/d31/dd1/fb2 [0,4194304] 0 2024-02-28T23:07:50.176 INFO:tasks.workunit.client.1.smithi084.stdout:0/957: chown d1/d8/d69/l125 0 1 2024-02-28T23:07:50.178 INFO:tasks.workunit.client.1.smithi084.stdout:3/965: dwrite d3/d27/d7d/db0/d1a/d29/f160 [0,4194304] 0 2024-02-28T23:07:50.180 INFO:tasks.workunit.client.1.smithi084.stdout:5/924: write dc/de/df/d31/dd1/f143 [176455,99710] 0 2024-02-28T23:07:50.181 INFO:tasks.workunit.client.1.smithi084.stdout:9/980: rename d4/cec to d4/d32/d3f/d89/c13c 0 2024-02-28T23:07:50.182 INFO:tasks.workunit.client.0.smithi057.stdout:7/780: dread d0/d5/d25/d3e/f5a [0,4194304] 0 2024-02-28T23:07:50.182 INFO:tasks.workunit.client.1.smithi084.stdout:3/966: dread - d3/d27/d7d/db0/d14/de1/f12e zero size 2024-02-28T23:07:50.184 INFO:tasks.workunit.client.1.smithi084.stdout:5/925: fdatasync dc/de/df/d16/d34/d3c/fb8 0 2024-02-28T23:07:50.186 INFO:tasks.workunit.client.1.smithi084.stdout:6/922: mkdir d1/d13/d50/dbd/d131 0 2024-02-28T23:07:50.190 INFO:tasks.workunit.client.1.smithi084.stdout:1/947: mkdir d12/d1b/d63/d132 0 2024-02-28T23:07:50.194 INFO:tasks.workunit.client.1.smithi084.stdout:8/913: write d0/d9/d1b/d38/d43/dae/fec [385390,87296] 0 2024-02-28T23:07:50.195 INFO:tasks.workunit.client.0.smithi057.stdout:8/811: dread d4/de9/d56/d6a/d42/f5e [0,4194304] 0 2024-02-28T23:07:50.200 INFO:tasks.workunit.client.1.smithi084.stdout:4/877: creat d1/d18/d20/d24/d2f/d3e/d52/f127 x:0 0 0 2024-02-28T23:07:50.200 INFO:tasks.workunit.client.1.smithi084.stdout:9/981: creat d4/d12f/f13d x:0 0 0 2024-02-28T23:07:50.205 INFO:tasks.workunit.client.0.smithi057.stdout:5/784: dread d4/d9/d14/d3b/f47 [0,4194304] 0 2024-02-28T23:07:50.207 INFO:tasks.workunit.client.1.smithi084.stdout:8/914: mkdir d0/d11/d126 0 2024-02-28T23:07:50.209 INFO:tasks.workunit.client.1.smithi084.stdout:9/982: fdatasync d4/d6/fd6 0 2024-02-28T23:07:50.211 INFO:tasks.workunit.client.1.smithi084.stdout:0/958: dread d1/d8/d69/db2/fdc [0,4194304] 0 2024-02-28T23:07:50.211 INFO:tasks.workunit.client.1.smithi084.stdout:4/878: creat d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f128 x:0 0 0 2024-02-28T23:07:50.212 INFO:tasks.workunit.client.0.smithi057.stdout:0/874: mknod d17/d20/d26/d30/c138 0 2024-02-28T23:07:50.213 INFO:tasks.workunit.client.0.smithi057.stdout:5/785: fsync d4/d9/d14/fab 0 2024-02-28T23:07:50.213 INFO:tasks.workunit.client.0.smithi057.stdout:8/812: dread d4/d13/d1a/f20 [0,4194304] 0 2024-02-28T23:07:50.217 INFO:tasks.workunit.client.0.smithi057.stdout:0/875: read d17/d20/d26/f41 [41274,95561] 0 2024-02-28T23:07:50.219 INFO:tasks.workunit.client.1.smithi084.stdout:0/959: mknod d1/d8/d69/d6b/d85/da5/c135 0 2024-02-28T23:07:50.221 INFO:tasks.workunit.client.0.smithi057.stdout:0/876: unlink d17/d19/dd2/ddb/fee 0 2024-02-28T23:07:50.221 INFO:tasks.workunit.client.1.smithi084.stdout:9/983: fsync d4/d6/d15/d3c/ff4 0 2024-02-28T23:07:50.225 INFO:tasks.workunit.client.1.smithi084.stdout:4/879: rename d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/cd1 to d1/d18/d20/d14/dd3/d120/c129 0 2024-02-28T23:07:50.228 INFO:tasks.workunit.client.0.smithi057.stdout:8/813: dwrite d4/d13/fa8 [0,4194304] 0 2024-02-28T23:07:50.229 INFO:tasks.workunit.client.0.smithi057.stdout:0/877: link d17/d19/d4d/l121 d17/d20/d23/d47/l139 0 2024-02-28T23:07:50.231 INFO:tasks.workunit.client.1.smithi084.stdout:8/915: dwrite d0/d9/d1b/d38/d43/dae/fec [0,4194304] 0 2024-02-28T23:07:50.232 INFO:tasks.workunit.client.0.smithi057.stdout:6/716: write d0/d17/d94/da0/ff0 [6301224,13235] 0 2024-02-28T23:07:50.233 INFO:tasks.workunit.client.0.smithi057.stdout:8/814: creat d4/de9/d56/d6a/dbd/f122 x:0 0 0 2024-02-28T23:07:50.237 INFO:tasks.workunit.client.0.smithi057.stdout:8/815: readlink d4/de9/d59/l121 0 2024-02-28T23:07:50.240 INFO:tasks.workunit.client.0.smithi057.stdout:8/816: dread - d4/d13/d1a/d4a/db3/dfc/fff zero size 2024-02-28T23:07:50.241 INFO:tasks.workunit.client.1.smithi084.stdout:0/960: link d1/d8/ld2 d1/d6/db/l136 0 2024-02-28T23:07:50.246 INFO:tasks.workunit.client.1.smithi084.stdout:9/984: rename d4/c5 to d4/d6/d15/d75/d2d/c13e 0 2024-02-28T23:07:50.251 INFO:tasks.workunit.client.1.smithi084.stdout:8/916: creat d0/d9/d1b/d38/f127 x:0 0 0 2024-02-28T23:07:50.252 INFO:tasks.workunit.client.1.smithi084.stdout:2/827: truncate d5/d7/d75/fc2 1914153 0 2024-02-28T23:07:50.253 INFO:tasks.workunit.client.0.smithi057.stdout:5/786: write d4/d9/d16/f18 [4587710,6315] 0 2024-02-28T23:07:50.255 INFO:tasks.workunit.client.1.smithi084.stdout:5/926: write dc/de/df/d31/dd1/d5f/fe2 [4914293,59882] 0 2024-02-28T23:07:50.257 INFO:tasks.workunit.client.1.smithi084.stdout:2/828: stat d5/d7/d12/d24/d3a/d48/f49 0 2024-02-28T23:07:50.257 INFO:tasks.workunit.client.0.smithi057.stdout:6/717: write d0/d1b/d3e/d3d/d42/d4f/d53/f93 [2020418,121219] 0 2024-02-28T23:07:50.259 INFO:tasks.workunit.client.0.smithi057.stdout:5/787: mknod d4/d32/df1/d24/da2/dbb/cf4 0 2024-02-28T23:07:50.262 INFO:tasks.workunit.client.1.smithi084.stdout:2/829: chown d5/d1f/d42/d8f/fd6 27581 1 2024-02-28T23:07:50.264 INFO:tasks.workunit.client.1.smithi084.stdout:9/985: truncate d4/d6/d15/d3c/f7a 1244104 0 2024-02-28T23:07:50.265 INFO:tasks.workunit.client.1.smithi084.stdout:3/967: write d3/d27/d7d/db0/d1a/d45/d5e/d7a/df4/ffb [2735520,19985] 0 2024-02-28T23:07:50.269 INFO:tasks.workunit.client.1.smithi084.stdout:5/927: creat dc/de/d83/dd9/f15a x:0 0 0 2024-02-28T23:07:50.269 INFO:tasks.workunit.client.0.smithi057.stdout:5/788: link d4/d7/d58/d78/dc1/cd4 d4/d7/d8/cf5 0 2024-02-28T23:07:50.269 INFO:tasks.workunit.client.1.smithi084.stdout:0/961: dwrite d1/d6/db/d35/d44/d92/f133 [0,4194304] 0 2024-02-28T23:07:50.271 INFO:tasks.workunit.client.0.smithi057.stdout:5/789: write d4/d7/d58/f89 [1021818,21930] 0 2024-02-28T23:07:50.277 INFO:tasks.workunit.client.0.smithi057.stdout:5/790: symlink d4/d9/d14/d3b/d61/ddc/lf6 0 2024-02-28T23:07:50.278 INFO:tasks.workunit.client.1.smithi084.stdout:0/962: write d1/d6/db/d35/d44/d92/f133 [1938103,75595] 0 2024-02-28T23:07:50.278 INFO:tasks.workunit.client.1.smithi084.stdout:3/968: creat d3/d27/d7d/db0/d1a/d29/f162 x:0 0 0 2024-02-28T23:07:50.279 INFO:tasks.workunit.client.1.smithi084.stdout:0/963: chown d1/d8/d18 132 1 2024-02-28T23:07:50.280 INFO:tasks.workunit.client.1.smithi084.stdout:2/830: read d5/d1f/d42/d74/fa1 [517290,51173] 0 2024-02-28T23:07:50.282 INFO:tasks.workunit.client.1.smithi084.stdout:0/964: readlink d1/d6/db/d35/d44/d113/l12b 0 2024-02-28T23:07:50.283 INFO:tasks.workunit.client.0.smithi057.stdout:5/791: symlink d4/d7/d8/lf7 0 2024-02-28T23:07:50.283 INFO:tasks.workunit.client.1.smithi084.stdout:9/986: rename d4/d6/d15/d3c/c10a to d4/d12f/c13f 0 2024-02-28T23:07:50.287 INFO:tasks.workunit.client.0.smithi057.stdout:8/817: dread d4/d13/d18/da7/ff3 [0,4194304] 0 2024-02-28T23:07:50.288 INFO:tasks.workunit.client.1.smithi084.stdout:8/917: write d0/d24/d6e/f8b [460190,37898] 0 2024-02-28T23:07:50.288 INFO:tasks.workunit.client.0.smithi057.stdout:5/792: write d4/d9/fcd [1448951,111854] 0 2024-02-28T23:07:50.289 INFO:tasks.workunit.client.1.smithi084.stdout:8/918: stat d0/d9/fb2 0 2024-02-28T23:07:50.289 INFO:tasks.workunit.client.0.smithi057.stdout:0/878: dread d17/d20/d4a/d78/d9d/fd6 [0,4194304] 0 2024-02-28T23:07:50.292 INFO:tasks.workunit.client.1.smithi084.stdout:8/919: chown d0/d24/d4d/ffe 7 1 2024-02-28T23:07:50.292 INFO:tasks.workunit.client.1.smithi084.stdout:4/880: mkdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/db4/d12a 0 2024-02-28T23:07:50.292 INFO:tasks.workunit.client.1.smithi084.stdout:0/965: rename d1/d6/db/d35/d44/d113/l12e to d1/d6/d59/l137 0 2024-02-28T23:07:50.297 INFO:tasks.workunit.client.1.smithi084.stdout:4/881: fdatasync d1/d18/d20/d14/d30/d47/f82 0 2024-02-28T23:07:50.298 INFO:tasks.workunit.client.1.smithi084.stdout:3/969: getdents d3/d50/d117/d3d/d43/d61/dce 0 2024-02-28T23:07:50.300 INFO:tasks.workunit.client.0.smithi057.stdout:8/818: mknod d4/de9/d56/d6a/d54/d6b/c123 0 2024-02-28T23:07:50.301 INFO:tasks.workunit.client.0.smithi057.stdout:0/879: creat d17/d20/d4a/d78/df7/f13a x:0 0 0 2024-02-28T23:07:50.304 INFO:tasks.workunit.client.0.smithi057.stdout:8/819: write d4/d13/d1a/d4a/fdc [1219058,66874] 0 2024-02-28T23:07:50.305 INFO:tasks.workunit.client.0.smithi057.stdout:5/793: write d4/d7/f83 [3929366,110720] 0 2024-02-28T23:07:50.306 INFO:tasks.workunit.client.1.smithi084.stdout:3/970: fsync d3/d50/d5a/d73/dc8/ff7 0 2024-02-28T23:07:50.306 INFO:tasks.workunit.client.0.smithi057.stdout:6/718: dread d0/d4/fd6 [0,4194304] 0 2024-02-28T23:07:50.307 INFO:tasks.workunit.client.0.smithi057.stdout:0/880: symlink d17/d20/d26/d48/l13b 0 2024-02-28T23:07:50.307 INFO:tasks.workunit.client.0.smithi057.stdout:8/820: mkdir d4/d13/d1a/d124 0 2024-02-28T23:07:50.311 INFO:tasks.workunit.client.0.smithi057.stdout:0/881: chown d17/d19/c3a 1738 1 2024-02-28T23:07:50.312 INFO:tasks.workunit.client.0.smithi057.stdout:4/723: sync 2024-02-28T23:07:50.312 INFO:tasks.workunit.client.0.smithi057.stdout:8/821: readlink d4/de9/d59/l85 0 2024-02-28T23:07:50.312 INFO:tasks.workunit.client.1.smithi084.stdout:3/971: symlink d3/d27/d7d/db0/d14/de1/l163 0 2024-02-28T23:07:50.316 INFO:tasks.workunit.client.0.smithi057.stdout:4/724: unlink d1a/d2f/db2/d49/d4d/f8d 0 2024-02-28T23:07:50.317 INFO:tasks.workunit.client.1.smithi084.stdout:2/831: dread d5/d7/d75/fac [0,4194304] 0 2024-02-28T23:07:50.319 INFO:tasks.workunit.client.1.smithi084.stdout:3/972: dread - d3/d50/d117/f108 zero size 2024-02-28T23:07:50.319 INFO:tasks.workunit.client.0.smithi057.stdout:8/822: rmdir d4/d13/d2c 39 2024-02-28T23:07:50.326 INFO:tasks.workunit.client.1.smithi084.stdout:3/973: write d3/f6 [10411604,56768] 0 2024-02-28T23:07:50.326 INFO:tasks.workunit.client.0.smithi057.stdout:0/882: link d17/d20/d26/d30/d6b/d71/la8 d17/l13c 0 2024-02-28T23:07:50.326 INFO:tasks.workunit.client.0.smithi057.stdout:8/823: dread - d4/f71 zero size 2024-02-28T23:07:50.326 INFO:tasks.workunit.client.0.smithi057.stdout:4/725: symlink d1a/d2f/db2/d49/d4d/dc9/lfb 0 2024-02-28T23:07:50.328 INFO:tasks.workunit.client.1.smithi084.stdout:0/966: truncate d1/d8/ff7 334135 0 2024-02-28T23:07:50.328 INFO:tasks.workunit.client.1.smithi084.stdout:3/974: write d3/d50/ff6 [2636034,84157] 0 2024-02-28T23:07:50.330 INFO:tasks.workunit.client.0.smithi057.stdout:0/883: fsync d17/d20/d4a/d78/db5/fcb 0 2024-02-28T23:07:50.331 INFO:tasks.workunit.client.0.smithi057.stdout:4/726: read - d1a/d48/d9c/fda zero size 2024-02-28T23:07:50.332 INFO:tasks.workunit.client.0.smithi057.stdout:8/824: truncate d4/de9/f2e 4289634 0 2024-02-28T23:07:50.335 INFO:tasks.workunit.client.1.smithi084.stdout:0/967: creat d1/d6/db/d35/d44/d92/f138 x:0 0 0 2024-02-28T23:07:50.335 INFO:tasks.workunit.client.1.smithi084.stdout:2/832: dread d5/d7/d12/d24/d3a/f96 [0,4194304] 0 2024-02-28T23:07:50.335 INFO:tasks.workunit.client.0.smithi057.stdout:5/794: dwrite d4/d32/df1/d24/da2/dbb/f93 [0,4194304] 0 2024-02-28T23:07:50.338 INFO:tasks.workunit.client.1.smithi084.stdout:3/975: rename d3/d27/d7d/db0/df/ded/ffd to d3/d50/d127/d14c/d15c/f164 0 2024-02-28T23:07:50.340 INFO:tasks.workunit.client.0.smithi057.stdout:4/727: write d1a/d48/d9c/d2e/d8f/db7/ff9 [725478,90158] 0 2024-02-28T23:07:50.343 INFO:tasks.workunit.client.1.smithi084.stdout:2/833: stat d5/d7/d12/d24/d2a/d35/ccf 0 2024-02-28T23:07:50.345 INFO:tasks.workunit.client.0.smithi057.stdout:0/884: chown d17/d20/d26/d48/d55/c101 5 1 2024-02-28T23:07:50.345 INFO:tasks.workunit.client.1.smithi084.stdout:3/976: write d3/d27/d7d/db0/f10a [83303,125531] 0 2024-02-28T23:07:50.347 INFO:tasks.workunit.client.1.smithi084.stdout:0/968: rename d1/d6/db/d35/d4a/d56 to d1/d6/db/d76/dee/df2/d139 0 2024-02-28T23:07:50.349 INFO:tasks.workunit.client.1.smithi084.stdout:4/882: dwrite d1/d18/d20/d65/d8e/d55/dc3/f10e [0,4194304] 0 2024-02-28T23:07:50.350 INFO:tasks.workunit.client.1.smithi084.stdout:8/920: dread d0/d9/d1b/dc1/fc7 [0,4194304] 0 2024-02-28T23:07:50.350 INFO:tasks.workunit.client.0.smithi057.stdout:0/885: write d17/d20/f66 [1869095,3394] 0 2024-02-28T23:07:50.351 INFO:tasks.workunit.client.1.smithi084.stdout:8/921: chown d0/d24 19860293 1 2024-02-28T23:07:50.352 INFO:tasks.workunit.client.0.smithi057.stdout:4/728: creat d1a/d48/d9c/d68/df0/ffc x:0 0 0 2024-02-28T23:07:50.354 INFO:tasks.workunit.client.1.smithi084.stdout:3/977: mknod d3/dae/c165 0 2024-02-28T23:07:50.354 INFO:tasks.workunit.client.1.smithi084.stdout:0/969: creat d1/d6/d59/f13a x:0 0 0 2024-02-28T23:07:50.357 INFO:tasks.workunit.client.1.smithi084.stdout:4/883: rename d1/d18/d20/d14/d101 to d1/d18/d20/d24/ddb/d12b 0 2024-02-28T23:07:50.366 INFO:tasks.workunit.client.1.smithi084.stdout:0/970: unlink d1/d8/d18/d4c/lf1 0 2024-02-28T23:07:50.368 INFO:tasks.workunit.client.0.smithi057.stdout:4/729: getdents d1a/d2f/db2/d49 0 2024-02-28T23:07:50.371 INFO:tasks.workunit.client.1.smithi084.stdout:2/834: getdents d5/d7/d2e/d4a/d5c 0 2024-02-28T23:07:50.371 INFO:tasks.workunit.client.0.smithi057.stdout:4/730: fsync f18 0 2024-02-28T23:07:50.373 INFO:tasks.workunit.client.1.smithi084.stdout:3/978: mknod d3/d50/c166 0 2024-02-28T23:07:50.373 INFO:tasks.workunit.client.0.smithi057.stdout:0/886: truncate d17/d20/d26/d48/d7c/d98/da6/fd5 1234179 0 2024-02-28T23:07:50.374 INFO:tasks.workunit.client.1.smithi084.stdout:0/971: mknod d1/d6/db/d35/d4a/c13b 0 2024-02-28T23:07:50.374 INFO:tasks.workunit.client.0.smithi057.stdout:0/887: fdatasync d17/d20/db6/f12c 0 2024-02-28T23:07:50.375 INFO:tasks.workunit.client.1.smithi084.stdout:2/835: write d5/d7/d9/d7c/d5b/f7f [1501440,112060] 0 2024-02-28T23:07:50.375 INFO:tasks.workunit.client.0.smithi057.stdout:5/795: dwrite d4/d9/d16/d52/f77 [0,4194304] 0 2024-02-28T23:07:50.378 INFO:tasks.workunit.client.0.smithi057.stdout:8/825: dwrite d4/de9/d56/da6/fd9 [0,4194304] 0 2024-02-28T23:07:50.380 INFO:tasks.workunit.client.1.smithi084.stdout:4/884: rename d1/d18/d20/d24/d2f/d3e/d52/d45/lb3 to d1/d18/d20/d14/d30/d47/da4/d10f/d115/l12c 0 2024-02-28T23:07:50.380 INFO:tasks.workunit.client.1.smithi084.stdout:2/836: chown d5/d7/d12/d24/d3a/d48/f8b 13054 1 2024-02-28T23:07:50.381 INFO:tasks.workunit.client.1.smithi084.stdout:8/922: write d0/d9/d1b/f1d [878244,65727] 0 2024-02-28T23:07:50.381 INFO:tasks.workunit.client.0.smithi057.stdout:5/796: write d4/d9/d14/d63/fb3 [1045402,87141] 0 2024-02-28T23:07:50.383 INFO:tasks.workunit.client.1.smithi084.stdout:2/837: fdatasync d5/d7/d12/d24/d3a/d48/f5a 0 2024-02-28T23:07:50.383 INFO:tasks.workunit.client.1.smithi084.stdout:3/979: rename d3/d27/d7d/db0/d1a/d29/d7e/l148 to d3/d27/d7d/db0/d1a/d45/l167 0 2024-02-28T23:07:50.388 INFO:tasks.workunit.client.0.smithi057.stdout:4/731: chown d1a/d2f/db2/d49/d4d/l96 3513643 1 2024-02-28T23:07:50.388 INFO:tasks.workunit.client.0.smithi057.stdout:5/797: symlink d4/d7/d8/lf8 0 2024-02-28T23:07:50.388 INFO:tasks.workunit.client.0.smithi057.stdout:8/826: write d4/de9/d56/d6a/ff4 [441307,87697] 0 2024-02-28T23:07:50.390 INFO:tasks.workunit.client.0.smithi057.stdout:0/888: write d17/f58 [313069,92740] 0 2024-02-28T23:07:50.394 INFO:tasks.workunit.client.0.smithi057.stdout:0/889: dread - d17/d20/d26/d30/f10a zero size 2024-02-28T23:07:50.396 INFO:tasks.workunit.client.0.smithi057.stdout:0/890: write d17/dda/f10f [425664,8037] 0 2024-02-28T23:07:50.397 INFO:tasks.workunit.client.1.smithi084.stdout:2/838: rename d5/d64/f6c to d5/ffe 0 2024-02-28T23:07:50.401 INFO:tasks.workunit.client.0.smithi057.stdout:0/891: write d17/d20/d26/d48/d7c/d98/da6/fdf [5067260,43068] 0 2024-02-28T23:07:50.405 INFO:tasks.workunit.client.0.smithi057.stdout:0/892: dread d17/d20/d26/d48/d7c/d98/da6/f11f [0,4194304] 0 2024-02-28T23:07:50.405 INFO:tasks.workunit.client.1.smithi084.stdout:2/839: creat d5/d7/d2e/d4a/d5c/fff x:0 0 0 2024-02-28T23:07:50.406 INFO:tasks.workunit.client.1.smithi084.stdout:0/972: dwrite d1/d6/db/d35/d82/f100 [0,4194304] 0 2024-02-28T23:07:50.410 INFO:tasks.workunit.client.0.smithi057.stdout:8/827: dread d4/de9/d7c/ff9 [0,4194304] 0 2024-02-28T23:07:50.411 INFO:tasks.workunit.client.0.smithi057.stdout:0/893: read d17/d19/fe1 [1092835,105222] 0 2024-02-28T23:07:50.412 INFO:tasks.workunit.client.1.smithi084.stdout:8/923: write d0/d9/d30/d33/f5e [524459,95025] 0 2024-02-28T23:07:50.412 INFO:tasks.workunit.client.0.smithi057.stdout:5/798: dread d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:50.412 INFO:tasks.workunit.client.1.smithi084.stdout:3/980: dwrite d3/d50/d117/d3d/f67 [0,4194304] 0 2024-02-28T23:07:50.415 INFO:tasks.workunit.client.0.smithi057.stdout:8/828: read d4/d13/d18/d62/d66/d74/fc7 [686583,9917] 0 2024-02-28T23:07:50.423 INFO:tasks.workunit.client.0.smithi057.stdout:8/829: link d4/d13/d18/d62/d66/c93 d4/de9/c125 0 2024-02-28T23:07:50.424 INFO:tasks.workunit.client.1.smithi084.stdout:0/973: rename d1/d8/d69/d6b/d85/da5/f102 to d1/d8/f13c 0 2024-02-28T23:07:50.424 INFO:tasks.workunit.client.1.smithi084.stdout:4/885: truncate d1/d18/d20/d24/d2f/d3e/fe5 1370811 0 2024-02-28T23:07:50.425 INFO:tasks.workunit.client.0.smithi057.stdout:0/894: write d17/d20/d26/d48/d7c/d98/da6/fd5 [255493,71011] 0 2024-02-28T23:07:50.426 INFO:tasks.workunit.client.0.smithi057.stdout:8/830: creat d4/de9/d56/d88/f126 x:0 0 0 2024-02-28T23:07:50.427 INFO:tasks.workunit.client.1.smithi084.stdout:3/981: dread - d3/d27/d7d/db0/d1a/d45/dde/f157 zero size 2024-02-28T23:07:50.428 INFO:tasks.workunit.client.1.smithi084.stdout:2/840: dread d5/d7/d9/f88 [0,4194304] 0 2024-02-28T23:07:50.429 INFO:tasks.workunit.client.0.smithi057.stdout:0/895: creat d17/d20/d23/d47/d60/f13d x:0 0 0 2024-02-28T23:07:50.430 INFO:tasks.workunit.client.0.smithi057.stdout:8/831: mknod d4/de9/d56/d6a/d54/c127 0 2024-02-28T23:07:50.432 INFO:tasks.workunit.client.0.smithi057.stdout:0/896: fdatasync d17/d20/f124 0 2024-02-28T23:07:50.433 INFO:tasks.workunit.client.1.smithi084.stdout:2/841: fsync d5/f15 0 2024-02-28T23:07:50.435 INFO:tasks.workunit.client.0.smithi057.stdout:8/832: dread d4/d13/d18/d62/d66/fbf [0,4194304] 0 2024-02-28T23:07:50.442 INFO:tasks.workunit.client.0.smithi057.stdout:0/897: link d17/d20/d23/d47/d60/fd3 d17/d20/d4a/d78/db5/dba/f13e 0 2024-02-28T23:07:50.443 INFO:tasks.workunit.client.0.smithi057.stdout:0/898: stat d17/d20/d26/d48/d7c/d98/da6/l129 0 2024-02-28T23:07:50.445 INFO:tasks.workunit.client.1.smithi084.stdout:3/982: chown d3/d27/d7d/db0/d1a/d45/dde/le8 11 1 2024-02-28T23:07:50.446 INFO:tasks.workunit.client.0.smithi057.stdout:5/799: dread d4/d9/d16/d52/fe1 [0,4194304] 0 2024-02-28T23:07:50.446 INFO:tasks.workunit.client.0.smithi057.stdout:0/899: creat d17/d19/dd2/f13f x:0 0 0 2024-02-28T23:07:50.447 INFO:tasks.workunit.client.1.smithi084.stdout:0/974: truncate d1/d8/d18/d4c/d6c/d74/fad 3676731 0 2024-02-28T23:07:50.449 INFO:tasks.workunit.client.0.smithi057.stdout:0/900: write d17/d20/d4a/d78/df7/f13a [680383,14354] 0 2024-02-28T23:07:50.449 INFO:tasks.workunit.client.1.smithi084.stdout:4/886: getdents d1/d18/d20/d24/d2f/d3e/d52/d48 0 2024-02-28T23:07:50.451 INFO:tasks.workunit.client.0.smithi057.stdout:5/800: getdents d4/d9/d16/d52/d7a 0 2024-02-28T23:07:50.451 INFO:tasks.workunit.client.1.smithi084.stdout:8/924: dread d0/d9/d30/db6/f89 [0,4194304] 0 2024-02-28T23:07:50.453 INFO:tasks.workunit.client.0.smithi057.stdout:0/901: dread d17/d20/d26/f69 [0,4194304] 0 2024-02-28T23:07:50.456 INFO:tasks.workunit.client.0.smithi057.stdout:0/902: creat d17/d20/d4a/d78/df7/f140 x:0 0 0 2024-02-28T23:07:50.457 INFO:tasks.workunit.client.1.smithi084.stdout:2/842: dwrite d5/d7/d12/d24/d2a/d35/f8e [0,4194304] 0 2024-02-28T23:07:50.458 INFO:tasks.workunit.client.1.smithi084.stdout:4/887: rmdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5 39 2024-02-28T23:07:50.460 INFO:tasks.workunit.client.1.smithi084.stdout:8/925: truncate d0/d9/d30/db6/fe9 864293 0 2024-02-28T23:07:50.460 INFO:tasks.workunit.client.1.smithi084.stdout:3/983: truncate d3/d50/d117/d7b/dbf/f114 2859958 0 2024-02-28T23:07:50.463 INFO:tasks.workunit.client.1.smithi084.stdout:2/843: mknod d5/d1f/c100 0 2024-02-28T23:07:50.466 INFO:tasks.workunit.client.0.smithi057.stdout:2/847: sync 2024-02-28T23:07:50.467 INFO:tasks.workunit.client.1.smithi084.stdout:2/844: chown d5/d7/d12/d24/d2a/c30 2115 1 2024-02-28T23:07:50.468 INFO:tasks.workunit.client.1.smithi084.stdout:4/888: dread d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fc6 [0,4194304] 0 2024-02-28T23:07:50.471 INFO:tasks.workunit.client.0.smithi057.stdout:5/801: link d4/d32/df1/d24/c5d d4/d9/d14/d3b/d61/d71/cf9 0 2024-02-28T23:07:50.471 INFO:tasks.workunit.client.0.smithi057.stdout:0/903: unlink l7 0 2024-02-28T23:07:50.471 INFO:tasks.workunit.client.1.smithi084.stdout:8/926: mknod d0/d24/d83/c128 0 2024-02-28T23:07:50.472 INFO:tasks.workunit.client.0.smithi057.stdout:5/802: dread - d4/d9/d14/d63/fad zero size 2024-02-28T23:07:50.473 INFO:tasks.workunit.client.1.smithi084.stdout:3/984: dread d3/d50/d117/d3d/faa [0,4194304] 0 2024-02-28T23:07:50.474 INFO:tasks.workunit.client.0.smithi057.stdout:2/848: mkdir d8/d11/db3/d107 0 2024-02-28T23:07:50.475 INFO:tasks.workunit.client.1.smithi084.stdout:8/927: read d0/d9/d1b/d38/d43/d51/ff5 [741276,40626] 0 2024-02-28T23:07:50.478 INFO:tasks.workunit.client.1.smithi084.stdout:3/985: creat d3/d27/d7d/db0/d1a/d29/f168 x:0 0 0 2024-02-28T23:07:50.480 INFO:tasks.workunit.client.0.smithi057.stdout:8/833: dread d4/de9/d56/da6/faf [0,4194304] 0 2024-02-28T23:07:50.481 INFO:tasks.workunit.client.1.smithi084.stdout:8/928: fdatasync d0/d24/d4d/f6a 0 2024-02-28T23:07:50.481 INFO:tasks.workunit.client.0.smithi057.stdout:5/803: read d4/d7/d8/fde [1073536,130743] 0 2024-02-28T23:07:50.483 INFO:tasks.workunit.client.1.smithi084.stdout:4/889: getdents d1/d18/d20/d24/d2f/d3e/d52/d45 0 2024-02-28T23:07:50.483 INFO:tasks.workunit.client.0.smithi057.stdout:8/834: mkdir d4/d13/d7f/d128 0 2024-02-28T23:07:50.485 INFO:tasks.workunit.client.0.smithi057.stdout:8/835: write d4/de9/d56/d6a/d42/fcd [791913,80679] 0 2024-02-28T23:07:50.487 INFO:tasks.workunit.client.0.smithi057.stdout:2/849: dread d8/d11/d39/f62 [0,4194304] 0 2024-02-28T23:07:50.487 INFO:tasks.workunit.client.1.smithi084.stdout:4/890: dread - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f11a zero size 2024-02-28T23:07:50.488 INFO:tasks.workunit.client.0.smithi057.stdout:8/836: mknod d4/de9/d56/d6a/dbd/c129 0 2024-02-28T23:07:50.489 INFO:tasks.workunit.client.0.smithi057.stdout:0/904: write d17/d20/d4a/d78/db5/dba/f13e [8167814,11754] 0 2024-02-28T23:07:50.489 INFO:tasks.workunit.client.1.smithi084.stdout:0/975: dread d1/d8/f3e [0,4194304] 0 2024-02-28T23:07:50.490 INFO:tasks.workunit.client.0.smithi057.stdout:2/850: mkdir d8/d11/d21/d108 0 2024-02-28T23:07:50.491 INFO:tasks.workunit.client.1.smithi084.stdout:0/976: chown d1/d6/d2d/lc4 0 1 2024-02-28T23:07:50.492 INFO:tasks.workunit.client.1.smithi084.stdout:3/986: mkdir d3/d27/d7d/db0/d1a/d45/d86/d15d/d169 0 2024-02-28T23:07:50.494 INFO:tasks.workunit.client.1.smithi084.stdout:2/845: write d5/d7/d2e/ffa [1149039,41214] 0 2024-02-28T23:07:50.497 INFO:tasks.workunit.client.1.smithi084.stdout:4/891: mkdir d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/d12d 0 2024-02-28T23:07:50.499 INFO:tasks.workunit.client.1.smithi084.stdout:3/987: fdatasync d3/d50/d117/d3d/d43/de3/f113 0 2024-02-28T23:07:50.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/851: dwrite d8/d11/d21/d28/d41/fd9 [0,4194304] 0 2024-02-28T23:07:50.509 INFO:tasks.workunit.client.1.smithi084.stdout:3/988: creat d3/d27/d7d/db0/d1a/d45/f16a x:0 0 0 2024-02-28T23:07:50.510 INFO:tasks.workunit.client.1.smithi084.stdout:8/929: dwrite d0/d24/d4d/ffe [0,4194304] 0 2024-02-28T23:07:50.510 INFO:tasks.workunit.client.1.smithi084.stdout:0/977: dwrite d1/d8/d18/f61 [0,4194304] 0 2024-02-28T23:07:50.520 INFO:tasks.workunit.client.1.smithi084.stdout:4/892: rename d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/db4 to d1/d12e 0 2024-02-28T23:07:50.525 INFO:tasks.workunit.client.1.smithi084.stdout:0/978: mkdir d1/d6/d13d 0 2024-02-28T23:07:50.531 INFO:tasks.workunit.client.1.smithi084.stdout:4/893: rename d1/d18/d20/d24/d2f/d3e/d52/d48/d73/f128 to d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/f12f 0 2024-02-28T23:07:50.533 INFO:tasks.workunit.client.1.smithi084.stdout:0/979: symlink d1/d6/db/d35/d82/l13e 0 2024-02-28T23:07:50.535 INFO:tasks.workunit.client.1.smithi084.stdout:4/894: chown d1/d18/d20/d24/ddb/f100 55634008 1 2024-02-28T23:07:50.540 INFO:tasks.workunit.client.1.smithi084.stdout:3/989: dwrite d3/d27/d7d/f8f [0,4194304] 0 2024-02-28T23:07:50.548 INFO:tasks.workunit.client.1.smithi084.stdout:3/990: symlink d3/d27/d7d/db0/d1a/d45/d86/d15d/l16b 0 2024-02-28T23:07:50.548 INFO:tasks.workunit.client.1.smithi084.stdout:0/980: truncate d1/d8/f3c 1364173 0 2024-02-28T23:07:50.550 INFO:tasks.workunit.client.1.smithi084.stdout:3/991: chown d3/d27/l26 554625 1 2024-02-28T23:07:50.550 INFO:tasks.workunit.client.1.smithi084.stdout:0/981: chown d1/d6/db/d37/f96 1232453 1 2024-02-28T23:07:50.554 INFO:tasks.workunit.client.1.smithi084.stdout:3/992: symlink d3/d27/d7d/db0/d14/de1/l16c 0 2024-02-28T23:07:50.556 INFO:tasks.workunit.client.1.smithi084.stdout:4/895: dwrite d1/d18/d20/d24/d2f/d3e/d52/d57/fa0 [0,4194304] 0 2024-02-28T23:07:50.556 INFO:tasks.workunit.client.1.smithi084.stdout:3/993: mkdir d3/d50/d5a/d16d 0 2024-02-28T23:07:50.559 INFO:tasks.workunit.client.1.smithi084.stdout:4/896: dread - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/fea zero size 2024-02-28T23:07:50.567 INFO:tasks.workunit.client.1.smithi084.stdout:4/897: fsync d1/d18/d20/d14/fab 0 2024-02-28T23:07:50.570 INFO:tasks.workunit.client.1.smithi084.stdout:4/898: stat d1/d18/d20/d14/d121/d122 0 2024-02-28T23:07:50.574 INFO:tasks.workunit.client.1.smithi084.stdout:4/899: write d1/d18/d20/d14/d30/d47/da4/d10f/d115/f11c [831665,2203] 0 2024-02-28T23:07:50.579 INFO:tasks.workunit.client.1.smithi084.stdout:3/994: dwrite d3/d50/d117/d7b/dbf/fc7 [0,4194304] 0 2024-02-28T23:07:50.583 INFO:tasks.workunit.client.1.smithi084.stdout:3/995: creat d3/d50/d117/d3d/d43/de3/f16e x:0 0 0 2024-02-28T23:07:50.587 INFO:tasks.workunit.client.1.smithi084.stdout:3/996: getdents d3/d27/d7d 0 2024-02-28T23:07:50.588 INFO:tasks.workunit.client.1.smithi084.stdout:3/997: readlink d3/d27/l82 0 2024-02-28T23:07:50.588 INFO:tasks.workunit.client.1.smithi084.stdout:0/982: dread d1/d6/db/d35/d44/f4e [0,4194304] 0 2024-02-28T23:07:50.589 INFO:tasks.workunit.client.1.smithi084.stdout:3/998: chown d3/f6 8056 1 2024-02-28T23:07:50.593 INFO:tasks.workunit.client.1.smithi084.stdout:0/983: mkdir d1/d8/d78/d111/d13f 0 2024-02-28T23:07:50.595 INFO:tasks.workunit.client.1.smithi084.stdout:0/984: mkdir d1/d8/d5f/d93/d140 0 2024-02-28T23:07:50.595 INFO:tasks.workunit.client.1.smithi084.stdout:3/999: write d3/d27/d7d/dd9/f110 [675258,126478] 0 2024-02-28T23:07:50.601 INFO:tasks.workunit.client.1.smithi084.stdout:0/985: dread d1/d8/d69/d6b/fbe [0,4194304] 0 2024-02-28T23:07:50.631 INFO:tasks.workunit.client.0.smithi057.stdout:2/852: creat d8/f109 x:0 0 0 2024-02-28T23:07:50.638 INFO:tasks.workunit.client.0.smithi057.stdout:2/853: creat d8/d11/d21/d70/d91/f10a x:0 0 0 2024-02-28T23:07:50.639 INFO:tasks.workunit.client.0.smithi057.stdout:0/905: link d17/d7a/lac d17/d20/d26/d48/d55/d88/l141 0 2024-02-28T23:07:50.646 INFO:tasks.workunit.client.0.smithi057.stdout:2/854: creat d8/d11/d15/daf/d105/f10b x:0 0 0 2024-02-28T23:07:50.648 INFO:tasks.workunit.client.0.smithi057.stdout:8/837: dwrite d4/d13/d18/f31 [0,4194304] 0 2024-02-28T23:07:50.656 INFO:tasks.workunit.client.0.smithi057.stdout:8/838: chown d4/d13/d2c/d4e/l117 11 1 2024-02-28T23:07:50.657 INFO:tasks.workunit.client.0.smithi057.stdout:0/906: truncate d17/d20/f27 3500745 0 2024-02-28T23:07:50.658 INFO:tasks.workunit.client.0.smithi057.stdout:2/855: unlink d8/d11/d21/d28/d49/d85/fe5 0 2024-02-28T23:07:50.662 INFO:tasks.workunit.client.0.smithi057.stdout:8/839: rename d4/de9/d56/d6a/d54/d61/l76 to d4/d13/d2c/l12a 0 2024-02-28T23:07:50.666 INFO:tasks.workunit.client.0.smithi057.stdout:8/840: read d4/d13/d1a/d4a/f6e [88557,60693] 0 2024-02-28T23:07:50.667 INFO:tasks.workunit.client.0.smithi057.stdout:2/856: fdatasync d8/d11/ffa 0 2024-02-28T23:07:50.669 INFO:tasks.workunit.client.0.smithi057.stdout:8/841: chown d4/de9/d56/d6a/d42 611606 1 2024-02-28T23:07:50.672 INFO:tasks.workunit.client.0.smithi057.stdout:8/842: write d4/d13/d2c/db1/fb7 [411851,111564] 0 2024-02-28T23:07:50.675 INFO:tasks.workunit.client.0.smithi057.stdout:0/907: rename d17/d19/f28 to d17/d20/d4a/d78/d9d/f142 0 2024-02-28T23:07:50.678 INFO:tasks.workunit.client.0.smithi057.stdout:2/857: mkdir d8/d11/d21/de6/d10c 0 2024-02-28T23:07:50.682 INFO:tasks.workunit.client.0.smithi057.stdout:8/843: mknod d4/de9/c12b 0 2024-02-28T23:07:50.683 INFO:tasks.workunit.client.0.smithi057.stdout:0/908: symlink d17/d19/dd2/d12d/l143 0 2024-02-28T23:07:50.686 INFO:tasks.workunit.client.0.smithi057.stdout:2/858: creat d8/d11/d39/d83/dde/f10d x:0 0 0 2024-02-28T23:07:50.686 INFO:tasks.workunit.client.1.smithi084.stdout:1/948: sync 2024-02-28T23:07:50.688 INFO:tasks.workunit.client.0.smithi057.stdout:0/909: creat d17/d20/d4a/d78/dd4/f144 x:0 0 0 2024-02-28T23:07:50.690 INFO:tasks.workunit.client.0.smithi057.stdout:2/859: mknod d8/dae/de1/c10e 0 2024-02-28T23:07:50.692 INFO:tasks.workunit.client.0.smithi057.stdout:8/844: rename d4/de9/d56/d6a/d54/d61/d113 to d4/de9/d12c 0 2024-02-28T23:07:50.692 INFO:tasks.workunit.client.0.smithi057.stdout:0/910: mknod d17/d19/c145 0 2024-02-28T23:07:50.693 INFO:tasks.workunit.client.0.smithi057.stdout:2/860: fdatasync d8/d11/d15/daf/fc0 0 2024-02-28T23:07:50.695 INFO:tasks.workunit.client.0.smithi057.stdout:8/845: write d4/de9/d56/d6a/d54/f70 [1344466,68791] 0 2024-02-28T23:07:50.698 INFO:tasks.workunit.client.0.smithi057.stdout:2/861: mkdir d8/dae/de1/d10f 0 2024-02-28T23:07:50.707 INFO:tasks.workunit.client.0.smithi057.stdout:2/862: link d8/d11/d21/d28/d41/la6 d8/d11/d21/d70/d99/l110 0 2024-02-28T23:07:50.711 INFO:tasks.workunit.client.0.smithi057.stdout:2/863: rename d8/d11/d15/f95 to d8/d11/d21/d108/f111 0 2024-02-28T23:07:50.711 INFO:tasks.workunit.client.0.smithi057.stdout:0/911: write d17/d19/f8f [1719152,109719] 0 2024-02-28T23:07:50.713 INFO:tasks.workunit.client.0.smithi057.stdout:0/912: fdatasync d17/d19/f53 0 2024-02-28T23:07:50.714 INFO:tasks.workunit.client.1.smithi084.stdout:1/949: dwrite d12/d1b/d91/d9b/da5/dcb/dd4/fe9 [0,4194304] 0 2024-02-28T23:07:50.717 INFO:tasks.workunit.client.0.smithi057.stdout:0/913: mkdir d17/d20/d23/d37/d146 0 2024-02-28T23:07:50.719 INFO:tasks.workunit.client.0.smithi057.stdout:2/864: rename d8/d11/d39/dbc/l103 to d8/d11/d15/l112 0 2024-02-28T23:07:50.722 INFO:tasks.workunit.client.0.smithi057.stdout:8/846: dwrite d4/de9/d56/d88/fe6 [0,4194304] 0 2024-02-28T23:07:50.723 INFO:tasks.workunit.client.1.smithi084.stdout:1/950: mkdir d12/d1b/d63/d132/d133 0 2024-02-28T23:07:50.728 INFO:tasks.workunit.client.1.smithi084.stdout:1/951: write d12/d1b/d2e/d30/d50/da8/f11a [603297,4704] 0 2024-02-28T23:07:50.735 INFO:tasks.workunit.client.1.smithi084.stdout:1/952: symlink d12/d1b/d2e/l134 0 2024-02-28T23:07:50.738 INFO:tasks.workunit.client.1.smithi084.stdout:1/953: write d12/d1b/d63/f56 [3819220,115013] 0 2024-02-28T23:07:50.739 INFO:tasks.workunit.client.0.smithi057.stdout:0/914: write d17/d20/d23/d47/f7f [1995792,75729] 0 2024-02-28T23:07:50.744 INFO:tasks.workunit.client.0.smithi057.stdout:0/915: mkdir d17/d20/d23/d37/d147 0 2024-02-28T23:07:50.745 INFO:tasks.workunit.client.1.smithi084.stdout:1/954: rename d12/da2/f112 to d12/d1b/d2e/d30/d77/da0/f135 0 2024-02-28T23:07:50.747 INFO:tasks.workunit.client.0.smithi057.stdout:8/847: dwrite d4/d13/d2c/db1/fde [0,4194304] 0 2024-02-28T23:07:50.749 INFO:tasks.workunit.client.0.smithi057.stdout:0/916: rename d17/d20/d23/d37 to d17/d19/d148 0 2024-02-28T23:07:50.750 INFO:tasks.workunit.client.1.smithi084.stdout:1/955: getdents d12/d1b/d2e/d52/d61 0 2024-02-28T23:07:50.752 INFO:tasks.workunit.client.0.smithi057.stdout:8/848: truncate d4/d13/d18/d62/d66/d74/ffa 364001 0 2024-02-28T23:07:50.763 INFO:tasks.workunit.client.0.smithi057.stdout:2/865: dread d8/fe [0,4194304] 0 2024-02-28T23:07:50.766 INFO:tasks.workunit.client.1.smithi084.stdout:1/956: dwrite d12/d84/fcf [0,4194304] 0 2024-02-28T23:07:50.767 INFO:tasks.workunit.client.1.smithi084.stdout:5/928: sync 2024-02-28T23:07:50.767 INFO:tasks.workunit.client.1.smithi084.stdout:9/987: sync 2024-02-28T23:07:50.767 INFO:tasks.workunit.client.1.smithi084.stdout:6/923: sync 2024-02-28T23:07:50.768 INFO:tasks.workunit.client.0.smithi057.stdout:2/866: read d8/d11/d39/f87 [173299,14855] 0 2024-02-28T23:07:50.770 INFO:tasks.workunit.client.0.smithi057.stdout:8/849: dwrite d4/d13/d18/da7/f101 [0,4194304] 0 2024-02-28T23:07:50.770 INFO:tasks.workunit.client.1.smithi084.stdout:6/924: write d1/d13/d2e/d41/d4d/d6c/fe7 [1168078,6692] 0 2024-02-28T23:07:50.774 INFO:tasks.workunit.client.0.smithi057.stdout:0/917: dwrite d17/d20/d26/d48/d55/d88/fcc [0,4194304] 0 2024-02-28T23:07:50.775 INFO:tasks.workunit.client.0.smithi057.stdout:2/867: mknod d8/d11/d39/d7b/d8b/c113 0 2024-02-28T23:07:50.775 INFO:tasks.workunit.client.0.smithi057.stdout:3/669: sync 2024-02-28T23:07:50.776 INFO:tasks.workunit.client.0.smithi057.stdout:9/735: sync 2024-02-28T23:07:50.779 INFO:tasks.workunit.client.1.smithi084.stdout:5/929: creat dc/de/df/d16/d34/daa/f15b x:0 0 0 2024-02-28T23:07:50.779 INFO:tasks.workunit.client.1.smithi084.stdout:1/957: link d12/d1b/d2e/d30/l53 d12/d1b/d63/d7d/l136 0 2024-02-28T23:07:50.785 INFO:tasks.workunit.client.1.smithi084.stdout:9/988: getdents d4/d6/d15/d75 0 2024-02-28T23:07:50.786 INFO:tasks.workunit.client.0.smithi057.stdout:0/918: creat d17/d20/d23/d47/d60/d125/f149 x:0 0 0 2024-02-28T23:07:50.791 INFO:tasks.workunit.client.0.smithi057.stdout:0/919: creat d17/d19/d4d/f14a x:0 0 0 2024-02-28T23:07:50.796 INFO:tasks.workunit.client.1.smithi084.stdout:6/925: dwrite d1/d9/d1d/d1e/d33/f11c [0,4194304] 0 2024-02-28T23:07:50.797 INFO:tasks.workunit.client.1.smithi084.stdout:1/958: rename d12/d1b/d2e/d52/f5a to d12/da2/f137 0 2024-02-28T23:07:50.797 INFO:tasks.workunit.client.0.smithi057.stdout:8/850: dwrite d4/de9/d59/f98 [0,4194304] 0 2024-02-28T23:07:50.797 INFO:tasks.workunit.client.0.smithi057.stdout:0/920: truncate d17/d20/d26/f67 1486107 0 2024-02-28T23:07:50.798 INFO:tasks.workunit.client.0.smithi057.stdout:3/670: write d3/f5a [1705479,82291] 0 2024-02-28T23:07:50.805 INFO:tasks.workunit.client.1.smithi084.stdout:1/959: creat d12/d1b/d2e/d30/d77/da0/d117/f138 x:0 0 0 2024-02-28T23:07:50.807 INFO:tasks.workunit.client.1.smithi084.stdout:6/926: link d1/d13/d2e/d41/d4d/f63 d1/d13/d50/dbd/d131/f132 0 2024-02-28T23:07:50.807 INFO:tasks.workunit.client.0.smithi057.stdout:3/671: creat d3/d4/d7/d9a/d9b/fd5 x:0 0 0 2024-02-28T23:07:50.809 INFO:tasks.workunit.client.0.smithi057.stdout:2/868: dwrite d8/d11/d39/d83/fa3 [0,4194304] 0 2024-02-28T23:07:50.810 INFO:tasks.workunit.client.0.smithi057.stdout:9/736: dwrite dd/f2d [0,4194304] 0 2024-02-28T23:07:50.814 INFO:tasks.workunit.client.1.smithi084.stdout:5/930: write dc/de/df/d16/ffa [954400,75466] 0 2024-02-28T23:07:50.816 INFO:tasks.workunit.client.1.smithi084.stdout:8/930: sync 2024-02-28T23:07:50.816 INFO:tasks.workunit.client.1.smithi084.stdout:0/986: sync 2024-02-28T23:07:50.818 INFO:tasks.workunit.client.0.smithi057.stdout:0/921: truncate d17/d20/d26/d30/d6b/d71/f7e 1090218 0 2024-02-28T23:07:50.818 INFO:tasks.workunit.client.1.smithi084.stdout:9/989: dwrite d4/d6/d15/d75/d2d/dbd/dde/f116 [0,4194304] 0 2024-02-28T23:07:50.818 INFO:tasks.workunit.client.0.smithi057.stdout:3/672: truncate d3/d8/d27/f77 267965 0 2024-02-28T23:07:50.818 INFO:tasks.workunit.client.0.smithi057.stdout:9/737: symlink dd/d46/d6f/lee 0 2024-02-28T23:07:50.821 INFO:tasks.workunit.client.1.smithi084.stdout:1/960: creat d12/d1b/d2e/f139 x:0 0 0 2024-02-28T23:07:50.823 INFO:tasks.workunit.client.1.smithi084.stdout:0/987: read d1/d6/d59/d70/f73 [1047114,79271] 0 2024-02-28T23:07:50.823 INFO:tasks.workunit.client.1.smithi084.stdout:4/900: sync 2024-02-28T23:07:50.823 INFO:tasks.workunit.client.1.smithi084.stdout:2/846: sync 2024-02-28T23:07:50.826 INFO:tasks.workunit.client.0.smithi057.stdout:8/851: read d4/d13/d1a/f89 [2555850,128910] 0 2024-02-28T23:07:50.827 INFO:tasks.workunit.client.0.smithi057.stdout:0/922: chown d17/d20/d26/d30/d6b/cfc 1 1 2024-02-28T23:07:50.828 INFO:tasks.workunit.client.0.smithi057.stdout:8/852: stat d4/d13/d2c/fb9 0 2024-02-28T23:07:50.829 INFO:tasks.workunit.client.0.smithi057.stdout:8/853: chown d4/d13/d18 19217282 1 2024-02-28T23:07:50.830 INFO:tasks.workunit.client.1.smithi084.stdout:8/931: dread - d0/d24/d78/fd2 zero size 2024-02-28T23:07:50.830 INFO:tasks.workunit.client.0.smithi057.stdout:3/673: creat d3/d4/d7/d9a/d9b/dab/dd0/fd6 x:0 0 0 2024-02-28T23:07:50.833 INFO:tasks.workunit.client.0.smithi057.stdout:4/732: sync 2024-02-28T23:07:50.833 INFO:tasks.workunit.client.0.smithi057.stdout:1/692: sync 2024-02-28T23:07:50.833 INFO:tasks.workunit.client.0.smithi057.stdout:6/719: sync 2024-02-28T23:07:50.833 INFO:tasks.workunit.client.0.smithi057.stdout:7/781: sync 2024-02-28T23:07:50.833 INFO:tasks.workunit.client.0.smithi057.stdout:5/804: sync 2024-02-28T23:07:50.839 INFO:tasks.workunit.client.0.smithi057.stdout:0/923: write d17/d20/d26/d48/d7c/d98/da6/fdf [1309752,2829] 0 2024-02-28T23:07:50.839 INFO:tasks.workunit.client.0.smithi057.stdout:1/693: readlink d2/d7/lda 0 2024-02-28T23:07:50.840 INFO:tasks.workunit.client.1.smithi084.stdout:9/990: link d4/d6/d15/d12/d41/fe5 d4/d6/f140 0 2024-02-28T23:07:50.840 INFO:tasks.workunit.client.0.smithi057.stdout:7/782: write d0/d3f/fe3 [130114,82728] 0 2024-02-28T23:07:50.844 INFO:tasks.workunit.client.0.smithi057.stdout:4/733: readlink d1a/d48/d9c/d2e/lf7 0 2024-02-28T23:07:50.847 INFO:tasks.workunit.client.1.smithi084.stdout:8/932: rename d0/d9/d30/db6/c11b to d0/d9/d30/d33/d71/d84/dd9/c129 0 2024-02-28T23:07:50.854 INFO:tasks.workunit.client.1.smithi084.stdout:8/933: mkdir d0/d44/dfb/d12a 0 2024-02-28T23:07:50.855 INFO:tasks.workunit.client.1.smithi084.stdout:9/991: truncate d4/d6/d15/d75/d2d/daa/f11c 340325 0 2024-02-28T23:07:50.855 INFO:tasks.workunit.client.1.smithi084.stdout:2/847: dwrite d5/d7/d2e/ffa [0,4194304] 0 2024-02-28T23:07:50.855 INFO:tasks.workunit.client.0.smithi057.stdout:6/720: dread d0/d4/d25/d60/fcf [0,4194304] 0 2024-02-28T23:07:50.858 INFO:tasks.workunit.client.0.smithi057.stdout:9/738: dwrite dd/de/f6b [0,4194304] 0 2024-02-28T23:07:50.860 INFO:tasks.workunit.client.0.smithi057.stdout:8/854: dread d4/d13/d18/d62/d66/fe5 [0,4194304] 0 2024-02-28T23:07:50.861 INFO:tasks.workunit.client.1.smithi084.stdout:8/934: readlink d0/d24/d78/le7 0 2024-02-28T23:07:50.863 INFO:tasks.workunit.client.1.smithi084.stdout:6/927: dread d1/d13/d47/d91/dba/f95 [0,4194304] 0 2024-02-28T23:07:50.868 INFO:tasks.workunit.client.0.smithi057.stdout:7/783: getdents d0/d5/d4a 0 2024-02-28T23:07:50.868 INFO:tasks.workunit.client.0.smithi057.stdout:9/739: creat dd/d46/d4d/fef x:0 0 0 2024-02-28T23:07:50.869 INFO:tasks.workunit.client.0.smithi057.stdout:8/855: creat d4/d13/d18/d62/d66/d74/f12d x:0 0 0 2024-02-28T23:07:50.873 INFO:tasks.workunit.client.1.smithi084.stdout:6/928: truncate d1/d13/d47/d91/fff 740796 0 2024-02-28T23:07:50.873 INFO:tasks.workunit.client.1.smithi084.stdout:9/992: rename d4/fc to d4/d6/d15/d75/d2d/f141 0 2024-02-28T23:07:50.873 INFO:tasks.workunit.client.1.smithi084.stdout:2/848: dread d5/d7/d12/d24/d3a/d48/f8b [0,4194304] 0 2024-02-28T23:07:50.874 INFO:tasks.workunit.client.0.smithi057.stdout:3/674: write d3/d8/d1b/f44 [4443283,34384] 0 2024-02-28T23:07:50.877 INFO:tasks.workunit.client.0.smithi057.stdout:6/721: rmdir d0/d1b/d3e/d3d/d74/dca 0 2024-02-28T23:07:50.877 INFO:tasks.workunit.client.0.smithi057.stdout:7/784: write d0/d5/d25/d3e/f5a [573247,47353] 0 2024-02-28T23:07:50.877 INFO:tasks.workunit.client.1.smithi084.stdout:6/929: unlink d1/d13/d2e/d41/d2c/dfb/lda 0 2024-02-28T23:07:50.879 INFO:tasks.workunit.client.0.smithi057.stdout:0/924: write d17/d20/d26/d30/f96 [181902,49269] 0 2024-02-28T23:07:50.880 INFO:tasks.workunit.client.0.smithi057.stdout:8/856: mkdir d4/df7/d12e 0 2024-02-28T23:07:50.883 INFO:tasks.workunit.client.1.smithi084.stdout:5/931: truncate dc/de/df/d31/dd1/d5f/fe2 3132598 0 2024-02-28T23:07:50.886 INFO:tasks.workunit.client.0.smithi057.stdout:9/740: truncate dd/de/d24/d63/d77/dcf/f50 4521841 0 2024-02-28T23:07:50.888 INFO:tasks.workunit.client.0.smithi057.stdout:2/869: dwrite d8/d11/d39/d83/dde/fcc [0,4194304] 0 2024-02-28T23:07:50.889 INFO:tasks.workunit.client.1.smithi084.stdout:6/930: creat d1/d13/d50/d74/d9e/dc2/f133 x:0 0 0 2024-02-28T23:07:50.891 INFO:tasks.workunit.client.1.smithi084.stdout:5/932: mknod dc/de/d144/c15c 0 2024-02-28T23:07:50.893 INFO:tasks.workunit.client.0.smithi057.stdout:3/675: creat d3/d4/d7/d9a/d9b/fd7 x:0 0 0 2024-02-28T23:07:50.893 INFO:tasks.workunit.client.1.smithi084.stdout:6/931: rmdir d1/d9/ddc/d12b 39 2024-02-28T23:07:50.894 INFO:tasks.workunit.client.0.smithi057.stdout:5/805: dwrite d4/d9/d14/d3b/d61/fa8 [0,4194304] 0 2024-02-28T23:07:50.894 INFO:tasks.workunit.client.0.smithi057.stdout:0/925: fsync d17/d20/d26/d48/ffa 0 2024-02-28T23:07:50.895 INFO:tasks.workunit.client.1.smithi084.stdout:4/901: dwrite d1/d4b/fa6 [0,4194304] 0 2024-02-28T23:07:50.896 INFO:tasks.workunit.client.1.smithi084.stdout:5/933: symlink dc/de/df/d16/d34/d3c/d128/l15d 0 2024-02-28T23:07:50.898 INFO:tasks.workunit.client.1.smithi084.stdout:0/988: dwrite d1/d6/d59/f5a [0,4194304] 0 2024-02-28T23:07:50.899 INFO:tasks.workunit.client.1.smithi084.stdout:4/902: read - d1/d18/ffa zero size 2024-02-28T23:07:50.899 INFO:tasks.workunit.client.1.smithi084.stdout:6/932: symlink d1/d13/d2e/l134 0 2024-02-28T23:07:50.900 INFO:tasks.workunit.client.0.smithi057.stdout:6/722: getdents d0/d1b/d3e 0 2024-02-28T23:07:50.901 INFO:tasks.workunit.client.1.smithi084.stdout:8/935: dread d0/d11/f2a [0,4194304] 0 2024-02-28T23:07:50.902 INFO:tasks.workunit.client.0.smithi057.stdout:0/926: chown d17/d20/d4a/d78/db5/dba/f137 0 1 2024-02-28T23:07:50.903 INFO:tasks.workunit.client.1.smithi084.stdout:6/933: rmdir d1/d9/ddc 39 2024-02-28T23:07:50.903 INFO:tasks.workunit.client.1.smithi084.stdout:1/961: dread d12/d1b/d63/fbc [0,4194304] 0 2024-02-28T23:07:50.903 INFO:tasks.workunit.client.1.smithi084.stdout:5/934: write dc/de/df/d31/dd1/d5f/d9b/f12f [1053499,62004] 0 2024-02-28T23:07:50.906 INFO:tasks.workunit.client.0.smithi057.stdout:5/806: fsync d4/d9/d16/d2c/f80 0 2024-02-28T23:07:50.906 INFO:tasks.workunit.client.0.smithi057.stdout:8/857: write d4/f71 [425997,34224] 0 2024-02-28T23:07:50.908 INFO:tasks.workunit.client.0.smithi057.stdout:2/870: truncate d8/d11/db3/de9/ff9 946772 0 2024-02-28T23:07:50.911 INFO:tasks.workunit.client.0.smithi057.stdout:3/676: creat d3/d8/d27/d70/fd8 x:0 0 0 2024-02-28T23:07:50.912 INFO:tasks.workunit.client.0.smithi057.stdout:6/723: stat d0/d1b/d3e/d3d/d42/d4f/d92/ce2 0 2024-02-28T23:07:50.912 INFO:tasks.workunit.client.1.smithi084.stdout:0/989: creat d1/d6/d1b/f141 x:0 0 0 2024-02-28T23:07:50.914 INFO:tasks.workunit.client.1.smithi084.stdout:2/849: write d5/d7/d12/d24/d3a/f96 [1719205,93564] 0 2024-02-28T23:07:50.914 INFO:tasks.workunit.client.1.smithi084.stdout:8/936: getdents d0/d11/d126 0 2024-02-28T23:07:50.919 INFO:tasks.workunit.client.1.smithi084.stdout:8/937: write d0/d9/d30/d33/d71/d84/dce/f109 [570554,109717] 0 2024-02-28T23:07:50.930 INFO:tasks.workunit.client.1.smithi084.stdout:5/935: rename dc/de/df/d16/l37 to dc/de/df/d31/dd1/d5f/d9b/l15e 0 2024-02-28T23:07:50.933 INFO:tasks.workunit.client.1.smithi084.stdout:0/990: symlink d1/l142 0 2024-02-28T23:07:50.934 INFO:tasks.workunit.client.1.smithi084.stdout:2/850: symlink d5/l101 0 2024-02-28T23:07:50.936 INFO:tasks.workunit.client.1.smithi084.stdout:2/851: chown d5/d7/d12/d24 24591094 1 2024-02-28T23:07:50.936 INFO:tasks.workunit.client.1.smithi084.stdout:1/962: getdents d12/d1b/d2e/d30/d50/da8/dd9/def 0 2024-02-28T23:07:50.938 INFO:tasks.workunit.client.1.smithi084.stdout:0/991: write d1/d8/da1/f132 [752169,84694] 0 2024-02-28T23:07:50.943 INFO:tasks.workunit.client.1.smithi084.stdout:8/938: link d0/d9/d30/d33/d71/d84/dc0/fc6 d0/d9/d30/d33/d71/d84/dce/f12b 0 2024-02-28T23:07:50.944 INFO:tasks.workunit.client.1.smithi084.stdout:2/852: read - d5/d7/d2e/f93 zero size 2024-02-28T23:07:50.948 INFO:tasks.workunit.client.1.smithi084.stdout:5/936: rename dc/de/df/d16/d34/d3c/d77/df8/fed to dc/de/df/d31/d8e/da9/dab/f15f 0 2024-02-28T23:07:50.949 INFO:tasks.workunit.client.1.smithi084.stdout:6/934: dwrite d1/d13/d47/d91/ff7 [0,4194304] 0 2024-02-28T23:07:50.956 INFO:tasks.workunit.client.1.smithi084.stdout:2/853: unlink d5/d7/d9/d7c/fdc 0 2024-02-28T23:07:50.956 INFO:tasks.workunit.client.1.smithi084.stdout:8/939: mkdir d0/d24/d78/df0/d12c 0 2024-02-28T23:07:50.957 INFO:tasks.workunit.client.1.smithi084.stdout:4/903: dread d1/f9 [0,4194304] 0 2024-02-28T23:07:50.967 INFO:tasks.workunit.client.1.smithi084.stdout:1/963: dwrite d12/d1b/d2e/d30/d50/f12c [0,4194304] 0 2024-02-28T23:07:50.971 INFO:tasks.workunit.client.1.smithi084.stdout:0/992: dread d1/d6/d1b/f58 [0,4194304] 0 2024-02-28T23:07:50.983 INFO:tasks.workunit.client.1.smithi084.stdout:9/993: sync 2024-02-28T23:07:50.987 INFO:tasks.workunit.client.0.smithi057.stdout:3/677: chown d3/d8/l6e 422 1 2024-02-28T23:07:50.987 INFO:tasks.workunit.client.0.smithi057.stdout:5/807: mkdir d4/d9/d14/d3b/d61/dfa 0 2024-02-28T23:07:50.987 INFO:tasks.workunit.client.0.smithi057.stdout:8/858: dread - d4/df7/f11e zero size 2024-02-28T23:07:50.987 INFO:tasks.workunit.client.0.smithi057.stdout:1/694: dwrite d2/d33/d54/f8b [0,4194304] 0 2024-02-28T23:07:50.987 INFO:tasks.workunit.client.0.smithi057.stdout:9/741: truncate dd/de/d24/d63/d77/dcf/d40/d44/da5/fea 229092 0 2024-02-28T23:07:50.990 INFO:tasks.workunit.client.1.smithi084.stdout:6/935: stat d1/d9/ddc/d12b/l12f 0 2024-02-28T23:07:50.990 INFO:tasks.workunit.client.1.smithi084.stdout:9/994: dread d4/d6/d15/d75/d2d/dbd/f108 [0,4194304] 0 2024-02-28T23:07:50.991 INFO:tasks.workunit.client.1.smithi084.stdout:5/937: truncate dc/d51/d7c/f93 526140 0 2024-02-28T23:07:50.993 INFO:tasks.workunit.client.1.smithi084.stdout:5/938: dread - dc/de/df/d16/d34/daa/dae/f142 zero size 2024-02-28T23:07:50.996 INFO:tasks.workunit.client.0.smithi057.stdout:2/871: dread d8/d11/d39/fcb [0,4194304] 0 2024-02-28T23:07:50.997 INFO:tasks.workunit.client.1.smithi084.stdout:4/904: mknod d1/d18/d20/d14/d30/d47/da4/d10f/d115/c130 0 2024-02-28T23:07:50.997 INFO:tasks.workunit.client.1.smithi084.stdout:5/939: write dc/de/df/d31/d8e/da9/fb3 [3650140,80991] 0 2024-02-28T23:07:50.997 INFO:tasks.workunit.client.0.smithi057.stdout:6/724: creat d0/d17/ff3 x:0 0 0 2024-02-28T23:07:50.997 INFO:tasks.workunit.client.0.smithi057.stdout:1/695: mkdir d2/d7/df/db/dfc 0 2024-02-28T23:07:50.997 INFO:tasks.workunit.client.0.smithi057.stdout:3/678: truncate d3/d4/d1d/d47/d62/fa5 1565271 0 2024-02-28T23:07:50.998 INFO:tasks.workunit.client.0.smithi057.stdout:5/808: mknod d4/d9/d16/d52/d7a/cfb 0 2024-02-28T23:07:51.000 INFO:tasks.workunit.client.0.smithi057.stdout:8/859: read - d4/de9/d56/d6a/fb0 zero size 2024-02-28T23:07:51.002 INFO:tasks.workunit.client.0.smithi057.stdout:7/785: dwrite d0/d5/d36/d4e/d15/f60 [0,4194304] 0 2024-02-28T23:07:51.003 INFO:tasks.workunit.client.1.smithi084.stdout:9/995: symlink d4/d12f/l142 0 2024-02-28T23:07:51.004 INFO:tasks.workunit.client.0.smithi057.stdout:6/725: mkdir d0/d17/d94/da0/de6/df4 0 2024-02-28T23:07:51.004 INFO:tasks.workunit.client.0.smithi057.stdout:9/742: dread dd/d46/dc4/f96 [0,4194304] 0 2024-02-28T23:07:51.007 INFO:tasks.workunit.client.0.smithi057.stdout:2/872: fdatasync d8/d11/d21/d28/f75 0 2024-02-28T23:07:51.009 INFO:tasks.workunit.client.1.smithi084.stdout:2/854: rename d5/d7/d12/d24/d2a/d35/l6b to d5/d7/d9/d94/l102 0 2024-02-28T23:07:51.013 INFO:tasks.workunit.client.1.smithi084.stdout:9/996: mknod d4/d32/d3f/d89/d10c/c143 0 2024-02-28T23:07:51.013 INFO:tasks.workunit.client.0.smithi057.stdout:3/679: readlink d3/d8/d1b/l28 0 2024-02-28T23:07:51.014 INFO:tasks.workunit.client.0.smithi057.stdout:1/696: mknod d2/d7/df/db/dfc/cfd 0 2024-02-28T23:07:51.014 INFO:tasks.workunit.client.0.smithi057.stdout:5/809: creat d4/d32/df1/d24/ffc x:0 0 0 2024-02-28T23:07:51.015 INFO:tasks.workunit.client.0.smithi057.stdout:6/726: dread - d0/d1b/d3e/d3d/d74/fb8 zero size 2024-02-28T23:07:51.016 INFO:tasks.workunit.client.0.smithi057.stdout:2/873: creat d8/d11/d39/dbc/f114 x:0 0 0 2024-02-28T23:07:51.017 INFO:tasks.workunit.client.1.smithi084.stdout:0/993: rename d1/d6/df6/f11f to d1/d6/db/d35/d4a/d12d/f143 0 2024-02-28T23:07:51.017 INFO:tasks.workunit.client.0.smithi057.stdout:9/743: rename dd/de/d24/d63/dc1/dd1 to dd/de/d24/d63/dc1/df0 0 2024-02-28T23:07:51.020 INFO:tasks.workunit.client.1.smithi084.stdout:9/997: unlink d4/lc8 0 2024-02-28T23:07:51.020 INFO:tasks.workunit.client.0.smithi057.stdout:9/744: stat ca 0 2024-02-28T23:07:51.021 INFO:tasks.workunit.client.0.smithi057.stdout:2/874: chown d8/d11/d39/d7b/d8b/f96 90 1 2024-02-28T23:07:51.023 INFO:tasks.workunit.client.0.smithi057.stdout:5/810: write d4/d9/d14/d3b/d61/d71/fa7 [146926,96791] 0 2024-02-28T23:07:51.023 INFO:tasks.workunit.client.1.smithi084.stdout:5/940: rename dc/de/c14d to dc/de/c160 0 2024-02-28T23:07:51.025 INFO:tasks.workunit.client.1.smithi084.stdout:0/994: creat d1/d8/d69/db2/f144 x:0 0 0 2024-02-28T23:07:51.025 INFO:tasks.workunit.client.1.smithi084.stdout:5/941: read - dc/de/df/d16/f14f zero size 2024-02-28T23:07:51.028 INFO:tasks.workunit.client.0.smithi057.stdout:9/745: chown dd/de/d24/d63/d77/dcf/d40/d44/d92/fa8 0 1 2024-02-28T23:07:51.029 INFO:tasks.workunit.client.0.smithi057.stdout:5/811: write d4/d9/fcd [2030211,66670] 0 2024-02-28T23:07:51.032 INFO:tasks.workunit.client.1.smithi084.stdout:0/995: read d1/d8/d78/fa4 [272367,17819] 0 2024-02-28T23:07:51.033 INFO:tasks.workunit.client.0.smithi057.stdout:2/875: read d8/d11/d39/d7b/d8b/fdf [1512904,33799] 0 2024-02-28T23:07:51.033 INFO:tasks.workunit.client.0.smithi057.stdout:8/860: link d4/d13/d18/d62/d66/c93 d4/d13/d1a/d4a/db3/c12f 0 2024-02-28T23:07:51.033 INFO:tasks.workunit.client.1.smithi084.stdout:9/998: unlink d4/d6/d15/d3c/f7a 0 2024-02-28T23:07:51.034 INFO:tasks.workunit.client.1.smithi084.stdout:0/996: read d1/d6/d59/d70/f8d [653862,95183] 0 2024-02-28T23:07:51.034 INFO:tasks.workunit.client.1.smithi084.stdout:2/855: getdents d5/d7/d9/d7c/d5b/d9f 0 2024-02-28T23:07:51.035 INFO:tasks.workunit.client.0.smithi057.stdout:5/812: write d4/d32/f6e [3908820,80574] 0 2024-02-28T23:07:51.035 INFO:tasks.workunit.client.0.smithi057.stdout:8/861: chown d4/d13/d7f 942604158 1 2024-02-28T23:07:51.036 INFO:tasks.workunit.client.1.smithi084.stdout:5/942: truncate dc/de/df/d31/dd1/d5f/d9b/f9c 314786 0 2024-02-28T23:07:51.042 INFO:tasks.workunit.client.0.smithi057.stdout:5/813: truncate d4/d9/d14/d63/d88/fd8 207820 0 2024-02-28T23:07:51.043 INFO:tasks.workunit.client.0.smithi057.stdout:9/746: dread dd/de/d24/f85 [0,4194304] 0 2024-02-28T23:07:51.044 INFO:tasks.workunit.client.1.smithi084.stdout:1/964: write d12/d1b/d2e/f4e [2658008,34101] 0 2024-02-28T23:07:51.046 INFO:tasks.workunit.client.1.smithi084.stdout:9/999: truncate d4/d6/d15/d3c/fb0 439248 0 2024-02-28T23:07:51.049 INFO:tasks.workunit.client.0.smithi057.stdout:8/862: rename d4/d13/d18/d62/d66/d74/la9 to d4/d13/d1a/d4a/db3/dfc/l130 0 2024-02-28T23:07:51.050 INFO:tasks.workunit.client.1.smithi084.stdout:4/905: dread d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf [0,4194304] 0 2024-02-28T23:07:51.051 INFO:tasks.workunit.client.0.smithi057.stdout:1/697: write d2/d7/df/db/d3a/f3c [262065,63983] 0 2024-02-28T23:07:51.051 INFO:tasks.workunit.client.1.smithi084.stdout:6/936: dwrite d1/d13/d50/f99 [0,4194304] 0 2024-02-28T23:07:51.052 INFO:tasks.workunit.client.0.smithi057.stdout:4/734: sync 2024-02-28T23:07:51.053 INFO:tasks.workunit.client.1.smithi084.stdout:1/965: creat d12/d1b/d91/f13a x:0 0 0 2024-02-28T23:07:51.054 INFO:tasks.workunit.client.0.smithi057.stdout:9/747: mknod dd/de/d24/d63/d77/dcf/d40/d44/d5c/cf1 0 2024-02-28T23:07:51.055 INFO:tasks.workunit.client.0.smithi057.stdout:6/727: truncate d0/d1b/fd5 845098 0 2024-02-28T23:07:51.056 INFO:tasks.workunit.client.0.smithi057.stdout:3/680: dread d3/d8/d27/d70/fc5 [0,4194304] 0 2024-02-28T23:07:51.056 INFO:tasks.workunit.client.1.smithi084.stdout:8/940: dwrite d0/d24/d9e/fa2 [0,4194304] 0 2024-02-28T23:07:51.058 INFO:tasks.workunit.client.1.smithi084.stdout:2/856: write d5/d7/d12/d24/d2a/d35/fa2 [2990465,75737] 0 2024-02-28T23:07:51.060 INFO:tasks.workunit.client.0.smithi057.stdout:8/863: mkdir d4/de9/d56/da6/d131 0 2024-02-28T23:07:51.060 INFO:tasks.workunit.client.0.smithi057.stdout:7/786: dwrite d0/d5/d25/d3e/f54 [0,4194304] 0 2024-02-28T23:07:51.063 INFO:tasks.workunit.client.1.smithi084.stdout:5/943: write dc/de/df/f110 [1023045,14460] 0 2024-02-28T23:07:51.063 INFO:tasks.workunit.client.0.smithi057.stdout:1/698: creat d2/d33/d54/dc8/ffe x:0 0 0 2024-02-28T23:07:51.064 INFO:tasks.workunit.client.0.smithi057.stdout:3/681: read - d3/d4f/d4c/fcd zero size 2024-02-28T23:07:51.069 INFO:tasks.workunit.client.0.smithi057.stdout:9/748: mkdir dd/de/d24/d63/d77/df2 0 2024-02-28T23:07:51.069 INFO:tasks.workunit.client.0.smithi057.stdout:4/735: mkdir d1a/d48/d9c/dfd 0 2024-02-28T23:07:51.069 INFO:tasks.workunit.client.1.smithi084.stdout:8/941: write d0/d24/d9e/ff8 [2970030,85131] 0 2024-02-28T23:07:51.071 INFO:tasks.workunit.client.0.smithi057.stdout:2/876: write d8/d11/d15/f80 [2586171,83415] 0 2024-02-28T23:07:51.072 INFO:tasks.workunit.client.1.smithi084.stdout:8/942: chown d0/d24/d83/c128 0 1 2024-02-28T23:07:51.074 INFO:tasks.workunit.client.1.smithi084.stdout:4/906: stat d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/c4c 0 2024-02-28T23:07:51.075 INFO:tasks.workunit.client.0.smithi057.stdout:9/749: stat dd/de/d14/d48/d4b/ce1 0 2024-02-28T23:07:51.075 INFO:tasks.workunit.client.1.smithi084.stdout:2/857: creat d5/d7/d75/dcd/f103 x:0 0 0 2024-02-28T23:07:51.076 INFO:tasks.workunit.client.1.smithi084.stdout:5/944: mkdir dc/de/df/d31/dd1/da4/d161 0 2024-02-28T23:07:51.076 INFO:tasks.workunit.client.0.smithi057.stdout:0/927: sync 2024-02-28T23:07:51.078 INFO:tasks.workunit.client.0.smithi057.stdout:8/864: creat d4/d13/d2c/db1/f132 x:0 0 0 2024-02-28T23:07:51.080 INFO:tasks.workunit.client.1.smithi084.stdout:0/997: dwrite d1/d6/db/d76/dee/df2/d139/f89 [0,4194304] 0 2024-02-28T23:07:51.080 INFO:tasks.workunit.client.0.smithi057.stdout:8/865: chown d4/d13/d2c/fb9 1619895 1 2024-02-28T23:07:51.081 INFO:tasks.workunit.client.0.smithi057.stdout:1/699: mkdir d2/d7/df/db/d12/d19/d48/dff 0 2024-02-28T23:07:51.087 INFO:tasks.workunit.client.0.smithi057.stdout:6/728: rename d0/d4/d25/d60/fd8 to d0/d17/ff5 0 2024-02-28T23:07:51.087 INFO:tasks.workunit.client.0.smithi057.stdout:2/877: truncate d8/d11/f1d 2686833 0 2024-02-28T23:07:51.089 INFO:tasks.workunit.client.1.smithi084.stdout:0/998: chown d1/d6/db/d37/f71 203311 1 2024-02-28T23:07:51.090 INFO:tasks.workunit.client.0.smithi057.stdout:0/928: creat d17/d20/df3/f14b x:0 0 0 2024-02-28T23:07:51.091 INFO:tasks.workunit.client.1.smithi084.stdout:0/999: chown d1/d6/d59/f13a 775001 1 2024-02-28T23:07:51.091 INFO:tasks.workunit.client.1.smithi084.stdout:6/937: dwrite d1/d13/d47/d91/ff7 [0,4194304] 0 2024-02-28T23:07:51.093 INFO:tasks.workunit.client.0.smithi057.stdout:6/729: read - d0/d1b/d3e/fdb zero size 2024-02-28T23:07:51.093 INFO:tasks.workunit.client.0.smithi057.stdout:2/878: chown d8/d11/d39/dda/f104 275851371 1 2024-02-28T23:07:51.093 INFO:tasks.workunit.client.0.smithi057.stdout:4/736: rename d1a/d48/d9c/d68/df0 to d1a/d2f/db2/dcb/dfe 0 2024-02-28T23:07:51.094 INFO:tasks.workunit.client.1.smithi084.stdout:5/945: fdatasync dc/de/df/d31/dd1/f61 0 2024-02-28T23:07:51.097 INFO:tasks.workunit.client.0.smithi057.stdout:9/750: truncate dd/de/d4a/d9f/fab 3533451 0 2024-02-28T23:07:51.098 INFO:tasks.workunit.client.1.smithi084.stdout:5/946: write dc/d51/f12b [204427,128653] 0 2024-02-28T23:07:51.099 INFO:tasks.workunit.client.0.smithi057.stdout:0/929: truncate d17/d20/d26/d48/d7c/d98/da6/f11f 558679 0 2024-02-28T23:07:51.099 INFO:tasks.workunit.client.0.smithi057.stdout:1/700: mkdir d2/d7/df/db/d12/d19/d48/dff/d100 0 2024-02-28T23:07:51.101 INFO:tasks.workunit.client.0.smithi057.stdout:7/787: getdents d0/d5/d36/d4e/d15/d29 0 2024-02-28T23:07:51.102 INFO:tasks.workunit.client.0.smithi057.stdout:8/866: truncate d4/de9/d56/f109 829031 0 2024-02-28T23:07:51.103 INFO:tasks.workunit.client.0.smithi057.stdout:2/879: creat d8/d11/d39/d83/f115 x:0 0 0 2024-02-28T23:07:51.106 INFO:tasks.workunit.client.0.smithi057.stdout:8/867: read - d4/df7/f111 zero size 2024-02-28T23:07:51.106 INFO:tasks.workunit.client.1.smithi084.stdout:2/858: rmdir d5 39 2024-02-28T23:07:51.110 INFO:tasks.workunit.client.1.smithi084.stdout:6/938: mkdir d1/d88/dbf/d135 0 2024-02-28T23:07:51.113 INFO:tasks.workunit.client.1.smithi084.stdout:6/939: stat d1/d13/d2e/d41/d2c 0 2024-02-28T23:07:51.113 INFO:tasks.workunit.client.1.smithi084.stdout:1/966: dwrite d12/d1b/d63/d7d/f95 [0,4194304] 0 2024-02-28T23:07:51.118 INFO:tasks.workunit.client.1.smithi084.stdout:6/940: chown d1/d88/dbf/fde 2989026 1 2024-02-28T23:07:51.119 INFO:tasks.workunit.client.1.smithi084.stdout:8/943: getdents d0/d24/d6e 0 2024-02-28T23:07:51.120 INFO:tasks.workunit.client.1.smithi084.stdout:1/967: dread - d12/d1b/d3b/fc8 zero size 2024-02-28T23:07:51.125 INFO:tasks.workunit.client.1.smithi084.stdout:5/947: rmdir dc/de/df/d16/d34/de8/d146 0 2024-02-28T23:07:51.126 INFO:tasks.workunit.client.1.smithi084.stdout:2/859: unlink d5/f31 0 2024-02-28T23:07:51.126 INFO:tasks.workunit.client.0.smithi057.stdout:5/814: dread f2 [0,4194304] 0 2024-02-28T23:07:51.132 INFO:tasks.workunit.client.0.smithi057.stdout:6/730: dread d0/d4/f50 [0,4194304] 0 2024-02-28T23:07:51.133 INFO:tasks.workunit.client.1.smithi084.stdout:1/968: mknod d12/d1b/d2e/d52/d61/c13b 0 2024-02-28T23:07:51.133 INFO:tasks.workunit.client.1.smithi084.stdout:4/907: link d1/d18/d20/d14/d30/l56 d1/d18/d20/d24/d2f/d3e/l131 0 2024-02-28T23:07:51.133 INFO:tasks.workunit.client.1.smithi084.stdout:8/944: truncate d0/d9/d30/d33/d71/d84/f8d 4799840 0 2024-02-28T23:07:51.134 INFO:tasks.workunit.client.1.smithi084.stdout:1/969: mkdir d12/d1b/d91/d9b/d13c 0 2024-02-28T23:07:51.136 INFO:tasks.workunit.client.1.smithi084.stdout:4/908: fsync d1/d18/d20/d24/d2f/d3e/fe5 0 2024-02-28T23:07:51.140 INFO:tasks.workunit.client.1.smithi084.stdout:4/909: rmdir d1/d18/d20/d65/d8e/d55 39 2024-02-28T23:07:51.141 INFO:tasks.workunit.client.0.smithi057.stdout:8/868: dwrite d4/d13/d18/d62/d66/d74/f110 [0,4194304] 0 2024-02-28T23:07:51.143 INFO:tasks.workunit.client.1.smithi084.stdout:2/860: dread d5/d7/d75/fac [0,4194304] 0 2024-02-28T23:07:51.145 INFO:tasks.workunit.client.0.smithi057.stdout:3/682: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:51.148 INFO:tasks.workunit.client.0.smithi057.stdout:2/880: fsync d8/d11/d15/db9/fbf 0 2024-02-28T23:07:51.148 INFO:tasks.workunit.client.1.smithi084.stdout:6/941: truncate d1/d9/d1d/f32 190621 0 2024-02-28T23:07:51.151 INFO:tasks.workunit.client.1.smithi084.stdout:8/945: dwrite d0/d24/d78/fbd [0,4194304] 0 2024-02-28T23:07:51.157 INFO:tasks.workunit.client.1.smithi084.stdout:8/946: dread - d0/d9/d1b/d38/d43/dae/db8/fd8 zero size 2024-02-28T23:07:51.157 INFO:tasks.workunit.client.0.smithi057.stdout:5/815: rename d4/d9/d14/d3b/d61/d71 to d4/d9/d16/d52/dfd 0 2024-02-28T23:07:51.157 INFO:tasks.workunit.client.1.smithi084.stdout:1/970: creat d12/d1b/d2e/d52/d61/d74/d12a/f13d x:0 0 0 2024-02-28T23:07:51.161 INFO:tasks.workunit.client.0.smithi057.stdout:4/737: getdents d1a/d2f/db2/dcb 0 2024-02-28T23:07:51.162 INFO:tasks.workunit.client.0.smithi057.stdout:4/738: stat d1a/d48/d9c 0 2024-02-28T23:07:51.165 INFO:tasks.workunit.client.0.smithi057.stdout:0/930: getdents d17/d19/dd2/d12d 0 2024-02-28T23:07:51.165 INFO:tasks.workunit.client.0.smithi057.stdout:8/869: dwrite d4/d13/d18/f31 [4194304,4194304] 0 2024-02-28T23:07:51.168 INFO:tasks.workunit.client.0.smithi057.stdout:0/931: write d17/d20/d26/f46 [683190,127326] 0 2024-02-28T23:07:51.170 INFO:tasks.workunit.client.1.smithi084.stdout:1/971: rmdir d12/d1b/d91 39 2024-02-28T23:07:51.170 INFO:tasks.workunit.client.1.smithi084.stdout:6/942: creat d1/d9/ddc/f136 x:0 0 0 2024-02-28T23:07:51.171 INFO:tasks.workunit.client.0.smithi057.stdout:2/881: dread - d8/d11/d21/d28/d41/fd6 zero size 2024-02-28T23:07:51.175 INFO:tasks.workunit.client.0.smithi057.stdout:4/739: mkdir d1a/d2f/db2/d49/d4d/dc9/dff 0 2024-02-28T23:07:51.175 INFO:tasks.workunit.client.0.smithi057.stdout:9/751: write dd/de/d24/d72/f91 [267219,40581] 0 2024-02-28T23:07:51.176 INFO:tasks.workunit.client.1.smithi084.stdout:8/947: dread - d0/d9/d1b/d38/d43/dae/db8/fde zero size 2024-02-28T23:07:51.178 INFO:tasks.workunit.client.0.smithi057.stdout:3/683: truncate d3/d4f/d4c/d91/fc0 2578903 0 2024-02-28T23:07:51.180 INFO:tasks.workunit.client.1.smithi084.stdout:2/861: dwrite d5/d7/d9/d7c/d5b/da6/fbb [0,4194304] 0 2024-02-28T23:07:51.181 INFO:tasks.workunit.client.0.smithi057.stdout:5/816: dread d4/d9/d14/d3b/d61/fa8 [0,4194304] 0 2024-02-28T23:07:51.182 INFO:tasks.workunit.client.1.smithi084.stdout:6/943: write d1/d13/d50/dbd/dd1/f121 [504724,30901] 0 2024-02-28T23:07:51.183 INFO:tasks.workunit.client.1.smithi084.stdout:1/972: creat d12/d1b/d2e/d52/dd7/f13e x:0 0 0 2024-02-28T23:07:51.188 INFO:tasks.workunit.client.0.smithi057.stdout:7/788: truncate d0/fd7 1500944 0 2024-02-28T23:07:51.189 INFO:tasks.workunit.client.1.smithi084.stdout:4/910: truncate d1/d18/d20/d65/d8e/d55/dc3/f10e 2819685 0 2024-02-28T23:07:51.189 INFO:tasks.workunit.client.1.smithi084.stdout:5/948: dread dc/de/df/d16/d34/f4d [0,4194304] 0 2024-02-28T23:07:51.189 INFO:tasks.workunit.client.1.smithi084.stdout:8/948: getdents d0/d9/d1b/d38/d43/d51/d54 0 2024-02-28T23:07:51.190 INFO:tasks.workunit.client.0.smithi057.stdout:7/789: stat d0/d98 0 2024-02-28T23:07:51.191 INFO:tasks.workunit.client.0.smithi057.stdout:0/932: creat d17/d20/df3/f14c x:0 0 0 2024-02-28T23:07:51.192 INFO:tasks.workunit.client.0.smithi057.stdout:9/752: mknod dd/de/d4a/d9f/cf3 0 2024-02-28T23:07:51.195 INFO:tasks.workunit.client.1.smithi084.stdout:4/911: mknod d1/d18/d20/d65/d8e/d55/dc3/c132 0 2024-02-28T23:07:51.198 INFO:tasks.workunit.client.1.smithi084.stdout:8/949: mkdir d0/d44/dfb/d12d 0 2024-02-28T23:07:51.198 INFO:tasks.workunit.client.1.smithi084.stdout:4/912: read - d1/d18/d20/d24/d2f/d3e/d52/d45/fed zero size 2024-02-28T23:07:51.199 INFO:tasks.workunit.client.0.smithi057.stdout:7/790: chown d0/d5/d36/d4e/cd5 156706 1 2024-02-28T23:07:51.204 INFO:tasks.workunit.client.0.smithi057.stdout:1/701: dwrite d2/d71/d62/f64 [0,4194304] 0 2024-02-28T23:07:51.206 INFO:tasks.workunit.client.1.smithi084.stdout:4/913: creat d1/d18/d20/d24/ddb/d12b/f133 x:0 0 0 2024-02-28T23:07:51.207 INFO:tasks.workunit.client.0.smithi057.stdout:1/702: read - d2/fe9 zero size 2024-02-28T23:07:51.209 INFO:tasks.workunit.client.1.smithi084.stdout:2/862: write d5/d7/d12/d24/fe2 [906203,567] 0 2024-02-28T23:07:51.209 INFO:tasks.workunit.client.1.smithi084.stdout:5/949: write dc/fc8 [3839751,41727] 0 2024-02-28T23:07:51.211 INFO:tasks.workunit.client.1.smithi084.stdout:6/944: write d1/d9/d1d/f32 [1183513,111675] 0 2024-02-28T23:07:51.211 INFO:tasks.workunit.client.1.smithi084.stdout:8/950: truncate d0/d9/d1b/d38/d43/d51/fb3 5731936 0 2024-02-28T23:07:51.213 INFO:tasks.workunit.client.1.smithi084.stdout:2/863: fdatasync d5/d7/d2e/d4a/d5c/fff 0 2024-02-28T23:07:51.215 INFO:tasks.workunit.client.1.smithi084.stdout:6/945: read d1/d13/d50/d74/d81/f94 [729703,45403] 0 2024-02-28T23:07:51.215 INFO:tasks.workunit.client.0.smithi057.stdout:4/740: dwrite d1a/d48/d9c/d68/dc5/fe7 [0,4194304] 0 2024-02-28T23:07:51.216 INFO:tasks.workunit.client.1.smithi084.stdout:5/950: creat dc/de/df/d6e/f162 x:0 0 0 2024-02-28T23:07:51.217 INFO:tasks.workunit.client.1.smithi084.stdout:8/951: mknod d0/d44/dfb/c12e 0 2024-02-28T23:07:51.221 INFO:tasks.workunit.client.1.smithi084.stdout:4/914: creat d1/d18/d20/d65/f134 x:0 0 0 2024-02-28T23:07:51.223 INFO:tasks.workunit.client.1.smithi084.stdout:6/946: dread - d1/d13/d47/d91/dba/de9/f113 zero size 2024-02-28T23:07:51.224 INFO:tasks.workunit.client.1.smithi084.stdout:8/952: creat d0/d44/dfb/f12f x:0 0 0 2024-02-28T23:07:51.225 INFO:tasks.workunit.client.1.smithi084.stdout:5/951: creat dc/de/df/d16/d34/d3c/d128/f163 x:0 0 0 2024-02-28T23:07:51.226 INFO:tasks.workunit.client.1.smithi084.stdout:4/915: truncate d1/d18/d20/d14/f4f 189549 0 2024-02-28T23:07:51.227 INFO:tasks.workunit.client.1.smithi084.stdout:6/947: fdatasync d1/f24 0 2024-02-28T23:07:51.227 INFO:tasks.workunit.client.0.smithi057.stdout:1/703: dwrite d2/d33/fe2 [0,4194304] 0 2024-02-28T23:07:51.229 INFO:tasks.workunit.client.1.smithi084.stdout:5/952: stat dc/de/df/d16/d34/d3c/d77/df8/f14c 0 2024-02-28T23:07:51.230 INFO:tasks.workunit.client.1.smithi084.stdout:4/916: readlink d1/d18/d20/d14/d30/l7c 0 2024-02-28T23:07:51.230 INFO:tasks.workunit.client.1.smithi084.stdout:8/953: creat d0/d9/d1b/d38/d43/dae/db8/f130 x:0 0 0 2024-02-28T23:07:51.236 INFO:tasks.workunit.client.1.smithi084.stdout:4/917: unlink d1/d18/d20/d14/d30/f3d 0 2024-02-28T23:07:51.237 INFO:tasks.workunit.client.0.smithi057.stdout:8/870: creat d4/de9/d56/dfd/f133 x:0 0 0 2024-02-28T23:07:51.238 INFO:tasks.workunit.client.1.smithi084.stdout:8/954: truncate d0/d9/d30/db6/fd7 261426 0 2024-02-28T23:07:51.241 INFO:tasks.workunit.client.1.smithi084.stdout:4/918: mknod d1/c135 0 2024-02-28T23:07:51.242 INFO:tasks.workunit.client.0.smithi057.stdout:9/753: mknod dd/de/d4a/d9f/cf4 0 2024-02-28T23:07:51.243 INFO:tasks.workunit.client.1.smithi084.stdout:4/919: chown d1/d18/d20/d65/d8e/d55/f16 2694983 1 2024-02-28T23:07:51.248 INFO:tasks.workunit.client.0.smithi057.stdout:5/817: dread d4/d9/d14/d63/fb3 [0,4194304] 0 2024-02-28T23:07:51.248 INFO:tasks.workunit.client.0.smithi057.stdout:0/933: creat d17/d20/d26/d30/d6b/d71/f14d x:0 0 0 2024-02-28T23:07:51.248 INFO:tasks.workunit.client.0.smithi057.stdout:4/741: mkdir d1a/d2f/db2/d49/d4d/de2/d6e/d100 0 2024-02-28T23:07:51.248 INFO:tasks.workunit.client.1.smithi084.stdout:8/955: getdents d0/d44/d9f 0 2024-02-28T23:07:51.250 INFO:tasks.workunit.client.0.smithi057.stdout:1/704: symlink d2/d7/df/db/d3a/l101 0 2024-02-28T23:07:51.257 INFO:tasks.workunit.client.0.smithi057.stdout:5/818: mkdir d4/d9/d16/d8f/dfe 0 2024-02-28T23:07:51.262 INFO:tasks.workunit.client.0.smithi057.stdout:3/684: getdents d3/d4f/d4c/d91 0 2024-02-28T23:07:51.262 INFO:tasks.workunit.client.1.smithi084.stdout:4/920: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf [4194304,4194304] 0 2024-02-28T23:07:51.265 INFO:tasks.workunit.client.1.smithi084.stdout:4/921: write d1/d18/d20/d65/d8e/d55/f16 [3033153,33717] 0 2024-02-28T23:07:51.266 INFO:tasks.workunit.client.0.smithi057.stdout:8/871: dwrite d4/de9/d56/d6a/ff4 [0,4194304] 0 2024-02-28T23:07:51.266 INFO:tasks.workunit.client.0.smithi057.stdout:4/742: readlink d1a/d48/d9c/d2e/d43/lba 0 2024-02-28T23:07:51.267 INFO:tasks.workunit.client.0.smithi057.stdout:0/934: dwrite d17/d20/d4a/f119 [0,4194304] 0 2024-02-28T23:07:51.271 INFO:tasks.workunit.client.0.smithi057.stdout:1/705: creat d2/dd4/dd5/f102 x:0 0 0 2024-02-28T23:07:51.273 INFO:tasks.workunit.client.1.smithi084.stdout:4/922: truncate d1/d18/ffa 987089 0 2024-02-28T23:07:51.273 INFO:tasks.workunit.client.0.smithi057.stdout:7/791: rename d0/d5/d25/d80/fae to d0/d5/d36/d4e/d15/d29/d3d/f102 0 2024-02-28T23:07:51.278 INFO:tasks.workunit.client.1.smithi084.stdout:5/953: dwrite dc/de/df/d16/f4a [4194304,4194304] 0 2024-02-28T23:07:51.278 INFO:tasks.workunit.client.1.smithi084.stdout:6/948: dwrite d1/d13/d2e/f119 [0,4194304] 0 2024-02-28T23:07:51.281 INFO:tasks.workunit.client.1.smithi084.stdout:8/956: link d0/d9/d30/db6/fd7 d0/d24/d78/df0/f131 0 2024-02-28T23:07:51.282 INFO:tasks.workunit.client.1.smithi084.stdout:4/923: creat d1/d18/d20/d24/d2f/d3e/d52/d45/db5/f136 x:0 0 0 2024-02-28T23:07:51.286 INFO:tasks.workunit.client.0.smithi057.stdout:4/743: rename d1a/d2f/db2/d49/d4d/dc9/lee to d1a/d2f/db2/d49/d4d/dc9/dd6/l101 0 2024-02-28T23:07:51.286 INFO:tasks.workunit.client.0.smithi057.stdout:7/792: creat d0/d5/d36/d4e/f103 x:0 0 0 2024-02-28T23:07:51.290 INFO:tasks.workunit.client.0.smithi057.stdout:8/872: creat d4/d13/d7f/d128/f134 x:0 0 0 2024-02-28T23:07:51.292 INFO:tasks.workunit.client.1.smithi084.stdout:4/924: rename d1/d18/d20/d14/l7f to d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/l137 0 2024-02-28T23:07:51.295 INFO:tasks.workunit.client.1.smithi084.stdout:4/925: mknod d1/d18/d20/d14/d30/d47/da4/c138 0 2024-02-28T23:07:51.302 INFO:tasks.workunit.client.1.smithi084.stdout:4/926: rename d1/d18/d20/d24/f107 to d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/f139 0 2024-02-28T23:07:51.316 INFO:tasks.workunit.client.0.smithi057.stdout:0/935: symlink d17/d20/d23/l14e 0 2024-02-28T23:07:51.319 INFO:tasks.workunit.client.0.smithi057.stdout:4/744: creat d1a/d2f/f102 x:0 0 0 2024-02-28T23:07:51.322 INFO:tasks.workunit.client.0.smithi057.stdout:1/706: symlink d2/d7/df/db/d12/d19/d48/dff/d100/l103 0 2024-02-28T23:07:51.324 INFO:tasks.workunit.client.0.smithi057.stdout:7/793: chown d0/d5/d36/f78 0 1 2024-02-28T23:07:51.327 INFO:tasks.workunit.client.0.smithi057.stdout:8/873: mknod d4/d13/d1a/d4a/c135 0 2024-02-28T23:07:51.328 INFO:tasks.workunit.client.0.smithi057.stdout:0/936: symlink d17/d20/d23/d47/d60/d6f/l14f 0 2024-02-28T23:07:51.328 INFO:tasks.workunit.client.1.smithi084.stdout:8/957: dread d0/d9/d30/d33/f5e [0,4194304] 0 2024-02-28T23:07:51.330 INFO:tasks.workunit.client.0.smithi057.stdout:0/937: readlink d17/d20/d26/d30/lf0 0 2024-02-28T23:07:51.330 INFO:tasks.workunit.client.0.smithi057.stdout:1/707: symlink d2/d7/df/db/dfc/l104 0 2024-02-28T23:07:51.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/794: mknod d0/d5/d25/d80/c104 0 2024-02-28T23:07:51.334 INFO:tasks.workunit.client.1.smithi084.stdout:8/958: mkdir d0/d132 0 2024-02-28T23:07:51.334 INFO:tasks.workunit.client.0.smithi057.stdout:8/874: write d4/d13/d1a/f58 [1626484,23636] 0 2024-02-28T23:07:51.336 INFO:tasks.workunit.client.0.smithi057.stdout:7/795: chown d0/d5/d36/d4e/d15/da0 257159097 1 2024-02-28T23:07:51.339 INFO:tasks.workunit.client.1.smithi084.stdout:8/959: creat d0/d9/d30/d33/d71/d84/f133 x:0 0 0 2024-02-28T23:07:51.340 INFO:tasks.workunit.client.0.smithi057.stdout:0/938: mknod d17/d20/d26/d48/d55/c150 0 2024-02-28T23:07:51.343 INFO:tasks.workunit.client.0.smithi057.stdout:4/745: link d1a/d48/d9c/f89 d1a/d48/db0/db1/f103 0 2024-02-28T23:07:51.344 INFO:tasks.workunit.client.1.smithi084.stdout:8/960: chown d0/d24/d83 14391 1 2024-02-28T23:07:51.345 INFO:tasks.workunit.client.0.smithi057.stdout:3/685: dread d3/d20/d2d/f3b [0,4194304] 0 2024-02-28T23:07:51.346 INFO:tasks.workunit.client.0.smithi057.stdout:7/796: write d0/d5/d36/d4e/d15/d29/f32 [824619,34996] 0 2024-02-28T23:07:51.349 INFO:tasks.workunit.client.1.smithi084.stdout:5/954: dread dc/de/df/d31/f7e [0,4194304] 0 2024-02-28T23:07:51.349 INFO:tasks.workunit.client.0.smithi057.stdout:4/746: creat d1a/d48/d9c/d68/dc5/f104 x:0 0 0 2024-02-28T23:07:51.352 INFO:tasks.workunit.client.0.smithi057.stdout:0/939: link d17/d20/d4a/f119 d17/d20/db6/f151 0 2024-02-28T23:07:51.353 INFO:tasks.workunit.client.1.smithi084.stdout:6/949: dwrite d1/d9/d1d/d1e/fcd [0,4194304] 0 2024-02-28T23:07:51.353 INFO:tasks.workunit.client.0.smithi057.stdout:7/797: creat d0/d5/d4a/f105 x:0 0 0 2024-02-28T23:07:51.354 INFO:tasks.workunit.client.1.smithi084.stdout:4/927: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/f93 [0,4194304] 0 2024-02-28T23:07:51.355 INFO:tasks.workunit.client.1.smithi084.stdout:5/955: creat dc/d51/d7c/f164 x:0 0 0 2024-02-28T23:07:51.356 INFO:tasks.workunit.client.0.smithi057.stdout:4/747: creat d1a/d2f/db2/dcb/f105 x:0 0 0 2024-02-28T23:07:51.357 INFO:tasks.workunit.client.1.smithi084.stdout:8/961: rename d0/d9/d30/d33/d71/d84/dc0/ldd to d0/d44/dfb/d12a/l134 0 2024-02-28T23:07:51.360 INFO:tasks.workunit.client.0.smithi057.stdout:2/882: truncate d8/d11/d39/d83/fa3 865417 0 2024-02-28T23:07:51.363 INFO:tasks.workunit.client.0.smithi057.stdout:8/875: dwrite d4/d13/d2c/db1/fb7 [0,4194304] 0 2024-02-28T23:07:51.365 INFO:tasks.workunit.client.1.smithi084.stdout:6/950: creat d1/d13/d47/d91/dba/de9/f137 x:0 0 0 2024-02-28T23:07:51.365 INFO:tasks.workunit.client.0.smithi057.stdout:2/883: fdatasync d8/d11/fa8 0 2024-02-28T23:07:51.375 INFO:tasks.workunit.client.0.smithi057.stdout:8/876: truncate d4/d13/d1a/d4a/db3/dfc/fff 732592 0 2024-02-28T23:07:51.376 INFO:tasks.workunit.client.0.smithi057.stdout:8/877: dread - d4/d13/d2c/db1/f11d zero size 2024-02-28T23:07:51.377 INFO:tasks.workunit.client.1.smithi084.stdout:2/864: sync 2024-02-28T23:07:51.377 INFO:tasks.workunit.client.1.smithi084.stdout:1/973: sync 2024-02-28T23:07:51.377 INFO:tasks.workunit.client.1.smithi084.stdout:5/956: dwrite dc/de/df/d16/d34/d3c/f5a [0,4194304] 0 2024-02-28T23:07:51.379 INFO:tasks.workunit.client.0.smithi057.stdout:2/884: symlink d8/d11/d15/daf/d105/l116 0 2024-02-28T23:07:51.381 INFO:tasks.workunit.client.1.smithi084.stdout:5/957: chown dc/de/df/d16/d34/daa/f131 4 1 2024-02-28T23:07:51.382 INFO:tasks.workunit.client.0.smithi057.stdout:7/798: dread d0/d3f/d76/dc1/dea/f5b [0,4194304] 0 2024-02-28T23:07:51.383 INFO:tasks.workunit.client.1.smithi084.stdout:2/865: mknod d5/d7/d2e/d4a/d5c/c104 0 2024-02-28T23:07:51.384 INFO:tasks.workunit.client.1.smithi084.stdout:1/974: symlink d12/d1b/d91/d9b/da5/dcb/l13f 0 2024-02-28T23:07:51.386 INFO:tasks.workunit.client.1.smithi084.stdout:5/958: fsync dc/de/df/d31/dd1/d126/f156 0 2024-02-28T23:07:51.389 INFO:tasks.workunit.client.0.smithi057.stdout:8/878: creat d4/f136 x:0 0 0 2024-02-28T23:07:51.389 INFO:tasks.workunit.client.1.smithi084.stdout:4/928: dread d1/d18/d20/d65/d8e/d55/dc3/f10e [0,4194304] 0 2024-02-28T23:07:51.390 INFO:tasks.workunit.client.1.smithi084.stdout:1/975: dread - d12/d1b/d2e/d52/d5b/f68 zero size 2024-02-28T23:07:51.391 INFO:tasks.workunit.client.0.smithi057.stdout:7/799: mknod d0/c106 0 2024-02-28T23:07:51.391 INFO:tasks.workunit.client.1.smithi084.stdout:1/976: stat d12/d1b/d2e/d30/f114 0 2024-02-28T23:07:51.393 INFO:tasks.workunit.client.0.smithi057.stdout:7/800: readlink d0/d5/d36/d4e/lbb 0 2024-02-28T23:07:51.394 INFO:tasks.workunit.client.1.smithi084.stdout:8/962: dread d0/d9/f5f [0,4194304] 0 2024-02-28T23:07:51.394 INFO:tasks.workunit.client.1.smithi084.stdout:4/929: readlink d1/d18/d20/d65/d8e/d55/l21 0 2024-02-28T23:07:51.402 INFO:tasks.workunit.client.0.smithi057.stdout:2/885: link d8/d11/d39/d83/f115 d8/d11/d21/d28/d49/d85/d92/df5/f117 0 2024-02-28T23:07:51.403 INFO:tasks.workunit.client.1.smithi084.stdout:5/959: dread dc/de/df/d31/dd1/d5f/d9b/fb9 [0,4194304] 0 2024-02-28T23:07:51.405 INFO:tasks.workunit.client.0.smithi057.stdout:2/886: dread - d8/d11/d56/f9d zero size 2024-02-28T23:07:51.407 INFO:tasks.workunit.client.1.smithi084.stdout:6/951: dwrite d1/d13/d50/d74/d81/f94 [0,4194304] 0 2024-02-28T23:07:51.407 INFO:tasks.workunit.client.0.smithi057.stdout:7/801: rmdir d0/d3f/d76/dc1/dea/d75 39 2024-02-28T23:07:51.410 INFO:tasks.workunit.client.1.smithi084.stdout:5/960: mknod dc/de/df/d31/dd1/d5f/d9b/d10e/c165 0 2024-02-28T23:07:51.410 INFO:tasks.workunit.client.0.smithi057.stdout:7/802: read d0/d5/d36/d4e/d15/d29/f90 [175980,100834] 0 2024-02-28T23:07:51.411 INFO:tasks.workunit.client.1.smithi084.stdout:8/963: write d0/fe6 [699698,18930] 0 2024-02-28T23:07:51.413 INFO:tasks.workunit.client.0.smithi057.stdout:5/819: dwrite d4/d9/d85/fa3 [0,4194304] 0 2024-02-28T23:07:51.415 INFO:tasks.workunit.client.1.smithi084.stdout:8/964: creat d0/d24/d83/f135 x:0 0 0 2024-02-28T23:07:51.417 INFO:tasks.workunit.client.1.smithi084.stdout:2/866: dread d5/d7/d9/fb8 [0,4194304] 0 2024-02-28T23:07:51.418 INFO:tasks.workunit.client.0.smithi057.stdout:1/708: write d2/d7/d4f/d5c/d6b/fb9 [231236,37999] 0 2024-02-28T23:07:51.420 INFO:tasks.workunit.client.0.smithi057.stdout:8/879: dread d4/de9/d56/d6a/d54/d61/f97 [0,4194304] 0 2024-02-28T23:07:51.420 INFO:tasks.workunit.client.0.smithi057.stdout:6/731: sync 2024-02-28T23:07:51.421 INFO:tasks.workunit.client.1.smithi084.stdout:4/930: dwrite d1/d12e/fde [0,4194304] 0 2024-02-28T23:07:51.422 INFO:tasks.workunit.client.1.smithi084.stdout:2/867: creat d5/d7/d9/f105 x:0 0 0 2024-02-28T23:07:51.427 INFO:tasks.workunit.client.0.smithi057.stdout:1/709: chown d2/d7/df/db/d3a/d2a/d82 20210 1 2024-02-28T23:07:51.429 INFO:tasks.workunit.client.1.smithi084.stdout:2/868: rename d5/ca9 to d5/d7/d12/d24/d3a/d48/c106 0 2024-02-28T23:07:51.430 INFO:tasks.workunit.client.1.smithi084.stdout:4/931: chown d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf 38857 1 2024-02-28T23:07:51.431 INFO:tasks.workunit.client.0.smithi057.stdout:2/887: creat d8/d11/d21/da4/f118 x:0 0 0 2024-02-28T23:07:51.432 INFO:tasks.workunit.client.1.smithi084.stdout:6/952: dwrite d1/d13/d47/d91/dba/de9/f125 [0,4194304] 0 2024-02-28T23:07:51.434 INFO:tasks.workunit.client.1.smithi084.stdout:2/869: mknod d5/d7/d9/d7c/d5b/da6/c107 0 2024-02-28T23:07:51.436 INFO:tasks.workunit.client.0.smithi057.stdout:5/820: unlink d4/d9/d16/d52/cc6 0 2024-02-28T23:07:51.437 INFO:tasks.workunit.client.0.smithi057.stdout:3/686: dwrite d3/d8/d27/f77 [0,4194304] 0 2024-02-28T23:07:51.438 INFO:tasks.workunit.client.0.smithi057.stdout:1/710: chown d2/d7/df/db/d12/d19/d80/l84 104 1 2024-02-28T23:07:51.441 INFO:tasks.workunit.client.1.smithi084.stdout:2/870: creat d5/d7/d9/d94/dbd/f108 x:0 0 0 2024-02-28T23:07:51.443 INFO:tasks.workunit.client.0.smithi057.stdout:7/803: mkdir d0/d3f/d76/d107 0 2024-02-28T23:07:51.448 INFO:tasks.workunit.client.0.smithi057.stdout:0/940: dwrite d17/d20/d26/d48/d7c/d98/da6/ff8 [0,4194304] 0 2024-02-28T23:07:51.450 INFO:tasks.workunit.client.1.smithi084.stdout:6/953: link d1/d13/d50/fa6 d1/d13/d50/d74/f138 0 2024-02-28T23:07:51.452 INFO:tasks.workunit.client.1.smithi084.stdout:5/961: truncate dc/de/df/d16/ffa 2031 0 2024-02-28T23:07:51.454 INFO:tasks.workunit.client.1.smithi084.stdout:8/965: dread d0/d9/d1b/fc4 [0,4194304] 0 2024-02-28T23:07:51.455 INFO:tasks.workunit.client.1.smithi084.stdout:4/932: dwrite d1/d12e/fde [0,4194304] 0 2024-02-28T23:07:51.456 INFO:tasks.workunit.client.1.smithi084.stdout:6/954: symlink d1/d13/d2e/d41/d4d/d6c/l139 0 2024-02-28T23:07:51.457 INFO:tasks.workunit.client.0.smithi057.stdout:4/748: dwrite d1a/d48/d9c/d2e/f98 [0,4194304] 0 2024-02-28T23:07:51.460 INFO:tasks.workunit.client.0.smithi057.stdout:6/732: truncate d0/d4/d11/f13 577819 0 2024-02-28T23:07:51.463 INFO:tasks.workunit.client.1.smithi084.stdout:5/962: symlink dc/de/df/db0/d104/d139/l166 0 2024-02-28T23:07:51.467 INFO:tasks.workunit.client.1.smithi084.stdout:4/933: read - d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/f12f zero size 2024-02-28T23:07:51.468 INFO:tasks.workunit.client.1.smithi084.stdout:8/966: symlink d0/d9/d1b/d38/d43/d51/d54/l136 0 2024-02-28T23:07:51.468 INFO:tasks.workunit.client.1.smithi084.stdout:5/963: rmdir dc/ddb 39 2024-02-28T23:07:51.472 INFO:tasks.workunit.client.1.smithi084.stdout:4/934: symlink d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/l13a 0 2024-02-28T23:07:51.474 INFO:tasks.workunit.client.0.smithi057.stdout:3/687: symlink d3/d4f/d4c/ld9 0 2024-02-28T23:07:51.476 INFO:tasks.workunit.client.0.smithi057.stdout:3/688: stat d3/d4/d7/la2 0 2024-02-28T23:07:51.476 INFO:tasks.workunit.client.1.smithi084.stdout:5/964: mknod dc/de/df/d16/d34/d3c/d128/c167 0 2024-02-28T23:07:51.476 INFO:tasks.workunit.client.1.smithi084.stdout:4/935: chown d1/d18/d20/d24/ldc 149 1 2024-02-28T23:07:51.481 INFO:tasks.workunit.client.0.smithi057.stdout:7/804: mknod d0/d5/d36/d4e/d15/d29/c108 0 2024-02-28T23:07:51.482 INFO:tasks.workunit.client.1.smithi084.stdout:6/955: dwrite d1/d13/d47/f129 [0,4194304] 0 2024-02-28T23:07:51.482 INFO:tasks.workunit.client.0.smithi057.stdout:5/821: dread d4/fb9 [0,4194304] 0 2024-02-28T23:07:51.483 INFO:tasks.workunit.client.1.smithi084.stdout:4/936: stat d1/d18/d20/d65/d8e/d55/dc3/dec/d11b/c123 0 2024-02-28T23:07:51.484 INFO:tasks.workunit.client.0.smithi057.stdout:9/754: sync 2024-02-28T23:07:51.485 INFO:tasks.workunit.client.1.smithi084.stdout:4/937: fdatasync d1/d18/d20/d24/d2f/d3e/d52/d57/fa0 0 2024-02-28T23:07:51.486 INFO:tasks.workunit.client.0.smithi057.stdout:6/733: mknod d0/d17/d20/cf6 0 2024-02-28T23:07:51.488 INFO:tasks.workunit.client.0.smithi057.stdout:4/749: rename d1a/d48/d97/d7f/fdf to d1a/d2f/db2/d49/d4d/dc9/f106 0 2024-02-28T23:07:51.492 INFO:tasks.workunit.client.0.smithi057.stdout:1/711: mknod d2/d71/d2d/d39/da3/db5/df7/c105 0 2024-02-28T23:07:51.496 INFO:tasks.workunit.client.0.smithi057.stdout:7/805: fsync d0/d5/d36/d4e/d15/d29/d3d/d51/f72 0 2024-02-28T23:07:51.496 INFO:tasks.workunit.client.0.smithi057.stdout:0/941: creat d17/d19/d148/d146/f152 x:0 0 0 2024-02-28T23:07:51.498 INFO:tasks.workunit.client.0.smithi057.stdout:8/880: rmdir d4/de9/d56/da6/d131 0 2024-02-28T23:07:51.501 INFO:tasks.workunit.client.1.smithi084.stdout:4/938: dwrite d1/d18/d20/d24/ddb/d12b/f133 [0,4194304] 0 2024-02-28T23:07:51.507 INFO:tasks.workunit.client.0.smithi057.stdout:4/750: truncate f18 4062486 0 2024-02-28T23:07:51.514 INFO:tasks.workunit.client.0.smithi057.stdout:4/751: stat d1a/d2f/c62 0 2024-02-28T23:07:51.516 INFO:tasks.workunit.client.1.smithi084.stdout:4/939: dread d1/d12e/fde [0,4194304] 0 2024-02-28T23:07:51.519 INFO:tasks.workunit.client.0.smithi057.stdout:1/712: rename d2/d7/df/db/d12/d19/c51 to d2/d7/df/db/d12/d3f/c106 0 2024-02-28T23:07:51.520 INFO:tasks.workunit.client.0.smithi057.stdout:0/942: truncate d17/d20/d23/d47/d60/f13d 792620 0 2024-02-28T23:07:51.520 INFO:tasks.workunit.client.1.smithi084.stdout:8/967: write d0/d9/d30/d33/d71/d84/dc0/fc6 [1918725,45870] 0 2024-02-28T23:07:51.520 INFO:tasks.workunit.client.1.smithi084.stdout:5/965: write dc/de/df/f71 [790222,14330] 0 2024-02-28T23:07:51.521 INFO:tasks.workunit.client.1.smithi084.stdout:5/966: fdatasync dc/de/df/d31/f7e 0 2024-02-28T23:07:51.523 INFO:tasks.workunit.client.0.smithi057.stdout:9/755: dwrite dd/de/d24/d63/d77/dcf/d40/d44/fa4 [0,4194304] 0 2024-02-28T23:07:51.524 INFO:tasks.workunit.client.0.smithi057.stdout:8/881: write d4/d13/d18/d62/d66/d74/fc7 [1086810,38955] 0 2024-02-28T23:07:51.524 INFO:tasks.workunit.client.1.smithi084.stdout:6/956: read d1/fe6 [117182,3021] 0 2024-02-28T23:07:51.525 INFO:tasks.workunit.client.1.smithi084.stdout:8/968: truncate d0/d11/fe4 103370 0 2024-02-28T23:07:51.527 INFO:tasks.workunit.client.1.smithi084.stdout:2/871: dread d5/d7/d12/d24/d2a/d35/f5d [0,4194304] 0 2024-02-28T23:07:51.528 INFO:tasks.workunit.client.0.smithi057.stdout:8/882: fdatasync d4/d13/d1a/f58 0 2024-02-28T23:07:51.529 INFO:tasks.workunit.client.1.smithi084.stdout:4/940: getdents d1/d18/d20/d14/d30/d47/da4/d10f/d115 0 2024-02-28T23:07:51.532 INFO:tasks.workunit.client.0.smithi057.stdout:0/943: rmdir d17/d20/d4a/d78/dd4 39 2024-02-28T23:07:51.532 INFO:tasks.workunit.client.1.smithi084.stdout:4/941: chown d1/d18/d20/d24/d2f/d3e/f60 463759411 1 2024-02-28T23:07:51.532 INFO:tasks.workunit.client.1.smithi084.stdout:6/957: mkdir d1/d9/d1d/d1e/d33/d13a 0 2024-02-28T23:07:51.534 INFO:tasks.workunit.client.0.smithi057.stdout:3/689: dread d3/d8/d27/d70/f74 [0,4194304] 0 2024-02-28T23:07:51.535 INFO:tasks.workunit.client.1.smithi084.stdout:6/958: write d1/d88/dbf/f110 [287021,34237] 0 2024-02-28T23:07:51.540 INFO:tasks.workunit.client.0.smithi057.stdout:7/806: rename d0/d5/d36/d4e/fd2 to d0/d5/d4a/f109 0 2024-02-28T23:07:51.546 INFO:tasks.workunit.client.0.smithi057.stdout:8/883: mknod d4/d13/d18/da7/c137 0 2024-02-28T23:07:51.548 INFO:tasks.workunit.client.0.smithi057.stdout:3/690: symlink d3/d4f/d4c/d91/lda 0 2024-02-28T23:07:51.551 INFO:tasks.workunit.client.0.smithi057.stdout:7/807: unlink d0/d5/d25/d3e/fa9 0 2024-02-28T23:07:51.556 INFO:tasks.workunit.client.0.smithi057.stdout:2/888: dwrite d8/f64 [0,4194304] 0 2024-02-28T23:07:51.558 INFO:tasks.workunit.client.0.smithi057.stdout:9/756: dread dd/de/d24/d63/d77/dcf/f69 [0,4194304] 0 2024-02-28T23:07:51.563 INFO:tasks.workunit.client.0.smithi057.stdout:7/808: creat d0/d5/d36/d4e/f10a x:0 0 0 2024-02-28T23:07:51.565 INFO:tasks.workunit.client.0.smithi057.stdout:6/734: write d0/d1b/d3e/d40/fa6 [62148,50919] 0 2024-02-28T23:07:51.565 INFO:tasks.workunit.client.0.smithi057.stdout:1/713: link d2/d7/df/db/d12/l91 d2/d7/df/db/d3a/l107 0 2024-02-28T23:07:51.566 INFO:tasks.workunit.client.0.smithi057.stdout:7/809: chown d0/d5/d36/d4e/d15/d29/d3d/c97 7919 1 2024-02-28T23:07:51.570 INFO:tasks.workunit.client.0.smithi057.stdout:0/944: rmdir d17/d19/d148/d147 0 2024-02-28T23:07:51.571 INFO:tasks.workunit.client.0.smithi057.stdout:1/714: fsync d2/d7/df/db/d12/f26 0 2024-02-28T23:07:51.572 INFO:tasks.workunit.client.0.smithi057.stdout:6/735: fsync d0/d17/f2c 0 2024-02-28T23:07:51.572 INFO:tasks.workunit.client.0.smithi057.stdout:3/691: truncate d3/d8/d27/f81 497188 0 2024-02-28T23:07:51.573 INFO:tasks.workunit.client.0.smithi057.stdout:4/752: write d1a/d48/db0/db1/fb6 [850421,100425] 0 2024-02-28T23:07:51.576 INFO:tasks.workunit.client.0.smithi057.stdout:6/736: fsync d0/d17/d20/fe9 0 2024-02-28T23:07:51.577 INFO:tasks.workunit.client.0.smithi057.stdout:2/889: rename d8/d11/d39/d83/dde/d50/f93 to d8/d11/db3/f119 0 2024-02-28T23:07:51.579 INFO:tasks.workunit.client.0.smithi057.stdout:2/890: write d8/dae/fb8 [2691888,54902] 0 2024-02-28T23:07:51.581 INFO:tasks.workunit.client.0.smithi057.stdout:9/757: truncate dd/de/d24/d63/d77/dcf/d40/d44/da5/fea 1169247 0 2024-02-28T23:07:51.583 INFO:tasks.workunit.client.0.smithi057.stdout:3/692: mknod d3/d4f/cdb 0 2024-02-28T23:07:51.584 INFO:tasks.workunit.client.0.smithi057.stdout:0/945: mkdir d17/d20/d26/d48/d7c/d98/da6/d153 0 2024-02-28T23:07:51.590 INFO:tasks.workunit.client.0.smithi057.stdout:1/715: rename d2/d7/df/db/dfc/l104 to d2/d7/df/db/d3a/d8f/l108 0 2024-02-28T23:07:51.591 INFO:tasks.workunit.client.1.smithi084.stdout:6/959: mkdir d1/d13/d50/dbd/dc4/d13b 0 2024-02-28T23:07:51.596 INFO:tasks.workunit.client.0.smithi057.stdout:2/891: creat d8/d11/d21/d28/d41/dbb/f11a x:0 0 0 2024-02-28T23:07:51.596 INFO:tasks.workunit.client.1.smithi084.stdout:6/960: read d1/d9/d1d/d1e/d33/f4c [224964,128184] 0 2024-02-28T23:07:51.598 INFO:tasks.workunit.client.1.smithi084.stdout:2/872: dwrite d5/d7/fc8 [0,4194304] 0 2024-02-28T23:07:51.599 INFO:tasks.workunit.client.0.smithi057.stdout:3/693: creat d3/d4/d1d/d47/d62/db4/fdc x:0 0 0 2024-02-28T23:07:51.603 INFO:tasks.workunit.client.1.smithi084.stdout:2/873: creat d5/d1f/f109 x:0 0 0 2024-02-28T23:07:51.604 INFO:tasks.workunit.client.0.smithi057.stdout:0/946: write d17/d19/fe1 [3220035,67035] 0 2024-02-28T23:07:51.605 INFO:tasks.workunit.client.1.smithi084.stdout:2/874: stat d5/d7/d75/db4/fec 0 2024-02-28T23:07:51.607 INFO:tasks.workunit.client.1.smithi084.stdout:2/875: chown d5/d7/d9/cc 61707920 1 2024-02-28T23:07:51.608 INFO:tasks.workunit.client.0.smithi057.stdout:6/737: dread d0/d1b/d54/f90 [0,4194304] 0 2024-02-28T23:07:51.609 INFO:tasks.workunit.client.0.smithi057.stdout:7/810: rename d0/d5 to d0/d3f/d76/dc1/dea/d75/dbf/d10b 0 2024-02-28T23:07:51.609 INFO:tasks.workunit.client.1.smithi084.stdout:6/961: link d1/d13/d2e/d41/dbb/lfe d1/d88/dbf/de2/l13c 0 2024-02-28T23:07:51.613 INFO:tasks.workunit.client.1.smithi084.stdout:6/962: creat d1/d9/ddc/d12b/f13d x:0 0 0 2024-02-28T23:07:51.616 INFO:tasks.workunit.client.0.smithi057.stdout:8/884: dwrite d4/de9/d7c/ff6 [0,4194304] 0 2024-02-28T23:07:51.616 INFO:tasks.workunit.client.1.smithi084.stdout:2/876: link d5/d7/d9/d7c/d5b/c62 d5/dd/d69/c10a 0 2024-02-28T23:07:51.617 INFO:tasks.workunit.client.0.smithi057.stdout:1/716: dwrite d2/f44 [0,4194304] 0 2024-02-28T23:07:51.620 INFO:tasks.workunit.client.1.smithi084.stdout:2/877: mkdir d5/d1f/d42/d8f/dbe/d10b 0 2024-02-28T23:07:51.628 INFO:tasks.workunit.client.0.smithi057.stdout:4/753: getdents d1a/d2f/db2/dcb 0 2024-02-28T23:07:51.629 INFO:tasks.workunit.client.0.smithi057.stdout:2/892: rename d8/d11/db3/de9/ff9 to d8/dae/de1/f11b 0 2024-02-28T23:07:51.629 INFO:tasks.workunit.client.0.smithi057.stdout:9/758: dwrite dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:51.635 INFO:tasks.workunit.client.0.smithi057.stdout:8/885: creat d4/d13/d1a/d4a/db3/dfc/f138 x:0 0 0 2024-02-28T23:07:51.638 INFO:tasks.workunit.client.0.smithi057.stdout:2/893: dread - d8/d11/d56/f9d zero size 2024-02-28T23:07:51.642 INFO:tasks.workunit.client.1.smithi084.stdout:4/942: dread d1/d18/d20/d24/d2f/d3e/d52/d48/dd7/f111 [0,4194304] 0 2024-02-28T23:07:51.648 INFO:tasks.workunit.client.0.smithi057.stdout:3/694: dwrite d3/d4/f30 [4194304,4194304] 0 2024-02-28T23:07:51.648 INFO:tasks.workunit.client.0.smithi057.stdout:0/947: rename d17/d19/d148 to d17/d20/d26/d48/d55/d88/d154 0 2024-02-28T23:07:51.650 INFO:tasks.workunit.client.1.smithi084.stdout:6/963: dread d1/d13/d47/f56 [0,4194304] 0 2024-02-28T23:07:51.650 INFO:tasks.workunit.client.0.smithi057.stdout:4/754: mkdir d1a/d2f/db2/d49/d4d/d107 0 2024-02-28T23:07:51.653 INFO:tasks.workunit.client.1.smithi084.stdout:6/964: write d1/d13/d50/f99 [5124484,83301] 0 2024-02-28T23:07:51.656 INFO:tasks.workunit.client.1.smithi084.stdout:6/965: read d1/d88/dbf/f120 [1947825,7433] 0 2024-02-28T23:07:51.656 INFO:tasks.workunit.client.0.smithi057.stdout:0/948: symlink d17/d20/d26/l155 0 2024-02-28T23:07:51.659 INFO:tasks.workunit.client.0.smithi057.stdout:9/759: rmdir dd/de/d24/d63/d77/dcf/d40/d44/d5c/daa 0 2024-02-28T23:07:51.661 INFO:tasks.workunit.client.0.smithi057.stdout:6/738: dread d0/d4/d11/f13 [0,4194304] 0 2024-02-28T23:07:51.662 INFO:tasks.workunit.client.1.smithi084.stdout:6/966: chown d1/d13/la1 565 1 2024-02-28T23:07:51.662 INFO:tasks.workunit.client.1.smithi084.stdout:4/943: dwrite d1/d18/d20/f11 [0,4194304] 0 2024-02-28T23:07:51.666 INFO:tasks.workunit.client.0.smithi057.stdout:8/886: getdents d4/d13/d18/da7/de1 0 2024-02-28T23:07:51.669 INFO:tasks.workunit.client.0.smithi057.stdout:3/695: creat d3/d4/d1d/fdd x:0 0 0 2024-02-28T23:07:51.670 INFO:tasks.workunit.client.1.smithi084.stdout:4/944: dread - d1/d18/d20/d14/d30/d47/da4/d10f/d115/f119 zero size 2024-02-28T23:07:51.672 INFO:tasks.workunit.client.1.smithi084.stdout:2/878: write d5/d7/f3f [1107734,124959] 0 2024-02-28T23:07:51.673 INFO:tasks.workunit.client.0.smithi057.stdout:3/696: stat d3/d4f/f7d 0 2024-02-28T23:07:51.674 INFO:tasks.workunit.client.1.smithi084.stdout:6/967: getdents d1/d9/ddc/d12b 0 2024-02-28T23:07:51.674 INFO:tasks.workunit.client.0.smithi057.stdout:4/755: rename d1a/d48/d97/d7f/cbc to d1a/d48/d9c/d68/c108 0 2024-02-28T23:07:51.676 INFO:tasks.workunit.client.1.smithi084.stdout:2/879: unlink d5/f15 0 2024-02-28T23:07:51.677 INFO:tasks.workunit.client.1.smithi084.stdout:6/968: write d1/d9/d1d/d1e/fcd [3139607,123152] 0 2024-02-28T23:07:51.678 INFO:tasks.workunit.client.0.smithi057.stdout:9/760: rmdir dd/de/d24/d63/d77/dcf 39 2024-02-28T23:07:51.679 INFO:tasks.workunit.client.1.smithi084.stdout:2/880: mknod d5/d1f/d42/d74/c10c 0 2024-02-28T23:07:51.682 INFO:tasks.workunit.client.0.smithi057.stdout:1/717: write d2/d7/d4f/d5c/d6b/f77 [1011885,21919] 0 2024-02-28T23:07:51.683 INFO:tasks.workunit.client.1.smithi084.stdout:4/945: dread d1/d12e/fde [0,4194304] 0 2024-02-28T23:07:51.684 INFO:tasks.workunit.client.0.smithi057.stdout:8/887: creat d4/d13/d18/d62/dbb/f139 x:0 0 0 2024-02-28T23:07:51.684 INFO:tasks.workunit.client.1.smithi084.stdout:6/969: link d1/d88/l8a d1/d13/d47/l13e 0 2024-02-28T23:07:51.684 INFO:tasks.workunit.client.1.smithi084.stdout:2/881: creat d5/d1f/d42/d74/f10d x:0 0 0 2024-02-28T23:07:51.685 INFO:tasks.workunit.client.0.smithi057.stdout:7/811: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/f55 [0,4194304] 0 2024-02-28T23:07:51.687 INFO:tasks.workunit.client.1.smithi084.stdout:4/946: mkdir d1/d12e/d13b 0 2024-02-28T23:07:51.687 INFO:tasks.workunit.client.0.smithi057.stdout:9/761: dread - dd/de/d14/d48/feb zero size 2024-02-28T23:07:51.687 INFO:tasks.workunit.client.0.smithi057.stdout:4/756: stat d1a/d2f/db2/d49/d4d/de2/f64 0 2024-02-28T23:07:51.689 INFO:tasks.workunit.client.1.smithi084.stdout:6/970: mknod d1/d13/d2e/d41/d4d/d6c/df6/c13f 0 2024-02-28T23:07:51.689 INFO:tasks.workunit.client.0.smithi057.stdout:2/894: dwrite d8/d11/d39/d83/dde/d50/f66 [0,4194304] 0 2024-02-28T23:07:51.690 INFO:tasks.workunit.client.1.smithi084.stdout:4/947: fsync d1/f6e 0 2024-02-28T23:07:51.692 INFO:tasks.workunit.client.1.smithi084.stdout:2/882: rename d5/d7/d12/d24/feb to d5/d7/d9/d7c/d5b/da6/f10e 0 2024-02-28T23:07:51.695 INFO:tasks.workunit.client.1.smithi084.stdout:2/883: creat d5/d1f/d42/d8f/f10f x:0 0 0 2024-02-28T23:07:51.696 INFO:tasks.workunit.client.0.smithi057.stdout:8/888: dread - d4/d13/d18/d62/d66/d74/f12d zero size 2024-02-28T23:07:51.697 INFO:tasks.workunit.client.0.smithi057.stdout:6/739: dwrite d0/d4/d25/fdd [0,4194304] 0 2024-02-28T23:07:51.698 INFO:tasks.workunit.client.0.smithi057.stdout:2/895: read - d8/d11/d39/d83/dde/f10d zero size 2024-02-28T23:07:51.699 INFO:tasks.workunit.client.0.smithi057.stdout:7/812: write d0/d3f/d76/dc1/dea/f100 [908302,124771] 0 2024-02-28T23:07:51.700 INFO:tasks.workunit.client.0.smithi057.stdout:8/889: readlink d4/de9/d56/da6/dae/l10f 0 2024-02-28T23:07:51.701 INFO:tasks.workunit.client.0.smithi057.stdout:3/697: link d3/d8/f3c d3/d4/d1d/fde 0 2024-02-28T23:07:51.704 INFO:tasks.workunit.client.0.smithi057.stdout:5/822: sync 2024-02-28T23:07:51.705 INFO:tasks.workunit.client.1.smithi084.stdout:2/884: link d5/d7/d9/d7c/d5b/da6/c107 d5/d1f/d42/d8f/dbe/d10b/c110 0 2024-02-28T23:07:51.706 INFO:tasks.workunit.client.0.smithi057.stdout:5/823: stat d4/d7/d8/ff 0 2024-02-28T23:07:51.706 INFO:tasks.workunit.client.1.smithi084.stdout:4/948: write d1/d18/d20/d65/d8e/d55/dc3/fdd [642520,29538] 0 2024-02-28T23:07:51.711 INFO:tasks.workunit.client.1.smithi084.stdout:2/885: rename d5/dd/f86 to d5/d7/f111 0 2024-02-28T23:07:51.711 INFO:tasks.workunit.client.1.smithi084.stdout:6/971: dwrite d1/fdb [0,4194304] 0 2024-02-28T23:07:51.713 INFO:tasks.workunit.client.0.smithi057.stdout:2/896: creat d8/d11/d39/dbc/f11c x:0 0 0 2024-02-28T23:07:51.713 INFO:tasks.workunit.client.0.smithi057.stdout:3/698: symlink d3/d20/ldf 0 2024-02-28T23:07:51.716 INFO:tasks.workunit.client.0.smithi057.stdout:5/824: read - d4/fd6 zero size 2024-02-28T23:07:51.722 INFO:tasks.workunit.client.0.smithi057.stdout:2/897: mknod d8/d11/d15/d97/c11d 0 2024-02-28T23:07:51.723 INFO:tasks.workunit.client.0.smithi057.stdout:7/813: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d3e/f54 [0,4194304] 0 2024-02-28T23:07:51.723 INFO:tasks.workunit.client.0.smithi057.stdout:4/757: link d1a/d48/db0/l8e d1a/d48/d9c/d2e/d8f/db7/l109 0 2024-02-28T23:07:51.723 INFO:tasks.workunit.client.1.smithi084.stdout:4/949: link d1/d18/d20/d24/da8/cff d1/d12e/d13b/c13c 0 2024-02-28T23:07:51.724 INFO:tasks.workunit.client.0.smithi057.stdout:8/890: link d4/d13/d2c/c35 d4/de9/d59/c13a 0 2024-02-28T23:07:51.725 INFO:tasks.workunit.client.1.smithi084.stdout:2/886: getdents d5/dd/d69 0 2024-02-28T23:07:51.726 INFO:tasks.workunit.client.0.smithi057.stdout:3/699: symlink d3/d4/d1d/d47/d62/dbc/le0 0 2024-02-28T23:07:51.726 INFO:tasks.workunit.client.0.smithi057.stdout:8/891: read d4/d13/d18/f31 [1734113,116006] 0 2024-02-28T23:07:51.727 INFO:tasks.workunit.client.0.smithi057.stdout:2/898: readlink d8/d11/d21/d28/l79 0 2024-02-28T23:07:51.728 INFO:tasks.workunit.client.1.smithi084.stdout:4/950: read d1/f4 [4190452,72069] 0 2024-02-28T23:07:51.730 INFO:tasks.workunit.client.1.smithi084.stdout:2/887: creat d5/f112 x:0 0 0 2024-02-28T23:07:51.732 INFO:tasks.workunit.client.0.smithi057.stdout:0/949: dread d17/d20/d26/d48/d7c/d98/fe7 [0,4194304] 0 2024-02-28T23:07:51.733 INFO:tasks.workunit.client.0.smithi057.stdout:4/758: symlink d1a/d2f/db2/d49/d4d/l10a 0 2024-02-28T23:07:51.733 INFO:tasks.workunit.client.1.smithi084.stdout:2/888: rmdir d5/d7/d12/d24/d2a/d35 39 2024-02-28T23:07:51.733 INFO:tasks.workunit.client.1.smithi084.stdout:4/951: chown d1/d18/d20/d24/d2f/d3e/d52/ce6 37732 1 2024-02-28T23:07:51.734 INFO:tasks.workunit.client.1.smithi084.stdout:1/977: sync 2024-02-28T23:07:51.735 INFO:tasks.workunit.client.0.smithi057.stdout:0/950: readlink d17/d20/d4a/d78/lca 0 2024-02-28T23:07:51.735 INFO:tasks.workunit.client.1.smithi084.stdout:6/972: dwrite d1/d13/d2e/d41/f37 [0,4194304] 0 2024-02-28T23:07:51.738 INFO:tasks.workunit.client.1.smithi084.stdout:6/973: stat d1/d9/d1d/d1e/d33/f89 0 2024-02-28T23:07:51.738 INFO:tasks.workunit.client.1.smithi084.stdout:6/974: stat d1/d13/d47/c6e 0 2024-02-28T23:07:51.739 INFO:tasks.workunit.client.0.smithi057.stdout:9/762: dread dd/de/d24/d63/d77/dcf/d40/d44/f62 [0,4194304] 0 2024-02-28T23:07:51.740 INFO:tasks.workunit.client.1.smithi084.stdout:2/889: mkdir d5/dd/d113 0 2024-02-28T23:07:51.740 INFO:tasks.workunit.client.0.smithi057.stdout:2/899: fsync d8/d11/d39/d7b/d8b/f96 0 2024-02-28T23:07:51.741 INFO:tasks.workunit.client.0.smithi057.stdout:8/892: creat d4/de9/d56/f13b x:0 0 0 2024-02-28T23:07:51.741 INFO:tasks.workunit.client.0.smithi057.stdout:4/759: symlink d1a/d2f/db2/l10b 0 2024-02-28T23:07:51.742 INFO:tasks.workunit.client.1.smithi084.stdout:1/978: mkdir d12/d1b/d2e/d30/d140 0 2024-02-28T23:07:51.743 INFO:tasks.workunit.client.1.smithi084.stdout:1/979: read - d12/d1b/d2e/d52/dd7/f13e zero size 2024-02-28T23:07:51.747 INFO:tasks.workunit.client.1.smithi084.stdout:6/975: stat d1/d9/ddc/cec 0 2024-02-28T23:07:51.747 INFO:tasks.workunit.client.0.smithi057.stdout:6/740: dwrite d0/d17/d20/fc3 [0,4194304] 0 2024-02-28T23:07:51.754 INFO:tasks.workunit.client.1.smithi084.stdout:1/980: rename d12/d1b/d2e/d52/dd7/f100 to d12/d1b/d2e/d52/d11f/f141 0 2024-02-28T23:07:51.755 INFO:tasks.workunit.client.1.smithi084.stdout:4/952: write d1/d18/d20/d24/d2f/d3e/d52/d45/f88 [3507694,85570] 0 2024-02-28T23:07:51.758 INFO:tasks.workunit.client.0.smithi057.stdout:4/760: symlink d1a/d48/d9c/d2e/d8f/l10c 0 2024-02-28T23:07:51.758 INFO:tasks.workunit.client.1.smithi084.stdout:2/890: write d5/d7/d75/fc2 [2579267,32812] 0 2024-02-28T23:07:51.760 INFO:tasks.workunit.client.0.smithi057.stdout:2/900: symlink d8/d11/d21/d28/d49/d85/d9b/l11e 0 2024-02-28T23:07:51.761 INFO:tasks.workunit.client.1.smithi084.stdout:1/981: symlink d12/d84/d9a/l142 0 2024-02-28T23:07:51.762 INFO:tasks.workunit.client.1.smithi084.stdout:4/953: write d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fd2 [1778256,108053] 0 2024-02-28T23:07:51.764 INFO:tasks.workunit.client.1.smithi084.stdout:8/969: sync 2024-02-28T23:07:51.764 INFO:tasks.workunit.client.1.smithi084.stdout:5/967: sync 2024-02-28T23:07:51.766 INFO:tasks.workunit.client.0.smithi057.stdout:9/763: rename dd/de/d4a/d60/fe3 to dd/de/d14/d48/d4b/d41/ff5 0 2024-02-28T23:07:51.766 INFO:tasks.workunit.client.0.smithi057.stdout:3/700: dwrite d3/d4/d7/d9a/d9b/dab/fc1 [0,4194304] 0 2024-02-28T23:07:51.766 INFO:tasks.workunit.client.0.smithi057.stdout:0/951: dwrite d17/d20/d23/f75 [0,4194304] 0 2024-02-28T23:07:51.769 INFO:tasks.workunit.client.1.smithi084.stdout:2/891: rename d5/d1f/c100 to d5/d7/d9/d94/dbd/c114 0 2024-02-28T23:07:51.769 INFO:tasks.workunit.client.1.smithi084.stdout:5/968: fdatasync dc/f70 0 2024-02-28T23:07:51.769 INFO:tasks.workunit.client.0.smithi057.stdout:1/718: dread d2/d7/d4f/f63 [0,4194304] 0 2024-02-28T23:07:51.769 INFO:tasks.workunit.client.1.smithi084.stdout:6/976: dwrite d1/d13/d50/dbd/df9/f100 [0,4194304] 0 2024-02-28T23:07:51.771 INFO:tasks.workunit.client.0.smithi057.stdout:3/701: write d3/d8/f93 [1401589,90551] 0 2024-02-28T23:07:51.773 INFO:tasks.workunit.client.1.smithi084.stdout:4/954: dread - d1/d18/d20/d24/d2f/d3e/d52/f105 zero size 2024-02-28T23:07:51.776 INFO:tasks.workunit.client.0.smithi057.stdout:2/901: symlink d8/d11/d39/dda/l11f 0 2024-02-28T23:07:51.780 INFO:tasks.workunit.client.0.smithi057.stdout:4/761: creat d1a/d48/d9c/d2e/d8f/db7/f10d x:0 0 0 2024-02-28T23:07:51.780 INFO:tasks.workunit.client.1.smithi084.stdout:8/970: creat d0/d44/dfb/f137 x:0 0 0 2024-02-28T23:07:51.781 INFO:tasks.workunit.client.0.smithi057.stdout:8/893: rename d4/d13/d18/da7/ff3 to d4/d13/d2c/db1/f13c 0 2024-02-28T23:07:51.782 INFO:tasks.workunit.client.0.smithi057.stdout:7/814: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/fbc [4194304,4194304] 0 2024-02-28T23:07:51.785 INFO:tasks.workunit.client.1.smithi084.stdout:5/969: symlink dc/de/df/d31/l168 0 2024-02-28T23:07:51.787 INFO:tasks.workunit.client.0.smithi057.stdout:7/815: mkdir d0/d98/d10c 0 2024-02-28T23:07:51.787 INFO:tasks.workunit.client.0.smithi057.stdout:3/702: creat d3/d8/d1b/fe1 x:0 0 0 2024-02-28T23:07:51.788 INFO:tasks.workunit.client.1.smithi084.stdout:6/977: write d1/d13/d47/f56 [6640474,39235] 0 2024-02-28T23:07:51.791 INFO:tasks.workunit.client.1.smithi084.stdout:8/971: symlink d0/d9/d30/d33/d71/d84/dc0/l138 0 2024-02-28T23:07:51.796 INFO:tasks.workunit.client.0.smithi057.stdout:9/764: read dd/de/d14/f1e [4404330,56497] 0 2024-02-28T23:07:51.798 INFO:tasks.workunit.client.1.smithi084.stdout:5/970: rename dc/d51/cb5 to dc/de/d83/c169 0 2024-02-28T23:07:51.798 INFO:tasks.workunit.client.1.smithi084.stdout:4/955: dread d1/d18/d20/d14/d30/d47/da4/d10f/d115/f11c [0,4194304] 0 2024-02-28T23:07:51.800 INFO:tasks.workunit.client.1.smithi084.stdout:8/972: fdatasync d0/d9/d30/d33/d71/d84/dd9/de3/df7/ffa 0 2024-02-28T23:07:51.800 INFO:tasks.workunit.client.0.smithi057.stdout:0/952: creat d17/d20/d4a/d78/dd4/d123/f156 x:0 0 0 2024-02-28T23:07:51.801 INFO:tasks.workunit.client.1.smithi084.stdout:6/978: chown d1/d13/d50/d74/d9e/d101/f111 420957 1 2024-02-28T23:07:51.802 INFO:tasks.workunit.client.0.smithi057.stdout:4/762: rename d1a/d48/d9c/d2e/f7d to d1a/d48/d9c/d68/f10e 0 2024-02-28T23:07:51.803 INFO:tasks.workunit.client.0.smithi057.stdout:7/816: creat d0/d3f/f10d x:0 0 0 2024-02-28T23:07:51.803 INFO:tasks.workunit.client.1.smithi084.stdout:8/973: chown d0/d9/f88 26691 1 2024-02-28T23:07:51.806 INFO:tasks.workunit.client.1.smithi084.stdout:5/971: rmdir dc/de/df/d16/d34/d3c/d98 39 2024-02-28T23:07:51.806 INFO:tasks.workunit.client.1.smithi084.stdout:1/982: dwrite d12/d1b/d2e/d52/f73 [0,4194304] 0 2024-02-28T23:07:51.806 INFO:tasks.workunit.client.0.smithi057.stdout:3/703: dread d3/d8/d27/d70/f74 [0,4194304] 0 2024-02-28T23:07:51.807 INFO:tasks.workunit.client.1.smithi084.stdout:6/979: fsync d1/d13/d2e/d41/d2c/dfb/f9a 0 2024-02-28T23:07:51.808 INFO:tasks.workunit.client.0.smithi057.stdout:2/902: rmdir d8/d11/d21/de6/d10c 0 2024-02-28T23:07:51.808 INFO:tasks.workunit.client.1.smithi084.stdout:4/956: chown d1/d18/d20/d24/d2f/d3e/d52/d48/d73/d9d/ce2 149 1 2024-02-28T23:07:51.811 INFO:tasks.workunit.client.0.smithi057.stdout:6/741: write d0/d1b/d3e/d3d/d74/fb8 [754115,87733] 0 2024-02-28T23:07:51.811 INFO:tasks.workunit.client.0.smithi057.stdout:8/894: getdents d4/de9/d37/d9d 0 2024-02-28T23:07:51.811 INFO:tasks.workunit.client.1.smithi084.stdout:5/972: write dc/d51/d7c/f164 [780391,40899] 0 2024-02-28T23:07:51.813 INFO:tasks.workunit.client.1.smithi084.stdout:1/983: truncate d12/d1b/d63/f130 78477 0 2024-02-28T23:07:51.814 INFO:tasks.workunit.client.1.smithi084.stdout:4/957: rmdir d1/d18/d20/d24/ddb 39 2024-02-28T23:07:51.816 INFO:tasks.workunit.client.1.smithi084.stdout:4/958: chown d1/d18/d20/d14/l78 1 1 2024-02-28T23:07:51.816 INFO:tasks.workunit.client.0.smithi057.stdout:1/719: link d2/d7/df/db/d12/d19/l40 d2/d33/d54/dc8/dfa/l109 0 2024-02-28T23:07:51.818 INFO:tasks.workunit.client.1.smithi084.stdout:5/973: stat dc/de/df/d16/d34/de8/lba 0 2024-02-28T23:07:51.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:51 smithi057 ceph-mon[29096]: pgmap v32: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 125 MiB/s rd, 294 MiB/s wr, 660 op/s 2024-02-28T23:07:51.823 INFO:tasks.workunit.client.1.smithi084.stdout:8/974: creat d0/d9/d1b/d38/f139 x:0 0 0 2024-02-28T23:07:51.824 INFO:tasks.workunit.client.1.smithi084.stdout:4/959: rmdir d1/d12e 39 2024-02-28T23:07:51.825 INFO:tasks.workunit.client.0.smithi057.stdout:0/953: chown d17/d19/d4d/l121 26 1 2024-02-28T23:07:51.826 INFO:tasks.workunit.client.1.smithi084.stdout:6/980: creat d1/d13/d47/d91/dba/f140 x:0 0 0 2024-02-28T23:07:51.826 INFO:tasks.workunit.client.1.smithi084.stdout:2/892: dread d5/d7/d12/d24/fa7 [0,4194304] 0 2024-02-28T23:07:51.826 INFO:tasks.workunit.client.0.smithi057.stdout:1/720: chown d2/f21 5572200 1 2024-02-28T23:07:51.827 INFO:tasks.workunit.client.0.smithi057.stdout:2/903: mknod d8/d11/d15/db9/c120 0 2024-02-28T23:07:51.827 INFO:tasks.workunit.client.1.smithi084.stdout:5/974: mkdir dc/de/df/d31/dd1/da4/d161/d16a 0 2024-02-28T23:07:51.827 INFO:tasks.workunit.client.0.smithi057.stdout:6/742: rename d0/d4/d25/c39 to d0/d1b/d3e/d40/cf7 0 2024-02-28T23:07:51.828 INFO:tasks.workunit.client.1.smithi084.stdout:4/960: readlink d1/d18/d20/d14/d30/l56 0 2024-02-28T23:07:51.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:51 smithi084 ceph-mon[34826]: pgmap v32: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 125 MiB/s rd, 294 MiB/s wr, 660 op/s 2024-02-28T23:07:51.832 INFO:tasks.workunit.client.1.smithi084.stdout:2/893: symlink d5/d7/d12/d24/d2a/d53/l115 0 2024-02-28T23:07:51.832 INFO:tasks.workunit.client.1.smithi084.stdout:8/975: link d0/d24/d4d/fcb d0/d9/d1b/d38/d43/d51/de8/f13a 0 2024-02-28T23:07:51.834 INFO:tasks.workunit.client.1.smithi084.stdout:5/975: symlink dc/de/df/d31/dd1/d5f/d9b/d10e/l16b 0 2024-02-28T23:07:51.834 INFO:tasks.workunit.client.0.smithi057.stdout:0/954: fsync d17/d20/db6/fc9 0 2024-02-28T23:07:51.838 INFO:tasks.workunit.client.1.smithi084.stdout:2/894: symlink d5/d7/d9/d7c/l116 0 2024-02-28T23:07:51.839 INFO:tasks.workunit.client.1.smithi084.stdout:1/984: dwrite d12/d1b/d91/d9b/ffd [0,4194304] 0 2024-02-28T23:07:51.839 INFO:tasks.workunit.client.0.smithi057.stdout:6/743: chown d0/d17/d20/l30 41282354 1 2024-02-28T23:07:51.839 INFO:tasks.workunit.client.0.smithi057.stdout:7/817: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/f10e x:0 0 0 2024-02-28T23:07:51.841 INFO:tasks.workunit.client.1.smithi084.stdout:5/976: fdatasync dc/de/df/d16/f45 0 2024-02-28T23:07:51.841 INFO:tasks.workunit.client.1.smithi084.stdout:2/895: write d5/d1f/d42/d8f/ff2 [119509,10471] 0 2024-02-28T23:07:51.843 INFO:tasks.workunit.client.1.smithi084.stdout:4/961: truncate d1/d18/d20/d14/d30/d47/da4/d10f/d115/f119 197735 0 2024-02-28T23:07:51.844 INFO:tasks.workunit.client.1.smithi084.stdout:1/985: chown d12/f70 43 1 2024-02-28T23:07:51.846 INFO:tasks.workunit.client.1.smithi084.stdout:8/976: rmdir d0/d24/d78/df0/d11f 0 2024-02-28T23:07:51.849 INFO:tasks.workunit.client.0.smithi057.stdout:9/765: dwrite dd/d46/fc0 [0,4194304] 0 2024-02-28T23:07:51.850 INFO:tasks.workunit.client.1.smithi084.stdout:1/986: mknod d12/d1b/d91/c143 0 2024-02-28T23:07:51.852 INFO:tasks.workunit.client.0.smithi057.stdout:6/744: readlink d0/d1b/d3e/lb1 0 2024-02-28T23:07:51.852 INFO:tasks.workunit.client.0.smithi057.stdout:4/763: dwrite d1a/d48/d97/fc0 [0,4194304] 0 2024-02-28T23:07:51.852 INFO:tasks.workunit.client.0.smithi057.stdout:3/704: dwrite d3/d4/d7/d9a/d9b/dab/fc1 [0,4194304] 0 2024-02-28T23:07:51.853 INFO:tasks.workunit.client.1.smithi084.stdout:4/962: readlink d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/dc0/l137 0 2024-02-28T23:07:51.853 INFO:tasks.workunit.client.1.smithi084.stdout:2/896: dread d5/dd/fc4 [0,4194304] 0 2024-02-28T23:07:51.854 INFO:tasks.workunit.client.0.smithi057.stdout:6/745: chown d0/d1b 36209 1 2024-02-28T23:07:51.857 INFO:tasks.workunit.client.0.smithi057.stdout:0/955: dread d17/d20/d26/f69 [0,4194304] 0 2024-02-28T23:07:51.858 INFO:tasks.workunit.client.0.smithi057.stdout:4/764: chown d1a/d1e/d81/fca 4856 1 2024-02-28T23:07:51.859 INFO:tasks.workunit.client.0.smithi057.stdout:8/895: dwrite d4/d13/d18/d62/d66/d74/f12d [0,4194304] 0 2024-02-28T23:07:51.860 INFO:tasks.workunit.client.0.smithi057.stdout:9/766: fdatasync dd/de/d14/d48/d76/dbf/fd2 0 2024-02-28T23:07:51.861 INFO:tasks.workunit.client.1.smithi084.stdout:6/981: dwrite d1/d13/d50/d74/fe4 [0,4194304] 0 2024-02-28T23:07:51.862 INFO:tasks.workunit.client.0.smithi057.stdout:3/705: mkdir d3/d4f/d4c/d91/de2 0 2024-02-28T23:07:51.865 INFO:tasks.workunit.client.0.smithi057.stdout:4/765: chown d1a/d48/d9c/d68/dc5/lc7 39 1 2024-02-28T23:07:51.865 INFO:tasks.workunit.client.0.smithi057.stdout:3/706: chown d3/d8/d1b/l28 51 1 2024-02-28T23:07:51.869 INFO:tasks.workunit.client.1.smithi084.stdout:8/977: rename d0/d9/d1b/d38/d43/d51/de8/feb to d0/d44/dfb/d12d/f13b 0 2024-02-28T23:07:51.869 INFO:tasks.workunit.client.1.smithi084.stdout:4/963: creat d1/d18/d20/d24/f13d x:0 0 0 2024-02-28T23:07:51.869 INFO:tasks.workunit.client.1.smithi084.stdout:1/987: rename d12/d1b/d3b/l49 to d12/d1b/d91/d9b/da5/dcb/dd4/df6/l144 0 2024-02-28T23:07:51.870 INFO:tasks.workunit.client.1.smithi084.stdout:5/977: dread dc/de/df/d31/f89 [0,4194304] 0 2024-02-28T23:07:51.873 INFO:tasks.workunit.client.1.smithi084.stdout:6/982: fdatasync d1/d13/d2e/d41/d2c/f78 0 2024-02-28T23:07:51.874 INFO:tasks.workunit.client.1.smithi084.stdout:4/964: mknod d1/d18/d20/d14/c13e 0 2024-02-28T23:07:51.877 INFO:tasks.workunit.client.1.smithi084.stdout:5/978: stat dc/de/df/d31/dd1/d5f/d66/l68 0 2024-02-28T23:07:51.882 INFO:tasks.workunit.client.1.smithi084.stdout:5/979: fdatasync dc/de/df/d31/fcb 0 2024-02-28T23:07:51.886 INFO:tasks.workunit.client.1.smithi084.stdout:1/988: dread d12/d1b/d2e/d30/f46 [0,4194304] 0 2024-02-28T23:07:51.887 INFO:tasks.workunit.client.1.smithi084.stdout:4/965: mknod d1/d18/d20/d24/d2f/d3e/d52/d48/c13f 0 2024-02-28T23:07:51.902 INFO:tasks.workunit.client.1.smithi084.stdout:4/966: mkdir d1/d140 0 2024-02-28T23:07:51.907 INFO:tasks.workunit.client.1.smithi084.stdout:4/967: stat d1/d18/d20/d24/d2f/d3e/fe5 0 2024-02-28T23:07:51.907 INFO:tasks.workunit.client.1.smithi084.stdout:2/897: dwrite d5/d7/d9/d94/f98 [0,4194304] 0 2024-02-28T23:07:51.912 INFO:tasks.workunit.client.1.smithi084.stdout:2/898: readlink d5/d1f/d42/d74/la4 0 2024-02-28T23:07:51.914 INFO:tasks.workunit.client.1.smithi084.stdout:8/978: dwrite d0/d9/fb2 [0,4194304] 0 2024-02-28T23:07:51.918 INFO:tasks.workunit.client.1.smithi084.stdout:8/979: chown d0/d24/d4d/c87 58199878 1 2024-02-28T23:07:51.920 INFO:tasks.workunit.client.1.smithi084.stdout:5/980: dread dc/fc8 [0,4194304] 0 2024-02-28T23:07:51.921 INFO:tasks.workunit.client.1.smithi084.stdout:1/989: write d12/d1b/d63/f2a [5192927,117133] 0 2024-02-28T23:07:51.921 INFO:tasks.workunit.client.1.smithi084.stdout:2/899: getdents d5/dd/d69 0 2024-02-28T23:07:51.924 INFO:tasks.workunit.client.1.smithi084.stdout:2/900: fsync d5/d1f/d83/f97 0 2024-02-28T23:07:51.931 INFO:tasks.workunit.client.1.smithi084.stdout:5/981: rmdir dc/de/df/d16/d34/d3c/d128 39 2024-02-28T23:07:51.934 INFO:tasks.workunit.client.1.smithi084.stdout:2/901: readlink d5/d7/d12/d24/d2a/lf8 0 2024-02-28T23:07:51.938 INFO:tasks.workunit.client.1.smithi084.stdout:8/980: dwrite d0/d9/d1b/d38/d43/d51/d54/f9c [0,4194304] 0 2024-02-28T23:07:51.939 INFO:tasks.workunit.client.1.smithi084.stdout:5/982: dread dc/de/df/d31/f7e [0,4194304] 0 2024-02-28T23:07:51.940 INFO:tasks.workunit.client.1.smithi084.stdout:2/902: creat d5/d7/d2e/d4a/d5c/f117 x:0 0 0 2024-02-28T23:07:51.946 INFO:tasks.workunit.client.1.smithi084.stdout:8/981: rename d0/d24/l6c to d0/d9/l13c 0 2024-02-28T23:07:51.953 INFO:tasks.workunit.client.1.smithi084.stdout:8/982: rmdir d0/d9/d30/d33/d71/d84/dce/d10f 0 2024-02-28T23:07:51.961 INFO:tasks.workunit.client.1.smithi084.stdout:8/983: mkdir d0/d24/d4d/d13d 0 2024-02-28T23:07:51.964 INFO:tasks.workunit.client.1.smithi084.stdout:5/983: dwrite dc/de/df/d31/f135 [0,4194304] 0 2024-02-28T23:07:51.966 INFO:tasks.workunit.client.1.smithi084.stdout:2/903: dread d5/d7/d12/d24/f66 [0,4194304] 0 2024-02-28T23:07:51.967 INFO:tasks.workunit.client.1.smithi084.stdout:2/904: chown d5/d1f/c60 62122 1 2024-02-28T23:07:51.984 INFO:tasks.workunit.client.0.smithi057.stdout:4/766: dread - d1a/d2f/db2/dcb/dfe/ffc zero size 2024-02-28T23:07:51.984 INFO:tasks.workunit.client.0.smithi057.stdout:3/707: chown d3/d4/d7/le 1053384 1 2024-02-28T23:07:51.987 INFO:tasks.workunit.client.1.smithi084.stdout:8/984: symlink d0/d9/d1b/d38/dd4/l13e 0 2024-02-28T23:07:51.987 INFO:tasks.workunit.client.0.smithi057.stdout:9/767: mkdir dd/de/df6 0 2024-02-28T23:07:51.988 INFO:tasks.workunit.client.1.smithi084.stdout:2/905: symlink d5/d7/d9/d7c/d5b/l118 0 2024-02-28T23:07:51.990 INFO:tasks.workunit.client.1.smithi084.stdout:2/906: fdatasync d5/d1f/d42/d74/fd0 0 2024-02-28T23:07:51.992 INFO:tasks.workunit.client.0.smithi057.stdout:0/956: rename d17/c9b to d17/d20/d23/c157 0 2024-02-28T23:07:51.993 INFO:tasks.workunit.client.1.smithi084.stdout:2/907: chown d5/d7/d12/d24/d3a/d48/l9c 0 1 2024-02-28T23:07:51.993 INFO:tasks.workunit.client.1.smithi084.stdout:5/984: dread dc/d51/f12a [0,4194304] 0 2024-02-28T23:07:51.995 INFO:tasks.workunit.client.1.smithi084.stdout:5/985: readlink dc/d51/ld6 0 2024-02-28T23:07:51.995 INFO:tasks.workunit.client.1.smithi084.stdout:2/908: dread - d5/d7/d75/ffd zero size 2024-02-28T23:07:51.997 INFO:tasks.workunit.client.1.smithi084.stdout:5/986: symlink dc/de/d83/d11c/l16c 0 2024-02-28T23:07:51.998 INFO:tasks.workunit.client.1.smithi084.stdout:2/909: creat d5/d1f/d42/d74/f119 x:0 0 0 2024-02-28T23:07:51.998 INFO:tasks.workunit.client.0.smithi057.stdout:7/818: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/c73 d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/c10f 0 2024-02-28T23:07:51.999 INFO:tasks.workunit.client.0.smithi057.stdout:8/896: getdents d4/de9/d56/d6a/dbd 0 2024-02-28T23:07:52.000 INFO:tasks.workunit.client.1.smithi084.stdout:2/910: dread - d5/d7/d75/dcd/f103 zero size 2024-02-28T23:07:52.000 INFO:tasks.workunit.client.0.smithi057.stdout:0/957: creat d17/d20/d4a/d78/dd4/d123/f158 x:0 0 0 2024-02-28T23:07:52.002 INFO:tasks.workunit.client.1.smithi084.stdout:5/987: dread dc/de/df/d31/dd1/d5f/d9b/fb9 [0,4194304] 0 2024-02-28T23:07:52.004 INFO:tasks.workunit.client.0.smithi057.stdout:1/721: dread d2/d7/df/f52 [0,4194304] 0 2024-02-28T23:07:52.007 INFO:tasks.workunit.client.0.smithi057.stdout:4/767: write d1a/d2f/f45 [560078,115839] 0 2024-02-28T23:07:52.009 INFO:tasks.workunit.client.0.smithi057.stdout:8/897: symlink d4/de9/d56/l13d 0 2024-02-28T23:07:52.010 INFO:tasks.workunit.client.0.smithi057.stdout:7/819: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f110 x:0 0 0 2024-02-28T23:07:52.012 INFO:tasks.workunit.client.0.smithi057.stdout:1/722: rmdir d2/d7/d4f 39 2024-02-28T23:07:52.013 INFO:tasks.workunit.client.0.smithi057.stdout:3/708: dread d3/f5a [0,4194304] 0 2024-02-28T23:07:52.015 INFO:tasks.workunit.client.0.smithi057.stdout:2/904: dread d8/d11/d21/d28/d49/d85/d92/fba [0,4194304] 0 2024-02-28T23:07:52.017 INFO:tasks.workunit.client.1.smithi084.stdout:8/985: truncate d0/d9/d30/d33/f5e 549542 0 2024-02-28T23:07:52.017 INFO:tasks.workunit.client.1.smithi084.stdout:2/911: dwrite d5/d7/d2e/fc0 [4194304,4194304] 0 2024-02-28T23:07:52.020 INFO:tasks.workunit.client.0.smithi057.stdout:2/905: write d8/f19 [1166061,41795] 0 2024-02-28T23:07:52.020 INFO:tasks.workunit.client.0.smithi057.stdout:4/768: dread d1a/d48/db0/db1/f103 [0,4194304] 0 2024-02-28T23:07:52.024 INFO:tasks.workunit.client.1.smithi084.stdout:8/986: mkdir d0/d9/d1b/d38/d43/d13f 0 2024-02-28T23:07:52.026 INFO:tasks.workunit.client.0.smithi057.stdout:0/958: dwrite d17/d20/d26/d30/f96 [0,4194304] 0 2024-02-28T23:07:52.027 INFO:tasks.workunit.client.1.smithi084.stdout:2/912: truncate d5/d7/d9/d7c/d5b/d9f/fa0 4169731 0 2024-02-28T23:07:52.028 INFO:tasks.workunit.client.1.smithi084.stdout:5/988: dwrite dc/de/df/d16/d34/daa/dae/fbb [0,4194304] 0 2024-02-28T23:07:52.029 INFO:tasks.workunit.client.0.smithi057.stdout:4/769: rename d1a/d2f to d1a/d2f/db2/d49/d4d/d107/d10f 22 2024-02-28T23:07:52.030 INFO:tasks.workunit.client.1.smithi084.stdout:2/913: chown d5/d7/d2e/d4a/dae 1 1 2024-02-28T23:07:52.034 INFO:tasks.workunit.client.1.smithi084.stdout:2/914: mkdir d5/dd/d69/d11a 0 2024-02-28T23:07:52.036 INFO:tasks.workunit.client.0.smithi057.stdout:4/770: chown d1a/d1e/d81 2312543 1 2024-02-28T23:07:52.036 INFO:tasks.workunit.client.0.smithi057.stdout:7/820: readlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/lfe 0 2024-02-28T23:07:52.036 INFO:tasks.workunit.client.0.smithi057.stdout:1/723: dread - d2/d7/df/db/d12/d3f/f8c zero size 2024-02-28T23:07:52.036 INFO:tasks.workunit.client.0.smithi057.stdout:8/898: truncate d4/de9/d56/d88/fd1 79739 0 2024-02-28T23:07:52.037 INFO:tasks.workunit.client.1.smithi084.stdout:2/915: mkdir d5/d7/d9/d7c/d5b/da6/d11b 0 2024-02-28T23:07:52.038 INFO:tasks.workunit.client.0.smithi057.stdout:0/959: symlink d17/d19/dd2/ddb/l159 0 2024-02-28T23:07:52.040 INFO:tasks.workunit.client.1.smithi084.stdout:2/916: chown d5/ffe 34 1 2024-02-28T23:07:52.043 INFO:tasks.workunit.client.0.smithi057.stdout:4/771: chown d1a/d2f/c62 1510089 1 2024-02-28T23:07:52.045 INFO:tasks.workunit.client.0.smithi057.stdout:1/724: write d2/d71/d2d/d39/da3/ff6 [390571,49659] 0 2024-02-28T23:07:52.045 INFO:tasks.workunit.client.0.smithi057.stdout:8/899: mkdir d4/d13/d18/d13e 0 2024-02-28T23:07:52.049 INFO:tasks.workunit.client.1.smithi084.stdout:2/917: write d5/d7/d12/d24/d2a/d35/fa2 [2621922,46042] 0 2024-02-28T23:07:52.050 INFO:tasks.workunit.client.1.smithi084.stdout:2/918: fdatasync d5/d7/d2e/d4a/d5c/fff 0 2024-02-28T23:07:52.050 INFO:tasks.workunit.client.0.smithi057.stdout:9/768: dread dd/de/d4a/d9f/fab [0,4194304] 0 2024-02-28T23:07:52.050 INFO:tasks.workunit.client.0.smithi057.stdout:0/960: creat d17/d19/dd2/d12d/f15a x:0 0 0 2024-02-28T23:07:52.051 INFO:tasks.workunit.client.0.smithi057.stdout:3/709: dwrite d3/d4/d7/d9a/d9b/dab/fc1 [0,4194304] 0 2024-02-28T23:07:52.052 INFO:tasks.workunit.client.0.smithi057.stdout:4/772: write d1a/d2f/db2/f63 [3272131,77962] 0 2024-02-28T23:07:52.056 INFO:tasks.workunit.client.1.smithi084.stdout:5/989: dwrite dc/de/df/d16/d34/d3c/d77/fbd [0,4194304] 0 2024-02-28T23:07:52.057 INFO:tasks.workunit.client.1.smithi084.stdout:2/919: creat d5/d7/d9/d7c/f11c x:0 0 0 2024-02-28T23:07:52.058 INFO:tasks.workunit.client.0.smithi057.stdout:0/961: stat d17/d19/dd2/d12d/l143 0 2024-02-28T23:07:52.058 INFO:tasks.workunit.client.0.smithi057.stdout:1/725: write d2/d7/df/db/d3a/f99 [5143065,3194] 0 2024-02-28T23:07:52.059 INFO:tasks.workunit.client.0.smithi057.stdout:9/769: chown dd/de/d14/l25 230669633 1 2024-02-28T23:07:52.063 INFO:tasks.workunit.client.0.smithi057.stdout:9/770: chown dd/de/d14/d48/d4b 6155832 1 2024-02-28T23:07:52.071 INFO:tasks.workunit.client.1.smithi084.stdout:5/990: truncate dc/de/df/d31/f7e 419321 0 2024-02-28T23:07:52.071 INFO:tasks.workunit.client.0.smithi057.stdout:0/962: mkdir d17/d20/d4a/d15b 0 2024-02-28T23:07:52.073 INFO:tasks.workunit.client.1.smithi084.stdout:5/991: creat dc/de/df/d31/d8e/f16d x:0 0 0 2024-02-28T23:07:52.077 INFO:tasks.workunit.client.0.smithi057.stdout:1/726: chown d2/d7/d4f 35041545 1 2024-02-28T23:07:52.077 INFO:tasks.workunit.client.0.smithi057.stdout:3/710: mknod d3/d8/ce3 0 2024-02-28T23:07:52.077 INFO:tasks.workunit.client.0.smithi057.stdout:4/773: mkdir d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110 0 2024-02-28T23:07:52.078 INFO:tasks.workunit.client.0.smithi057.stdout:9/771: mkdir dd/de/d24/d63/d77/dcf/d40/d44/da5/df7 0 2024-02-28T23:07:52.081 INFO:tasks.workunit.client.1.smithi084.stdout:5/992: dread dc/de/df/d31/d8e/da9/dab/fe6 [0,4194304] 0 2024-02-28T23:07:52.083 INFO:tasks.workunit.client.0.smithi057.stdout:4/774: fdatasync d1a/d48/db0/db1/fb6 0 2024-02-28T23:07:52.083 INFO:tasks.workunit.client.0.smithi057.stdout:3/711: creat d3/d20/d2d/fe4 x:0 0 0 2024-02-28T23:07:52.083 INFO:tasks.workunit.client.0.smithi057.stdout:9/772: fsync dd/de/d14/f18 0 2024-02-28T23:07:52.083 INFO:tasks.workunit.client.0.smithi057.stdout:1/727: symlink d2/d7/df/db/d12/l10a 0 2024-02-28T23:07:52.087 INFO:tasks.workunit.client.1.smithi084.stdout:2/920: truncate d5/d1f/d42/d74/fa1 765105 0 2024-02-28T23:07:52.088 INFO:tasks.workunit.client.1.smithi084.stdout:2/921: chown d5/d1f/lb6 6 1 2024-02-28T23:07:52.089 INFO:tasks.workunit.client.1.smithi084.stdout:2/922: chown d5/d7/d75/de9 5591 1 2024-02-28T23:07:52.093 INFO:tasks.workunit.client.0.smithi057.stdout:3/712: mknod d3/d20/d2d/ce5 0 2024-02-28T23:07:52.095 INFO:tasks.workunit.client.1.smithi084.stdout:5/993: dwrite dc/de/d83/dd9/ffb [0,4194304] 0 2024-02-28T23:07:52.095 INFO:tasks.workunit.client.1.smithi084.stdout:5/994: stat dc/c75 0 2024-02-28T23:07:52.095 INFO:tasks.workunit.client.1.smithi084.stdout:5/995: dread - dc/fad zero size 2024-02-28T23:07:52.096 INFO:tasks.workunit.client.0.smithi057.stdout:0/963: dwrite d17/d19/f8f [4194304,4194304] 0 2024-02-28T23:07:52.096 INFO:tasks.workunit.client.0.smithi057.stdout:1/728: rename d2/d7/d4f/d5c to d2/d7/df/db/d10b 0 2024-02-28T23:07:52.099 INFO:tasks.workunit.client.1.smithi084.stdout:6/983: sync 2024-02-28T23:07:52.100 INFO:tasks.workunit.client.0.smithi057.stdout:7/821: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/faf [0,4194304] 0 2024-02-28T23:07:52.103 INFO:tasks.workunit.client.0.smithi057.stdout:3/713: fdatasync d3/d8/d27/f5f 0 2024-02-28T23:07:52.105 INFO:tasks.workunit.client.0.smithi057.stdout:0/964: unlink d17/d20/d26/l35 0 2024-02-28T23:07:52.109 INFO:tasks.workunit.client.0.smithi057.stdout:0/965: stat d17/d20/d26/c3d 0 2024-02-28T23:07:52.110 INFO:tasks.workunit.client.0.smithi057.stdout:1/729: write d2/d71/d62/f74 [2511900,38920] 0 2024-02-28T23:07:52.113 INFO:tasks.workunit.client.0.smithi057.stdout:4/775: write d1a/d48/d9c/f38 [4279732,120233] 0 2024-02-28T23:07:52.113 INFO:tasks.workunit.client.0.smithi057.stdout:9/773: write dd/de/d24/d63/d77/dcf/fe6 [4505984,42372] 0 2024-02-28T23:07:52.116 INFO:tasks.workunit.client.0.smithi057.stdout:0/966: chown d17/d20/d23/d47/d60/d125/f149 97021583 1 2024-02-28T23:07:52.122 INFO:tasks.workunit.client.0.smithi057.stdout:8/900: dwrite d4/d13/d1a/d4a/db3/dfc/dab/fd5 [0,4194304] 0 2024-02-28T23:07:52.122 INFO:tasks.workunit.client.0.smithi057.stdout:4/776: truncate d1a/d1e/d51/dc3/feb 512867 0 2024-02-28T23:07:52.123 INFO:tasks.workunit.client.1.smithi084.stdout:2/923: mknod d5/d7/d9/d7c/d5b/da6/c11d 0 2024-02-28T23:07:52.125 INFO:tasks.workunit.client.0.smithi057.stdout:4/777: chown d1a/d1e/d81/ce3 6462623 1 2024-02-28T23:07:52.125 INFO:tasks.workunit.client.1.smithi084.stdout:5/996: creat dc/de/df/d31/d8e/da9/f16e x:0 0 0 2024-02-28T23:07:52.125 INFO:tasks.workunit.client.1.smithi084.stdout:2/924: chown d5/d7/lee 11 1 2024-02-28T23:07:52.126 INFO:tasks.workunit.client.1.smithi084.stdout:6/984: mknod d1/d9/d1d/d1e/d33/d128/c141 0 2024-02-28T23:07:52.128 INFO:tasks.workunit.client.1.smithi084.stdout:2/925: creat d5/d1f/d42/d74/f11e x:0 0 0 2024-02-28T23:07:52.130 INFO:tasks.workunit.client.0.smithi057.stdout:4/778: chown d1a/d2f/db2/d49/d4d/l99 0 1 2024-02-28T23:07:52.130 INFO:tasks.workunit.client.0.smithi057.stdout:0/967: symlink d17/d20/d4a/d78/l15c 0 2024-02-28T23:07:52.131 INFO:tasks.workunit.client.1.smithi084.stdout:6/985: truncate d1/d13/d2e/d41/fa9 615242 0 2024-02-28T23:07:52.131 INFO:tasks.workunit.client.1.smithi084.stdout:2/926: chown d5/d7/d75/db4/cfb 0 1 2024-02-28T23:07:52.132 INFO:tasks.workunit.client.1.smithi084.stdout:6/986: chown d1/d9/f10b 1 1 2024-02-28T23:07:52.134 INFO:tasks.workunit.client.1.smithi084.stdout:2/927: symlink d5/d7/d9/d7c/l11f 0 2024-02-28T23:07:52.137 INFO:tasks.workunit.client.0.smithi057.stdout:8/901: truncate d4/de9/d56/d6a/d42/f5e 4839406 0 2024-02-28T23:07:52.138 INFO:tasks.workunit.client.1.smithi084.stdout:2/928: creat d5/d7/d9/d94/dbd/f120 x:0 0 0 2024-02-28T23:07:52.139 INFO:tasks.workunit.client.1.smithi084.stdout:6/987: rename d1/d13/d50/dbd/d10f/c11a to d1/d13/d47/d91/c142 0 2024-02-28T23:07:52.141 INFO:tasks.workunit.client.1.smithi084.stdout:6/988: fdatasync d1/d13/d47/f129 0 2024-02-28T23:07:52.142 INFO:tasks.workunit.client.0.smithi057.stdout:1/730: link d2/d7/df/db/d12/d19/f23 d2/d7/df/f10c 0 2024-02-28T23:07:52.142 INFO:tasks.workunit.client.0.smithi057.stdout:8/902: fdatasync d4/d13/d18/f65 0 2024-02-28T23:07:52.143 INFO:tasks.workunit.client.1.smithi084.stdout:2/929: mknod d5/d7/d9/d7c/d5b/da6/c121 0 2024-02-28T23:07:52.145 INFO:tasks.workunit.client.1.smithi084.stdout:6/989: symlink d1/d88/dbf/d135/l143 0 2024-02-28T23:07:52.148 INFO:tasks.workunit.client.1.smithi084.stdout:6/990: creat d1/f144 x:0 0 0 2024-02-28T23:07:52.149 INFO:tasks.workunit.client.0.smithi057.stdout:8/903: creat d4/d13/d2c/d4e/f13f x:0 0 0 2024-02-28T23:07:52.150 INFO:tasks.workunit.client.1.smithi084.stdout:6/991: fdatasync d1/d13/d2e/f119 0 2024-02-28T23:07:52.151 INFO:tasks.workunit.client.0.smithi057.stdout:4/779: mkdir d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/d111 0 2024-02-28T23:07:52.151 INFO:tasks.workunit.client.0.smithi057.stdout:7/822: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/fb2 [0,4194304] 0 2024-02-28T23:07:52.153 INFO:tasks.workunit.client.1.smithi084.stdout:5/997: dread dc/de/df/d31/dd1/f5d [0,4194304] 0 2024-02-28T23:07:52.154 INFO:tasks.workunit.client.0.smithi057.stdout:7/823: stat d0/d98 0 2024-02-28T23:07:52.155 INFO:tasks.workunit.client.1.smithi084.stdout:5/998: fsync dc/f2a 0 2024-02-28T23:07:52.157 INFO:tasks.workunit.client.0.smithi057.stdout:9/774: dwrite dd/de/d24/d63/d77/dcf/d40/d44/d92/fa8 [0,4194304] 0 2024-02-28T23:07:52.158 INFO:tasks.workunit.client.1.smithi084.stdout:6/992: link d1/d13/d50/dbd/cd4 d1/d13/d50/dbd/dc4/d13b/c145 0 2024-02-28T23:07:52.158 INFO:tasks.workunit.client.1.smithi084.stdout:2/930: dwrite d5/d7/d12/d24/d3a/d48/f5a [0,4194304] 0 2024-02-28T23:07:52.159 INFO:tasks.workunit.client.0.smithi057.stdout:1/731: creat d2/d7/df/db/d3a/d8f/f10d x:0 0 0 2024-02-28T23:07:52.160 INFO:tasks.workunit.client.1.smithi084.stdout:5/999: mknod dc/dc7/c16f 0 2024-02-28T23:07:52.164 INFO:tasks.workunit.client.1.smithi084.stdout:6/993: write d1/d13/d47/d91/ff7 [5013622,75770] 0 2024-02-28T23:07:52.165 INFO:tasks.workunit.client.1.smithi084.stdout:2/931: mknod d5/d7/d12/d24/d3a/c122 0 2024-02-28T23:07:52.165 INFO:tasks.workunit.client.0.smithi057.stdout:3/714: dread d3/d4/d7/fb9 [0,4194304] 0 2024-02-28T23:07:52.165 INFO:tasks.workunit.client.0.smithi057.stdout:4/780: mknod d1a/d1e/d51/c112 0 2024-02-28T23:07:52.165 INFO:tasks.workunit.client.0.smithi057.stdout:0/968: dwrite d17/d20/d23/d47/d60/d125/f149 [0,4194304] 0 2024-02-28T23:07:52.166 INFO:tasks.workunit.client.0.smithi057.stdout:4/781: chown d1a/d1e/fa8 1 1 2024-02-28T23:07:52.168 INFO:tasks.workunit.client.0.smithi057.stdout:3/715: fsync d3/d8/d1b/fa4 0 2024-02-28T23:07:52.168 INFO:tasks.workunit.client.0.smithi057.stdout:0/969: fdatasync d17/d19/dd2/fe6 0 2024-02-28T23:07:52.169 INFO:tasks.workunit.client.0.smithi057.stdout:1/732: rmdir d2/d7/df/db/d12/d19 39 2024-02-28T23:07:52.172 INFO:tasks.workunit.client.0.smithi057.stdout:9/775: symlink dd/de/d24/d63/d77/dcf/d40/lf8 0 2024-02-28T23:07:52.174 INFO:tasks.workunit.client.0.smithi057.stdout:8/904: dread d4/d13/d1a/d4a/db3/dfc/fff [0,4194304] 0 2024-02-28T23:07:52.174 INFO:tasks.workunit.client.0.smithi057.stdout:0/970: write d17/d20/d4a/d78/dd4/d123/f158 [804236,77723] 0 2024-02-28T23:07:52.175 INFO:tasks.workunit.client.0.smithi057.stdout:9/776: read dd/d46/fc0 [3616515,50614] 0 2024-02-28T23:07:52.178 INFO:tasks.workunit.client.0.smithi057.stdout:3/716: readlink d3/d4f/l43 0 2024-02-28T23:07:52.178 INFO:tasks.workunit.client.0.smithi057.stdout:8/905: chown d4/de9/d56/f13b 9 1 2024-02-28T23:07:52.179 INFO:tasks.workunit.client.0.smithi057.stdout:4/782: truncate d1a/d48/d97/f21 292362 0 2024-02-28T23:07:52.181 INFO:tasks.workunit.client.0.smithi057.stdout:8/906: stat d4/d13/d2c 0 2024-02-28T23:07:52.182 INFO:tasks.workunit.client.0.smithi057.stdout:0/971: symlink d17/d20/d23/d47/d60/d6f/l15d 0 2024-02-28T23:07:52.183 INFO:tasks.workunit.client.1.smithi084.stdout:6/994: dwrite d1/d13/d50/d74/d9e/fb9 [0,4194304] 0 2024-02-28T23:07:52.184 INFO:tasks.workunit.client.0.smithi057.stdout:9/777: chown dd/de/c13 361830183 1 2024-02-28T23:07:52.188 INFO:tasks.workunit.client.0.smithi057.stdout:9/778: chown dd/de/d14/f1e 215 1 2024-02-28T23:07:52.203 INFO:tasks.workunit.client.0.smithi057.stdout:9/779: readlink dd/de/d4a/d9f/la3 0 2024-02-28T23:07:52.203 INFO:tasks.workunit.client.0.smithi057.stdout:0/972: creat d17/d20/d4a/d78/d9d/f15e x:0 0 0 2024-02-28T23:07:52.205 INFO:tasks.workunit.client.0.smithi057.stdout:1/733: dread d2/d71/d2d/d5b/f7d [0,4194304] 0 2024-02-28T23:07:52.217 INFO:tasks.workunit.client.0.smithi057.stdout:5/825: sync 2024-02-28T23:07:52.217 INFO:tasks.workunit.client.1.smithi084.stdout:4/968: sync 2024-02-28T23:07:52.218 INFO:tasks.workunit.client.1.smithi084.stdout:4/969: fsync d1/d4b/fa6 0 2024-02-28T23:07:52.220 INFO:tasks.workunit.client.0.smithi057.stdout:4/783: dwrite d1a/d2f/f45 [4194304,4194304] 0 2024-02-28T23:07:52.222 INFO:tasks.workunit.client.0.smithi057.stdout:4/784: chown d1a/l7c 49 1 2024-02-28T23:07:52.224 INFO:tasks.workunit.client.0.smithi057.stdout:8/907: write d4/d13/d18/d62/dbb/f86 [396195,51811] 0 2024-02-28T23:07:52.233 INFO:tasks.workunit.client.0.smithi057.stdout:9/780: dwrite dd/d46/d6f/fa0 [0,4194304] 0 2024-02-28T23:07:52.238 INFO:tasks.workunit.client.0.smithi057.stdout:7/824: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f102 [0,4194304] 0 2024-02-28T23:07:52.243 INFO:tasks.workunit.client.1.smithi084.stdout:4/970: chown d1/d18/d20/d24/ddb/d12b/f133 302727 1 2024-02-28T23:07:52.244 INFO:tasks.workunit.client.0.smithi057.stdout:7/825: read d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f49 [1138590,112043] 0 2024-02-28T23:07:52.247 INFO:tasks.workunit.client.0.smithi057.stdout:7/826: stat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/c8f 0 2024-02-28T23:07:52.247 INFO:tasks.workunit.client.1.smithi084.stdout:4/971: truncate d1/d18/d20/d14/d30/d47/f82 4525764 0 2024-02-28T23:07:52.250 INFO:tasks.workunit.client.1.smithi084.stdout:4/972: stat d1/d18/d20/d24/d2f/d3e/d52/d57/c112 0 2024-02-28T23:07:52.251 INFO:tasks.workunit.client.1.smithi084.stdout:8/987: sync 2024-02-28T23:07:52.251 INFO:tasks.workunit.client.1.smithi084.stdout:1/990: sync 2024-02-28T23:07:52.256 INFO:tasks.workunit.client.1.smithi084.stdout:4/973: dread - d1/d18/d20/d24/d2f/d3e/d52/d45/fed zero size 2024-02-28T23:07:52.256 INFO:tasks.workunit.client.0.smithi057.stdout:7/827: truncate d0/d3f/d76/dc1/dea/d75/dbf/d10b/f10e 431933 0 2024-02-28T23:07:52.259 INFO:tasks.workunit.client.1.smithi084.stdout:1/991: unlink d12/d1b/d2e/d30/d77/lcd 0 2024-02-28T23:07:52.259 INFO:tasks.workunit.client.0.smithi057.stdout:3/717: symlink d3/d4/d7/le6 0 2024-02-28T23:07:52.260 INFO:tasks.workunit.client.1.smithi084.stdout:2/932: dwrite d5/d7/d9/d94/dbd/fdb [0,4194304] 0 2024-02-28T23:07:52.261 INFO:tasks.workunit.client.1.smithi084.stdout:8/988: creat d0/d9/d1b/d38/d43/dae/f140 x:0 0 0 2024-02-28T23:07:52.265 INFO:tasks.workunit.client.1.smithi084.stdout:4/974: symlink d1/d18/d20/d14/d121/l141 0 2024-02-28T23:07:52.265 INFO:tasks.workunit.client.0.smithi057.stdout:3/718: truncate d3/d4f/d4c/fcd 290103 0 2024-02-28T23:07:52.265 INFO:tasks.workunit.client.0.smithi057.stdout:7/828: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/lff 0 1 2024-02-28T23:07:52.266 INFO:tasks.workunit.client.1.smithi084.stdout:1/992: fdatasync d12/d1b/d2e/fe5 0 2024-02-28T23:07:52.267 INFO:tasks.workunit.client.0.smithi057.stdout:3/719: chown d3/d8/f5e 35043066 1 2024-02-28T23:07:52.269 INFO:tasks.workunit.client.1.smithi084.stdout:1/993: chown d12/d1b/d2e/d30/d77/la6 0 1 2024-02-28T23:07:52.277 INFO:tasks.workunit.client.1.smithi084.stdout:1/994: mkdir d12/d1b/d63/d7d/dd3/d145 0 2024-02-28T23:07:52.280 INFO:tasks.workunit.client.0.smithi057.stdout:9/781: rmdir dd/de/d4a/d9f 39 2024-02-28T23:07:52.281 INFO:tasks.workunit.client.0.smithi057.stdout:2/906: sync 2024-02-28T23:07:52.282 INFO:tasks.workunit.client.0.smithi057.stdout:9/782: stat dd/de/d14/f18 0 2024-02-28T23:07:52.282 INFO:tasks.workunit.client.0.smithi057.stdout:6/746: sync 2024-02-28T23:07:52.282 INFO:tasks.workunit.client.1.smithi084.stdout:4/975: creat d1/d18/d20/d24/d2f/d3e/d52/d57/f142 x:0 0 0 2024-02-28T23:07:52.285 INFO:tasks.workunit.client.0.smithi057.stdout:9/783: fsync dd/fd6 0 2024-02-28T23:07:52.287 INFO:tasks.workunit.client.0.smithi057.stdout:9/784: fdatasync dd/de/d14/f8e 0 2024-02-28T23:07:52.288 INFO:tasks.workunit.client.1.smithi084.stdout:4/976: symlink d1/d18/d20/d24/l143 0 2024-02-28T23:07:52.289 INFO:tasks.workunit.client.0.smithi057.stdout:4/785: creat d1a/d2f/db2/d49/d4d/dc9/dec/f113 x:0 0 0 2024-02-28T23:07:52.291 INFO:tasks.workunit.client.0.smithi057.stdout:7/829: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/l111 0 2024-02-28T23:07:52.292 INFO:tasks.workunit.client.0.smithi057.stdout:3/720: creat d3/d4/d7/d9a/d9b/dab/dd0/fe7 x:0 0 0 2024-02-28T23:07:52.293 INFO:tasks.workunit.client.0.smithi057.stdout:4/786: chown d1a/d2f/db2/d49/cd4 179 1 2024-02-28T23:07:52.296 INFO:tasks.workunit.client.0.smithi057.stdout:0/973: creat d17/d20/f15f x:0 0 0 2024-02-28T23:07:52.298 INFO:tasks.workunit.client.0.smithi057.stdout:6/747: mknod d0/d1b/cf8 0 2024-02-28T23:07:52.300 INFO:tasks.workunit.client.0.smithi057.stdout:1/734: link d2/d7/df/db/d3a/d8f/lde d2/d71/d2d/d39/l10e 0 2024-02-28T23:07:52.306 INFO:tasks.workunit.client.0.smithi057.stdout:2/907: rename d8/d11/d21/d70 to d8/d11/d21/d28/d41/d121 0 2024-02-28T23:07:52.306 INFO:tasks.workunit.client.1.smithi084.stdout:4/977: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/db5/f136 [0,4194304] 0 2024-02-28T23:07:52.309 INFO:tasks.workunit.client.0.smithi057.stdout:3/721: dwrite d3/d4/f30 [4194304,4194304] 0 2024-02-28T23:07:52.310 INFO:tasks.workunit.client.0.smithi057.stdout:5/826: creat d4/d9/fff x:0 0 0 2024-02-28T23:07:52.314 INFO:tasks.workunit.client.0.smithi057.stdout:9/785: dread - dd/de/d24/d63/d77/dcf/d40/d44/d5c/fa7 zero size 2024-02-28T23:07:52.316 INFO:tasks.workunit.client.0.smithi057.stdout:9/786: dread - dd/d46/d4d/fef zero size 2024-02-28T23:07:52.319 INFO:tasks.workunit.client.0.smithi057.stdout:4/787: symlink d1a/d48/d9c/d2e/d43/l114 0 2024-02-28T23:07:52.320 INFO:tasks.workunit.client.0.smithi057.stdout:0/974: truncate f10 3354875 0 2024-02-28T23:07:52.321 INFO:tasks.workunit.client.1.smithi084.stdout:4/978: dread d1/d18/d20/f11 [0,4194304] 0 2024-02-28T23:07:52.324 INFO:tasks.workunit.client.0.smithi057.stdout:0/975: write d17/d20/d23/d47/d60/dea/f11d [785907,28131] 0 2024-02-28T23:07:52.326 INFO:tasks.workunit.client.1.smithi084.stdout:4/979: chown d1/d18/d20/d65/d8e/d55/l41 796 1 2024-02-28T23:07:52.329 INFO:tasks.workunit.client.0.smithi057.stdout:7/830: mkdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/d112 0 2024-02-28T23:07:52.329 INFO:tasks.workunit.client.1.smithi084.stdout:4/980: chown d1/d18/d20/d65/d8e/d55/l41 7779 1 2024-02-28T23:07:52.332 INFO:tasks.workunit.client.0.smithi057.stdout:7/831: chown d0/d3f/d76/dc1/dea/c61 4986 1 2024-02-28T23:07:52.337 INFO:tasks.workunit.client.0.smithi057.stdout:2/908: write d8/d11/d56/f102 [952102,31459] 0 2024-02-28T23:07:52.341 INFO:tasks.workunit.client.1.smithi084.stdout:2/933: truncate d5/d1f/d42/d74/fa1 522928 0 2024-02-28T23:07:52.343 INFO:tasks.workunit.client.0.smithi057.stdout:9/787: creat dd/de/d24/d63/d77/dcf/d40/d44/ff9 x:0 0 0 2024-02-28T23:07:52.344 INFO:tasks.workunit.client.1.smithi084.stdout:8/989: write d0/d44/d9f/fd1 [780871,74105] 0 2024-02-28T23:07:52.344 INFO:tasks.workunit.client.1.smithi084.stdout:6/995: dwrite d1/d13/d50/d74/f138 [0,4194304] 0 2024-02-28T23:07:52.345 INFO:tasks.workunit.client.0.smithi057.stdout:8/908: truncate d4/de9/d56/d88/fe6 1694789 0 2024-02-28T23:07:52.348 INFO:tasks.workunit.client.0.smithi057.stdout:6/748: mknod d0/d17/d94/da0/de6/df4/cf9 0 2024-02-28T23:07:52.349 INFO:tasks.workunit.client.1.smithi084.stdout:8/990: fsync d0/d9/d30/db6/f80 0 2024-02-28T23:07:52.351 INFO:tasks.workunit.client.1.smithi084.stdout:8/991: stat d0/d9/d30/d33/d71 0 2024-02-28T23:07:52.353 INFO:tasks.workunit.client.1.smithi084.stdout:6/996: write d1/d9/d1d/d1e/d33/f8f [4447717,112781] 0 2024-02-28T23:07:52.356 INFO:tasks.workunit.client.0.smithi057.stdout:7/832: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/faf [0,4194304] 0 2024-02-28T23:07:52.357 INFO:tasks.workunit.client.1.smithi084.stdout:6/997: write d1/d13/d47/d91/dba/fe3 [1583558,16681] 0 2024-02-28T23:07:52.359 INFO:tasks.workunit.client.1.smithi084.stdout:8/992: unlink d0/d24/c108 0 2024-02-28T23:07:52.359 INFO:tasks.workunit.client.0.smithi057.stdout:3/722: mkdir d3/d4/de8 0 2024-02-28T23:07:52.368 INFO:tasks.workunit.client.1.smithi084.stdout:8/993: creat d0/d9/d1b/d38/dd4/f141 x:0 0 0 2024-02-28T23:07:52.368 INFO:tasks.workunit.client.0.smithi057.stdout:9/788: unlink dd/d46/d4d/c4e 0 2024-02-28T23:07:52.369 INFO:tasks.workunit.client.0.smithi057.stdout:9/789: dread - dd/de/d24/d72/f7c zero size 2024-02-28T23:07:52.370 INFO:tasks.workunit.client.1.smithi084.stdout:1/995: creat d12/d1b/d2e/f146 x:0 0 0 2024-02-28T23:07:52.372 INFO:tasks.workunit.client.0.smithi057.stdout:0/976: creat d17/d20/d26/d48/d7c/d98/da6/d153/f160 x:0 0 0 2024-02-28T23:07:52.374 INFO:tasks.workunit.client.1.smithi084.stdout:1/996: symlink d12/d1b/d91/df9/l147 0 2024-02-28T23:07:52.374 INFO:tasks.workunit.client.0.smithi057.stdout:6/749: dwrite d0/d17/d94/fe5 [0,4194304] 0 2024-02-28T23:07:52.374 INFO:tasks.workunit.client.0.smithi057.stdout:1/735: creat d2/d7/df/db/d12/d19/da4/f10f x:0 0 0 2024-02-28T23:07:52.375 INFO:tasks.workunit.client.1.smithi084.stdout:4/981: write d1/d18/d20/d65/d8e/d55/dc3/f10e [1204449,129081] 0 2024-02-28T23:07:52.375 INFO:tasks.workunit.client.0.smithi057.stdout:2/909: mknod d8/d11/db3/d107/c122 0 2024-02-28T23:07:52.375 INFO:tasks.workunit.client.1.smithi084.stdout:1/997: fdatasync d12/d1b/d63/f130 0 2024-02-28T23:07:52.379 INFO:tasks.workunit.client.1.smithi084.stdout:4/982: fsync d1/d18/d20/d14/d30/f81 0 2024-02-28T23:07:52.380 INFO:tasks.workunit.client.0.smithi057.stdout:3/723: mknod d3/d4/d1d/d47/d62/ce9 0 2024-02-28T23:07:52.380 INFO:tasks.workunit.client.0.smithi057.stdout:4/788: link d1a/d48/d9c/d2e/d43/c94 d1a/d2f/c115 0 2024-02-28T23:07:52.383 INFO:tasks.workunit.client.0.smithi057.stdout:4/789: chown d1a/d48/d6c/f8b 229531 1 2024-02-28T23:07:52.384 INFO:tasks.workunit.client.0.smithi057.stdout:1/736: read d2/d7/df/db/d12/d3f/d47/f9f [4117403,72875] 0 2024-02-28T23:07:52.388 INFO:tasks.workunit.client.1.smithi084.stdout:2/934: dwrite d5/d7/d9/d7c/d5b/da6/f10e [4194304,4194304] 0 2024-02-28T23:07:52.389 INFO:tasks.workunit.client.0.smithi057.stdout:7/833: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d3e/c113 0 2024-02-28T23:07:52.392 INFO:tasks.workunit.client.1.smithi084.stdout:2/935: fdatasync d5/d7/d12/d24/d2a/f52 0 2024-02-28T23:07:52.398 INFO:tasks.workunit.client.0.smithi057.stdout:5/827: getdents d4/d7/d8 0 2024-02-28T23:07:52.399 INFO:tasks.workunit.client.1.smithi084.stdout:2/936: truncate d5/d7/d2e/d4a/d5c/fe7 449305 0 2024-02-28T23:07:52.400 INFO:tasks.workunit.client.1.smithi084.stdout:6/998: write d1/d13/d2e/d41/dbb/f127 [966435,4029] 0 2024-02-28T23:07:52.404 INFO:tasks.workunit.client.1.smithi084.stdout:4/983: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf [4194304,4194304] 0 2024-02-28T23:07:52.404 INFO:tasks.workunit.client.0.smithi057.stdout:0/977: dwrite d17/d20/d26/f46 [4194304,4194304] 0 2024-02-28T23:07:52.404 INFO:tasks.workunit.client.0.smithi057.stdout:3/724: creat d3/d8/d1b/fea x:0 0 0 2024-02-28T23:07:52.404 INFO:tasks.workunit.client.0.smithi057.stdout:9/790: creat dd/d46/dc4/de5/ffa x:0 0 0 2024-02-28T23:07:52.406 INFO:tasks.workunit.client.0.smithi057.stdout:8/909: dread d4/d13/d18/d62/d66/fc4 [0,4194304] 0 2024-02-28T23:07:52.411 INFO:tasks.workunit.client.0.smithi057.stdout:4/790: chown d1a/d48/d9c/d2e/d8f/db7/cdc 20130 1 2024-02-28T23:07:52.411 INFO:tasks.workunit.client.1.smithi084.stdout:1/998: write d12/da2/f137 [685148,122982] 0 2024-02-28T23:07:52.412 INFO:tasks.workunit.client.1.smithi084.stdout:4/984: truncate d1/d18/d20/d14/d30/f113 795272 0 2024-02-28T23:07:52.414 INFO:tasks.workunit.client.1.smithi084.stdout:1/999: chown d12/d1b/f31 60052 1 2024-02-28T23:07:52.419 INFO:tasks.workunit.client.0.smithi057.stdout:6/750: creat d0/d1b/d3e/d3d/d42/d4f/dd0/ffa x:0 0 0 2024-02-28T23:07:52.421 INFO:tasks.workunit.client.0.smithi057.stdout:7/834: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/d9f/c114 0 2024-02-28T23:07:52.423 INFO:tasks.workunit.client.1.smithi084.stdout:6/999: write d1/d13/d47/f122 [2805879,117260] 0 2024-02-28T23:07:52.431 INFO:tasks.workunit.client.0.smithi057.stdout:7/835: readlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d4a/lba 0 2024-02-28T23:07:52.432 INFO:tasks.workunit.client.0.smithi057.stdout:3/725: dwrite d3/d4/d1d/fcb [0,4194304] 0 2024-02-28T23:07:52.435 INFO:tasks.workunit.client.0.smithi057.stdout:4/791: mknod d1a/d2f/db2/d49/d4d/dc9/dec/c116 0 2024-02-28T23:07:52.435 INFO:tasks.workunit.client.0.smithi057.stdout:7/836: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/lab 445099 1 2024-02-28T23:07:52.436 INFO:tasks.workunit.client.1.smithi084.stdout:4/985: dwrite d1/d18/d20/d24/d2f/d3e/d52/f9a [4194304,4194304] 0 2024-02-28T23:07:52.437 INFO:tasks.workunit.client.0.smithi057.stdout:6/751: readlink d0/d1b/d3e/d3d/d42/lee 0 2024-02-28T23:07:52.437 INFO:tasks.workunit.client.0.smithi057.stdout:5/828: symlink d4/d7/l100 0 2024-02-28T23:07:52.438 INFO:tasks.workunit.client.0.smithi057.stdout:8/910: mkdir d4/de9/d56/d6a/d140 0 2024-02-28T23:07:52.440 INFO:tasks.workunit.client.0.smithi057.stdout:3/726: creat d3/d20/d2d/feb x:0 0 0 2024-02-28T23:07:52.441 INFO:tasks.workunit.client.0.smithi057.stdout:4/792: write d1a/d48/db0/f5d [4873847,123595] 0 2024-02-28T23:07:52.442 INFO:tasks.workunit.client.0.smithi057.stdout:8/911: fsync d4/d13/d18/f65 0 2024-02-28T23:07:52.444 INFO:tasks.workunit.client.0.smithi057.stdout:3/727: fdatasync d3/d8/d27/f77 0 2024-02-28T23:07:52.444 INFO:tasks.workunit.client.1.smithi084.stdout:4/986: mkdir d1/d18/d20/d144 0 2024-02-28T23:07:52.445 INFO:tasks.workunit.client.0.smithi057.stdout:2/910: write d8/d11/d21/f86 [353299,110359] 0 2024-02-28T23:07:52.445 INFO:tasks.workunit.client.0.smithi057.stdout:5/829: write d4/d9/d16/d4e/fed [713078,80195] 0 2024-02-28T23:07:52.447 INFO:tasks.workunit.client.0.smithi057.stdout:6/752: truncate d0/d1b/d3e/d40/fa6 529038 0 2024-02-28T23:07:52.449 INFO:tasks.workunit.client.1.smithi084.stdout:2/937: dread d5/d7/d12/d24/d2a/f52 [0,4194304] 0 2024-02-28T23:07:52.453 INFO:tasks.workunit.client.0.smithi057.stdout:7/837: dread - d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f110 zero size 2024-02-28T23:07:52.453 INFO:tasks.workunit.client.1.smithi084.stdout:4/987: creat d1/d18/d20/d144/f145 x:0 0 0 2024-02-28T23:07:52.457 INFO:tasks.workunit.client.0.smithi057.stdout:6/753: stat d0/d17/d94/da0/de6/df4 0 2024-02-28T23:07:52.459 INFO:tasks.workunit.client.1.smithi084.stdout:4/988: truncate d1/f6 841622 0 2024-02-28T23:07:52.460 INFO:tasks.workunit.client.0.smithi057.stdout:3/728: truncate d3/d8/d27/fbb 237528 0 2024-02-28T23:07:52.462 INFO:tasks.workunit.client.0.smithi057.stdout:4/793: creat d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/f117 x:0 0 0 2024-02-28T23:07:52.473 INFO:tasks.workunit.client.0.smithi057.stdout:1/737: dwrite d2/f8 [0,4194304] 0 2024-02-28T23:07:52.475 INFO:tasks.workunit.client.0.smithi057.stdout:5/830: write d4/d9/d14/fcc [3826336,107625] 0 2024-02-28T23:07:52.475 INFO:tasks.workunit.client.0.smithi057.stdout:9/791: dwrite dd/de/d4a/d60/f83 [0,4194304] 0 2024-02-28T23:07:52.477 INFO:tasks.workunit.client.0.smithi057.stdout:2/911: link d8/d11/d21/d28/d41/f53 d8/d11/d21/d108/f123 0 2024-02-28T23:07:52.480 INFO:tasks.workunit.client.0.smithi057.stdout:5/831: chown d4/d7/d8/lf7 125931450 1 2024-02-28T23:07:52.480 INFO:tasks.workunit.client.0.smithi057.stdout:8/912: dwrite d4/de9/d56/d6a/d54/d6b/f6c [0,4194304] 0 2024-02-28T23:07:52.483 INFO:tasks.workunit.client.0.smithi057.stdout:8/913: chown d4/d13/d2c/db1/fde 1774497947 1 2024-02-28T23:07:52.484 INFO:tasks.workunit.client.0.smithi057.stdout:3/729: creat d3/d4/d1d/d47/d62/fec x:0 0 0 2024-02-28T23:07:52.489 INFO:tasks.workunit.client.1.smithi084.stdout:2/938: dread d5/dd/d69/f85 [4194304,4194304] 0 2024-02-28T23:07:52.492 INFO:tasks.workunit.client.0.smithi057.stdout:9/792: fdatasync dd/de/d24/f85 0 2024-02-28T23:07:52.492 INFO:tasks.workunit.client.0.smithi057.stdout:8/914: fsync d4/d13/d7f/d128/f134 0 2024-02-28T23:07:52.492 INFO:tasks.workunit.client.0.smithi057.stdout:8/915: fsync d4/f5 0 2024-02-28T23:07:52.496 INFO:tasks.workunit.client.0.smithi057.stdout:3/730: fdatasync d3/d4/d7/d9a/d9b/dab/dd0/fd6 0 2024-02-28T23:07:52.496 INFO:tasks.workunit.client.0.smithi057.stdout:8/916: fsync d4/de9/d56/d6a/d42/fcd 0 2024-02-28T23:07:52.501 INFO:tasks.workunit.client.0.smithi057.stdout:8/917: stat d4/de9/d56/d6a/d54/d6b/dca/ffb 0 2024-02-28T23:07:52.503 INFO:tasks.workunit.client.0.smithi057.stdout:4/794: rmdir d1a/d2f 39 2024-02-28T23:07:52.503 INFO:tasks.workunit.client.0.smithi057.stdout:2/912: chown d8/d11/d39/d83/fa3 6 1 2024-02-28T23:07:52.504 INFO:tasks.workunit.client.0.smithi057.stdout:2/913: fsync d8/d11/d21/f86 0 2024-02-28T23:07:52.506 INFO:tasks.workunit.client.0.smithi057.stdout:1/738: rename d2/d7/f37 to d2/d7/df/db/d3a/d2a/f110 0 2024-02-28T23:07:52.506 INFO:tasks.workunit.client.0.smithi057.stdout:0/978: dread d17/d20/d23/d47/fd8 [0,4194304] 0 2024-02-28T23:07:52.506 INFO:tasks.workunit.client.1.smithi084.stdout:2/939: dwrite d5/dd/fc4 [0,4194304] 0 2024-02-28T23:07:52.515 INFO:tasks.workunit.client.0.smithi057.stdout:8/918: symlink d4/df7/l141 0 2024-02-28T23:07:52.521 INFO:tasks.workunit.client.0.smithi057.stdout:7/838: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fb3 [0,4194304] 0 2024-02-28T23:07:52.524 INFO:tasks.workunit.client.0.smithi057.stdout:6/754: dread d0/d1b/d3e/d3d/d42/d4f/d53/f80 [0,4194304] 0 2024-02-28T23:07:52.524 INFO:tasks.workunit.client.1.smithi084.stdout:2/940: write d5/d1f/d42/d8f/fa8 [2336764,60859] 0 2024-02-28T23:07:52.527 INFO:tasks.workunit.client.0.smithi057.stdout:6/755: write d0/d1b/fed [159704,89164] 0 2024-02-28T23:07:52.527 INFO:tasks.workunit.client.1.smithi084.stdout:2/941: rmdir d5/d1f/d42/d8f/dbe/d10b 39 2024-02-28T23:07:52.530 INFO:tasks.workunit.client.1.smithi084.stdout:2/942: write d5/d7/d9/d7c/d5b/da6/fbb [1672765,16462] 0 2024-02-28T23:07:52.532 INFO:tasks.workunit.client.1.smithi084.stdout:2/943: readlink d5/d7/d9/d7c/d5b/d9f/lb5 0 2024-02-28T23:07:52.540 INFO:tasks.workunit.client.0.smithi057.stdout:2/914: rename d8/d11/d39/c46 to d8/d11/d21/d28/dec/c124 0 2024-02-28T23:07:52.545 INFO:tasks.workunit.client.1.smithi084.stdout:2/944: write d5/d7/d1d/f50 [4763980,110644] 0 2024-02-28T23:07:52.545 INFO:tasks.workunit.client.0.smithi057.stdout:4/795: unlink d1a/d48/d97/d7f/cbf 0 2024-02-28T23:07:52.546 INFO:tasks.workunit.client.1.smithi084.stdout:2/945: chown d5/d7/d12/d24/d2a/d35/f5d 3763 1 2024-02-28T23:07:52.549 INFO:tasks.workunit.client.1.smithi084.stdout:2/946: unlink d5/d7/d2e/ffa 0 2024-02-28T23:07:52.555 INFO:tasks.workunit.client.0.smithi057.stdout:0/979: dread d17/d20/d26/d30/d6b/fc3 [0,4194304] 0 2024-02-28T23:07:52.555 INFO:tasks.workunit.client.0.smithi057.stdout:6/756: truncate d0/d4/f19 4432900 0 2024-02-28T23:07:52.571 INFO:tasks.workunit.client.0.smithi057.stdout:1/739: truncate d2/f21 2689033 0 2024-02-28T23:07:52.572 INFO:tasks.workunit.client.0.smithi057.stdout:2/915: mknod d8/dae/c125 0 2024-02-28T23:07:52.572 INFO:tasks.workunit.client.0.smithi057.stdout:5/832: getdents d4/d9/d16/d8f 0 2024-02-28T23:07:52.575 INFO:tasks.workunit.client.0.smithi057.stdout:7/839: mkdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d115 0 2024-02-28T23:07:52.576 INFO:tasks.workunit.client.0.smithi057.stdout:5/833: stat d4/d9/d16/f18 0 2024-02-28T23:07:52.576 INFO:tasks.workunit.client.0.smithi057.stdout:3/731: getdents d3/d4 0 2024-02-28T23:07:52.577 INFO:tasks.workunit.client.0.smithi057.stdout:6/757: creat d0/d1b/d3e/d3d/d42/ffb x:0 0 0 2024-02-28T23:07:52.577 INFO:tasks.workunit.client.0.smithi057.stdout:8/919: creat d4/de9/d56/d6a/d54/f142 x:0 0 0 2024-02-28T23:07:52.581 INFO:tasks.workunit.client.0.smithi057.stdout:7/840: truncate d0/d3f/f6c 335761 0 2024-02-28T23:07:52.581 INFO:tasks.workunit.client.0.smithi057.stdout:0/980: write d17/d19/d4d/d7b/f94 [1041542,17042] 0 2024-02-28T23:07:52.586 INFO:tasks.workunit.client.0.smithi057.stdout:5/834: creat d4/d7/d58/f101 x:0 0 0 2024-02-28T23:07:52.589 INFO:tasks.workunit.client.0.smithi057.stdout:3/732: stat d3/d8/d1b/c7b 0 2024-02-28T23:07:52.589 INFO:tasks.workunit.client.0.smithi057.stdout:6/758: creat d0/d1b/d3e/ffc x:0 0 0 2024-02-28T23:07:52.591 INFO:tasks.workunit.client.0.smithi057.stdout:0/981: mkdir d17/d20/d26/d48/d7c/d98/da6/d161 0 2024-02-28T23:07:52.595 INFO:tasks.workunit.client.0.smithi057.stdout:5/835: chown d4/d32/df1/d24/d30/d62/fd5 5 1 2024-02-28T23:07:52.596 INFO:tasks.workunit.client.0.smithi057.stdout:7/841: link d0/c3 d0/d3f/c116 0 2024-02-28T23:07:52.601 INFO:tasks.workunit.client.0.smithi057.stdout:6/759: creat d0/d1b/d3e/d3d/d42/d4f/ffd x:0 0 0 2024-02-28T23:07:52.602 INFO:tasks.workunit.client.0.smithi057.stdout:5/836: fdatasync d4/d7/d8/ff 0 2024-02-28T23:07:52.605 INFO:tasks.workunit.client.0.smithi057.stdout:3/733: write d3/d8/d27/f5f [5130887,68051] 0 2024-02-28T23:07:52.606 INFO:tasks.workunit.client.0.smithi057.stdout:1/740: dread d2/d7/f41 [0,4194304] 0 2024-02-28T23:07:52.607 INFO:tasks.workunit.client.0.smithi057.stdout:6/760: readlink d0/d1b/d3e/lb1 0 2024-02-28T23:07:52.608 INFO:tasks.workunit.client.0.smithi057.stdout:4/796: dread d1a/d2f/f45 [4194304,4194304] 0 2024-02-28T23:07:52.609 INFO:tasks.workunit.client.0.smithi057.stdout:2/916: dwrite d8/d11/d21/d28/f73 [0,4194304] 0 2024-02-28T23:07:52.610 INFO:tasks.workunit.client.0.smithi057.stdout:0/982: link d17/d20/d26/f41 d17/d20/d26/d48/d7c/d98/da6/d153/f162 0 2024-02-28T23:07:52.613 INFO:tasks.workunit.client.0.smithi057.stdout:8/920: dwrite d4/d13/d1a/d4a/db3/dfc/fa0 [4194304,4194304] 0 2024-02-28T23:07:52.614 INFO:tasks.workunit.client.0.smithi057.stdout:6/761: creat d0/d1b/ffe x:0 0 0 2024-02-28T23:07:52.618 INFO:tasks.workunit.client.0.smithi057.stdout:8/921: fdatasync d4/d13/d18/d62/d66/fd8 0 2024-02-28T23:07:52.618 INFO:tasks.workunit.client.0.smithi057.stdout:6/762: fdatasync d0/d17/d94/fe5 0 2024-02-28T23:07:52.620 INFO:tasks.workunit.client.0.smithi057.stdout:8/922: write d4/df7/f111 [145451,13517] 0 2024-02-28T23:07:52.623 INFO:tasks.workunit.client.0.smithi057.stdout:2/917: chown d8/d11/db3/d107 11 1 2024-02-28T23:07:52.628 INFO:tasks.workunit.client.0.smithi057.stdout:3/734: mknod d3/d4/d7/d9a/ced 0 2024-02-28T23:07:52.631 INFO:tasks.workunit.client.0.smithi057.stdout:1/741: read d2/d71/d2d/f46 [67233,60893] 0 2024-02-28T23:07:52.633 INFO:tasks.workunit.client.0.smithi057.stdout:0/983: mkdir d17/d20/d26/d48/d55/d88/d154/d163 0 2024-02-28T23:07:52.634 INFO:tasks.workunit.client.0.smithi057.stdout:1/742: chown d2/d71/d2d/d5b/faf 8564922 1 2024-02-28T23:07:52.635 INFO:tasks.workunit.client.1.smithi084.stdout:8/994: sync 2024-02-28T23:07:52.635 INFO:tasks.workunit.client.1.smithi084.stdout:4/989: sync 2024-02-28T23:07:52.636 INFO:tasks.workunit.client.0.smithi057.stdout:0/984: write d17/d19/dd2/fe6 [981203,75608] 0 2024-02-28T23:07:52.637 INFO:tasks.workunit.client.0.smithi057.stdout:5/837: rename d4/d32/df1/d24/c6d to d4/d32/df1/d24/da2/dbb/d95/c102 0 2024-02-28T23:07:52.639 INFO:tasks.workunit.client.1.smithi084.stdout:8/995: rmdir d0/d44/dfb/d12d 39 2024-02-28T23:07:52.639 INFO:tasks.workunit.client.0.smithi057.stdout:7/842: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/fb2 [0,4194304] 0 2024-02-28T23:07:52.640 INFO:tasks.workunit.client.0.smithi057.stdout:0/985: dread - d17/d20/d4a/d78/dd4/d123/f156 zero size 2024-02-28T23:07:52.643 INFO:tasks.workunit.client.0.smithi057.stdout:3/735: mkdir d3/d4/de8/dee 0 2024-02-28T23:07:52.643 INFO:tasks.workunit.client.1.smithi084.stdout:8/996: creat d0/d9/d1b/d38/d43/f142 x:0 0 0 2024-02-28T23:07:52.644 INFO:tasks.workunit.client.0.smithi057.stdout:1/743: creat d2/dd4/f111 x:0 0 0 2024-02-28T23:07:52.644 INFO:tasks.workunit.client.0.smithi057.stdout:4/797: link d1a/d48/d9c/d68/dc5/ld2 d1a/d2f/db2/d49/d4d/dc9/dff/l118 0 2024-02-28T23:07:52.645 INFO:tasks.workunit.client.0.smithi057.stdout:0/986: write d17/d19/d4d/f115 [874155,87857] 0 2024-02-28T23:07:52.645 INFO:tasks.workunit.client.0.smithi057.stdout:5/838: creat d4/d9/d14/f103 x:0 0 0 2024-02-28T23:07:52.647 INFO:tasks.workunit.client.1.smithi084.stdout:8/997: dread - d0/d9/d30/d33/d71/d84/f133 zero size 2024-02-28T23:07:52.651 INFO:tasks.workunit.client.0.smithi057.stdout:2/918: getdents d8/d11/d21/d28/d49/d85/d9b 0 2024-02-28T23:07:52.652 INFO:tasks.workunit.client.1.smithi084.stdout:8/998: symlink d0/d44/dfb/l143 0 2024-02-28T23:07:52.652 INFO:tasks.workunit.client.0.smithi057.stdout:1/744: mknod d2/d7/d4f/c112 0 2024-02-28T23:07:52.656 INFO:tasks.workunit.client.0.smithi057.stdout:7/843: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/c117 0 2024-02-28T23:07:52.657 INFO:tasks.workunit.client.1.smithi084.stdout:8/999: creat d0/d9/d30/d33/d71/d84/dce/f144 x:0 0 0 2024-02-28T23:07:52.657 INFO:tasks.workunit.client.1.smithi084.stdout:4/990: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/dd4/da5/fbf [8388608,4194304] 0 2024-02-28T23:07:52.659 INFO:tasks.workunit.client.0.smithi057.stdout:1/745: chown d2/d71/d62/f74 24 1 2024-02-28T23:07:52.660 INFO:tasks.workunit.client.0.smithi057.stdout:7/844: chown d0/d3f/d76/dc1/dea/f100 166 1 2024-02-28T23:07:52.660 INFO:tasks.workunit.client.0.smithi057.stdout:8/923: rename d4/d13/d1a/c21 to d4/d13/c143 0 2024-02-28T23:07:52.663 INFO:tasks.workunit.client.0.smithi057.stdout:0/987: truncate d17/d20/d26/d48/d7c/d98/da6/d153/f160 144224 0 2024-02-28T23:07:52.664 INFO:tasks.workunit.client.0.smithi057.stdout:4/798: unlink d1a/d2f/db2/d49/d4d/l10a 0 2024-02-28T23:07:52.665 INFO:tasks.workunit.client.0.smithi057.stdout:7/845: chown d0/d3f/d76/dc1/dea/d75/fa5 1000755 1 2024-02-28T23:07:52.668 INFO:tasks.workunit.client.0.smithi057.stdout:1/746: mknod d2/d71/def/c113 0 2024-02-28T23:07:52.670 INFO:tasks.workunit.client.0.smithi057.stdout:4/799: readlink d1a/d2f/db2/d49/d4d/la4 0 2024-02-28T23:07:52.670 INFO:tasks.workunit.client.0.smithi057.stdout:2/919: dread d8/f19 [0,4194304] 0 2024-02-28T23:07:52.672 INFO:tasks.workunit.client.0.smithi057.stdout:3/736: link d3/d4/d1d/d47/l92 d3/d4/d7/d9a/d9b/dab/lef 0 2024-02-28T23:07:52.673 INFO:tasks.workunit.client.0.smithi057.stdout:8/924: creat d4/de9/d37/d9d/f144 x:0 0 0 2024-02-28T23:07:52.673 INFO:tasks.workunit.client.0.smithi057.stdout:0/988: truncate d17/d20/d4a/d78/d9d/f15e 795476 0 2024-02-28T23:07:52.674 INFO:tasks.workunit.client.0.smithi057.stdout:4/800: stat d1a/d2f/db2/d49/d4d/dc9/lfb 0 2024-02-28T23:07:52.677 INFO:tasks.workunit.client.1.smithi084.stdout:4/991: write d1/d18/d20/d24/f8b [1894492,38716] 0 2024-02-28T23:07:52.678 INFO:tasks.workunit.client.0.smithi057.stdout:1/747: creat d2/d7/df/db/d3a/f114 x:0 0 0 2024-02-28T23:07:52.680 INFO:tasks.workunit.client.1.smithi084.stdout:4/992: mkdir d1/d18/d20/d14/d30/d47/da4/d10f/d115/d146 0 2024-02-28T23:07:52.680 INFO:tasks.workunit.client.0.smithi057.stdout:3/737: write d3/d20/d2d/f3b [2386377,107319] 0 2024-02-28T23:07:52.682 INFO:tasks.workunit.client.0.smithi057.stdout:3/738: chown d3/d20/f23 13 1 2024-02-28T23:07:52.683 INFO:tasks.workunit.client.0.smithi057.stdout:2/920: creat d8/d11/d21/d28/d49/d85/d9b/f126 x:0 0 0 2024-02-28T23:07:52.686 INFO:tasks.workunit.client.1.smithi084.stdout:4/993: truncate d1/d18/d20/d24/d2f/d3e/d52/f9a 8576352 0 2024-02-28T23:07:52.687 INFO:tasks.workunit.client.0.smithi057.stdout:8/925: creat d4/d13/d18/f145 x:0 0 0 2024-02-28T23:07:52.689 INFO:tasks.workunit.client.0.smithi057.stdout:1/748: rmdir d2/d7/df/db 39 2024-02-28T23:07:52.691 INFO:tasks.workunit.client.0.smithi057.stdout:2/921: creat d8/d11/d21/d28/dec/f127 x:0 0 0 2024-02-28T23:07:52.693 INFO:tasks.workunit.client.1.smithi084.stdout:4/994: write d1/d18/d20/d14/f39 [4306020,29141] 0 2024-02-28T23:07:52.694 INFO:tasks.workunit.client.0.smithi057.stdout:3/739: mknod d3/d4/d1d/cf0 0 2024-02-28T23:07:52.695 INFO:tasks.workunit.client.0.smithi057.stdout:5/839: truncate d4/d9/d14/f5e 1483438 0 2024-02-28T23:07:52.697 INFO:tasks.workunit.client.0.smithi057.stdout:8/926: mkdir d4/d13/d18/d62/d66/d74/d8b/d146 0 2024-02-28T23:07:52.697 INFO:tasks.workunit.client.1.smithi084.stdout:4/995: creat d1/d18/d20/d14/d30/d47/da4/d10f/d115/f147 x:0 0 0 2024-02-28T23:07:52.701 INFO:tasks.workunit.client.0.smithi057.stdout:3/740: truncate d3/d4/d1d/f39 3017837 0 2024-02-28T23:07:52.702 INFO:tasks.workunit.client.1.smithi084.stdout:4/996: creat d1/d18/d20/f148 x:0 0 0 2024-02-28T23:07:52.704 INFO:tasks.workunit.client.0.smithi057.stdout:8/927: rename d4/de9/d56/dfd/f133 to d4/d13/d18/d62/d66/f147 0 2024-02-28T23:07:52.709 INFO:tasks.workunit.client.0.smithi057.stdout:3/741: creat d3/d4/de8/ff1 x:0 0 0 2024-02-28T23:07:52.715 INFO:tasks.workunit.client.0.smithi057.stdout:0/989: dwrite d17/f83 [0,4194304] 0 2024-02-28T23:07:52.717 INFO:tasks.workunit.client.1.smithi084.stdout:2/947: sync 2024-02-28T23:07:52.721 INFO:tasks.workunit.client.1.smithi084.stdout:4/997: dwrite d1/d18/d20/d24/d2f/d3e/d52/d45/db5/fd2 [0,4194304] 0 2024-02-28T23:07:52.721 INFO:tasks.workunit.client.1.smithi084.stdout:2/948: fdatasync d5/d7/d12/f51 0 2024-02-28T23:07:52.725 INFO:tasks.workunit.client.0.smithi057.stdout:4/801: dwrite d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:52.726 INFO:tasks.workunit.client.0.smithi057.stdout:5/840: truncate d4/fe6 53274 0 2024-02-28T23:07:52.728 INFO:tasks.workunit.client.0.smithi057.stdout:3/742: mknod d3/d8/d27/cf2 0 2024-02-28T23:07:52.728 INFO:tasks.workunit.client.0.smithi057.stdout:1/749: dwrite d2/d7/df/db/d3a/d2a/d82/fe4 [4194304,4194304] 0 2024-02-28T23:07:52.728 INFO:tasks.workunit.client.0.smithi057.stdout:2/922: creat d8/d11/d21/d28/d49/d85/d9b/f128 x:0 0 0 2024-02-28T23:07:52.728 INFO:tasks.workunit.client.1.smithi084.stdout:4/998: getdents d1/d18/d20/d14/d121/d122 0 2024-02-28T23:07:52.729 INFO:tasks.workunit.client.0.smithi057.stdout:7/846: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f46 [0,4194304] 0 2024-02-28T23:07:52.730 INFO:tasks.workunit.client.0.smithi057.stdout:2/923: chown d8 94243 1 2024-02-28T23:07:52.731 INFO:tasks.workunit.client.0.smithi057.stdout:5/841: symlink d4/d9/d14/d63/d88/l104 0 2024-02-28T23:07:52.733 INFO:tasks.workunit.client.0.smithi057.stdout:4/802: chown d1a/d48/f79 15835 1 2024-02-28T23:07:52.735 INFO:tasks.workunit.client.1.smithi084.stdout:4/999: rmdir d1/d18/d20/d24/d2f/d3e 39 2024-02-28T23:07:52.736 INFO:tasks.workunit.client.0.smithi057.stdout:8/928: dwrite d4/de9/d56/d6a/f4d [4194304,4194304] 0 2024-02-28T23:07:52.736 INFO:tasks.workunit.client.0.smithi057.stdout:0/990: creat d17/d20/d4a/d78/f164 x:0 0 0 2024-02-28T23:07:52.737 INFO:tasks.workunit.client.0.smithi057.stdout:1/750: readlink d2/d7/df/db/d3a/l60 0 2024-02-28T23:07:52.745 INFO:tasks.workunit.client.1.smithi084.stdout:2/949: dwrite d5/d7/d9/d94/dbd/f108 [0,4194304] 0 2024-02-28T23:07:52.749 INFO:tasks.workunit.client.0.smithi057.stdout:8/929: symlink d4/d13/d2c/db1/l148 0 2024-02-28T23:07:52.752 INFO:tasks.workunit.client.0.smithi057.stdout:5/842: read d4/d9/d16/d52/dfd/fa7 [51024,99700] 0 2024-02-28T23:07:52.763 INFO:tasks.workunit.client.0.smithi057.stdout:0/991: creat d17/d20/d23/f165 x:0 0 0 2024-02-28T23:07:52.766 INFO:tasks.workunit.client.0.smithi057.stdout:5/843: fsync d4/d9/d16/d52/dfd/fa7 0 2024-02-28T23:07:52.766 INFO:tasks.workunit.client.0.smithi057.stdout:4/803: link d1a/d1e/d51/dc3/lea d1a/d2f/db2/d49/d4d/l119 0 2024-02-28T23:07:52.776 INFO:tasks.workunit.client.0.smithi057.stdout:3/743: dwrite d3/d4/f7a [0,4194304] 0 2024-02-28T23:07:52.777 INFO:tasks.workunit.client.0.smithi057.stdout:5/844: rename d4/d9/f1c to d4/d9/d16/d52/dfd/da1/f105 0 2024-02-28T23:07:52.782 INFO:tasks.workunit.client.0.smithi057.stdout:5/845: write d4/d7/d58/f101 [937956,89542] 0 2024-02-28T23:07:52.790 INFO:tasks.workunit.client.0.smithi057.stdout:8/930: dwrite d4/d13/d1a/d4a/db3/dfc/fa0 [4194304,4194304] 0 2024-02-28T23:07:52.798 INFO:tasks.workunit.client.0.smithi057.stdout:2/924: write d8/d11/d21/d28/d41/f9e [1268135,1278] 0 2024-02-28T23:07:52.808 INFO:tasks.workunit.client.0.smithi057.stdout:0/992: write d17/d20/d26/f52 [465868,129453] 0 2024-02-28T23:07:52.811 INFO:tasks.workunit.client.0.smithi057.stdout:0/993: chown d17/d20/db6/f151 1462101 1 2024-02-28T23:07:52.813 INFO:tasks.workunit.client.0.smithi057.stdout:5/846: symlink d4/d9/d14/df2/l106 0 2024-02-28T23:07:52.815 INFO:tasks.workunit.client.0.smithi057.stdout:2/925: dread d8/d11/d39/d83/fd1 [0,4194304] 0 2024-02-28T23:07:52.819 INFO:tasks.workunit.client.0.smithi057.stdout:8/931: dread d4/d13/d1a/f58 [0,4194304] 0 2024-02-28T23:07:52.822 INFO:tasks.workunit.client.1.smithi084.stdout:2/950: dwrite d5/d7/d12/d24/d3a/fc6 [0,4194304] 0 2024-02-28T23:07:52.823 INFO:tasks.workunit.client.0.smithi057.stdout:2/926: stat d8/d11/d15/f78 0 2024-02-28T23:07:52.824 INFO:tasks.workunit.client.0.smithi057.stdout:3/744: dwrite d3/d4/d7/f6a [4194304,4194304] 0 2024-02-28T23:07:52.825 INFO:tasks.workunit.client.0.smithi057.stdout:7/847: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/f28 [0,4194304] 0 2024-02-28T23:07:52.825 INFO:tasks.workunit.client.0.smithi057.stdout:1/751: dwrite d2/d71/d2d/f3d [4194304,4194304] 0 2024-02-28T23:07:52.825 INFO:tasks.workunit.client.1.smithi084.stdout:2/951: mkdir d5/d7/d9/d7c/d5b/da6/d123 0 2024-02-28T23:07:52.827 INFO:tasks.workunit.client.1.smithi084.stdout:2/952: fsync d5/d7/d75/db4/ff9 0 2024-02-28T23:07:52.828 INFO:tasks.workunit.client.0.smithi057.stdout:5/847: write d4/d32/df1/f76 [2293,97599] 0 2024-02-28T23:07:52.829 INFO:tasks.workunit.client.1.smithi084.stdout:2/953: chown d5/d7/d2e/d4a/d5c/fff 0 1 2024-02-28T23:07:52.829 INFO:tasks.workunit.client.0.smithi057.stdout:0/994: dread d17/d20/d26/f69 [0,4194304] 0 2024-02-28T23:07:52.831 INFO:tasks.workunit.client.1.smithi084.stdout:2/954: write d5/d7/d75/dcd/ff3 [180405,14001] 0 2024-02-28T23:07:52.837 INFO:tasks.workunit.client.0.smithi057.stdout:3/745: unlink d3/c18 0 2024-02-28T23:07:52.841 INFO:tasks.workunit.client.0.smithi057.stdout:7/848: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/fb2 [1278406,17104] 0 2024-02-28T23:07:52.841 INFO:tasks.workunit.client.0.smithi057.stdout:1/752: rmdir d2/dd4/dd5 39 2024-02-28T23:07:52.841 INFO:tasks.workunit.client.0.smithi057.stdout:4/804: dread d1a/d48/d97/f2c [0,4194304] 0 2024-02-28T23:07:52.845 INFO:tasks.workunit.client.0.smithi057.stdout:0/995: fsync d17/d20/d23/fcd 0 2024-02-28T23:07:52.853 INFO:tasks.workunit.client.1.smithi084.stdout:2/955: dread d5/d7/fc8 [0,4194304] 0 2024-02-28T23:07:52.854 INFO:tasks.workunit.client.0.smithi057.stdout:4/805: fdatasync d1a/d48/d9c/d68/f10e 0 2024-02-28T23:07:52.855 INFO:tasks.workunit.client.1.smithi084.stdout:2/956: write d5/d7/d75/db4/ff9 [195480,116054] 0 2024-02-28T23:07:52.857 INFO:tasks.workunit.client.0.smithi057.stdout:0/996: mknod d17/d20/d4a/d78/db5/c166 0 2024-02-28T23:07:52.858 INFO:tasks.workunit.client.0.smithi057.stdout:7/849: dread - d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/fdc zero size 2024-02-28T23:07:52.858 INFO:tasks.workunit.client.1.smithi084.stdout:2/957: fdatasync d5/d7/d9/f88 0 2024-02-28T23:07:52.859 INFO:tasks.workunit.client.0.smithi057.stdout:3/746: dread d3/d8/d1b/f4a [0,4194304] 0 2024-02-28T23:07:52.863 INFO:tasks.workunit.client.0.smithi057.stdout:3/747: fsync d3/d4/f7a 0 2024-02-28T23:07:52.863 INFO:tasks.workunit.client.1.smithi084.stdout:2/958: rename d5/d7/d12/d24/d3a/d48/l9c to d5/d1f/d42/l124 0 2024-02-28T23:07:52.866 INFO:tasks.workunit.client.1.smithi084.stdout:2/959: write d5/d7/d9/d7c/d5b/d9f/fa0 [2347586,64897] 0 2024-02-28T23:07:52.868 INFO:tasks.workunit.client.0.smithi057.stdout:4/806: rename d1a/d2f/db2/dcb/fd9 to d1a/d48/d9c/d2e/df8/f11a 0 2024-02-28T23:07:52.868 INFO:tasks.workunit.client.0.smithi057.stdout:1/753: dwrite d2/d71/d2d/f3d [4194304,4194304] 0 2024-02-28T23:07:52.868 INFO:tasks.workunit.client.0.smithi057.stdout:0/997: symlink d17/d20/d26/d48/d7c/d98/l167 0 2024-02-28T23:07:52.869 INFO:tasks.workunit.client.1.smithi084.stdout:2/960: creat d5/d7/d9/d7c/f125 x:0 0 0 2024-02-28T23:07:52.871 INFO:tasks.workunit.client.0.smithi057.stdout:8/932: write d4/d13/d2c/d4e/f11b [988894,110880] 0 2024-02-28T23:07:52.871 INFO:tasks.workunit.client.1.smithi084.stdout:2/961: creat d5/dd/d69/d11a/f126 x:0 0 0 2024-02-28T23:07:52.874 INFO:tasks.workunit.client.1.smithi084.stdout:2/962: symlink d5/d1f/d83/l127 0 2024-02-28T23:07:52.875 INFO:tasks.workunit.client.0.smithi057.stdout:9/793: sync 2024-02-28T23:07:52.875 INFO:tasks.workunit.client.0.smithi057.stdout:6/763: sync 2024-02-28T23:07:52.876 INFO:tasks.workunit.client.0.smithi057.stdout:1/754: read d2/d71/d62/fa6 [1366252,72190] 0 2024-02-28T23:07:52.876 INFO:tasks.workunit.client.0.smithi057.stdout:7/850: read d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f90 [83399,123862] 0 2024-02-28T23:07:52.876 INFO:tasks.workunit.client.0.smithi057.stdout:2/927: truncate d8/d11/d39/d83/dde/fcc 3140128 0 2024-02-28T23:07:52.879 INFO:tasks.workunit.client.0.smithi057.stdout:5/848: dwrite d4/d9/fa6 [4194304,4194304] 0 2024-02-28T23:07:52.880 INFO:tasks.workunit.client.0.smithi057.stdout:8/933: mknod d4/de9/d56/dc9/c149 0 2024-02-28T23:07:52.881 INFO:tasks.workunit.client.0.smithi057.stdout:1/755: write d2/fe9 [275175,67808] 0 2024-02-28T23:07:52.883 INFO:tasks.workunit.client.0.smithi057.stdout:4/807: truncate d1a/d2f/db2/f3c 4970497 0 2024-02-28T23:07:52.894 INFO:tasks.workunit.client.0.smithi057.stdout:7/851: readlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/lff 0 2024-02-28T23:07:52.894 INFO:tasks.workunit.client.0.smithi057.stdout:5/849: readlink d4/d9/d16/d52/dfd/lc2 0 2024-02-28T23:07:52.895 INFO:tasks.workunit.client.0.smithi057.stdout:2/928: dread d8/d11/d15/f78 [0,4194304] 0 2024-02-28T23:07:52.896 INFO:tasks.workunit.client.0.smithi057.stdout:0/998: dwrite d17/d20/d4a/d78/f110 [0,4194304] 0 2024-02-28T23:07:52.897 INFO:tasks.workunit.client.1.smithi084.stdout:2/963: dwrite d5/d7/d12/d24/d2a/d35/f5d [4194304,4194304] 0 2024-02-28T23:07:52.899 INFO:tasks.workunit.client.0.smithi057.stdout:2/929: dread - d8/d11/d21/d28/dec/f127 zero size 2024-02-28T23:07:52.900 INFO:tasks.workunit.client.0.smithi057.stdout:0/999: chown d17/d20/d26/d48/d7c/d98/fcf 3752 1 2024-02-28T23:07:52.902 INFO:tasks.workunit.client.1.smithi084.stdout:2/964: stat d5/d7/d9/d7c/d5b/d9f/lb5 0 2024-02-28T23:07:52.920 INFO:tasks.workunit.client.1.smithi084.stdout:2/965: write d5/d7/d9/d7c/d5b/f7e [3446902,44119] 0 2024-02-28T23:07:52.927 INFO:tasks.workunit.client.1.smithi084.stdout:2/966: rename d5/d7/d9/d94/f9e to d5/d7/d9/d94/f128 0 2024-02-28T23:07:52.929 INFO:tasks.workunit.client.1.smithi084.stdout:2/967: chown d5/d7/d9/d7c/d5b 60544 1 2024-02-28T23:07:52.933 INFO:tasks.workunit.client.1.smithi084.stdout:2/968: fdatasync d5/d7/d9/d94/f128 0 2024-02-28T23:07:52.935 INFO:tasks.workunit.client.1.smithi084.stdout:2/969: fsync d5/d1f/d42/d8f/f10f 0 2024-02-28T23:07:52.936 INFO:tasks.workunit.client.0.smithi057.stdout:3/748: truncate d3/d8/f29 308338 0 2024-02-28T23:07:52.937 INFO:tasks.workunit.client.1.smithi084.stdout:2/970: chown d5/d7/d9/d7c/d5b/d9f 222577 1 2024-02-28T23:07:52.938 INFO:tasks.workunit.client.0.smithi057.stdout:6/764: dread d0/d1b/fd5 [0,4194304] 0 2024-02-28T23:07:52.938 INFO:tasks.workunit.client.0.smithi057.stdout:3/749: write d3/d4/d7/d9a/d9b/dab/dd0/fe7 [748503,61091] 0 2024-02-28T23:07:52.940 INFO:tasks.workunit.client.0.smithi057.stdout:8/934: creat d4/de9/d56/dfd/f14a x:0 0 0 2024-02-28T23:07:52.941 INFO:tasks.workunit.client.1.smithi084.stdout:2/971: rmdir d5/d7/d9/d94 39 2024-02-28T23:07:52.951 INFO:tasks.workunit.client.0.smithi057.stdout:6/765: unlink d0/d1b/d3e/d40/cf7 0 2024-02-28T23:07:52.956 INFO:tasks.workunit.client.0.smithi057.stdout:6/766: mkdir d0/d4/d25/d60/d6a/dff 0 2024-02-28T23:07:52.958 INFO:tasks.workunit.client.0.smithi057.stdout:8/935: link d4/l114 d4/de9/d59/l14b 0 2024-02-28T23:07:52.961 INFO:tasks.workunit.client.1.smithi084.stdout:2/972: dwrite d5/d7/d2e/d4a/d5c/f117 [0,4194304] 0 2024-02-28T23:07:52.972 INFO:tasks.workunit.client.0.smithi057.stdout:4/808: creat d1a/d48/d9c/d2e/d8f/f11b x:0 0 0 2024-02-28T23:07:52.974 INFO:tasks.workunit.client.0.smithi057.stdout:4/809: chown d1a/d2f/db2/f7b 0 1 2024-02-28T23:07:52.983 INFO:tasks.workunit.client.0.smithi057.stdout:4/810: getdents d1a/d48/d9c/d2e/df8 0 2024-02-28T23:07:52.998 INFO:tasks.workunit.client.0.smithi057.stdout:3/750: write d3/d4f/d4c/d91/fc0 [2309319,31105] 0 2024-02-28T23:07:53.004 INFO:tasks.workunit.client.0.smithi057.stdout:9/794: rename dd/de/d24/d63/d77/dcf/d40/d44/d92/fa8 to dd/de/d24/d63/d77/dcf/d40/ffb 0 2024-02-28T23:07:53.010 INFO:tasks.workunit.client.0.smithi057.stdout:5/850: rename d4/d9/d16/d8f to d4/d32/d107 0 2024-02-28T23:07:53.011 INFO:tasks.workunit.client.0.smithi057.stdout:9/795: fsync dd/de/d4a/d60/f75 0 2024-02-28T23:07:53.013 INFO:tasks.workunit.client.0.smithi057.stdout:3/751: dwrite d3/d4/d7/f10 [8388608,4194304] 0 2024-02-28T23:07:53.016 INFO:tasks.workunit.client.0.smithi057.stdout:8/936: rename d4/d13/d1a/d4a/db3/dfc/f138 to d4/d13/d7f/d128/f14c 0 2024-02-28T23:07:53.017 INFO:tasks.workunit.client.0.smithi057.stdout:6/767: dread d0/fe7 [0,4194304] 0 2024-02-28T23:07:53.022 INFO:tasks.workunit.client.0.smithi057.stdout:3/752: rename d3/d8/d1b/c7b to d3/d4/d1d/d47/cf3 0 2024-02-28T23:07:53.025 INFO:tasks.workunit.client.0.smithi057.stdout:6/768: mkdir d0/d1b/d3e/d100 0 2024-02-28T23:07:53.027 INFO:tasks.workunit.client.0.smithi057.stdout:3/753: mknod d3/d20/cf4 0 2024-02-28T23:07:53.027 INFO:tasks.workunit.client.0.smithi057.stdout:6/769: chown d0/d4/d11/ld4 15931187 1 2024-02-28T23:07:53.029 INFO:tasks.workunit.client.1.smithi084.stdout:2/973: readlink d5/d7/d9/d94/l102 0 2024-02-28T23:07:53.031 INFO:tasks.workunit.client.0.smithi057.stdout:6/770: creat d0/d17/d94/f101 x:0 0 0 2024-02-28T23:07:53.033 INFO:tasks.workunit.client.0.smithi057.stdout:6/771: chown d0/d4 0 1 2024-02-28T23:07:53.035 INFO:tasks.workunit.client.0.smithi057.stdout:3/754: creat d3/ff5 x:0 0 0 2024-02-28T23:07:53.035 INFO:tasks.workunit.client.0.smithi057.stdout:8/937: dwrite d4/de9/d59/f98 [0,4194304] 0 2024-02-28T23:07:53.037 INFO:tasks.workunit.client.0.smithi057.stdout:4/811: truncate d1a/d48/f79 3210176 0 2024-02-28T23:07:53.037 INFO:tasks.workunit.client.0.smithi057.stdout:3/755: write d3/ff5 [609543,82860] 0 2024-02-28T23:07:53.038 INFO:tasks.workunit.client.0.smithi057.stdout:2/930: mknod d8/d11/d39/d83/c129 0 2024-02-28T23:07:53.044 INFO:tasks.workunit.client.0.smithi057.stdout:3/756: unlink d3/c4b 0 2024-02-28T23:07:53.046 INFO:tasks.workunit.client.0.smithi057.stdout:8/938: symlink d4/d13/d1a/l14d 0 2024-02-28T23:07:53.047 INFO:tasks.workunit.client.0.smithi057.stdout:3/757: read d3/d8/d27/f5f [2560733,127383] 0 2024-02-28T23:07:53.048 INFO:tasks.workunit.client.0.smithi057.stdout:3/758: fsync d3/d4f/d4c/fcd 0 2024-02-28T23:07:53.050 INFO:tasks.workunit.client.0.smithi057.stdout:6/772: rename d0/d1b/d3e/d3d/d42/d4f/dd0 to d0/d17/d94/da0/de6/df4/d102 0 2024-02-28T23:07:53.050 INFO:tasks.workunit.client.0.smithi057.stdout:8/939: write d4/d13/d18/da7/f101 [555130,26931] 0 2024-02-28T23:07:53.058 INFO:tasks.workunit.client.0.smithi057.stdout:6/773: creat d0/d1b/f103 x:0 0 0 2024-02-28T23:07:53.060 INFO:tasks.workunit.client.0.smithi057.stdout:7/852: mkdir d0/d3f/d76/d118 0 2024-02-28T23:07:53.060 INFO:tasks.workunit.client.0.smithi057.stdout:6/774: chown d0/d4/d11/f9a 837961721 1 2024-02-28T23:07:53.067 INFO:tasks.workunit.client.0.smithi057.stdout:9/796: write dd/d46/d6f/f79 [1898627,16373] 0 2024-02-28T23:07:53.071 INFO:tasks.workunit.client.0.smithi057.stdout:3/759: getdents d3/d8 0 2024-02-28T23:07:53.072 INFO:tasks.workunit.client.0.smithi057.stdout:2/931: dwrite d8/f19 [0,4194304] 0 2024-02-28T23:07:53.073 INFO:tasks.workunit.client.0.smithi057.stdout:8/940: dwrite d4/d13/d18/d62/d66/fd8 [0,4194304] 0 2024-02-28T23:07:53.073 INFO:tasks.workunit.client.0.smithi057.stdout:7/853: dread d0/d3f/d76/dc1/dea/f9e [0,4194304] 0 2024-02-28T23:07:53.077 INFO:tasks.workunit.client.0.smithi057.stdout:3/760: fsync d3/d4/fca 0 2024-02-28T23:07:53.079 INFO:tasks.workunit.client.0.smithi057.stdout:4/812: write d1a/d48/d97/d7f/fcc [399139,63963] 0 2024-02-28T23:07:53.082 INFO:tasks.workunit.client.0.smithi057.stdout:2/932: write d8/d11/d21/d28/d49/d85/d9b/f126 [1036704,64301] 0 2024-02-28T23:07:53.086 INFO:tasks.workunit.client.0.smithi057.stdout:9/797: truncate dd/de/f43 580749 0 2024-02-28T23:07:53.087 INFO:tasks.workunit.client.0.smithi057.stdout:3/761: chown d3/d4f/d4c/d91/f9f 51504420 1 2024-02-28T23:07:53.090 INFO:tasks.workunit.client.0.smithi057.stdout:5/851: dwrite d4/d9/d16/f40 [0,4194304] 0 2024-02-28T23:07:53.093 INFO:tasks.workunit.client.0.smithi057.stdout:8/941: truncate d4/d13/d1a/d4a/fd4 165499 0 2024-02-28T23:07:53.093 INFO:tasks.workunit.client.0.smithi057.stdout:2/933: rmdir d8/d11/d39/d83 39 2024-02-28T23:07:53.096 INFO:tasks.workunit.client.0.smithi057.stdout:6/775: write d0/d4/fb4 [696517,25937] 0 2024-02-28T23:07:53.097 INFO:tasks.workunit.client.0.smithi057.stdout:5/852: readlink d4/d32/df1/d24/da2/dbb/d95/l9d 0 2024-02-28T23:07:53.100 INFO:tasks.workunit.client.0.smithi057.stdout:9/798: getdents dd/de/d24/d63/d77/dcf/d40/d44/da5/de4 0 2024-02-28T23:07:53.103 INFO:tasks.workunit.client.0.smithi057.stdout:8/942: rename d4/f71 to d4/d13/d1a/d4a/db3/f14e 0 2024-02-28T23:07:53.110 INFO:tasks.workunit.client.0.smithi057.stdout:7/854: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f90 [25888,5711] 0 2024-02-28T23:07:53.114 INFO:tasks.workunit.client.0.smithi057.stdout:2/934: truncate d8/d11/d21/d28/f75 2584648 0 2024-02-28T23:07:53.115 INFO:tasks.workunit.client.0.smithi057.stdout:8/943: creat d4/de9/d56/d88/f14f x:0 0 0 2024-02-28T23:07:53.116 INFO:tasks.workunit.client.0.smithi057.stdout:5/853: getdents d4/d9/d14/d3b/df3 0 2024-02-28T23:07:53.116 INFO:tasks.workunit.client.0.smithi057.stdout:6/776: dwrite d0/d17/d20/fe9 [0,4194304] 0 2024-02-28T23:07:53.117 INFO:tasks.workunit.client.0.smithi057.stdout:4/813: write d1a/d48/d9c/d2e/fa3 [560928,5879] 0 2024-02-28T23:07:53.120 INFO:tasks.workunit.client.0.smithi057.stdout:9/799: write dd/d46/fc0 [4054718,126360] 0 2024-02-28T23:07:53.120 INFO:tasks.workunit.client.0.smithi057.stdout:7/855: fdatasync d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fa8 0 2024-02-28T23:07:53.126 INFO:tasks.workunit.client.0.smithi057.stdout:6/777: creat d0/d1b/d3e/f104 x:0 0 0 2024-02-28T23:07:53.127 INFO:tasks.workunit.client.0.smithi057.stdout:4/814: read - d1a/d48/d9c/d2e/d8f/db7/f10d zero size 2024-02-28T23:07:53.129 INFO:tasks.workunit.client.0.smithi057.stdout:8/944: mknod d4/d13/d1a/d4a/db3/c150 0 2024-02-28T23:07:53.135 INFO:tasks.workunit.client.0.smithi057.stdout:9/800: symlink dd/de/d14/d48/d4b/d41/lfc 0 2024-02-28T23:07:53.137 INFO:tasks.workunit.client.0.smithi057.stdout:2/935: truncate d8/dae/fb8 3724590 0 2024-02-28T23:07:53.138 INFO:tasks.workunit.client.0.smithi057.stdout:6/778: truncate d0/f4a 5117826 0 2024-02-28T23:07:53.139 INFO:tasks.workunit.client.0.smithi057.stdout:9/801: read - dd/de/d24/d72/f7c zero size 2024-02-28T23:07:53.140 INFO:tasks.workunit.client.0.smithi057.stdout:8/945: creat d4/de9/d56/da6/dae/f151 x:0 0 0 2024-02-28T23:07:53.144 INFO:tasks.workunit.client.0.smithi057.stdout:5/854: dwrite d4/d9/d16/d4e/fed [0,4194304] 0 2024-02-28T23:07:53.146 INFO:tasks.workunit.client.0.smithi057.stdout:4/815: getdents d1a/d1e/d51 0 2024-02-28T23:07:53.147 INFO:tasks.workunit.client.0.smithi057.stdout:8/946: mkdir d4/d13/df0/d104/d152 0 2024-02-28T23:07:53.151 INFO:tasks.workunit.client.0.smithi057.stdout:2/936: rename d8/d11/d39/f87 to d8/d11/d15/daf/f12a 0 2024-02-28T23:07:53.152 INFO:tasks.workunit.client.0.smithi057.stdout:8/947: chown d4/de9/d56/dfd/f14a 15700517 1 2024-02-28T23:07:53.152 INFO:tasks.workunit.client.0.smithi057.stdout:7/856: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f32 [0,4194304] 0 2024-02-28T23:07:53.154 INFO:tasks.workunit.client.0.smithi057.stdout:8/948: fdatasync d4/f136 0 2024-02-28T23:07:53.158 INFO:tasks.workunit.client.0.smithi057.stdout:4/816: rmdir d1a/d2f/db2/d49/d4d 39 2024-02-28T23:07:53.159 INFO:tasks.workunit.client.0.smithi057.stdout:9/802: creat dd/de/d24/d63/d77/df2/ffd x:0 0 0 2024-02-28T23:07:53.160 INFO:tasks.workunit.client.0.smithi057.stdout:5/855: rename d4/d32/df1/d24/da2/dbb/f93 to d4/d7/d58/f108 0 2024-02-28T23:07:53.162 INFO:tasks.workunit.client.0.smithi057.stdout:6/779: read d0/d1b/d3e/d3d/d42/d4f/d53/f5b [57472,46940] 0 2024-02-28T23:07:53.163 INFO:tasks.workunit.client.0.smithi057.stdout:5/856: chown d4/d9/d14/d3b/f84 7 1 2024-02-28T23:07:53.165 INFO:tasks.workunit.client.0.smithi057.stdout:4/817: stat d1a/d2f/db2/d49/d4d/de2/d6e 0 2024-02-28T23:07:53.167 INFO:tasks.workunit.client.0.smithi057.stdout:4/818: stat d1a/d48/db0/db1/cd3 0 2024-02-28T23:07:53.172 INFO:tasks.workunit.client.0.smithi057.stdout:5/857: mknod d4/d32/df1/d24/da2/dbb/c109 0 2024-02-28T23:07:53.173 INFO:tasks.workunit.client.0.smithi057.stdout:7/857: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d115 to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119 0 2024-02-28T23:07:53.175 INFO:tasks.workunit.client.0.smithi057.stdout:4/819: creat d1a/d48/db0/f11c x:0 0 0 2024-02-28T23:07:53.177 INFO:tasks.workunit.client.0.smithi057.stdout:8/949: link d4/de9/d56/d88/lc2 d4/d13/d18/da7/de1/dee/d118/l153 0 2024-02-28T23:07:53.179 INFO:tasks.workunit.client.0.smithi057.stdout:9/803: write dd/fd7 [2683832,111072] 0 2024-02-28T23:07:53.180 INFO:tasks.workunit.client.0.smithi057.stdout:5/858: fsync d4/f6 0 2024-02-28T23:07:53.185 INFO:tasks.workunit.client.0.smithi057.stdout:7/858: read d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/f8b [4007463,49640] 0 2024-02-28T23:07:53.189 INFO:tasks.workunit.client.0.smithi057.stdout:6/780: dwrite d0/d4/f50 [0,4194304] 0 2024-02-28T23:07:53.189 INFO:tasks.workunit.client.0.smithi057.stdout:4/820: link d1a/d48/cbb d1a/d48/d9c/d68/c11d 0 2024-02-28T23:07:53.192 INFO:tasks.workunit.client.0.smithi057.stdout:9/804: link dd/de/d14/d48/d4b/c97 dd/de/d4a/d9f/cfe 0 2024-02-28T23:07:53.197 INFO:tasks.workunit.client.0.smithi057.stdout:8/950: dwrite d4/d13/d18/f65 [0,4194304] 0 2024-02-28T23:07:53.206 INFO:tasks.workunit.client.0.smithi057.stdout:9/805: fdatasync dd/de/d24/d63/d77/dcf/d40/d44/f62 0 2024-02-28T23:07:53.212 INFO:tasks.workunit.client.0.smithi057.stdout:9/806: truncate dd/de/d14/d48/d4b/d41/ff5 308446 0 2024-02-28T23:07:53.213 INFO:tasks.workunit.client.0.smithi057.stdout:9/807: stat dd/d46/d4d/c8b 0 2024-02-28T23:07:53.214 INFO:tasks.workunit.client.0.smithi057.stdout:4/821: write d1a/d2f/db2/d49/d4d/de2/fb3 [892039,55766] 0 2024-02-28T23:07:53.221 INFO:tasks.workunit.client.1.smithi084.stdout:2/974: sync 2024-02-28T23:07:53.224 INFO:tasks.workunit.client.1.smithi084.stdout:2/975: write d5/d7/d2e/fc0 [5609803,93519] 0 2024-02-28T23:07:53.226 INFO:tasks.workunit.client.0.smithi057.stdout:8/951: dwrite d4/df7/f11e [0,4194304] 0 2024-02-28T23:07:53.227 INFO:tasks.workunit.client.1.smithi084.stdout:2/976: write d5/d7/d12/d24/d3a/fc6 [2075911,128845] 0 2024-02-28T23:07:53.227 INFO:tasks.workunit.client.0.smithi057.stdout:1/756: sync 2024-02-28T23:07:53.232 INFO:tasks.workunit.client.0.smithi057.stdout:4/822: symlink d1a/d48/d9c/d2e/l11e 0 2024-02-28T23:07:53.234 INFO:tasks.workunit.client.0.smithi057.stdout:1/757: stat d2/d7/df/db/d3a/l107 0 2024-02-28T23:07:53.235 INFO:tasks.workunit.client.0.smithi057.stdout:8/952: creat d4/d13/d18/d62/de4/f154 x:0 0 0 2024-02-28T23:07:53.238 INFO:tasks.workunit.client.0.smithi057.stdout:9/808: getdents dd/de/d14/d48/d76 0 2024-02-28T23:07:53.239 INFO:tasks.workunit.client.0.smithi057.stdout:9/809: chown dd/de/d14/f1e 180051 1 2024-02-28T23:07:53.245 INFO:tasks.workunit.client.0.smithi057.stdout:4/823: getdents d1a/d2f/db2/d49/d4d/de2 0 2024-02-28T23:07:53.247 INFO:tasks.workunit.client.0.smithi057.stdout:4/824: fdatasync d1a/d48/d9c/d2e/f98 0 2024-02-28T23:07:53.255 INFO:tasks.workunit.client.0.smithi057.stdout:8/953: dwrite d4/de9/d56/d88/f126 [0,4194304] 0 2024-02-28T23:07:53.260 INFO:tasks.workunit.client.0.smithi057.stdout:9/810: dwrite dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:53.271 INFO:tasks.workunit.client.0.smithi057.stdout:9/811: chown dd/de/d24/d63/d77/dcf/d40/dd0 13321 1 2024-02-28T23:07:53.272 INFO:tasks.workunit.client.0.smithi057.stdout:8/954: dread d4/d13/d18/d62/dbb/f86 [0,4194304] 0 2024-02-28T23:07:53.274 INFO:tasks.workunit.client.0.smithi057.stdout:4/825: dwrite d1a/d2f/db2/dcb/f105 [0,4194304] 0 2024-02-28T23:07:53.278 INFO:tasks.workunit.client.0.smithi057.stdout:1/758: dwrite d2/d7/df/db/d3a/d5a/fcb [0,4194304] 0 2024-02-28T23:07:53.282 INFO:tasks.workunit.client.0.smithi057.stdout:8/955: symlink d4/de9/d56/dc9/l155 0 2024-02-28T23:07:53.283 INFO:tasks.workunit.client.0.smithi057.stdout:1/759: mknod d2/d7/df/db/d12/d19/d48/c115 0 2024-02-28T23:07:53.285 INFO:tasks.workunit.client.0.smithi057.stdout:4/826: rename d1a/c33 to d1a/d2f/db2/d49/d4d/c11f 0 2024-02-28T23:07:53.288 INFO:tasks.workunit.client.0.smithi057.stdout:8/956: mknod d4/d13/df0/d104/d152/c156 0 2024-02-28T23:07:53.288 INFO:tasks.workunit.client.0.smithi057.stdout:1/760: chown d2/d7/df/db/d3a/d8f/l108 6288 1 2024-02-28T23:07:53.296 INFO:tasks.workunit.client.0.smithi057.stdout:1/761: unlink d2/d7/df/db/d12/d19/da4/dc5/ldd 0 2024-02-28T23:07:53.296 INFO:tasks.workunit.client.0.smithi057.stdout:8/957: truncate d4/d13/d18/d62/d66/f99 206733 0 2024-02-28T23:07:53.308 INFO:tasks.workunit.client.0.smithi057.stdout:8/958: creat d4/d13/d1a/d4a/db3/f157 x:0 0 0 2024-02-28T23:07:53.312 INFO:tasks.workunit.client.0.smithi057.stdout:4/827: dwrite d1a/d1e/d51/dc3/fce [0,4194304] 0 2024-02-28T23:07:53.314 INFO:tasks.workunit.client.0.smithi057.stdout:8/959: write d4/d13/d18/d62/d66/f147 [168919,99580] 0 2024-02-28T23:07:53.317 INFO:tasks.workunit.client.0.smithi057.stdout:1/762: dwrite d2/d7/df/db/d3a/d2a/d82/fe4 [4194304,4194304] 0 2024-02-28T23:07:53.319 INFO:tasks.workunit.client.0.smithi057.stdout:4/828: read d1a/d48/d97/d7f/fa5 [4756032,121095] 0 2024-02-28T23:07:53.324 INFO:tasks.workunit.client.0.smithi057.stdout:4/829: mkdir d1a/d48/d9c/d68/dc5/d120 0 2024-02-28T23:07:53.326 INFO:tasks.workunit.client.0.smithi057.stdout:8/960: getdents d4/d13/d18/d62 0 2024-02-28T23:07:53.329 INFO:tasks.workunit.client.0.smithi057.stdout:8/961: symlink d4/d13/d2c/d4e/l158 0 2024-02-28T23:07:53.329 INFO:tasks.workunit.client.0.smithi057.stdout:4/830: truncate d1a/d2f/db2/d49/d4d/de2/db5/fcd 1031337 0 2024-02-28T23:07:53.331 INFO:tasks.workunit.client.0.smithi057.stdout:4/831: dread - d1a/d48/d9c/d2e/d8f/f11b zero size 2024-02-28T23:07:53.333 INFO:tasks.workunit.client.0.smithi057.stdout:4/832: chown d1a/d48/d9c/d2e/d8f/db7/cdc 10571736 1 2024-02-28T23:07:53.336 INFO:tasks.workunit.client.0.smithi057.stdout:4/833: fsync d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/f117 0 2024-02-28T23:07:53.337 INFO:tasks.workunit.client.0.smithi057.stdout:4/834: chown d1a/d48/d9c/d68 17207720 1 2024-02-28T23:07:53.337 INFO:tasks.workunit.client.0.smithi057.stdout:1/763: dwrite d2/d7/df/db/d12/d3f/fea [0,4194304] 0 2024-02-28T23:07:53.339 INFO:tasks.workunit.client.0.smithi057.stdout:4/835: read - d1a/d48/d9c/d68/dc5/f104 zero size 2024-02-28T23:07:53.341 INFO:tasks.workunit.client.0.smithi057.stdout:1/764: creat d2/d7/df/db/d3a/d2a/d82/f116 x:0 0 0 2024-02-28T23:07:53.345 INFO:tasks.workunit.client.0.smithi057.stdout:1/765: fdatasync d2/d7/df/db/f55 0 2024-02-28T23:07:53.346 INFO:tasks.workunit.client.1.smithi084.stdout:2/977: sync 2024-02-28T23:07:53.352 INFO:tasks.workunit.client.0.smithi057.stdout:1/766: link d2/d7/df/db/d3a/d8f/l108 d2/dd4/de5/l117 0 2024-02-28T23:07:53.353 INFO:tasks.workunit.client.1.smithi084.stdout:2/978: dread d5/d7/d12/d24/f56 [0,4194304] 0 2024-02-28T23:07:53.355 INFO:tasks.workunit.client.1.smithi084.stdout:2/979: truncate d5/f112 209009 0 2024-02-28T23:07:53.357 INFO:tasks.workunit.client.0.smithi057.stdout:4/836: dwrite d1a/d48/db0/f11c [0,4194304] 0 2024-02-28T23:07:53.358 INFO:tasks.workunit.client.0.smithi057.stdout:1/767: rename d2/d7/df/db/d12/d19/d48 to d2/dd4/dd5/d118 0 2024-02-28T23:07:53.363 INFO:tasks.workunit.client.0.smithi057.stdout:1/768: creat d2/d7/df/db/d12/d3f/d47/f119 x:0 0 0 2024-02-28T23:07:53.371 INFO:tasks.workunit.client.1.smithi084.stdout:2/980: dwrite d5/f112 [0,4194304] 0 2024-02-28T23:07:53.372 INFO:tasks.workunit.client.0.smithi057.stdout:4/837: dread d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:53.375 INFO:tasks.workunit.client.0.smithi057.stdout:4/838: mknod d1a/d2f/db2/dcb/dfe/c121 0 2024-02-28T23:07:53.380 INFO:tasks.workunit.client.0.smithi057.stdout:4/839: symlink d1a/d2f/db2/d49/d4d/de2/db5/l122 0 2024-02-28T23:07:53.381 INFO:tasks.workunit.client.0.smithi057.stdout:1/769: dwrite d2/d7/df/db/d3a/d2a/d82/fe4 [4194304,4194304] 0 2024-02-28T23:07:53.392 INFO:tasks.workunit.client.0.smithi057.stdout:1/770: link d2/d7/df/db/f69 d2/d71/def/f11a 0 2024-02-28T23:07:53.394 INFO:tasks.workunit.client.0.smithi057.stdout:1/771: creat d2/d7/df/db/d12/dd6/f11b x:0 0 0 2024-02-28T23:07:53.394 INFO:tasks.workunit.client.1.smithi084.stdout:2/981: dwrite d5/d7/d9/d7c/d5b/f7f [0,4194304] 0 2024-02-28T23:07:53.396 INFO:tasks.workunit.client.0.smithi057.stdout:4/840: dwrite d1a/d1e/f5b [0,4194304] 0 2024-02-28T23:07:53.398 INFO:tasks.workunit.client.1.smithi084.stdout:2/982: unlink d5/dd/d69/c84 0 2024-02-28T23:07:53.399 INFO:tasks.workunit.client.0.smithi057.stdout:4/841: mknod d1a/d48/db0/db1/c123 0 2024-02-28T23:07:53.400 INFO:tasks.workunit.client.0.smithi057.stdout:1/772: symlink d2/d7/df/db/d12/l11c 0 2024-02-28T23:07:53.402 INFO:tasks.workunit.client.0.smithi057.stdout:4/842: chown d1a/d2f/db2/d49/d4d/de2/d6e/d100 29145 1 2024-02-28T23:07:53.403 INFO:tasks.workunit.client.1.smithi084.stdout:2/983: truncate d5/d7/d9/f3c 1172704 0 2024-02-28T23:07:53.405 INFO:tasks.workunit.client.0.smithi057.stdout:1/773: mknod d2/d71/d2d/d5b/c11d 0 2024-02-28T23:07:53.406 INFO:tasks.workunit.client.0.smithi057.stdout:4/843: write d1a/d2f/db2/d49/d4d/dc9/dec/f113 [524915,40056] 0 2024-02-28T23:07:53.408 INFO:tasks.workunit.client.1.smithi084.stdout:2/984: mknod d5/d7/d75/de9/c129 0 2024-02-28T23:07:53.411 INFO:tasks.workunit.client.0.smithi057.stdout:4/844: mknod d1a/d1e/d81/c124 0 2024-02-28T23:07:53.413 INFO:tasks.workunit.client.0.smithi057.stdout:4/845: chown d1a/d2f/db2/dcb/dfe/c121 17 1 2024-02-28T23:07:53.414 INFO:tasks.workunit.client.1.smithi084.stdout:2/985: getdents d5/d64 0 2024-02-28T23:07:53.417 INFO:tasks.workunit.client.0.smithi057.stdout:4/846: truncate d1a/d48/d6c/f8b 1032129 0 2024-02-28T23:07:53.420 INFO:tasks.workunit.client.0.smithi057.stdout:4/847: fsync d1a/d2f/db2/d49/d4d/de2/fb3 0 2024-02-28T23:07:53.422 INFO:tasks.workunit.client.0.smithi057.stdout:4/848: fdatasync d1a/d2f/db2/d49/d4d/f78 0 2024-02-28T23:07:53.425 INFO:tasks.workunit.client.0.smithi057.stdout:4/849: read d1a/d2f/db2/f35 [388726,71655] 0 2024-02-28T23:07:53.431 INFO:tasks.workunit.client.0.smithi057.stdout:4/850: rename d1a/d48/d97 to d1a/d2f/db2/d49/d4d/dc9/dff/d125 0 2024-02-28T23:07:53.434 INFO:tasks.workunit.client.0.smithi057.stdout:4/851: fsync d1a/d48/d9c/f89 0 2024-02-28T23:07:53.437 INFO:tasks.workunit.client.0.smithi057.stdout:1/774: write d2/d7/df/db/d12/d19/f23 [6145937,28979] 0 2024-02-28T23:07:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:4/852: unlink d1a/d2f/db2/d49/d4d/dc9/dec/c116 0 2024-02-28T23:07:53.440 INFO:tasks.workunit.client.1.smithi084.stdout:2/986: dread d5/d7/d12/f51 [0,4194304] 0 2024-02-28T23:07:53.442 INFO:tasks.workunit.client.0.smithi057.stdout:4/853: chown d1a/d2f/db2/d49/d4d/de2 400 1 2024-02-28T23:07:53.442 INFO:tasks.workunit.client.0.smithi057.stdout:1/775: mknod d2/d71/d2d/d5b/c11e 0 2024-02-28T23:07:53.443 INFO:tasks.workunit.client.1.smithi084.stdout:2/987: truncate d5/d7/d9/d7c/f95 3688307 0 2024-02-28T23:07:53.444 INFO:tasks.workunit.client.0.smithi057.stdout:4/854: mkdir d1a/d48/d9c/d2e/d8f/d126 0 2024-02-28T23:07:53.444 INFO:tasks.workunit.client.1.smithi084.stdout:2/988: chown d5/d1f/f109 235553779 1 2024-02-28T23:07:53.446 INFO:tasks.workunit.client.1.smithi084.stdout:2/989: fsync d5/d7/fc8 0 2024-02-28T23:07:53.448 INFO:tasks.workunit.client.1.smithi084.stdout:2/990: dread - d5/d7/d75/dcd/f103 zero size 2024-02-28T23:07:53.451 INFO:tasks.workunit.client.0.smithi057.stdout:4/855: dread d1a/d2f/db2/d49/d4d/dc9/dff/d125/f21 [0,4194304] 0 2024-02-28T23:07:53.451 INFO:tasks.workunit.client.1.smithi084.stdout:2/991: getdents d5/d7/d75/db4 0 2024-02-28T23:07:53.454 INFO:tasks.workunit.client.0.smithi057.stdout:4/856: truncate d1a/d48/d6c/f92 6085974 0 2024-02-28T23:07:53.456 INFO:tasks.workunit.client.0.smithi057.stdout:1/776: dwrite d2/d71/d62/ff8 [0,4194304] 0 2024-02-28T23:07:53.458 INFO:tasks.workunit.client.0.smithi057.stdout:1/777: read - d2/d71/f9e zero size 2024-02-28T23:07:53.460 INFO:tasks.workunit.client.0.smithi057.stdout:4/857: read d1a/d2f/db2/d49/d4d/dc9/dff/d125/f74 [3054885,92760] 0 2024-02-28T23:07:53.466 INFO:tasks.workunit.client.0.smithi057.stdout:1/778: symlink d2/l11f 0 2024-02-28T23:07:53.469 INFO:tasks.workunit.client.1.smithi084.stdout:2/992: dwrite d5/d7/d9/d7c/d5b/da6/f10e [4194304,4194304] 0 2024-02-28T23:07:53.470 INFO:tasks.workunit.client.0.smithi057.stdout:1/779: getdents d2/d7/df/db/da9 0 2024-02-28T23:07:53.473 INFO:tasks.workunit.client.1.smithi084.stdout:2/993: write d5/d7/d2e/f46 [2283226,94974] 0 2024-02-28T23:07:53.476 INFO:tasks.workunit.client.1.smithi084.stdout:2/994: write d5/d7/d9/d7c/f11c [72111,119866] 0 2024-02-28T23:07:53.477 INFO:tasks.workunit.client.0.smithi057.stdout:4/858: write d1a/d2f/db2/d49/d4d/de2/f64 [3249288,84109] 0 2024-02-28T23:07:53.481 INFO:tasks.workunit.client.1.smithi084.stdout:2/995: mkdir d5/d7/d2e/d4a/d12a 0 2024-02-28T23:07:53.482 INFO:tasks.workunit.client.0.smithi057.stdout:4/859: symlink d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/l127 0 2024-02-28T23:07:53.483 INFO:tasks.workunit.client.1.smithi084.stdout:2/996: chown d5/d1f/c60 0 1 2024-02-28T23:07:53.486 INFO:tasks.workunit.client.0.smithi057.stdout:4/860: truncate d1a/d2f/db2/f3c 848720 0 2024-02-28T23:07:53.486 INFO:tasks.workunit.client.1.smithi084.stdout:2/997: mkdir d5/d7/d75/de9/d12b 0 2024-02-28T23:07:53.504 INFO:tasks.workunit.client.0.smithi057.stdout:4/861: dwrite d1a/d48/d9c/d2e/fa2 [0,4194304] 0 2024-02-28T23:07:53.507 INFO:tasks.workunit.client.0.smithi057.stdout:4/862: symlink d1a/l128 0 2024-02-28T23:07:53.513 INFO:tasks.workunit.client.1.smithi084.stdout:2/998: dread d5/d7/d1d/f50 [0,4194304] 0 2024-02-28T23:07:53.528 INFO:tasks.workunit.client.1.smithi084.stdout:2/999: write d5/d7/f111 [683045,39906] 0 2024-02-28T23:07:53.540 INFO:tasks.workunit.client.1.smithi084.stderr:+ rm -rf -- ./tmp.Tih5bPNKzk 2024-02-28T23:07:53.602 INFO:tasks.workunit.client.0.smithi057.stdout:3/762: sync 2024-02-28T23:07:53.656 INFO:tasks.workunit.client.0.smithi057.stdout:3/763: dwrite d3/d4f/d4c/d91/fc0 [0,4194304] 0 2024-02-28T23:07:53.661 INFO:tasks.workunit.client.0.smithi057.stdout:3/764: write d3/d4/d1d/f36 [3800881,59175] 0 2024-02-28T23:07:53.682 INFO:tasks.workunit.client.0.smithi057.stdout:3/765: truncate d3/d8/d27/d70/f74 1299467 0 2024-02-28T23:07:53.689 INFO:tasks.workunit.client.0.smithi057.stdout:2/937: sync 2024-02-28T23:07:53.690 INFO:tasks.workunit.client.0.smithi057.stdout:3/766: chown d3/d8/d27/lbd 309595 1 2024-02-28T23:07:53.697 INFO:tasks.workunit.client.0.smithi057.stdout:3/767: symlink d3/d4/d1d/d47/d62/dbc/lf6 0 2024-02-28T23:07:53.700 INFO:tasks.workunit.client.0.smithi057.stdout:2/938: creat d8/d11/f12b x:0 0 0 2024-02-28T23:07:53.701 INFO:tasks.workunit.client.0.smithi057.stdout:3/768: write d3/d4/d1d/f36 [3710998,108426] 0 2024-02-28T23:07:53.705 INFO:tasks.workunit.client.0.smithi057.stdout:2/939: rmdir d8/d11/d39/d7b 39 2024-02-28T23:07:53.707 INFO:tasks.workunit.client.0.smithi057.stdout:3/769: write d3/d20/d2d/fb8 [634813,77990] 0 2024-02-28T23:07:53.713 INFO:tasks.workunit.client.0.smithi057.stdout:3/770: write d3/d8/f93 [1685147,101982] 0 2024-02-28T23:07:53.714 INFO:tasks.workunit.client.0.smithi057.stdout:2/940: mknod d8/dae/de1/c12c 0 2024-02-28T23:07:53.744 INFO:tasks.workunit.client.0.smithi057.stdout:2/941: dread - d8/d11/d39/dda/f104 zero size 2024-02-28T23:07:53.744 INFO:tasks.workunit.client.0.smithi057.stdout:3/771: write d3/d4/d1d/fdd [109541,129957] 0 2024-02-28T23:07:53.753 INFO:tasks.workunit.client.0.smithi057.stdout:2/942: mknod d8/d11/d15/def/c12d 0 2024-02-28T23:07:53.761 INFO:tasks.workunit.client.0.smithi057.stdout:3/772: getdents d3/d4/d7/d9a/d9b/dab 0 2024-02-28T23:07:53.764 INFO:tasks.workunit.client.0.smithi057.stdout:2/943: symlink d8/d11/d21/d28/d41/dbb/l12e 0 2024-02-28T23:07:53.764 INFO:tasks.workunit.client.0.smithi057.stdout:3/773: chown d3/f5a 17 1 2024-02-28T23:07:53.779 INFO:tasks.workunit.client.0.smithi057.stdout:3/774: mknod d3/d8/d1b/cf7 0 2024-02-28T23:07:53.789 INFO:tasks.workunit.client.0.smithi057.stdout:3/775: mkdir d3/d4/d7/df8 0 2024-02-28T23:07:53.796 INFO:tasks.workunit.client.0.smithi057.stdout:2/944: write d8/d11/d39/d83/f115 [987394,17776] 0 2024-02-28T23:07:53.797 INFO:tasks.workunit.client.0.smithi057.stdout:3/776: fsync d3/d4f/d4c/d91/f9f 0 2024-02-28T23:07:53.804 INFO:tasks.workunit.client.0.smithi057.stdout:7/859: sync 2024-02-28T23:07:53.804 INFO:tasks.workunit.client.0.smithi057.stdout:5/859: sync 2024-02-28T23:07:53.804 INFO:tasks.workunit.client.0.smithi057.stdout:3/777: fdatasync d3/d4/d1d/fcb 0 2024-02-28T23:07:53.808 INFO:tasks.workunit.client.0.smithi057.stdout:5/860: fsync d4/f92 0 2024-02-28T23:07:53.813 INFO:tasks.workunit.client.0.smithi057.stdout:3/778: mknod d3/d4/d7/d9a/cf9 0 2024-02-28T23:07:53.814 INFO:tasks.workunit.client.0.smithi057.stdout:2/945: write d8/d11/d21/d28/d41/dbb/fd5 [342240,39665] 0 2024-02-28T23:07:53.818 INFO:tasks.workunit.client.0.smithi057.stdout:5/861: rmdir d4/d9/ddd 39 2024-02-28T23:07:53.825 INFO:tasks.workunit.client.0.smithi057.stdout:3/779: mkdir d3/d4/d1d/d47/d62/db4/dfa 0 2024-02-28T23:07:53.831 INFO:tasks.workunit.client.0.smithi057.stdout:2/946: symlink d8/d11/d21/d28/d49/d85/d9b/ddc/l12f 0 2024-02-28T23:07:53.832 INFO:tasks.workunit.client.0.smithi057.stdout:3/780: symlink d3/d4/d7/d9a/d9b/dab/dd0/lfb 0 2024-02-28T23:07:53.835 INFO:tasks.workunit.client.0.smithi057.stdout:2/947: creat d8/d11/d39/dbc/f130 x:0 0 0 2024-02-28T23:07:53.844 INFO:tasks.workunit.client.0.smithi057.stdout:2/948: symlink d8/d11/d21/d28/d49/d85/d9b/ddc/l131 0 2024-02-28T23:07:53.847 INFO:tasks.workunit.client.0.smithi057.stdout:5/862: dwrite f2 [0,4194304] 0 2024-02-28T23:07:53.850 INFO:tasks.workunit.client.0.smithi057.stdout:3/781: getdents d3/d4/d7/d9a/d9b/dab/dd0 0 2024-02-28T23:07:53.851 INFO:tasks.workunit.client.0.smithi057.stdout:2/949: readlink d8/d11/d21/d28/d49/d85/d9b/l11e 0 2024-02-28T23:07:53.853 INFO:tasks.workunit.client.0.smithi057.stdout:2/950: stat d8/d11/d15/daf/ff0 0 2024-02-28T23:07:53.854 INFO:tasks.workunit.client.0.smithi057.stdout:5/863: creat d4/d9/d16/d52/dfd/f10a x:0 0 0 2024-02-28T23:07:53.854 INFO:tasks.workunit.client.0.smithi057.stdout:3/782: read d3/d4/d7/fb9 [91110,109020] 0 2024-02-28T23:07:53.858 INFO:tasks.workunit.client.0.smithi057.stdout:3/783: write d3/d8/d27/f90 [128857,130701] 0 2024-02-28T23:07:53.859 INFO:tasks.workunit.client.0.smithi057.stdout:2/951: mkdir d8/d11/d21/d132 0 2024-02-28T23:07:53.860 INFO:tasks.workunit.client.0.smithi057.stdout:3/784: chown d3/d20/d2d/l40 6 1 2024-02-28T23:07:53.862 INFO:tasks.workunit.client.0.smithi057.stdout:5/864: getdents d4/d9/d14 0 2024-02-28T23:07:53.865 INFO:tasks.workunit.client.0.smithi057.stdout:3/785: unlink d3/d8/cc8 0 2024-02-28T23:07:53.865 INFO:tasks.workunit.client.0.smithi057.stdout:5/865: chown d4/d7/d8/lf7 28 1 2024-02-28T23:07:53.867 INFO:tasks.workunit.client.0.smithi057.stdout:2/952: creat d8/d11/d39/d7b/d8b/da2/f133 x:0 0 0 2024-02-28T23:07:53.867 INFO:tasks.workunit.client.0.smithi057.stdout:5/866: chown d4/d32/df1/d24/f51 327 1 2024-02-28T23:07:53.871 INFO:tasks.workunit.client.0.smithi057.stdout:5/867: symlink d4/d7/d58/d78/l10b 0 2024-02-28T23:07:53.874 INFO:tasks.workunit.client.0.smithi057.stdout:5/868: write d4/d9/d16/f1a [3514710,120924] 0 2024-02-28T23:07:53.876 INFO:tasks.workunit.client.0.smithi057.stdout:2/953: write d8/f84 [1911964,8855] 0 2024-02-28T23:07:53.881 INFO:tasks.workunit.client.0.smithi057.stdout:2/954: mkdir d8/d11/d39/d7b/d8b/da2/d134 0 2024-02-28T23:07:53.885 INFO:tasks.workunit.client.0.smithi057.stdout:5/869: link d4/d9/d14/d3b/d61/db2/fd7 d4/d9/ddd/f10c 0 2024-02-28T23:07:53.887 INFO:tasks.workunit.client.0.smithi057.stdout:2/955: unlink d8/d11/f1d 0 2024-02-28T23:07:53.888 INFO:tasks.workunit.client.0.smithi057.stdout:2/956: chown d8/f84 166 1 2024-02-28T23:07:53.889 INFO:tasks.workunit.client.0.smithi057.stdout:5/870: fdatasync d4/d9/d14/d63/fe2 0 2024-02-28T23:07:53.891 INFO:tasks.workunit.client.0.smithi057.stdout:3/786: dwrite d3/d8/d27/fbb [0,4194304] 0 2024-02-28T23:07:53.899 INFO:tasks.workunit.client.0.smithi057.stdout:3/787: write d3/d8/d1b/f4a [774656,119802] 0 2024-02-28T23:07:53.903 INFO:tasks.workunit.client.0.smithi057.stdout:2/957: write d8/dae/de1/ff7 [513311,28598] 0 2024-02-28T23:07:53.911 INFO:tasks.workunit.client.0.smithi057.stdout:8/962: sync 2024-02-28T23:07:53.911 INFO:tasks.workunit.client.0.smithi057.stdout:9/812: sync 2024-02-28T23:07:53.911 INFO:tasks.workunit.client.0.smithi057.stdout:6/781: sync 2024-02-28T23:07:53.913 INFO:tasks.workunit.client.0.smithi057.stdout:5/871: dwrite d4/d32/df1/f2d [0,4194304] 0 2024-02-28T23:07:53.916 INFO:tasks.workunit.client.0.smithi057.stdout:6/782: write d0/d17/d94/fe5 [224019,65864] 0 2024-02-28T23:07:53.917 INFO:tasks.workunit.client.0.smithi057.stdout:5/872: fsync d4/d7/d58/f101 0 2024-02-28T23:07:53.922 INFO:tasks.workunit.client.0.smithi057.stdout:9/813: symlink dd/de/d24/d63/d77/dcf/d40/d44/d92/lff 0 2024-02-28T23:07:53.928 INFO:tasks.workunit.client.0.smithi057.stdout:9/814: write dd/f3c [1251672,11666] 0 2024-02-28T23:07:53.930 INFO:tasks.workunit.client.0.smithi057.stdout:3/788: dwrite d3/d20/d2d/f82 [0,4194304] 0 2024-02-28T23:07:53.942 INFO:tasks.workunit.client.0.smithi057.stdout:2/958: dwrite d8/f23 [0,4194304] 0 2024-02-28T23:07:53.948 INFO:tasks.workunit.client.0.smithi057.stdout:3/789: creat d3/d8/d27/d70/ffc x:0 0 0 2024-02-28T23:07:53.951 INFO:tasks.workunit.client.0.smithi057.stdout:8/963: dwrite d4/de9/d56/d6a/dbd/f122 [0,4194304] 0 2024-02-28T23:07:53.952 INFO:tasks.workunit.client.0.smithi057.stdout:9/815: creat dd/de/d24/d63/d77/dcf/d40/d44/da5/df7/f100 x:0 0 0 2024-02-28T23:07:53.954 INFO:tasks.workunit.client.0.smithi057.stdout:3/790: mkdir d3/d4/dfd 0 2024-02-28T23:07:53.956 INFO:tasks.workunit.client.0.smithi057.stdout:2/959: read d8/d11/d21/f31 [7489969,99626] 0 2024-02-28T23:07:53.963 INFO:tasks.workunit.client.0.smithi057.stdout:6/783: dwrite d0/d4/d25/fa9 [0,4194304] 0 2024-02-28T23:07:53.967 INFO:tasks.workunit.client.0.smithi057.stdout:9/816: mkdir dd/de/d14/dc2/d101 0 2024-02-28T23:07:53.970 INFO:tasks.workunit.client.0.smithi057.stdout:6/784: chown d0/d4/d25/d60 3861740 1 2024-02-28T23:07:53.973 INFO:tasks.workunit.client.0.smithi057.stdout:2/960: mkdir d8/d11/d21/d28/d41/d121/d99/d135 0 2024-02-28T23:07:53.974 INFO:tasks.workunit.client.0.smithi057.stdout:6/785: creat d0/d1b/f105 x:0 0 0 2024-02-28T23:07:53.979 INFO:tasks.workunit.client.0.smithi057.stdout:2/961: mknod d8/d11/d15/db9/c136 0 2024-02-28T23:07:53.980 INFO:tasks.workunit.client.0.smithi057.stdout:3/791: write d3/f64 [524025,26327] 0 2024-02-28T23:07:53.982 INFO:tasks.workunit.client.0.smithi057.stdout:2/962: truncate d8/f109 19487 0 2024-02-28T23:07:53.983 INFO:tasks.workunit.client.0.smithi057.stdout:9/817: write dd/de/d24/d63/dc1/fc5 [1165521,104491] 0 2024-02-28T23:07:53.989 INFO:tasks.workunit.client.0.smithi057.stdout:4/863: sync 2024-02-28T23:07:53.989 INFO:tasks.workunit.client.0.smithi057.stdout:1/780: sync 2024-02-28T23:07:53.989 INFO:tasks.workunit.client.0.smithi057.stdout:7/860: sync 2024-02-28T23:07:53.990 INFO:tasks.workunit.client.0.smithi057.stdout:8/964: dwrite d4/de9/d56/d6a/d42/ff2 [0,4194304] 0 2024-02-28T23:07:53.996 INFO:tasks.workunit.client.0.smithi057.stdout:9/818: symlink dd/de/d14/d48/d4b/d41/l102 0 2024-02-28T23:07:53.998 INFO:tasks.workunit.client.0.smithi057.stdout:7/861: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/fbd 2 1 2024-02-28T23:07:54.000 INFO:tasks.workunit.client.0.smithi057.stdout:5/873: dread d4/d32/df1/d24/d30/d62/faa [0,4194304] 0 2024-02-28T23:07:54.005 INFO:tasks.workunit.client.0.smithi057.stdout:6/786: dwrite d0/d1b/d3e/d40/f4d [0,4194304] 0 2024-02-28T23:07:54.006 INFO:tasks.workunit.client.0.smithi057.stdout:4/864: creat d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/f129 x:0 0 0 2024-02-28T23:07:54.013 INFO:tasks.workunit.client.0.smithi057.stdout:1/781: truncate d2/d7/df/db/d12/dd6/ffb 618363 0 2024-02-28T23:07:54.013 INFO:tasks.workunit.client.0.smithi057.stdout:9/819: creat dd/de/d14/d48/f103 x:0 0 0 2024-02-28T23:07:54.013 INFO:tasks.workunit.client.0.smithi057.stdout:5/874: fsync d4/fb0 0 2024-02-28T23:07:54.017 INFO:tasks.workunit.client.0.smithi057.stdout:7/862: creat d0/d3f/d76/dc1/dea/f11a x:0 0 0 2024-02-28T23:07:54.022 INFO:tasks.workunit.client.0.smithi057.stdout:2/963: dwrite d8/d11/d15/f20 [0,4194304] 0 2024-02-28T23:07:54.026 INFO:tasks.workunit.client.0.smithi057.stdout:7/863: mknod d0/d3f/c11b 0 2024-02-28T23:07:54.028 INFO:tasks.workunit.client.0.smithi057.stdout:1/782: fdatasync d2/d7/df/db/d12/d3f/d47/f9f 0 2024-02-28T23:07:54.029 INFO:tasks.workunit.client.0.smithi057.stdout:2/964: stat d8/d11/d39/f62 0 2024-02-28T23:07:54.031 INFO:tasks.workunit.client.0.smithi057.stdout:9/820: fsync dd/de/d24/d63/d77/dcf/f69 0 2024-02-28T23:07:54.037 INFO:tasks.workunit.client.0.smithi057.stdout:6/787: write d0/fe1 [1239622,72671] 0 2024-02-28T23:07:54.043 INFO:tasks.workunit.client.0.smithi057.stdout:4/865: dread d1a/d2f/db2/d49/d4d/de2/fb3 [0,4194304] 0 2024-02-28T23:07:54.048 INFO:tasks.workunit.client.0.smithi057.stdout:7/864: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/c11c 0 2024-02-28T23:07:54.050 INFO:tasks.workunit.client.0.smithi057.stdout:5/875: dwrite d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:54.058 INFO:tasks.workunit.client.0.smithi057.stdout:4/866: unlink d1a/d48/d9c/d68/f10e 0 2024-02-28T23:07:54.059 INFO:tasks.workunit.client.0.smithi057.stdout:1/783: write d2/d71/d2d/f46 [573699,65361] 0 2024-02-28T23:07:54.061 INFO:tasks.workunit.client.0.smithi057.stdout:7/865: rename d0/d3f/d76/dc1/dea/f9d to d0/d3f/d76/dc1/dea/d75/dbf/f11d 0 2024-02-28T23:07:54.064 INFO:tasks.workunit.client.0.smithi057.stdout:8/965: dread d4/de9/d59/f91 [0,4194304] 0 2024-02-28T23:07:54.068 INFO:tasks.workunit.client.0.smithi057.stdout:9/821: dwrite dd/d46/d6f/f79 [0,4194304] 0 2024-02-28T23:07:54.068 INFO:tasks.workunit.client.0.smithi057.stdout:8/966: stat d4/d13/d2c 0 2024-02-28T23:07:54.071 INFO:tasks.workunit.client.0.smithi057.stdout:7/866: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fb3 258 1 2024-02-28T23:07:54.074 INFO:tasks.workunit.client.0.smithi057.stdout:4/867: dread d1a/d2f/db2/f59 [0,4194304] 0 2024-02-28T23:07:54.075 INFO:tasks.workunit.client.0.smithi057.stdout:1/784: symlink d2/d7/df/db/d12/dd6/l120 0 2024-02-28T23:07:54.088 INFO:tasks.workunit.client.0.smithi057.stdout:9/822: rename dd/de/d24/d63/d77/dcf/d40/d44/da5/cd9 to dd/de/d24/d63/d77/dcf/d40/d44/ded/c104 0 2024-02-28T23:07:54.090 INFO:tasks.workunit.client.0.smithi057.stdout:7/867: creat d0/d98/d10c/f11e x:0 0 0 2024-02-28T23:07:54.094 INFO:tasks.workunit.client.0.smithi057.stdout:2/965: dwrite d8/d11/d21/d28/d41/d121/d91/fb6 [0,4194304] 0 2024-02-28T23:07:54.095 INFO:tasks.workunit.client.0.smithi057.stdout:8/967: dread d4/de9/d7c/ff9 [0,4194304] 0 2024-02-28T23:07:54.098 INFO:tasks.workunit.client.0.smithi057.stdout:5/876: dread d4/d7/d8/f26 [4194304,4194304] 0 2024-02-28T23:07:54.099 INFO:tasks.workunit.client.0.smithi057.stdout:9/823: mkdir dd/d46/dc4/dd3/d105 0 2024-02-28T23:07:54.100 INFO:tasks.workunit.client.0.smithi057.stdout:8/968: write d4/de9/d56/d88/f126 [4382585,12082] 0 2024-02-28T23:07:54.104 INFO:tasks.workunit.client.0.smithi057.stdout:5/877: fsync d4/d9/d16/f18 0 2024-02-28T23:07:54.107 INFO:tasks.workunit.client.0.smithi057.stdout:1/785: dread d2/d71/d62/ff8 [0,4194304] 0 2024-02-28T23:07:54.112 INFO:tasks.workunit.client.0.smithi057.stdout:9/824: creat dd/de/d24/d63/d77/dcf/d40/f106 x:0 0 0 2024-02-28T23:07:54.114 INFO:tasks.workunit.client.0.smithi057.stdout:4/868: dwrite f16 [0,4194304] 0 2024-02-28T23:07:54.118 INFO:tasks.workunit.client.0.smithi057.stdout:5/878: write d4/fb0 [211257,77753] 0 2024-02-28T23:07:54.123 INFO:tasks.workunit.client.0.smithi057.stdout:5/879: stat d4/d7/d8 0 2024-02-28T23:07:54.124 INFO:tasks.workunit.client.0.smithi057.stdout:9/825: rename dd/d46/d4d/l58 to dd/d46/dc4/l107 0 2024-02-28T23:07:54.129 INFO:tasks.workunit.client.0.smithi057.stdout:2/966: dwrite d8/d11/d21/d28/f73 [0,4194304] 0 2024-02-28T23:07:54.136 INFO:tasks.workunit.client.0.smithi057.stdout:5/880: chown d4/d32/d107/ce7 6 1 2024-02-28T23:07:54.137 INFO:tasks.workunit.client.0.smithi057.stdout:7/868: dwrite d0/d3f/d76/dc1/feb [0,4194304] 0 2024-02-28T23:07:54.143 INFO:tasks.workunit.client.0.smithi057.stdout:1/786: truncate d2/d7/df/db/d12/d19/f23 4720656 0 2024-02-28T23:07:54.143 INFO:tasks.workunit.client.0.smithi057.stdout:9/826: mkdir dd/de/d24/d63/d77/dcf/d40/d108 0 2024-02-28T23:07:54.145 INFO:tasks.workunit.client.0.smithi057.stdout:8/969: dwrite d4/d13/d18/d62/d66/d74/f12d [0,4194304] 0 2024-02-28T23:07:54.149 INFO:tasks.workunit.client.0.smithi057.stdout:5/881: getdents d4/d32/df1/d24/d30 0 2024-02-28T23:07:54.153 INFO:tasks.workunit.client.0.smithi057.stdout:7/869: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/ce1 653 1 2024-02-28T23:07:54.154 INFO:tasks.workunit.client.0.smithi057.stdout:1/787: mkdir d2/d33/d54/dc8/dfa/d121 0 2024-02-28T23:07:54.154 INFO:tasks.workunit.client.0.smithi057.stdout:4/869: dwrite d1a/d48/d9c/f38 [0,4194304] 0 2024-02-28T23:07:54.156 INFO:tasks.workunit.client.0.smithi057.stdout:5/882: write d4/d32/df1/d24/fbf [457712,120782] 0 2024-02-28T23:07:54.160 INFO:tasks.workunit.client.0.smithi057.stdout:8/970: creat d4/d13/df0/d104/d10c/f159 x:0 0 0 2024-02-28T23:07:54.166 INFO:tasks.workunit.client.0.smithi057.stdout:4/870: write d1a/d48/d9c/f38 [1197527,72264] 0 2024-02-28T23:07:54.169 INFO:tasks.workunit.client.0.smithi057.stdout:1/788: write d2/d7/df/db/d12/d19/da4/f10f [829540,130251] 0 2024-02-28T23:07:54.170 INFO:tasks.workunit.client.0.smithi057.stdout:8/971: creat d4/de9/d7c/f15a x:0 0 0 2024-02-28T23:07:54.180 INFO:tasks.workunit.client.0.smithi057.stdout:7/870: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/c104 to d0/d3f/d76/dc1/c11f 0 2024-02-28T23:07:54.180 INFO:tasks.workunit.client.0.smithi057.stdout:9/827: dwrite dd/de/d14/d48/d76/dbf/fd2 [0,4194304] 0 2024-02-28T23:07:54.185 INFO:tasks.workunit.client.0.smithi057.stdout:5/883: dread d4/fb9 [0,4194304] 0 2024-02-28T23:07:54.187 INFO:tasks.workunit.client.0.smithi057.stdout:8/972: chown d4/d13/d1a/d4a/db3/c12f 331143538 1 2024-02-28T23:07:54.196 INFO:tasks.workunit.client.0.smithi057.stdout:9/828: fsync dd/de/d24/d63/d77/dcf/d40/d44/f7a 0 2024-02-28T23:07:54.202 INFO:tasks.workunit.client.0.smithi057.stdout:5/884: creat d4/d9/d14/d3b/d61/dfa/f10d x:0 0 0 2024-02-28T23:07:54.211 INFO:tasks.workunit.client.0.smithi057.stdout:8/973: truncate d4/f11 2982057 0 2024-02-28T23:07:54.216 INFO:tasks.workunit.client.0.smithi057.stdout:9/829: creat dd/de/d14/dc2/d101/f109 x:0 0 0 2024-02-28T23:07:54.218 INFO:tasks.workunit.client.0.smithi057.stdout:1/789: dwrite d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:54.220 INFO:tasks.workunit.client.0.smithi057.stdout:9/830: chown dd/de/d24/d63/d77/dcf/d40/lf8 160128 1 2024-02-28T23:07:54.222 INFO:tasks.workunit.client.0.smithi057.stdout:4/871: dwrite d1a/d2f/db2/d49/f86 [4194304,4194304] 0 2024-02-28T23:07:54.223 INFO:tasks.workunit.client.0.smithi057.stdout:1/790: write d2/d71/d2d/d5b/f7c [896301,62514] 0 2024-02-28T23:07:54.225 INFO:tasks.workunit.client.0.smithi057.stdout:9/831: read dd/d46/d6f/fa0 [2333739,108656] 0 2024-02-28T23:07:54.230 INFO:tasks.workunit.client.0.smithi057.stdout:9/832: readlink dd/de/d14/d48/d4b/d41/l102 0 2024-02-28T23:07:54.241 INFO:tasks.workunit.client.0.smithi057.stdout:7/871: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/lcd to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/l120 0 2024-02-28T23:07:54.243 INFO:tasks.workunit.client.0.smithi057.stdout:5/885: creat d4/d7/d58/f10e x:0 0 0 2024-02-28T23:07:54.246 INFO:tasks.workunit.client.0.smithi057.stdout:4/872: creat d1a/d2f/db2/f12a x:0 0 0 2024-02-28T23:07:54.247 INFO:tasks.workunit.client.0.smithi057.stdout:5/886: read - d4/d7/f8d zero size 2024-02-28T23:07:54.250 INFO:tasks.workunit.client.0.smithi057.stdout:1/791: unlink d2/d7/df/db/f69 0 2024-02-28T23:07:54.254 INFO:tasks.workunit.client.0.smithi057.stdout:7/872: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fe6 to d0/d98/f121 0 2024-02-28T23:07:54.255 INFO:tasks.workunit.client.0.smithi057.stdout:9/833: fsync dd/f2f 0 2024-02-28T23:07:54.256 INFO:tasks.workunit.client.0.smithi057.stdout:4/873: stat d1a/d48/d9c/d68/c108 0 2024-02-28T23:07:54.257 INFO:tasks.workunit.client.0.smithi057.stdout:5/887: mkdir d4/d9/d14/d9e/d10f 0 2024-02-28T23:07:54.260 INFO:tasks.workunit.client.0.smithi057.stdout:1/792: mknod d2/d7/df/db/d12/dd6/c122 0 2024-02-28T23:07:54.262 INFO:tasks.workunit.client.0.smithi057.stdout:7/873: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f90 [0,4194304] 0 2024-02-28T23:07:54.263 INFO:tasks.workunit.client.0.smithi057.stdout:4/874: read d1a/d2f/db2/d49/d4d/dc9/dec/f113 [91907,83231] 0 2024-02-28T23:07:54.264 INFO:tasks.workunit.client.0.smithi057.stdout:1/793: stat d2/d7/df/db/d12/l59 0 2024-02-28T23:07:54.267 INFO:tasks.workunit.client.0.smithi057.stdout:5/888: mkdir d4/d9/d14/d3b/d61/dfa/d110 0 2024-02-28T23:07:54.267 INFO:tasks.workunit.client.0.smithi057.stdout:1/794: readlink d2/d7/df/db/d12/l91 0 2024-02-28T23:07:54.268 INFO:tasks.workunit.client.0.smithi057.stdout:3/792: sync 2024-02-28T23:07:54.272 INFO:tasks.workunit.client.0.smithi057.stdout:1/795: creat d2/d71/def/f123 x:0 0 0 2024-02-28T23:07:54.273 INFO:tasks.workunit.client.0.smithi057.stdout:4/875: truncate d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/f129 904099 0 2024-02-28T23:07:54.275 INFO:tasks.workunit.client.0.smithi057.stdout:5/889: rmdir d4/d32/d107 39 2024-02-28T23:07:54.281 INFO:tasks.workunit.client.0.smithi057.stdout:4/876: mknod d1a/d48/c12b 0 2024-02-28T23:07:54.283 INFO:tasks.workunit.client.0.smithi057.stdout:4/877: write d1a/d48/d9c/fda [615864,937] 0 2024-02-28T23:07:54.283 INFO:tasks.workunit.client.0.smithi057.stdout:1/796: creat d2/d7/df/db/d12/d19/f124 x:0 0 0 2024-02-28T23:07:54.284 INFO:tasks.workunit.client.0.smithi057.stdout:9/834: dread dd/d46/fc0 [0,4194304] 0 2024-02-28T23:07:54.285 INFO:tasks.workunit.client.0.smithi057.stdout:1/797: chown d2/dd4/dd5/d118 3 1 2024-02-28T23:07:54.288 INFO:tasks.workunit.client.0.smithi057.stdout:5/890: creat d4/d9/d16/d4e/f111 x:0 0 0 2024-02-28T23:07:54.289 INFO:tasks.workunit.client.0.smithi057.stdout:5/891: chown d4/d9/d14/d63 1432885805 1 2024-02-28T23:07:54.296 INFO:tasks.workunit.client.0.smithi057.stdout:9/835: creat dd/de/d24/d63/d77/df2/f10a x:0 0 0 2024-02-28T23:07:54.299 INFO:tasks.workunit.client.0.smithi057.stdout:5/892: mkdir d4/d32/d107/d112 0 2024-02-28T23:07:54.302 INFO:tasks.workunit.client.0.smithi057.stdout:8/974: write d4/d13/d18/d62/d66/fc4 [1566390,107199] 0 2024-02-28T23:07:54.303 INFO:tasks.workunit.client.0.smithi057.stdout:5/893: chown d4/d9/d16/d52/d7a 1732691897 1 2024-02-28T23:07:54.308 INFO:tasks.workunit.client.0.smithi057.stdout:9/836: symlink dd/de/d24/d63/d77/dcf/d40/d44/d92/l10b 0 2024-02-28T23:07:54.314 INFO:tasks.workunit.client.0.smithi057.stdout:4/878: dwrite d1a/d2f/db2/dcb/dfe/ffc [0,4194304] 0 2024-02-28T23:07:54.317 INFO:tasks.workunit.client.0.smithi057.stdout:2/967: sync 2024-02-28T23:07:54.317 INFO:tasks.workunit.client.0.smithi057.stdout:6/788: sync 2024-02-28T23:07:54.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:54 smithi057 ceph-mon[29096]: pgmap v33: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 96 MiB/s rd, 213 MiB/s wr, 484 op/s 2024-02-28T23:07:54.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:54 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:54.320 INFO:tasks.workunit.client.0.smithi057.stdout:5/894: creat d4/d32/df1/d24/d30/d62/f113 x:0 0 0 2024-02-28T23:07:54.324 INFO:tasks.workunit.client.0.smithi057.stdout:4/879: fdatasync d1a/d2f/db2/f12a 0 2024-02-28T23:07:54.332 INFO:tasks.workunit.client.0.smithi057.stdout:9/837: creat dd/de/d24/d63/d77/dcf/f10c x:0 0 0 2024-02-28T23:07:54.332 INFO:tasks.workunit.client.0.smithi057.stdout:1/798: dread d2/d71/d62/fe0 [0,4194304] 0 2024-02-28T23:07:54.338 INFO:tasks.workunit.client.0.smithi057.stdout:6/789: mkdir d0/d1b/d3e/d100/d106 0 2024-02-28T23:07:54.340 INFO:tasks.workunit.client.0.smithi057.stdout:7/874: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f22 [0,4194304] 0 2024-02-28T23:07:54.341 INFO:tasks.workunit.client.0.smithi057.stdout:8/975: dwrite d4/d13/d18/d62/d66/d74/f110 [0,4194304] 0 2024-02-28T23:07:54.353 INFO:tasks.workunit.client.0.smithi057.stdout:2/968: write d8/d11/d21/d28/d41/d121/fbd [407548,27191] 0 2024-02-28T23:07:54.354 INFO:tasks.workunit.client.0.smithi057.stdout:6/790: dread - d0/d1b/d3e/d3d/d74/fa4 zero size 2024-02-28T23:07:54.364 INFO:tasks.workunit.client.0.smithi057.stdout:1/799: write d2/d71/f9e [148901,64925] 0 2024-02-28T23:07:54.364 INFO:tasks.workunit.client.0.smithi057.stdout:5/895: dwrite d4/d32/df1/f2d [4194304,4194304] 0 2024-02-28T23:07:54.368 INFO:tasks.workunit.client.0.smithi057.stdout:1/800: rmdir d2/d71 39 2024-02-28T23:07:54.370 INFO:tasks.workunit.client.0.smithi057.stdout:5/896: mknod d4/d32/c114 0 2024-02-28T23:07:54.381 INFO:tasks.workunit.client.0.smithi057.stdout:8/976: dwrite d4/d13/d18/d62/d66/d74/ffa [0,4194304] 0 2024-02-28T23:07:54.381 INFO:tasks.workunit.client.0.smithi057.stdout:5/897: mkdir d4/d9/d14/d3b/d61/d115 0 2024-02-28T23:07:54.383 INFO:tasks.workunit.client.0.smithi057.stdout:2/969: getdents d8/d11/d21/d28/d49/d85/d9b 0 2024-02-28T23:07:54.384 INFO:tasks.workunit.client.0.smithi057.stdout:1/801: truncate d2/d7/df/db/d3a/f6c 1652091 0 2024-02-28T23:07:54.387 INFO:tasks.workunit.client.0.smithi057.stdout:8/977: creat d4/de9/d12c/f15b x:0 0 0 2024-02-28T23:07:54.390 INFO:tasks.workunit.client.0.smithi057.stdout:5/898: read - d4/d9/d14/d63/d88/fd3 zero size 2024-02-28T23:07:54.394 INFO:tasks.workunit.client.0.smithi057.stdout:7/875: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f60 [0,4194304] 0 2024-02-28T23:07:54.398 INFO:tasks.workunit.client.0.smithi057.stdout:4/880: dread d1a/d1e/d51/f90 [0,4194304] 0 2024-02-28T23:07:54.399 INFO:tasks.workunit.client.0.smithi057.stdout:1/802: creat d2/d71/d2d/d39/da3/db5/df7/f125 x:0 0 0 2024-02-28T23:07:54.402 INFO:tasks.workunit.client.0.smithi057.stdout:8/978: link d4/c46 d4/c15c 0 2024-02-28T23:07:54.402 INFO:tasks.workunit.client.0.smithi057.stdout:5/899: creat d4/d7/d58/f116 x:0 0 0 2024-02-28T23:07:54.402 INFO:tasks.workunit.client.0.smithi057.stdout:9/838: dread dd/de/d14/f17 [0,4194304] 0 2024-02-28T23:07:54.406 INFO:tasks.workunit.client.0.smithi057.stdout:8/979: mkdir d4/de9/d56/dc9/d15d 0 2024-02-28T23:07:54.413 INFO:tasks.workunit.client.0.smithi057.stdout:7/876: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/f122 x:0 0 0 2024-02-28T23:07:54.413 INFO:tasks.workunit.client.0.smithi057.stdout:8/980: symlink d4/de9/d56/d88/l15e 0 2024-02-28T23:07:54.422 INFO:tasks.workunit.client.0.smithi057.stdout:2/970: dwrite d8/d11/d39/dbc/f114 [0,4194304] 0 2024-02-28T23:07:54.422 INFO:tasks.workunit.client.0.smithi057.stdout:1/803: write d2/d7/df/db/d3a/d66/fbc [268543,38506] 0 2024-02-28T23:07:54.424 INFO:tasks.workunit.client.0.smithi057.stdout:8/981: rmdir d4/d13/d1a/d4a/db3/dfc/dab 39 2024-02-28T23:07:54.427 INFO:tasks.workunit.client.0.smithi057.stdout:1/804: truncate d2/fe9 975482 0 2024-02-28T23:07:54.433 INFO:tasks.workunit.client.0.smithi057.stdout:9/839: truncate dd/de/d14/d48/d76/dbf/fd2 1693322 0 2024-02-28T23:07:54.435 INFO:tasks.workunit.client.0.smithi057.stdout:4/881: dread d1a/d1e/f25 [0,4194304] 0 2024-02-28T23:07:54.436 INFO:tasks.workunit.client.0.smithi057.stdout:5/900: dwrite d4/d9/d16/d4e/fed [0,4194304] 0 2024-02-28T23:07:54.436 INFO:tasks.workunit.client.0.smithi057.stdout:7/877: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/f123 x:0 0 0 2024-02-28T23:07:54.439 INFO:tasks.workunit.client.0.smithi057.stdout:8/982: mknod d4/de9/d37/d9d/c15f 0 2024-02-28T23:07:54.442 INFO:tasks.workunit.client.0.smithi057.stdout:2/971: write d8/fb [2441205,56236] 0 2024-02-28T23:07:54.443 INFO:tasks.workunit.client.0.smithi057.stdout:1/805: mknod d2/d71/d2d/d39/da3/c126 0 2024-02-28T23:07:54.446 INFO:tasks.workunit.client.0.smithi057.stdout:9/840: mknod dd/de/d24/d72/c10d 0 2024-02-28T23:07:54.453 INFO:tasks.workunit.client.0.smithi057.stdout:4/882: dread - d1a/d1e/d81/fca zero size 2024-02-28T23:07:54.453 INFO:tasks.workunit.client.0.smithi057.stdout:5/901: creat d4/d9/d14/f117 x:0 0 0 2024-02-28T23:07:54.454 INFO:tasks.workunit.client.0.smithi057.stdout:9/841: unlink dd/d46/l68 0 2024-02-28T23:07:54.454 INFO:tasks.workunit.client.0.smithi057.stdout:7/878: rmdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d 39 2024-02-28T23:07:54.456 INFO:tasks.workunit.client.0.smithi057.stdout:1/806: chown d2/d33/fae 14625806 1 2024-02-28T23:07:54.457 INFO:tasks.workunit.client.0.smithi057.stdout:5/902: mkdir d4/d32/df1/d118 0 2024-02-28T23:07:54.457 INFO:tasks.workunit.client.0.smithi057.stdout:9/842: write dd/de/d14/dc2/fe8 [486653,47884] 0 2024-02-28T23:07:54.460 INFO:tasks.workunit.client.0.smithi057.stdout:7/879: rename d0/fc0 to d0/d3f/d76/d118/f124 0 2024-02-28T23:07:54.474 INFO:tasks.workunit.client.0.smithi057.stdout:5/903: rename d4/d9/d16/d4e/fac to d4/d9/d14/d3b/d61/dfa/f119 0 2024-02-28T23:07:54.481 INFO:tasks.workunit.client.0.smithi057.stdout:7/880: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d3e to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d125 0 2024-02-28T23:07:54.481 INFO:tasks.workunit.client.0.smithi057.stdout:9/843: creat dd/de/f10e x:0 0 0 2024-02-28T23:07:54.484 INFO:tasks.workunit.client.0.smithi057.stdout:5/904: getdents d4/d9/d14/d9e/d10f 0 2024-02-28T23:07:54.486 INFO:tasks.workunit.client.0.smithi057.stdout:2/972: dread d8/d11/d15/d97/f106 [0,4194304] 0 2024-02-28T23:07:54.486 INFO:tasks.workunit.client.0.smithi057.stdout:7/881: read d0/d3f/d76/dc1/dea/fbe [2516685,105848] 0 2024-02-28T23:07:54.488 INFO:tasks.workunit.client.0.smithi057.stdout:4/883: dwrite d1a/d1e/d51/f90 [0,4194304] 0 2024-02-28T23:07:54.491 INFO:tasks.workunit.client.0.smithi057.stdout:7/882: symlink d0/d3f/d76/d118/l126 0 2024-02-28T23:07:54.493 INFO:tasks.workunit.client.0.smithi057.stdout:5/905: mkdir d4/d9/d16/d52/dfd/da1/d11a 0 2024-02-28T23:07:54.497 INFO:tasks.workunit.client.0.smithi057.stdout:2/973: read - d8/d11/d39/d83/dde/ffc zero size 2024-02-28T23:07:54.498 INFO:tasks.workunit.client.0.smithi057.stdout:4/884: readlink d1a/d2f/db2/d49/d4d/dc9/dff/l118 0 2024-02-28T23:07:54.501 INFO:tasks.workunit.client.0.smithi057.stdout:1/807: dwrite d2/d71/def/ff2 [0,4194304] 0 2024-02-28T23:07:54.503 INFO:tasks.workunit.client.0.smithi057.stdout:9/844: getdents dd/de 0 2024-02-28T23:07:54.508 INFO:tasks.workunit.client.0.smithi057.stdout:5/906: mkdir d4/d7/d58/d78/dc1/d11b 0 2024-02-28T23:07:54.516 INFO:tasks.workunit.client.0.smithi057.stdout:3/793: sync 2024-02-28T23:07:54.520 INFO:tasks.workunit.client.0.smithi057.stdout:2/974: rename d8/dae to d8/d137 0 2024-02-28T23:07:54.521 INFO:tasks.workunit.client.0.smithi057.stdout:7/883: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f22 [8388608,4194304] 0 2024-02-28T23:07:54.524 INFO:tasks.workunit.client.0.smithi057.stdout:5/907: mknod d4/d9/d16/d52/dfd/c11c 0 2024-02-28T23:07:54.528 INFO:tasks.workunit.client.0.smithi057.stdout:1/808: creat d2/d7/df/db/d12/d3f/f127 x:0 0 0 2024-02-28T23:07:54.528 INFO:tasks.workunit.client.0.smithi057.stdout:4/885: link d1a/d48/db0/c5a d1a/d2f/db2/d49/c12c 0 2024-02-28T23:07:54.529 INFO:tasks.workunit.client.0.smithi057.stdout:3/794: mknod d3/cfe 0 2024-02-28T23:07:54.530 INFO:tasks.workunit.client.0.smithi057.stdout:7/884: creat d0/d3f/d76/dc1/f127 x:0 0 0 2024-02-28T23:07:54.530 INFO:tasks.workunit.client.0.smithi057.stdout:2/975: fdatasync d8/d11/d39/fcb 0 2024-02-28T23:07:54.533 INFO:tasks.workunit.client.0.smithi057.stdout:5/908: write d4/d9/d16/f2b [578235,109431] 0 2024-02-28T23:07:54.533 INFO:tasks.workunit.client.0.smithi057.stdout:7/885: chown d0/l33 3 1 2024-02-28T23:07:54.546 INFO:tasks.workunit.client.0.smithi057.stdout:2/976: unlink d8/d11/d15/l63 0 2024-02-28T23:07:54.549 INFO:tasks.workunit.client.0.smithi057.stdout:2/977: write d8/d11/d15/daf/fc0 [3169411,115457] 0 2024-02-28T23:07:54.551 INFO:tasks.workunit.client.0.smithi057.stdout:4/886: truncate d1a/d2f/db2/d49/d4d/de2/f53 1437416 0 2024-02-28T23:07:54.558 INFO:tasks.workunit.client.0.smithi057.stdout:2/978: creat d8/d11/d21/d28/d49/f138 x:0 0 0 2024-02-28T23:07:54.562 INFO:tasks.workunit.client.0.smithi057.stdout:7/886: unlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/f58 0 2024-02-28T23:07:54.567 INFO:tasks.workunit.client.0.smithi057.stdout:9/845: dread dd/de/d24/d63/d77/dcf/f69 [0,4194304] 0 2024-02-28T23:07:54.573 INFO:tasks.workunit.client.0.smithi057.stdout:1/809: dread d2/d33/d54/fc7 [0,4194304] 0 2024-02-28T23:07:54.573 INFO:tasks.workunit.client.0.smithi057.stdout:3/795: link d3/d20/lb0 d3/d4/d7/df8/lff 0 2024-02-28T23:07:54.573 INFO:tasks.workunit.client.0.smithi057.stdout:7/887: rmdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d125 39 2024-02-28T23:07:54.573 INFO:tasks.workunit.client.0.smithi057.stdout:5/909: dwrite d4/d32/f6e [0,4194304] 0 2024-02-28T23:07:54.578 INFO:tasks.workunit.client.0.smithi057.stdout:2/979: rename d8/d11/d39/d83/dde/fcc to d8/d11/d15/f139 0 2024-02-28T23:07:54.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:54 smithi084 ceph-mon[34826]: pgmap v33: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 96 MiB/s rd, 213 MiB/s wr, 484 op/s 2024-02-28T23:07:54.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:54 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:07:54.580 INFO:tasks.workunit.client.0.smithi057.stdout:7/888: stat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/ff0 0 2024-02-28T23:07:54.585 INFO:tasks.workunit.client.0.smithi057.stdout:2/980: chown d8/d11/d15/daf/ff0 27210 1 2024-02-28T23:07:54.587 INFO:tasks.workunit.client.0.smithi057.stdout:4/887: dwrite d1a/d48/d9c/d2e/d8f/db7/ff9 [0,4194304] 0 2024-02-28T23:07:54.590 INFO:tasks.workunit.client.0.smithi057.stdout:1/810: unlink d2/d7/df/db/ca8 0 2024-02-28T23:07:54.591 INFO:tasks.workunit.client.0.smithi057.stdout:3/796: mkdir d3/d4/d1d/d47/d100 0 2024-02-28T23:07:54.595 INFO:tasks.workunit.client.0.smithi057.stdout:1/811: write d2/d7/df/db/d12/d19/f23 [1927923,30699] 0 2024-02-28T23:07:54.604 INFO:tasks.workunit.client.0.smithi057.stdout:9/846: write dd/de/d24/d63/d77/dcf/d40/d44/d5c/f8a [1361920,101037] 0 2024-02-28T23:07:54.612 INFO:tasks.workunit.client.0.smithi057.stdout:2/981: read d8/d11/d21/d28/d41/f53 [3657273,11400] 0 2024-02-28T23:07:54.612 INFO:tasks.workunit.client.0.smithi057.stdout:7/889: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/f128 x:0 0 0 2024-02-28T23:07:54.616 INFO:tasks.workunit.client.0.smithi057.stdout:4/888: write d1a/d48/f61 [91797,88158] 0 2024-02-28T23:07:54.616 INFO:tasks.workunit.client.0.smithi057.stdout:3/797: write d3/d4/d1d/d47/d62/faf [189581,113912] 0 2024-02-28T23:07:54.616 INFO:tasks.workunit.client.0.smithi057.stdout:2/982: chown d8/d11/d39/d7b/d8b/f96 1 1 2024-02-28T23:07:54.620 INFO:tasks.workunit.client.0.smithi057.stdout:9/847: mknod dd/de/d14/d48/d76/dbf/c10f 0 2024-02-28T23:07:54.628 INFO:tasks.workunit.client.0.smithi057.stdout:9/848: symlink dd/d46/dc4/l110 0 2024-02-28T23:07:54.629 INFO:tasks.workunit.client.0.smithi057.stdout:7/890: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/fca [450832,122119] 0 2024-02-28T23:07:54.629 INFO:tasks.workunit.client.0.smithi057.stdout:2/983: symlink d8/d11/d21/d28/d41/d121/d99/d135/l13a 0 2024-02-28T23:07:54.630 INFO:tasks.workunit.client.0.smithi057.stdout:4/889: chown d1a/d2f/l76 815 1 2024-02-28T23:07:54.636 INFO:tasks.workunit.client.0.smithi057.stdout:2/984: mknod d8/d11/d56/c13b 0 2024-02-28T23:07:54.637 INFO:tasks.workunit.client.0.smithi057.stdout:4/890: dread - d1a/d1e/d81/fca zero size 2024-02-28T23:07:54.642 INFO:tasks.workunit.client.0.smithi057.stdout:2/985: mknod d8/d11/d39/d7b/c13c 0 2024-02-28T23:07:54.644 INFO:tasks.workunit.client.0.smithi057.stdout:7/891: mknod d0/d3f/d76/dc1/dea/c129 0 2024-02-28T23:07:54.644 INFO:tasks.workunit.client.0.smithi057.stdout:4/891: write d1a/d48/d9c/d2e/f98 [1425738,37283] 0 2024-02-28T23:07:54.647 INFO:tasks.workunit.client.0.smithi057.stdout:9/849: truncate dd/de/d24/d63/d77/dcf/fe6 250912 0 2024-02-28T23:07:54.648 INFO:tasks.workunit.client.0.smithi057.stdout:1/812: dwrite d2/d7/df/db/d12/f5e [0,4194304] 0 2024-02-28T23:07:54.652 INFO:tasks.workunit.client.0.smithi057.stdout:2/986: mkdir d8/d11/d21/d28/d49/d85/d9b/dad/d13d 0 2024-02-28T23:07:54.652 INFO:tasks.workunit.client.0.smithi057.stdout:7/892: truncate d0/fd7 395434 0 2024-02-28T23:07:54.664 INFO:tasks.workunit.client.0.smithi057.stdout:9/850: creat dd/f111 x:0 0 0 2024-02-28T23:07:54.672 INFO:tasks.workunit.client.0.smithi057.stdout:9/851: symlink dd/de/d4a/d60/d8c/l112 0 2024-02-28T23:07:54.680 INFO:tasks.workunit.client.0.smithi057.stdout:1/813: dread d2/d7/df/db/d12/d19/f23 [0,4194304] 0 2024-02-28T23:07:54.682 INFO:tasks.workunit.client.0.smithi057.stdout:7/893: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/ff0 [0,4194304] 0 2024-02-28T23:07:54.686 INFO:tasks.workunit.client.0.smithi057.stdout:4/892: dread d1a/d2f/db2/d49/d4d/de2/f53 [0,4194304] 0 2024-02-28T23:07:54.687 INFO:tasks.workunit.client.0.smithi057.stdout:9/852: chown dd/de/d24/d72/cbd 953557 1 2024-02-28T23:07:54.692 INFO:tasks.workunit.client.0.smithi057.stdout:9/853: chown dd/de/d4a/d9f/fab 2966 1 2024-02-28T23:07:54.694 INFO:tasks.workunit.client.0.smithi057.stdout:1/814: dread d2/fe9 [0,4194304] 0 2024-02-28T23:07:54.699 INFO:tasks.workunit.client.0.smithi057.stdout:2/987: dwrite d8/d11/d39/dbc/f130 [0,4194304] 0 2024-02-28T23:07:54.718 INFO:tasks.workunit.client.0.smithi057.stdout:7/894: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/l12a 0 2024-02-28T23:07:54.718 INFO:tasks.workunit.client.0.smithi057.stdout:9/854: creat dd/de/d14/dc2/d101/f113 x:0 0 0 2024-02-28T23:07:54.720 INFO:tasks.workunit.client.0.smithi057.stdout:1/815: symlink d2/d33/d54/dc8/l128 0 2024-02-28T23:07:54.721 INFO:tasks.workunit.client.0.smithi057.stdout:6/791: sync 2024-02-28T23:07:54.728 INFO:tasks.workunit.client.0.smithi057.stdout:6/792: chown d0/d1b/d3e/d3d/d42/d4f/fd9 126 1 2024-02-28T23:07:54.730 INFO:tasks.workunit.client.0.smithi057.stdout:4/893: rename d1a/d2f/db2/d49/d4d/cf2 to d1a/d1e/c12d 0 2024-02-28T23:07:54.732 INFO:tasks.workunit.client.0.smithi057.stdout:1/816: mknod d2/d33/daa/c129 0 2024-02-28T23:07:54.736 INFO:tasks.workunit.client.0.smithi057.stdout:6/793: dread - d0/d1b/d3e/d3d/d42/f8f zero size 2024-02-28T23:07:54.750 INFO:tasks.workunit.client.0.smithi057.stdout:1/817: creat d2/dd4/dd5/d118/dff/d100/f12a x:0 0 0 2024-02-28T23:07:54.754 INFO:tasks.workunit.client.0.smithi057.stdout:2/988: truncate d8/d137/de1/ff7 315282 0 2024-02-28T23:07:54.757 INFO:tasks.workunit.client.0.smithi057.stdout:1/818: truncate d2/d71/d2d/f4c 1188403 0 2024-02-28T23:07:54.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.758+0000 7f8994800700 1 -- 172.21.15.57:0/3662115066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 msgr2=0x7f898c071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:54.760 INFO:tasks.workunit.client.0.smithi057.stdout:6/794: read d0/d4/d25/d60/d6a/f7e [3107902,111010] 0 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.758+0000 7f8994800700 1 --2- 172.21.15.57:0/3662115066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c071e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f898800b0a0 tx=0x7f898800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 -- 172.21.15.57:0/3662115066 shutdown_connections 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 --2- 172.21.15.57:0/3662115066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 --2- 172.21.15.57:0/3662115066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c071e10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 -- 172.21.15.57:0/3662115066 >> 172.21.15.57:0/3662115066 conn(0x7f898c06d110 msgr2=0x7f898c06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 -- 172.21.15.57:0/3662115066 shutdown_connections 2024-02-28T23:07:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.759+0000 7f8994800700 1 -- 172.21.15.57:0/3662115066 wait complete. 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.760+0000 7f8994800700 1 Processor -- start 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.761+0000 7f8994800700 1 -- start start 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.761+0000 7f8994800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c10e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.761+0000 7f8994800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.761+0000 7f8994800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c10b4d0 con 0x7f898c071a00 2024-02-28T23:07:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.761+0000 7f8994800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c10b610 con 0x7f898c0723e0 2024-02-28T23:07:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.762+0000 7f8991400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.762+0000 7f8991400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:34732/0 (socket says 172.21.15.57:34732) 2024-02-28T23:07:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.762+0000 7f8991400700 1 -- 172.21.15.57:0/2776592229 learned_addr learned my addr 172.21.15.57:0/2776592229 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:54.764 INFO:tasks.workunit.client.0.smithi057.stdout:6/795: stat d0/d4/feb 0 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.763+0000 7f8991e00700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c10e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.764+0000 7f8991400700 1 -- 172.21.15.57:0/2776592229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 msgr2=0x7f898c10e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.764+0000 7f8991400700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c10e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.764+0000 7f8991400700 1 -- 172.21.15.57:0/2776592229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988009d00 con 0x7f898c0723e0 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.764+0000 7f8991e00700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:07:54.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.765+0000 7f8991400700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f898000e920 tx=0x7f898000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:54.767 INFO:tasks.workunit.client.0.smithi057.stdout:1/819: creat d2/d7/d4f/db1/dce/f12b x:0 0 0 2024-02-28T23:07:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.765+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f898000c9d0 con 0x7f898c0723e0 2024-02-28T23:07:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.765+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8980010430 con 0x7f898c0723e0 2024-02-28T23:07:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.765+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8980018650 con 0x7f898c0723e0 2024-02-28T23:07:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.765+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c10b8f0 con 0x7f898c0723e0 2024-02-28T23:07:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.766+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c10be40 con 0x7f898c0723e0 2024-02-28T23:07:54.768 INFO:tasks.workunit.client.0.smithi057.stdout:6/796: stat d0/d4/d25/f26 0 2024-02-28T23:07:54.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.768+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c19a670 con 0x7f898c0723e0 2024-02-28T23:07:54.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.769+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f89800105a0 con 0x7f898c0723e0 2024-02-28T23:07:54.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.770+0000 7f8986a00700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 0x7f897406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:54.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.770+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8980014070 con 0x7f898c0723e0 2024-02-28T23:07:54.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.771+0000 7f8991e00700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 0x7f897406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:54.773 INFO:tasks.workunit.client.0.smithi057.stdout:1/820: creat d2/d33/d54/dc8/dd2/f12c x:0 0 0 2024-02-28T23:07:54.773 INFO:tasks.workunit.client.0.smithi057.stdout:6/797: mkdir d0/d1b/d3e/d3d/d42/d4f/d92/d107 0 2024-02-28T23:07:54.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.773+0000 7f8991e00700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 0x7f897406f3b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8988006070 tx=0x7f8988006000 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:54.774 INFO:tasks.workunit.client.0.smithi057.stdout:1/821: chown d2/d7/df/db/d3a/f99 4174 1 2024-02-28T23:07:54.775 INFO:tasks.workunit.client.0.smithi057.stdout:6/798: write d0/d1b/d3e/d3d/d42/d4f/ffd [116214,79173] 0 2024-02-28T23:07:54.782 INFO:tasks.workunit.client.0.smithi057.stdout:9/855: dread dd/de/d24/d63/d77/dcf/d40/f59 [4194304,4194304] 0 2024-02-28T23:07:54.783 INFO:tasks.workunit.client.0.smithi057.stdout:2/989: truncate d8/d11/d39/d83/dde/d50/f66 2326132 0 2024-02-28T23:07:54.787 INFO:tasks.workunit.client.0.smithi057.stdout:9/856: creat dd/f114 x:0 0 0 2024-02-28T23:07:54.789 INFO:tasks.workunit.client.0.smithi057.stdout:2/990: symlink d8/d11/d21/d28/d49/d85/d92/dc5/df1/l13e 0 2024-02-28T23:07:54.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:54.793+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8980052a50 con 0x7f898c0723e0 2024-02-28T23:07:54.793 INFO:tasks.workunit.client.0.smithi057.stdout:6/799: dwrite d0/fd1 [0,4194304] 0 2024-02-28T23:07:54.798 INFO:tasks.workunit.client.0.smithi057.stdout:6/800: symlink d0/d17/d94/l108 0 2024-02-28T23:07:54.800 INFO:tasks.workunit.client.0.smithi057.stdout:6/801: readlink d0/d1b/d3e/d40/l47 0 2024-02-28T23:07:54.803 INFO:tasks.workunit.client.0.smithi057.stdout:6/802: dread - d0/d17/d20/f64 zero size 2024-02-28T23:07:54.807 INFO:tasks.workunit.client.0.smithi057.stdout:6/803: mknod d0/d4/d25/d60/d6a/dff/c109 0 2024-02-28T23:07:54.814 INFO:tasks.workunit.client.0.smithi057.stdout:6/804: readlink d0/d1b/d3e/d3d/d42/d4f/d53/le4 0 2024-02-28T23:07:54.816 INFO:tasks.workunit.client.0.smithi057.stdout:9/857: write dd/de/d4a/d60/f75 [1625587,32935] 0 2024-02-28T23:07:54.830 INFO:tasks.workunit.client.0.smithi057.stdout:6/805: mkdir d0/d17/d94/da0/de6/df4/d102/d10a 0 2024-02-28T23:07:54.830 INFO:tasks.workunit.client.0.smithi057.stdout:2/991: dread d8/d11/d21/f36 [8388608,4194304] 0 2024-02-28T23:07:54.830 INFO:tasks.workunit.client.0.smithi057.stdout:9/858: mkdir dd/de/d14/d48/d76/dbf/d115 0 2024-02-28T23:07:54.830 INFO:tasks.workunit.client.0.smithi057.stdout:6/806: symlink d0/d4/d25/d60/d6a/l10b 0 2024-02-28T23:07:54.831 INFO:tasks.workunit.client.0.smithi057.stdout:6/807: stat d0/d1b/d3e/d3d/d42/lee 0 2024-02-28T23:07:54.834 INFO:tasks.workunit.client.0.smithi057.stdout:6/808: creat d0/d17/d94/da0/db3/f10c x:0 0 0 2024-02-28T23:07:54.836 INFO:tasks.workunit.client.0.smithi057.stdout:6/809: rmdir d0/d17/d20 39 2024-02-28T23:07:54.840 INFO:tasks.workunit.client.0.smithi057.stdout:9/859: dread dd/de/d24/d63/d77/dcf/d40/f59 [4194304,4194304] 0 2024-02-28T23:07:54.844 INFO:tasks.workunit.client.0.smithi057.stdout:2/992: dwrite d8/d11/d21/da4/f118 [0,4194304] 0 2024-02-28T23:07:54.848 INFO:tasks.workunit.client.0.smithi057.stdout:2/993: rmdir d8/d11/d39 39 2024-02-28T23:07:54.849 INFO:tasks.workunit.client.0.smithi057.stdout:6/810: rmdir d0/d17 39 2024-02-28T23:07:54.857 INFO:tasks.workunit.client.0.smithi057.stdout:6/811: mknod d0/d17/d20/c10d 0 2024-02-28T23:07:54.861 INFO:tasks.workunit.client.0.smithi057.stdout:6/812: mkdir d0/d4/d25/d60/dc9/d10e 0 2024-02-28T23:07:54.862 INFO:tasks.workunit.client.0.smithi057.stdout:6/813: chown d0/d17/fcd 12 1 2024-02-28T23:07:54.867 INFO:tasks.workunit.client.0.smithi057.stdout:6/814: creat d0/d1b/d3e/d3d/d42/d4f/d53/f10f x:0 0 0 2024-02-28T23:07:54.867 INFO:tasks.workunit.client.0.smithi057.stdout:2/994: dwrite d8/d11/d39/d7b/d8b/fdf [0,4194304] 0 2024-02-28T23:07:54.879 INFO:tasks.workunit.client.0.smithi057.stdout:6/815: creat d0/d17/d94/da0/de6/df4/d102/d10a/f110 x:0 0 0 2024-02-28T23:07:54.879 INFO:tasks.workunit.client.0.smithi057.stdout:2/995: rename d8/d11/d56 to d8/d137/de1/d10f/d13f 0 2024-02-28T23:07:54.883 INFO:tasks.workunit.client.0.smithi057.stdout:2/996: mkdir d8/d11/d21/d28/d41/d121/d99/d135/d140 0 2024-02-28T23:07:54.884 INFO:tasks.workunit.client.0.smithi057.stdout:6/816: rmdir d0/d1b/d3e/d3d/d42/d4f 39 2024-02-28T23:07:54.887 INFO:tasks.workunit.client.0.smithi057.stdout:2/997: mknod d8/d11/d21/c141 0 2024-02-28T23:07:54.889 INFO:tasks.workunit.client.0.smithi057.stdout:6/817: creat d0/d1b/d3e/d3d/f111 x:0 0 0 2024-02-28T23:07:54.891 INFO:tasks.workunit.client.0.smithi057.stdout:2/998: symlink d8/d11/d15/def/l142 0 2024-02-28T23:07:54.896 INFO:tasks.workunit.client.0.smithi057.stdout:2/999: rename d8/d11/d15/c81 to d8/d11/d21/d28/d49/d85/c143 0 2024-02-28T23:07:54.896 INFO:tasks.workunit.client.0.smithi057.stdout:8/983: sync 2024-02-28T23:07:54.897 INFO:tasks.workunit.client.0.smithi057.stdout:6/818: write d0/d4/d25/f2a [1336541,17717] 0 2024-02-28T23:07:54.902 INFO:tasks.workunit.client.0.smithi057.stdout:8/984: mknod d4/d13/d1a/d124/c160 0 2024-02-28T23:07:54.902 INFO:tasks.workunit.client.0.smithi057.stdout:6/819: write d0/d17/d94/da0/db3/f10c [1028535,82789] 0 2024-02-28T23:07:54.912 INFO:tasks.workunit.client.0.smithi057.stdout:6/820: getdents d0/d17 0 2024-02-28T23:07:54.914 INFO:tasks.workunit.client.0.smithi057.stdout:6/821: write d0/d1b/d3e/f104 [834050,52683] 0 2024-02-28T23:07:54.917 INFO:tasks.workunit.client.0.smithi057.stdout:6/822: fsync d0/d1b/d3e/d3d/f111 0 2024-02-28T23:07:54.922 INFO:tasks.workunit.client.0.smithi057.stdout:8/985: dwrite d4/d13/d18/f65 [4194304,4194304] 0 2024-02-28T23:07:54.933 INFO:tasks.workunit.client.0.smithi057.stdout:8/986: getdents d4/d13 0 2024-02-28T23:07:54.934 INFO:tasks.workunit.client.0.smithi057.stdout:6/823: truncate d0/d17/d20/fe9 863407 0 2024-02-28T23:07:54.935 INFO:tasks.workunit.client.0.smithi057.stdout:8/987: chown d4/de9/d56/d6a/d54/f142 62128027 1 2024-02-28T23:07:54.958 INFO:tasks.workunit.client.0.smithi057.stdout:6/824: dwrite d0/d4/d25/f71 [4194304,4194304] 0 2024-02-28T23:07:54.958 INFO:tasks.workunit.client.0.smithi057.stdout:8/988: dwrite d4/d13/d1a/d4a/db3/fd2 [0,4194304] 0 2024-02-28T23:07:54.961 INFO:tasks.workunit.client.0.smithi057.stdout:8/989: chown d4/d13/l50 70681759 1 2024-02-28T23:07:54.963 INFO:tasks.workunit.client.0.smithi057.stdout:6/825: symlink d0/d4/d25/d60/dc9/l112 0 2024-02-28T23:07:54.970 INFO:tasks.workunit.client.0.smithi057.stdout:8/990: read - d4/d13/d18/d62/de4/f154 zero size 2024-02-28T23:07:54.972 INFO:tasks.workunit.client.0.smithi057.stdout:8/991: fsync d4/d13/d2c/db1/fb2 0 2024-02-28T23:07:54.975 INFO:tasks.workunit.client.0.smithi057.stdout:8/992: mknod d4/d13/c161 0 2024-02-28T23:07:54.981 INFO:tasks.workunit.client.0.smithi057.stdout:6/826: fsync d0/d17/f35 0 2024-02-28T23:07:54.984 INFO:tasks.workunit.client.0.smithi057.stdout:6/827: read d0/d1b/fed [92175,122695] 0 2024-02-28T23:07:54.989 INFO:tasks.workunit.client.0.smithi057.stdout:8/993: write d4/d13/d1a/fdb [977581,68702] 0 2024-02-28T23:07:55.007 INFO:tasks.workunit.client.0.smithi057.stdout:6/828: dread d0/d1b/f27 [0,4194304] 0 2024-02-28T23:07:55.015 INFO:tasks.workunit.client.0.smithi057.stdout:8/994: dread d4/d13/d18/d62/d66/fd8 [0,4194304] 0 2024-02-28T23:07:55.020 INFO:tasks.workunit.client.0.smithi057.stdout:8/995: rename d4/de9/d37/d9d/f144 to d4/de9/d56/dfd/f162 0 2024-02-28T23:07:55.025 INFO:tasks.workunit.client.0.smithi057.stdout:8/996: creat d4/d13/df0/d104/f163 x:0 0 0 2024-02-28T23:07:55.027 INFO:tasks.workunit.client.0.smithi057.stdout:8/997: chown d4/de9/d59/fa2 22 1 2024-02-28T23:07:55.030 INFO:tasks.workunit.client.0.smithi057.stdout:8/998: creat d4/d13/f164 x:0 0 0 2024-02-28T23:07:55.032 INFO:tasks.workunit.client.0.smithi057.stdout:6/829: dread d0/f6c [0,4194304] 0 2024-02-28T23:07:55.036 INFO:tasks.workunit.client.0.smithi057.stdout:6/830: truncate d0/d17/f35 8959260 0 2024-02-28T23:07:55.037 INFO:tasks.workunit.client.0.smithi057.stdout:5/910: sync 2024-02-28T23:07:55.037 INFO:tasks.workunit.client.0.smithi057.stdout:6/831: dread - d0/d1b/ffe zero size 2024-02-28T23:07:55.042 INFO:tasks.workunit.client.0.smithi057.stdout:5/911: unlink d4/d32/df1/f67 0 2024-02-28T23:07:55.043 INFO:tasks.workunit.client.0.smithi057.stdout:6/832: fdatasync d0/d1b/d3e/d3d/f62 0 2024-02-28T23:07:55.045 INFO:tasks.workunit.client.0.smithi057.stdout:6/833: readlink d0/d4/d11/lea 0 2024-02-28T23:07:55.051 INFO:tasks.workunit.client.0.smithi057.stdout:5/912: mknod d4/d7/d58/d78/c11d 0 2024-02-28T23:07:55.052 INFO:tasks.workunit.client.0.smithi057.stdout:6/834: write d0/d1b/d3e/d3d/d74/fb8 [1166380,106991] 0 2024-02-28T23:07:55.053 INFO:tasks.workunit.client.0.smithi057.stdout:5/913: write d4/d9/d14/fec [789903,21557] 0 2024-02-28T23:07:55.054 INFO:tasks.workunit.client.0.smithi057.stdout:6/835: stat d0/d17/d20/f5c 0 2024-02-28T23:07:55.054 INFO:tasks.workunit.client.0.smithi057.stdout:8/999: dwrite d4/d13/d2c/db1/fb7 [8388608,4194304] 0 2024-02-28T23:07:55.054 INFO:tasks.workunit.client.0.smithi057.stdout:5/914: chown d4/d7/l97 241538299 1 2024-02-28T23:07:55.064 INFO:tasks.workunit.client.0.smithi057.stdout:6/836: mkdir d0/d1b/d3e/d3d/d42/d4f/d113 0 2024-02-28T23:07:55.064 INFO:tasks.workunit.client.0.smithi057.stdout:5/915: getdents d4/d9/d14/d63/d88 0 2024-02-28T23:07:55.066 INFO:tasks.workunit.client.0.smithi057.stdout:6/837: chown d0/d1b/d3e/d3d/d42/d4f/d53/le4 105077 1 2024-02-28T23:07:55.068 INFO:tasks.workunit.client.0.smithi057.stdout:5/916: truncate d4/d7/d58/f116 521002 0 2024-02-28T23:07:55.069 INFO:tasks.workunit.client.0.smithi057.stdout:6/838: chown d0/d4/d25/d60/d6a/ca8 444 1 2024-02-28T23:07:55.077 INFO:tasks.workunit.client.0.smithi057.stdout:6/839: rename d0/d1b/d3e/d3d/d42/d4f/d92/lb9 to d0/d17/d94/da0/l114 0 2024-02-28T23:07:55.084 INFO:tasks.workunit.client.0.smithi057.stdout:6/840: write d0/d1b/d54/fdf [928160,5827] 0 2024-02-28T23:07:55.088 INFO:tasks.workunit.client.0.smithi057.stdout:6/841: chown d0/d1b/d3e/d3d/fc4 49619807 1 2024-02-28T23:07:55.088 INFO:tasks.workunit.client.0.smithi057.stdout:5/917: dwrite d4/d9/d16/d52/fe1 [0,4194304] 0 2024-02-28T23:07:55.091 INFO:tasks.workunit.client.0.smithi057.stdout:5/918: write d4/d32/df1/f2d [4780848,24302] 0 2024-02-28T23:07:55.091 INFO:tasks.workunit.client.0.smithi057.stdout:6/842: mknod d0/d17/d94/da0/db3/c115 0 2024-02-28T23:07:55.095 INFO:tasks.workunit.client.0.smithi057.stdout:6/843: chown d0/d17/d94/da0/de6/df4/d102/d10a/f110 63107352 1 2024-02-28T23:07:55.096 INFO:tasks.workunit.client.0.smithi057.stdout:3/798: sync 2024-02-28T23:07:55.100 INFO:tasks.workunit.client.0.smithi057.stdout:5/919: link d4/d32/df1/d24/da2/dbb/fa4 d4/d9/d14/d3b/d61/dfa/d110/f11e 0 2024-02-28T23:07:55.100 INFO:tasks.workunit.client.0.smithi057.stdout:3/799: mkdir d3/d4f/d101 0 2024-02-28T23:07:55.104 INFO:tasks.workunit.client.0.smithi057.stdout:6/844: unlink d0/d1b/d3e/d3d/d42/d4f/d53/f5b 0 2024-02-28T23:07:55.106 INFO:tasks.workunit.client.0.smithi057.stdout:3/800: write d3/d4/d1d/d47/d62/fa5 [86388,18408] 0 2024-02-28T23:07:55.108 INFO:tasks.workunit.client.0.smithi057.stdout:3/801: unlink d3/d20/d2d/c97 0 2024-02-28T23:07:55.110 INFO:tasks.workunit.client.0.smithi057.stdout:3/802: chown d3/d4/d7/l1c 457005 1 2024-02-28T23:07:55.113 INFO:tasks.workunit.client.0.smithi057.stdout:3/803: mknod d3/d20/c102 0 2024-02-28T23:07:55.117 INFO:tasks.workunit.client.0.smithi057.stdout:3/804: stat d3/d4/d1d/d47 0 2024-02-28T23:07:55.133 INFO:tasks.workunit.client.0.smithi057.stdout:3/805: truncate d3/d4/d7/fa9 101319 0 2024-02-28T23:07:55.137 INFO:tasks.workunit.client.0.smithi057.stdout:5/920: dread d4/d9/fa6 [4194304,4194304] 0 2024-02-28T23:07:55.137 INFO:tasks.workunit.client.0.smithi057.stdout:6/845: dwrite d0/d4/d25/f26 [0,4194304] 0 2024-02-28T23:07:55.139 INFO:tasks.workunit.client.0.smithi057.stdout:3/806: link d3/d4/d1d/d47/l52 d3/d4/dfd/l103 0 2024-02-28T23:07:55.143 INFO:tasks.workunit.client.0.smithi057.stdout:3/807: creat d3/d4/d7/d9a/f104 x:0 0 0 2024-02-28T23:07:55.144 INFO:tasks.workunit.client.0.smithi057.stdout:6/846: creat d0/d17/d94/da0/de6/df4/d102/f116 x:0 0 0 2024-02-28T23:07:55.144 INFO:tasks.workunit.client.0.smithi057.stdout:3/808: fsync d3/d4/d1d/f1e 0 2024-02-28T23:07:55.145 INFO:tasks.workunit.client.0.smithi057.stdout:3/809: fdatasync d3/f64 0 2024-02-28T23:07:55.147 INFO:tasks.workunit.client.0.smithi057.stdout:6/847: truncate d0/d1b/d3e/d3d/d42/ffb 458242 0 2024-02-28T23:07:55.148 INFO:tasks.workunit.client.0.smithi057.stdout:3/810: mknod d3/d4/d7/c105 0 2024-02-28T23:07:55.153 INFO:tasks.workunit.client.0.smithi057.stdout:5/921: write d4/d32/df1/d24/d30/d62/faa [118727,4159] 0 2024-02-28T23:07:55.155 INFO:tasks.workunit.client.0.smithi057.stdout:9/860: sync 2024-02-28T23:07:55.155 INFO:tasks.workunit.client.0.smithi057.stdout:7/895: sync 2024-02-28T23:07:55.156 INFO:tasks.workunit.client.0.smithi057.stdout:1/822: sync 2024-02-28T23:07:55.156 INFO:tasks.workunit.client.0.smithi057.stdout:4/894: sync 2024-02-28T23:07:55.161 INFO:tasks.workunit.client.0.smithi057.stdout:6/848: symlink d0/d1b/d3e/d3d/d42/d4f/d92/d107/l117 0 2024-02-28T23:07:55.165 INFO:tasks.workunit.client.0.smithi057.stdout:1/823: symlink d2/d33/d54/dc8/l12d 0 2024-02-28T23:07:55.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.166+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f898c061190 con 0x7f897406cf00 2024-02-28T23:07:55.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.168+0000 7f8986a00700 1 -- 172.21.15.57:0/2776592229 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f898c061190 con 0x7f897406cf00 2024-02-28T23:07:55.168 INFO:tasks.workunit.client.0.smithi057.stdout:9/861: creat dd/de/d4a/d60/d8c/f116 x:0 0 0 2024-02-28T23:07:55.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.173+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 msgr2=0x7f897406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.173+0000 7f8994800700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 0x7f897406f3b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8988006070 tx=0x7f8988006000 comp rx=0 tx=0).stop 2024-02-28T23:07:55.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.173+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 msgr2=0x7f898c10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.173+0000 7f8994800700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f898000e920 tx=0x7f898000ec30 comp rx=0 tx=0).stop 2024-02-28T23:07:55.173 INFO:tasks.workunit.client.0.smithi057.stdout:5/922: rmdir d4/d7/d58/d78/dc1/d11b 0 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.174+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 shutdown_connections 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.174+0000 7f8994800700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f897406cf00 0x7f897406f3b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.174+0000 7f8994800700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898c071a00 0x7f898c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.174+0000 7f8994800700 1 --2- 172.21.15.57:0/2776592229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f898c0723e0 0x7f898c10eed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.174+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 >> 172.21.15.57:0/2776592229 conn(0x7f898c06d110 msgr2=0x7f898c108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.175+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 shutdown_connections 2024-02-28T23:07:55.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.176+0000 7f8994800700 1 -- 172.21.15.57:0/2776592229 wait complete. 2024-02-28T23:07:55.176 INFO:tasks.workunit.client.0.smithi057.stdout:1/824: mkdir d2/d7/df/db/d12/d3f/d47/d12e 0 2024-02-28T23:07:55.183 INFO:tasks.workunit.client.0.smithi057.stdout:7/896: rename d0/d3f/d76/ld3 to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/l12b 0 2024-02-28T23:07:55.188 INFO:tasks.workunit.client.0.smithi057.stdout:6/849: getdents d0/d4/d11 0 2024-02-28T23:07:55.189 INFO:tasks.workunit.client.0.smithi057.stdout:7/897: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/f10e 23688111 1 2024-02-28T23:07:55.197 INFO:tasks.workunit.client.0.smithi057.stdout:3/811: dread d3/d4f/d4c/f54 [0,4194304] 0 2024-02-28T23:07:55.199 INFO:tasks.workunit.client.0.smithi057.stdout:6/850: truncate d0/d4/d11/fec 399360 0 2024-02-28T23:07:55.200 INFO:tasks.workunit.client.0.smithi057.stdout:9/862: symlink dd/de/d14/d48/d4b/d41/dde/l117 0 2024-02-28T23:07:55.201 INFO:tasks.workunit.client.0.smithi057.stdout:1/825: symlink d2/dd4/l12f 0 2024-02-28T23:07:55.201 INFO:tasks.workunit.client.0.smithi057.stdout:5/923: dwrite d4/d7/d58/f108 [0,4194304] 0 2024-02-28T23:07:55.201 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:07:55.202 INFO:tasks.workunit.client.0.smithi057.stdout:9/863: fsync dd/f2d 0 2024-02-28T23:07:55.208 INFO:tasks.workunit.client.0.smithi057.stdout:4/895: rename d1a/d2f/db2/d49/d4d/dc9/dff/d125/d7f/fcc to d1a/d2f/db2/d49/d4d/de2/f12e 0 2024-02-28T23:07:55.221 INFO:tasks.workunit.client.0.smithi057.stdout:5/924: symlink d4/d9/d16/d52/d7a/l11f 0 2024-02-28T23:07:55.221 INFO:tasks.workunit.client.0.smithi057.stdout:9/864: unlink dd/de/d24/d63/d77/dcf/d40/f106 0 2024-02-28T23:07:55.221 INFO:tasks.workunit.client.0.smithi057.stdout:1/826: creat d2/d71/d2d/d5b/f130 x:0 0 0 2024-02-28T23:07:55.228 INFO:tasks.workunit.client.0.smithi057.stdout:9/865: chown dd/de/d24/d63/d77/dcf/d40/d44/d5c/fe7 26382485 1 2024-02-28T23:07:55.230 INFO:tasks.workunit.client.0.smithi057.stdout:9/866: readlink dd/de/d24/d63/d77/dcf/d40/l42 0 2024-02-28T23:07:55.244 INFO:tasks.workunit.client.0.smithi057.stdout:7/898: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/f8c [0,4194304] 0 2024-02-28T23:07:55.249 INFO:tasks.workunit.client.0.smithi057.stdout:6/851: dread d0/d17/d20/f5e [0,4194304] 0 2024-02-28T23:07:55.265 INFO:tasks.workunit.client.0.smithi057.stdout:4/896: creat d1a/d2f/db2/d49/d4d/de2/f12f x:0 0 0 2024-02-28T23:07:55.269 INFO:tasks.workunit.client.0.smithi057.stdout:5/925: rename d4/d9/d16/d52/d7a/ce4 to d4/d32/df1/d24/da2/c120 0 2024-02-28T23:07:55.276 INFO:tasks.workunit.client.0.smithi057.stdout:9/867: rename dd/de/d24/d63/d77/dcf/d40/d44/da5/lc6 to dd/de/d24/d63/d77/df2/l118 0 2024-02-28T23:07:55.284 INFO:tasks.workunit.client.0.smithi057.stdout:4/897: rename d1a/d48/d9c/d2e/f98 to d1a/d2f/db2/d49/d4d/f130 0 2024-02-28T23:07:55.287 INFO:tasks.workunit.client.0.smithi057.stdout:9/868: symlink dd/de/d4a/d60/l119 0 2024-02-28T23:07:55.288 INFO:tasks.workunit.client.0.smithi057.stdout:6/852: rmdir d0/d1b/d3e/d3d/d42/d4f/d113 0 2024-02-28T23:07:55.292 INFO:tasks.workunit.client.0.smithi057.stdout:7/899: getdents d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25 0 2024-02-28T23:07:55.295 INFO:tasks.workunit.client.0.smithi057.stdout:7/900: dread - d0/d3f/d76/dc1/dea/f11a zero size 2024-02-28T23:07:55.297 INFO:tasks.workunit.client.0.smithi057.stdout:4/898: creat d1a/d2f/db2/d49/d4d/de2/d6e/f131 x:0 0 0 2024-02-28T23:07:55.297 INFO:tasks.workunit.client.0.smithi057.stdout:6/853: truncate d0/d1b/fd5 1013835 0 2024-02-28T23:07:55.303 INFO:tasks.workunit.client.0.smithi057.stdout:4/899: chown d1a/d2f/db2/d49/d4d/dc9/f106 28750 1 2024-02-28T23:07:55.306 INFO:tasks.workunit.client.0.smithi057.stdout:7/901: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/l12c 0 2024-02-28T23:07:55.313 INFO:tasks.workunit.client.0.smithi057.stdout:9/869: dwrite dd/f114 [0,4194304] 0 2024-02-28T23:07:55.321 INFO:tasks.workunit.client.0.smithi057.stdout:3/812: truncate d3/d4/d7/d9a/d9b/fd2 752557 0 2024-02-28T23:07:55.327 INFO:tasks.workunit.client.0.smithi057.stdout:4/900: dwrite d1a/d2f/f102 [0,4194304] 0 2024-02-28T23:07:55.334 INFO:tasks.workunit.client.0.smithi057.stdout:1/827: write d2/d71/d2d/d5b/fe7 [857361,59658] 0 2024-02-28T23:07:55.349 INFO:tasks.workunit.client.0.smithi057.stdout:6/854: write d0/d4/d11/f13 [155520,1447] 0 2024-02-28T23:07:55.350 INFO:tasks.workunit.client.0.smithi057.stdout:9/870: link dd/de/c9c dd/de/d24/d63/d77/dcf/d40/d44/c11a 0 2024-02-28T23:07:55.352 INFO:tasks.workunit.client.0.smithi057.stdout:7/902: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/f19 [3206623,69119] 0 2024-02-28T23:07:55.355 INFO:tasks.workunit.client.0.smithi057.stdout:3/813: dwrite d3/ff5 [0,4194304] 0 2024-02-28T23:07:55.362 INFO:tasks.workunit.client.0.smithi057.stdout:1/828: truncate d2/d7/df/db/d12/d3f/d47/f8e 184326 0 2024-02-28T23:07:55.368 INFO:tasks.workunit.client.0.smithi057.stdout:6/855: creat d0/d17/d94/da0/de6/df4/d102/d10a/f118 x:0 0 0 2024-02-28T23:07:55.376 INFO:tasks.workunit.client.0.smithi057.stdout:1/829: creat d2/d7/d4f/db4/f131 x:0 0 0 2024-02-28T23:07:55.379 INFO:tasks.workunit.client.0.smithi057.stdout:5/926: dwrite d4/d9/d14/d63/f7b [0,4194304] 0 2024-02-28T23:07:55.385 INFO:tasks.workunit.client.0.smithi057.stdout:4/901: dwrite d1a/d2f/db2/d49/d4d/dc9/dff/d125/d7f/faa [0,4194304] 0 2024-02-28T23:07:55.398 INFO:tasks.workunit.client.0.smithi057.stdout:7/903: rename d0/d3f/d76/dc1/fdf to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/f12d 0 2024-02-28T23:07:55.400 INFO:tasks.workunit.client.0.smithi057.stdout:1/830: write d2/d71/def/f123 [880169,20175] 0 2024-02-28T23:07:55.402 INFO:tasks.workunit.client.0.smithi057.stdout:9/871: dread dd/de/d24/d63/d77/dcf/d40/d44/f7a [0,4194304] 0 2024-02-28T23:07:55.404 INFO:tasks.workunit.client.0.smithi057.stdout:4/902: mkdir d1a/d48/d9c/d2e/df8/d132 0 2024-02-28T23:07:55.409 INFO:tasks.workunit.client.0.smithi057.stdout:1/831: readlink d2/d7/df/l2f 0 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.411+0000 7fa8c1800700 1 -- 172.21.15.57:0/590701543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 msgr2=0x7fa8bc0722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.411+0000 7fa8c1800700 1 --2- 172.21.15.57:0/590701543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc0722d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80074e0 tx=0x7fa8a800af90 comp rx=0 tx=0).stop 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.414+0000 7fa8c1800700 1 -- 172.21.15.57:0/590701543 shutdown_connections 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.414+0000 7fa8c1800700 1 --2- 172.21.15.57:0/590701543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.414+0000 7fa8c1800700 1 --2- 172.21.15.57:0/590701543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc0722d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.414+0000 7fa8c1800700 1 -- 172.21.15.57:0/590701543 >> 172.21.15.57:0/590701543 conn(0x7fa8bc06d110 msgr2=0x7fa8bc06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:55.418 INFO:tasks.workunit.client.0.smithi057.stdout:5/927: getdents d4/d32/df1/d24/da2/dbb 0 2024-02-28T23:07:55.418 INFO:tasks.workunit.client.0.smithi057.stdout:4/903: mknod d1a/d48/d9c/d2e/df8/c133 0 2024-02-28T23:07:55.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.415+0000 7fa8c1800700 1 -- 172.21.15.57:0/590701543 shutdown_connections 2024-02-28T23:07:55.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.415+0000 7fa8c1800700 1 -- 172.21.15.57:0/590701543 wait complete. 2024-02-28T23:07:55.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.416+0000 7fa8c1800700 1 Processor -- start 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.417+0000 7fa8c1800700 1 -- start start 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.417+0000 7fa8c1800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.417+0000 7fa8c1800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.417+0000 7fa8c1800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc19cc10 con 0x7fa8bc107e90 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.417+0000 7fa8c1800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc19cd50 con 0x7fa8bc071ec0 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.418+0000 7fa8ba000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.418+0000 7fa8ba000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59388/0 (socket says 172.21.15.57:59388) 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.418+0000 7fa8ba000700 1 -- 172.21.15.57:0/447551991 learned_addr learned my addr 172.21.15.57:0/447551991 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.418+0000 7fa8baa00700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.419+0000 7fa8ba000700 1 -- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 msgr2=0x7fa8bc19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.419+0000 7fa8ba000700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.419+0000 7fa8ba000700 1 -- 172.21.15.57:0/447551991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80071c0 con 0x7fa8bc107e90 2024-02-28T23:07:55.420 INFO:tasks.workunit.client.0.smithi057.stdout:9/872: creat dd/de/d24/d63/d77/dcf/f11b x:0 0 0 2024-02-28T23:07:55.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.422+0000 7fa8ba000700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4007ed0 tx=0x7fa8b400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:55.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.422+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b4012630 con 0x7fa8bc107e90 2024-02-28T23:07:55.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.423+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc1a1800 con 0x7fa8bc107e90 2024-02-28T23:07:55.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.423+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc1a1d50 con 0x7fa8bc107e90 2024-02-28T23:07:55.424 INFO:tasks.workunit.client.0.smithi057.stdout:6/856: dread d0/d1b/fd5 [0,4194304] 0 2024-02-28T23:07:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.424+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8b4009950 con 0x7fa8bc107e90 2024-02-28T23:07:55.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.425+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b4011b10 con 0x7fa8bc107e90 2024-02-28T23:07:55.426 INFO:tasks.workunit.client.0.smithi057.stdout:4/904: dread - d1a/d2f/db2/d49/d4d/de2/f12f zero size 2024-02-28T23:07:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.427+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c005320 con 0x7fa8bc107e90 2024-02-28T23:07:55.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.432+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7fa8b40113c0 con 0x7fa8bc107e90 2024-02-28T23:07:55.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.433+0000 7fa8b3400700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 0x7fa8a006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:55.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.434+0000 7fa8baa00700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 0x7fa8a006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:55.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.434+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa8b4058db0 con 0x7fa8bc107e90 2024-02-28T23:07:55.440 INFO:tasks.workunit.client.0.smithi057.stdout:7/904: dwrite d0/d3f/d76/dc1/dea/f5b [0,4194304] 0 2024-02-28T23:07:55.440 INFO:tasks.workunit.client.0.smithi057.stdout:3/814: dread d3/d20/d2d/f82 [0,4194304] 0 2024-02-28T23:07:55.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.441+0000 7fa8baa00700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 0x7fa8a006f3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8008b30 tx=0x7fa8a800c330 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:55.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.447+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa8b405c510 con 0x7fa8bc107e90 2024-02-28T23:07:55.453 INFO:tasks.workunit.client.0.smithi057.stdout:5/928: dwrite d4/d9/d14/fec [0,4194304] 0 2024-02-28T23:07:55.467 INFO:tasks.workunit.client.0.smithi057.stdout:5/929: mkdir d4/d7/d58/d121 0 2024-02-28T23:07:55.470 INFO:tasks.workunit.client.0.smithi057.stdout:7/905: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/c12e 0 2024-02-28T23:07:55.472 INFO:tasks.workunit.client.0.smithi057.stdout:6/857: link d0/d1b/d3e/d40/c79 d0/d1b/d3e/c119 0 2024-02-28T23:07:55.473 INFO:tasks.workunit.client.0.smithi057.stdout:3/815: mknod d3/d4/de8/dee/c106 0 2024-02-28T23:07:55.477 INFO:tasks.workunit.client.0.smithi057.stdout:7/906: mknod d0/d3f/d76/dc1/dea/d75/c12f 0 2024-02-28T23:07:55.477 INFO:tasks.workunit.client.0.smithi057.stdout:5/930: read d4/d32/df1/d24/f51 [979639,61857] 0 2024-02-28T23:07:55.490 INFO:tasks.workunit.client.0.smithi057.stdout:7/907: fsync d0/d3f/d76/d118/f124 0 2024-02-28T23:07:55.493 INFO:tasks.workunit.client.0.smithi057.stdout:9/873: dread dd/de/d24/d63/d77/dcf/d40/d44/f62 [0,4194304] 0 2024-02-28T23:07:55.493 INFO:tasks.workunit.client.0.smithi057.stdout:6/858: link d0/d1b/l5f d0/d17/d94/l11a 0 2024-02-28T23:07:55.499 INFO:tasks.workunit.client.0.smithi057.stdout:5/931: link d4/d9/d16/d52/d7a/f7f d4/d9/d16/d52/dfd/da1/d11a/f122 0 2024-02-28T23:07:55.502 INFO:tasks.workunit.client.0.smithi057.stdout:5/932: chown d4/d9/d16/d2c/ldf 0 1 2024-02-28T23:07:55.503 INFO:tasks.workunit.client.0.smithi057.stdout:7/908: mkdir d0/d3f/d76/dc1/d130 0 2024-02-28T23:07:55.513 INFO:tasks.workunit.client.0.smithi057.stdout:5/933: symlink d4/d7/d58/l123 0 2024-02-28T23:07:55.513 INFO:tasks.workunit.client.0.smithi057.stdout:7/909: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f90 [1138624,46227] 0 2024-02-28T23:07:55.520 INFO:tasks.workunit.client.0.smithi057.stdout:5/934: creat d4/d7/d8/f124 x:0 0 0 2024-02-28T23:07:55.520 INFO:tasks.workunit.client.0.smithi057.stdout:3/816: truncate d3/d4/f17 2803054 0 2024-02-28T23:07:55.523 INFO:tasks.workunit.client.0.smithi057.stdout:9/874: write dd/de/d14/f51 [710972,26924] 0 2024-02-28T23:07:55.525 INFO:tasks.workunit.client.0.smithi057.stdout:7/910: read - d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/fc7 zero size 2024-02-28T23:07:55.538 INFO:tasks.workunit.client.0.smithi057.stdout:3/817: write d3/d8/d27/f90 [2553108,26091] 0 2024-02-28T23:07:55.540 INFO:tasks.workunit.client.0.smithi057.stdout:7/911: fsync d0/d3f/d76/dc1/dea/d75/dbf/d10b/f2e 0 2024-02-28T23:07:55.540 INFO:tasks.workunit.client.0.smithi057.stdout:6/859: dwrite d0/d17/d20/fe9 [0,4194304] 0 2024-02-28T23:07:55.556 INFO:tasks.workunit.client.0.smithi057.stdout:6/860: symlink d0/d17/d94/da0/db3/l11b 0 2024-02-28T23:07:55.562 INFO:tasks.workunit.client.0.smithi057.stdout:9/875: dwrite dd/de/d4a/d60/f83 [0,4194304] 0 2024-02-28T23:07:55.570 INFO:tasks.workunit.client.0.smithi057.stdout:3/818: getdents d3/d20/d2d 0 2024-02-28T23:07:55.579 INFO:tasks.workunit.client.0.smithi057.stdout:5/935: dread d4/d9/d16/d52/dfd/da1/f105 [0,4194304] 0 2024-02-28T23:07:55.586 INFO:tasks.workunit.client.0.smithi057.stdout:7/912: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/lff d0/d3f/d76/d107/l131 0 2024-02-28T23:07:55.586 INFO:tasks.workunit.client.0.smithi057.stdout:3/819: truncate d3/d4/d1d/d47/d62/fbe 886486 0 2024-02-28T23:07:55.590 INFO:tasks.workunit.client.0.smithi057.stdout:5/936: mknod d4/d32/df1/d24/d72/c125 0 2024-02-28T23:07:55.597 INFO:tasks.workunit.client.0.smithi057.stdout:5/937: chown d4/d32/df1/d24/d30/lbe 55 1 2024-02-28T23:07:55.598 INFO:tasks.workunit.client.0.smithi057.stdout:7/913: readlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/lce 0 2024-02-28T23:07:55.600 INFO:tasks.workunit.client.0.smithi057.stdout:9/876: dread dd/de/d24/d63/d77/dcf/d40/d44/f7a [0,4194304] 0 2024-02-28T23:07:55.601 INFO:tasks.workunit.client.0.smithi057.stdout:6/861: dwrite d0/d17/d94/da0/de6/df4/d102/d10a/f118 [0,4194304] 0 2024-02-28T23:07:55.604 INFO:tasks.workunit.client.0.smithi057.stdout:5/938: chown d4/d9/d14/d3b/d61/db2 23417 1 2024-02-28T23:07:55.609 INFO:tasks.workunit.client.0.smithi057.stdout:5/939: stat d4/d9/d14/d63/fad 0 2024-02-28T23:07:55.614 INFO:tasks.workunit.client.0.smithi057.stdout:6/862: chown d0/d1b/d3e/d3d/d42/d4f/fbb 14207227 1 2024-02-28T23:07:55.615 INFO:tasks.workunit.client.0.smithi057.stdout:6/863: chown d0/d4/d25/f2a 15777 1 2024-02-28T23:07:55.617 INFO:tasks.workunit.client.0.smithi057.stdout:6/864: chown d0/d17/f82 15 1 2024-02-28T23:07:55.618 INFO:tasks.workunit.client.0.smithi057.stdout:5/940: unlink d4/d32/df1/fd2 0 2024-02-28T23:07:55.623 INFO:tasks.workunit.client.0.smithi057.stdout:9/877: mknod dd/de/d24/d63/d77/dcf/d40/d44/da5/c11c 0 2024-02-28T23:07:55.638 INFO:tasks.workunit.client.0.smithi057.stdout:7/914: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fa8 [0,4194304] 0 2024-02-28T23:07:55.639 INFO:tasks.workunit.client.0.smithi057.stdout:1/832: sync 2024-02-28T23:07:55.639 INFO:tasks.workunit.client.0.smithi057.stdout:9/878: dread dd/d46/dc4/f96 [0,4194304] 0 2024-02-28T23:07:55.648 INFO:tasks.workunit.client.0.smithi057.stdout:5/941: dwrite d4/d9/d14/f103 [0,4194304] 0 2024-02-28T23:07:55.649 INFO:tasks.workunit.client.0.smithi057.stdout:3/820: dread d3/d4/d1d/fcb [0,4194304] 0 2024-02-28T23:07:55.661 INFO:tasks.workunit.client.0.smithi057.stdout:1/833: mknod d2/d33/d54/dc8/dfa/d121/c132 0 2024-02-28T23:07:55.668 INFO:tasks.workunit.client.0.smithi057.stdout:7/915: mknod d0/d3f/d76/dc1/dea/d75/de8/c132 0 2024-02-28T23:07:55.669 INFO:tasks.workunit.client.0.smithi057.stdout:3/821: creat d3/d4/d1d/d47/d62/db4/dfa/f107 x:0 0 0 2024-02-28T23:07:55.671 INFO:tasks.workunit.client.0.smithi057.stdout:7/916: chown d0/d3f/d76/dc1/dea/f100 100718 1 2024-02-28T23:07:55.672 INFO:tasks.workunit.client.0.smithi057.stdout:3/822: mknod d3/d4/d7/d9a/c108 0 2024-02-28T23:07:55.677 INFO:tasks.workunit.client.0.smithi057.stdout:3/823: mknod d3/d4/d7/df8/c109 0 2024-02-28T23:07:55.679 INFO:tasks.workunit.client.0.smithi057.stdout:3/824: chown d3/d4/d7/d9a/cf9 1287689 1 2024-02-28T23:07:55.685 INFO:tasks.workunit.client.0.smithi057.stdout:1/834: dwrite d2/d71/def/ff2 [0,4194304] 0 2024-02-28T23:07:55.689 INFO:tasks.workunit.client.0.smithi057.stdout:3/825: truncate d3/f8e 196334 0 2024-02-28T23:07:55.690 INFO:tasks.workunit.client.0.smithi057.stdout:7/917: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f6a to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/f133 0 2024-02-28T23:07:55.691 INFO:tasks.workunit.client.0.smithi057.stdout:3/826: fdatasync d3/d20/d2d/fe4 0 2024-02-28T23:07:55.699 INFO:tasks.workunit.client.0.smithi057.stdout:6/865: dread d0/d4/d25/d60/f97 [0,4194304] 0 2024-02-28T23:07:55.704 INFO:tasks.workunit.client.0.smithi057.stdout:1/835: write d2/d71/d2d/f31 [756402,116991] 0 2024-02-28T23:07:55.705 INFO:tasks.workunit.client.0.smithi057.stdout:6/866: symlink d0/d17/d94/da0/l11c 0 2024-02-28T23:07:55.710 INFO:tasks.workunit.client.0.smithi057.stdout:6/867: chown d0/d4/d11/lea 1430477517 1 2024-02-28T23:07:55.716 INFO:tasks.workunit.client.0.smithi057.stdout:3/827: dwrite d3/d20/f23 [4194304,4194304] 0 2024-02-28T23:07:55.720 INFO:tasks.workunit.client.0.smithi057.stdout:7/918: dwrite d0/d3f/d76/dc1/dea/d75/f94 [0,4194304] 0 2024-02-28T23:07:55.720 INFO:tasks.workunit.client.0.smithi057.stdout:3/828: fdatasync d3/d8/d27/fbb 0 2024-02-28T23:07:55.722 INFO:tasks.workunit.client.0.smithi057.stdout:6/868: getdents d0/d4/d25/d60/d6a 0 2024-02-28T23:07:55.729 INFO:tasks.workunit.client.0.smithi057.stdout:3/829: creat d3/d4f/d4c/d91/f10a x:0 0 0 2024-02-28T23:07:55.731 INFO:tasks.workunit.client.0.smithi057.stdout:4/905: sync 2024-02-28T23:07:55.739 INFO:tasks.workunit.client.0.smithi057.stdout:7/919: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/f134 x:0 0 0 2024-02-28T23:07:55.739 INFO:tasks.workunit.client.0.smithi057.stdout:6/869: creat d0/d4/f11d x:0 0 0 2024-02-28T23:07:55.742 INFO:tasks.workunit.client.0.smithi057.stdout:6/870: fsync d0/d17/d94/da0/db3/f10c 0 2024-02-28T23:07:55.746 INFO:tasks.workunit.client.0.smithi057.stdout:7/920: creat d0/d3f/d76/d118/f135 x:0 0 0 2024-02-28T23:07:55.746 INFO:tasks.workunit.client.0.smithi057.stdout:3/830: creat d3/d4/d1d/f10b x:0 0 0 2024-02-28T23:07:55.746 INFO:tasks.workunit.client.0.smithi057.stdout:1/836: dread d2/d71/d62/fa6 [0,4194304] 0 2024-02-28T23:07:55.746 INFO:tasks.workunit.client.0.smithi057.stdout:7/921: readlink d0/l33 0 2024-02-28T23:07:55.750 INFO:tasks.workunit.client.0.smithi057.stdout:4/906: truncate d1a/d1e/f25 1341952 0 2024-02-28T23:07:55.754 INFO:tasks.workunit.client.0.smithi057.stdout:3/831: mknod d3/d4/d1d/c10c 0 2024-02-28T23:07:55.761 INFO:tasks.workunit.client.0.smithi057.stdout:6/871: rename d0/f4a to d0/d1b/d3e/d3d/d42/f11e 0 2024-02-28T23:07:55.761 INFO:tasks.workunit.client.0.smithi057.stdout:4/907: truncate d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/f129 1220445 0 2024-02-28T23:07:55.766 INFO:tasks.workunit.client.0.smithi057.stdout:3/832: unlink d3/d4/d7/c32 0 2024-02-28T23:07:55.770 INFO:tasks.workunit.client.0.smithi057.stdout:7/922: truncate d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/ff4 669444 0 2024-02-28T23:07:55.781 INFO:tasks.workunit.client.0.smithi057.stdout:3/833: rename d3/d4f/d4c/fcd to d3/d8/f10d 0 2024-02-28T23:07:55.784 INFO:tasks.workunit.client.0.smithi057.stdout:4/908: dread d1a/d48/d9c/d2e/d8f/fe5 [0,4194304] 0 2024-02-28T23:07:55.786 INFO:tasks.workunit.client.0.smithi057.stdout:1/837: dwrite d2/f10 [0,4194304] 0 2024-02-28T23:07:55.805 INFO:tasks.workunit.client.0.smithi057.stdout:4/909: truncate d1a/d2f/db2/d49/d4d/de2/f53 1702346 0 2024-02-28T23:07:55.806 INFO:tasks.workunit.client.0.smithi057.stdout:6/872: dwrite d0/d17/d20/f64 [0,4194304] 0 2024-02-28T23:07:55.808 INFO:tasks.workunit.client.0.smithi057.stdout:7/923: getdents d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e 0 2024-02-28T23:07:55.809 INFO:tasks.workunit.client.0.smithi057.stdout:4/910: chown d1a/d2f/db2 7831852 1 2024-02-28T23:07:55.811 INFO:tasks.workunit.client.0.smithi057.stdout:3/834: dwrite d3/d20/f23 [4194304,4194304] 0 2024-02-28T23:07:55.815 INFO:tasks.workunit.client.0.smithi057.stdout:6/873: chown d0/d1b/d3e/d3d/d42/d4f/l7f 0 1 2024-02-28T23:07:55.820 INFO:tasks.workunit.client.0.smithi057.stdout:1/838: getdents d2/d7/d4f 0 2024-02-28T23:07:55.830 INFO:tasks.workunit.client.0.smithi057.stdout:3/835: unlink d3/d4/d7/df8/lff 0 2024-02-28T23:07:55.839 INFO:tasks.workunit.client.0.smithi057.stdout:6/874: rename d0/d4/d25/d60/d6a/ldc to d0/d4/d25/d60/l11f 0 2024-02-28T23:07:55.839 INFO:tasks.workunit.client.0.smithi057.stdout:7/924: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/f2e [0,4194304] 0 2024-02-28T23:07:55.839 INFO:tasks.workunit.client.0.smithi057.stdout:1/839: write d2/d71/d2d/f4c [2074007,35304] 0 2024-02-28T23:07:55.842 INFO:tasks.workunit.client.0.smithi057.stdout:1/840: dread - d2/d33/d54/dc8/ffe zero size 2024-02-28T23:07:55.847 INFO:tasks.workunit.client.0.smithi057.stdout:3/836: write d3/d8/d27/f81 [1214948,94363] 0 2024-02-28T23:07:55.849 INFO:tasks.workunit.client.0.smithi057.stdout:7/925: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/l136 0 2024-02-28T23:07:55.851 INFO:tasks.workunit.client.0.smithi057.stdout:1/841: dread - d2/d7/df/db/d10b/fec zero size 2024-02-28T23:07:55.860 INFO:tasks.workunit.client.0.smithi057.stdout:1/842: chown d2/l53 10227 1 2024-02-28T23:07:55.862 INFO:tasks.workunit.client.0.smithi057.stdout:7/926: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f137 x:0 0 0 2024-02-28T23:07:55.867 INFO:tasks.workunit.client.0.smithi057.stdout:4/911: dread d1a/d1e/fa8 [0,4194304] 0 2024-02-28T23:07:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.869+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa89c000bf0 con 0x7fa8a006cf00 2024-02-28T23:07:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.871+0000 7fa8b3400700 1 -- 172.21.15.57:0/447551991 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa89c000bf0 con 0x7fa8a006cf00 2024-02-28T23:07:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.875+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 msgr2=0x7fa8a006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.875+0000 7fa8c1800700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 0x7fa8a006f3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8008b30 tx=0x7fa8a800c330 comp rx=0 tx=0).stop 2024-02-28T23:07:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.875+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 msgr2=0x7fa8bc19c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:55.876 INFO:tasks.workunit.client.0.smithi057.stdout:7/927: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/ff4 [0,4194304] 0 2024-02-28T23:07:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.875+0000 7fa8c1800700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4007ed0 tx=0x7fa8b400f5c0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 shutdown_connections 2024-02-28T23:07:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc071ec0 0x7fa8bc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fa8a006cf00 0x7fa8a006f3b0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8008b30 tx=0x7fa8a800c330 comp rx=0 tx=0).stop 2024-02-28T23:07:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 --2- 172.21.15.57:0/447551991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8bc107e90 0x7fa8bc19c5f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 >> 172.21.15.57:0/447551991 conn(0x7fa8bc06d110 msgr2=0x7fa8bc10b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.876+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 shutdown_connections 2024-02-28T23:07:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:55.878+0000 7fa8c1800700 1 -- 172.21.15.57:0/447551991 wait complete. 2024-02-28T23:07:55.879 INFO:tasks.workunit.client.0.smithi057.stdout:3/837: dwrite d3/d4/d7/d9a/d9b/dab/dd0/fe7 [0,4194304] 0 2024-02-28T23:07:55.885 INFO:tasks.workunit.client.0.smithi057.stdout:1/843: dread d2/d7/df/db/d3a/d2a/d82/fe3 [0,4194304] 0 2024-02-28T23:07:55.896 INFO:tasks.workunit.client.0.smithi057.stdout:7/928: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f138 x:0 0 0 2024-02-28T23:07:55.897 INFO:tasks.workunit.client.0.smithi057.stdout:7/929: readlink d0/l33 0 2024-02-28T23:07:55.906 INFO:tasks.workunit.client.0.smithi057.stdout:4/912: dread d1a/d2f/db2/d49/d4d/dc9/dff/d125/f32 [0,4194304] 0 2024-02-28T23:07:55.911 INFO:tasks.workunit.client.0.smithi057.stdout:1/844: write d2/d7/df/db/d12/d3f/d47/f8e [1016011,82436] 0 2024-02-28T23:07:55.913 INFO:tasks.workunit.client.0.smithi057.stdout:4/913: rmdir d1a/d48/d9c/d2e/df8/d132 0 2024-02-28T23:07:55.916 INFO:tasks.workunit.client.0.smithi057.stdout:1/845: symlink d2/d33/d54/dc8/dfa/l133 0 2024-02-28T23:07:55.917 INFO:tasks.workunit.client.0.smithi057.stdout:4/914: mknod d1a/c134 0 2024-02-28T23:07:55.919 INFO:tasks.workunit.client.0.smithi057.stdout:7/930: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/f1e [0,4194304] 0 2024-02-28T23:07:55.925 INFO:tasks.workunit.client.0.smithi057.stdout:1/846: dread d2/d71/d2d/f49 [0,4194304] 0 2024-02-28T23:07:55.929 INFO:tasks.workunit.client.0.smithi057.stdout:1/847: symlink d2/d7/df/db/d12/d19/l134 0 2024-02-28T23:07:55.932 INFO:tasks.workunit.client.0.smithi057.stdout:1/848: chown d2/d7/df/db/d3a/d5a/dd3 788283 1 2024-02-28T23:07:55.938 INFO:tasks.workunit.client.0.smithi057.stdout:1/849: rename d2/d7/df/db/d10b/d6b/f77 to d2/d71/d2d/d5b/f135 0 2024-02-28T23:07:55.940 INFO:tasks.workunit.client.0.smithi057.stdout:1/850: readlink d2/d7/df/db/d12/d3f/d47/l86 0 2024-02-28T23:07:55.943 INFO:tasks.workunit.client.0.smithi057.stdout:1/851: write d2/d71/d2d/d39/da3/db5/df7/f125 [113779,79785] 0 2024-02-28T23:07:55.947 INFO:tasks.workunit.client.0.smithi057.stdout:1/852: rename d2/d33/d54/dc8/dd2/f12c to d2/d7/df/db/d3a/f136 0 2024-02-28T23:07:55.950 INFO:tasks.workunit.client.0.smithi057.stdout:1/853: mkdir d2/d7/df/db/d10b/d137 0 2024-02-28T23:07:55.952 INFO:tasks.workunit.client.0.smithi057.stdout:1/854: write d2/d7/df/db/d3a/d8f/f10d [1905,20441] 0 2024-02-28T23:07:55.980 INFO:tasks.workunit.client.0.smithi057.stdout:1/855: dread d2/d7/df/db/d12/d19/f9d [0,4194304] 0 2024-02-28T23:07:55.983 INFO:tasks.workunit.client.0.smithi057.stdout:1/856: creat d2/d71/d2d/d39/f138 x:0 0 0 2024-02-28T23:07:55.998 INFO:tasks.workunit.client.0.smithi057.stdout:1/857: dwrite d2/d33/d54/dc8/ffe [0,4194304] 0 2024-02-28T23:07:56.004 INFO:tasks.workunit.client.0.smithi057.stdout:1/858: creat d2/dd4/dd5/d118/f139 x:0 0 0 2024-02-28T23:07:56.009 INFO:tasks.workunit.client.0.smithi057.stdout:1/859: unlink d2/d7/df/db/d12/d19/d65/f98 0 2024-02-28T23:07:56.012 INFO:tasks.workunit.client.0.smithi057.stdout:1/860: creat d2/d71/d2d/d39/da3/f13a x:0 0 0 2024-02-28T23:07:56.013 INFO:tasks.workunit.client.0.smithi057.stdout:1/861: chown d2/d7/df/db/d3a/c78 260367 1 2024-02-28T23:07:56.016 INFO:tasks.workunit.client.0.smithi057.stdout:1/862: mknod d2/d7/df/db/d12/dd6/c13b 0 2024-02-28T23:07:56.030 INFO:tasks.workunit.client.0.smithi057.stdout:1/863: write d2/d71/d2d/fbf [756508,72063] 0 2024-02-28T23:07:56.032 INFO:tasks.workunit.client.0.smithi057.stdout:1/864: chown d2/dd4/dd5/f102 817943 1 2024-02-28T23:07:56.034 INFO:tasks.workunit.client.0.smithi057.stdout:1/865: dread - d2/dd4/f111 zero size 2024-02-28T23:07:56.036 INFO:tasks.workunit.client.0.smithi057.stdout:1/866: fsync d2/d71/d2d/d39/fd1 0 2024-02-28T23:07:56.039 INFO:tasks.workunit.client.0.smithi057.stdout:1/867: dread - d2/d7/df/db/d10b/fec zero size 2024-02-28T23:07:56.044 INFO:tasks.workunit.client.0.smithi057.stdout:1/868: read d2/d7/df/db/d10b/d6b/fb9 [214565,60926] 0 2024-02-28T23:07:56.047 INFO:tasks.workunit.client.0.smithi057.stdout:1/869: readlink d2/d7/df/db/d12/dd6/l120 0 2024-02-28T23:07:56.054 INFO:tasks.workunit.client.0.smithi057.stdout:1/870: write d2/d7/df/db/d3a/fc9 [1080862,78781] 0 2024-02-28T23:07:56.064 INFO:tasks.workunit.client.0.smithi057.stdout:1/871: fdatasync d2/d7/df/db/d3a/d2a/f83 0 2024-02-28T23:07:56.065 INFO:tasks.workunit.client.0.smithi057.stdout:1/872: fsync d2/d71/d2d/d5b/fe7 0 2024-02-28T23:07:56.065 INFO:tasks.workunit.client.0.smithi057.stdout:1/873: truncate d2/f8 4880320 0 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 -- 172.21.15.57:0/1902526591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 msgr2=0x7fbb40105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 --2- 172.21.15.57:0/1902526591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb40105a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009870 tx=0x7fbb2c009b80 comp rx=0 tx=0).stop 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 -- 172.21.15.57:0/1902526591 shutdown_connections 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 --2- 172.21.15.57:0/1902526591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb40105a60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 --2- 172.21.15.57:0/1902526591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 0x7fbb40103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 -- 172.21.15.57:0/1902526591 >> 172.21.15.57:0/1902526591 conn(0x7fbb400fa780 msgr2=0x7fbb400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 -- 172.21.15.57:0/1902526591 shutdown_connections 2024-02-28T23:07:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.064+0000 7fbb45a00700 1 -- 172.21.15.57:0/1902526591 wait complete. 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 Processor -- start 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 -- start start 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 0x7fbb401006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400ff250 con 0x7fbb40103680 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.065+0000 7fbb45a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400ff390 con 0x7fbb40100d60 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59406/0 (socket says 172.21.15.57:59406) 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 -- 172.21.15.57:0/2049352280 learned_addr learned my addr 172.21.15.57:0/2049352280 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3ea00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 0x7fbb401006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 -- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 msgr2=0x7fbb401006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 0x7fbb401006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 -- 172.21.15.57:0/2049352280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009520 con 0x7fbb40103680 2024-02-28T23:07:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.066+0000 7fbb3e000700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009f50 tx=0x7fbb2c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.067+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2c01c070 con 0x7fbb40103680 2024-02-28T23:07:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.067+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb2c00be80 con 0x7fbb40103680 2024-02-28T23:07:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.067+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2c0173f0 con 0x7fbb40103680 2024-02-28T23:07:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.067+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb400ff610 con 0x7fbb40103680 2024-02-28T23:07:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.067+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb400ffab0 con 0x7fbb40103680 2024-02-28T23:07:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.068+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4018fbe0 con 0x7fbb40103680 2024-02-28T23:07:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.069+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7fbb2c020070 con 0x7fbb40103680 2024-02-28T23:07:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.070+0000 7fbb37400700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 0x7fbb2406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.070+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbb2c08da00 con 0x7fbb40103680 2024-02-28T23:07:56.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.071+0000 7fbb3ea00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 0x7fbb2406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.072+0000 7fbb3ea00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 0x7fbb2406f3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb40069000 tx=0x7fbb30009310 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:56.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.076+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fbb2c05b1c0 con 0x7fbb40103680 2024-02-28T23:07:56.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:56 smithi057 ceph-mon[29096]: pgmap v34: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 134 MiB/s rd, 301 MiB/s wr, 693 op/s 2024-02-28T23:07:56.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:56 smithi057 ceph-mon[29096]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:3/838: sync 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:9/879: sync 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:6/875: sync 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:1/874: sync 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:7/931: sync 2024-02-28T23:07:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:4/915: sync 2024-02-28T23:07:56.320 INFO:tasks.workunit.client.0.smithi057.stdout:5/942: sync 2024-02-28T23:07:56.323 INFO:tasks.workunit.client.0.smithi057.stdout:4/916: chown d1a/d1e/f42 13131357 1 2024-02-28T23:07:56.326 INFO:tasks.workunit.client.0.smithi057.stdout:5/943: fdatasync d4/d9/d16/feb 0 2024-02-28T23:07:56.328 INFO:tasks.workunit.client.0.smithi057.stdout:9/880: creat dd/d46/dc4/de5/f11d x:0 0 0 2024-02-28T23:07:56.330 INFO:tasks.workunit.client.0.smithi057.stdout:1/875: fsync d2/d33/f76 0 2024-02-28T23:07:56.337 INFO:tasks.workunit.client.0.smithi057.stdout:3/839: rename d3/d4f/d4c/ld9 to d3/d4f/d4c/d91/de2/l10e 0 2024-02-28T23:07:56.341 INFO:tasks.workunit.client.0.smithi057.stdout:7/932: read d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d125/f54 [325844,82915] 0 2024-02-28T23:07:56.348 INFO:tasks.workunit.client.0.smithi057.stdout:6/876: creat d0/d4/d25/d60/f120 x:0 0 0 2024-02-28T23:07:56.358 INFO:tasks.workunit.client.0.smithi057.stdout:1/876: dwrite d2/dd4/f111 [0,4194304] 0 2024-02-28T23:07:56.359 INFO:tasks.workunit.client.0.smithi057.stdout:6/877: rename d0/d4/d25/d60/dc9/d10e to d0/d17/d94/da0/d121 0 2024-02-28T23:07:56.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.360+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb40061190 con 0x7fbb2406cf00 2024-02-28T23:07:56.360 INFO:tasks.workunit.client.0.smithi057.stdout:3/840: getdents d3/d20 0 2024-02-28T23:07:56.363 INFO:tasks.workunit.client.0.smithi057.stdout:6/878: fdatasync d0/d17/d94/f101 0 2024-02-28T23:07:56.364 INFO:tasks.workunit.client.0.smithi057.stdout:1/877: mkdir d2/d71/d2d/d39/da3/db5/d13c 0 2024-02-28T23:07:56.366 INFO:tasks.workunit.client.0.smithi057.stdout:3/841: write d3/ff5 [2106778,43995] 0 2024-02-28T23:07:56.367 INFO:tasks.workunit.client.0.smithi057.stdout:1/878: write d2/d71/d2d/d5b/fe7 [4048164,96237] 0 2024-02-28T23:07:56.368 INFO:tasks.workunit.client.0.smithi057.stdout:6/879: creat d0/d1b/d3e/d100/f122 x:0 0 0 2024-02-28T23:07:56.369 INFO:tasks.workunit.client.0.smithi057.stdout:3/842: stat d3/d20/d2d/c34 0 2024-02-28T23:07:56.370 INFO:tasks.workunit.client.0.smithi057.stdout:1/879: readlink d2/d71/d2d/la1 0 2024-02-28T23:07:56.371 INFO:tasks.workunit.client.0.smithi057.stdout:3/843: readlink d3/d4/d7/l5d 0 2024-02-28T23:07:56.377 INFO:tasks.workunit.client.0.smithi057.stdout:7/933: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d125/f5a [0,4194304] 0 2024-02-28T23:07:56.387 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:07:56.387 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (9m) 47s ago 11m 14.0M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (7m) 47s ago 11m 7822k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (10m) 50s ago 10m 7730k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (11m) 47s ago 11m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (10m) 50s ago 10m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (9m) 47s ago 11m 73.7M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (4m) 47s ago 4m 54.0M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (4m) 47s ago 4m 92.0M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (4m) 50s ago 3m 50.8M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (4m) 50s ago 4m 91.1M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:9283,8765,8443 running (13m) 47s ago 13m 333M - 18.2.0 dc2bc1663786 7afdee645aa5 2024-02-28T23:07:56.388 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (82s) 50s ago 9m 487M - 19.0.0-1670-g1fb5d106 2272b2c42f9b ae1c65e2de51 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (13m) 47s ago 13m 48.4M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (9m) 50s ago 9m 39.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (11m) 47s ago 11m 7642k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (10m) 50s ago 10m 7750k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (8m) 47s ago 8m 101M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (8m) 47s ago 8m 115M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (7m) 47s ago 7m 95.9M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (7m) 50s ago 7m 117M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (6m) 50s ago 6m 112M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (5m) 50s ago 5m 101M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:07:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 starting - - - - 2024-02-28T23:07:56.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.384+0000 7fbb37400700 1 -- 172.21.15.57:0/2049352280 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbb40061190 con 0x7fbb2406cf00 2024-02-28T23:07:56.390 INFO:tasks.workunit.client.0.smithi057.stdout:5/944: write d4/fb9 [597669,54102] 0 2024-02-28T23:07:56.390 INFO:tasks.workunit.client.0.smithi057.stdout:4/917: write d1a/d2f/db2/f59 [313952,59727] 0 2024-02-28T23:07:56.391 INFO:tasks.workunit.client.0.smithi057.stdout:3/844: rmdir d3/d4f/d4c/d91 39 2024-02-28T23:07:56.391 INFO:tasks.workunit.client.0.smithi057.stdout:9/881: write dd/de/f4f [164815,116329] 0 2024-02-28T23:07:56.391 INFO:tasks.workunit.client.0.smithi057.stdout:5/945: truncate d4/d32/df1/f76 777653 0 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 msgr2=0x7fbb2406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 0x7fbb2406f3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb40069000 tx=0x7fbb30009310 comp rx=0 tx=0).stop 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 msgr2=0x7fbb400fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009f50 tx=0x7fbb2c00f740 comp rx=0 tx=0).stop 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 shutdown_connections 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb40100d60 0x7fbb401006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7fbb2406cf00 0x7fbb2406f3b0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb40069000 tx=0x7fbb30009310 comp rx=0 tx=0).stop 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 --2- 172.21.15.57:0/2049352280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb40103680 0x7fbb400fed10 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009f50 tx=0x7fbb2c00f740 comp rx=0 tx=0).stop 2024-02-28T23:07:56.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.393+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 >> 172.21.15.57:0/2049352280 conn(0x7fbb400fa780 msgr2=0x7fbb400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:56.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.394+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 shutdown_connections 2024-02-28T23:07:56.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.394+0000 7fbb45a00700 1 -- 172.21.15.57:0/2049352280 wait complete. 2024-02-28T23:07:56.396 INFO:tasks.workunit.client.0.smithi057.stdout:5/946: rmdir d4/d32/df1/d24/da2 39 2024-02-28T23:07:56.399 INFO:tasks.workunit.client.0.smithi057.stdout:3/845: write d3/d4/d7/d9a/d9b/fd7 [1012999,49918] 0 2024-02-28T23:07:56.400 INFO:tasks.workunit.client.0.smithi057.stdout:9/882: rename dd/de/d14/d48/d4b/cb5 to dd/de/d4a/d60/d8c/c11e 0 2024-02-28T23:07:56.403 INFO:tasks.workunit.client.0.smithi057.stdout:5/947: chown d4/d9/d16/d52/fe1 232 1 2024-02-28T23:07:56.405 INFO:tasks.workunit.client.0.smithi057.stdout:6/880: dwrite d0/d1b/d3e/d40/f73 [0,4194304] 0 2024-02-28T23:07:56.409 INFO:tasks.workunit.client.0.smithi057.stdout:4/918: dwrite d1a/d1e/f5b [4194304,4194304] 0 2024-02-28T23:07:56.410 INFO:tasks.workunit.client.0.smithi057.stdout:9/883: rmdir dd/de/d24/d63/d77/dcf/d40 39 2024-02-28T23:07:56.419 INFO:tasks.workunit.client.0.smithi057.stdout:3/846: rename d3/d4/d1d/d47/d62/db4/dfa/f107 to d3/d4/d1d/d47/d62/db4/dfa/f10f 0 2024-02-28T23:07:56.419 INFO:tasks.workunit.client.0.smithi057.stdout:5/948: creat d4/d9/d14/d3b/d61/dfa/d110/f126 x:0 0 0 2024-02-28T23:07:56.422 INFO:tasks.workunit.client.0.smithi057.stdout:3/847: write d3/d20/f23 [6944181,105536] 0 2024-02-28T23:07:56.429 INFO:tasks.workunit.client.0.smithi057.stdout:3/848: creat d3/d4f/d4c/f110 x:0 0 0 2024-02-28T23:07:56.432 INFO:tasks.workunit.client.0.smithi057.stdout:6/881: write d0/d1b/d3e/d3d/f55 [4482818,5009] 0 2024-02-28T23:07:56.435 INFO:tasks.workunit.client.0.smithi057.stdout:6/882: write d0/d4/f11d [538263,39427] 0 2024-02-28T23:07:56.437 INFO:tasks.workunit.client.0.smithi057.stdout:3/849: creat d3/d4f/d4c/f111 x:0 0 0 2024-02-28T23:07:56.440 INFO:tasks.workunit.client.0.smithi057.stdout:3/850: chown d3/d4/de8 6 1 2024-02-28T23:07:56.441 INFO:tasks.workunit.client.0.smithi057.stdout:9/884: write dd/de/d4a/d60/d8c/fda [888687,115652] 0 2024-02-28T23:07:56.441 INFO:tasks.workunit.client.0.smithi057.stdout:4/919: write d1a/d48/f79 [259847,85525] 0 2024-02-28T23:07:56.442 INFO:tasks.workunit.client.0.smithi057.stdout:6/883: creat d0/d1b/d3e/d100/f123 x:0 0 0 2024-02-28T23:07:56.444 INFO:tasks.workunit.client.0.smithi057.stdout:9/885: readlink dd/de/l84 0 2024-02-28T23:07:56.451 INFO:tasks.workunit.client.0.smithi057.stdout:3/851: mkdir d3/d4/d1d/d47/d100/d112 0 2024-02-28T23:07:56.453 INFO:tasks.workunit.client.0.smithi057.stdout:4/920: chown d1a/d2f/db2/d49/d4d/dc9/dff/d125/d7f/fa5 20358777 1 2024-02-28T23:07:56.456 INFO:tasks.workunit.client.0.smithi057.stdout:5/949: dread d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:56.457 INFO:tasks.workunit.client.0.smithi057.stdout:4/921: readlink d1a/d48/d9c/d2e/d43/l85 0 2024-02-28T23:07:56.460 INFO:tasks.workunit.client.0.smithi057.stdout:5/950: fsync d4/d32/df1/d24/d30/d62/f113 0 2024-02-28T23:07:56.461 INFO:tasks.workunit.client.0.smithi057.stdout:9/886: link dd/d46/d6f/f79 dd/de/d24/d63/d77/dcf/d40/d44/da5/de4/f11f 0 2024-02-28T23:07:56.463 INFO:tasks.workunit.client.0.smithi057.stdout:3/852: link d3/d4/dfd/l103 d3/d4/d1d/d47/d100/d112/l113 0 2024-02-28T23:07:56.464 INFO:tasks.workunit.client.0.smithi057.stdout:4/922: mknod d1a/d2f/db2/dcb/dfe/c135 0 2024-02-28T23:07:56.466 INFO:tasks.workunit.client.0.smithi057.stdout:9/887: symlink dd/d46/dc4/l120 0 2024-02-28T23:07:56.466 INFO:tasks.workunit.client.0.smithi057.stdout:5/951: write d4/d9/d16/d52/dfd/da1/f105 [2650412,81763] 0 2024-02-28T23:07:56.472 INFO:tasks.workunit.client.0.smithi057.stdout:3/853: creat d3/d4/d7/d9a/d9b/dab/dd0/f114 x:0 0 0 2024-02-28T23:07:56.476 INFO:tasks.workunit.client.0.smithi057.stdout:6/884: dread d0/d1b/d54/fdf [0,4194304] 0 2024-02-28T23:07:56.476 INFO:tasks.workunit.client.0.smithi057.stdout:5/952: mkdir d4/d9/d16/d52/dfd/da1/d127 0 2024-02-28T23:07:56.477 INFO:tasks.workunit.client.0.smithi057.stdout:3/854: creat d3/d20/f115 x:0 0 0 2024-02-28T23:07:56.479 INFO:tasks.workunit.client.0.smithi057.stdout:6/885: unlink d0/d17/l1a 0 2024-02-28T23:07:56.480 INFO:tasks.workunit.client.0.smithi057.stdout:9/888: link dd/d46/c7b dd/d46/dc4/c121 0 2024-02-28T23:07:56.482 INFO:tasks.workunit.client.0.smithi057.stdout:5/953: dread - d4/d9/d14/d3b/d61/db2/fe9 zero size 2024-02-28T23:07:56.485 INFO:tasks.workunit.client.0.smithi057.stdout:3/855: truncate d3/d8/d27/f5f 3540607 0 2024-02-28T23:07:56.487 INFO:tasks.workunit.client.0.smithi057.stdout:4/923: write d1a/d48/d9c/d2e/df8/f11a [1109525,5560] 0 2024-02-28T23:07:56.488 INFO:tasks.workunit.client.0.smithi057.stdout:5/954: symlink d4/d9/d14/d3b/df3/l128 0 2024-02-28T23:07:56.490 INFO:tasks.workunit.client.0.smithi057.stdout:3/856: fsync d3/d4f/d4c/f54 0 2024-02-28T23:07:56.491 INFO:tasks.workunit.client.0.smithi057.stdout:3/857: chown d3/d20/d2d 17305 1 2024-02-28T23:07:56.494 INFO:tasks.workunit.client.0.smithi057.stdout:4/924: truncate d1a/d1e/fe6 1937714 0 2024-02-28T23:07:56.494 INFO:tasks.workunit.client.0.smithi057.stdout:3/858: chown d3/d8/d27/d70/f9d 122004422 1 2024-02-28T23:07:56.499 INFO:tasks.workunit.client.0.smithi057.stdout:9/889: getdents dd/de/d24/d63/d77/dcf 0 2024-02-28T23:07:56.504 INFO:tasks.workunit.client.0.smithi057.stdout:4/925: symlink d1a/d2f/db2/d49/d4d/dc9/dff/l136 0 2024-02-28T23:07:56.507 INFO:tasks.workunit.client.0.smithi057.stdout:9/890: creat dd/de/d24/d63/d77/dcf/d40/d44/d5c/f122 x:0 0 0 2024-02-28T23:07:56.512 INFO:tasks.workunit.client.0.smithi057.stdout:5/955: dwrite d4/d9/d16/d4e/f111 [0,4194304] 0 2024-02-28T23:07:56.517 INFO:tasks.workunit.client.0.smithi057.stdout:9/891: write dd/de/d24/d63/d77/dcf/d40/d44/d5c/fa7 [1012185,115235] 0 2024-02-28T23:07:56.523 INFO:tasks.workunit.client.0.smithi057.stdout:6/886: dwrite d0/d17/ff3 [0,4194304] 0 2024-02-28T23:07:56.532 INFO:tasks.workunit.client.0.smithi057.stdout:9/892: link dd/de/d4a/d60/f7e dd/de/d24/d63/d77/dcf/d40/d44/d5c/f123 0 2024-02-28T23:07:56.535 INFO:tasks.workunit.client.0.smithi057.stdout:3/859: dread d3/d8/f5e [0,4194304] 0 2024-02-28T23:07:56.536 INFO:tasks.workunit.client.0.smithi057.stdout:9/893: creat dd/d46/f124 x:0 0 0 2024-02-28T23:07:56.537 INFO:tasks.workunit.client.0.smithi057.stdout:9/894: write dd/de/d14/f8e [4426183,98395] 0 2024-02-28T23:07:56.538 INFO:tasks.workunit.client.0.smithi057.stdout:5/956: dread d4/d9/d85/fce [0,4194304] 0 2024-02-28T23:07:56.544 INFO:tasks.workunit.client.0.smithi057.stdout:9/895: dread dd/de/d14/d48/d4b/d41/f93 [0,4194304] 0 2024-02-28T23:07:56.545 INFO:tasks.workunit.client.0.smithi057.stdout:6/887: truncate d0/d4/f50 489478 0 2024-02-28T23:07:56.546 INFO:tasks.workunit.client.0.smithi057.stdout:9/896: chown dd/d46/d6f/cae 55 1 2024-02-28T23:07:56.547 INFO:tasks.workunit.client.0.smithi057.stdout:5/957: creat d4/d32/df1/d24/da2/dbb/d95/f129 x:0 0 0 2024-02-28T23:07:56.548 INFO:tasks.workunit.client.0.smithi057.stdout:6/888: dread - d0/d1b/d3e/d3d/d42/f8f zero size 2024-02-28T23:07:56.548 INFO:tasks.workunit.client.0.smithi057.stdout:3/860: dread d3/f5a [0,4194304] 0 2024-02-28T23:07:56.549 INFO:tasks.workunit.client.0.smithi057.stdout:5/958: chown d4/d9/d14/d3b/df3/l128 24625503 1 2024-02-28T23:07:56.550 INFO:tasks.workunit.client.0.smithi057.stdout:6/889: readlink d0/d4/d25/d60/dc9/l112 0 2024-02-28T23:07:56.552 INFO:tasks.workunit.client.0.smithi057.stdout:3/861: write d3/d4/d7/f10 [8249616,104710] 0 2024-02-28T23:07:56.554 INFO:tasks.workunit.client.0.smithi057.stdout:5/959: chown d4/d9/d16/d4e/f87 3325671 1 2024-02-28T23:07:56.555 INFO:tasks.workunit.client.0.smithi057.stdout:3/862: stat d3/d8/d27/lbd 0 2024-02-28T23:07:56.556 INFO:tasks.workunit.client.0.smithi057.stdout:9/897: rename dd/de/d4a/d60/d8c/fda to dd/de/d14/d48/d76/f125 0 2024-02-28T23:07:56.558 INFO:tasks.workunit.client.0.smithi057.stdout:5/960: chown d4/d9/d16/d52/d7a/cfb 506681202 1 2024-02-28T23:07:56.559 INFO:tasks.workunit.client.0.smithi057.stdout:3/863: read d3/d8/d1b/f4a [155810,9225] 0 2024-02-28T23:07:56.560 INFO:tasks.workunit.client.0.smithi057.stdout:6/890: symlink d0/d4/d25/d60/dc9/l124 0 2024-02-28T23:07:56.565 INFO:tasks.workunit.client.0.smithi057.stdout:5/961: creat d4/d9/d14/d3b/d61/dfa/d110/f12a x:0 0 0 2024-02-28T23:07:56.566 INFO:tasks.workunit.client.0.smithi057.stdout:3/864: rename d3/l26 to d3/d4/d7/df8/l116 0 2024-02-28T23:07:56.571 INFO:tasks.workunit.client.0.smithi057.stdout:6/891: link d0/d17/d94/l108 d0/d4/d25/d60/dc9/l125 0 2024-02-28T23:07:56.572 INFO:tasks.workunit.client.0.smithi057.stdout:5/962: dread d4/f7d [0,4194304] 0 2024-02-28T23:07:56.574 INFO:tasks.workunit.client.0.smithi057.stdout:6/892: write d0/d1b/d3e/d3d/d74/fb8 [776925,89936] 0 2024-02-28T23:07:56.575 INFO:tasks.workunit.client.0.smithi057.stdout:5/963: write d4/d9/d16/d52/dfd/f10a [205198,31159] 0 2024-02-28T23:07:56.576 INFO:tasks.workunit.client.0.smithi057.stdout:3/865: link d3/d8/d27/f5f d3/d4/d1d/d47/d62/f117 0 2024-02-28T23:07:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:56 smithi084 ceph-mon[34826]: pgmap v34: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 134 MiB/s rd, 301 MiB/s wr, 693 op/s 2024-02-28T23:07:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:56 smithi084 ceph-mon[34826]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:56.580 INFO:tasks.workunit.client.0.smithi057.stdout:6/893: truncate d0/d17/f82 5195526 0 2024-02-28T23:07:56.586 INFO:tasks.workunit.client.0.smithi057.stdout:6/894: unlink d0/d17/d94/da0/de6/df4/d102/d10a/f118 0 2024-02-28T23:07:56.595 INFO:tasks.workunit.client.0.smithi057.stdout:3/866: write d3/d8/d27/d70/fd8 [242670,22550] 0 2024-02-28T23:07:56.598 INFO:tasks.workunit.client.0.smithi057.stdout:6/895: write d0/d4/f19 [2146937,82796] 0 2024-02-28T23:07:56.601 INFO:tasks.workunit.client.0.smithi057.stdout:3/867: truncate d3/d4f/f48 3105277 0 2024-02-28T23:07:56.601 INFO:tasks.workunit.client.0.smithi057.stdout:6/896: chown d0/d17/d94/da0/de6/df4/d102/d10a 66527459 1 2024-02-28T23:07:56.605 INFO:tasks.workunit.client.0.smithi057.stdout:5/964: dwrite d4/d7/f8d [0,4194304] 0 2024-02-28T23:07:56.608 INFO:tasks.workunit.client.0.smithi057.stdout:5/965: symlink d4/d32/df1/d24/d30/d62/l12b 0 2024-02-28T23:07:56.611 INFO:tasks.workunit.client.0.smithi057.stdout:5/966: fdatasync d4/d9/d14/d3b/d61/dfa/d110/f12a 0 2024-02-28T23:07:56.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.617+0000 7f0baa800700 1 -- 172.21.15.57:0/304901698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba40feb50 msgr2=0x7f0ba4100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.617+0000 7f0baa800700 1 --2- 172.21.15.57:0/304901698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba40feb50 0x7f0ba4100f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009910 tx=0x7f0b94009c20 comp rx=0 tx=0).stop 2024-02-28T23:07:56.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.619+0000 7f0baa800700 1 -- 172.21.15.57:0/304901698 shutdown_connections 2024-02-28T23:07:56.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.619+0000 7f0baa800700 1 --2- 172.21.15.57:0/304901698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba41014b0 0x7f0ba4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.619+0000 7f0baa800700 1 --2- 172.21.15.57:0/304901698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba40feb50 0x7f0ba4100f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.619+0000 7f0baa800700 1 -- 172.21.15.57:0/304901698 >> 172.21.15.57:0/304901698 conn(0x7f0ba40fa780 msgr2=0x7f0ba40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:56.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.620+0000 7f0baa800700 1 -- 172.21.15.57:0/304901698 shutdown_connections 2024-02-28T23:07:56.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.620+0000 7f0baa800700 1 -- 172.21.15.57:0/304901698 wait complete. 2024-02-28T23:07:56.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.621+0000 7f0baa800700 1 Processor -- start 2024-02-28T23:07:56.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.621+0000 7f0baa800700 1 -- start start 2024-02-28T23:07:56.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0baa800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba40feb50 0x7f0ba419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0baa800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0baa800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba419cc10 con 0x7f0ba41014b0 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0baa800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba419cd50 con 0x7f0ba40feb50 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0b9be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0b9be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59426/0 (socket says 172.21.15.57:59426) 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0b9be00700 1 -- 172.21.15.57:0/523754242 learned_addr learned my addr 172.21.15.57:0/523754242 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.622+0000 7f0ba3400700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba40feb50 0x7f0ba419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.623+0000 7f0b9be00700 1 -- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba40feb50 msgr2=0x7f0ba419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:56.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.623+0000 7f0b9be00700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba40feb50 0x7f0ba419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:56.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.623+0000 7f0b9be00700 1 -- 172.21.15.57:0/523754242 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940095f0 con 0x7f0ba41014b0 2024-02-28T23:07:56.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.623+0000 7f0b9be00700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00d8d0 tx=0x7f0b8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:56.623 INFO:tasks.workunit.client.0.smithi057.stdout:6/897: dwrite d0/d1b/d3e/d3d/f111 [0,4194304] 0 2024-02-28T23:07:56.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.624+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8c0098a0 con 0x7f0ba41014b0 2024-02-28T23:07:56.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.624+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c010460 con 0x7f0ba41014b0 2024-02-28T23:07:56.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.624+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8c00f5f0 con 0x7f0ba41014b0 2024-02-28T23:07:56.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.626+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba41a1800 con 0x7f0ba41014b0 2024-02-28T23:07:56.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.626+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba41a1d20 con 0x7f0ba41014b0 2024-02-28T23:07:56.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.628+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba41961c0 con 0x7f0ba41014b0 2024-02-28T23:07:56.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.629+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f0b8c009a00 con 0x7f0ba41014b0 2024-02-28T23:07:56.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.630+0000 7f0ba0c00700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 0x7f0b8806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:56.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.631+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0b8c08c310 con 0x7f0ba41014b0 2024-02-28T23:07:56.631 INFO:tasks.workunit.client.0.smithi057.stdout:3/868: write d3/d4/f12 [5898965,87995] 0 2024-02-28T23:07:56.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.632+0000 7f0ba3400700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 0x7f0b8806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:56.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.632+0000 7f0ba3400700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 0x7f0b8806f4c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006010 tx=0x7f0b9400b560 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:56.634 INFO:tasks.workunit.client.0.smithi057.stdout:6/898: creat d0/d4/d25/d60/d6a/f126 x:0 0 0 2024-02-28T23:07:56.635 INFO:tasks.workunit.client.0.smithi057.stdout:3/869: dread - d3/d8/d27/d70/f9d zero size 2024-02-28T23:07:56.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:56.637+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f0b8c059ab0 con 0x7f0ba41014b0 2024-02-28T23:07:56.637 INFO:tasks.workunit.client.0.smithi057.stdout:5/967: dwrite d4/d9/d16/f1a [0,4194304] 0 2024-02-28T23:07:56.642 INFO:tasks.workunit.client.0.smithi057.stdout:1/880: sync 2024-02-28T23:07:56.644 INFO:tasks.workunit.client.0.smithi057.stdout:5/968: creat d4/d9/d16/d52/dfd/f12c x:0 0 0 2024-02-28T23:07:56.653 INFO:tasks.workunit.client.0.smithi057.stdout:5/969: creat d4/d9/d16/f12d x:0 0 0 2024-02-28T23:07:56.654 INFO:tasks.workunit.client.0.smithi057.stdout:6/899: dread d0/d17/d20/f5e [0,4194304] 0 2024-02-28T23:07:56.661 INFO:tasks.workunit.client.0.smithi057.stdout:5/970: symlink d4/d9/d14/d9e/d10f/l12e 0 2024-02-28T23:07:56.672 INFO:tasks.workunit.client.0.smithi057.stdout:6/900: write d0/d1b/d3e/d3d/d42/f11e [545752,4870] 0 2024-02-28T23:07:56.674 INFO:tasks.workunit.client.0.smithi057.stdout:6/901: readlink d0/d4/d25/l29 0 2024-02-28T23:07:56.745 INFO:tasks.workunit.client.0.smithi057.stdout:6/902: mkdir d0/d1b/d3e/d3d/d42/d127 0 2024-02-28T23:07:56.746 INFO:tasks.workunit.client.0.smithi057.stdout:3/870: dread d3/d4/d7/f96 [0,4194304] 0 2024-02-28T23:07:56.750 INFO:tasks.workunit.client.0.smithi057.stdout:3/871: creat d3/f118 x:0 0 0 2024-02-28T23:07:56.755 INFO:tasks.workunit.client.0.smithi057.stdout:3/872: dread - d3/d4/fca zero size 2024-02-28T23:07:56.761 INFO:tasks.workunit.client.0.smithi057.stdout:3/873: rename d3/d4/c1a to d3/d8/d1b/c119 0 2024-02-28T23:07:56.762 INFO:tasks.workunit.client.0.smithi057.stdout:5/971: dread d4/d7/d8/ff [0,4194304] 0 2024-02-28T23:07:56.764 INFO:tasks.workunit.client.0.smithi057.stdout:3/874: creat d3/d20/f11a x:0 0 0 2024-02-28T23:07:56.767 INFO:tasks.workunit.client.0.smithi057.stdout:3/875: write d3/fd [3826874,2475] 0 2024-02-28T23:07:56.771 INFO:tasks.workunit.client.0.smithi057.stdout:3/876: symlink d3/d4/d7/d9a/l11b 0 2024-02-28T23:07:56.778 INFO:tasks.workunit.client.0.smithi057.stdout:3/877: rename d3/d4/d1d/c10c to d3/d4/d7/c11c 0 2024-02-28T23:07:56.781 INFO:tasks.workunit.client.0.smithi057.stdout:6/903: dread d0/d17/d94/da0/de6/fa2 [0,4194304] 0 2024-02-28T23:07:56.782 INFO:tasks.workunit.client.0.smithi057.stdout:5/972: truncate d4/d32/df1/f2d 2059349 0 2024-02-28T23:07:56.782 INFO:tasks.workunit.client.0.smithi057.stdout:3/878: write d3/d8/d27/d70/fd8 [267829,100197] 0 2024-02-28T23:07:56.784 INFO:tasks.workunit.client.0.smithi057.stdout:6/904: readlink d0/d1b/d54/l51 0 2024-02-28T23:07:56.785 INFO:tasks.workunit.client.0.smithi057.stdout:5/973: creat d4/d9/d14/f12f x:0 0 0 2024-02-28T23:07:56.788 INFO:tasks.workunit.client.0.smithi057.stdout:6/905: mkdir d0/d1b/d54/d128 0 2024-02-28T23:07:56.790 INFO:tasks.workunit.client.0.smithi057.stdout:3/879: creat d3/d4/de8/dee/f11d x:0 0 0 2024-02-28T23:07:56.791 INFO:tasks.workunit.client.0.smithi057.stdout:5/974: mknod d4/d9/d14/df2/c130 0 2024-02-28T23:07:56.803 INFO:tasks.workunit.client.0.smithi057.stdout:6/906: write d0/d17/d94/da0/de6/fa2 [2983907,47184] 0 2024-02-28T23:07:56.807 INFO:tasks.workunit.client.0.smithi057.stdout:6/907: mkdir d0/d4/d11/d129 0 2024-02-28T23:07:56.811 INFO:tasks.workunit.client.0.smithi057.stdout:6/908: symlink d0/d4/d25/d60/d6a/dff/l12a 0 2024-02-28T23:07:56.812 INFO:tasks.workunit.client.0.smithi057.stdout:3/880: dwrite d3/d8/f3c [4194304,4194304] 0 2024-02-28T23:07:56.814 INFO:tasks.workunit.client.0.smithi057.stdout:6/909: rmdir d0/d4/d25/d60/dc9 39 2024-02-28T23:07:56.815 INFO:tasks.workunit.client.0.smithi057.stdout:3/881: mknod d3/d4/d1d/d47/d62/c11e 0 2024-02-28T23:07:56.816 INFO:tasks.workunit.client.0.smithi057.stdout:7/934: sync 2024-02-28T23:07:56.819 INFO:tasks.workunit.client.0.smithi057.stdout:6/910: creat d0/d17/d94/da0/f12b x:0 0 0 2024-02-28T23:07:56.819 INFO:tasks.workunit.client.0.smithi057.stdout:3/882: chown d3/d4/d1d/fdd 6 1 2024-02-28T23:07:56.819 INFO:tasks.workunit.client.0.smithi057.stdout:5/975: dread d4/d32/f6e [0,4194304] 0 2024-02-28T23:07:56.820 INFO:tasks.workunit.client.0.smithi057.stdout:3/883: chown d3/d4f/f7d 8 1 2024-02-28T23:07:56.823 INFO:tasks.workunit.client.0.smithi057.stdout:6/911: mknod d0/d17/d94/da0/de6/c12c 0 2024-02-28T23:07:56.827 INFO:tasks.workunit.client.0.smithi057.stdout:3/884: creat d3/d4/d1d/d47/d62/db4/dfa/f11f x:0 0 0 2024-02-28T23:07:56.828 INFO:tasks.workunit.client.0.smithi057.stdout:5/976: getdents d4/d9/d16/d52/dfd/da1/d127 0 2024-02-28T23:07:56.830 INFO:tasks.workunit.client.0.smithi057.stdout:6/912: symlink d0/d17/l12d 0 2024-02-28T23:07:56.832 INFO:tasks.workunit.client.0.smithi057.stdout:3/885: creat d3/d4/d7/d9a/d9b/f120 x:0 0 0 2024-02-28T23:07:56.833 INFO:tasks.workunit.client.0.smithi057.stdout:3/886: write d3/f64 [29260,96162] 0 2024-02-28T23:07:56.835 INFO:tasks.workunit.client.0.smithi057.stdout:6/913: dread d0/d1b/fed [0,4194304] 0 2024-02-28T23:07:56.836 INFO:tasks.workunit.client.0.smithi057.stdout:3/887: write d3/d4/d1d/f36 [2776510,83654] 0 2024-02-28T23:07:56.836 INFO:tasks.workunit.client.0.smithi057.stdout:7/935: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/faf [0,4194304] 0 2024-02-28T23:07:56.838 INFO:tasks.workunit.client.0.smithi057.stdout:6/914: chown d0/d17/d94/fe5 14518985 1 2024-02-28T23:07:56.839 INFO:tasks.workunit.client.0.smithi057.stdout:3/888: fdatasync d3/d8/d27/f3e 0 2024-02-28T23:07:56.841 INFO:tasks.workunit.client.0.smithi057.stdout:3/889: fsync d3/d4/d1d/fde 0 2024-02-28T23:07:56.842 INFO:tasks.workunit.client.0.smithi057.stdout:6/915: creat d0/d17/d94/f12e x:0 0 0 2024-02-28T23:07:56.843 INFO:tasks.workunit.client.0.smithi057.stdout:6/916: fdatasync d0/d4/d25/fa9 0 2024-02-28T23:07:56.844 INFO:tasks.workunit.client.0.smithi057.stdout:3/890: truncate d3/ff5 5170135 0 2024-02-28T23:07:56.847 INFO:tasks.workunit.client.0.smithi057.stdout:7/936: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/df7/cc2 to d0/d3f/d76/d107/c139 0 2024-02-28T23:07:56.848 INFO:tasks.workunit.client.0.smithi057.stdout:6/917: mkdir d0/d1b/d54/d12f 0 2024-02-28T23:07:56.849 INFO:tasks.workunit.client.0.smithi057.stdout:3/891: mknod d3/d4/d7/d9a/c121 0 2024-02-28T23:07:56.860 INFO:tasks.workunit.client.0.smithi057.stdout:7/937: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d125/f7d d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/f13a 0 2024-02-28T23:07:56.861 INFO:tasks.workunit.client.0.smithi057.stdout:7/938: stat d0/d3f/d76/dc1/d130 0 2024-02-28T23:07:56.870 INFO:tasks.workunit.client.0.smithi057.stdout:6/918: dwrite d0/d4/fc1 [4194304,4194304] 0 2024-02-28T23:07:56.873 INFO:tasks.workunit.client.0.smithi057.stdout:3/892: dwrite d3/d4/d1d/f39 [0,4194304] 0 2024-02-28T23:07:56.876 INFO:tasks.workunit.client.0.smithi057.stdout:6/919: read d0/d4/fa [1053428,79315] 0 2024-02-28T23:07:56.876 INFO:tasks.workunit.client.0.smithi057.stdout:3/893: read d3/fd [7730228,72377] 0 2024-02-28T23:07:56.884 INFO:tasks.workunit.client.0.smithi057.stdout:3/894: mknod d3/d4/d7/c122 0 2024-02-28T23:07:56.890 INFO:tasks.workunit.client.0.smithi057.stdout:7/939: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f9c [4194304,4194304] 0 2024-02-28T23:07:56.898 INFO:tasks.workunit.client.0.smithi057.stdout:7/940: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/d9f/fc4 [2143884,61587] 0 2024-02-28T23:07:56.904 INFO:tasks.workunit.client.0.smithi057.stdout:6/920: dwrite d0/d17/f2c [0,4194304] 0 2024-02-28T23:07:56.909 INFO:tasks.workunit.client.0.smithi057.stdout:6/921: fdatasync d0/d1b/d3e/d40/f4d 0 2024-02-28T23:07:56.921 INFO:tasks.workunit.client.0.smithi057.stdout:7/941: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f1a d0/d98/d10c/f13b 0 2024-02-28T23:07:56.925 INFO:tasks.workunit.client.0.smithi057.stdout:6/922: write d0/d1b/d3e/d3d/d42/d4f/d53/f80 [1180032,108078] 0 2024-02-28T23:07:56.929 INFO:tasks.workunit.client.0.smithi057.stdout:3/895: dread d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:56.933 INFO:tasks.workunit.client.0.smithi057.stdout:3/896: truncate d3/d20/f115 940092 0 2024-02-28T23:07:56.936 INFO:tasks.workunit.client.0.smithi057.stdout:3/897: write d3/d4/d1d/f10b [277431,5906] 0 2024-02-28T23:07:56.937 INFO:tasks.workunit.client.0.smithi057.stdout:6/923: write d0/d4/f50 [632184,64109] 0 2024-02-28T23:07:56.942 INFO:tasks.workunit.client.0.smithi057.stdout:3/898: truncate d3/fd 328000 0 2024-02-28T23:07:56.963 INFO:tasks.workunit.client.0.smithi057.stdout:6/924: dwrite d0/d1b/d3e/ffc [0,4194304] 0 2024-02-28T23:07:56.965 INFO:tasks.workunit.client.0.smithi057.stdout:7/942: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/fb3 [0,4194304] 0 2024-02-28T23:07:56.970 INFO:tasks.workunit.client.0.smithi057.stdout:3/899: dwrite d3/d4/f30 [0,4194304] 0 2024-02-28T23:07:56.971 INFO:tasks.workunit.client.0.smithi057.stdout:6/925: mkdir d0/d4/d25/d60/d6a/d130 0 2024-02-28T23:07:56.978 INFO:tasks.workunit.client.0.smithi057.stdout:3/900: symlink d3/l123 0 2024-02-28T23:07:56.980 INFO:tasks.workunit.client.0.smithi057.stdout:3/901: stat d3/d4f/d4c/f111 0 2024-02-28T23:07:56.983 INFO:tasks.workunit.client.0.smithi057.stdout:6/926: dread d0/d4/d25/d60/d6a/f69 [0,4194304] 0 2024-02-28T23:07:56.987 INFO:tasks.workunit.client.0.smithi057.stdout:3/902: rename d3/d8/d27/f5f to d3/d4/d7/d9a/d9b/dab/f124 0 2024-02-28T23:07:56.987 INFO:tasks.workunit.client.0.smithi057.stdout:7/943: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f79 [0,4194304] 0 2024-02-28T23:07:56.989 INFO:tasks.workunit.client.0.smithi057.stdout:3/903: mknod d3/d8/d27/d70/c125 0 2024-02-28T23:07:56.992 INFO:tasks.workunit.client.0.smithi057.stdout:6/927: write d0/d1b/d3e/d3d/d42/fb0 [1485112,35099] 0 2024-02-28T23:07:56.994 INFO:tasks.workunit.client.0.smithi057.stdout:7/944: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/f8b to d0/d3f/d76/dc1/d130/f13c 0 2024-02-28T23:07:56.995 INFO:tasks.workunit.client.0.smithi057.stdout:3/904: symlink d3/d4f/d4c/d91/l126 0 2024-02-28T23:07:56.996 INFO:tasks.workunit.client.0.smithi057.stdout:6/928: symlink d0/d17/d94/da0/de6/l131 0 2024-02-28T23:07:56.999 INFO:tasks.workunit.client.0.smithi057.stdout:1/881: sync 2024-02-28T23:07:56.999 INFO:tasks.workunit.client.0.smithi057.stdout:9/898: sync 2024-02-28T23:07:57.000 INFO:tasks.workunit.client.0.smithi057.stdout:4/926: sync 2024-02-28T23:07:57.002 INFO:tasks.workunit.client.0.smithi057.stdout:7/945: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d71/f13d x:0 0 0 2024-02-28T23:07:57.003 INFO:tasks.workunit.client.0.smithi057.stdout:4/927: chown d1a/d2f/f102 14897657 1 2024-02-28T23:07:57.003 INFO:tasks.workunit.client.0.smithi057.stdout:6/929: mknod d0/d1b/d3e/d3d/d42/c132 0 2024-02-28T23:07:57.017 INFO:tasks.workunit.client.0.smithi057.stdout:3/905: dwrite d3/d8/d27/f77 [0,4194304] 0 2024-02-28T23:07:57.018 INFO:tasks.workunit.client.0.smithi057.stdout:1/882: dread d2/d71/d2d/f49 [0,4194304] 0 2024-02-28T23:07:57.020 INFO:tasks.workunit.client.0.smithi057.stdout:6/930: truncate d0/d17/d94/fe5 3874486 0 2024-02-28T23:07:57.029 INFO:tasks.workunit.client.0.smithi057.stdout:9/899: dwrite dd/d46/dc4/f96 [0,4194304] 0 2024-02-28T23:07:57.030 INFO:tasks.workunit.client.0.smithi057.stdout:6/931: getdents d0/d4/d11 0 2024-02-28T23:07:57.041 INFO:tasks.workunit.client.0.smithi057.stdout:9/900: unlink dd/d46/dc4/de5/ffa 0 2024-02-28T23:07:57.041 INFO:tasks.workunit.client.0.smithi057.stdout:7/946: dread d0/d3f/d76/dc1/dea/f5b [0,4194304] 0 2024-02-28T23:07:57.047 INFO:tasks.workunit.client.0.smithi057.stdout:6/932: link d0/d17/d94/da0/de6/cc0 d0/d17/d94/da0/c133 0 2024-02-28T23:07:57.048 INFO:tasks.workunit.client.0.smithi057.stdout:9/901: creat dd/d46/d6f/f126 x:0 0 0 2024-02-28T23:07:57.050 INFO:tasks.workunit.client.0.smithi057.stdout:4/928: dread d1a/d2f/db2/d49/d4d/dc9/dff/d125/fc0 [0,4194304] 0 2024-02-28T23:07:57.056 INFO:tasks.workunit.client.0.smithi057.stdout:1/883: dwrite d2/d71/d2d/d39/f138 [0,4194304] 0 2024-02-28T23:07:57.061 INFO:tasks.workunit.client.0.smithi057.stdout:7/947: rmdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d 39 2024-02-28T23:07:57.062 INFO:tasks.workunit.client.0.smithi057.stdout:4/929: chown d1a/d48/d9c/d2e/d8f/f11b 186 1 2024-02-28T23:07:57.063 INFO:tasks.workunit.client.0.smithi057.stdout:3/906: dwrite d3/d8/d27/fbb [0,4194304] 0 2024-02-28T23:07:57.065 INFO:tasks.workunit.client.0.smithi057.stdout:6/933: truncate d0/d4/fd6 1926517 0 2024-02-28T23:07:57.075 INFO:tasks.workunit.client.0.smithi057.stdout:9/902: dwrite dd/de/d24/d63/d77/dcf/d40/d44/da5/fea [0,4194304] 0 2024-02-28T23:07:57.076 INFO:tasks.workunit.client.0.smithi057.stdout:4/930: fdatasync d1a/d1e/fa8 0 2024-02-28T23:07:57.080 INFO:tasks.workunit.client.0.smithi057.stdout:3/907: fsync d3/d20/f24 0 2024-02-28T23:07:57.094 INFO:tasks.workunit.client.0.smithi057.stdout:4/931: dread d1a/d2f/db2/d49/d4d/dc9/dff/d125/f32 [0,4194304] 0 2024-02-28T23:07:57.094 INFO:tasks.workunit.client.0.smithi057.stdout:3/908: creat d3/d4f/f127 x:0 0 0 2024-02-28T23:07:57.094 INFO:tasks.workunit.client.0.smithi057.stdout:9/903: getdents dd/de/d24/d63/d77/dcf/d40/dd0 0 2024-02-28T23:07:57.097 INFO:tasks.workunit.client.0.smithi057.stdout:4/932: readlink d1a/d48/db0/l8e 0 2024-02-28T23:07:57.099 INFO:tasks.workunit.client.0.smithi057.stdout:1/884: write d2/d33/d54/fc7 [63781,11234] 0 2024-02-28T23:07:57.105 INFO:tasks.workunit.client.0.smithi057.stdout:3/909: creat d3/d4/d1d/d47/f128 x:0 0 0 2024-02-28T23:07:57.108 INFO:tasks.workunit.client.0.smithi057.stdout:9/904: fdatasync dd/de/d14/d48/d4b/f54 0 2024-02-28T23:07:57.113 INFO:tasks.workunit.client.0.smithi057.stdout:7/948: link d0/d98/ldd d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/l13e 0 2024-02-28T23:07:57.115 INFO:tasks.workunit.client.0.smithi057.stdout:1/885: dread d2/d7/df/db/d12/d19/f9d [0,4194304] 0 2024-02-28T23:07:57.118 INFO:tasks.workunit.client.0.smithi057.stdout:9/905: rmdir dd/d46 39 2024-02-28T23:07:57.121 INFO:tasks.workunit.client.0.smithi057.stdout:6/934: dwrite d0/d1b/d3e/d3d/d42/d4f/fd9 [0,4194304] 0 2024-02-28T23:07:57.124 INFO:tasks.workunit.client.0.smithi057.stdout:4/933: write d1a/d2f/f45 [283209,43595] 0 2024-02-28T23:07:57.128 INFO:tasks.workunit.client.0.smithi057.stdout:9/906: truncate dd/de/d14/d48/fdc 1010620 0 2024-02-28T23:07:57.129 INFO:tasks.workunit.client.0.smithi057.stdout:3/910: getdents d3/d4/dfd 0 2024-02-28T23:07:57.129 INFO:tasks.workunit.client.0.smithi057.stdout:5/977: sync 2024-02-28T23:07:57.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.131+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0ba402cc30 con 0x7f0ba41014b0 2024-02-28T23:07:57.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.132+0000 7f0ba0c00700 1 -- 172.21.15.57:0/523754242 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f0b8c059640 con 0x7f0ba41014b0 2024-02-28T23:07:57.134 INFO:tasks.workunit.client.0.smithi057.stdout:9/907: dread dd/f2f [0,4194304] 0 2024-02-28T23:07:57.134 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:07:57.134 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:07:57.134 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:07:57.134 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:07:57.135 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:07:57.136 INFO:tasks.workunit.client.0.smithi057.stdout:4/934: unlink d1a/d2f/db2/dcb/dfe/c121 0 2024-02-28T23:07:57.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 msgr2=0x7f0b8806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 0x7f0b8806f4c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006010 tx=0x7f0b9400b560 comp rx=0 tx=0).stop 2024-02-28T23:07:57.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 msgr2=0x7f0ba419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.137 INFO:tasks.workunit.client.0.smithi057.stdout:5/978: symlink d4/d32/df1/d24/da2/l131 0 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00d8d0 tx=0x7f0b8c00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 shutdown_connections 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba40feb50 0x7f0ba419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f0b8806d010 0x7f0b8806f4c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 --2- 172.21.15.57:0/523754242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ba41014b0 0x7f0ba419c5f0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00d8d0 tx=0x7f0b8c00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 >> 172.21.15.57:0/523754242 conn(0x7f0ba40fa780 msgr2=0x7f0ba40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:57.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.137+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 shutdown_connections 2024-02-28T23:07:57.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.138+0000 7f0baa800700 1 -- 172.21.15.57:0/523754242 wait complete. 2024-02-28T23:07:57.138 INFO:tasks.workunit.client.0.smithi057.stdout:7/949: truncate d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/fbd 649402 0 2024-02-28T23:07:57.138 INFO:tasks.workunit.client.0.smithi057.stdout:6/935: link d0/d1b/cf8 d0/d4/d25/d60/d6a/dff/c134 0 2024-02-28T23:07:57.139 INFO:tasks.workunit.client.0.smithi057.stdout:1/886: dwrite d2/d71/d2d/f31 [0,4194304] 0 2024-02-28T23:07:57.139 INFO:tasks.workunit.client.0.smithi057.stdout:9/908: write dd/de/d14/f18 [2556071,71911] 0 2024-02-28T23:07:57.195 INFO:tasks.workunit.client.0.smithi057.stdout:6/936: fdatasync d0/d1b/d3e/d100/f123 0 2024-02-28T23:07:57.196 INFO:tasks.workunit.client.0.smithi057.stdout:5/979: fsync d4/d9/d14/f5e 0 2024-02-28T23:07:57.197 INFO:tasks.workunit.client.0.smithi057.stdout:4/935: write d1a/d2f/db2/dcb/dfe/ffc [3521116,19456] 0 2024-02-28T23:07:57.204 INFO:tasks.workunit.client.0.smithi057.stdout:1/887: unlink d2/d7/df/db/d12/d19/l134 0 2024-02-28T23:07:57.205 INFO:tasks.workunit.client.0.smithi057.stdout:9/909: mkdir dd/de/d24/d127 0 2024-02-28T23:07:57.207 INFO:tasks.workunit.client.0.smithi057.stdout:6/937: mknod d0/d1b/d3e/d40/c135 0 2024-02-28T23:07:57.214 INFO:tasks.workunit.client.0.smithi057.stdout:9/910: mknod dd/de/d24/d63/dc1/c128 0 2024-02-28T23:07:57.217 INFO:tasks.workunit.client.0.smithi057.stdout:7/950: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/d51/f5e [4194304,4194304] 0 2024-02-28T23:07:57.219 INFO:tasks.workunit.client.0.smithi057.stdout:1/888: rename d2/d7/df/db/d3a/d2a/d82/fe8 to d2/d71/d2d/f13d 0 2024-02-28T23:07:57.225 INFO:tasks.workunit.client.0.smithi057.stdout:1/889: write d2/d7/df/db/d12/d19/da4/f10f [26315,37679] 0 2024-02-28T23:07:57.226 INFO:tasks.workunit.client.0.smithi057.stdout:7/951: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f137 [8475,8215] 0 2024-02-28T23:07:57.228 INFO:tasks.workunit.client.0.smithi057.stdout:9/911: creat dd/d46/dc4/dd3/f129 x:0 0 0 2024-02-28T23:07:57.229 INFO:tasks.workunit.client.0.smithi057.stdout:4/936: write d1a/d2f/db2/d49/d4d/dc9/dff/d125/f3e [1968661,79623] 0 2024-02-28T23:07:57.230 INFO:tasks.workunit.client.0.smithi057.stdout:6/938: link d0/d1b/d3e/d3d/d42/d4f/d53/cda d0/d1b/d54/d12f/c136 0 2024-02-28T23:07:57.252 INFO:tasks.workunit.client.0.smithi057.stdout:9/912: chown dd/de/d14/dc2/d101/f109 1048 1 2024-02-28T23:07:57.253 INFO:tasks.workunit.client.0.smithi057.stdout:1/890: write d2/d71/d2d/d5b/f7d [976813,24285] 0 2024-02-28T23:07:57.256 INFO:tasks.workunit.client.0.smithi057.stdout:1/891: chown d2/d7/df/f10c 1728 1 2024-02-28T23:07:57.296 INFO:tasks.workunit.client.0.smithi057.stdout:7/952: fdatasync d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/f12d 0 2024-02-28T23:07:57.297 INFO:tasks.workunit.client.0.smithi057.stdout:1/892: fsync d2/d7/df/db/d12/d3f/f43 0 2024-02-28T23:07:57.301 INFO:tasks.workunit.client.0.smithi057.stdout:6/939: fdatasync d0/d4/fd6 0 2024-02-28T23:07:57.305 INFO:tasks.workunit.client.0.smithi057.stdout:4/937: creat d1a/f137 x:0 0 0 2024-02-28T23:07:57.306 INFO:tasks.workunit.client.0.smithi057.stdout:1/893: creat d2/d7/df/db/d10b/f13e x:0 0 0 2024-02-28T23:07:57.308 INFO:tasks.workunit.client.0.smithi057.stdout:1/894: fsync d2/f8 0 2024-02-28T23:07:57.316 INFO:tasks.workunit.client.0.smithi057.stdout:6/940: fsync d0/d17/fa7 0 2024-02-28T23:07:57.318 INFO:tasks.workunit.client.0.smithi057.stdout:6/941: chown d0/d4/d11/f13 32218 1 2024-02-28T23:07:57.318 INFO:tasks.workunit.client.0.smithi057.stdout:7/953: rename d0/d3f/d76/dc1/dea/fbe to d0/d3f/d76/dc1/dea/f13f 0 2024-02-28T23:07:57.320 INFO:tasks.workunit.client.0.smithi057.stdout:4/938: mknod d1a/d2f/db2/d49/d4d/d107/c138 0 2024-02-28T23:07:57.324 INFO:tasks.workunit.client.0.smithi057.stdout:6/942: mkdir d0/d17/d94/da0/db3/d137 0 2024-02-28T23:07:57.332 INFO:tasks.workunit.client.0.smithi057.stdout:4/939: rename d1a/d2f/db2/d49/d4d/dc9/dff/d125/lef to d1a/d2f/db2/d49/d4d/dc9/dec/l139 0 2024-02-28T23:07:57.334 INFO:tasks.workunit.client.0.smithi057.stdout:6/943: creat d0/d17/d94/da0/de6/df4/f138 x:0 0 0 2024-02-28T23:07:57.336 INFO:tasks.workunit.client.0.smithi057.stdout:4/940: mknod d1a/d2f/db2/d49/d4d/d107/c13a 0 2024-02-28T23:07:57.339 INFO:tasks.workunit.client.0.smithi057.stdout:4/941: stat d1a/d2f/db2/d49/d4d/dc9/dff/d125/f21 0 2024-02-28T23:07:57.341 INFO:tasks.workunit.client.0.smithi057.stdout:4/942: fdatasync d1a/d48/d9c/d2e/d8f/f11b 0 2024-02-28T23:07:57.341 INFO:tasks.workunit.client.0.smithi057.stdout:6/944: link d0/d1b/d3e/d3d/l86 d0/d1b/d54/l139 0 2024-02-28T23:07:57.344 INFO:tasks.workunit.client.0.smithi057.stdout:4/943: truncate d1a/d48/db0/ff6 442356 0 2024-02-28T23:07:57.345 INFO:tasks.workunit.client.0.smithi057.stdout:6/945: write d0/d1b/d3e/d3d/d42/d4f/d53/f10f [252981,103878] 0 2024-02-28T23:07:57.346 INFO:tasks.workunit.client.0.smithi057.stdout:4/944: chown d1a/d48/d9c/d68/fa7 51682160 1 2024-02-28T23:07:57.349 INFO:tasks.workunit.client.0.smithi057.stdout:9/913: write dd/de/d24/d63/d77/dcf/d40/d44/d5c/f123 [2100984,95024] 0 2024-02-28T23:07:57.352 INFO:tasks.workunit.client.0.smithi057.stdout:6/946: rename d0/d1b/d3e/d3d/f5a to d0/d17/d94/da0/de6/f13a 0 2024-02-28T23:07:57.354 INFO:tasks.workunit.client.0.smithi057.stdout:4/945: symlink d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/l13b 0 2024-02-28T23:07:57.357 INFO:tasks.workunit.client.0.smithi057.stdout:9/914: creat dd/de/d24/d63/d77/dcf/d40/f12a x:0 0 0 2024-02-28T23:07:57.364 INFO:tasks.workunit.client.0.smithi057.stdout:9/915: getdents dd/de/d24/d63/d77/df2 0 2024-02-28T23:07:57.372 INFO:tasks.workunit.client.0.smithi057.stdout:6/947: dwrite d0/d4/d25/d60/f120 [0,4194304] 0 2024-02-28T23:07:57.373 INFO:tasks.workunit.client.0.smithi057.stdout:4/946: write d1a/d2f/db2/f7b [894230,37793] 0 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 -- 172.21.15.57:0/3766020636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 msgr2=0x7f1468103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 --2- 172.21.15.57:0/3766020636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 0x7f1468103a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1450009870 tx=0x7f1450009b80 comp rx=0 tx=0).stop 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 -- 172.21.15.57:0/3766020636 shutdown_connections 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 --2- 172.21.15.57:0/3766020636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 0x7f1468103a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 --2- 172.21.15.57:0/3766020636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 -- 172.21.15.57:0/3766020636 >> 172.21.15.57:0/3766020636 conn(0x7f14680fd9f0 msgr2=0x7f14680ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:57.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.376+0000 7f146f800700 1 -- 172.21.15.57:0/3766020636 shutdown_connections 2024-02-28T23:07:57.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.377+0000 7f146f800700 1 -- 172.21.15.57:0/3766020636 wait complete. 2024-02-28T23:07:57.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.380+0000 7f146f800700 1 Processor -- start 2024-02-28T23:07:57.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.380+0000 7f146f800700 1 -- start start 2024-02-28T23:07:57.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146f800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146f800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 0x7f1468198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146f800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468198880 con 0x7f1468103640 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146f800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146819d290 con 0x7f1468102440 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:43738/0 (socket says 172.21.15.57:43738) 2024-02-28T23:07:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.381+0000 7f146ce00700 1 -- 172.21.15.57:0/1554443415 learned_addr learned my addr 172.21.15.57:0/1554443415 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:07:57.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.382+0000 7f146ce00700 1 -- 172.21.15.57:0/1554443415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 msgr2=0x7f1468198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.382+0000 7f146ce00700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 0x7f1468198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.382+0000 7f146ce00700 1 -- 172.21.15.57:0/1554443415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450009520 con 0x7f1468102440 2024-02-28T23:07:57.383 INFO:tasks.workunit.client.0.smithi057.stdout:6/948: mknod d0/d1b/d3e/d3d/d42/d4f/d92/d107/c13b 0 2024-02-28T23:07:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.382+0000 7f146ce00700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f146000d8d0 tx=0x7f146000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.383+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1460009900 con 0x7f1468102440 2024-02-28T23:07:57.384 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:57 smithi057 ceph-mon[29096]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:57.384 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:57 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/523754242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:07:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.383+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1460010460 con 0x7f1468102440 2024-02-28T23:07:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.383+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146806a830 con 0x7f1468102440 2024-02-28T23:07:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.383+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f146000f5f0 con 0x7f1468102440 2024-02-28T23:07:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.383+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146806ad50 con 0x7f1468102440 2024-02-28T23:07:57.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.386+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1444005320 con 0x7f1468102440 2024-02-28T23:07:57.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.388+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 90728+0+0 (secure 0 0 0) 0x7f146000f7e0 con 0x7f1468102440 2024-02-28T23:07:57.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.389+0000 7f1465600700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 0x7f144c06f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:07:57.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.390+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f146008c7c0 con 0x7f1468102440 2024-02-28T23:07:57.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.390+0000 7f1467e00700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 0x7f144c06f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:07:57.393 INFO:tasks.workunit.client.0.smithi057.stdout:9/916: dwrite dd/de/d14/f8e [0,4194304] 0 2024-02-28T23:07:57.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.392+0000 7f1467e00700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 0x7f144c06f450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1450009870 tx=0x7f145000b5a0 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:07:57.394 INFO:tasks.workunit.client.0.smithi057.stdout:4/947: link d1a/d48/d9c/le0 d1a/d2f/db2/d49/d4d/dc9/dff/d125/d7f/l13c 0 2024-02-28T23:07:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.399+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f14600563b0 con 0x7f1468102440 2024-02-28T23:07:57.399 INFO:tasks.workunit.client.0.smithi057.stdout:4/948: dread - d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/f117 zero size 2024-02-28T23:07:57.402 INFO:tasks.workunit.client.0.smithi057.stdout:4/949: chown d1a/d2f/db2/d49/d4d/f88 965960619 1 2024-02-28T23:07:57.403 INFO:tasks.workunit.client.0.smithi057.stdout:9/917: mknod dd/de/d24/d63/dc1/df0/c12b 0 2024-02-28T23:07:57.406 INFO:tasks.workunit.client.0.smithi057.stdout:6/949: write d0/d1b/d3e/d3d/fc4 [159634,61664] 0 2024-02-28T23:07:57.414 INFO:tasks.workunit.client.0.smithi057.stdout:9/918: creat dd/de/d24/d63/d77/dcf/d40/d108/f12c x:0 0 0 2024-02-28T23:07:57.417 INFO:tasks.workunit.client.0.smithi057.stdout:4/950: creat d1a/f13d x:0 0 0 2024-02-28T23:07:57.424 INFO:tasks.workunit.client.0.smithi057.stdout:6/950: dread d0/d1b/d3e/d3d/d42/d4f/fbb [0,4194304] 0 2024-02-28T23:07:57.425 INFO:tasks.workunit.client.0.smithi057.stdout:9/919: mkdir dd/de/d24/d63/d77/dcf/d40/d44/da5/df7/d12d 0 2024-02-28T23:07:57.432 INFO:tasks.workunit.client.0.smithi057.stdout:6/951: creat d0/d17/d94/da0/db3/f13c x:0 0 0 2024-02-28T23:07:57.436 INFO:tasks.workunit.client.0.smithi057.stdout:6/952: mknod d0/c13d 0 2024-02-28T23:07:57.437 INFO:tasks.workunit.client.0.smithi057.stdout:9/920: read dd/de/d14/d48/d4b/d41/fcb [339468,2040] 0 2024-02-28T23:07:57.442 INFO:tasks.workunit.client.0.smithi057.stdout:4/951: dwrite d1a/d2f/db2/d49/d4d/de2/f12e [0,4194304] 0 2024-02-28T23:07:57.443 INFO:tasks.workunit.client.0.smithi057.stdout:9/921: chown dd/de/d4a/d60/d8c/c11e 118370629 1 2024-02-28T23:07:57.452 INFO:tasks.workunit.client.0.smithi057.stdout:6/953: getdents d0/d1b/d3e/d3d/d42/d4f 0 2024-02-28T23:07:57.452 INFO:tasks.workunit.client.0.smithi057.stdout:4/952: creat d1a/d2f/db2/d49/d4d/de2/d6e/f13e x:0 0 0 2024-02-28T23:07:57.454 INFO:tasks.workunit.client.0.smithi057.stdout:3/911: sync 2024-02-28T23:07:57.464 INFO:tasks.workunit.client.0.smithi057.stdout:9/922: rename dd/de/d24/d63/d77/dcf/d40/d44/da5/de4 to dd/de/d4a/d60/d8c/d12e 0 2024-02-28T23:07:57.465 INFO:tasks.workunit.client.0.smithi057.stdout:4/953: truncate f18 2781916 0 2024-02-28T23:07:57.466 INFO:tasks.workunit.client.0.smithi057.stdout:3/912: mkdir d3/d4/d7/d9a/d129 0 2024-02-28T23:07:57.470 INFO:tasks.workunit.client.0.smithi057.stdout:4/954: symlink d1a/d2f/db2/d49/d4d/de2/d6e/db4/l13f 0 2024-02-28T23:07:57.472 INFO:tasks.workunit.client.0.smithi057.stdout:3/913: creat d3/d4/d1d/d47/f12a x:0 0 0 2024-02-28T23:07:57.475 INFO:tasks.workunit.client.0.smithi057.stdout:3/914: dread - d3/d20/d2d/fe4 zero size 2024-02-28T23:07:57.476 INFO:tasks.workunit.client.0.smithi057.stdout:3/915: stat d3/d4f/d101 0 2024-02-28T23:07:57.480 INFO:tasks.workunit.client.0.smithi057.stdout:9/923: unlink dd/de/d24/d63/d77/dcf/d40/d44/c11a 0 2024-02-28T23:07:57.480 INFO:tasks.workunit.client.0.smithi057.stdout:6/954: dwrite d0/d17/fa7 [0,4194304] 0 2024-02-28T23:07:57.482 INFO:tasks.workunit.client.0.smithi057.stdout:3/916: mknod d3/d4/d1d/d47/d62/c12b 0 2024-02-28T23:07:57.486 INFO:tasks.workunit.client.0.smithi057.stdout:6/955: readlink d0/d17/d94/da0/l114 0 2024-02-28T23:07:57.487 INFO:tasks.workunit.client.0.smithi057.stdout:3/917: creat d3/d8/d27/d70/f12c x:0 0 0 2024-02-28T23:07:57.487 INFO:tasks.workunit.client.0.smithi057.stdout:9/924: chown dd/de/d24/d63/dc1/df0/cd5 6349095 1 2024-02-28T23:07:57.489 INFO:tasks.workunit.client.0.smithi057.stdout:9/925: write dd/de/d14/f18 [3574070,98702] 0 2024-02-28T23:07:57.491 INFO:tasks.workunit.client.0.smithi057.stdout:9/926: readlink dd/de/d24/d63/l7d 0 2024-02-28T23:07:57.494 INFO:tasks.workunit.client.0.smithi057.stdout:3/918: mknod d3/d4/d7/c12d 0 2024-02-28T23:07:57.504 INFO:tasks.workunit.client.0.smithi057.stdout:6/956: write d0/d1b/f27 [2923008,43162] 0 2024-02-28T23:07:57.505 INFO:tasks.workunit.client.0.smithi057.stdout:3/919: rename d3/d4/d1d/d47/d62/db4/dfa/f10f to d3/d4/d7/d9a/d9b/dab/dd0/f12e 0 2024-02-28T23:07:57.509 INFO:tasks.workunit.client.0.smithi057.stdout:6/957: rename d0/d1b/d3e/d40/l98 to d0/d1b/d3e/d40/l13e 0 2024-02-28T23:07:57.511 INFO:tasks.workunit.client.0.smithi057.stdout:3/920: symlink d3/l12f 0 2024-02-28T23:07:57.512 INFO:tasks.workunit.client.0.smithi057.stdout:6/958: read d0/d17/d94/f9e [3612478,83899] 0 2024-02-28T23:07:57.575 INFO:tasks.workunit.client.0.smithi057.stdout:4/955: dread d1a/d2f/db2/d49/d4d/de2/fb3 [0,4194304] 0 2024-02-28T23:07:57.575 INFO:tasks.workunit.client.0.smithi057.stdout:3/921: link d3/d4f/f127 d3/d4/d7/d9a/d129/f130 0 2024-02-28T23:07:57.578 INFO:tasks.workunit.client.0.smithi057.stdout:4/956: mkdir d1a/d48/d9c/dfd/d140 0 2024-02-28T23:07:57.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:57 smithi084 ceph-mon[34826]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:57.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:57 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/523754242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:07:57.580 INFO:tasks.workunit.client.0.smithi057.stdout:3/922: mknod d3/d4/d7/d9a/d9b/c131 0 2024-02-28T23:07:57.582 INFO:tasks.workunit.client.0.smithi057.stdout:3/923: readlink d3/d4f/d4c/d91/l126 0 2024-02-28T23:07:57.586 INFO:tasks.workunit.client.0.smithi057.stdout:3/924: write d3/d4f/d4c/d91/fc0 [3160084,44104] 0 2024-02-28T23:07:57.590 INFO:tasks.workunit.client.0.smithi057.stdout:6/959: dwrite d0/d1b/d3e/d3d/d42/d4f/fac [0,4194304] 0 2024-02-28T23:07:57.593 INFO:tasks.workunit.client.0.smithi057.stdout:3/925: creat d3/d4/d7/d9a/d9b/dab/f132 x:0 0 0 2024-02-28T23:07:57.605 INFO:tasks.workunit.client.0.smithi057.stdout:3/926: write d3/d8/f73 [2478960,22485] 0 2024-02-28T23:07:57.609 INFO:tasks.workunit.client.0.smithi057.stdout:4/957: dread d1a/d2f/f102 [0,4194304] 0 2024-02-28T23:07:57.615 INFO:tasks.workunit.client.0.smithi057.stdout:4/958: creat d1a/d2f/db2/d49/d4d/de2/d6e/db4/de9/d110/d111/f141 x:0 0 0 2024-02-28T23:07:57.623 INFO:tasks.workunit.client.0.smithi057.stdout:4/959: link f18 d1a/d48/d9c/d2e/df8/f142 0 2024-02-28T23:07:57.624 INFO:tasks.workunit.client.0.smithi057.stdout:3/927: dwrite d3/d8/d1b/f4a [0,4194304] 0 2024-02-28T23:07:57.626 INFO:tasks.workunit.client.0.smithi057.stdout:6/960: dread d0/d17/d94/fe5 [0,4194304] 0 2024-02-28T23:07:57.628 INFO:tasks.workunit.client.0.smithi057.stdout:3/928: mkdir d3/d4/d133 0 2024-02-28T23:07:57.628 INFO:tasks.workunit.client.0.smithi057.stdout:6/961: readlink d0/d17/l12d 0 2024-02-28T23:07:57.628 INFO:tasks.workunit.client.0.smithi057.stdout:4/960: symlink d1a/d48/d9c/dfd/d140/l143 0 2024-02-28T23:07:57.631 INFO:tasks.workunit.client.0.smithi057.stdout:6/962: symlink d0/d4/d25/l13f 0 2024-02-28T23:07:57.634 INFO:tasks.workunit.client.0.smithi057.stdout:5/980: sync 2024-02-28T23:07:57.634 INFO:tasks.workunit.client.0.smithi057.stdout:3/929: rename d3/d8/d1b/f44 to d3/d8/f134 0 2024-02-28T23:07:57.639 INFO:tasks.workunit.client.0.smithi057.stdout:5/981: creat d4/d9/f132 x:0 0 0 2024-02-28T23:07:57.643 INFO:tasks.workunit.client.0.smithi057.stdout:3/930: read d3/d8/f3c [2008513,78783] 0 2024-02-28T23:07:57.647 INFO:tasks.workunit.client.0.smithi057.stdout:6/963: dwrite d0/d4/d25/f71 [0,4194304] 0 2024-02-28T23:07:57.648 INFO:tasks.workunit.client.0.smithi057.stdout:4/961: truncate d1a/d1e/d51/f90 3169763 0 2024-02-28T23:07:57.648 INFO:tasks.workunit.client.0.smithi057.stdout:3/931: chown d3/d4/d7/d9a/d9b/fd2 3671 1 2024-02-28T23:07:57.652 INFO:tasks.workunit.client.0.smithi057.stdout:6/964: write d0/d1b/d3e/d3d/d42/d4f/d53/f80 [2616898,130288] 0 2024-02-28T23:07:57.655 INFO:tasks.workunit.client.0.smithi057.stdout:3/932: rename d3/d4/d133 to d3/d4/d7/d135 0 2024-02-28T23:07:57.659 INFO:tasks.workunit.client.0.smithi057.stdout:3/933: chown d3/d4/d7/d9a/ld3 0 1 2024-02-28T23:07:57.666 INFO:tasks.workunit.client.0.smithi057.stdout:4/962: write f14 [2193581,43204] 0 2024-02-28T23:07:57.670 INFO:tasks.workunit.client.0.smithi057.stdout:6/965: truncate d0/d4/f19 3992291 0 2024-02-28T23:07:57.679 INFO:tasks.workunit.client.0.smithi057.stdout:3/934: dwrite d3/d4/d7/f10 [12582912,4194304] 0 2024-02-28T23:07:57.684 INFO:tasks.workunit.client.0.smithi057.stdout:5/982: dread d4/d9/d14/d3b/fa0 [0,4194304] 0 2024-02-28T23:07:57.685 INFO:tasks.workunit.client.0.smithi057.stdout:3/935: mknod d3/d20/c136 0 2024-02-28T23:07:57.685 INFO:tasks.workunit.client.0.smithi057.stdout:4/963: dwrite d1a/d1e/f42 [0,4194304] 0 2024-02-28T23:07:57.688 INFO:tasks.workunit.client.0.smithi057.stdout:5/983: fdatasync d4/d9/d16/d4e/f111 0 2024-02-28T23:07:57.688 INFO:tasks.workunit.client.0.smithi057.stdout:3/936: readlink d3/d4/d7/d9a/l80 0 2024-02-28T23:07:57.690 INFO:tasks.workunit.client.0.smithi057.stdout:1/895: sync 2024-02-28T23:07:57.690 INFO:tasks.workunit.client.0.smithi057.stdout:7/954: sync 2024-02-28T23:07:57.696 INFO:tasks.workunit.client.0.smithi057.stdout:3/937: chown d3/d8/d27/d70/fd8 229 1 2024-02-28T23:07:57.696 INFO:tasks.workunit.client.0.smithi057.stdout:5/984: read d4/d9/d16/d4e/fe8 [336948,25075] 0 2024-02-28T23:07:57.697 INFO:tasks.workunit.client.0.smithi057.stdout:7/955: stat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/d9f/c114 0 2024-02-28T23:07:57.697 INFO:tasks.workunit.client.0.smithi057.stdout:6/966: dwrite d0/d1b/fed [0,4194304] 0 2024-02-28T23:07:57.699 INFO:tasks.workunit.client.0.smithi057.stdout:1/896: mkdir d2/d7/df/db/d10b/d6b/d13f 0 2024-02-28T23:07:57.701 INFO:tasks.workunit.client.0.smithi057.stdout:3/938: write d3/d4/f7a [2528253,96864] 0 2024-02-28T23:07:57.705 INFO:tasks.workunit.client.0.smithi057.stdout:5/985: chown d4/d9/d16/d4e/fc8 773877 1 2024-02-28T23:07:57.706 INFO:tasks.workunit.client.0.smithi057.stdout:3/939: write d3/d8/f3c [2401930,22035] 0 2024-02-28T23:07:57.712 INFO:tasks.workunit.client.0.smithi057.stdout:6/967: symlink d0/d17/d94/da0/db3/l140 0 2024-02-28T23:07:57.712 INFO:tasks.workunit.client.0.smithi057.stdout:6/968: chown d0/d4/fd6 233 1 2024-02-28T23:07:57.712 INFO:tasks.workunit.client.0.smithi057.stdout:3/940: fsync d3/d8/f5e 0 2024-02-28T23:07:57.715 INFO:tasks.workunit.client.0.smithi057.stdout:6/969: write d0/d4/d11/f9a [3908690,77196] 0 2024-02-28T23:07:57.719 INFO:tasks.workunit.client.0.smithi057.stdout:3/941: creat d3/d4/d7/d9a/d129/f137 x:0 0 0 2024-02-28T23:07:57.723 INFO:tasks.workunit.client.0.smithi057.stdout:1/897: dwrite d2/d71/def/ff2 [0,4194304] 0 2024-02-28T23:07:57.724 INFO:tasks.workunit.client.0.smithi057.stdout:6/970: unlink d0/d1b/d3e/d3d/d42/d4f/d53/f93 0 2024-02-28T23:07:57.725 INFO:tasks.workunit.client.0.smithi057.stdout:6/971: chown d0/c13d 478 1 2024-02-28T23:07:57.726 INFO:tasks.workunit.client.0.smithi057.stdout:1/898: write d2/d7/f41 [3419149,86050] 0 2024-02-28T23:07:57.728 INFO:tasks.workunit.client.0.smithi057.stdout:3/942: truncate d3/d4/d1d/d47/d62/fcf 381747 0 2024-02-28T23:07:57.729 INFO:tasks.workunit.client.0.smithi057.stdout:6/972: write d0/d17/fa7 [388693,56947] 0 2024-02-28T23:07:57.739 INFO:tasks.workunit.client.0.smithi057.stdout:4/964: write d1a/d2f/db2/f3c [1730057,27421] 0 2024-02-28T23:07:57.740 INFO:tasks.workunit.client.0.smithi057.stdout:1/899: chown d2/d71/d2d/d39/da3/db5/led 253 1 2024-02-28T23:07:57.740 INFO:tasks.workunit.client.0.smithi057.stdout:7/956: write d0/d3f/d76/fb8 [1031470,75057] 0 2024-02-28T23:07:57.740 INFO:tasks.workunit.client.0.smithi057.stdout:5/986: write d4/d7/d58/f96 [718443,43762] 0 2024-02-28T23:07:57.744 INFO:tasks.workunit.client.0.smithi057.stdout:3/943: truncate d3/d8/f86 861713 0 2024-02-28T23:07:57.746 INFO:tasks.workunit.client.0.smithi057.stdout:1/900: creat d2/d71/d2d/d39/da3/f140 x:0 0 0 2024-02-28T23:07:57.747 INFO:tasks.workunit.client.0.smithi057.stdout:4/965: truncate d1a/d2f/db2/f63 2199816 0 2024-02-28T23:07:57.750 INFO:tasks.workunit.client.0.smithi057.stdout:5/987: rename d4/d9/d16/d52/dfd/da1/d127 to d4/d32/df1/d24/d30/d62/d133 0 2024-02-28T23:07:57.751 INFO:tasks.workunit.client.0.smithi057.stdout:1/901: truncate d2/d71/f9e 1126149 0 2024-02-28T23:07:57.752 INFO:tasks.workunit.client.0.smithi057.stdout:6/973: mkdir d0/d17/d20/d141 0 2024-02-28T23:07:57.755 INFO:tasks.workunit.client.0.smithi057.stdout:1/902: chown d2/d71/d2d/d5b/fe7 3398 1 2024-02-28T23:07:57.755 INFO:tasks.workunit.client.0.smithi057.stdout:5/988: chown d4/d32/df1/d24/da2/l131 3415508 1 2024-02-28T23:07:57.757 INFO:tasks.workunit.client.0.smithi057.stdout:5/989: fsync d4/d9/d14/f117 0 2024-02-28T23:07:57.759 INFO:tasks.workunit.client.0.smithi057.stdout:4/966: unlink d1a/d1e/f5b 0 2024-02-28T23:07:57.760 INFO:tasks.workunit.client.0.smithi057.stdout:5/990: write d4/d9/d16/f12d [990983,108580] 0 2024-02-28T23:07:57.762 INFO:tasks.workunit.client.0.smithi057.stdout:5/991: readlink d4/d32/l99 0 2024-02-28T23:07:57.764 INFO:tasks.workunit.client.0.smithi057.stdout:6/974: mknod d0/d1b/d3e/d3d/c142 0 2024-02-28T23:07:57.765 INFO:tasks.workunit.client.0.smithi057.stdout:7/957: getdents d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0 0 2024-02-28T23:07:57.768 INFO:tasks.workunit.client.0.smithi057.stdout:7/958: chown d0/d98/fef 6596 1 2024-02-28T23:07:57.770 INFO:tasks.workunit.client.0.smithi057.stdout:6/975: write d0/d4/fb4 [29832,11549] 0 2024-02-28T23:07:57.770 INFO:tasks.workunit.client.0.smithi057.stdout:4/967: mkdir d1a/d2f/db2/dcb/dfe/d144 0 2024-02-28T23:07:57.774 INFO:tasks.workunit.client.0.smithi057.stdout:5/992: rename d4/d9/d16/d52/dfd/da1 to d4/d32/df1/d24/d72/d134 0 2024-02-28T23:07:57.775 INFO:tasks.workunit.client.0.smithi057.stdout:4/968: fdatasync d1a/d48/d9c/d68/dc5/f104 0 2024-02-28T23:07:57.777 INFO:tasks.workunit.client.0.smithi057.stdout:5/993: dread - d4/d7/d8/f124 zero size 2024-02-28T23:07:57.780 INFO:tasks.workunit.client.0.smithi057.stdout:4/969: read - d1a/d48/d9c/d68/fd7 zero size 2024-02-28T23:07:57.780 INFO:tasks.workunit.client.0.smithi057.stdout:6/976: symlink d0/d17/d20/d141/l143 0 2024-02-28T23:07:57.782 INFO:tasks.workunit.client.0.smithi057.stdout:3/944: dwrite d3/f2c [4194304,4194304] 0 2024-02-28T23:07:57.786 INFO:tasks.workunit.client.0.smithi057.stdout:4/970: truncate d1a/d2f/db2/d49/d4d/dc9/dff/d125/fc0 5227875 0 2024-02-28T23:07:57.788 INFO:tasks.workunit.client.0.smithi057.stdout:3/945: creat d3/d4/d1d/d47/f138 x:0 0 0 2024-02-28T23:07:57.796 INFO:tasks.workunit.client.0.smithi057.stdout:6/977: creat d0/f144 x:0 0 0 2024-02-28T23:07:57.803 INFO:tasks.workunit.client.0.smithi057.stdout:7/959: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/f28 [0,4194304] 0 2024-02-28T23:07:57.804 INFO:tasks.workunit.client.0.smithi057.stdout:6/978: dread - d0/d17/d94/da0/de6/df4/d102/ffa zero size 2024-02-28T23:07:57.806 INFO:tasks.workunit.client.0.smithi057.stdout:4/971: dwrite d1a/d1e/d51/dc3/fce [0,4194304] 0 2024-02-28T23:07:57.807 INFO:tasks.workunit.client.0.smithi057.stdout:9/927: sync 2024-02-28T23:07:57.815 INFO:tasks.workunit.client.0.smithi057.stdout:3/946: dwrite d3/d4/d1d/d47/f138 [0,4194304] 0 2024-02-28T23:07:57.821 INFO:tasks.workunit.client.0.smithi057.stdout:4/972: mkdir d1a/d2f/db2/dcb/dfe/d145 0 2024-02-28T23:07:57.822 INFO:tasks.workunit.client.0.smithi057.stdout:6/979: symlink d0/d4/d25/l145 0 2024-02-28T23:07:57.826 INFO:tasks.workunit.client.0.smithi057.stdout:6/980: mkdir d0/d1b/d3e/d146 0 2024-02-28T23:07:57.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.828+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 --> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1444000bf0 con 0x7f144c06cfa0 2024-02-28T23:07:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.830+0000 7f1465600700 1 -- 172.21.15.57:0/1554443415 <== mgr.24375 v2:172.21.15.84:6828/1810777701 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f1444000bf0 con 0x7f144c06cfa0 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "1/2 daemons upgraded", 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mgr daemons", 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:07:57.830 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:07:57.831 INFO:tasks.workunit.client.0.smithi057.stdout:9/928: rename dd/d46/dc4/c121 to dd/de/d14/d48/d76/dbf/c12f 0 2024-02-28T23:07:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 msgr2=0x7f144c06f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 0x7f144c06f450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1450009870 tx=0x7f145000b5a0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 msgr2=0x7f1468197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:07:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f146000d8d0 tx=0x7f146000dc90 comp rx=0 tx=0).stop 2024-02-28T23:07:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 shutdown_connections 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1468102440 0x7f1468197d20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.84:6828/1810777701,v1:172.21.15.84:6829/1810777701] conn(0x7f144c06cfa0 0x7f144c06f450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 --2- 172.21.15.57:0/1554443415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468103640 0x7f1468198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.834+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 >> 172.21.15.57:0/1554443415 conn(0x7f14680fd9f0 msgr2=0x7f1468106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.835+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 shutdown_connections 2024-02-28T23:07:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:07:57.835+0000 7f146f800700 1 -- 172.21.15.57:0/1554443415 wait complete. 2024-02-28T23:07:57.835 INFO:tasks.workunit.client.0.smithi057.stdout:9/929: mknod dd/de/c130 0 2024-02-28T23:07:57.837 INFO:tasks.workunit.client.0.smithi057.stdout:6/981: getdents d0/d1b/d3e/d3d/d42/d4f/d92 0 2024-02-28T23:07:57.844 INFO:tasks.workunit.client.0.smithi057.stdout:4/973: write d1a/d1e/fa8 [3313586,71999] 0 2024-02-28T23:07:57.850 INFO:tasks.workunit.client.0.smithi057.stdout:3/947: truncate d3/d4/d1d/d47/f138 138949 0 2024-02-28T23:07:57.853 INFO:tasks.workunit.client.0.smithi057.stdout:4/974: write d1a/d2f/db2/d49/d4d/f130 [2495582,19729] 0 2024-02-28T23:07:57.864 INFO:tasks.workunit.client.0.smithi057.stdout:6/982: dwrite d0/d17/d94/da0/db3/f13c [0,4194304] 0 2024-02-28T23:07:57.874 INFO:tasks.workunit.client.0.smithi057.stdout:9/930: dwrite dd/de/d24/d63/d77/dcf/d40/d44/f7a [4194304,4194304] 0 2024-02-28T23:07:57.875 INFO:tasks.workunit.client.0.smithi057.stdout:7/960: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f46 [0,4194304] 0 2024-02-28T23:07:57.878 INFO:tasks.workunit.client.0.smithi057.stdout:6/983: symlink d0/d17/d94/da0/de6/df4/l147 0 2024-02-28T23:07:57.879 INFO:tasks.workunit.client.0.smithi057.stdout:3/948: dwrite d3/d4/d1d/fde [4194304,4194304] 0 2024-02-28T23:07:57.879 INFO:tasks.workunit.client.0.smithi057.stdout:7/961: dread - d0/d98/f121 zero size 2024-02-28T23:07:57.883 INFO:tasks.workunit.client.0.smithi057.stdout:6/984: write d0/d17/d20/fe9 [194956,121991] 0 2024-02-28T23:07:57.884 INFO:tasks.workunit.client.0.smithi057.stdout:9/931: mkdir dd/de/d24/d63/d77/dcf/d40/d44/d92/d131 0 2024-02-28T23:07:57.886 INFO:tasks.workunit.client.0.smithi057.stdout:9/932: dread - dd/d46/dc4/de5/f11d zero size 2024-02-28T23:07:57.902 INFO:tasks.workunit.client.0.smithi057.stdout:4/975: dread d1a/d2f/db2/d49/d4d/dc9/dff/d125/f3e [0,4194304] 0 2024-02-28T23:07:57.961 INFO:tasks.workunit.client.0.smithi057.stdout:7/962: creat d0/d3f/d76/dc1/f140 x:0 0 0 2024-02-28T23:07:57.965 INFO:tasks.workunit.client.0.smithi057.stdout:9/933: truncate dd/f5b 819809 0 2024-02-28T23:07:57.970 INFO:tasks.workunit.client.0.smithi057.stdout:3/949: symlink d3/d4/d7/l139 0 2024-02-28T23:07:57.972 INFO:tasks.workunit.client.0.smithi057.stdout:7/963: fsync d0/f50 0 2024-02-28T23:07:57.986 INFO:tasks.workunit.client.0.smithi057.stdout:3/950: dread d3/d8/d27/f3e [0,4194304] 0 2024-02-28T23:07:57.994 INFO:tasks.workunit.client.0.smithi057.stdout:3/951: symlink d3/d20/d2d/l13a 0 2024-02-28T23:07:57.996 INFO:tasks.workunit.client.0.smithi057.stdout:4/976: getdents d1a/d2f/db2/d49/d4d/dc9/dec 0 2024-02-28T23:07:57.998 INFO:tasks.workunit.client.0.smithi057.stdout:4/977: chown d1a/d48/f61 243 1 2024-02-28T23:07:58.001 INFO:tasks.workunit.client.0.smithi057.stdout:3/952: unlink d3/d4/d7/le6 0 2024-02-28T23:07:58.012 INFO:tasks.workunit.client.0.smithi057.stdout:4/978: getdents d1a/d2f/db2/d49/d4d/dc9/dff 0 2024-02-28T23:07:58.032 INFO:tasks.workunit.client.0.smithi057.stdout:3/953: dread d3/d8/f3c [0,4194304] 0 2024-02-28T23:07:58.039 INFO:tasks.workunit.client.0.smithi057.stdout:1/903: sync 2024-02-28T23:07:58.040 INFO:tasks.workunit.client.0.smithi057.stdout:5/994: sync 2024-02-28T23:07:58.043 INFO:tasks.workunit.client.0.smithi057.stdout:1/904: read d2/d71/d2d/d39/f138 [1183757,54545] 0 2024-02-28T23:07:58.058 INFO:tasks.workunit.client.0.smithi057.stdout:5/995: dwrite d4/d7/d58/f96 [0,4194304] 0 2024-02-28T23:07:58.079 INFO:tasks.workunit.client.0.smithi057.stdout:6/985: write d0/d4/fa [350173,33714] 0 2024-02-28T23:07:58.085 INFO:tasks.workunit.client.0.smithi057.stdout:6/986: write d0/d17/d94/da0/de6/df4/d102/ffa [854131,112375] 0 2024-02-28T23:07:58.096 INFO:tasks.workunit.client.0.smithi057.stdout:6/987: truncate d0/d1b/d3e/d3d/d42/f8f 56665 0 2024-02-28T23:07:58.101 INFO:tasks.workunit.client.0.smithi057.stdout:7/964: dwrite d0/d3f/d76/dc1/dea/d75/f8a [0,4194304] 0 2024-02-28T23:07:58.102 INFO:tasks.workunit.client.0.smithi057.stdout:3/954: truncate d3/d4/f12 4359388 0 2024-02-28T23:07:58.102 INFO:tasks.workunit.client.0.smithi057.stdout:6/988: chown d0/d1b/f27 1 1 2024-02-28T23:07:58.102 INFO:tasks.workunit.client.0.smithi057.stdout:9/934: dwrite dd/de/d14/f61 [0,4194304] 0 2024-02-28T23:07:58.108 INFO:tasks.workunit.client.0.smithi057.stdout:6/989: write d0/d17/d94/da0/db3/f10c [911255,14078] 0 2024-02-28T23:07:58.108 INFO:tasks.workunit.client.0.smithi057.stdout:7/965: creat d0/d3f/d76/dc1/f141 x:0 0 0 2024-02-28T23:07:58.110 INFO:tasks.workunit.client.0.smithi057.stdout:4/979: dwrite d1a/d48/d9c/d2e/df8/f142 [0,4194304] 0 2024-02-28T23:07:58.116 INFO:tasks.workunit.client.0.smithi057.stdout:3/955: dread d3/d4/d1d/d47/d62/faf [0,4194304] 0 2024-02-28T23:07:58.125 INFO:tasks.workunit.client.0.smithi057.stdout:6/990: mkdir d0/d1b/d3e/d148 0 2024-02-28T23:07:58.126 INFO:tasks.workunit.client.0.smithi057.stdout:6/991: stat d0/lc7 0 2024-02-28T23:07:58.126 INFO:tasks.workunit.client.0.smithi057.stdout:4/980: rmdir d1a/d1e 39 2024-02-28T23:07:58.126 INFO:tasks.workunit.client.0.smithi057.stdout:9/935: mknod dd/d46/d4d/dc9/c132 0 2024-02-28T23:07:58.126 INFO:tasks.workunit.client.0.smithi057.stdout:6/992: read d0/d1b/d3e/d3d/d42/ffb [4963,91817] 0 2024-02-28T23:07:58.126 INFO:tasks.workunit.client.0.smithi057.stdout:5/996: dwrite d4/d9/d14/d3b/f47 [0,4194304] 0 2024-02-28T23:07:58.139 INFO:tasks.workunit.client.0.smithi057.stdout:5/997: mkdir d4/d9/d14/d3b/d61/dfa/d135 0 2024-02-28T23:07:58.140 INFO:tasks.workunit.client.0.smithi057.stdout:3/956: dread d3/d20/d2d/fb8 [0,4194304] 0 2024-02-28T23:07:58.140 INFO:tasks.workunit.client.0.smithi057.stdout:6/993: symlink d0/d4/d25/d60/d6a/d130/l149 0 2024-02-28T23:07:58.140 INFO:tasks.workunit.client.0.smithi057.stdout:9/936: link dd/de/d14/d48/d4b/l66 dd/de/d14/d48/d4b/d41/l133 0 2024-02-28T23:07:58.144 INFO:tasks.workunit.client.0.smithi057.stdout:4/981: getdents d1a/d48/d9c/d2e 0 2024-02-28T23:07:58.145 INFO:tasks.workunit.client.0.smithi057.stdout:9/937: dread - dd/de/d24/d63/d77/dcf/f10c zero size 2024-02-28T23:07:58.145 INFO:tasks.workunit.client.0.smithi057.stdout:5/998: creat d4/d32/d107/dfe/f136 x:0 0 0 2024-02-28T23:07:58.149 INFO:tasks.workunit.client.0.smithi057.stdout:3/957: unlink d3/d4/d7/f7f 0 2024-02-28T23:07:58.151 INFO:tasks.workunit.client.0.smithi057.stdout:6/994: link d0/d4/d25/fbe d0/d1b/f14a 0 2024-02-28T23:07:58.153 INFO:tasks.workunit.client.0.smithi057.stdout:5/999: rename d4/d32/df1/d24/f51 to d4/d7/f137 0 2024-02-28T23:07:58.155 INFO:tasks.workunit.client.0.smithi057.stdout:3/958: symlink d3/d4f/d4c/d91/de2/l13b 0 2024-02-28T23:07:58.164 INFO:tasks.workunit.client.0.smithi057.stdout:3/959: mknod d3/d4/d7/d9a/d9b/dab/dd0/c13c 0 2024-02-28T23:07:58.167 INFO:tasks.workunit.client.0.smithi057.stdout:3/960: read d3/d8/d27/f77 [2973211,113725] 0 2024-02-28T23:07:58.169 INFO:tasks.workunit.client.0.smithi057.stdout:6/995: dwrite d0/d1b/d3e/d3d/d74/fb8 [0,4194304] 0 2024-02-28T23:07:58.174 INFO:tasks.workunit.client.0.smithi057.stdout:9/938: truncate dd/de/f4f 993879 0 2024-02-28T23:07:58.174 INFO:tasks.workunit.client.0.smithi057.stdout:3/961: symlink d3/d4/d1d/d47/d62/db4/l13d 0 2024-02-28T23:07:58.179 INFO:tasks.workunit.client.0.smithi057.stdout:6/996: getdents d0/d17/d94/da0/de6 0 2024-02-28T23:07:58.181 INFO:tasks.workunit.client.0.smithi057.stdout:9/939: dread dd/de/d24/d63/d77/dcf/fe6 [0,4194304] 0 2024-02-28T23:07:58.182 INFO:tasks.workunit.client.0.smithi057.stdout:6/997: fsync d0/d1b/d3e/d100/f122 0 2024-02-28T23:07:58.186 INFO:tasks.workunit.client.0.smithi057.stdout:9/940: chown dd/de/l5e 582674219 1 2024-02-28T23:07:58.186 INFO:tasks.workunit.client.0.smithi057.stdout:6/998: symlink d0/d17/d94/da0/de6/df4/d102/l14b 0 2024-02-28T23:07:58.189 INFO:tasks.workunit.client.0.smithi057.stdout:3/962: dread d3/d8/f93 [0,4194304] 0 2024-02-28T23:07:58.189 INFO:tasks.workunit.client.0.smithi057.stdout:6/999: fsync d0/d1b/d3e/d100/f123 0 2024-02-28T23:07:58.189 INFO:tasks.workunit.client.0.smithi057.stdout:9/941: symlink dd/de/d24/d63/d77/dcf/l134 0 2024-02-28T23:07:58.190 INFO:tasks.workunit.client.0.smithi057.stdout:9/942: stat dd/de/d24/d63/dc1/de0 0 2024-02-28T23:07:58.194 INFO:tasks.workunit.client.0.smithi057.stdout:9/943: getdents dd/de/d14/dc2 0 2024-02-28T23:07:58.196 INFO:tasks.workunit.client.0.smithi057.stdout:9/944: creat dd/d46/d4d/f135 x:0 0 0 2024-02-28T23:07:58.199 INFO:tasks.workunit.client.0.smithi057.stdout:3/963: dread d3/d8/f9c [4194304,4194304] 0 2024-02-28T23:07:58.200 INFO:tasks.workunit.client.0.smithi057.stdout:9/945: mkdir dd/d46/d4d/d136 0 2024-02-28T23:07:58.202 INFO:tasks.workunit.client.0.smithi057.stdout:3/964: dread - d3/d4f/d4c/f89 zero size 2024-02-28T23:07:58.204 INFO:tasks.workunit.client.0.smithi057.stdout:3/965: chown d3/d8/d27/f55 110864 1 2024-02-28T23:07:58.206 INFO:tasks.workunit.client.0.smithi057.stdout:3/966: mknod d3/d8/c13e 0 2024-02-28T23:07:58.209 INFO:tasks.workunit.client.0.smithi057.stdout:9/946: truncate dd/de/d14/d48/d76/dbf/fd2 142186 0 2024-02-28T23:07:58.210 INFO:tasks.workunit.client.0.smithi057.stdout:9/947: fsync dd/de/d4a/d60/f83 0 2024-02-28T23:07:58.213 INFO:tasks.workunit.client.0.smithi057.stdout:9/948: symlink dd/l137 0 2024-02-28T23:07:58.217 INFO:tasks.workunit.client.0.smithi057.stdout:9/949: mknod dd/de/d24/d127/c138 0 2024-02-28T23:07:58.218 INFO:tasks.workunit.client.0.smithi057.stdout:3/967: dread d3/d4/f30 [0,4194304] 0 2024-02-28T23:07:58.220 INFO:tasks.workunit.client.0.smithi057.stdout:3/968: mkdir d3/d4/d7/d9a/d129/d13f 0 2024-02-28T23:07:58.221 INFO:tasks.workunit.client.0.smithi057.stdout:9/950: truncate dd/de/d14/d48/d4b/fec 252921 0 2024-02-28T23:07:58.223 INFO:tasks.workunit.client.0.smithi057.stdout:3/969: chown d3/d4/d7/fa9 987 1 2024-02-28T23:07:58.223 INFO:tasks.workunit.client.0.smithi057.stdout:9/951: mkdir dd/de/d14/d48/d76/dbf/d139 0 2024-02-28T23:07:58.226 INFO:tasks.workunit.client.0.smithi057.stdout:9/952: creat dd/d46/dc4/dd3/d105/f13a x:0 0 0 2024-02-28T23:07:58.228 INFO:tasks.workunit.client.0.smithi057.stdout:9/953: symlink dd/d46/d4d/d136/l13b 0 2024-02-28T23:07:58.235 INFO:tasks.workunit.client.0.smithi057.stdout:9/954: rmdir dd/d46/dc4/dd3/d105 39 2024-02-28T23:07:58.239 INFO:tasks.workunit.client.0.smithi057.stdout:9/955: link dd/de/d4a/d9f/cf4 dd/de/d4a/d9f/c13c 0 2024-02-28T23:07:58.259 INFO:tasks.workunit.client.0.smithi057.stdout:9/956: dwrite dd/de/d24/d72/f91 [0,4194304] 0 2024-02-28T23:07:58.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:58 smithi084 ceph-mon[34826]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:58.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:58 smithi084 ceph-mon[34826]: pgmap v35: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 87 MiB/s rd, 192 MiB/s wr, 455 op/s 2024-02-28T23:07:58.410 INFO:tasks.workunit.client.0.smithi057.stdout:9/957: readlink dd/de/d14/d48/d4b/l64 0 2024-02-28T23:07:58.456 INFO:tasks.workunit.client.0.smithi057.stdout:9/958: dwrite dd/de/d4a/d9f/fab [0,4194304] 0 2024-02-28T23:07:58.460 INFO:tasks.workunit.client.0.smithi057.stdout:9/959: rmdir dd/d46/dc4/de5 39 2024-02-28T23:07:58.464 INFO:tasks.workunit.client.0.smithi057.stdout:9/960: dread - dd/de/d24/d63/d77/dcf/d40/d44/d5c/f122 zero size 2024-02-28T23:07:58.468 INFO:tasks.workunit.client.0.smithi057.stdout:9/961: symlink dd/de/d24/d63/d77/l13d 0 2024-02-28T23:07:58.477 INFO:tasks.workunit.client.0.smithi057.stdout:9/962: write dd/de/d14/f17 [8322432,14777] 0 2024-02-28T23:07:58.489 INFO:tasks.workunit.client.0.smithi057.stdout:9/963: write dd/d46/d6f/f79 [4640673,46372] 0 2024-02-28T23:07:58.503 INFO:tasks.workunit.client.0.smithi057.stdout:9/964: dwrite dd/de/d4a/d60/fd8 [0,4194304] 0 2024-02-28T23:07:58.509 INFO:tasks.workunit.client.0.smithi057.stdout:9/965: rmdir dd 39 2024-02-28T23:07:58.519 INFO:tasks.workunit.client.0.smithi057.stdout:9/966: rename dd/de/d4a/d9f/cf4 to dd/de/d4a/d9f/c13e 0 2024-02-28T23:07:58.519 INFO:tasks.workunit.client.0.smithi057.stdout:9/967: fsync dd/f114 0 2024-02-28T23:07:58.532 INFO:tasks.workunit.client.0.smithi057.stdout:9/968: dwrite dd/de/d14/f5a [0,4194304] 0 2024-02-28T23:07:58.536 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:58 smithi057 ceph-mon[29096]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:58.536 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:58 smithi057 ceph-mon[29096]: pgmap v35: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 87 MiB/s rd, 192 MiB/s wr, 455 op/s 2024-02-28T23:07:58.554 INFO:tasks.workunit.client.0.smithi057.stdout:9/969: dwrite dd/de/d24/d63/d77/dcf/d40/fb2 [0,4194304] 0 2024-02-28T23:07:58.557 INFO:tasks.workunit.client.0.smithi057.stdout:9/970: stat dd/d46/dc4/dd3/d105/f13a 0 2024-02-28T23:07:58.564 INFO:tasks.workunit.client.0.smithi057.stdout:9/971: write dd/de/d14/d48/d4b/d41/f93 [1404594,21577] 0 2024-02-28T23:07:58.565 INFO:tasks.workunit.client.0.smithi057.stdout:1/905: sync 2024-02-28T23:07:58.565 INFO:tasks.workunit.client.0.smithi057.stdout:7/966: sync 2024-02-28T23:07:58.568 INFO:tasks.workunit.client.0.smithi057.stdout:1/906: chown d2/d7/df/db/d12/d19/da4 8063 1 2024-02-28T23:07:58.574 INFO:tasks.workunit.client.0.smithi057.stdout:1/907: write d2/d71/d2d/d39/da3/ff6 [102031,129452] 0 2024-02-28T23:07:58.578 INFO:tasks.workunit.client.0.smithi057.stdout:1/908: rmdir d2/d7/df/db/d12/d19/d80 39 2024-02-28T23:07:58.582 INFO:tasks.workunit.client.0.smithi057.stdout:1/909: mkdir d2/d71/d2d/d5b/d141 0 2024-02-28T23:07:58.585 INFO:tasks.workunit.client.0.smithi057.stdout:9/972: dwrite dd/de/d24/fa6 [0,4194304] 0 2024-02-28T23:07:58.587 INFO:tasks.workunit.client.0.smithi057.stdout:1/910: rmdir d2/d7/df/db/d3a/d5a/dd3 39 2024-02-28T23:07:58.589 INFO:tasks.workunit.client.0.smithi057.stdout:1/911: rmdir d2/d33 39 2024-02-28T23:07:58.592 INFO:tasks.workunit.client.0.smithi057.stdout:1/912: readlink d2/d7/df/db/d12/dd6/le6 0 2024-02-28T23:07:58.597 INFO:tasks.workunit.client.0.smithi057.stdout:1/913: symlink d2/d7/df/db/d12/d19/da4/l142 0 2024-02-28T23:07:58.599 INFO:tasks.workunit.client.0.smithi057.stdout:9/973: dread dd/de/d4a/d60/f75 [0,4194304] 0 2024-02-28T23:07:58.599 INFO:tasks.workunit.client.0.smithi057.stdout:1/914: dread - d2/d7/df/db/d12/d19/f124 zero size 2024-02-28T23:07:58.600 INFO:tasks.workunit.client.0.smithi057.stdout:7/967: dread d0/f85 [0,4194304] 0 2024-02-28T23:07:58.603 INFO:tasks.workunit.client.0.smithi057.stdout:9/974: chown dd/f5b 271689449 1 2024-02-28T23:07:58.603 INFO:tasks.workunit.client.0.smithi057.stdout:1/915: write d2/d71/d2d/f46 [1355520,20332] 0 2024-02-28T23:07:58.605 INFO:tasks.workunit.client.0.smithi057.stdout:1/916: chown d2/d7/df/db/d3a/l107 181236000 1 2024-02-28T23:07:58.608 INFO:tasks.workunit.client.0.smithi057.stdout:9/975: creat dd/de/d24/d63/d77/dcf/d40/d44/d92/f13f x:0 0 0 2024-02-28T23:07:58.609 INFO:tasks.workunit.client.0.smithi057.stdout:1/917: fsync d2/d7/df/db/d12/d3f/d47/f8e 0 2024-02-28T23:07:58.628 INFO:tasks.workunit.client.0.smithi057.stdout:7/968: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f137 [0,4194304] 0 2024-02-28T23:07:58.636 INFO:tasks.workunit.client.0.smithi057.stdout:9/976: dwrite dd/de/d14/f55 [0,4194304] 0 2024-02-28T23:07:58.638 INFO:tasks.workunit.client.0.smithi057.stdout:1/918: dwrite d2/d7/df/db/d12/d3f/d47/f9f [4194304,4194304] 0 2024-02-28T23:07:58.639 INFO:tasks.workunit.client.0.smithi057.stdout:9/977: dread - dd/de/d14/dc2/d101/f109 zero size 2024-02-28T23:07:58.640 INFO:tasks.workunit.client.0.smithi057.stdout:7/969: mkdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d142 0 2024-02-28T23:07:58.646 INFO:tasks.workunit.client.0.smithi057.stdout:7/970: unlink d0/d3f/d76/dc1/dea/c81 0 2024-02-28T23:07:58.649 INFO:tasks.workunit.client.0.smithi057.stdout:9/978: mknod dd/de/d24/d63/d77/dcf/d40/d44/d92/d131/c140 0 2024-02-28T23:07:58.656 INFO:tasks.workunit.client.0.smithi057.stdout:7/971: mkdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/d143 0 2024-02-28T23:07:58.658 INFO:tasks.workunit.client.0.smithi057.stdout:1/919: dread d2/d71/d2d/f46 [0,4194304] 0 2024-02-28T23:07:58.667 INFO:tasks.workunit.client.0.smithi057.stdout:1/920: truncate d2/d7/df/db/d12/d3f/d47/f119 562759 0 2024-02-28T23:07:58.674 INFO:tasks.workunit.client.0.smithi057.stdout:1/921: write d2/d71/d2d/f49 [1247682,87262] 0 2024-02-28T23:07:58.674 INFO:tasks.workunit.client.0.smithi057.stdout:1/922: stat d2/d7/df/db/d12/l59 0 2024-02-28T23:07:58.674 INFO:tasks.workunit.client.0.smithi057.stdout:9/979: dwrite dd/de/d4a/d60/f7e [0,4194304] 0 2024-02-28T23:07:58.681 INFO:tasks.workunit.client.0.smithi057.stdout:7/972: write d0/d3f/d76/dc1/dea/d75/fa5 [3903869,88462] 0 2024-02-28T23:07:58.682 INFO:tasks.workunit.client.0.smithi057.stdout:1/923: creat d2/d7/d4f/f143 x:0 0 0 2024-02-28T23:07:58.686 INFO:tasks.workunit.client.0.smithi057.stdout:9/980: rename dd/f31 to dd/de/d4a/d60/d8c/d12e/f141 0 2024-02-28T23:07:58.688 INFO:tasks.workunit.client.0.smithi057.stdout:1/924: read d2/d7/df/db/d10b/d6b/fb9 [154759,36804] 0 2024-02-28T23:07:58.693 INFO:tasks.workunit.client.0.smithi057.stdout:7/973: rename d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/f8c to d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/da0/f144 0 2024-02-28T23:07:58.693 INFO:tasks.workunit.client.0.smithi057.stdout:9/981: dread dd/de/d14/d48/d4b/d41/ff5 [0,4194304] 0 2024-02-28T23:07:58.695 INFO:tasks.workunit.client.0.smithi057.stdout:1/925: creat d2/d7/df/db/d3a/d5a/dd3/f144 x:0 0 0 2024-02-28T23:07:58.698 INFO:tasks.workunit.client.0.smithi057.stdout:3/970: sync 2024-02-28T23:07:58.698 INFO:tasks.workunit.client.0.smithi057.stdout:4/982: sync 2024-02-28T23:07:58.699 INFO:tasks.workunit.client.0.smithi057.stdout:7/974: symlink d0/d3f/d76/dc1/dea/d75/de8/l145 0 2024-02-28T23:07:58.702 INFO:tasks.workunit.client.0.smithi057.stdout:9/982: symlink dd/de/d24/d63/d77/dcf/d40/d44/d92/d131/l142 0 2024-02-28T23:07:58.704 INFO:tasks.workunit.client.0.smithi057.stdout:1/926: symlink d2/d7/df/db/d3a/d8f/l145 0 2024-02-28T23:07:58.707 INFO:tasks.workunit.client.0.smithi057.stdout:3/971: creat d3/d4/d7/d9a/d9b/dab/f140 x:0 0 0 2024-02-28T23:07:58.708 INFO:tasks.workunit.client.0.smithi057.stdout:9/983: symlink dd/de/d4a/d60/d8c/d12e/l143 0 2024-02-28T23:07:58.716 INFO:tasks.workunit.client.0.smithi057.stdout:9/984: creat dd/d46/d4d/d136/f144 x:0 0 0 2024-02-28T23:07:58.717 INFO:tasks.workunit.client.0.smithi057.stdout:3/972: creat d3/d4/d7/d9a/d9b/dab/dd0/f141 x:0 0 0 2024-02-28T23:07:58.718 INFO:tasks.workunit.client.0.smithi057.stdout:7/975: truncate d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f1a 850440 0 2024-02-28T23:07:58.719 INFO:tasks.workunit.client.0.smithi057.stdout:3/973: chown d3/d8/f3c 1114 1 2024-02-28T23:07:58.721 INFO:tasks.workunit.client.0.smithi057.stdout:4/983: dwrite d1a/d2f/db2/dcb/f105 [0,4194304] 0 2024-02-28T23:07:58.724 INFO:tasks.workunit.client.0.smithi057.stdout:9/985: truncate dd/de/d24/d63/d77/dcf/f10c 162061 0 2024-02-28T23:07:58.727 INFO:tasks.workunit.client.0.smithi057.stdout:3/974: mkdir d3/d4f/d4c/d91/de2/d142 0 2024-02-28T23:07:58.730 INFO:tasks.workunit.client.0.smithi057.stdout:4/984: unlink d1a/d48/d9c/d2e/ld8 0 2024-02-28T23:07:58.731 INFO:tasks.workunit.client.0.smithi057.stdout:3/975: readlink d3/d8/d27/lbd 0 2024-02-28T23:07:58.732 INFO:tasks.workunit.client.0.smithi057.stdout:1/927: dwrite d2/d7/df/db/fdb [0,4194304] 0 2024-02-28T23:07:58.740 INFO:tasks.workunit.client.0.smithi057.stdout:9/986: mknod dd/de/d14/d48/c145 0 2024-02-28T23:07:58.740 INFO:tasks.workunit.client.0.smithi057.stdout:7/976: mknod d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/c146 0 2024-02-28T23:07:58.742 INFO:tasks.workunit.client.0.smithi057.stdout:4/985: write d1a/d2f/db2/d49/d4d/dc9/dff/d125/f3e [5051128,56457] 0 2024-02-28T23:07:58.742 INFO:tasks.workunit.client.0.smithi057.stdout:1/928: symlink d2/d7/df/l146 0 2024-02-28T23:07:58.743 INFO:tasks.workunit.client.0.smithi057.stdout:9/987: rmdir dd/de/d4a 39 2024-02-28T23:07:58.743 INFO:tasks.workunit.client.0.smithi057.stdout:3/976: write d3/d20/d2d/f82 [3287478,4788] 0 2024-02-28T23:07:58.752 INFO:tasks.workunit.client.0.smithi057.stdout:1/929: readlink d2/d7/df/db/d3a/l107 0 2024-02-28T23:07:58.754 INFO:tasks.workunit.client.0.smithi057.stdout:1/930: fdatasync d2/d71/d2d/d5b/f7d 0 2024-02-28T23:07:58.754 INFO:tasks.workunit.client.0.smithi057.stdout:4/986: rename d1a/d48/d6c/f8b to d1a/d2f/db2/dcb/f146 0 2024-02-28T23:07:58.757 INFO:tasks.workunit.client.0.smithi057.stdout:3/977: getdents d3/d4f/d101 0 2024-02-28T23:07:58.764 INFO:tasks.workunit.client.0.smithi057.stdout:3/978: truncate d3/d20/d2d/fb8 1595120 0 2024-02-28T23:07:58.766 INFO:tasks.workunit.client.0.smithi057.stdout:7/977: dwrite d0/d3f/d76/dc1/dea/d75/f94 [0,4194304] 0 2024-02-28T23:07:58.773 INFO:tasks.workunit.client.0.smithi057.stdout:3/979: mknod d3/d4/d1d/d47/c143 0 2024-02-28T23:07:58.778 INFO:tasks.workunit.client.0.smithi057.stdout:9/988: dwrite dd/de/d14/dc2/d101/f109 [0,4194304] 0 2024-02-28T23:07:58.782 INFO:tasks.workunit.client.0.smithi057.stdout:3/980: truncate d3/d4/d7/fb9 5160545 0 2024-02-28T23:07:58.786 INFO:tasks.workunit.client.0.smithi057.stdout:4/987: getdents d1a/d2f/db2/d49/d4d/de2/d6e/db4 0 2024-02-28T23:07:58.792 INFO:tasks.workunit.client.0.smithi057.stdout:3/981: link d3/d4/d1d/f10b d3/d8/d27/d70/f144 0 2024-02-28T23:07:58.795 INFO:tasks.workunit.client.0.smithi057.stdout:9/989: symlink dd/de/d14/d48/d76/l146 0 2024-02-28T23:07:58.799 INFO:tasks.workunit.client.0.smithi057.stdout:4/988: unlink d1a/d1e/d81/fca 0 2024-02-28T23:07:58.803 INFO:tasks.workunit.client.0.smithi057.stdout:7/978: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/c1b d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/c147 0 2024-02-28T23:07:58.809 INFO:tasks.workunit.client.0.smithi057.stdout:3/982: rmdir d3/d4f/d4c/d91/de2/d142 0 2024-02-28T23:07:58.812 INFO:tasks.workunit.client.0.smithi057.stdout:7/979: write d0/d3f/d76/fb8 [1236921,86582] 0 2024-02-28T23:07:58.813 INFO:tasks.workunit.client.0.smithi057.stdout:1/931: dread d2/d7/df/db/d12/d19/d65/f96 [0,4194304] 0 2024-02-28T23:07:58.817 INFO:tasks.workunit.client.0.smithi057.stdout:3/983: write d3/d4/d1d/fde [238706,123085] 0 2024-02-28T23:07:58.820 INFO:tasks.workunit.client.0.smithi057.stdout:1/932: write d2/d7/df/db/d10b/f13e [688030,50028] 0 2024-02-28T23:07:58.824 INFO:tasks.workunit.client.0.smithi057.stdout:3/984: mkdir d3/d20/d145 0 2024-02-28T23:07:58.831 INFO:tasks.workunit.client.0.smithi057.stdout:1/933: symlink d2/d7/df/db/d3a/d66/l147 0 2024-02-28T23:07:58.834 INFO:tasks.workunit.client.0.smithi057.stdout:4/989: dwrite d1a/d48/d9c/f38 [0,4194304] 0 2024-02-28T23:07:58.835 INFO:tasks.workunit.client.0.smithi057.stdout:1/934: chown d2/d71/def/ff2 11399056 1 2024-02-28T23:07:58.837 INFO:tasks.workunit.client.0.smithi057.stdout:9/990: dwrite dd/de/d14/d48/d4b/d41/ff5 [0,4194304] 0 2024-02-28T23:07:58.838 INFO:tasks.workunit.client.0.smithi057.stdout:4/990: stat d1a/d48/d9c/d2e/l11e 0 2024-02-28T23:07:58.850 INFO:tasks.workunit.client.0.smithi057.stdout:9/991: readlink dd/de/d24/d63/d77/dcf/d40/d44/d92/ldf 0 2024-02-28T23:07:58.850 INFO:tasks.workunit.client.0.smithi057.stdout:9/992: fsync dd/de/d14/f61 0 2024-02-28T23:07:58.855 INFO:tasks.workunit.client.0.smithi057.stdout:3/985: link d3/d4/d7/df8/l116 d3/d4/d1d/d47/l146 0 2024-02-28T23:07:58.856 INFO:tasks.workunit.client.0.smithi057.stdout:7/980: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f1a [0,4194304] 0 2024-02-28T23:07:58.868 INFO:tasks.workunit.client.0.smithi057.stdout:7/981: creat d0/d3f/d76/dc1/dea/d75/dbf/d10b/d4a/f148 x:0 0 0 2024-02-28T23:07:58.869 INFO:tasks.workunit.client.0.smithi057.stdout:3/986: symlink d3/d4/d7/d9a/d129/l147 0 2024-02-28T23:07:58.875 INFO:tasks.workunit.client.0.smithi057.stdout:1/935: dwrite d2/d33/d54/fc7 [0,4194304] 0 2024-02-28T23:07:58.877 INFO:tasks.workunit.client.0.smithi057.stdout:3/987: fdatasync d3/d4/d7/d9a/d129/f137 0 2024-02-28T23:07:58.885 INFO:tasks.workunit.client.0.smithi057.stdout:9/993: dwrite dd/de/d14/f1e [0,4194304] 0 2024-02-28T23:07:58.912 INFO:tasks.workunit.client.0.smithi057.stdout:7/982: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/f49 [0,4194304] 0 2024-02-28T23:07:58.951 INFO:tasks.workunit.client.0.smithi057.stdout:1/936: link d2/d7/d4f/f143 d2/d7/df/db/d12/d19/da4/f148 0 2024-02-28T23:07:58.951 INFO:tasks.workunit.client.0.smithi057.stdout:7/983: mknod d0/d3f/d76/dc1/dea/d75/dbf/c149 0 2024-02-28T23:07:58.952 INFO:tasks.workunit.client.0.smithi057.stdout:3/988: getdents d3/d8/d27 0 2024-02-28T23:07:58.954 INFO:tasks.workunit.client.0.smithi057.stdout:1/937: mknod d2/dd4/c149 0 2024-02-28T23:07:58.955 INFO:tasks.workunit.client.0.smithi057.stdout:3/989: chown d3/d4/dfd/l103 0 1 2024-02-28T23:07:58.958 INFO:tasks.workunit.client.0.smithi057.stdout:1/938: truncate d2/fe9 127359 0 2024-02-28T23:07:58.961 INFO:tasks.workunit.client.0.smithi057.stdout:3/990: getdents d3/d8 0 2024-02-28T23:07:58.963 INFO:tasks.workunit.client.0.smithi057.stdout:3/991: dread - d3/d4/d1d/d47/f128 zero size 2024-02-28T23:07:58.966 INFO:tasks.workunit.client.0.smithi057.stdout:7/984: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d4a/f105 [771848,12687] 0 2024-02-28T23:07:58.968 INFO:tasks.workunit.client.0.smithi057.stdout:3/992: write d3/d4/d7/fb9 [1763087,6661] 0 2024-02-28T23:07:58.978 INFO:tasks.workunit.client.0.smithi057.stdout:7/985: unlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/f137 0 2024-02-28T23:07:58.980 INFO:tasks.workunit.client.0.smithi057.stdout:3/993: write d3/d4f/f127 [634438,127123] 0 2024-02-28T23:07:58.984 INFO:tasks.workunit.client.0.smithi057.stdout:1/939: dwrite d2/d7/df/db/d3a/f6c [0,4194304] 0 2024-02-28T23:07:58.987 INFO:tasks.workunit.client.0.smithi057.stdout:7/986: chown d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d/de7/l120 998 1 2024-02-28T23:07:58.989 INFO:tasks.workunit.client.0.smithi057.stdout:3/994: rename d3/d8/l2a to d3/d4/d7/d9a/d9b/dab/l148 0 2024-02-28T23:07:58.990 INFO:tasks.workunit.client.0.smithi057.stdout:7/987: chown d0/d3f/d76/dc1/f127 21401953 1 2024-02-28T23:07:58.991 INFO:tasks.workunit.client.0.smithi057.stdout:1/940: truncate d2/d7/df/db/d3a/f99 3308322 0 2024-02-28T23:07:58.993 INFO:tasks.workunit.client.0.smithi057.stdout:1/941: fsync d2/d7/df/db/d10b/f13e 0 2024-02-28T23:07:58.994 INFO:tasks.workunit.client.0.smithi057.stdout:3/995: creat d3/d8/d1b/f149 x:0 0 0 2024-02-28T23:07:58.997 INFO:tasks.workunit.client.0.smithi057.stdout:7/988: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/fbc [1504680,108844] 0 2024-02-28T23:07:59.003 INFO:tasks.workunit.client.0.smithi057.stdout:7/989: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/l14a 0 2024-02-28T23:07:59.008 INFO:tasks.workunit.client.0.smithi057.stdout:3/996: dread d3/f64 [0,4194304] 0 2024-02-28T23:07:59.013 INFO:tasks.workunit.client.0.smithi057.stdout:3/997: symlink d3/d4/d7/d9a/l14a 0 2024-02-28T23:07:59.014 INFO:tasks.workunit.client.0.smithi057.stdout:7/990: link d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/f79 d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/f14b 0 2024-02-28T23:07:59.019 INFO:tasks.workunit.client.0.smithi057.stdout:3/998: symlink d3/d4/d1d/l14b 0 2024-02-28T23:07:59.020 INFO:tasks.workunit.client.0.smithi057.stdout:3/999: chown d3/f118 2041855 1 2024-02-28T23:07:59.020 INFO:tasks.workunit.client.0.smithi057.stdout:1/942: dread d2/d71/d62/fe0 [0,4194304] 0 2024-02-28T23:07:59.023 INFO:tasks.workunit.client.0.smithi057.stdout:7/991: dread d0/d3f/d76/fb8 [0,4194304] 0 2024-02-28T23:07:59.024 INFO:tasks.workunit.client.0.smithi057.stdout:1/943: creat d2/dd4/dd5/d118/dff/f14a x:0 0 0 2024-02-28T23:07:59.038 INFO:tasks.workunit.client.0.smithi057.stdout:7/992: write d0/d3f/d76/dc1/dea/d75/dbf/d10b/f2e [790282,4108] 0 2024-02-28T23:07:59.050 INFO:tasks.workunit.client.0.smithi057.stdout:7/993: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/l14c 0 2024-02-28T23:07:59.078 INFO:tasks.workunit.client.0.smithi057.stdout:7/994: dread d0/d3f/d76/dc1/dea/d75/dbf/d10b/d25/d80/d9f/fc4 [0,4194304] 0 2024-02-28T23:07:59.085 INFO:tasks.workunit.client.0.smithi057.stdout:7/995: symlink d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/l14d 0 2024-02-28T23:07:59.102 INFO:tasks.workunit.client.0.smithi057.stdout:7/996: dwrite d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d119/f122 [0,4194304] 0 2024-02-28T23:07:59.109 INFO:tasks.workunit.client.0.smithi057.stdout:7/997: rmdir d0/d3f/d76/dc1/dea 39 2024-02-28T23:07:59.110 INFO:tasks.workunit.client.0.smithi057.stdout:7/998: symlink d0/d98/l14e 0 2024-02-28T23:07:59.116 INFO:tasks.workunit.client.0.smithi057.stdout:7/999: rmdir d0/d3f/d76/dc1/dea/d75/dbf/d10b/d36/d4e/d15/d29/d3d 39 2024-02-28T23:07:59.134 INFO:tasks.workunit.client.0.smithi057.stdout:4/991: sync 2024-02-28T23:07:59.134 INFO:tasks.workunit.client.0.smithi057.stdout:9/994: sync 2024-02-28T23:07:59.135 INFO:tasks.workunit.client.0.smithi057.stdout:4/992: chown d1a/d2f/db2/d49 854999831 1 2024-02-28T23:07:59.138 INFO:tasks.workunit.client.0.smithi057.stdout:9/995: truncate dd/de/d24/d63/d77/dcf/d40/fb2 5214272 0 2024-02-28T23:07:59.138 INFO:tasks.workunit.client.0.smithi057.stdout:4/993: truncate d1a/d2f/db2/d49/d4d/dc9/fd5 5084072 0 2024-02-28T23:07:59.141 INFO:tasks.workunit.client.0.smithi057.stdout:9/996: creat dd/de/d24/d72/f147 x:0 0 0 2024-02-28T23:07:59.143 INFO:tasks.workunit.client.0.smithi057.stdout:4/994: mknod d1a/c147 0 2024-02-28T23:07:59.156 INFO:tasks.workunit.client.0.smithi057.stdout:4/995: dwrite d1a/d48/db0/f11c [0,4194304] 0 2024-02-28T23:07:59.179 INFO:tasks.workunit.client.0.smithi057.stdout:4/996: dwrite d1a/d1e/fe6 [0,4194304] 0 2024-02-28T23:07:59.185 INFO:tasks.workunit.client.0.smithi057.stdout:4/997: symlink d1a/d2f/db2/d49/d4d/dc9/l148 0 2024-02-28T23:07:59.195 INFO:tasks.workunit.client.0.smithi057.stdout:4/998: rename d1a/d48/d9c/d2e to d1a/d2f/db2/d49/d4d/de2/d6e/d149 0 2024-02-28T23:07:59.199 INFO:tasks.workunit.client.0.smithi057.stdout:4/999: mkdir d1a/d2f/db2/d49/d4d/de2/d6e/d100/d14a 0 2024-02-28T23:07:59.324 INFO:tasks.workunit.client.0.smithi057.stdout:1/944: sync 2024-02-28T23:07:59.328 INFO:tasks.workunit.client.0.smithi057.stdout:1/945: readlink d2/d7/df/db/d12/l30 0 2024-02-28T23:07:59.342 INFO:tasks.workunit.client.0.smithi057.stdout:9/997: sync 2024-02-28T23:07:59.351 INFO:tasks.workunit.client.0.smithi057.stdout:9/998: dread dd/de/d14/d48/d4b/d41/fad [0,4194304] 0 2024-02-28T23:07:59.365 INFO:tasks.workunit.client.0.smithi057.stdout:1/946: dread d2/d33/d54/dc8/ffe [0,4194304] 0 2024-02-28T23:07:59.367 INFO:tasks.workunit.client.0.smithi057.stdout:1/947: readlink d2/d7/df/db/d3a/l60 0 2024-02-28T23:07:59.370 INFO:tasks.workunit.client.0.smithi057.stdout:9/999: dwrite dd/de/d14/d48/d4b/d41/f93 [0,4194304] 0 2024-02-28T23:07:59.373 INFO:tasks.workunit.client.0.smithi057.stdout:1/948: rmdir d2/d71/d2d/d39/da3/db5 39 2024-02-28T23:07:59.394 INFO:tasks.workunit.client.0.smithi057.stdout:1/949: dwrite d2/d33/f76 [0,4194304] 0 2024-02-28T23:07:59.405 INFO:tasks.workunit.client.0.smithi057.stdout:1/950: link d2/d7/df/db/d3a/d2a/d82/fe4 d2/d71/d2d/d5b/f14b 0 2024-02-28T23:07:59.412 INFO:tasks.workunit.client.0.smithi057.stdout:1/951: rename d2/d7/df/db/fdb to d2/d7/df/db/d12/d3f/f14c 0 2024-02-28T23:07:59.428 INFO:tasks.workunit.client.0.smithi057.stdout:1/952: write d2/d71/d62/fa6 [2823060,71042] 0 2024-02-28T23:07:59.443 INFO:tasks.workunit.client.0.smithi057.stdout:1/953: dread d2/d7/df/db/d3a/d2a/f6d [0,4194304] 0 2024-02-28T23:07:59.449 INFO:tasks.workunit.client.0.smithi057.stdout:1/954: symlink d2/d90/l14d 0 2024-02-28T23:07:59.457 INFO:tasks.workunit.client.0.smithi057.stdout:1/955: mknod d2/d71/d2d/d39/da3/db5/c14e 0 2024-02-28T23:07:59.464 INFO:tasks.workunit.client.0.smithi057.stdout:1/956: unlink d2/d71/d2d/d39/da3/lb2 0 2024-02-28T23:07:59.473 INFO:tasks.workunit.client.0.smithi057.stdout:1/957: truncate d2/d7/df/db/d12/f26 1762055 0 2024-02-28T23:07:59.482 INFO:tasks.workunit.client.0.smithi057.stdout:1/958: unlink d2/d7/df/db/d3a/d8f/l108 0 2024-02-28T23:07:59.486 INFO:tasks.workunit.client.0.smithi057.stdout:1/959: symlink d2/d33/d54/dc8/dfa/l14f 0 2024-02-28T23:07:59.490 INFO:tasks.workunit.client.0.smithi057.stdout:1/960: creat d2/d7/df/db/d10b/d6b/f150 x:0 0 0 2024-02-28T23:07:59.500 INFO:tasks.workunit.client.0.smithi057.stdout:1/961: unlink d2/d7/df/db/d3a/d5a/dd3/f144 0 2024-02-28T23:07:59.504 INFO:tasks.workunit.client.0.smithi057.stdout:1/962: truncate d2/d7/d4f/f63 559455 0 2024-02-28T23:07:59.507 INFO:tasks.workunit.client.0.smithi057.stdout:1/963: readlink d2/d7/df/db/d12/d19/da4/l142 0 2024-02-28T23:07:59.527 INFO:tasks.workunit.client.0.smithi057.stdout:1/964: dread d2/d7/df/db/f28 [0,4194304] 0 2024-02-28T23:07:59.529 INFO:tasks.workunit.client.0.smithi057.stdout:1/965: symlink d2/d71/def/l151 0 2024-02-28T23:07:59.552 INFO:tasks.workunit.client.0.smithi057.stdout:1/966: dwrite d2/d7/df/f52 [0,4194304] 0 2024-02-28T23:07:59.557 INFO:tasks.workunit.client.0.smithi057.stdout:1/967: creat d2/d7/df/db/d12/f152 x:0 0 0 2024-02-28T23:07:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:59 smithi057 ceph-mon[29096]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:59 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:59 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:07:59 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:07:59.578 INFO:tasks.workunit.client.0.smithi057.stdout:1/968: dread d2/d71/d2d/d39/da3/db5/fb7 [0,4194304] 0 2024-02-28T23:07:59.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:59 smithi084 ceph-mon[34826]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:07:59.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:59 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:59.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:59 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:07:59.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:07:59 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:07:59.580 INFO:tasks.workunit.client.0.smithi057.stdout:1/969: symlink d2/d7/d4f/db4/l153 0 2024-02-28T23:07:59.583 INFO:tasks.workunit.client.0.smithi057.stdout:1/970: fdatasync d2/d71/d2d/f13d 0 2024-02-28T23:07:59.584 INFO:tasks.workunit.client.0.smithi057.stdout:1/971: stat d2/cbb 0 2024-02-28T23:07:59.602 INFO:tasks.workunit.client.0.smithi057.stdout:1/972: dwrite d2/d33/d54/f8b [0,4194304] 0 2024-02-28T23:07:59.605 INFO:tasks.workunit.client.0.smithi057.stdout:1/973: write d2/dd4/dd5/d118/dff/d100/f12a [121348,45929] 0 2024-02-28T23:07:59.615 INFO:tasks.workunit.client.0.smithi057.stdout:1/974: mkdir d2/d71/d154 0 2024-02-28T23:07:59.632 INFO:tasks.workunit.client.0.smithi057.stdout:1/975: dwrite d2/d71/d2d/d5b/fe7 [0,4194304] 0 2024-02-28T23:07:59.635 INFO:tasks.workunit.client.0.smithi057.stdout:1/976: creat d2/d7/df/db/d12/d3f/f155 x:0 0 0 2024-02-28T23:07:59.638 INFO:tasks.workunit.client.0.smithi057.stdout:1/977: creat d2/d7/df/db/d3a/d2a/f156 x:0 0 0 2024-02-28T23:07:59.643 INFO:tasks.workunit.client.0.smithi057.stdout:1/978: mknod d2/d71/d2d/d39/da3/c157 0 2024-02-28T23:07:59.645 INFO:tasks.workunit.client.0.smithi057.stdout:1/979: write d2/f10 [1417650,73185] 0 2024-02-28T23:07:59.650 INFO:tasks.workunit.client.0.smithi057.stdout:1/980: unlink d2/d7/df/db/d3a/d66/fbc 0 2024-02-28T23:07:59.654 INFO:tasks.workunit.client.0.smithi057.stdout:1/981: creat d2/d7/df/db/d12/d3f/f158 x:0 0 0 2024-02-28T23:07:59.664 INFO:tasks.workunit.client.0.smithi057.stdout:1/982: write d2/d7/df/db/d3a/d2a/d82/fe4 [6261910,41275] 0 2024-02-28T23:07:59.686 INFO:tasks.workunit.client.0.smithi057.stdout:1/983: dread d2/f44 [0,4194304] 0 2024-02-28T23:07:59.687 INFO:tasks.workunit.client.0.smithi057.stdout:1/984: readlink d2/d90/l14d 0 2024-02-28T23:07:59.689 INFO:tasks.workunit.client.0.smithi057.stdout:1/985: chown d2/d71/d2d/d39/da3/c126 27 1 2024-02-28T23:07:59.693 INFO:tasks.workunit.client.0.smithi057.stdout:1/986: truncate d2/d33/fe2 335897 0 2024-02-28T23:07:59.696 INFO:tasks.workunit.client.0.smithi057.stdout:1/987: symlink d2/d71/def/l159 0 2024-02-28T23:07:59.699 INFO:tasks.workunit.client.0.smithi057.stdout:1/988: mknod d2/d7/df/db/d3a/d5a/dd3/c15a 0 2024-02-28T23:07:59.705 INFO:tasks.workunit.client.0.smithi057.stdout:1/989: dread d2/d7/df/db/d10b/d6b/fb9 [0,4194304] 0 2024-02-28T23:07:59.711 INFO:tasks.workunit.client.0.smithi057.stdout:1/990: creat d2/d7/df/db/d12/d19/f15b x:0 0 0 2024-02-28T23:07:59.713 INFO:tasks.workunit.client.0.smithi057.stdout:1/991: mknod d2/d90/c15c 0 2024-02-28T23:07:59.716 INFO:tasks.workunit.client.0.smithi057.stdout:1/992: fdatasync d2/d7/df/db/d12/d3f/f8c 0 2024-02-28T23:07:59.728 INFO:tasks.workunit.client.0.smithi057.stdout:1/993: link d2/d7/df/db/d3a/l107 d2/d7/df/db/d12/d19/l15d 0 2024-02-28T23:07:59.730 INFO:tasks.workunit.client.0.smithi057.stdout:1/994: fdatasync d2/d71/d62/fe0 0 2024-02-28T23:07:59.733 INFO:tasks.workunit.client.0.smithi057.stdout:1/995: rmdir d2/dd4/dd5/d118/dff 39 2024-02-28T23:07:59.741 INFO:tasks.workunit.client.0.smithi057.stdout:1/996: truncate d2/d71/d2d/d5b/f135 411022 0 2024-02-28T23:07:59.747 INFO:tasks.workunit.client.0.smithi057.stdout:1/997: write d2/d7/d4f/fd9 [734143,33039] 0 2024-02-28T23:07:59.764 INFO:tasks.workunit.client.0.smithi057.stdout:1/998: dread d2/d71/d2d/f49 [0,4194304] 0 2024-02-28T23:07:59.771 INFO:tasks.workunit.client.0.smithi057.stdout:1/999: symlink d2/d71/l15e 0 2024-02-28T23:07:59.780 INFO:tasks.workunit.client.0.smithi057.stderr:+ rm -rf -- ./tmp.6RtNsWrv6j 2024-02-28T23:08:00.467 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:00 smithi057 ceph-mon[29096]: pgmap v36: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 208 MiB/s wr, 500 op/s 2024-02-28T23:08:00.467 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:00 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:00.467 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:00 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:00.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:00 smithi084 ceph-mon[34826]: pgmap v36: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 208 MiB/s wr, 500 op/s 2024-02-28T23:08:00.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:00 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:00.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:00 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:01 smithi057 ceph-mon[29096]: pgmap v37: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 205 MiB/s wr, 550 op/s 2024-02-28T23:08:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:01 smithi084 ceph-mon[34826]: pgmap v37: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 205 MiB/s wr, 550 op/s 2024-02-28T23:08:04.509 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:04 smithi057 ceph-mon[29096]: pgmap v38: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 58 MiB/s rd, 130 MiB/s wr, 352 op/s 2024-02-28T23:08:04.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:04 smithi084 ceph-mon[34826]: pgmap v38: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 58 MiB/s rd, 130 MiB/s wr, 352 op/s 2024-02-28T23:08:06.443 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:06 smithi057 ceph-mon[29096]: pgmap v39: 65 pgs: 65 active+clean; 3.2 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 133 MiB/s wr, 433 op/s 2024-02-28T23:08:06.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:06 smithi084 ceph-mon[34826]: pgmap v39: 65 pgs: 65 active+clean; 3.2 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 133 MiB/s wr, 433 op/s 2024-02-28T23:08:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:07 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:07 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:07 smithi057 ceph-mon[29096]: pgmap v40: 65 pgs: 65 active+clean; 3.2 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 23 MiB/s rd, 45 MiB/s wr, 224 op/s 2024-02-28T23:08:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:07 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:07 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:07 smithi084 ceph-mon[34826]: pgmap v40: 65 pgs: 65 active+clean; 3.2 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 23 MiB/s rd, 45 MiB/s wr, 224 op/s 2024-02-28T23:08:09.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:08 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:08 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:10.427 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:10 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:10.427 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:10 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:10.427 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:10 smithi057 ceph-mon[29096]: pgmap v41: 65 pgs: 65 active+clean; 2.9 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 23 MiB/s rd, 45 MiB/s wr, 238 op/s 2024-02-28T23:08:10.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:10 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:10 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:10 smithi084 ceph-mon[34826]: pgmap v41: 65 pgs: 65 active+clean; 2.9 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 23 MiB/s rd, 45 MiB/s wr, 238 op/s 2024-02-28T23:08:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: Standby manager daemon smithi057.zrrldj restarted 2024-02-28T23:08:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: Standby manager daemon smithi057.zrrldj started 2024-02-28T23:08:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/crt"}]: dispatch 2024-02-28T23:08:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:08:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/key"}]: dispatch 2024-02-28T23:08:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:08:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:11 smithi057 ceph-mon[29096]: pgmap v42: 65 pgs: 65 active+clean; 2.2 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 17 MiB/s rd, 29 MiB/s wr, 254 op/s 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: Standby manager daemon smithi057.zrrldj restarted 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: Standby manager daemon smithi057.zrrldj started 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/crt"}]: dispatch 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi057.zrrldj/key"}]: dispatch 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:08:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:11 smithi084 ceph-mon[34826]: pgmap v42: 65 pgs: 65 active+clean; 2.2 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 17 MiB/s rd, 29 MiB/s wr, 254 op/s 2024-02-28T23:08:12.552 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:12 smithi057 ceph-mon[29096]: mgrmap e26: smithi084.fhkzis(active, since 78s), standbys: smithi057.zrrldj 2024-02-28T23:08:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:12 smithi084 ceph-mon[34826]: mgrmap e26: smithi084.fhkzis(active, since 78s), standbys: smithi057.zrrldj 2024-02-28T23:08:13.246 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-28T23:08:13.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-28T23:08:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:13 smithi084 ceph-mon[34826]: pgmap v43: 65 pgs: 65 active+clean; 2.2 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 155 op/s 2024-02-28T23:08:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:13 smithi057 ceph-mon[29096]: pgmap v43: 65 pgs: 65 active+clean; 2.2 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 155 op/s 2024-02-28T23:08:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: osdmap e49: 6 total, 6 up, 6 in 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd='[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]': finished 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: mgrmap e27: smithi057.zrrldj(active, starting, since 0.010536s) 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:08:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:08:15.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: Manager daemon smithi057.zrrldj is now available 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:08:15.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:15 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 172.21.15.84:0/553492475' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: osdmap e49: 6 total, 6 up, 6 in 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.24375 ' entity='mgr.smithi084.fhkzis' cmd='[{"prefix": "mgr fail", "who": "smithi084.fhkzis"}]': finished 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: mgrmap e27: smithi057.zrrldj(active, starting, since 0.010536s) 2024-02-28T23:08:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:08:15.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: Manager daemon smithi057.zrrldj is now available 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:08:15.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:15 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T23:08:16.767 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:16 smithi084 ceph-mon[34826]: mgrmap e28: smithi057.zrrldj(active, since 1.01354s) 2024-02-28T23:08:16.768 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:16 smithi084 ceph-mon[34826]: pgmap v3: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:16.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:16 smithi057 ceph-mon[29096]: mgrmap e28: smithi057.zrrldj(active, since 1.01354s) 2024-02-28T23:08:16.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:16 smithi057 ceph-mon[29096]: pgmap v3: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:17 smithi084 ceph-mon[34826]: [28/Feb/2024:23:08:16] ENGINE Bus STARTING 2024-02-28T23:08:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:17 smithi084 ceph-mon[34826]: [28/Feb/2024:23:08:16] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T23:08:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:17 smithi084 ceph-mon[34826]: [28/Feb/2024:23:08:16] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T23:08:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:17 smithi084 ceph-mon[34826]: [28/Feb/2024:23:08:16] ENGINE Bus STARTED 2024-02-28T23:08:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:17 smithi084 ceph-mon[34826]: pgmap v4: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:17 smithi057 ceph-mon[29096]: [28/Feb/2024:23:08:16] ENGINE Bus STARTING 2024-02-28T23:08:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:17 smithi057 ceph-mon[29096]: [28/Feb/2024:23:08:16] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T23:08:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:17 smithi057 ceph-mon[29096]: [28/Feb/2024:23:08:16] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T23:08:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:17 smithi057 ceph-mon[29096]: [28/Feb/2024:23:08:16] ENGINE Bus STARTED 2024-02-28T23:08:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:17 smithi057 ceph-mon[29096]: pgmap v4: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:18.531 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:18 smithi084 ceph-mon[34826]: mgrmap e29: smithi057.zrrldj(active, since 3s) 2024-02-28T23:08:18.564 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-28T23:08:18.565 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-28T23:08:18.650 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:18 smithi057 ceph-mon[29096]: mgrmap e29: smithi057.zrrldj(active, since 3s) 2024-02-28T23:08:19.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:19 smithi057 ceph-mon[29096]: pgmap v5: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:19.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:19 smithi084 ceph-mon[34826]: pgmap v5: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:21.522 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:21 smithi084 ceph-mon[34826]: pgmap v6: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:21 smithi057 ceph-mon[29096]: pgmap v6: 65 pgs: 65 active+clean; 876 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-02-28T23:08:23.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:23 smithi084 ceph-mon[34826]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 888 KiB/s rd, 804 KiB/s wr, 193 op/s 2024-02-28T23:08:23.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:23 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:23.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:23 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:23 smithi057 ceph-mon[29096]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 888 KiB/s rd, 804 KiB/s wr, 193 op/s 2024-02-28T23:08:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:23 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:23 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:25.720 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:25 smithi084 ceph-mon[34826]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 690 KiB/s rd, 625 KiB/s wr, 150 op/s 2024-02-28T23:08:25.720 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:25.720 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:25 smithi057 ceph-mon[29096]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 690 KiB/s rd, 625 KiB/s wr, 150 op/s 2024-02-28T23:08:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:26.652 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:26 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:08:26.652 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:26 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:08:26.653 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:26 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:08:26.653 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:26 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:08:26.653 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:26 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:08:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:26 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:08:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:26 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:08:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:26 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:08:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:26 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:08:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:26 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/3749252562' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:08:27.701 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:27 smithi057 ceph-mon[29096]: pgmap v9: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-02-28T23:08:27.701 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:27 smithi057 ceph-mon[29096]: mgrmap e30: smithi057.zrrldj(active, since 12s), standbys: smithi084.fhkzis 2024-02-28T23:08:27.702 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:27 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:27 smithi084 ceph-mon[34826]: pgmap v9: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-02-28T23:08:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:27 smithi084 ceph-mon[34826]: mgrmap e30: smithi057.zrrldj(active, since 12s), standbys: smithi084.fhkzis 2024-02-28T23:08:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:27 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:08:28.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 -- 172.21.15.57:0/4290576684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24074d20 msgr2=0x7f8c24073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 --2- 172.21.15.57:0/4290576684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24074d20 0x7f8c24073180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b0a0 tx=0x7f8c1400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 -- 172.21.15.57:0/4290576684 shutdown_connections 2024-02-28T23:08:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 --2- 172.21.15.57:0/4290576684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24073750 0x7f8c24073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 --2- 172.21.15.57:0/4290576684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24074d20 0x7f8c24073180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.087+0000 7f8c28c00700 1 -- 172.21.15.57:0/4290576684 >> 172.21.15.57:0/4290576684 conn(0x7f8c240fb7f0 msgr2=0x7f8c240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:28.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.089+0000 7f8c28c00700 1 -- 172.21.15.57:0/4290576684 shutdown_connections 2024-02-28T23:08:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.090+0000 7f8c28c00700 1 -- 172.21.15.57:0/4290576684 wait complete. 2024-02-28T23:08:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.091+0000 7f8c28c00700 1 Processor -- start 2024-02-28T23:08:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.092+0000 7f8c28c00700 1 -- start start 2024-02-28T23:08:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.093+0000 7f8c28c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.094+0000 7f8c28c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 0x7f8c240722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.094+0000 7f8c28c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240728e0 con 0x7f8c24074d20 2024-02-28T23:08:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.095+0000 7f8c28c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c241a9e50 con 0x7f8c24073750 2024-02-28T23:08:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.095+0000 7f8c21e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.096+0000 7f8c21e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:59968/0 (socket says 172.21.15.57:59968) 2024-02-28T23:08:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.096+0000 7f8c21e00700 1 -- 172.21.15.57:0/1345682210 learned_addr learned my addr 172.21.15.57:0/1345682210 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:08:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.096+0000 7f8c1be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 0x7f8c240722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.097+0000 7f8c21e00700 1 -- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 msgr2=0x7f8c240722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.098+0000 7f8c21e00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 0x7f8c240722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.098+0000 7f8c21e00700 1 -- 172.21.15.57:0/1345682210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14009d00 con 0x7f8c24073750 2024-02-28T23:08:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.098+0000 7f8c1be00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 0x7f8c240722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:08:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.099+0000 7f8c21e00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b830 tx=0x7f8c14003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.100+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1400e040 con 0x7f8c24073750 2024-02-28T23:08:28.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.100+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c14007e90 con 0x7f8c24073750 2024-02-28T23:08:28.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.100+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c241a9ff0 con 0x7f8c24073750 2024-02-28T23:08:28.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.101+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c14012850 con 0x7f8c24073750 2024-02-28T23:08:28.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.102+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c241aa460 con 0x7f8c24073750 2024-02-28T23:08:28.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.106+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2410f380 con 0x7f8c24073750 2024-02-28T23:08:28.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.107+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f8c14019070 con 0x7f8c24073750 2024-02-28T23:08:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.110+0000 7f8c1aa00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 0x7f8c0806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.111+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8c1408e240 con 0x7f8c24073750 2024-02-28T23:08:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.112+0000 7f8c1be00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 0x7f8c0806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:28.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.117+0000 7f8c1be00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 0x7f8c0806fb40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c007fd0 tx=0x7f8c0c0073d0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:28.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.118+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8c1405b950 con 0x7f8c24073750 2024-02-28T23:08:28.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.523+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c241032f0 con 0x7f8c0806d690 2024-02-28T23:08:28.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.525+0000 7f8c1aa00700 1 -- 172.21.15.57:0/1345682210 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f8c241032f0 con 0x7f8c0806d690 2024-02-28T23:08:28.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 msgr2=0x7f8c0806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 0x7f8c0806fb40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c007fd0 tx=0x7f8c0c0073d0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 msgr2=0x7f8c24071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b830 tx=0x7f8c14003c30 comp rx=0 tx=0).stop 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 shutdown_connections 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f8c0806d690 0x7f8c0806fb40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c24073750 0x7f8c24071d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 --2- 172.21.15.57:0/1345682210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c24074d20 0x7f8c240722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 >> 172.21.15.57:0/1345682210 conn(0x7f8c240fb7f0 msgr2=0x7f8c24101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 shutdown_connections 2024-02-28T23:08:28.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.529+0000 7f8c28c00700 1 -- 172.21.15.57:0/1345682210 wait complete. 2024-02-28T23:08:28.547 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:08:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.713+0000 7f655e800700 1 -- 172.21.15.57:0/690316692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558102490 msgr2=0x7f65581028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.713+0000 7f655e800700 1 --2- 172.21.15.57:0/690316692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558102490 0x7f65581028a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6548009910 tx=0x7f6548009c20 comp rx=0 tx=0).stop 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 -- 172.21.15.57:0/690316692 shutdown_connections 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 --2- 172.21.15.57:0/690316692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558103640 0x7f6558103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 --2- 172.21.15.57:0/690316692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558102490 0x7f65581028a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 -- 172.21.15.57:0/690316692 >> 172.21.15.57:0/690316692 conn(0x7f65580fda60 msgr2=0x7f65580ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 -- 172.21.15.57:0/690316692 shutdown_connections 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.714+0000 7f655e800700 1 -- 172.21.15.57:0/690316692 wait complete. 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.715+0000 7f655e800700 1 Processor -- start 2024-02-28T23:08:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.715+0000 7f655e800700 1 -- start start 2024-02-28T23:08:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f655e800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f655e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558103640 0x7f6558198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f655e800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558198860 con 0x7f6558103640 2024-02-28T23:08:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f655e800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655819d270 con 0x7f6558102490 2024-02-28T23:08:28.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f6557400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:28.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f6557400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:59978/0 (socket says 172.21.15.57:59978) 2024-02-28T23:08:28.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.716+0000 7f6557400700 1 -- 172.21.15.57:0/3454603093 learned_addr learned my addr 172.21.15.57:0/3454603093 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:08:28.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.717+0000 7f6557400700 1 -- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558103640 msgr2=0x7f6558198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:28.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.717+0000 7f6557400700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558103640 0x7f6558198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:28.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.717+0000 7f6557400700 1 -- 172.21.15.57:0/3454603093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480095f0 con 0x7f6558102490 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.718+0000 7f6557400700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654800bda0 tx=0x7f654800be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.719+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654801d070 con 0x7f6558102490 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.719+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f654800f460 con 0x7f6558102490 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.719+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6548021620 con 0x7f6558102490 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.719+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655819d410 con 0x7f6558102490 2024-02-28T23:08:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.719+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655819d8d0 con 0x7f6558102490 2024-02-28T23:08:28.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.721+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655810af90 con 0x7f6558102490 2024-02-28T23:08:28.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.722+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f654800f5d0 con 0x7f6558102490 2024-02-28T23:08:28.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.723+0000 7f654fe00700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 0x7f653c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:28.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.723+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f654808e520 con 0x7f6558102490 2024-02-28T23:08:28.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.724+0000 7f6556a00700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 0x7f653c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:28.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.725+0000 7f6556a00700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 0x7f653c06fb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6544005950 tx=0x7f65440058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:28.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:28.745+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6548057f40 con 0x7f6558102490 2024-02-28T23:08:29.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.138+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6558107f90 con 0x7f653c06d6a0 2024-02-28T23:08:29.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.139+0000 7f654fe00700 1 -- 172.21.15.57:0/3454603093 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f6558107f90 con 0x7f653c06d6a0 2024-02-28T23:08:29.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 msgr2=0x7f653c06fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 0x7f653c06fb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6544005950 tx=0x7f65440058e0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 msgr2=0x7f6558197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654800bda0 tx=0x7f654800be80 comp rx=0 tx=0).stop 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 shutdown_connections 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f653c06d6a0 0x7f653c06fb50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6558102490 0x7f6558197d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 --2- 172.21.15.57:0/3454603093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6558103640 0x7f6558198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.142+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 >> 172.21.15.57:0/3454603093 conn(0x7f65580fda60 msgr2=0x7f6558106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.143+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 shutdown_connections 2024-02-28T23:08:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.143+0000 7f655e800700 1 -- 172.21.15.57:0/3454603093 wait complete. 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.302+0000 7f8405400700 1 -- 172.21.15.57:0/1580473245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 msgr2=0x7f8400073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.302+0000 7f8405400700 1 --2- 172.21.15.57:0/1580473245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f8400073180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009910 tx=0x7f83f0009c20 comp rx=0 tx=0).stop 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 -- 172.21.15.57:0/1580473245 shutdown_connections 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 --2- 172.21.15.57:0/1580473245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 0x7f8400073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 --2- 172.21.15.57:0/1580473245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f8400073180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 -- 172.21.15.57:0/1580473245 >> 172.21.15.57:0/1580473245 conn(0x7f84000fb7f0 msgr2=0x7f84000fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 -- 172.21.15.57:0/1580473245 shutdown_connections 2024-02-28T23:08:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.303+0000 7f8405400700 1 -- 172.21.15.57:0/1580473245 wait complete. 2024-02-28T23:08:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.304+0000 7f8405400700 1 Processor -- start 2024-02-28T23:08:29.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f8405400700 1 -- start start 2024-02-28T23:08:29.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f8405400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 0x7f8400197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f8405400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f8405400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8400198750 con 0x7f8400074d20 2024-02-28T23:08:29.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f8405400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84001988c0 con 0x7f8400073750 2024-02-28T23:08:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.305+0000 7f83fde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38740/0 (socket says 172.21.15.57:38740) 2024-02-28T23:08:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 -- 172.21.15.57:0/2094274927 learned_addr learned my addr 172.21.15.57:0/2094274927 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:08:29.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fe800700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 0x7f8400197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 -- 172.21.15.57:0/2094274927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 msgr2=0x7f8400197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 0x7f8400197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 -- 172.21.15.57:0/2094274927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00095f0 con 0x7f8400074d20 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.306+0000 7f83fde00700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00c930 tx=0x7f83ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.307+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ec007ad0 con 0x7f8400074d20 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.307+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83ec00ce80 con 0x7f8400074d20 2024-02-28T23:08:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.307+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ec018630 con 0x7f8400074d20 2024-02-28T23:08:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.308+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840019d360 con 0x7f8400074d20 2024-02-28T23:08:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.308+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840019d880 con 0x7f8400074d20 2024-02-28T23:08:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.309+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840010af80 con 0x7f8400074d20 2024-02-28T23:08:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.311+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f83ec007c30 con 0x7f8400074d20 2024-02-28T23:08:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.312+0000 7f83f7400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 0x7f83e406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.312+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f83ec08d220 con 0x7f8400074d20 2024-02-28T23:08:29.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.313+0000 7f83fe800700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 0x7f83e406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.314+0000 7f83fe800700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 0x7f83e406fb00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006010 tx=0x7f83f000b560 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.319+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f83ec056d40 con 0x7f8400074d20 2024-02-28T23:08:29.434 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:29 smithi057 ceph-mon[29096]: pgmap v10: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.0 MiB/s wr, 245 op/s 2024-02-28T23:08:29.434 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:29 smithi057 ceph-mon[29096]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:29.434 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:29.434 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:29.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.704+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84001077a0 con 0x7f83e406d650 2024-02-28T23:08:29.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.717+0000 7f83f7400700 1 -- 172.21.15.57:0/2094274927 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f84001077a0 con 0x7f83e406d650 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (9m) 1s ago 11m 14.3M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (8m) 1s ago 12m 7914k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (10m) 6s ago 10m 7843k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (12m) 1s ago 12m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (10m) 6s ago 10m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:08:29.716 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (9m) 1s ago 11m 74.0M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (4m) 1s ago 4m 85.5M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (4m) 1s ago 4m 147M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (4m) 6s ago 4m 87.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (4m) 6s ago 4m 142M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (33s) 1s ago 13m 473M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (116s) 6s ago 10m 70.0M - 19.0.0-1670-g1fb5d106 2272b2c42f9b ae1c65e2de51 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (13m) 1s ago 14m 51.7M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (10m) 6s ago 10m 42.7M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (12m) 1s ago 12m 8232k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (10m) 6s ago 10m 8120k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (9m) 1s ago 9m 196M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (8m) 1s ago 8m 218M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (8m) 1s ago 8m 172M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:08:29.717 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (7m) 6s ago 7m 244M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:08:29.718 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (7m) 6s ago 7m 203M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:08:29.718 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (6m) 6s ago 6m 215M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:08:29.718 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (63s) 1s ago 11m 47.2M - 2.43.0 a07b618ecd1d e554cbda23d0 2024-02-28T23:08:29.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.720+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 msgr2=0x7f83e406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.720+0000 7f8405400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 0x7f83e406fb00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006010 tx=0x7f83f000b560 comp rx=0 tx=0).stop 2024-02-28T23:08:29.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 msgr2=0x7f84001981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00c930 tx=0x7f83ec00cc40 comp rx=0 tx=0).stop 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 shutdown_connections 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f83e406d650 0x7f83e406fb00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8400073750 0x7f8400197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 --2- 172.21.15.57:0/2094274927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8400074d20 0x7f84001981e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 >> 172.21.15.57:0/2094274927 conn(0x7f84000fb7f0 msgr2=0x7f8400106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 shutdown_connections 2024-02-28T23:08:29.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.721+0000 7f8405400700 1 -- 172.21.15.57:0/2094274927 wait complete. 2024-02-28T23:08:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:29 smithi084 ceph-mon[34826]: pgmap v10: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.0 MiB/s wr, 245 op/s 2024-02-28T23:08:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:29 smithi084 ceph-mon[34826]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 -- 172.21.15.57:0/2622210147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c700737b0 msgr2=0x7f3c70073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 --2- 172.21.15.57:0/2622210147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c700737b0 0x7f3c70073c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009910 tx=0x7f3c5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 -- 172.21.15.57:0/2622210147 shutdown_connections 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 --2- 172.21.15.57:0/2622210147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c700737b0 0x7f3c70073c20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 --2- 172.21.15.57:0/2622210147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c70074d80 0x7f3c700731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 -- 172.21.15.57:0/2622210147 >> 172.21.15.57:0/2622210147 conn(0x7f3c700fb7f0 msgr2=0x7f3c700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:29.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 -- 172.21.15.57:0/2622210147 shutdown_connections 2024-02-28T23:08:29.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.917+0000 7f3c78600700 1 -- 172.21.15.57:0/2622210147 wait complete. 2024-02-28T23:08:29.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.918+0000 7f3c78600700 1 Processor -- start 2024-02-28T23:08:29.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.918+0000 7f3c78600700 1 -- start start 2024-02-28T23:08:29.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.919+0000 7f3c78600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.920+0000 7f3c78600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c70074d80 0x7f3c7019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.920+0000 7f3c78600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7019cbd0 con 0x7f3c70074d80 2024-02-28T23:08:29.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.920+0000 7f3c78600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7019cd10 con 0x7f3c700737b0 2024-02-28T23:08:29.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.920+0000 7f3c75c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:60018/0 (socket says 172.21.15.57:60018) 2024-02-28T23:08:29.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75c00700 1 -- 172.21.15.57:0/3072978714 learned_addr learned my addr 172.21.15.57:0/3072978714 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:08:29.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75200700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c70074d80 0x7f3c7019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75c00700 1 -- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c70074d80 msgr2=0x7f3c7019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75c00700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c70074d80 0x7f3c7019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.921+0000 7f3c75c00700 1 -- 172.21.15.57:0/3072978714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0095f0 con 0x7f3c700737b0 2024-02-28T23:08:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.922+0000 7f3c75c00700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400e920 tx=0x7f3c6400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.922+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c64009790 con 0x7f3c700737b0 2024-02-28T23:08:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.922+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c701a17c0 con 0x7f3c700737b0 2024-02-28T23:08:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.922+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c6400cba0 con 0x7f3c700737b0 2024-02-28T23:08:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.922+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c701a1d10 con 0x7f3c700737b0 2024-02-28T23:08:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.923+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c64018630 con 0x7f3c700737b0 2024-02-28T23:08:29.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.924+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7010f380 con 0x7f3c700737b0 2024-02-28T23:08:29.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.926+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f3c6400cd10 con 0x7f3c700737b0 2024-02-28T23:08:29.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.927+0000 7f3c6e000700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 0x7f3c5806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:29.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.927+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3c64014070 con 0x7f3c700737b0 2024-02-28T23:08:29.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.927+0000 7f3c75200700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 0x7f3c5806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:29.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.928+0000 7f3c75200700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 0x7f3c5806fb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b6f0 tx=0x7f3c5c009f90 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:29.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:29.941+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3c6405ade0 con 0x7f3c700737b0 2024-02-28T23:08:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.411+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3c701a2410 con 0x7f3c700737b0 2024-02-28T23:08:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.414+0000 7f3c6e000700 1 -- 172.21.15.57:0/3072978714 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f3c64026070 con 0x7f3c700737b0 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:08:30.413 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:08:30.414 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:08:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.417+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 msgr2=0x7f3c5806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.417+0000 7f3c78600700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 0x7f3c5806fb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b6f0 tx=0x7f3c5c009f90 comp rx=0 tx=0).stop 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.417+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 msgr2=0x7f3c7019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.417+0000 7f3c78600700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400e920 tx=0x7f3c6400ece0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 shutdown_connections 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3c5806d650 0x7f3c5806fb00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700737b0 0x7f3c7019c070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 --2- 172.21.15.57:0/3072978714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c70074d80 0x7f3c7019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 >> 172.21.15.57:0/3072978714 conn(0x7f3c700fb7f0 msgr2=0x7f3c70101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.418+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 shutdown_connections 2024-02-28T23:08:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.419+0000 7f3c78600700 1 -- 172.21.15.57:0/3072978714 wait complete. 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.657+0000 7f2b6e800700 1 -- 172.21.15.57:0/3931321184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b68103310 msgr2=0x7f2b68105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.657+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3931321184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b68103310 0x7f2b68105790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009910 tx=0x7f2b54009c20 comp rx=0 tx=0).stop 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 -- 172.21.15.57:0/3931321184 shutdown_connections 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3931321184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b68103310 0x7f2b68105790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3931321184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68069270 0x7f2b68102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 -- 172.21.15.57:0/3931321184 >> 172.21.15.57:0/3931321184 conn(0x7f2b680fa780 msgr2=0x7f2b680fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 -- 172.21.15.57:0/3931321184 shutdown_connections 2024-02-28T23:08:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.658+0000 7f2b6e800700 1 -- 172.21.15.57:0/3931321184 wait complete. 2024-02-28T23:08:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.659+0000 7f2b6e800700 1 Processor -- start 2024-02-28T23:08:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.660+0000 7f2b6e800700 1 -- start start 2024-02-28T23:08:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.660+0000 7f2b6e800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:30.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.660+0000 7f2b6e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:30.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.660+0000 7f2b6e800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68112870 con 0x7f2b6810ed00 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.660+0000 7f2b6e800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681129e0 con 0x7f2b68103310 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.661+0000 7f2b66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.661+0000 7f2b67400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.661+0000 7f2b66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38760/0 (socket says 172.21.15.57:38760) 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.661+0000 7f2b66a00700 1 -- 172.21.15.57:0/3410439478 learned_addr learned my addr 172.21.15.57:0/3410439478 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:08:30.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.661+0000 7f2b67400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:60038/0 (socket says 172.21.15.57:60038) 2024-02-28T23:08:30.661 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:30 smithi057 ceph-mon[29096]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:30.661 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:30.662 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:30 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3072978714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:08:30.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.662+0000 7f2b67400700 1 -- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 msgr2=0x7f2b6810f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.662+0000 7f2b67400700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.662+0000 7f2b67400700 1 -- 172.21.15.57:0/3410439478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540095f0 con 0x7f2b68103310 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.662+0000 7f2b66a00700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.663+0000 7f2b67400700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800e940 tx=0x7f2b5800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.663+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5800c990 con 0x7f2b68103310 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.664+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b58010430 con 0x7f2b68103310 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.664+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6810f7d0 con 0x7f2b68103310 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.664+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b58018630 con 0x7f2b68103310 2024-02-28T23:08:30.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.664+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b681aaab0 con 0x7f2b68103310 2024-02-28T23:08:30.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.666+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b68108c00 con 0x7f2b68103310 2024-02-28T23:08:30.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.667+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f2b580105a0 con 0x7f2b68103310 2024-02-28T23:08:30.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.668+0000 7f2b5fe00700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 0x7f2b4c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:08:30.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.669+0000 7f2b66a00700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 0x7f2b4c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:08:30.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.670+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2b58014070 con 0x7f2b68103310 2024-02-28T23:08:30.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.670+0000 7f2b66a00700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 0x7f2b4c06fb50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b6f0 tx=0x7f2b54009f90 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:08:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.682+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f2b5805aeb0 con 0x7f2b68103310 2024-02-28T23:08:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:30 smithi084 ceph-mon[34826]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:30 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3072978714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:08:30.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.978+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b6802cda0 con 0x7f2b4c06d6a0 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.980+0000 7f2b5fe00700 1 -- 172.21.15.57:0/3410439478 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f2b6802cda0 con 0x7f2b4c06d6a0 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:08:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:08:30.980 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2024-02-28T23:08:30.980 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2024-02-28T23:08:30.980 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:08:30.980 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:08:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.985+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 msgr2=0x7f2b4c06fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 0x7f2b4c06fb50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b6f0 tx=0x7f2b54009f90 comp rx=0 tx=0).stop 2024-02-28T23:08:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 msgr2=0x7f2b681122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:08:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800e940 tx=0x7f2b5800ec50 comp rx=0 tx=0).stop 2024-02-28T23:08:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 shutdown_connections 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f2b4c06d6a0 0x7f2b4c06fb50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b68103310 0x7f2b681122a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 --2- 172.21.15.57:0/3410439478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b6810ed00 0x7f2b6810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 >> 172.21.15.57:0/3410439478 conn(0x7f2b680fa780 msgr2=0x7f2b680fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.986+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 shutdown_connections 2024-02-28T23:08:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:08:30.987+0000 7f2b6e800700 1 -- 172.21.15.57:0/3410439478 wait complete. 2024-02-28T23:08:31.731 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:31 smithi084 ceph-mon[34826]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:31.731 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:31 smithi084 ceph-mon[34826]: pgmap v11: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.0 MiB/s wr, 245 op/s 2024-02-28T23:08:31.731 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:31 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:31.731 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:31 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:31 smithi057 ceph-mon[29096]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:31 smithi057 ceph-mon[29096]: pgmap v11: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.0 MiB/s wr, 245 op/s 2024-02-28T23:08:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:31 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:31 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:32.750 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:32 smithi084 ceph-mon[34826]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:32.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:32 smithi057 ceph-mon[29096]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:08:33.816 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:33 smithi084 ceph-mon[34826]: pgmap v12: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 411 op/s 2024-02-28T23:08:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:33 smithi057 ceph-mon[29096]: pgmap v12: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 411 op/s 2024-02-28T23:08:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:35 smithi057 ceph-mon[29096]: pgmap v13: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-02-28T23:08:35.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:35 smithi084 ceph-mon[34826]: pgmap v13: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:08:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:08:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:37 smithi057 ceph-mon[29096]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 475 op/s 2024-02-28T23:08:37.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:37.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:08:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:08:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:08:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:37 smithi084 ceph-mon[34826]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 475 op/s 2024-02-28T23:08:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:38 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:08:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:38 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:08:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:38 smithi057 ceph-mon[29096]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:08:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:38 smithi057 ceph-mon[29096]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:08:38.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:38 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:08:38.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:38 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:08:38.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:38 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:08:38.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:38 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:08:39.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:08:39.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:40 smithi057 ceph-mon[29096]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T23:08:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:40 smithi057 ceph-mon[29096]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T23:08:40.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:40 smithi084 ceph-mon[34826]: Reconfiguring prometheus.smithi057 (dependencies changed)... 2024-02-28T23:08:40.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:40 smithi084 ceph-mon[34826]: Reconfiguring daemon prometheus.smithi057 on smithi057 2024-02-28T23:08:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:41 smithi057 ceph-mon[29096]: pgmap v16: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 390 op/s 2024-02-28T23:08:41.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:41 smithi084 ceph-mon[34826]: pgmap v16: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 390 op/s 2024-02-28T23:08:43.543 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:43 smithi057 ceph-mon[29096]: pgmap v17: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 532 op/s 2024-02-28T23:08:43.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:43 smithi084 ceph-mon[34826]: pgmap v17: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 532 op/s 2024-02-28T23:08:45.801 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:45 smithi057 ceph-mon[29096]: pgmap v18: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 366 op/s 2024-02-28T23:08:45.801 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:45 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:45.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:45 smithi084 ceph-mon[34826]: pgmap v18: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 366 op/s 2024-02-28T23:08:45.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:45 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:47.660 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:47 smithi057 ceph-mon[29096]: pgmap v19: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 535 op/s 2024-02-28T23:08:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:47 smithi084 ceph-mon[34826]: pgmap v19: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 535 op/s 2024-02-28T23:08:49.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:49.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:49.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:08:49.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:08:49.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:49.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:48 smithi057 ceph-mon[29096]: pgmap v20: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 358 op/s 2024-02-28T23:08:49.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:49.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:49.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:08:49.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:08:49.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:49.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:48 smithi084 ceph-mon[34826]: pgmap v20: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 358 op/s 2024-02-28T23:08:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:51 smithi057 ceph-mon[29096]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-02-28T23:08:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:51 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:08:51.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:51 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:08:51.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:51 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:51 smithi084 ceph-mon[34826]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-02-28T23:08:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:51 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:08:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:51 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:08:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:51 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:08:52.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:52 smithi084 ceph-mon[34826]: Upgrade: Updating mgr.smithi084.fhkzis 2024-02-28T23:08:52.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:52 smithi084 ceph-mon[34826]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:08:52.703 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:52 smithi057 ceph-mon[29096]: Upgrade: Updating mgr.smithi084.fhkzis 2024-02-28T23:08:52.704 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:52 smithi057 ceph-mon[29096]: Deploying daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:08:53.454 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:53 smithi084 ceph-mon[34826]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 490 op/s 2024-02-28T23:08:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:53 smithi057 ceph-mon[29096]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 490 op/s 2024-02-28T23:08:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:55 smithi057 ceph-mon[29096]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 347 op/s 2024-02-28T23:08:55.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:55 smithi084 ceph-mon[34826]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 347 op/s 2024-02-28T23:08:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:57 smithi057 ceph-mon[29096]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 505 op/s 2024-02-28T23:08:57.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:57 smithi084 ceph-mon[34826]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 505 op/s 2024-02-28T23:08:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:58 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:08:58.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:58 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:08:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:59 smithi057 ceph-mon[29096]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-02-28T23:08:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:08:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:08:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:59 smithi084 ceph-mon[34826]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-02-28T23:08:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:08:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:08:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:08:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:01.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 -- 172.21.15.57:0/2961000109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928103640 msgr2=0x7fa928103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2961000109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928103640 0x7fa928103a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa914009910 tx=0x7fa914009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:01.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 -- 172.21.15.57:0/2961000109 shutdown_connections 2024-02-28T23:09:01.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2961000109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928103640 0x7fa928103a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2961000109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928102490 0x7fa9281028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 -- 172.21.15.57:0/2961000109 >> 172.21.15.57:0/2961000109 conn(0x7fa9280fda60 msgr2=0x7fa9280ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:01.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 -- 172.21.15.57:0/2961000109 shutdown_connections 2024-02-28T23:09:01.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.178+0000 7fa92ca00700 1 -- 172.21.15.57:0/2961000109 wait complete. 2024-02-28T23:09:01.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.179+0000 7fa92ca00700 1 Processor -- start 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.179+0000 7fa92ca00700 1 -- start start 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa92ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa92ca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 0x7fa9281981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa92ca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281987f0 con 0x7fa928102490 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa92ca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928198960 con 0x7fa928103640 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa925e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa925e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54582/0 (socket says 172.21.15.57:54582) 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa925e00700 1 -- 172.21.15.57:0/2295262660 learned_addr learned my addr 172.21.15.57:0/2295262660 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.180+0000 7fa925400700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 0x7fa9281981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa925e00700 1 -- 172.21.15.57:0/2295262660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 msgr2=0x7fa9281981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa925e00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 0x7fa9281981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa925e00700 1 -- 172.21.15.57:0/2295262660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140095f0 con 0x7fa928102490 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa925400700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 0x7fa9281981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:09:01.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa925e00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e8f0 tx=0x7fa91800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa918009790 con 0x7fa928102490 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9180044d0 con 0x7fa928102490 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92819d3f0 con 0x7fa928102490 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa918003da0 con 0x7fa928102490 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.181+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92819d940 con 0x7fa928102490 2024-02-28T23:09:01.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.183+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92810af90 con 0x7fa928102490 2024-02-28T23:09:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.185+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7fa91800cba0 con 0x7fa928102490 2024-02-28T23:09:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.186+0000 7fa91ea00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 0x7fa90c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.186+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa918014070 con 0x7fa928102490 2024-02-28T23:09:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.188+0000 7fa925400700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 0x7fa90c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.189+0000 7fa925400700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 0x7fa90c06fb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9140059d0 tx=0x7fa914005940 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:01.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.192+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa9180572c0 con 0x7fa928102490 2024-02-28T23:09:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.553+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa928107f90 con 0x7fa90c06d660 2024-02-28T23:09:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.556+0000 7fa91ea00700 1 -- 172.21.15.57:0/2295262660 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fa928107f90 con 0x7fa90c06d660 2024-02-28T23:09:01.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 msgr2=0x7fa90c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 0x7fa90c06fb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9140059d0 tx=0x7fa914005940 comp rx=0 tx=0).stop 2024-02-28T23:09:01.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 msgr2=0x7fa928197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e8f0 tx=0x7fa91800ecb0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 shutdown_connections 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa90c06d660 0x7fa90c06fb10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa928102490 0x7fa928197cb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 --2- 172.21.15.57:0/2295262660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa928103640 0x7fa9281981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.559+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 >> 172.21.15.57:0/2295262660 conn(0x7fa9280fda60 msgr2=0x7fa928106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.560+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 shutdown_connections 2024-02-28T23:09:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.560+0000 7fa92ca00700 1 -- 172.21.15.57:0/2295262660 wait complete. 2024-02-28T23:09:01.581 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:09:01.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.768+0000 7f3b25600700 1 -- 172.21.15.57:0/3735544720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 msgr2=0x7f3b20103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.768+0000 7f3b25600700 1 --2- 172.21.15.57:0/3735544720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b20103930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009910 tx=0x7f3b0c009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:01.768 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:01 smithi057 ceph-mon[29096]: pgmap v26: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 371 op/s 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 -- 172.21.15.57:0/3735544720 shutdown_connections 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 --2- 172.21.15.57:0/3735544720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b20103930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 --2- 172.21.15.57:0/3735544720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b20100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 -- 172.21.15.57:0/3735544720 >> 172.21.15.57:0/3735544720 conn(0x7f3b200fa780 msgr2=0x7f3b200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 -- 172.21.15.57:0/3735544720 shutdown_connections 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.769+0000 7f3b25600700 1 -- 172.21.15.57:0/3735544720 wait complete. 2024-02-28T23:09:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.770+0000 7f3b25600700 1 Processor -- start 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.771+0000 7f3b25600700 1 -- start start 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.771+0000 7f3b25600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.771+0000 7f3b25600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b2019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.771+0000 7f3b25600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019ccb0 con 0x7f3b201014b0 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.771+0000 7f3b25600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019cdf0 con 0x7f3b200feb50 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:57050/0 (socket says 172.21.15.57:57050) 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 -- 172.21.15.57:0/324443277 learned_addr learned my addr 172.21.15.57:0/324443277 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1e000700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b2019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 -- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 msgr2=0x7f3b2019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b2019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.772+0000 7f3b1ea00700 1 -- 172.21.15.57:0/324443277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0095f0 con 0x7f3b200feb50 2024-02-28T23:09:01.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b1ea00700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b4d0 tx=0x7f3b1000b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:01.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b10011800 con 0x7f3b200feb50 2024-02-28T23:09:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b201a18a0 con 0x7f3b200feb50 2024-02-28T23:09:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b201a1df0 con 0x7f3b200feb50 2024-02-28T23:09:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b10011e10 con 0x7f3b200feb50 2024-02-28T23:09:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.773+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1000f2e0 con 0x7f3b200feb50 2024-02-28T23:09:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.775+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00005320 con 0x7f3b200feb50 2024-02-28T23:09:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.776+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f3b10011960 con 0x7f3b200feb50 2024-02-28T23:09:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.777+0000 7f3b17400700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 0x7f3b0406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.777+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3b1008cd40 con 0x7f3b200feb50 2024-02-28T23:09:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.778+0000 7f3b1e000700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 0x7f3b0406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.779+0000 7f3b1e000700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 0x7f3b0406fb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009fd0 tx=0x7f3b0c00b540 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:01.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:01.793+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3b1005a3a0 con 0x7f3b200feb50 2024-02-28T23:09:01.821 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:01 smithi084 ceph-mon[34826]: pgmap v26: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 371 op/s 2024-02-28T23:09:02.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.147+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b00000bf0 con 0x7f3b0406d6a0 2024-02-28T23:09:02.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.149+0000 7f3b17400700 1 -- 172.21.15.57:0/324443277 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f3b00000bf0 con 0x7f3b0406d6a0 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 msgr2=0x7f3b0406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 0x7f3b0406fb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009fd0 tx=0x7f3b0c00b540 comp rx=0 tx=0).stop 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 msgr2=0x7f3b2019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b4d0 tx=0x7f3b1000b7e0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 shutdown_connections 2024-02-28T23:09:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3b0406d6a0 0x7f3b0406fb50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b200feb50 0x7f3b2019c150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 --2- 172.21.15.57:0/324443277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b201014b0 0x7f3b2019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 >> 172.21.15.57:0/324443277 conn(0x7f3b200fa780 msgr2=0x7f3b200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 shutdown_connections 2024-02-28T23:09:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.152+0000 7f3b25600700 1 -- 172.21.15.57:0/324443277 wait complete. 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 -- 172.21.15.57:0/1948900675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3720074d80 msgr2=0x7f37200731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 --2- 172.21.15.57:0/1948900675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3720074d80 0x7f37200731e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3710009910 tx=0x7f3710009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 -- 172.21.15.57:0/1948900675 shutdown_connections 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 --2- 172.21.15.57:0/1948900675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37200737b0 0x7f3720073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 --2- 172.21.15.57:0/1948900675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3720074d80 0x7f37200731e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.336+0000 7f3725600700 1 -- 172.21.15.57:0/1948900675 >> 172.21.15.57:0/1948900675 conn(0x7f37200fb7f0 msgr2=0x7f37200fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.337+0000 7f3725600700 1 -- 172.21.15.57:0/1948900675 shutdown_connections 2024-02-28T23:09:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.337+0000 7f3725600700 1 -- 172.21.15.57:0/1948900675 wait complete. 2024-02-28T23:09:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 Processor -- start 2024-02-28T23:09:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 -- start start 2024-02-28T23:09:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3720074d80 0x7f372019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372019cbc0 con 0x7f37200737b0 2024-02-28T23:09:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.338+0000 7f3725600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372019cd00 con 0x7f3720074d80 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54606/0 (socket says 172.21.15.57:54606) 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 -- 172.21.15.57:0/2464955175 learned_addr learned my addr 172.21.15.57:0/2464955175 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f3716000700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3720074d80 0x7f372019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 -- 172.21.15.57:0/2464955175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3720074d80 msgr2=0x7f372019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3720074d80 0x7f372019c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.339+0000 7f371ea00700 1 -- 172.21.15.57:0/2464955175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100095f0 con 0x7f37200737b0 2024-02-28T23:09:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f371ea00700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3710005b40 tx=0x7f371000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:02.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f371001d070 con 0x7f37200737b0 2024-02-28T23:09:02.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f371000f460 con 0x7f37200737b0 2024-02-28T23:09:02.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37201a1750 con 0x7f37200737b0 2024-02-28T23:09:02.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710021620 con 0x7f37200737b0 2024-02-28T23:09:02.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.340+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37201a1c40 con 0x7f37200737b0 2024-02-28T23:09:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.341+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc005320 con 0x7f37200737b0 2024-02-28T23:09:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.342+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f3710003a60 con 0x7f37200737b0 2024-02-28T23:09:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.343+0000 7f3717e00700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 0x7f370406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.343+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f371008e3c0 con 0x7f37200737b0 2024-02-28T23:09:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.343+0000 7f3716000700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 0x7f370406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.344+0000 7f3716000700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 0x7f370406fb00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3708007be0 tx=0x7f37080073d0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:02.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.348+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f371005b990 con 0x7f37200737b0 2024-02-28T23:09:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.647+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f36fc000bf0 con 0x7f370406d650 2024-02-28T23:09:02.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.673+0000 7f3717e00700 1 -- 172.21.15.57:0/2464955175 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f36fc000bf0 con 0x7f370406d650 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (10m) 33s ago 12m 14.3M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (8m) 33s ago 12m 7914k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (11m) 39s ago 11m 7843k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (12m) 33s ago 12m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (11m) 39s ago 11m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (10m) 33s ago 12m 74.0M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (5m) 33s ago 5m 85.5M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (5m) 33s ago 5m 147M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (5m) 39s ago 5m 87.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:09:02.673 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (5m) 39s ago 5m 142M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (66s) 33s ago 14m 473M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 starting - - - - 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (14m) 33s ago 14m 51.7M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (11m) 39s ago 10m 42.7M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (12m) 33s ago 12m 8232k - 1.5.0 0da6a335fe13 f5f372b635b2 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (11m) 39s ago 11m 8120k - 1.5.0 0da6a335fe13 ec057efbcb2a 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (10m) 33s ago 10m 196M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (9m) 33s ago 9m 218M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (8m) 33s ago 8m 172M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (8m) 39s ago 8m 244M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (7m) 39s ago 7m 203M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (7m) 39s ago 7m 215M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:09:02.674 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 starting - - - - 2024-02-28T23:09:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.676+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 msgr2=0x7f370406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.676+0000 7f3725600700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 0x7f370406fb00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3708007be0 tx=0x7f37080073d0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.676+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 msgr2=0x7f372019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.676+0000 7f3725600700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3710005b40 tx=0x7f371000bd70 comp rx=0 tx=0).stop 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 shutdown_connections 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f370406d650 0x7f370406fb00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37200737b0 0x7f372019c060 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 --2- 172.21.15.57:0/2464955175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3720074d80 0x7f372019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 >> 172.21.15.57:0/2464955175 conn(0x7f37200fb7f0 msgr2=0x7f3720101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 shutdown_connections 2024-02-28T23:09:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.677+0000 7f3725600700 1 -- 172.21.15.57:0/2464955175 wait complete. 2024-02-28T23:09:02.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:02 smithi057 ceph-mon[29096]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:02.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:02 smithi084 ceph-mon[34826]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:02.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.878+0000 7f6cac600700 1 -- 172.21.15.57:0/2081512369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4069270 msgr2=0x7f6ca4102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.878+0000 7f6cac600700 1 --2- 172.21.15.57:0/2081512369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4069270 0x7f6ca4102dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009870 tx=0x7f6c94009b80 comp rx=0 tx=0).stop 2024-02-28T23:09:02.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.879+0000 7f6cac600700 1 -- 172.21.15.57:0/2081512369 shutdown_connections 2024-02-28T23:09:02.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.879+0000 7f6cac600700 1 --2- 172.21.15.57:0/2081512369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.879+0000 7f6cac600700 1 --2- 172.21.15.57:0/2081512369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4069270 0x7f6ca4102dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.879+0000 7f6cac600700 1 -- 172.21.15.57:0/2081512369 >> 172.21.15.57:0/2081512369 conn(0x7f6ca40fa780 msgr2=0x7f6ca40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.879+0000 7f6cac600700 1 -- 172.21.15.57:0/2081512369 shutdown_connections 2024-02-28T23:09:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.880+0000 7f6cac600700 1 -- 172.21.15.57:0/2081512369 wait complete. 2024-02-28T23:09:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.881+0000 7f6cac600700 1 Processor -- start 2024-02-28T23:09:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.881+0000 7f6cac600700 1 -- start start 2024-02-28T23:09:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6cac600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6cac600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6cac600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41986b0 con 0x7f6ca4103310 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6cac600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4198820 con 0x7f6ca4198240 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:57104/0 (socket says 172.21.15.57:57104) 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.882+0000 7f6ca9200700 1 -- 172.21.15.57:0/3549183562 learned_addr learned my addr 172.21.15.57:0/3549183562 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.883+0000 7f6ca9200700 1 -- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 msgr2=0x7f6ca4197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.884+0000 7f6ca9c00700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.884+0000 7f6ca9200700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.884+0000 7f6ca9200700 1 -- 172.21.15.57:0/3549183562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94009520 con 0x7f6ca4198240 2024-02-28T23:09:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.884+0000 7f6ca9c00700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:09:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.885+0000 7f6ca9200700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e910 tx=0x7f6c9800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.885+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c980041f0 con 0x7f6ca4198240 2024-02-28T23:09:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.885+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c98004350 con 0x7f6ca4198240 2024-02-28T23:09:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.885+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c98003d20 con 0x7f6ca4198240 2024-02-28T23:09:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.885+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca406a7d0 con 0x7f6ca4198240 2024-02-28T23:09:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.886+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca406ad20 con 0x7f6ca4198240 2024-02-28T23:09:02.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.889+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca40fc340 con 0x7f6ca4198240 2024-02-28T23:09:02.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.890+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f6c9800c760 con 0x7f6ca4198240 2024-02-28T23:09:02.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.891+0000 7f6ca2000700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 0x7f6c8c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:02.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.891+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6c9808cd10 con 0x7f6ca4198240 2024-02-28T23:09:02.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.892+0000 7f6ca9c00700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 0x7f6c8c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:02.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.893+0000 7f6ca9c00700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 0x7f6c8c06fb40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400a010 tx=0x7f6c94019040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:02.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:02.901+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6c98056830 con 0x7f6ca4198240 2024-02-28T23:09:03.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.303+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ca4069270 con 0x7f6ca4198240 2024-02-28T23:09:03.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.304+0000 7f6ca2000700 1 -- 172.21.15.57:0/3549183562 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f6c9801f2c0 con 0x7f6ca4198240 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:03.304 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:09:03.305 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:09:03.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.307+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 msgr2=0x7f6c8c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.307+0000 7f6cac600700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 0x7f6c8c06fb40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400a010 tx=0x7f6c94019040 comp rx=0 tx=0).stop 2024-02-28T23:09:03.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.307+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 msgr2=0x7f6ca419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.307+0000 7f6cac600700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e910 tx=0x7f6c9800ecd0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 shutdown_connections 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6c8c06d690 0x7f6c8c06fb40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca4103310 0x7f6ca4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 --2- 172.21.15.57:0/3549183562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca4198240 0x7f6ca419d270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 >> 172.21.15.57:0/3549183562 conn(0x7f6ca40fa780 msgr2=0x7f6ca4100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:03.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 shutdown_connections 2024-02-28T23:09:03.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.308+0000 7f6cac600700 1 -- 172.21.15.57:0/3549183562 wait complete. 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 -- 172.21.15.57:0/508175652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c103310 msgr2=0x7f416c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 --2- 172.21.15.57:0/508175652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c103310 0x7f416c105790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f4158009b80 comp rx=0 tx=0).stop 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 -- 172.21.15.57:0/508175652 shutdown_connections 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 --2- 172.21.15.57:0/508175652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c103310 0x7f416c105790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 --2- 172.21.15.57:0/508175652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c069270 0x7f416c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 -- 172.21.15.57:0/508175652 >> 172.21.15.57:0/508175652 conn(0x7f416c0fa780 msgr2=0x7f416c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 -- 172.21.15.57:0/508175652 shutdown_connections 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.511+0000 7f4172000700 1 -- 172.21.15.57:0/508175652 wait complete. 2024-02-28T23:09:03.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.512+0000 7f4172000700 1 Processor -- start 2024-02-28T23:09:03.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.513+0000 7f4172000700 1 -- start start 2024-02-28T23:09:03.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.513+0000 7f4172000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:03.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.513+0000 7f4172000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c0fed70 0x7f416c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:03.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.513+0000 7f4172000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c199700 con 0x7f416c103310 2024-02-28T23:09:03.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.513+0000 7f4172000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c199870 con 0x7f416c0fed70 2024-02-28T23:09:03.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:03.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54646/0 (socket says 172.21.15.57:54646) 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 -- 172.21.15.57:0/3233091963 learned_addr learned my addr 172.21.15.57:0/3233091963 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416aa00700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c0fed70 0x7f416c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 -- 172.21.15.57:0/3233091963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c0fed70 msgr2=0x7f416c0ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c0fed70 0x7f416c0ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.514+0000 7f416b400700 1 -- 172.21.15.57:0/3233091963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158009520 con 0x7f416c103310 2024-02-28T23:09:03.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.515+0000 7f416b400700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f416400e970 tx=0x7f416400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.519+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416400c9d0 con 0x7f416c103310 2024-02-28T23:09:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.520+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4164010430 con 0x7f416c103310 2024-02-28T23:09:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.520+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4164018630 con 0x7f416c103310 2024-02-28T23:09:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.520+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c0ff720 con 0x7f416c103310 2024-02-28T23:09:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.520+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c1a2300 con 0x7f416c103310 2024-02-28T23:09:03.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.522+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c18fbf0 con 0x7f416c103310 2024-02-28T23:09:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.523+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 30) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f41640105a0 con 0x7f416c103310 2024-02-28T23:09:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.524+0000 7f4163e00700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 0x7f415006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.524+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4164014070 con 0x7f416c103310 2024-02-28T23:09:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.525+0000 7f416aa00700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 0x7f415006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.526+0000 7f416aa00700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 0x7f415006fb50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4158009f50 tx=0x7f4158019040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:03.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.530+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f4164052a50 con 0x7f416c103310 2024-02-28T23:09:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:03 smithi057 ceph-mon[29096]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:03 smithi057 ceph-mon[29096]: pgmap v27: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 455 op/s 2024-02-28T23:09:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:03 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3549183562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:03 smithi084 ceph-mon[34826]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:03 smithi084 ceph-mon[34826]: pgmap v27: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 455 op/s 2024-02-28T23:09:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:03 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3549183562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.873+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f416c0ff8b0 con 0x7f415006d6a0 2024-02-28T23:09:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.875+0000 7f4163e00700 1 -- 172.21.15.57:0/3233091963 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f416c0ff8b0 con 0x7f415006d6a0 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "", 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mgr daemons", 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:09:03.878 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:09:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 msgr2=0x7f415006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 0x7f415006fb50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4158009f50 tx=0x7f4158019040 comp rx=0 tx=0).stop 2024-02-28T23:09:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 msgr2=0x7f416c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f416400e970 tx=0x7f416400ed30 comp rx=0 tx=0).stop 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 shutdown_connections 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f415006d6a0 0x7f415006fb50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f416c103310 0x7f416c100720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 --2- 172.21.15.57:0/3233091963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f416c0fed70 0x7f416c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.880+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 >> 172.21.15.57:0/3233091963 conn(0x7f416c0fa780 msgr2=0x7f416c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.881+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 shutdown_connections 2024-02-28T23:09:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:03.881+0000 7f4172000700 1 -- 172.21.15.57:0/3233091963 wait complete. 2024-02-28T23:09:04.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:04 smithi057 ceph-mon[29096]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:04.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:04 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:04.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:04 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:04.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:04 smithi084 ceph-mon[34826]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:04.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:04 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:04.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:04 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:05.517 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:05 smithi084 ceph-mon[34826]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:05.517 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:05 smithi084 ceph-mon[34826]: pgmap v28: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-02-28T23:09:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:05 smithi057 ceph-mon[29096]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:05 smithi057 ceph-mon[29096]: pgmap v28: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-02-28T23:09:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:07 smithi084 ceph-mon[34826]: pgmap v29: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 423 op/s 2024-02-28T23:09:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:07 smithi057 ceph-mon[29096]: pgmap v29: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 423 op/s 2024-02-28T23:09:08.417 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:08 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:08.417 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:08 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:08.754 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:08 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:08.754 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:08 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:09 smithi057 ceph-mon[29096]: pgmap v30: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 958 KiB/s rd, 968 KiB/s wr, 265 op/s 2024-02-28T23:09:09.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:09 smithi084 ceph-mon[34826]: pgmap v30: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 958 KiB/s rd, 968 KiB/s wr, 265 op/s 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: pgmap v31: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 332 op/s 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:09:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi057.zrrldj"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi057.zrrldj"}]': finished 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi084.fhkzis"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi084.fhkzis"}]': finished 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: pgmap v31: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 332 op/s 2024-02-28T23:09:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/1615528038' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi057.zrrldj"}]: dispatch 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi057.zrrldj"}]': finished 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi084.fhkzis"}]: dispatch 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi084.fhkzis"}]': finished 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:11.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all mgr 2024-02-28T23:09:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all rgw 2024-02-28T23:09:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all rbd-mirror 2024-02-28T23:09:12.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-28T23:09:12.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all iscsi 2024-02-28T23:09:12.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all nfs 2024-02-28T23:09:12.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all nvmeof 2024-02-28T23:09:12.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:12 smithi084 ceph-mon[34826]: mgrmap e31: smithi057.zrrldj(active, since 57s), standbys: smithi084.fhkzis 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all mgr 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all rgw 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all rbd-mirror 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all iscsi 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all nfs 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all nvmeof 2024-02-28T23:09:12.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:12 smithi057 ceph-mon[29096]: mgrmap e31: smithi057.zrrldj(active, since 57s), standbys: smithi084.fhkzis 2024-02-28T23:09:13.801 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:13 smithi057 ceph-mon[29096]: pgmap v32: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-02-28T23:09:13.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:13 smithi084 ceph-mon[34826]: pgmap v32: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-02-28T23:09:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:15.731 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:15 smithi057 ceph-mon[29096]: pgmap v33: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-02-28T23:09:15.731 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:15 smithi057 ceph-mon[29096]: Upgrade: Updating node-exporter.smithi057 (1/2) 2024-02-28T23:09:15.731 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:15 smithi057 ceph-mon[29096]: Deploying daemon node-exporter.smithi057 on smithi057 2024-02-28T23:09:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:15 smithi084 ceph-mon[34826]: pgmap v33: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-02-28T23:09:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:15 smithi084 ceph-mon[34826]: Upgrade: Updating node-exporter.smithi057 (1/2) 2024-02-28T23:09:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:15 smithi084 ceph-mon[34826]: Deploying daemon node-exporter.smithi057 on smithi057 2024-02-28T23:09:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:17 smithi057 ceph-mon[29096]: pgmap v34: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 469 op/s 2024-02-28T23:09:17.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:17 smithi084 ceph-mon[34826]: pgmap v34: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 469 op/s 2024-02-28T23:09:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:19 smithi057 ceph-mon[29096]: pgmap v35: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 335 op/s 2024-02-28T23:09:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:19 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:19 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:19.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:19 smithi084 ceph-mon[34826]: pgmap v35: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 335 op/s 2024-02-28T23:09:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:19 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:19 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:21 smithi057 ceph-mon[29096]: pgmap v36: 65 pgs: 65 active+clean; 267 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 442 op/s 2024-02-28T23:09:21.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:21 smithi084 ceph-mon[34826]: pgmap v36: 65 pgs: 65 active+clean; 267 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 442 op/s 2024-02-28T23:09:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:22 smithi057 ceph-mon[29096]: Upgrade: Updating node-exporter.smithi084 (2/2) 2024-02-28T23:09:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:22 smithi057 ceph-mon[29096]: Deploying daemon node-exporter.smithi084 on smithi084 2024-02-28T23:09:22.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:22 smithi084 ceph-mon[34826]: Upgrade: Updating node-exporter.smithi084 (2/2) 2024-02-28T23:09:22.834 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:22 smithi084 ceph-mon[34826]: Deploying daemon node-exporter.smithi084 on smithi084 2024-02-28T23:09:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:23 smithi057 ceph-mon[29096]: pgmap v37: 65 pgs: 65 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 433 op/s 2024-02-28T23:09:23.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:23 smithi084 ceph-mon[34826]: pgmap v37: 65 pgs: 65 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 433 op/s 2024-02-28T23:09:25.812 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:25 smithi057 ceph-mon[29096]: pgmap v38: 65 pgs: 65 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 949 KiB/s rd, 1.0 MiB/s wr, 331 op/s 2024-02-28T23:09:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:25 smithi084 ceph-mon[34826]: pgmap v38: 65 pgs: 65 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 949 KiB/s rd, 1.0 MiB/s wr, 331 op/s 2024-02-28T23:09:27.045 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-28T23:09:27.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-28T23:09:27.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:27 smithi084 ceph-mon[34826]: pgmap v39: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 472 op/s 2024-02-28T23:09:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:27 smithi057 ceph-mon[29096]: pgmap v39: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 472 op/s 2024-02-28T23:09:28.242 DEBUG:teuthology.parallel:result is None 2024-02-28T23:09:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:29 smithi057 ceph-mon[29096]: pgmap v40: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 974 KiB/s rd, 982 KiB/s wr, 304 op/s 2024-02-28T23:09:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:29 smithi084 ceph-mon[34826]: pgmap v40: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 974 KiB/s rd, 982 KiB/s wr, 304 op/s 2024-02-28T23:09:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:30.499 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-28T23:09:30.499 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-28T23:09:31.702 DEBUG:teuthology.parallel:result is None 2024-02-28T23:09:31.703 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-28T23:09:31.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:31 smithi057 ceph-mon[29096]: pgmap v41: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2024-02-28T23:09:31.813 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-28T23:09:31.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-28T23:09:31.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:31 smithi084 ceph-mon[34826]: pgmap v41: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2024-02-28T23:09:31.998 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-28T23:09:31.998 DEBUG:teuthology.parallel:result is None 2024-02-28T23:09:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:33 smithi057 ceph-mon[29096]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 315 op/s 2024-02-28T23:09:33.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:33 smithi084 ceph-mon[34826]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 315 op/s 2024-02-28T23:09:34.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 -- 172.21.15.57:0/1821670678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c1036d0 msgr2=0x7f092c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 --2- 172.21.15.57:0/1821670678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c1036d0 0x7f092c103b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0918009910 tx=0x7f0918009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 -- 172.21.15.57:0/1821670678 shutdown_connections 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 --2- 172.21.15.57:0/1821670678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c1036d0 0x7f092c103b40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 --2- 172.21.15.57:0/1821670678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c102430 0x7f092c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.057+0000 7f0934000700 1 -- 172.21.15.57:0/1821670678 >> 172.21.15.57:0/1821670678 conn(0x7f092c0fda80 msgr2=0x7f092c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.058+0000 7f0934000700 1 -- 172.21.15.57:0/1821670678 shutdown_connections 2024-02-28T23:09:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.058+0000 7f0934000700 1 -- 172.21.15.57:0/1821670678 wait complete. 2024-02-28T23:09:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 Processor -- start 2024-02-28T23:09:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 -- start start 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 0x7f092c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c198740 con 0x7f092c102430 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0934000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c1988b0 con 0x7f092c1036d0 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0931600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0931600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35996/0 (socket says 172.21.15.57:35996) 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.059+0000 7f0931600700 1 -- 172.21.15.57:0/535485348 learned_addr learned my addr 172.21.15.57:0/535485348 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0930c00700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 0x7f092c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0931600700 1 -- 172.21.15.57:0/535485348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 msgr2=0x7f092c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0931600700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 0x7f092c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0931600700 1 -- 172.21.15.57:0/535485348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180095f0 con 0x7f092c102430 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0930c00700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 0x7f092c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:09:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.060+0000 7f0931600700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f092000c930 tx=0x7f092000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.061+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0920007ad0 con 0x7f092c102430 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.061+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0920007c30 con 0x7f092c102430 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.061+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0920018630 con 0x7f092c102430 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.061+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c19d350 con 0x7f092c102430 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.061+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c19d8a0 con 0x7f092c102430 2024-02-28T23:09:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.062+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c10af10 con 0x7f092c102430 2024-02-28T23:09:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.063+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 31) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f092000f450 con 0x7f092c102430 2024-02-28T23:09:34.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.064+0000 7f092a000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 0x7f091406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.064+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f092008d3a0 con 0x7f092c102430 2024-02-28T23:09:34.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.064+0000 7f0930c00700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 0x7f091406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.065+0000 7f0930c00700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 0x7f091406fb00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f092c1991a0 tx=0x7f091800b560 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.070+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f0920056e40 con 0x7f092c102430 2024-02-28T23:09:34.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.325+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f092c108020 con 0x7f091406d650 2024-02-28T23:09:34.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.327+0000 7f092a000700 1 -- 172.21.15.57:0/535485348 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f092c108020 con 0x7f091406d650 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 msgr2=0x7f091406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 0x7f091406fb00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f092c1991a0 tx=0x7f091800b560 comp rx=0 tx=0).stop 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 msgr2=0x7f092c197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f092000c930 tx=0x7f092000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 shutdown_connections 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f091406d650 0x7f091406fb00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c102430 0x7f092c197c90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 --2- 172.21.15.57:0/535485348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f092c1036d0 0x7f092c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 >> 172.21.15.57:0/535485348 conn(0x7f092c0fda80 msgr2=0x7f092c106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 shutdown_connections 2024-02-28T23:09:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.330+0000 7f0934000700 1 -- 172.21.15.57:0/535485348 wait complete. 2024-02-28T23:09:34.353 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:09:34.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 -- 172.21.15.57:0/3348212187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b40feb50 msgr2=0x7fb9b4100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/3348212187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b40feb50 0x7fb9b4100f70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009910 tx=0x7fb9a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:34.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 -- 172.21.15.57:0/3348212187 shutdown_connections 2024-02-28T23:09:34.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/3348212187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b41014b0 0x7fb9b4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/3348212187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b40feb50 0x7fb9b4100f70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.502+0000 7fb9bbc00700 1 -- 172.21.15.57:0/3348212187 >> 172.21.15.57:0/3348212187 conn(0x7fb9b40fa780 msgr2=0x7fb9b40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:34.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.503+0000 7fb9bbc00700 1 -- 172.21.15.57:0/3348212187 shutdown_connections 2024-02-28T23:09:34.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.503+0000 7fb9bbc00700 1 -- 172.21.15.57:0/3348212187 wait complete. 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 Processor -- start 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 -- start start 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b40feb50 0x7fb9b419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419cc10 con 0x7fb9b41014b0 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9bbc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419cd50 con 0x7fb9b40feb50 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9b3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9b3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36012/0 (socket says 172.21.15.57:36012) 2024-02-28T23:09:34.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.504+0000 7fb9b3e00700 1 -- 172.21.15.57:0/624121607 learned_addr learned my addr 172.21.15.57:0/624121607 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:34.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.505+0000 7fb9b3e00700 1 -- 172.21.15.57:0/624121607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b40feb50 msgr2=0x7fb9b419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.505+0000 7fb9b9200700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b40feb50 0x7fb9b419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.505+0000 7fb9b3e00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b40feb50 0x7fb9b419c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.505+0000 7fb9b3e00700 1 -- 172.21.15.57:0/624121607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80095f0 con 0x7fb9b41014b0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.505+0000 7fb9b3e00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000d900 tx=0x7fb9a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.506+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a00041f0 con 0x7fb9b41014b0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.506+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9a0004ce0 con 0x7fb9b41014b0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.506+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41a1800 con 0x7fb9b41014b0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.506+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41a1d50 con 0x7fb9b41014b0 2024-02-28T23:09:34.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.506+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a000b750 con 0x7fb9b41014b0 2024-02-28T23:09:34.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.507+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b41961c0 con 0x7fb9b41014b0 2024-02-28T23:09:34.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.508+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 31) v1 ==== 91440+0+0 (secure 0 0 0) 0x7fb9a0004350 con 0x7fb9b41014b0 2024-02-28T23:09:34.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.509+0000 7fb9b1600700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 0x7fb99c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.509+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb9a008d340 con 0x7fb9b41014b0 2024-02-28T23:09:34.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.509+0000 7fb9b9200700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 0x7fb99c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.510+0000 7fb9b9200700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 0x7fb99c06fb00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000c00 tx=0x7fb9a8005fb0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.514+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb9a005a930 con 0x7fb9b41014b0 2024-02-28T23:09:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.767+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9b4061190 con 0x7fb99c06d650 2024-02-28T23:09:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.773+0000 7fb9b1600700 1 -- 172.21.15.57:0/624121607 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fb9b4061190 con 0x7fb99c06d650 2024-02-28T23:09:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 msgr2=0x7fb99c06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 0x7fb99c06fb00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000c00 tx=0x7fb9a8005fb0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 msgr2=0x7fb9b419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000d900 tx=0x7fb9a000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 shutdown_connections 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb99c06d650 0x7fb99c06fb00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b40feb50 0x7fb9b419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 --2- 172.21.15.57:0/624121607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9b41014b0 0x7fb9b419c5f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 >> 172.21.15.57:0/624121607 conn(0x7fb9b40fa780 msgr2=0x7fb9b40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 shutdown_connections 2024-02-28T23:09:34.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.776+0000 7fb9bbc00700 1 -- 172.21.15.57:0/624121607 wait complete. 2024-02-28T23:09:34.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 -- 172.21.15.57:0/3360103504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824101440 msgr2=0x7fd824101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 --2- 172.21.15.57:0/3360103504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824101440 0x7fd824101890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd810009910 tx=0x7fd810009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:34.944 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:34 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:34.944 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:34 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:34.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 -- 172.21.15.57:0/3360103504 shutdown_connections 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 --2- 172.21.15.57:0/3360103504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824101440 0x7fd824101890 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 --2- 172.21.15.57:0/3360103504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824100240 0x7fd824100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.944+0000 7fd82a600700 1 -- 172.21.15.57:0/3360103504 >> 172.21.15.57:0/3360103504 conn(0x7fd8240fb7f0 msgr2=0x7fd8240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.945+0000 7fd82a600700 1 -- 172.21.15.57:0/3360103504 shutdown_connections 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.945+0000 7fd82a600700 1 -- 172.21.15.57:0/3360103504 wait complete. 2024-02-28T23:09:34.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.946+0000 7fd82a600700 1 Processor -- start 2024-02-28T23:09:34.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.947+0000 7fd82a600700 1 -- start start 2024-02-28T23:09:34.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.947+0000 7fd82a600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.947+0000 7fd82a600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824101440 0x7fd8241981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.947+0000 7fd82a600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8241987f0 con 0x7fd824100240 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.947+0000 7fd82a600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824198960 con 0x7fd824101440 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36020/0 (socket says 172.21.15.57:36020) 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 -- 172.21.15.57:0/365299015 learned_addr learned my addr 172.21.15.57:0/365299015 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd822a00700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824101440 0x7fd8241981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 -- 172.21.15.57:0/365299015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824101440 msgr2=0x7fd8241981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824101440 0x7fd8241981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.948+0000 7fd823400700 1 -- 172.21.15.57:0/365299015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100095f0 con 0x7fd824100240 2024-02-28T23:09:34.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd823400700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd81400d900 tx=0x7fd81400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8140041f0 con 0x7fd824100240 2024-02-28T23:09:34.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82419d3f0 con 0x7fd824100240 2024-02-28T23:09:34.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd814004350 con 0x7fd824100240 2024-02-28T23:09:34.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd814003d20 con 0x7fd824100240 2024-02-28T23:09:34.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.949+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82419d940 con 0x7fd824100240 2024-02-28T23:09:34.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.950+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd824105520 con 0x7fd824100240 2024-02-28T23:09:34.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.951+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 31) v1 ==== 91440+0+0 (secure 0 0 0) 0x7fd814009730 con 0x7fd824100240 2024-02-28T23:09:34.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.953+0000 7fd81be00700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 0x7fd80806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:34.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.953+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd81408d120 con 0x7fd824100240 2024-02-28T23:09:34.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.953+0000 7fd822a00700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 0x7fd80806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:34.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.954+0000 7fd822a00700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 0x7fd80806fb60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8241991c0 tx=0x7fd81000b560 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:34.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:34.959+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd814056b40 con 0x7fd824100240 2024-02-28T23:09:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:34 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:34 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.207+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd824061190 con 0x7fd80806d6b0 2024-02-28T23:09:35.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.223+0000 7fd81be00700 1 -- 172.21.15.57:0/365299015 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd824061190 con 0x7fd80806d6b0 2024-02-28T23:09:35.222 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (11m) 27s ago 13m 14.7M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (9m) 27s ago 13m 7977k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (11m) 1s ago 11m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (13m) 27s ago 13m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (11m) 1s ago 11m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (10m) 27s ago 12m 73.5M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (5m) 27s ago 5m 68.0M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (5m) 27s ago 5m 140M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (5m) 1s ago 5m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (5m) 1s ago 5m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (99s) 27s ago 14m 494M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (37s) 1s ago 11m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:09:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (14m) 27s ago 15m 53.0M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (11m) 1s ago 11m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 starting - - - - 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (6s) 1s ago 11m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (10m) 27s ago 10m 209M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (9m) 27s ago 9m 226M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (9m) 27s ago 9m 182M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (8m) 1s ago 8m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (8m) 1s ago 8m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (7m) 1s ago 7m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:09:35.224 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (49s) 27s ago 12m 43.9M - 2.43.0 a07b618ecd1d e3fb082fb414 2024-02-28T23:09:35.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.226+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 msgr2=0x7fd80806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.226+0000 7fd82a600700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 0x7fd80806fb60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8241991c0 tx=0x7fd81000b560 comp rx=0 tx=0).stop 2024-02-28T23:09:35.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.226+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 msgr2=0x7fd824197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.226+0000 7fd82a600700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd81400d900 tx=0x7fd81400dc10 comp rx=0 tx=0).stop 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 shutdown_connections 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd80806d6b0 0x7fd80806fb60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd824100240 0x7fd824197cb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 --2- 172.21.15.57:0/365299015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd824101440 0x7fd8241981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 >> 172.21.15.57:0/365299015 conn(0x7fd8240fb7f0 msgr2=0x7fd824102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 shutdown_connections 2024-02-28T23:09:35.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.227+0000 7fd82a600700 1 -- 172.21.15.57:0/365299015 wait complete. 2024-02-28T23:09:35.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 -- 172.21.15.57:0/2988012123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0105600 msgr2=0x7f65c01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 --2- 172.21.15.57:0/2988012123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0105600 0x7f65c01079e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009910 tx=0x7f65ac009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 -- 172.21.15.57:0/2988012123 shutdown_connections 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 --2- 172.21.15.57:0/2988012123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0105600 0x7f65c01079e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 --2- 172.21.15.57:0/2988012123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0069250 0x7f65c01050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 -- 172.21.15.57:0/2988012123 >> 172.21.15.57:0/2988012123 conn(0x7f65c00fa610 msgr2=0x7f65c00fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 -- 172.21.15.57:0/2988012123 shutdown_connections 2024-02-28T23:09:35.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.389+0000 7f65c7400700 1 -- 172.21.15.57:0/2988012123 wait complete. 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.390+0000 7f65c7400700 1 Processor -- start 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.390+0000 7f65c7400700 1 -- start start 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c7400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0105600 0x7f65c0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c7400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01986b0 con 0x7f65c0069250 2024-02-28T23:09:35.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c7400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01987f0 con 0x7f65c0105600 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c4a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c4a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39662/0 (socket says 172.21.15.57:39662) 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65c4a00700 1 -- 172.21.15.57:0/1407169347 learned_addr learned my addr 172.21.15.57:0/1407169347 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.391+0000 7f65bfe00700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0105600 0x7f65c0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.392+0000 7f65c4a00700 1 -- 172.21.15.57:0/1407169347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0105600 msgr2=0x7f65c0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.392+0000 7f65c4a00700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0105600 0x7f65c0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.392+0000 7f65c4a00700 1 -- 172.21.15.57:0/1407169347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0095f0 con 0x7f65c0069250 2024-02-28T23:09:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.392+0000 7f65c4a00700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e8c0 tx=0x7f65b400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.392+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b400c9b0 con 0x7f65c0069250 2024-02-28T23:09:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.393+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f65b4010430 con 0x7f65c0069250 2024-02-28T23:09:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.393+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b4018650 con 0x7f65c0069250 2024-02-28T23:09:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.393+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c019d2a0 con 0x7f65c0069250 2024-02-28T23:09:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.393+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c019d7f0 con 0x7f65c0069250 2024-02-28T23:09:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.394+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c0191ce0 con 0x7f65c0069250 2024-02-28T23:09:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.395+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 31) v1 ==== 91440+0+0 (secure 0 0 0) 0x7f65b400cb10 con 0x7f65c0069250 2024-02-28T23:09:35.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.397+0000 7f65bd600700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 0x7f65a806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.397+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f65b4014070 con 0x7f65c0069250 2024-02-28T23:09:35.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.397+0000 7f65bfe00700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 0x7f65a806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.398+0000 7f65bfe00700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 0x7f65a806fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009fd0 tx=0x7f65ac005fd0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:35.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.401+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f65b4052a50 con 0x7f65c0069250 2024-02-28T23:09:35.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.736+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f65c002ce90 con 0x7f65c0069250 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.737+0000 7f65bd600700 1 -- 172.21.15.57:0/1407169347 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f65b405a9d0 con 0x7f65c0069250 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:09:35.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:09:35.737 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.740+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 msgr2=0x7f65a806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.740+0000 7f65c7400700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 0x7f65a806fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009fd0 tx=0x7f65ac005fd0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.740+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 msgr2=0x7f65c0197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.740+0000 7f65c7400700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e8c0 tx=0x7f65b400ebd0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 shutdown_connections 2024-02-28T23:09:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f65a806d650 0x7f65a806fb00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65c0069250 0x7f65c0197b50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 --2- 172.21.15.57:0/1407169347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0105600 0x7f65c0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 >> 172.21.15.57:0/1407169347 conn(0x7f65c00fa610 msgr2=0x7f65c00fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 shutdown_connections 2024-02-28T23:09:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.741+0000 7f65c7400700 1 -- 172.21.15.57:0/1407169347 wait complete. 2024-02-28T23:09:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:35 smithi057 ceph-mon[29096]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:35 smithi057 ceph-mon[29096]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 870 KiB/s rd, 859 KiB/s wr, 258 op/s 2024-02-28T23:09:35.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 -- 172.21.15.57:0/2323582184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4103680 msgr2=0x7ffae4105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 --2- 172.21.15.57:0/2323582184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4103680 0x7ffae4105a60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009910 tx=0x7ffad0009c20 comp rx=0 tx=0).stop 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 -- 172.21.15.57:0/2323582184 shutdown_connections 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 --2- 172.21.15.57:0/2323582184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4103680 0x7ffae4105a60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 --2- 172.21.15.57:0/2323582184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4100d60 0x7ffae4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 -- 172.21.15.57:0/2323582184 >> 172.21.15.57:0/2323582184 conn(0x7ffae40fa780 msgr2=0x7ffae40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 -- 172.21.15.57:0/2323582184 shutdown_connections 2024-02-28T23:09:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.902+0000 7ffae9200700 1 -- 172.21.15.57:0/2323582184 wait complete. 2024-02-28T23:09:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.904+0000 7ffae9200700 1 Processor -- start 2024-02-28T23:09:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.904+0000 7ffae9200700 1 -- start start 2024-02-28T23:09:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.904+0000 7ffae9200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.904+0000 7ffae9200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4103680 0x7ffae4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae9200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4196640 con 0x7ffae4100d60 2024-02-28T23:09:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae9200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4196780 con 0x7ffae4103680 2024-02-28T23:09:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39668/0 (socket says 172.21.15.57:39668) 2024-02-28T23:09:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 -- 172.21.15.57:0/3266629762 learned_addr learned my addr 172.21.15.57:0/3266629762 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:09:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1400700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4103680 0x7ffae4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 -- 172.21.15.57:0/3266629762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4103680 msgr2=0x7ffae4196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4103680 0x7ffae4196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.905+0000 7ffae1e00700 1 -- 172.21.15.57:0/3266629762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00095f0 con 0x7ffae4100d60 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffae1e00700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffad400e8f0 tx=0x7ffad400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffad4009790 con 0x7ffae4100d60 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffad400c810 con 0x7ffae4100d60 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffad40055b0 con 0x7ffae4100d60 2024-02-28T23:09:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae419b230 con 0x7ffae4100d60 2024-02-28T23:09:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.906+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae419b780 con 0x7ffae4100d60 2024-02-28T23:09:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.908+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae418fbe0 con 0x7ffae4100d60 2024-02-28T23:09:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.909+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 31) v1 ==== 91440+0+0 (secure 0 0 0) 0x7ffad4004a00 con 0x7ffae4100d60 2024-02-28T23:09:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.910+0000 7ffadaa00700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 0x7ffac806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:09:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.910+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffad4014070 con 0x7ffae4100d60 2024-02-28T23:09:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.910+0000 7ffae1400700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 0x7ffac806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:09:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.911+0000 7ffae1400700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 0x7ffac806fb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009fd0 tx=0x7ffad0005fd0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:09:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:35.917+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ffad405aa30 con 0x7ffae4100d60 2024-02-28T23:09:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:35 smithi084 ceph-mon[34826]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:35 smithi084 ceph-mon[34826]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 870 KiB/s rd, 859 KiB/s wr, 258 op/s 2024-02-28T23:09:36.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.173+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffae4061190 con 0x7ffac806d660 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.175+0000 7ffadaa00700 1 -- 172.21.15.57:0/3266629762 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ffae4061190 con 0x7ffac806d660 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:09:36.174 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:09:36.175 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2024-02-28T23:09:36.175 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-28T23:09:36.175 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:09:36.175 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.178+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 msgr2=0x7ffac806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.178+0000 7ffae9200700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 0x7ffac806fb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009fd0 tx=0x7ffad0005fd0 comp rx=0 tx=0).stop 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.178+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 msgr2=0x7ffae4195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.178+0000 7ffae9200700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffad400e8f0 tx=0x7ffad400ec00 comp rx=0 tx=0).stop 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 shutdown_connections 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ffac806d660 0x7ffac806fb10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffae4100d60 0x7ffae4195ae0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 --2- 172.21.15.57:0/3266629762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4103680 0x7ffae4196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:09:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 >> 172.21.15.57:0/3266629762 conn(0x7ffae40fa780 msgr2=0x7ffae40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:09:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 shutdown_connections 2024-02-28T23:09:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:09:36.179+0000 7ffae9200700 1 -- 172.21.15.57:0/3266629762 wait complete. 2024-02-28T23:09:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:36 smithi057 ceph-mon[29096]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:36 smithi057 ceph-mon[29096]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:36 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1407169347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:36.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:36 smithi084 ceph-mon[34826]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:36.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:36 smithi084 ceph-mon[34826]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:36.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:36.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:36.857 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:36 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1407169347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:37.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:37 smithi057 ceph-mon[29096]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:37.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:37 smithi057 ceph-mon[29096]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 KiB/s rd, 906 KiB/s wr, 298 op/s 2024-02-28T23:09:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:37 smithi084 ceph-mon[34826]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:09:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:37 smithi084 ceph-mon[34826]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 KiB/s rd, 906 KiB/s wr, 298 op/s 2024-02-28T23:09:39.957 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:39 smithi084 ceph-mon[34826]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 433 KiB/s rd, 418 KiB/s wr, 157 op/s 2024-02-28T23:09:39.958 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:39.958 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:39 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:39 smithi057 ceph-mon[29096]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 433 KiB/s rd, 418 KiB/s wr, 157 op/s 2024-02-28T23:09:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:39 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:41.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:41 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:41.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:41 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:41.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:41 smithi057 ceph-mon[29096]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 434 KiB/s rd, 418 KiB/s wr, 158 op/s 2024-02-28T23:09:41.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:41 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:41.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:41 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:41.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:41 smithi084 ceph-mon[34826]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 434 KiB/s rd, 418 KiB/s wr, 158 op/s 2024-02-28T23:09:42.726 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:42 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:42.726 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:42 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:42.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:42 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:42.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:42 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:43.812 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:43 smithi057 ceph-mon[29096]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 258 KiB/s rd, 266 KiB/s wr, 87 op/s 2024-02-28T23:09:43.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:43 smithi084 ceph-mon[34826]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 258 KiB/s rd, 266 KiB/s wr, 87 op/s 2024-02-28T23:09:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:44 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:44 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:09:45.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:45 smithi057 ceph-mon[29096]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 47 KiB/s wr, 42 op/s 2024-02-28T23:09:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:45 smithi084 ceph-mon[34826]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 47 KiB/s wr, 42 op/s 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: mgrmap e32: smithi057.zrrldj(active, since 91s), standbys: smithi084.fhkzis 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:09:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:46 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: mgrmap e32: smithi057.zrrldj(active, since 91s), standbys: smithi084.fhkzis 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:09:46.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:46 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 47 KiB/s wr, 44 op/s 2024-02-28T23:09:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:47 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 47 KiB/s wr, 44 op/s 2024-02-28T23:09:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:47.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:47 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:09:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:49 smithi057 ceph-mon[29096]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-28T23:09:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:49 smithi084 ceph-mon[34826]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-28T23:09:50.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:50 smithi057 ceph-mon[29096]: Upgrade: Updating prometheus.smithi057 2024-02-28T23:09:50.789 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:50 smithi057 ceph-mon[29096]: Deploying daemon prometheus.smithi057 on smithi057 2024-02-28T23:09:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:50 smithi084 ceph-mon[34826]: Upgrade: Updating prometheus.smithi057 2024-02-28T23:09:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:50 smithi084 ceph-mon[34826]: Deploying daemon prometheus.smithi057 on smithi057 2024-02-28T23:09:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:51 smithi057 ceph-mon[29096]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-28T23:09:51.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:51 smithi084 ceph-mon[34826]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-28T23:09:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:53 smithi057 ceph-mon[29096]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-28T23:09:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:53 smithi084 ceph-mon[34826]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-28T23:09:55.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:55 smithi084 ceph-mon[34826]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:09:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:55 smithi057 ceph-mon[29096]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-28T23:09:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:57 smithi084 ceph-mon[34826]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:09:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:57 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:57 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:57 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:57.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:57 smithi057 ceph-mon[29096]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:09:57.966 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:57 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:57.966 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:57 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:09:57.966 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:57 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:09:59.901 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:59 smithi057 ceph-mon[29096]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:09:59.901 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:09:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:59 smithi084 ceph-mon[34826]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:10:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:09:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:00 smithi057 ceph-mon[29096]: overall HEALTH_OK 2024-02-28T23:10:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:00 smithi084 ceph-mon[34826]: overall HEALTH_OK 2024-02-28T23:10:02.065 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:01 smithi057 ceph-mon[29096]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-28T23:10:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:01 smithi084 ceph-mon[34826]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-28T23:10:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:03 smithi084 ceph-mon[34826]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-28T23:10:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:03 smithi057 ceph-mon[29096]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-28T23:10:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:05 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:05 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:05 smithi057 ceph-mon[29096]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:05.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:05 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:05.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:05 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:05.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:05 smithi084 ceph-mon[34826]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.355+0000 7fd35c400700 1 -- 172.21.15.57:0/2005390322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354100240 msgr2=0x7fd354100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 --2- 172.21.15.57:0/2005390322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354100240 0x7fd354100650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd348009910 tx=0x7fd348009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 -- 172.21.15.57:0/2005390322 shutdown_connections 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 --2- 172.21.15.57:0/2005390322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354101440 0x7fd354101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 --2- 172.21.15.57:0/2005390322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354100240 0x7fd354100650 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 -- 172.21.15.57:0/2005390322 >> 172.21.15.57:0/2005390322 conn(0x7fd3540fb7f0 msgr2=0x7fd3540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 -- 172.21.15.57:0/2005390322 shutdown_connections 2024-02-28T23:10:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.356+0000 7fd35c400700 1 -- 172.21.15.57:0/2005390322 wait complete. 2024-02-28T23:10:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.357+0000 7fd35c400700 1 Processor -- start 2024-02-28T23:10:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.357+0000 7fd35c400700 1 -- start start 2024-02-28T23:10:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd35c400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354100240 0x7fd354074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd35c400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd35c400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354073680 con 0x7fd354101440 2024-02-28T23:10:06.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd35c400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540737c0 con 0x7fd354100240 2024-02-28T23:10:06.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37250/0 (socket says 172.21.15.57:37250) 2024-02-28T23:10:06.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 -- 172.21.15.57:0/1704096830 learned_addr learned my addr 172.21.15.57:0/1704096830 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359a00700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354100240 0x7fd354074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 -- 172.21.15.57:0/1704096830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354100240 msgr2=0x7fd354074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354100240 0x7fd354074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.358+0000 7fd359000700 1 -- 172.21.15.57:0/1704096830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480095f0 con 0x7fd354101440 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd359000700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd34000d8d0 tx=0x7fd34000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd340009940 con 0x7fd354101440 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd340010460 con 0x7fd354101440 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd34000f5f0 con 0x7fd354101440 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd354073aa0 con 0x7fd354101440 2024-02-28T23:10:06.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.359+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35419db10 con 0x7fd354101440 2024-02-28T23:10:06.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.361+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd354108d90 con 0x7fd354101440 2024-02-28T23:10:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.363+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fd3400105d0 con 0x7fd354101440 2024-02-28T23:10:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.363+0000 7fd352000700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 0x7fd33c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.364+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd34008cf20 con 0x7fd354101440 2024-02-28T23:10:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.364+0000 7fd359a00700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 0x7fd33c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.365+0000 7fd359a00700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 0x7fd33c06f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd348006010 tx=0x7fd34800b540 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:06.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.369+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd34005a710 con 0x7fd354101440 2024-02-28T23:10:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.633+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd35419e0c0 con 0x7fd33c06d520 2024-02-28T23:10:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.636+0000 7fd352000700 1 -- 172.21.15.57:0/1704096830 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fd35419e0c0 con 0x7fd33c06d520 2024-02-28T23:10:06.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 msgr2=0x7fd33c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 0x7fd33c06f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd348006010 tx=0x7fd34800b540 comp rx=0 tx=0).stop 2024-02-28T23:10:06.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 msgr2=0x7fd354073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd34000d8d0 tx=0x7fd34000dc90 comp rx=0 tx=0).stop 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 shutdown_connections 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd33c06d520 0x7fd33c06f9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd354100240 0x7fd354074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 --2- 172.21.15.57:0/1704096830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd354101440 0x7fd354073140 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 >> 172.21.15.57:0/1704096830 conn(0x7fd3540fb7f0 msgr2=0x7fd354104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 shutdown_connections 2024-02-28T23:10:06.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.639+0000 7fd35c400700 1 -- 172.21.15.57:0/1704096830 wait complete. 2024-02-28T23:10:06.655 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 -- 172.21.15.57:0/1169691445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970073750 msgr2=0x7f5970073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 --2- 172.21.15.57:0/1169691445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970073750 0x7f5970073bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f595c009910 tx=0x7f595c009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 -- 172.21.15.57:0/1169691445 shutdown_connections 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 --2- 172.21.15.57:0/1169691445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970073750 0x7f5970073bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 --2- 172.21.15.57:0/1169691445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970074d20 0x7f5970073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 -- 172.21.15.57:0/1169691445 >> 172.21.15.57:0/1169691445 conn(0x7f59700fb7f0 msgr2=0x7f59700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 -- 172.21.15.57:0/1169691445 shutdown_connections 2024-02-28T23:10:06.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.807+0000 7f5977600700 1 -- 172.21.15.57:0/1169691445 wait complete. 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.808+0000 7f5977600700 1 Processor -- start 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f5977600700 1 -- start start 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f5977600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970073750 0x7f5970197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f5977600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f5977600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59701987e0 con 0x7f5970074d20 2024-02-28T23:10:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f5977600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970198950 con 0x7f5970073750 2024-02-28T23:10:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f596fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f596fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37260/0 (socket says 172.21.15.57:37260) 2024-02-28T23:10:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.809+0000 7f596fe00700 1 -- 172.21.15.57:0/1790827753 learned_addr learned my addr 172.21.15.57:0/1790827753 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f5974c00700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970073750 0x7f5970197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596fe00700 1 -- 172.21.15.57:0/1790827753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970073750 msgr2=0x7f5970197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:06.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596fe00700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970073750 0x7f5970197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:06.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596fe00700 1 -- 172.21.15.57:0/1790827753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c0095f0 con 0x7f5970074d20 2024-02-28T23:10:06.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596fe00700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f595c005b40 tx=0x7f595c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f595c01d070 con 0x7f5970074d20 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f595c00f460 con 0x7f5970074d20 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.810+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597019d380 con 0x7f5970074d20 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.811+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f595c021640 con 0x7f5970074d20 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.811+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597019d870 con 0x7f5970074d20 2024-02-28T23:10:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.812+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597010af80 con 0x7f5970074d20 2024-02-28T23:10:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.813+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f595c00fa90 con 0x7f5970074d20 2024-02-28T23:10:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.814+0000 7f596d600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 0x7f595806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.814+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f595c08e1c0 con 0x7f5970074d20 2024-02-28T23:10:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.814+0000 7f5974c00700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 0x7f595806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:06.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.815+0000 7f5974c00700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 0x7f595806f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5964005950 tx=0x7f59640058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:06.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:06.819+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f595c057f20 con 0x7f5970074d20 2024-02-28T23:10:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.079+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f597019dfc0 con 0x7f595806d520 2024-02-28T23:10:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.081+0000 7f596d600700 1 -- 172.21.15.57:0/1790827753 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f597019dfc0 con 0x7f595806d520 2024-02-28T23:10:07.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 msgr2=0x7f595806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 0x7f595806f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5964005950 tx=0x7f59640058e0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 msgr2=0x7f59701981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f595c005b40 tx=0x7f595c00be20 comp rx=0 tx=0).stop 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 shutdown_connections 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f595806d520 0x7f595806f9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5970073750 0x7f5970197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 --2- 172.21.15.57:0/1790827753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5970074d20 0x7f59701981e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 >> 172.21.15.57:0/1790827753 conn(0x7f59700fb7f0 msgr2=0x7f5970106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 shutdown_connections 2024-02-28T23:10:07.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.084+0000 7f5977600700 1 -- 172.21.15.57:0/1790827753 wait complete. 2024-02-28T23:10:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 -- 172.21.15.57:0/3585489468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 msgr2=0x7fa200103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 --2- 172.21.15.57:0/3585489468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200103140 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009910 tx=0x7fa1f0009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 -- 172.21.15.57:0/3585489468 shutdown_connections 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 --2- 172.21.15.57:0/3585489468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 --2- 172.21.15.57:0/3585489468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200103140 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 -- 172.21.15.57:0/3585489468 >> 172.21.15.57:0/3585489468 conn(0x7fa2000fa780 msgr2=0x7fa2000fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 -- 172.21.15.57:0/3585489468 shutdown_connections 2024-02-28T23:10:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.242+0000 7fa206400700 1 -- 172.21.15.57:0/3585489468 wait complete. 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.243+0000 7fa206400700 1 Processor -- start 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.243+0000 7fa206400700 1 -- start start 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa206400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa206400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa206400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200198860 con 0x7fa200100d60 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa206400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20019d270 con 0x7fa200103680 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37284/0 (socket says 172.21.15.57:37284) 2024-02-28T23:10:07.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 -- 172.21.15.57:0/3495618941 learned_addr learned my addr 172.21.15.57:0/3495618941 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1f7e00700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 -- 172.21.15.57:0/3495618941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 msgr2=0x7fa200198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1ff400700 1 -- 172.21.15.57:0/3495618941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00095f0 con 0x7fa200100d60 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.244+0000 7fa1f7e00700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa1ff400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000bd70 tx=0x7fa1f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f001d070 con 0x7fa200100d60 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa1f000f460 con 0x7fa200100d60 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20019d410 con 0x7fa200100d60 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f0021640 con 0x7fa200100d60 2024-02-28T23:10:07.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.245+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20019d8d0 con 0x7fa200100d60 2024-02-28T23:10:07.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.246+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000fc340 con 0x7fa200100d60 2024-02-28T23:10:07.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.248+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fa1f000fab0 con 0x7fa200100d60 2024-02-28T23:10:07.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.249+0000 7fa1fcc00700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 0x7fa1e406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.249+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa1f008e820 con 0x7fa200100d60 2024-02-28T23:10:07.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.250+0000 7fa1f7e00700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 0x7fa1e406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.251+0000 7fa1f7e00700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 0x7fa1e406f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8005fd0 tx=0x7fa1e8005dc0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:07.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.253+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa1f0058560 con 0x7fa200100d60 2024-02-28T23:10:07.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.489+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa200061190 con 0x7fa1e406d520 2024-02-28T23:10:07.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.521+0000 7fa1fcc00700 1 -- 172.21.15.57:0/3495618941 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa200061190 con 0x7fa1e406d520 2024-02-28T23:10:07.521 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (11m) 3s ago 13m 14.9M - 0.25.0 c8568f914cd2 32c7c56ed9c8 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (9m) 3s ago 13m 8048k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (12m) 33s ago 12m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (13m) 3s ago 13m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (12m) 33s ago 12m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (11m) 3s ago 13m 73.9M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (6m) 3s ago 6m 57.1M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (6m) 3s ago 6m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (6m) 33s ago 6m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (6m) 33s ago 6m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:10:07.522 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (2m) 3s ago 15m 499M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (69s) 33s ago 12m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (15m) 3s ago 15m 55.5M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (12m) 33s ago 12m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (49s) 3s ago 13m 7554k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (39s) 33s ago 12m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (11m) 3s ago 11m 227M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (10m) 3s ago 10m 239M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (9m) 3s ago 9m 201M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (9m) 33s ago 9m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (8m) 33s ago 8m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (8m) 33s ago 8m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:10:07.523 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (11s) 3s ago 12m 38.8M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:10:07.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 msgr2=0x7fa1e406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 0x7fa1e406f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8005fd0 tx=0x7fa1e8005dc0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 msgr2=0x7fa200197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000bd70 tx=0x7fa1f000be50 comp rx=0 tx=0).stop 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 shutdown_connections 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fa1e406d520 0x7fa1e406f9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa200100d60 0x7fa200197d00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 --2- 172.21.15.57:0/3495618941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa200103680 0x7fa200198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 >> 172.21.15.57:0/3495618941 conn(0x7fa2000fa780 msgr2=0x7fa200100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 shutdown_connections 2024-02-28T23:10:07.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.524+0000 7fa206400700 1 -- 172.21.15.57:0/3495618941 wait complete. 2024-02-28T23:10:07.655 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:07.655 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:07.655 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:07 smithi057 ceph-mon[29096]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 -- 172.21.15.57:0/107931222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 msgr2=0x7fece0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 --2- 172.21.15.57:0/107931222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0102e90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009910 tx=0x7fecd4009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 -- 172.21.15.57:0/107931222 shutdown_connections 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 --2- 172.21.15.57:0/107931222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 0x7fece01057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 --2- 172.21.15.57:0/107931222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0102e90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 -- 172.21.15.57:0/107931222 >> 172.21.15.57:0/107931222 conn(0x7fece00fa4d0 msgr2=0x7fece00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 -- 172.21.15.57:0/107931222 shutdown_connections 2024-02-28T23:10:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.671+0000 7fece6e00700 1 -- 172.21.15.57:0/107931222 wait complete. 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.672+0000 7fece6e00700 1 Processor -- start 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece6e00700 1 -- start start 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece6e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece6e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 0x7fece00fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece6e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00fefa0 con 0x7fece0100ab0 2024-02-28T23:10:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece6e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00ff0e0 con 0x7fece01033d0 2024-02-28T23:10:07.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37304/0 (socket says 172.21.15.57:37304) 2024-02-28T23:10:07.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.673+0000 7fece5a00700 1 -- 172.21.15.57:0/1954049216 learned_addr learned my addr 172.21.15.57:0/1954049216 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:07.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.674+0000 7fece5000700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 0x7fece00fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.674+0000 7fece5a00700 1 -- 172.21.15.57:0/1954049216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 msgr2=0x7fece00fea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:07.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.674+0000 7fece5a00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 0x7fece00fea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:07.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.674+0000 7fece5a00700 1 -- 172.21.15.57:0/1954049216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40095f0 con 0x7fece0100ab0 2024-02-28T23:10:07.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.674+0000 7fece5a00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fecd400bd70 tx=0x7fecd400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.675+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd401d070 con 0x7fece0100ab0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.675+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece00ff360 con 0x7fece0100ab0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.675+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fecd4022470 con 0x7fece0100ab0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.675+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd400f670 con 0x7fece0100ab0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.675+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece00ff850 con 0x7fece0100ab0 2024-02-28T23:10:07.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.676+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece018fa40 con 0x7fece0100ab0 2024-02-28T23:10:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.678+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fecd4022a50 con 0x7fece0100ab0 2024-02-28T23:10:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.680+0000 7fecde000700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.680+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fecd408ea10 con 0x7fece0100ab0 2024-02-28T23:10:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.680+0000 7fece5000700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.681+0000 7fece5000700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feccc005950 tx=0x7feccc0058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:07.684+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fecd405c350 con 0x7fece0100ab0 2024-02-28T23:10:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:07 smithi084 ceph-mon[34826]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:08.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.015+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fece002cc30 con 0x7fece0100ab0 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.017+0000 7fecde000700 1 -- 172.21.15.57:0/1954049216 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fecd40272c0 con 0x7fece0100ab0 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:10:08.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:10:08.017 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 msgr2=0x7fecc806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feccc005950 tx=0x7feccc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 msgr2=0x7fece0100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fecd400bd70 tx=0x7fecd400be50 comp rx=0 tx=0).stop 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 shutdown_connections 2024-02-28T23:10:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece0100ab0 0x7fece0100410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 --2- 172.21.15.57:0/1954049216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece01033d0 0x7fece00fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.019+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 >> 172.21.15.57:0/1954049216 conn(0x7fece00fa4d0 msgr2=0x7fece00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.020+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 shutdown_connections 2024-02-28T23:10:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.020+0000 7fece6e00700 1 -- 172.21.15.57:0/1954049216 wait complete. 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 -- 172.21.15.57:0/3491615308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c0737b0 msgr2=0x7f0c1c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 --2- 172.21.15.57:0/3491615308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c073c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009940 tx=0x7f0c04009c50 comp rx=0 tx=0).stop 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 -- 172.21.15.57:0/3491615308 shutdown_connections 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 --2- 172.21.15.57:0/3491615308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c073c20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 --2- 172.21.15.57:0/3491615308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c074d80 0x7f0c1c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 -- 172.21.15.57:0/3491615308 >> 172.21.15.57:0/3491615308 conn(0x7f0c1c0fb7f0 msgr2=0x7f0c1c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 -- 172.21.15.57:0/3491615308 shutdown_connections 2024-02-28T23:10:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.183+0000 7f0c21000700 1 -- 172.21.15.57:0/3491615308 wait complete. 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 Processor -- start 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 -- start start 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c1987e0 con 0x7f0c1c074d80 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.184+0000 7f0c21000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c198950 con 0x7f0c1c0737b0 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37320/0 (socket says 172.21.15.57:37320) 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 -- 172.21.15.57:0/1059166686 learned_addr learned my addr 172.21.15.57:0/1059166686 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c19e00700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 -- 172.21.15.57:0/1059166686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 msgr2=0x7f0c1c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c11400700 1 -- 172.21.15.57:0/1059166686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040095f0 con 0x7f0c1c074d80 2024-02-28T23:10:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.185+0000 7f0c19e00700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:10:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c11400700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c040055f0 tx=0x7f0c040056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0401d070 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c04003ba0 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c04021710 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c19d380 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.186+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c19d870 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.187+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c10af80 con 0x7f0c1c074d80 2024-02-28T23:10:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.188+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f0c0400faa0 con 0x7f0c1c074d80 2024-02-28T23:10:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.189+0000 7f0c13400700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 0x7f0c0006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.190+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0c0408e320 con 0x7f0c1c074d80 2024-02-28T23:10:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.190+0000 7f0c19e00700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 0x7f0c0006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.190+0000 7f0c19e00700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 0x7f0c0006f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0060b0 tx=0x7f0c0c0076c0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:08.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.194+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f0c04057fe0 con 0x7f0c1c074d80 2024-02-28T23:10:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.439+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c1c107740 con 0x7f0c0006d520 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.440+0000 7f0c13400700 1 -- 172.21.15.57:0/1059166686 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f0c1c107740 con 0x7f0c0006d520 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:10:08.440 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:10:08.441 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2024-02-28T23:10:08.441 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading prometheus daemons", 2024-02-28T23:10:08.441 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:10:08.441 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:10:08.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 msgr2=0x7f0c0006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 0x7f0c0006f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0060b0 tx=0x7f0c0c0076c0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 msgr2=0x7f0c1c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c040055f0 tx=0x7f0c040056d0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 shutdown_connections 2024-02-28T23:10:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f0c0006d520 0x7f0c0006f9d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.443+0000 7f0c21000700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c1c0737b0 0x7f0c1c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.444+0000 7f0c21000700 1 --2- 172.21.15.57:0/1059166686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c1c074d80 0x7f0c1c1981e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.444+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 >> 172.21.15.57:0/1059166686 conn(0x7f0c1c0fb7f0 msgr2=0x7f0c1c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.444+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 shutdown_connections 2024-02-28T23:10:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:08.444+0000 7f0c21000700 1 -- 172.21.15.57:0/1059166686 wait complete. 2024-02-28T23:10:08.633 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:08 smithi057 ceph-mon[29096]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.633 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:08 smithi057 ceph-mon[29096]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.634 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:08 smithi057 ceph-mon[29096]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.634 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:08 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1954049216' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:08 smithi084 ceph-mon[34826]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:08 smithi084 ceph-mon[34826]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:08 smithi084 ceph-mon[34826]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:08 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1954049216' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:09.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:09 smithi057 ceph-mon[29096]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:09.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:09 smithi057 ceph-mon[29096]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:09 smithi084 ceph-mon[34826]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:09 smithi084 ceph-mon[34826]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:10:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:10:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:12.741 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.743 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:12 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:10:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:12.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:12 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:13.647 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:13 smithi057 ceph-mon[29096]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:13.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:13 smithi084 ceph-mon[34826]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:14.766 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:15.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:15 smithi057 ceph-mon[29096]: Upgrade: Updating alertmanager.smithi057 2024-02-28T23:10:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:15 smithi057 ceph-mon[29096]: Deploying daemon alertmanager.smithi057 on smithi057 2024-02-28T23:10:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:15 smithi057 ceph-mon[29096]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:15 smithi084 ceph-mon[34826]: Upgrade: Updating alertmanager.smithi057 2024-02-28T23:10:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:15 smithi084 ceph-mon[34826]: Deploying daemon alertmanager.smithi057 on smithi057 2024-02-28T23:10:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:15 smithi084 ceph-mon[34826]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:17.712 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:17 smithi057 ceph-mon[29096]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:17 smithi084 ceph-mon[34826]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:19 smithi057 ceph-mon[29096]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:19 smithi084 ceph-mon[34826]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:21 smithi057 ceph-mon[29096]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:21 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:21 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:21 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:21.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:21 smithi084 ceph-mon[34826]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:21.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:21 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:21.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:21 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:21.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:21 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:23 smithi057 ceph-mon[29096]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:23 smithi084 ceph-mon[34826]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:25.810 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:25 smithi057 ceph-mon[29096]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:25 smithi084 ceph-mon[34826]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:27 smithi057 ceph-mon[29096]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:27 smithi084 ceph-mon[34826]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:29 smithi057 ceph-mon[29096]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:29 smithi084 ceph-mon[34826]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:31.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:31 smithi057 ceph-mon[29096]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:31 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:31 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:31 smithi084 ceph-mon[34826]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:31 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:31 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:33.632 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:33 smithi057 ceph-mon[29096]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:33 smithi084 ceph-mon[34826]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:35 smithi057 ceph-mon[29096]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:35 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:35 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:35 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:10:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:35 smithi084 ceph-mon[34826]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:35 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:35 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:35 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:10:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:10:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T23:10:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.821 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:36 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:10:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:36.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:36 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:37 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T23:10:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:37 smithi084 ceph-mon[34826]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:37.936 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:37 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-28T23:10:37.936 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:37 smithi057 ceph-mon[29096]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 -- 172.21.15.57:0/3178413826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd540737b0 msgr2=0x7efd54073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 --2- 172.21.15.57:0/3178413826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd540737b0 0x7efd54073c20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efd40009910 tx=0x7efd40009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 -- 172.21.15.57:0/3178413826 shutdown_connections 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 --2- 172.21.15.57:0/3178413826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd540737b0 0x7efd54073c20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 --2- 172.21.15.57:0/3178413826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd54074d80 0x7efd540731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 -- 172.21.15.57:0/3178413826 >> 172.21.15.57:0/3178413826 conn(0x7efd540fb7f0 msgr2=0x7efd540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 -- 172.21.15.57:0/3178413826 shutdown_connections 2024-02-28T23:10:38.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.622+0000 7efd5ac00700 1 -- 172.21.15.57:0/3178413826 wait complete. 2024-02-28T23:10:38.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 Processor -- start 2024-02-28T23:10:38.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 -- start start 2024-02-28T23:10:38.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd540737b0 0x7efd5419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:38.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:38.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419cc00 con 0x7efd54074d80 2024-02-28T23:10:38.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.624+0000 7efd5ac00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419cd40 con 0x7efd540737b0 2024-02-28T23:10:38.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35108/0 (socket says 172.21.15.57:35108) 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 -- 172.21.15.57:0/1090546017 learned_addr learned my addr 172.21.15.57:0/1090546017 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53e00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd540737b0 0x7efd5419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 -- 172.21.15.57:0/1090546017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd540737b0 msgr2=0x7efd5419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd540737b0 0x7efd5419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.625+0000 7efd53400700 1 -- 172.21.15.57:0/1090546017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd400095f0 con 0x7efd54074d80 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd53400700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd40005b40 tx=0x7efd4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4001d070 con 0x7efd54074d80 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd4000f460 con 0x7efd54074d80 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541a1790 con 0x7efd54074d80 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd40021620 con 0x7efd54074d80 2024-02-28T23:10:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.626+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd541a1c80 con 0x7efd54074d80 2024-02-28T23:10:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.628+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5410f380 con 0x7efd54074d80 2024-02-28T23:10:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.629+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7efd4000fa40 con 0x7efd54074d80 2024-02-28T23:10:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.630+0000 7efd50c00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 0x7efd3806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.630+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efd4008e1f0 con 0x7efd54074d80 2024-02-28T23:10:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.630+0000 7efd53e00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 0x7efd3806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.631+0000 7efd53e00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 0x7efd3806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd44005950 tx=0x7efd4400b410 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:38.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.635+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7efd4005ba00 con 0x7efd54074d80 2024-02-28T23:10:38.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.891+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd541a2380 con 0x7efd3806d520 2024-02-28T23:10:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.893+0000 7efd50c00700 1 -- 172.21.15.57:0/1090546017 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7efd541a2380 con 0x7efd3806d520 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 msgr2=0x7efd3806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 0x7efd3806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd44005950 tx=0x7efd4400b410 comp rx=0 tx=0).stop 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 msgr2=0x7efd5419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd40005b40 tx=0x7efd4000bd70 comp rx=0 tx=0).stop 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 shutdown_connections 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efd3806d520 0x7efd3806f9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd540737b0 0x7efd5419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 --2- 172.21.15.57:0/1090546017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd54074d80 0x7efd5419c5e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 >> 172.21.15.57:0/1090546017 conn(0x7efd540fb7f0 msgr2=0x7efd54101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:38.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.896+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 shutdown_connections 2024-02-28T23:10:38.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:38.897+0000 7efd5ac00700 1 -- 172.21.15.57:0/1090546017 wait complete. 2024-02-28T23:10:38.916 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 -- 172.21.15.57:0/2313863735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54980737b0 msgr2=0x7f5498073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 --2- 172.21.15.57:0/2313863735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54980737b0 0x7f5498073c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5484009940 tx=0x7f5484009c50 comp rx=0 tx=0).stop 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 -- 172.21.15.57:0/2313863735 shutdown_connections 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 --2- 172.21.15.57:0/2313863735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54980737b0 0x7f5498073c20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 --2- 172.21.15.57:0/2313863735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5498074d80 0x7f54980731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.065+0000 7f549d200700 1 -- 172.21.15.57:0/2313863735 >> 172.21.15.57:0/2313863735 conn(0x7f54980fb7f0 msgr2=0x7f54980fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:39.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.066+0000 7f549d200700 1 -- 172.21.15.57:0/2313863735 shutdown_connections 2024-02-28T23:10:39.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.066+0000 7f549d200700 1 -- 172.21.15.57:0/2313863735 wait complete. 2024-02-28T23:10:39.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.067+0000 7f549d200700 1 Processor -- start 2024-02-28T23:10:39.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f549d200700 1 -- start start 2024-02-28T23:10:39.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f549d200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54980737b0 0x7f5498197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f549d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f549d200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54981987e0 con 0x7f5498074d80 2024-02-28T23:10:39.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f549d200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498198950 con 0x7f54980737b0 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f5495400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.068+0000 7f5495400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35136/0 (socket says 172.21.15.57:35136) 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495400700 1 -- 172.21.15.57:0/2051352415 learned_addr learned my addr 172.21.15.57:0/2051352415 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495e00700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54980737b0 0x7f5498197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495400700 1 -- 172.21.15.57:0/2051352415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54980737b0 msgr2=0x7f5498197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495400700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54980737b0 0x7f5498197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495400700 1 -- 172.21.15.57:0/2051352415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840095f0 con 0x7f5498074d80 2024-02-28T23:10:39.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f5495400700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54840055f0 tx=0x7f54840056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.069+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548401d070 con 0x7f5498074d80 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.070+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f548400f460 con 0x7f5498074d80 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.070+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5484021620 con 0x7f5498074d80 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.070+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549819d380 con 0x7f5498074d80 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.070+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549819d820 con 0x7f5498074d80 2024-02-28T23:10:39.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.071+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549810af80 con 0x7f5498074d80 2024-02-28T23:10:39.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.072+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f548400f5d0 con 0x7f5498074d80 2024-02-28T23:10:39.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.073+0000 7f548ea00700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 0x7f547c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.073+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f548408e2f0 con 0x7f5498074d80 2024-02-28T23:10:39.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.074+0000 7f5495e00700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 0x7f547c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.074+0000 7f5495e00700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 0x7f547c06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5488005950 tx=0x7f54880058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.078+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f5484057fd0 con 0x7f5498074d80 2024-02-28T23:10:39.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.321+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f549819df70 con 0x7f547c06d520 2024-02-28T23:10:39.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.323+0000 7f548ea00700 1 -- 172.21.15.57:0/2051352415 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f549819df70 con 0x7f547c06d520 2024-02-28T23:10:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.325+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 msgr2=0x7f547c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.325+0000 7f549d200700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 0x7f547c06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5488005950 tx=0x7f54880058e0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.325+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 msgr2=0x7f54981981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.325+0000 7f549d200700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54840055f0 tx=0x7f54840056d0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 shutdown_connections 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f547c06d520 0x7f547c06f9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54980737b0 0x7f5498197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 --2- 172.21.15.57:0/2051352415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5498074d80 0x7f54981981e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 >> 172.21.15.57:0/2051352415 conn(0x7f54980fb7f0 msgr2=0x7f5498106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 shutdown_connections 2024-02-28T23:10:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.326+0000 7f549d200700 1 -- 172.21.15.57:0/2051352415 wait complete. 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 -- 172.21.15.57:0/3924304818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 msgr2=0x7fdc70101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 --2- 172.21.15.57:0/3924304818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70101890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009910 tx=0x7fdc5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 -- 172.21.15.57:0/3924304818 shutdown_connections 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 --2- 172.21.15.57:0/3924304818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70101890 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 --2- 172.21.15.57:0/3924304818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 0x7fdc70100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 -- 172.21.15.57:0/3924304818 >> 172.21.15.57:0/3924304818 conn(0x7fdc700fb7f0 msgr2=0x7fdc700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 -- 172.21.15.57:0/3924304818 shutdown_connections 2024-02-28T23:10:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.486+0000 7fdc77a00700 1 -- 172.21.15.57:0/3924304818 wait complete. 2024-02-28T23:10:39.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.487+0000 7fdc77a00700 1 Processor -- start 2024-02-28T23:10:39.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.487+0000 7fdc77a00700 1 -- start start 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc77a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 0x7fdc70195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc77a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc77a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701965f0 con 0x7fdc70101440 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc77a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70196730 con 0x7fdc70100240 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc6fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc6fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35156/0 (socket says 172.21.15.57:35156) 2024-02-28T23:10:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.488+0000 7fdc6fe00700 1 -- 172.21.15.57:0/4145362812 learned_addr learned my addr 172.21.15.57:0/4145362812 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:39.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc75000700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 0x7fdc70195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6fe00700 1 -- 172.21.15.57:0/4145362812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 msgr2=0x7fdc70195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6fe00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 0x7fdc70195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6fe00700 1 -- 172.21.15.57:0/4145362812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c0095f0 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6fe00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00bd70 tx=0x7fdc5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc5c01d070 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c022470 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc5c00f670 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7019b180 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.489+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7019b670 con 0x7fdc70101440 2024-02-28T23:10:39.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.490+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc70108d90 con 0x7fdc70101440 2024-02-28T23:10:39.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.492+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fdc5c022a50 con 0x7fdc70101440 2024-02-28T23:10:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.492+0000 7fdc6d600700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 0x7fdc5806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.493+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdc5c08e6e0 con 0x7fdc70101440 2024-02-28T23:10:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.493+0000 7fdc75000700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 0x7fdc5806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.493+0000 7fdc75000700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 0x7fdc5806f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc701012a0 tx=0x7fdc64005cf0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.497+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fdc5c05c0a0 con 0x7fdc70101440 2024-02-28T23:10:39.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.742+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc7019bdc0 con 0x7fdc5806d520 2024-02-28T23:10:39.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.761+0000 7fdc6d600700 1 -- 172.21.15.57:0/4145362812 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdc7019bdc0 con 0x7fdc5806d520 2024-02-28T23:10:39.760 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:10:39.760 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (19s) 11s ago 14m 13.9M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (10m) 11s ago 14m 8082k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (13m) 66s ago 13m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (14m) 11s ago 14m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (12m) 66s ago 12m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (11m) 11s ago 13m 73.9M - 9.4.7 954c08fa6188 0deb89854cb5 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (6m) 11s ago 6m 57.1M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (6m) 11s ago 6m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (6m) 66s ago 6m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (6m) 66s ago 6m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (2m) 11s ago 15m 500M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (101s) 66s ago 12m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:10:39.761 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (16m) 11s ago 16m 56.1M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (12m) 66s ago 12m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (81s) 11s ago 14m 7277k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (71s) 66s ago 12m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (11m) 11s ago 11m 227M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (11m) 11s ago 11m 239M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (10m) 11s ago 10m 201M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (9m) 66s ago 9m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (9m) 66s ago 9m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (8m) 66s ago 8m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:10:39.762 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (43s) 11s ago 13m 44.4M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:10:39.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 msgr2=0x7fdc5806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 0x7fdc5806f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc701012a0 tx=0x7fdc64005cf0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 msgr2=0x7fdc70195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00bd70 tx=0x7fdc5c00be50 comp rx=0 tx=0).stop 2024-02-28T23:10:39.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 shutdown_connections 2024-02-28T23:10:39.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fdc5806d520 0x7fdc5806f9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc70100240 0x7fdc70195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 --2- 172.21.15.57:0/4145362812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc70101440 0x7fdc70195fd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 >> 172.21.15.57:0/4145362812 conn(0x7fdc700fb7f0 msgr2=0x7fdc70104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:39.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 shutdown_connections 2024-02-28T23:10:39.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.765+0000 7fdc77a00700 1 -- 172.21.15.57:0/4145362812 wait complete. 2024-02-28T23:10:39.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:39 smithi084 ceph-mon[34826]: Upgrade: Updating grafana.smithi057 2024-02-28T23:10:39.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:39 smithi084 ceph-mon[34826]: Deploying daemon grafana.smithi057 on smithi057 2024-02-28T23:10:39.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:39 smithi084 ceph-mon[34826]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:39.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 -- 172.21.15.57:0/4103436210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 msgr2=0x7fe460106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 --2- 172.21.15.57:0/4103436210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460106a80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe450009910 tx=0x7fe450009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:39.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 -- 172.21.15.57:0/4103436210 shutdown_connections 2024-02-28T23:10:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 --2- 172.21.15.57:0/4103436210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 0x7fe460107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 --2- 172.21.15.57:0/4103436210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460106a80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 -- 172.21.15.57:0/4103436210 >> 172.21.15.57:0/4103436210 conn(0x7fe460075740 msgr2=0x7fe460077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 -- 172.21.15.57:0/4103436210 shutdown_connections 2024-02-28T23:10:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.942+0000 7fe45f200700 1 -- 172.21.15.57:0/4103436210 wait complete. 2024-02-28T23:10:39.942 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:39 smithi057 ceph-mon[29096]: Upgrade: Updating grafana.smithi057 2024-02-28T23:10:39.942 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:39 smithi057 ceph-mon[29096]: Deploying daemon grafana.smithi057 on smithi057 2024-02-28T23:10:39.943 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:39 smithi057 ceph-mon[29096]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.943+0000 7fe45f200700 1 Processor -- start 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.943+0000 7fe45f200700 1 -- start start 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 0x7fe4600810a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600816c0 con 0x7fe460106670 2024-02-28T23:10:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45f200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a9c40 con 0x7fe460107870 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35180/0 (socket says 172.21.15.57:35180) 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.944+0000 7fe45de00700 1 -- 172.21.15.57:0/1162636361 learned_addr learned my addr 172.21.15.57:0/1162636361 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.945+0000 7fe45d400700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 0x7fe4600810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.945+0000 7fe45de00700 1 -- 172.21.15.57:0/1162636361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 msgr2=0x7fe4600810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.945+0000 7fe45de00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 0x7fe4600810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.945+0000 7fe45de00700 1 -- 172.21.15.57:0/1162636361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500095f0 con 0x7fe460106670 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.945+0000 7fe45de00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe45000bda0 tx=0x7fe45000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.946+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe45001d070 con 0x7fe460106670 2024-02-28T23:10:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.946+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe450004c50 con 0x7fe460106670 2024-02-28T23:10:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.946+0000 7fe45f200700 1 -- 172.21.15.57:0/1162636361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4601a9de0 con 0x7fe460106670 2024-02-28T23:10:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.946+0000 7fe45f200700 1 -- 172.21.15.57:0/1162636361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4601aa220 con 0x7fe460106670 2024-02-28T23:10:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.946+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe45000f690 con 0x7fe460106670 2024-02-28T23:10:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.948+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4380052f0 con 0x7fe460106670 2024-02-28T23:10:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.949+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fe45000f7f0 con 0x7fe460106670 2024-02-28T23:10:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.950+0000 7fe456a00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 0x7fe44406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.951+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe45008e7a0 con 0x7fe460106670 2024-02-28T23:10:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.951+0000 7fe45d400700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 0x7fe44406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.952+0000 7fe45d400700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 0x7fe44406f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005fd0 tx=0x7fe44c00a560 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:39.955+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe45005bf90 con 0x7fe460106670 2024-02-28T23:10:40.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.323+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4380061d0 con 0x7fe460106670 2024-02-28T23:10:40.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.324+0000 7fe456a00700 1 -- 172.21.15.57:0/1162636361 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fe45005bb20 con 0x7fe460106670 2024-02-28T23:10:40.322 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:10:40.322 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:10:40.322 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:10:40.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:10:40.323 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:10:40.324 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 msgr2=0x7fe44406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 0x7fe44406f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005fd0 tx=0x7fe44c00a560 comp rx=0 tx=0).stop 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 msgr2=0x7fe460080b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe45000bda0 tx=0x7fe45000be80 comp rx=0 tx=0).stop 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 shutdown_connections 2024-02-28T23:10:40.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fe44406d520 0x7fe44406f9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe460106670 0x7fe460080b60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.326+0000 7fe44be00700 1 --2- 172.21.15.57:0/1162636361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe460107870 0x7fe4600810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.327+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 >> 172.21.15.57:0/1162636361 conn(0x7fe460075740 msgr2=0x7fe46010aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:40.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.327+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 shutdown_connections 2024-02-28T23:10:40.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.327+0000 7fe44be00700 1 -- 172.21.15.57:0/1162636361 wait complete. 2024-02-28T23:10:40.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 -- 172.21.15.57:0/2981083066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 msgr2=0x7f968c103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 --2- 172.21.15.57:0/2981083066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c103140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f967c009910 tx=0x7f967c009c20 comp rx=0 tx=0).stop 2024-02-28T23:10:40.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 -- 172.21.15.57:0/2981083066 shutdown_connections 2024-02-28T23:10:40.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 --2- 172.21.15.57:0/2981083066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 0x7f968c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 --2- 172.21.15.57:0/2981083066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c103140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 -- 172.21.15.57:0/2981083066 >> 172.21.15.57:0/2981083066 conn(0x7f968c0fa780 msgr2=0x7f968c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:40.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 -- 172.21.15.57:0/2981083066 shutdown_connections 2024-02-28T23:10:40.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.504+0000 7f9691e00700 1 -- 172.21.15.57:0/2981083066 wait complete. 2024-02-28T23:10:40.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.506+0000 7f9691e00700 1 Processor -- start 2024-02-28T23:10:40.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.506+0000 7f9691e00700 1 -- start start 2024-02-28T23:10:40.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f9691e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:40.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f9691e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 0x7f968c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:40.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f9691e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1987f0 con 0x7f968c100d60 2024-02-28T23:10:40.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f9691e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c198960 con 0x7f968c103680 2024-02-28T23:10:40.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:40.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35196/0 (socket says 172.21.15.57:35196) 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 -- 172.21.15.57:0/613858733 learned_addr learned my addr 172.21.15.57:0/613858733 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968a000700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 0x7f968c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 -- 172.21.15.57:0/613858733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 msgr2=0x7f968c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 0x7f968c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.507+0000 7f968aa00700 1 -- 172.21.15.57:0/613858733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0095f0 con 0x7f968c100d60 2024-02-28T23:10:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.508+0000 7f968aa00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f967c00bda0 tx=0x7f967c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.508+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967c01d070 con 0x7f968c100d60 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.508+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c19d390 con 0x7f968c100d60 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.508+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f967c00f460 con 0x7f968c100d60 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.508+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c19d880 con 0x7f968c100d60 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.509+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967c02b610 con 0x7f968c100d60 2024-02-28T23:10:40.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.510+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c0fc340 con 0x7f968c100d60 2024-02-28T23:10:40.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.511+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f967c00f5d0 con 0x7f968c100d60 2024-02-28T23:10:40.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.512+0000 7f9690a00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 0x7f967006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:10:40.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.512+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f967c08e250 con 0x7f968c100d60 2024-02-28T23:10:40.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.512+0000 7f968a000700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 0x7f967006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:10:40.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.513+0000 7f968a000700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 0x7f967006f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9678005950 tx=0x7f96780058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:10:40.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.517+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f967c057f40 con 0x7f968c100d60 2024-02-28T23:10:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.814+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f968c061190 con 0x7f967006d520 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.816+0000 7f9690a00700 1 -- 172.21.15.57:0/613858733 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f968c061190 con 0x7f967006d520 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:10:40.814 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading grafana daemons", 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:10:40.815 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:10:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.819+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 msgr2=0x7f967006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.819+0000 7f9691e00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 0x7f967006f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9678005950 tx=0x7f96780058e0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.819+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 msgr2=0x7f968c197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.819+0000 7f9691e00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f967c00bda0 tx=0x7f967c00be80 comp rx=0 tx=0).stop 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 shutdown_connections 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f967006d520 0x7f967006f9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f968c100d60 0x7f968c197cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 --2- 172.21.15.57:0/613858733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f968c103680 0x7f968c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 >> 172.21.15.57:0/613858733 conn(0x7f968c0fa780 msgr2=0x7f968c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 shutdown_connections 2024-02-28T23:10:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:10:40.820+0000 7f9691e00700 1 -- 172.21.15.57:0/613858733 wait complete. 2024-02-28T23:10:40.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:40 smithi057 ceph-mon[29096]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:40.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:40 smithi057 ceph-mon[29096]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:40.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:40 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1162636361' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:40 smithi084 ceph-mon[34826]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:40 smithi084 ceph-mon[34826]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:40 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1162636361' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:10:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:41 smithi057 ceph-mon[29096]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:42.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:41 smithi057 ceph-mon[29096]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:41 smithi084 ceph-mon[34826]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:41 smithi084 ceph-mon[34826]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:10:42.802 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:42 smithi057 ceph-mon[29096]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:42 smithi084 ceph-mon[34826]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:10:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:43 smithi057 ceph-mon[29096]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:43 smithi084 ceph-mon[34826]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:44 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:44 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:45 smithi057 ceph-mon[29096]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:45 smithi084 ceph-mon[34826]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:47 smithi057 ceph-mon[29096]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:47 smithi084 ceph-mon[34826]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:49 smithi057 ceph-mon[29096]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:49 smithi084 ceph-mon[34826]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:51 smithi057 ceph-mon[29096]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:51 smithi084 ceph-mon[34826]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:52 smithi057 ceph-mon[29096]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:52 smithi084 ceph-mon[34826]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:10:55.681 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:55 smithi057 ceph-mon[29096]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:55.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:55 smithi084 ceph-mon[34826]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:57.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:57 smithi084 ceph-mon[34826]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:57.737 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:57 smithi057 ceph-mon[29096]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:10:59.632 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:59 smithi057 ceph-mon[29096]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:59.632 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:10:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:10:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:59 smithi084 ceph-mon[34826]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:10:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:10:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:00 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:00 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:00 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:00.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:00 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:00.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:00 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:00.854 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:00 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:01.877 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:01 smithi057 ceph-mon[29096]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:01 smithi084 ceph-mon[34826]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:03.957 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:03 smithi057 ceph-mon[29096]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:03 smithi084 ceph-mon[34826]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:04 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:11:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:04 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:11:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:05 smithi057 ceph-mon[29096]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:05 smithi084 ceph-mon[34826]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:07 smithi084 ceph-mon[34826]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:07.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:07.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:07 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:07.889 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:07 smithi057 ceph-mon[29096]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:07.889 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:07.889 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:07 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:09.924 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:09 smithi057 ceph-mon[29096]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:09.924 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:09 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:09.924 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:09 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:09 smithi084 ceph-mon[34826]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:09 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:09 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.995+0000 7f1a8ee00700 1 -- 172.21.15.57:0/3002962423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 msgr2=0x7f1a88102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.995+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/3002962423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88102df0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009910 tx=0x7f1a78009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 -- 172.21.15.57:0/3002962423 shutdown_connections 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/3002962423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/3002962423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88102df0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 -- 172.21.15.57:0/3002962423 >> 172.21.15.57:0/3002962423 conn(0x7f1a880fa780 msgr2=0x7f1a880fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 -- 172.21.15.57:0/3002962423 shutdown_connections 2024-02-28T23:11:10.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.996+0000 7f1a8ee00700 1 -- 172.21.15.57:0/3002962423 wait complete. 2024-02-28T23:11:10.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 Processor -- start 2024-02-28T23:11:10.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 -- start start 2024-02-28T23:11:10.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:10.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a881987e0 con 0x7f1a88069250 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.997+0000 7f1a8ee00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88198950 con 0x7f1a88103330 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39832/0 (socket says 172.21.15.57:39832) 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 -- 172.21.15.57:0/1940630933 learned_addr learned my addr 172.21.15.57:0/1940630933 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:10.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87400700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 -- 172.21.15.57:0/1940630933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 msgr2=0x7f1a88198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87e00700 1 -- 172.21.15.57:0/1940630933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780095f0 con 0x7f1a88069250 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.998+0000 7f1a87400700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a87e00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800bda0 tx=0x7f1a7800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7801d070 con 0x7f1a88069250 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8819d340 con 0x7f1a88069250 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8819d830 con 0x7f1a88069250 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a78004c50 con 0x7f1a88069250 2024-02-28T23:11:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:10.999+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a78021640 con 0x7f1a88069250 2024-02-28T23:11:11.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.000+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a880fc340 con 0x7f1a88069250 2024-02-28T23:11:11.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.003+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f1a7800f460 con 0x7f1a88069250 2024-02-28T23:11:11.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.003+0000 7f1a84c00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 0x7f1a6c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.004+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1a7808e330 con 0x7f1a88069250 2024-02-28T23:11:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.004+0000 7f1a87400700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 0x7f1a6c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.006+0000 7f1a87400700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 0x7f1a6c06f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1a74005fd0 tx=0x7f1a74005e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:11.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.009+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f1a780581e0 con 0x7f1a88069250 2024-02-28T23:11:11.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.267+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a88061190 con 0x7f1a6c06d520 2024-02-28T23:11:11.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.270+0000 7f1a84c00700 1 -- 172.21.15.57:0/1940630933 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f1a88061190 con 0x7f1a6c06d520 2024-02-28T23:11:11.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 msgr2=0x7f1a6c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 0x7f1a6c06f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1a74005fd0 tx=0x7f1a74005e30 comp rx=0 tx=0).stop 2024-02-28T23:11:11.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 msgr2=0x7f1a88197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800bda0 tx=0x7f1a7800be80 comp rx=0 tx=0).stop 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 shutdown_connections 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f1a6c06d520 0x7f1a6c06f9d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a88069250 0x7f1a88197d10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 --2- 172.21.15.57:0/1940630933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a88103330 0x7f1a88198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 >> 172.21.15.57:0/1940630933 conn(0x7f1a880fa780 msgr2=0x7f1a88104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 shutdown_connections 2024-02-28T23:11:11.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.273+0000 7f1a8ee00700 1 -- 172.21.15.57:0/1940630933 wait complete. 2024-02-28T23:11:11.291 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:11:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 -- 172.21.15.57:0/2844052896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f540737b0 msgr2=0x7f9f54073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 --2- 172.21.15.57:0/2844052896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f540737b0 0x7f9f54073c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009940 tx=0x7f9f40009c50 comp rx=0 tx=0).stop 2024-02-28T23:11:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 -- 172.21.15.57:0/2844052896 shutdown_connections 2024-02-28T23:11:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 --2- 172.21.15.57:0/2844052896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f540737b0 0x7f9f54073c20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 --2- 172.21.15.57:0/2844052896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54074d80 0x7f9f540731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 -- 172.21.15.57:0/2844052896 >> 172.21.15.57:0/2844052896 conn(0x7f9f540fb7f0 msgr2=0x7f9f540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.440+0000 7f9f5b800700 1 -- 172.21.15.57:0/2844052896 shutdown_connections 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.441+0000 7f9f5b800700 1 -- 172.21.15.57:0/2844052896 wait complete. 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 Processor -- start 2024-02-28T23:11:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 -- start start 2024-02-28T23:11:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f540737b0 0x7f9f54197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541987e0 con 0x7f9f54074d80 2024-02-28T23:11:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.442+0000 7f9f5b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54198950 con 0x7f9f540737b0 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.443+0000 7f9f53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.443+0000 7f9f58e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f540737b0 0x7f9f54197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.443+0000 7f9f53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39840/0 (socket says 172.21.15.57:39840) 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.443+0000 7f9f53e00700 1 -- 172.21.15.57:0/1668497633 learned_addr learned my addr 172.21.15.57:0/1668497633 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.444+0000 7f9f53e00700 1 -- 172.21.15.57:0/1668497633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f540737b0 msgr2=0x7f9f54197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.444+0000 7f9f53e00700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f540737b0 0x7f9f54197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.444+0000 7f9f53e00700 1 -- 172.21.15.57:0/1668497633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400095f0 con 0x7f9f54074d80 2024-02-28T23:11:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.444+0000 7f9f53e00700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f400055f0 tx=0x7f9f40005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.445+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4001d070 con 0x7f9f54074d80 2024-02-28T23:11:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.445+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5419d380 con 0x7f9f54074d80 2024-02-28T23:11:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.445+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f4000f460 con 0x7f9f54074d80 2024-02-28T23:11:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.445+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f40021640 con 0x7f9f54074d80 2024-02-28T23:11:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.445+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5419d870 con 0x7f9f54074d80 2024-02-28T23:11:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.447+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5410af80 con 0x7f9f54074d80 2024-02-28T23:11:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.448+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f9f4000fad0 con 0x7f9f54074d80 2024-02-28T23:11:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.449+0000 7f9f51600700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.450+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f4008e130 con 0x7f9f54074d80 2024-02-28T23:11:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.450+0000 7f9f58e00700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.451+0000 7f9f58e00700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f54198b80 tx=0x7f9f4800b410 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:11.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.455+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9f40057e90 con 0x7f9f54074d80 2024-02-28T23:11:11.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.708+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5419df70 con 0x7f9f3c06d520 2024-02-28T23:11:11.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.711+0000 7f9f51600700 1 -- 172.21.15.57:0/1668497633 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f9f5419df70 con 0x7f9f3c06d520 2024-02-28T23:11:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.714+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 msgr2=0x7f9f3c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.714+0000 7f9f5b800700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f54198b80 tx=0x7f9f4800b410 comp rx=0 tx=0).stop 2024-02-28T23:11:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.714+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 msgr2=0x7f9f541981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.714+0000 7f9f5b800700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f400055f0 tx=0x7f9f40005620 comp rx=0 tx=0).stop 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 shutdown_connections 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f540737b0 0x7f9f54197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 --2- 172.21.15.57:0/1668497633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f54074d80 0x7f9f541981e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 >> 172.21.15.57:0/1668497633 conn(0x7f9f540fb7f0 msgr2=0x7f9f54106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 shutdown_connections 2024-02-28T23:11:11.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.715+0000 7f9f5b800700 1 -- 172.21.15.57:0/1668497633 wait complete. 2024-02-28T23:11:11.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.861+0000 7f6b77e00700 1 -- 172.21.15.57:0/1432153046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 msgr2=0x7f6b70075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.861+0000 7f6b77e00700 1 --2- 172.21.15.57:0/1432153046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b70075b10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009910 tx=0x7f6b64009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:11.861 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:11 smithi057 ceph-mon[29096]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 -- 172.21.15.57:0/1432153046 shutdown_connections 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 --2- 172.21.15.57:0/1432153046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b70076d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 --2- 172.21.15.57:0/1432153046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b70075b10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 -- 172.21.15.57:0/1432153046 >> 172.21.15.57:0/1432153046 conn(0x7f6b700fda60 msgr2=0x7f6b700ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 -- 172.21.15.57:0/1432153046 shutdown_connections 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.862+0000 7f6b77e00700 1 -- 172.21.15.57:0/1432153046 wait complete. 2024-02-28T23:11:11.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.863+0000 7f6b77e00700 1 Processor -- start 2024-02-28T23:11:11.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.863+0000 7f6b77e00700 1 -- start start 2024-02-28T23:11:11.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.863+0000 7f6b77e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b77e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b7019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b77e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7019cb90 con 0x7f6b70075700 2024-02-28T23:11:11.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b77e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7019ccd0 con 0x7f6b70076900 2024-02-28T23:11:11.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b75400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b74a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b7019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b74a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b7019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:50200/0 (socket says 172.21.15.57:50200) 2024-02-28T23:11:11.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b74a00700 1 -- 172.21.15.57:0/3196770530 learned_addr learned my addr 172.21.15.57:0/3196770530 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.864+0000 7f6b75400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39858/0 (socket says 172.21.15.57:39858) 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b75400700 1 -- 172.21.15.57:0/3196770530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 msgr2=0x7f6b7019c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b75400700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b7019c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b75400700 1 -- 172.21.15.57:0/3196770530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640095f0 con 0x7f6b70075700 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b75400700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000c00 tx=0x7f6b6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:11.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6401d070 con 0x7f6b70075700 2024-02-28T23:11:11.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b6400f460 con 0x7f6b70075700 2024-02-28T23:11:11.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.865+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b701a1730 con 0x7f6b70075700 2024-02-28T23:11:11.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.866+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b64021640 con 0x7f6b70075700 2024-02-28T23:11:11.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.866+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b701a1bf0 con 0x7f6b70075700 2024-02-28T23:11:11.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.867+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7010f380 con 0x7f6b70075700 2024-02-28T23:11:11.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.868+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f6b6400fa20 con 0x7f6b70075700 2024-02-28T23:11:11.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.869+0000 7f6b6e000700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 0x7f6b5806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:11.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.869+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6b6408e2e0 con 0x7f6b70075700 2024-02-28T23:11:11.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.869+0000 7f6b74a00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 0x7f6b5806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:11.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.870+0000 7f6b74a00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 0x7f6b5806f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00ad20 tx=0x7f6b5c005f90 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:11.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:11.874+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6b6405ba50 con 0x7f6b70075700 2024-02-28T23:11:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:11 smithi084 ceph-mon[34826]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.105+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b701a22f0 con 0x7f6b5806d520 2024-02-28T23:11:12.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.144+0000 7f6b6e000700 1 -- 172.21.15.57:0/3196770530 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6b701a22f0 con 0x7f6b5806d520 2024-02-28T23:11:12.144 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (51s) 5s ago 14m 13.9M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (10m) 5s ago 15m 8144k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (13m) 98s ago 13m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (14m) 5s ago 14m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (13m) 98s ago 13m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (13s) 5s ago 14m 50.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (7m) 5s ago 7m 57.1M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (7m) 5s ago 7m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (7m) 98s ago 7m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (7m) 98s ago 7m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (3m) 5s ago 16m 501M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (2m) 98s ago 13m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (16m) 5s ago 16m 54.0M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:11:12.145 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (13m) 98s ago 13m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (113s) 5s ago 14m 7633k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (103s) 98s ago 13m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (12m) 5s ago 12m 227M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (11m) 5s ago 11m 240M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (10m) 5s ago 10m 201M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (10m) 98s ago 10m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (9m) 98s ago 9m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (9m) 98s ago 9m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:11:12.146 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (75s) 5s ago 13m 47.7M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 msgr2=0x7f6b5806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 0x7f6b5806f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00ad20 tx=0x7f6b5c005f90 comp rx=0 tx=0).stop 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 msgr2=0x7f6b7019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000c00 tx=0x7f6b6400bd70 comp rx=0 tx=0).stop 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 shutdown_connections 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6b5806d520 0x7f6b5806f9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b70075700 0x7f6b7019c0c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 --2- 172.21.15.57:0/3196770530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b70076900 0x7f6b7019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.148+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 >> 172.21.15.57:0/3196770530 conn(0x7f6b700fda60 msgr2=0x7f6b7010c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:12.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.149+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 shutdown_connections 2024-02-28T23:11:12.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.149+0000 7f6b77e00700 1 -- 172.21.15.57:0/3196770530 wait complete. 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 -- 172.21.15.57:0/3543628895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 msgr2=0x7f6d94073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 --2- 172.21.15.57:0/3543628895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d94073c20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009910 tx=0x7f6d84009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 -- 172.21.15.57:0/3543628895 shutdown_connections 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 --2- 172.21.15.57:0/3543628895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d94073c20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 --2- 172.21.15.57:0/3543628895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 0x7f6d940731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.315+0000 7f6d9b800700 1 -- 172.21.15.57:0/3543628895 >> 172.21.15.57:0/3543628895 conn(0x7f6d940fb7f0 msgr2=0x7f6d940fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.316+0000 7f6d9b800700 1 -- 172.21.15.57:0/3543628895 shutdown_connections 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.316+0000 7f6d9b800700 1 -- 172.21.15.57:0/3543628895 wait complete. 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 Processor -- start 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 -- start start 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 0x7f6d9419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419cc00 con 0x7f6d940737b0 2024-02-28T23:11:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d9b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419cd40 con 0x7f6d94074d80 2024-02-28T23:11:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.317+0000 7f6d98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39884/0 (socket says 172.21.15.57:39884) 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 -- 172.21.15.57:0/4082929040 learned_addr learned my addr 172.21.15.57:0/4082929040 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d93e00700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 0x7f6d9419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 -- 172.21.15.57:0/4082929040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 msgr2=0x7f6d9419c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 0x7f6d9419c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 -- 172.21.15.57:0/4082929040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840095f0 con 0x7f6d940737b0 2024-02-28T23:11:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.318+0000 7f6d98e00700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e920 tx=0x7f6d8800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:12.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.319+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d88009790 con 0x7f6d940737b0 2024-02-28T23:11:12.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.319+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d8800cba0 con 0x7f6d940737b0 2024-02-28T23:11:12.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.319+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d88018630 con 0x7f6d940737b0 2024-02-28T23:11:12.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.319+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d941a17f0 con 0x7f6d940737b0 2024-02-28T23:11:12.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.319+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d941a1d40 con 0x7f6d940737b0 2024-02-28T23:11:12.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.320+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9410f380 con 0x7f6d940737b0 2024-02-28T23:11:12.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.321+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f6d8800cd10 con 0x7f6d940737b0 2024-02-28T23:11:12.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.322+0000 7f6d91600700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 0x7f6d7c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.322+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6d88014070 con 0x7f6d940737b0 2024-02-28T23:11:12.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.322+0000 7f6d93e00700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 0x7f6d7c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.323+0000 7f6d93e00700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 0x7f6d7c06f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009fd0 tx=0x7f6d84005fd0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:12.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.327+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6d88052a50 con 0x7f6d940737b0 2024-02-28T23:11:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.662+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d941a2430 con 0x7f6d940737b0 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.663+0000 7f6d91600700 1 -- 172.21.15.57:0/4082929040 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f6d8805a810 con 0x7f6d940737b0 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:11:12.663 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:11:12.664 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:11:12.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.666+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 msgr2=0x7f6d7c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.666+0000 7f6d9b800700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 0x7f6d7c06f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009fd0 tx=0x7f6d84005fd0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.666+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 msgr2=0x7f6d9419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.666+0000 7f6d9b800700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e920 tx=0x7f6d8800ece0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 shutdown_connections 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f6d7c06d520 0x7f6d7c06f9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6d940737b0 0x7f6d9419c0a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 --2- 172.21.15.57:0/4082929040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d94074d80 0x7f6d9419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 >> 172.21.15.57:0/4082929040 conn(0x7f6d940fb7f0 msgr2=0x7f6d94101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 shutdown_connections 2024-02-28T23:11:12.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.667+0000 7f6d9b800700 1 -- 172.21.15.57:0/4082929040 wait complete. 2024-02-28T23:11:12.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 -- 172.21.15.57:0/1556721091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f4100d80 msgr2=0x7f62f4103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 --2- 172.21.15.57:0/1556721091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f4100d80 0x7f62f4103160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009910 tx=0x7f62e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:12.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 -- 172.21.15.57:0/1556721091 shutdown_connections 2024-02-28T23:11:12.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 --2- 172.21.15.57:0/1556721091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f41036a0 0x7f62f4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 --2- 172.21.15.57:0/1556721091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f4100d80 0x7f62f4103160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.830+0000 7f62fb000700 1 -- 172.21.15.57:0/1556721091 >> 172.21.15.57:0/1556721091 conn(0x7f62f40fa7a0 msgr2=0x7f62f40fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:12.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.831+0000 7f62fb000700 1 -- 172.21.15.57:0/1556721091 shutdown_connections 2024-02-28T23:11:12.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.831+0000 7f62fb000700 1 -- 172.21.15.57:0/1556721091 wait complete. 2024-02-28T23:11:12.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 Processor -- start 2024-02-28T23:11:12.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 -- start start 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f4100d80 0x7f62f4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4198b70 con 0x7f62f41036a0 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.832+0000 7f62fb000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4198cb0 con 0x7f62f4100d80 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39900/0 (socket says 172.21.15.57:39900) 2024-02-28T23:11:12.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 -- 172.21.15.57:0/2097673673 learned_addr learned my addr 172.21.15.57:0/2097673673 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:12.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3e00700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f4100d80 0x7f62f4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 -- 172.21.15.57:0/2097673673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f4100d80 msgr2=0x7f62f4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:12.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f4100d80 0x7f62f4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:12.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.833+0000 7f62f3400700 1 -- 172.21.15.57:0/2097673673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40095f0 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62f3400700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f62e000d8d0 tx=0x7f62e000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e00098a0 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62e0010460 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f419d760 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e000f5f0 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.834+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f419dcb0 con 0x7f62f41036a0 2024-02-28T23:11:12.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.835+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f4068ab0 con 0x7f62f41036a0 2024-02-28T23:11:12.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.836+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f62e0009a00 con 0x7f62f41036a0 2024-02-28T23:11:12.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.838+0000 7f62f0c00700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 0x7f62d806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:12.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.838+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f62e008ca90 con 0x7f62f41036a0 2024-02-28T23:11:12.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.838+0000 7f62f3e00700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 0x7f62d806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:12.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.839+0000 7f62f3e00700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 0x7f62d806f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006010 tx=0x7f62e400b560 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:12.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:12.842+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f62e005a320 con 0x7f62f41036a0 2024-02-28T23:11:13.023 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:12 smithi057 ceph-mon[29096]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:12 smithi084 ceph-mon[34826]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:13.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.099+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62f4100450 con 0x7f62d806d520 2024-02-28T23:11:13.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.101+0000 7f62f0c00700 1 -- 172.21.15.57:0/2097673673 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f62f4100450 con 0x7f62d806d520 2024-02-28T23:11:13.100 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:11:13.100 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:11:13.100 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:11:13.100 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/2 daemons upgraded", 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading grafana daemons", 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:11:13.101 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 msgr2=0x7f62d806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 0x7f62d806f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006010 tx=0x7f62e400b560 comp rx=0 tx=0).stop 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 msgr2=0x7f62f4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f62e000d8d0 tx=0x7f62e000dbe0 comp rx=0 tx=0).stop 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 shutdown_connections 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f62d806d520 0x7f62d806f9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62f4100d80 0x7f62f4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 --2- 172.21.15.57:0/2097673673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f41036a0 0x7f62f4198550 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.103+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 >> 172.21.15.57:0/2097673673 conn(0x7f62f40fa7a0 msgr2=0x7f62f40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.104+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 shutdown_connections 2024-02-28T23:11:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:13.104+0000 7f62fb000700 1 -- 172.21.15.57:0/2097673673 wait complete. 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/4082929040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:13.973 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.974 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:13.975 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-28T23:11:13.976 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-28T23:11:13.977 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:13.978 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:13.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:13.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:13 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/4082929040' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-28T23:11:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:14.085 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:13 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: Upgrade: Finalizing container_image settings 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: Upgrade: Complete! 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-28T23:11:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: Upgrade: Finalizing container_image settings 2024-02-28T23:11:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: Upgrade: Complete! 2024-02-28T23:11:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:11:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:11:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:11:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:11:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:15 smithi057 ceph-mon[29096]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:15 smithi084 ceph-mon[34826]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:17 smithi057 ceph-mon[29096]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:17 smithi084 ceph-mon[34826]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:18 smithi057 ceph-mon[29096]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:18 smithi084 ceph-mon[34826]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:21 smithi057 ceph-mon[29096]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:21.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:21 smithi084 ceph-mon[34826]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:23 smithi057 ceph-mon[29096]: pgmap v97: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:23 smithi084 ceph-mon[34826]: pgmap v97: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:25 smithi057 ceph-mon[29096]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:25 smithi084 ceph-mon[34826]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:27 smithi057 ceph-mon[29096]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:27 smithi084 ceph-mon[34826]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:29 smithi057 ceph-mon[29096]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:29 smithi084 ceph-mon[34826]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:31 smithi057 ceph-mon[29096]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:31 smithi084 ceph-mon[34826]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:33 smithi084 ceph-mon[34826]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:33 smithi057 ceph-mon[29096]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:35 smithi057 ceph-mon[29096]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:35 smithi084 ceph-mon[34826]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:37.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:37 smithi057 ceph-mon[29096]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:37 smithi084 ceph-mon[34826]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:39 smithi057 ceph-mon[29096]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:39.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:39 smithi084 ceph-mon[34826]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:41 smithi057 ceph-mon[29096]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:41.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:41 smithi084 ceph-mon[34826]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 -- 172.21.15.57:0/3481654477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8103460 msgr2=0x7fcec81038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 --2- 172.21.15.57:0/3481654477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8103460 0x7fcec81038b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009910 tx=0x7fceb4009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 -- 172.21.15.57:0/3481654477 shutdown_connections 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 --2- 172.21.15.57:0/3481654477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8103460 0x7fcec81038b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 --2- 172.21.15.57:0/3481654477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8102260 0x7fcec8102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 -- 172.21.15.57:0/3481654477 >> 172.21.15.57:0/3481654477 conn(0x7fcec80fd8d0 msgr2=0x7fcec80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 -- 172.21.15.57:0/3481654477 shutdown_connections 2024-02-28T23:11:43.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.283+0000 7fcecd000700 1 -- 172.21.15.57:0/3481654477 wait complete. 2024-02-28T23:11:43.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.284+0000 7fcecd000700 1 Processor -- start 2024-02-28T23:11:43.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.284+0000 7fcecd000700 1 -- start start 2024-02-28T23:11:43.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcecd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcecd000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 0x7fcec8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcecd000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80755f0 con 0x7fcec8102260 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcecd000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8075760 con 0x7fcec8103460 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcec7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcec7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37578/0 (socket says 172.21.15.57:37578) 2024-02-28T23:11:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.285+0000 7fcec7400700 1 -- 172.21.15.57:0/737644099 learned_addr learned my addr 172.21.15.57:0/737644099 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec7400700 1 -- 172.21.15.57:0/737644099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 msgr2=0x7fcec8079040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec6a00700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 0x7fcec8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec7400700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 0x7fcec8079040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec7400700 1 -- 172.21.15.57:0/737644099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb40095f0 con 0x7fcec8102260 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec6a00700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 0x7fcec8079040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:11:43.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcec7400700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b480 tx=0x7fceb800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb80118c0 con 0x7fcec8102260 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec8075a40 con 0x7fcec8102260 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.286+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceb8010380 con 0x7fcec8102260 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.287+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb800f4f0 con 0x7fcec8102260 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.287+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec8076010 con 0x7fcec8102260 2024-02-28T23:11:43.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.288+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec810ae50 con 0x7fcec8102260 2024-02-28T23:11:43.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.289+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fceb80104f0 con 0x7fcec8102260 2024-02-28T23:11:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.290+0000 7fcebfe00700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 0x7fceac06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.290+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fceb808cc70 con 0x7fcec8102260 2024-02-28T23:11:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.290+0000 7fcec6a00700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 0x7fceac06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.291+0000 7fcec6a00700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 0x7fceac06f9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb400b540 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:43.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.295+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fceb8093080 con 0x7fcec8102260 2024-02-28T23:11:43.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.541+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcec8107db0 con 0x7fceac06d520 2024-02-28T23:11:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.543+0000 7fcebfe00700 1 -- 172.21.15.57:0/737644099 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fcec8107db0 con 0x7fceac06d520 2024-02-28T23:11:43.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 msgr2=0x7fceac06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:43.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 0x7fceac06f9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb400b540 comp rx=0 tx=0).stop 2024-02-28T23:11:43.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 msgr2=0x7fcec8078b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:43.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b480 tx=0x7fceb800b840 comp rx=0 tx=0).stop 2024-02-28T23:11:43.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 shutdown_connections 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fceac06d520 0x7fceac06f9d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcec8102260 0x7fcec8078b00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.546+0000 7fcecd000700 1 --2- 172.21.15.57:0/737644099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcec8103460 0x7fcec8079040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.547+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 >> 172.21.15.57:0/737644099 conn(0x7fcec80fd8d0 msgr2=0x7fcec8106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.547+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 shutdown_connections 2024-02-28T23:11:43.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:43.547+0000 7fcecd000700 1 -- 172.21.15.57:0/737644099 wait complete. 2024-02-28T23:11:43.805 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:43 smithi057 ceph-mon[29096]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:43.805 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:43 smithi057 ceph-mon[29096]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:11:43.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:43 smithi084 ceph-mon[34826]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:43.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:43 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:11:44.191 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-02-28T23:11:44.677 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:11:44.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:44 smithi057 ceph-mon[29096]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:44.868 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:44 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:44 smithi084 ceph-mon[34826]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:11:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:44 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:11:45.880 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:45 smithi057 ceph-mon[29096]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:45 smithi084 ceph-mon[34826]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 -- 172.21.15.57:0/346305247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 msgr2=0x7ff54c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 --2- 172.21.15.57:0/346305247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c106e40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff538009910 tx=0x7ff538009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 -- 172.21.15.57:0/346305247 shutdown_connections 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 --2- 172.21.15.57:0/346305247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c106e40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 --2- 172.21.15.57:0/346305247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 0x7ff54c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:46.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 -- 172.21.15.57:0/346305247 >> 172.21.15.57:0/346305247 conn(0x7ff54c0fdd90 msgr2=0x7ff54c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:46.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 -- 172.21.15.57:0/346305247 shutdown_connections 2024-02-28T23:11:46.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.783+0000 7ff552c00700 1 -- 172.21.15.57:0/346305247 wait complete. 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.784+0000 7ff552c00700 1 Processor -- start 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff552c00700 1 -- start start 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff552c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 0x7ff54c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff552c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff552c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1989a0 con 0x7ff54c102e70 2024-02-28T23:11:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff552c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c198ae0 con 0x7ff54c1024d0 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff550e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff550e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35856/0 (socket says 172.21.15.57:35856) 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.785+0000 7ff550e00700 1 -- 172.21.15.57:0/913984130 learned_addr learned my addr 172.21.15.57:0/913984130 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff551800700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 0x7ff54c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff550e00700 1 -- 172.21.15.57:0/913984130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 msgr2=0x7ff54c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff550e00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 0x7ff54c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff550e00700 1 -- 172.21.15.57:0/913984130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380095f0 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff550e00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff5380092e0 tx=0x7ff53800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.786+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53801d070 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.787+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c19c8d0 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.787+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff53800f460 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.787+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5380175f0 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.787+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c19cdc0 con 0x7ff54c102e70 2024-02-28T23:11:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.788+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c10a840 con 0x7ff54c102e70 2024-02-28T23:11:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.789+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7ff53800f5d0 con 0x7ff54c102e70 2024-02-28T23:11:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.790+0000 7ff54a000700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 0x7ff53406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.790+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff53808e580 con 0x7ff54c102e70 2024-02-28T23:11:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.790+0000 7ff551800700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 0x7ff53406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.791+0000 7ff551800700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 0x7ff53406f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff540005ea0 tx=0x7ff540005e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:46.795+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ff5380581c0 con 0x7ff54c102e70 2024-02-28T23:11:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.118+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff54c062380 con 0x7ff54c102e70 2024-02-28T23:11:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.119+0000 7ff54a000700 1 -- 172.21.15.57:0/913984130 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff53805b880 con 0x7ff54c102e70 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 msgr2=0x7ff53406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 0x7ff53406f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff540005ea0 tx=0x7ff540005e30 comp rx=0 tx=0).stop 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 msgr2=0x7ff54c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff5380092e0 tx=0x7ff53800be20 comp rx=0 tx=0).stop 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 shutdown_connections 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff53406d520 0x7ff53406f9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff54c1024d0 0x7ff54c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 --2- 172.21.15.57:0/913984130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff54c102e70 0x7ff54c198350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 >> 172.21.15.57:0/913984130 conn(0x7ff54c0fdd90 msgr2=0x7ff54c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 shutdown_connections 2024-02-28T23:11:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:47.122+0000 7ff552c00700 1 -- 172.21.15.57:0/913984130 wait complete. 2024-02-28T23:11:47.137 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:11:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:47 smithi057 ceph-mon[29096]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:47 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/913984130' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:47.946 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-02-28T23:11:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:47 smithi084 ceph-mon[34826]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:47 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/913984130' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:11:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:49 smithi057 ceph-mon[29096]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:49 smithi084 ceph-mon[34826]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 -- 172.21.15.57:0/2981630409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90102470 msgr2=0x7efc901028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 --2- 172.21.15.57:0/2981630409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90102470 0x7efc901028e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009910 tx=0x7efc7c009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 -- 172.21.15.57:0/2981630409 shutdown_connections 2024-02-28T23:11:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 --2- 172.21.15.57:0/2981630409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90102470 0x7efc901028e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 --2- 172.21.15.57:0/2981630409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90108470 0x7efc90108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:50.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.920+0000 7efc96e00700 1 -- 172.21.15.57:0/2981630409 >> 172.21.15.57:0/2981630409 conn(0x7efc900fdf90 msgr2=0x7efc901003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:50.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.921+0000 7efc96e00700 1 -- 172.21.15.57:0/2981630409 shutdown_connections 2024-02-28T23:11:50.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.921+0000 7efc96e00700 1 -- 172.21.15.57:0/2981630409 wait complete. 2024-02-28T23:11:50.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.922+0000 7efc96e00700 1 Processor -- start 2024-02-28T23:11:50.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.922+0000 7efc96e00700 1 -- start start 2024-02-28T23:11:50.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc96e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90102470 0x7efc90198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:50.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc96e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:50.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc96e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90198db0 con 0x7efc90108470 2024-02-28T23:11:50.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc96e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9019cb40 con 0x7efc90102470 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc8f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc8f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35878/0 (socket says 172.21.15.57:35878) 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc8f400700 1 -- 172.21.15.57:0/1271606102 learned_addr learned my addr 172.21.15.57:0/1271606102 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.923+0000 7efc8fe00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90102470 0x7efc90198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8f400700 1 -- 172.21.15.57:0/1271606102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90102470 msgr2=0x7efc90198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8f400700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90102470 0x7efc90198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8f400700 1 -- 172.21.15.57:0/1271606102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c0095f0 con 0x7efc90108470 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8f400700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b5c0 tx=0x7efc7c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc7c01d070 con 0x7efc90108470 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc7c00f460 con 0x7efc90108470 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9019cdc0 con 0x7efc90108470 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc7c017610 con 0x7efc90108470 2024-02-28T23:11:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.924+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9019d2b0 con 0x7efc90108470 2024-02-28T23:11:50.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.926+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9010aa00 con 0x7efc90108470 2024-02-28T23:11:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.927+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7efc7c00f5d0 con 0x7efc90108470 2024-02-28T23:11:50.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.928+0000 7efc8cc00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 0x7efc7406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:50.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.928+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efc7c08e590 con 0x7efc90108470 2024-02-28T23:11:50.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.928+0000 7efc8fe00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 0x7efc7406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:50.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.929+0000 7efc8fe00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 0x7efc7406f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc80006fd0 tx=0x7efc80008040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:50.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:50.933+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7efc7c058250 con 0x7efc90108470 2024-02-28T23:11:51.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.255+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efc9004f2a0 con 0x7efc90108470 2024-02-28T23:11:51.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.257+0000 7efc8cc00700 1 -- 172.21.15.57:0/1271606102 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7efc7c05b910 con 0x7efc90108470 2024-02-28T23:11:51.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 msgr2=0x7efc7406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:51.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 0x7efc7406f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc80006fd0 tx=0x7efc80008040 comp rx=0 tx=0).stop 2024-02-28T23:11:51.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 msgr2=0x7efc901986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:51.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b5c0 tx=0x7efc7c00be00 comp rx=0 tx=0).stop 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 shutdown_connections 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7efc7406d520 0x7efc7406f9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc90102470 0x7efc90198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 --2- 172.21.15.57:0/1271606102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc90108470 0x7efc901986d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.260+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 >> 172.21.15.57:0/1271606102 conn(0x7efc900fdf90 msgr2=0x7efc900feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.261+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 shutdown_connections 2024-02-28T23:11:51.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:51.261+0000 7efc96e00700 1 -- 172.21.15.57:0/1271606102 wait complete. 2024-02-28T23:11:51.275 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)" 2024-02-28T23:11:52.054 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:51 smithi057 ceph-mon[29096]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:52.054 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:51 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1271606102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:51 smithi084 ceph-mon[34826]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:51 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1271606102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:52.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-02-28T23:11:52.562 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:11:53.757 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:53 smithi057 ceph-mon[29096]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:53.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:53 smithi084 ceph-mon[34826]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:54.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.626+0000 7ff8ff600700 1 -- 172.21.15.57:0/3743562157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075280 msgr2=0x7ff8f8075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.626+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3743562157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075280 0x7ff8f8075650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009910 tx=0x7ff8ec009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 -- 172.21.15.57:0/3743562157 shutdown_connections 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3743562157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075b90 0x7ff8f8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3743562157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075280 0x7ff8f8075650 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 -- 172.21.15.57:0/3743562157 >> 172.21.15.57:0/3743562157 conn(0x7ff8f80fdf90 msgr2=0x7ff8f81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 -- 172.21.15.57:0/3743562157 shutdown_connections 2024-02-28T23:11:54.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.627+0000 7ff8ff600700 1 -- 172.21.15.57:0/3743562157 wait complete. 2024-02-28T23:11:54.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.628+0000 7ff8ff600700 1 Processor -- start 2024-02-28T23:11:54.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.628+0000 7ff8ff600700 1 -- start start 2024-02-28T23:11:54.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.629+0000 7ff8ff600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 0x7ff8f819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:54.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.629+0000 7ff8ff600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 0x7ff8f819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:54.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.629+0000 7ff8ff600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f819d050 con 0x7ff8f8075b90 2024-02-28T23:11:54.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.629+0000 7ff8ff600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f81a0de0 con 0x7ff8f8075280 2024-02-28T23:11:54.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.629+0000 7ff8fcc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 0x7ff8f819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:54.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8fcc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 0x7ff8f819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:38760/0 (socket says 172.21.15.57:38760) 2024-02-28T23:11:54.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8fcc00700 1 -- 172.21.15.57:0/3728591354 learned_addr learned my addr 172.21.15.57:0/3728591354 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:54.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8f7e00700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 0x7ff8f819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:54.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8f7e00700 1 -- 172.21.15.57:0/3728591354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 msgr2=0x7ff8f819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:54.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8f7e00700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 0x7ff8f819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8f7e00700 1 -- 172.21.15.57:0/3728591354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0095f0 con 0x7ff8f8075b90 2024-02-28T23:11:54.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.630+0000 7ff8f7e00700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 0x7ff8f819c970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d8d0 tx=0x7ff8e400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.631+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e400f860 con 0x7ff8f8075b90 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.631+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e400fe80 con 0x7ff8f8075b90 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.631+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e4010ea0 con 0x7ff8f8075b90 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.631+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f81a10c0 con 0x7ff8f8075b90 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.631+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f81a1610 con 0x7ff8f8075b90 2024-02-28T23:11:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.632+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f810e340 con 0x7ff8f8075b90 2024-02-28T23:11:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.633+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7ff8e4010460 con 0x7ff8f8075b90 2024-02-28T23:11:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.634+0000 7ff8f5600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 0x7ff8e006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.634+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff8e408cbb0 con 0x7ff8f8075b90 2024-02-28T23:11:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.635+0000 7ff8fcc00700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 0x7ff8e006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.635+0000 7ff8fcc00700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 0x7ff8e006f9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b850 tx=0x7ff8ec005ad0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:54.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.639+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ff8e4056870 con 0x7ff8f8075b90 2024-02-28T23:11:54.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.972+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8f804f2a0 con 0x7ff8f8075b90 2024-02-28T23:11:54.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.973+0000 7ff8f5600700 1 -- 172.21.15.57:0/3728591354 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff8e4059f30 con 0x7ff8f8075b90 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.976+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 msgr2=0x7ff8e006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.976+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 0x7ff8e006f9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b850 tx=0x7ff8ec005ad0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.976+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 msgr2=0x7ff8f819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.976+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 0x7ff8f819c970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d8d0 tx=0x7ff8e400dbe0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 shutdown_connections 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7ff8e006d520 0x7ff8e006f9d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f8075280 0x7ff8f819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 --2- 172.21.15.57:0/3728591354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8f8075b90 0x7ff8f819c970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:54.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 >> 172.21.15.57:0/3728591354 conn(0x7ff8f80fdf90 msgr2=0x7ff8f80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:54.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 shutdown_connections 2024-02-28T23:11:54.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:54.977+0000 7ff8ff600700 1 -- 172.21.15.57:0/3728591354 wait complete. 2024-02-28T23:11:54.992 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:11:55.765 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-02-28T23:11:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:55 smithi057 ceph-mon[29096]: pgmap v113: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:55 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3728591354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:55 smithi084 ceph-mon[34826]: pgmap v113: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:11:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:55 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3728591354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:11:56.204 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:11:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:57 smithi057 ceph-mon[29096]: pgmap v114: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:57 smithi084 ceph-mon[34826]: pgmap v114: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.392+0000 7f4f9f200700 1 -- 172.21.15.57:0/3980345095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98108460 msgr2=0x7f4f98108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3980345095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98108460 0x7f4f98108830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009910 tx=0x7f4f88009c20 comp rx=0 tx=0).stop 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 -- 172.21.15.57:0/3980345095 shutdown_connections 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3980345095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98102460 0x7f4f981028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3980345095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98108460 0x7f4f98108830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 -- 172.21.15.57:0/3980345095 >> 172.21.15.57:0/3980345095 conn(0x7f4f980fdf40 msgr2=0x7f4f98100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 -- 172.21.15.57:0/3980345095 shutdown_connections 2024-02-28T23:11:58.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.393+0000 7f4f9f200700 1 -- 172.21.15.57:0/3980345095 wait complete. 2024-02-28T23:11:58.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.394+0000 7f4f9f200700 1 Processor -- start 2024-02-28T23:11:58.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.394+0000 7f4f9f200700 1 -- start start 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f9f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f9f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 0x7f4f98198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f9f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98198e60 con 0x7f4f98102460 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f9f200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9819cbf0 con 0x7f4f98108460 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37502/0 (socket says 172.21.15.57:37502) 2024-02-28T23:11:58.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.395+0000 7f4f97e00700 1 -- 172.21.15.57:0/3576973070 learned_addr learned my addr 172.21.15.57:0/3576973070 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97400700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 0x7f4f98198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97e00700 1 -- 172.21.15.57:0/3576973070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 msgr2=0x7f4f98198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97e00700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 0x7f4f98198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97e00700 1 -- 172.21.15.57:0/3576973070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880095f0 con 0x7f4f98102460 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97400700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 0x7f4f98198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:11:58.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f97e00700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b5c0 tx=0x7f4f8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.396+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8801d070 con 0x7f4f98102460 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.397+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f88005e00 con 0x7f4f98102460 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.397+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9819ce70 con 0x7f4f98102460 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.397+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f88017400 con 0x7f4f98102460 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.397+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9819d360 con 0x7f4f98102460 2024-02-28T23:11:58.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.398+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9810a9d0 con 0x7f4f98102460 2024-02-28T23:11:58.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.399+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f4f8800f460 con 0x7f4f98102460 2024-02-28T23:11:58.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.400+0000 7f4f94c00700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 0x7f4f7c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:11:58.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.401+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4f8808e510 con 0x7f4f98102460 2024-02-28T23:11:58.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.401+0000 7f4f97400700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 0x7f4f7c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:11:58.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.402+0000 7f4f97400700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 0x7f4f7c06f9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005fd0 tx=0x7f4f84005ee0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:11:58.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.405+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f4f88058150 con 0x7f4f98102460 2024-02-28T23:11:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:11:58.656+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f9819d580 con 0x7f4f7c06d520 2024-02-28T23:12:00.065 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:59 smithi057 ceph-mon[29096]: pgmap v115: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:00.065 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:11:59 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:59 smithi084 ceph-mon[34826]: pgmap v115: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:11:59 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:00 smithi057 ceph-mon[29096]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:00 smithi084 ceph-mon[34826]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.543+0000 7f4f94c00700 1 -- 172.21.15.57:0/3576973070 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f4f9819d580 con 0x7f4f7c06d520 2024-02-28T23:12:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 msgr2=0x7f4f7c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 0x7f4f7c06f9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005fd0 tx=0x7f4f84005ee0 comp rx=0 tx=0).stop 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 msgr2=0x7f4f98198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b5c0 tx=0x7f4f8800be20 comp rx=0 tx=0).stop 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 shutdown_connections 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f4f7c06d520 0x7f4f7c06f9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f98102460 0x7f4f98198240 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 --2- 172.21.15.57:0/3576973070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f98108460 0x7f4f98198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.547+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 >> 172.21.15.57:0/3576973070 conn(0x7f4f980fdf40 msgr2=0x7f4f98100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.548+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 shutdown_connections 2024-02-28T23:12:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:01.548+0000 7f4f9f200700 1 -- 172.21.15.57:0/3576973070 wait complete. 2024-02-28T23:12:01.564 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:12:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:01 smithi057 ceph-mon[29096]: pgmap v116: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:01.981 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:01 smithi084 ceph-mon[34826]: pgmap v116: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:02.326 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch ps' 2024-02-28T23:12:02.766 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:02.923 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:02 smithi057 ceph-mon[29096]: pgmap v117: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:02 smithi084 ceph-mon[34826]: pgmap v117: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:05.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.052+0000 7f7aace00700 1 -- 172.21.15.57:0/3732112104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 msgr2=0x7f7aa81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:05.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.052+0000 7f7aace00700 1 --2- 172.21.15.57:0/3732112104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81028e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009910 tx=0x7f7a90009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:05.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 -- 172.21.15.57:0/3732112104 shutdown_connections 2024-02-28T23:12:05.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 --2- 172.21.15.57:0/3732112104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81028e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 --2- 172.21.15.57:0/3732112104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 0x7f7aa8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 -- 172.21.15.57:0/3732112104 >> 172.21.15.57:0/3732112104 conn(0x7f7aa80fdf90 msgr2=0x7f7aa81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:05.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 -- 172.21.15.57:0/3732112104 shutdown_connections 2024-02-28T23:12:05.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.053+0000 7f7aace00700 1 -- 172.21.15.57:0/3732112104 wait complete. 2024-02-28T23:12:05.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 Processor -- start 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 -- start start 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 0x7f7aa8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8198e10 con 0x7f7aa8102470 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.054+0000 7f7aace00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa819cba0 con 0x7f7aa8108470 2024-02-28T23:12:05.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55126/0 (socket says 172.21.15.57:55126) 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 -- 172.21.15.57:0/1795989091 learned_addr learned my addr 172.21.15.57:0/1795989091 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7a9fe00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 0x7f7aa8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 -- 172.21.15.57:0/1795989091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 msgr2=0x7f7aa8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 0x7f7aa8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.055+0000 7f7aa5e00700 1 -- 172.21.15.57:0/1795989091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900095f0 con 0x7f7aa8102470 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7aa5e00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c8a0 tx=0x7f7a9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:05.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a9800cea0 con 0x7f7aa8102470 2024-02-28T23:12:05.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a98004830 con 0x7f7aa8102470 2024-02-28T23:12:05.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a980055b0 con 0x7f7aa8102470 2024-02-28T23:12:05.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa819ce80 con 0x7f7aa8102470 2024-02-28T23:12:05.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.056+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa819d3d0 con 0x7f7aa8102470 2024-02-28T23:12:05.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.058+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa810a970 con 0x7f7aa8102470 2024-02-28T23:12:05.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.059+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f7a980049a0 con 0x7f7aa8102470 2024-02-28T23:12:05.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.059+0000 7f7a9ea00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 0x7f7a8c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:05.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.060+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7a9808cbd0 con 0x7f7aa8102470 2024-02-28T23:12:05.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.060+0000 7f7a9fe00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 0x7f7a8c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:05.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.061+0000 7f7a9fe00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 0x7f7a8c06f9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8199810 tx=0x7f7a9001a040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:05.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.064+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7a98056890 con 0x7f7aa8102470 2024-02-28T23:12:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.302+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aa8199690 con 0x7f7a8c06d520 2024-02-28T23:12:05.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.315+0000 7f7a9ea00700 1 -- 172.21.15.57:0/1795989091 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7a900056c0 con 0x7f7a8c06d520 2024-02-28T23:12:05.314 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:12:05.314 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (104s) 58s ago 15m 13.9M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:12:05.314 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (11m) 58s ago 15m 8144k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (14m) 2m ago 14m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (15m) 58s ago 15m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (14m) 2m ago 14m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (66s) 58s ago 15m 50.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (8m) 58s ago 8m 57.1M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (8m) 58s ago 8m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (8m) 2m ago 8m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (8m) 2m ago 8m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (4m) 58s ago 17m 501M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (3m) 2m ago 14m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (17m) 58s ago 17m 54.0M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (14m) 2m ago 14m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (2m) 58s ago 15m 7633k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (2m) 2m ago 14m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (13m) 58s ago 13m 227M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:12:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (12m) 58s ago 12m 240M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:12:05.316 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (11m) 58s ago 11m 201M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:12:05.316 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (11m) 2m ago 11m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:12:05.316 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (10m) 2m ago 10m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:12:05.316 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (10m) 2m ago 10m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:12:05.316 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (2m) 58s ago 14m 47.7M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:12:05.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 msgr2=0x7f7a8c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:05.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 0x7f7a8c06f9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8199810 tx=0x7f7a9001a040 comp rx=0 tx=0).stop 2024-02-28T23:12:05.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 msgr2=0x7f7aa81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:05.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c8a0 tx=0x7f7a9800cc60 comp rx=0 tx=0).stop 2024-02-28T23:12:05.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 shutdown_connections 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f7a8c06d520 0x7f7a8c06f9d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7aa8102470 0x7f7aa81981f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 --2- 172.21.15.57:0/1795989091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa8108470 0x7f7aa8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 >> 172.21.15.57:0/1795989091 conn(0x7f7aa80fdf90 msgr2=0x7f7aa80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 shutdown_connections 2024-02-28T23:12:05.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:05.319+0000 7f7aace00700 1 -- 172.21.15.57:0/1795989091 wait complete. 2024-02-28T23:12:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:05 smithi057 ceph-mon[29096]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:05.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:05 smithi084 ceph-mon[34826]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:06.091 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-28T23:12:06.092 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:12:06.092 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-02-28T23:12:06.527 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:06.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:06 smithi057 ceph-mon[29096]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:06.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:06 smithi084 ceph-mon[34826]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:07.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:07 smithi057 ceph-mon[29096]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:07 smithi084 ceph-mon[34826]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 -- 172.21.15.57:0/3678166127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 msgr2=0x7f3db4106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 --2- 172.21.15.57:0/3678166127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4106e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009910 tx=0x7f3da0009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 -- 172.21.15.57:0/3678166127 shutdown_connections 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 --2- 172.21.15.57:0/3678166127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4106e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 --2- 172.21.15.57:0/3678166127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 -- 172.21.15.57:0/3678166127 >> 172.21.15.57:0/3678166127 conn(0x7f3db40fdd20 msgr2=0x7f3db4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 -- 172.21.15.57:0/3678166127 shutdown_connections 2024-02-28T23:12:08.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.915+0000 7f3db9a00700 1 -- 172.21.15.57:0/3678166127 wait complete. 2024-02-28T23:12:08.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.916+0000 7f3db9a00700 1 Processor -- start 2024-02-28T23:12:08.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db9a00700 1 -- start start 2024-02-28T23:12:08.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db9a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:08.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:08.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db9a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db41989e0 con 0x7f3db4102e60 2024-02-28T23:12:08.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db9a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4198b20 con 0x7f3db41024c0 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55132/0 (socket says 172.21.15.57:55132) 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.917+0000 7f3db3400700 1 -- 172.21.15.57:0/979834987 learned_addr learned my addr 172.21.15.57:0/979834987 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3e00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3400700 1 -- 172.21.15.57:0/979834987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 msgr2=0x7f3db4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3400700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3400700 1 -- 172.21.15.57:0/979834987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00095f0 con 0x7f3db4102e60 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3e00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db3400700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005f50 tx=0x7f3da000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.918+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da001d070 con 0x7f3db4102e60 2024-02-28T23:12:08.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.919+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3da000f460 con 0x7f3db4102e60 2024-02-28T23:12:08.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.919+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db419c910 con 0x7f3db4102e60 2024-02-28T23:12:08.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.919+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da0017630 con 0x7f3db4102e60 2024-02-28T23:12:08.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.919+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db419ce00 con 0x7f3db4102e60 2024-02-28T23:12:08.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.920+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db410a830 con 0x7f3db4102e60 2024-02-28T23:12:08.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.921+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f3da000f5d0 con 0x7f3db4102e60 2024-02-28T23:12:08.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.922+0000 7f3db0c00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 0x7f3d9806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:08.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.922+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3da008e510 con 0x7f3db4102e60 2024-02-28T23:12:08.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.922+0000 7f3db3e00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 0x7f3d9806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:08.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.923+0000 7f3db3e00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 0x7f3d9806f9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005fd0 tx=0x7f3da4005e20 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:08.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:08.927+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3da00581d0 con 0x7f3db4102e60 2024-02-28T23:12:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.158+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f3db4062380 con 0x7f3db4102e60 2024-02-28T23:12:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.169+0000 7f3db0c00700 1 -- 172.21.15.57:0/979834987 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3da005b890 con 0x7f3db4102e60 2024-02-28T23:12:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 msgr2=0x7f3d9806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 0x7f3d9806f9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005fd0 tx=0x7f3da4005e20 comp rx=0 tx=0).stop 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 msgr2=0x7f3db4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005f50 tx=0x7f3da000be20 comp rx=0 tx=0).stop 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 shutdown_connections 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f3d9806d520 0x7f3d9806f9d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3db41024c0 0x7f3db4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 --2- 172.21.15.57:0/979834987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3db4102e60 0x7f3db4198390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:09.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.172+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 >> 172.21.15.57:0/979834987 conn(0x7f3db40fdd20 msgr2=0x7f3db41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:09.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.173+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 shutdown_connections 2024-02-28T23:12:09.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:09.173+0000 7f3db9a00700 1 -- 172.21.15.57:0/979834987 wait complete. 2024-02-28T23:12:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:09 smithi057 ceph-mon[29096]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:09 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/979834987' entity='client.admin' 2024-02-28T23:12:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:09 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:09 smithi084 ceph-mon[34826]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:09 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/979834987' entity='client.admin' 2024-02-28T23:12:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:09 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:10.096 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-28T23:12:10.097 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:12:10.097 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-28T23:12:10.541 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:11 smithi057 ceph-mon[29096]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:12:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:11 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:11 smithi084 ceph-mon[34826]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:12:11.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:11 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.731+0000 7f9b20e00700 1 -- 172.21.15.57:0/1826022451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 msgr2=0x7f9b1c110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.731+0000 7f9b20e00700 1 --2- 172.21.15.57:0/1826022451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c110b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009910 tx=0x7f9b08009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 -- 172.21.15.57:0/1826022451 shutdown_connections 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 --2- 172.21.15.57:0/1826022451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c110b40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 --2- 172.21.15.57:0/1826022451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 -- 172.21.15.57:0/1826022451 >> 172.21.15.57:0/1826022451 conn(0x7f9b1c0fbbb0 msgr2=0x7f9b1c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 -- 172.21.15.57:0/1826022451 shutdown_connections 2024-02-28T23:12:12.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.732+0000 7f9b20e00700 1 -- 172.21.15.57:0/1826022451 wait complete. 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.733+0000 7f9b20e00700 1 Processor -- start 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.733+0000 7f9b20e00700 1 -- start start 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b20e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b20e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b20e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c19ce30 con 0x7f9b1c073a00 2024-02-28T23:12:12.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b20e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1a0bc0 con 0x7f9b1c0730f0 2024-02-28T23:12:12.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b1aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:12.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b1aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55158/0 (socket says 172.21.15.57:55158) 2024-02-28T23:12:12.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.734+0000 7f9b1aa00700 1 -- 172.21.15.57:0/191425497 learned_addr learned my addr 172.21.15.57:0/191425497 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:12.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1b400700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1aa00700 1 -- 172.21.15.57:0/191425497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 msgr2=0x7f9b1c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1aa00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c19c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1aa00700 1 -- 172.21.15.57:0/191425497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080095f0 con 0x7f9b1c073a00 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1b400700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b1aa00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b5c0 tx=0x7f9b0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0801d070 con 0x7f9b1c073a00 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b08005e00 con 0x7f9b1c073a00 2024-02-28T23:12:12.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c1a0ea0 con 0x7f9b1c073a00 2024-02-28T23:12:12.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.735+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b08017400 con 0x7f9b1c073a00 2024-02-28T23:12:12.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.736+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c1a13f0 con 0x7f9b1c073a00 2024-02-28T23:12:12.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.737+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c10e2c0 con 0x7f9b1c073a00 2024-02-28T23:12:12.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.738+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f9b0800f460 con 0x7f9b1c073a00 2024-02-28T23:12:12.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.739+0000 7f9b13e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 0x7f9b0006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:12.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.739+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9b0808e4d0 con 0x7f9b1c073a00 2024-02-28T23:12:12.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.739+0000 7f9b1b400700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 0x7f9b0006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:12.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.740+0000 7f9b1b400700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 0x7f9b0006f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005d90 tx=0x7f9b0c005d00 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:12.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.745+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9b08058110 con 0x7f9b1c073a00 2024-02-28T23:12:12.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.977+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f9b1c062380 con 0x7f9b1c073a00 2024-02-28T23:12:12.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.978+0000 7f9b13e00700 1 -- 172.21.15.57:0/191425497 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f9b0805b7d0 con 0x7f9b1c073a00 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 msgr2=0x7f9b0006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 0x7f9b0006f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005d90 tx=0x7f9b0c005d00 comp rx=0 tx=0).stop 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 msgr2=0x7f9b1c19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b5c0 tx=0x7f9b0800be20 comp rx=0 tx=0).stop 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 shutdown_connections 2024-02-28T23:12:12.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9b0006d520 0x7f9b0006f9d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1c0730f0 0x7f9b1c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 --2- 172.21.15.57:0/191425497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b1c073a00 0x7f9b1c19c750 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:12.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.981+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 >> 172.21.15.57:0/191425497 conn(0x7f9b1c0fbbb0 msgr2=0x7f9b1c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:12.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.982+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 shutdown_connections 2024-02-28T23:12:12.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:12.982+0000 7f9b20e00700 1 -- 172.21.15.57:0/191425497 wait complete. 2024-02-28T23:12:13.694 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:13 smithi057 ceph-mon[29096]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:13 smithi084 ceph-mon[34826]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:13.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-28T23:12:14.271 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:14.774 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:14 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:14 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:15 smithi057 ceph-mon[29096]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:15.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:15 smithi084 ceph-mon[34826]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 -- 172.21.15.57:0/880443901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 msgr2=0x7f0594106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 --2- 172.21.15.57:0/880443901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f0594106ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0580009910 tx=0x7f0580009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 -- 172.21.15.57:0/880443901 shutdown_connections 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 --2- 172.21.15.57:0/880443901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f0594106ff0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 --2- 172.21.15.57:0/880443901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 0x7f059410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 -- 172.21.15.57:0/880443901 >> 172.21.15.57:0/880443901 conn(0x7f0594075e90 msgr2=0x7f05940782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 -- 172.21.15.57:0/880443901 shutdown_connections 2024-02-28T23:12:16.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.501+0000 7f0599c00700 1 -- 172.21.15.57:0/880443901 wait complete. 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.503+0000 7f0599c00700 1 Processor -- start 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.503+0000 7f0599c00700 1 -- start start 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0599c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0599c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 0x7f059419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0599c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059419ce00 con 0x7f0594106b80 2024-02-28T23:12:16.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0599c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05941a0b90 con 0x7f059410c660 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0593e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0593e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54404/0 (socket says 172.21.15.57:54404) 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0593e00700 1 -- 172.21.15.57:0/2101412177 learned_addr learned my addr 172.21.15.57:0/2101412177 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.504+0000 7f0593400700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 0x7f059419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0593e00700 1 -- 172.21.15.57:0/2101412177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 msgr2=0x7f059419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0593e00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 0x7f059419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0593e00700 1 -- 172.21.15.57:0/2101412177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800095f0 con 0x7f0594106b80 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0593e00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f058400c8a0 tx=0x7f058400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058400cea0 con 0x7f0594106b80 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0584004830 con 0x7f0594106b80 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05941a0e70 con 0x7f0594106b80 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.505+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05840055b0 con 0x7f0594106b80 2024-02-28T23:12:16.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.506+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05941a13c0 con 0x7f0594106b80 2024-02-28T23:12:16.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.507+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059410eb40 con 0x7f0594106b80 2024-02-28T23:12:16.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.508+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f05840049a0 con 0x7f0594106b80 2024-02-28T23:12:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.509+0000 7f0590c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 0x7f057806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.509+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f058408cca0 con 0x7f0594106b80 2024-02-28T23:12:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.510+0000 7f0593400700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 0x7f057806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.510+0000 7f0593400700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 0x7f057806f9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f059419d800 tx=0x7f058001a040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.514+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f05840568e0 con 0x7f0594106b80 2024-02-28T23:12:16.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.743+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f05941a1640 con 0x7f0594106b80 2024-02-28T23:12:16.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.744+0000 7f0590c00700 1 -- 172.21.15.57:0/2101412177 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0584004c50 con 0x7f0594106b80 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 msgr2=0x7f057806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 0x7f057806f9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f059419d800 tx=0x7f058001a040 comp rx=0 tx=0).stop 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 msgr2=0x7f059419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f058400c8a0 tx=0x7f058400cc60 comp rx=0 tx=0).stop 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 shutdown_connections 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f057806d520 0x7f057806f9d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0594106b80 0x7f059419c1e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 --2- 172.21.15.57:0/2101412177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059410c660 0x7f059419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.747+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 >> 172.21.15.57:0/2101412177 conn(0x7f0594075e90 msgr2=0x7f05940778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.748+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 shutdown_connections 2024-02-28T23:12:16.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:16.748+0000 7f0599c00700 1 -- 172.21.15.57:0/2101412177 wait complete. 2024-02-28T23:12:17.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-28T23:12:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:17 smithi057 ceph-mon[29096]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:17 smithi084 ceph-mon[34826]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:19.803 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:19 smithi057 ceph-mon[29096]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:19 smithi084 ceph-mon[34826]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:20.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.345+0000 7fece9e00700 1 -- 172.21.15.57:0/216364976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece4100e80 msgr2=0x7fece4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.345+0000 7fece9e00700 1 --2- 172.21.15.57:0/216364976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece4100e80 0x7fece4104e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009910 tx=0x7fecd0009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 -- 172.21.15.57:0/216364976 shutdown_connections 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 --2- 172.21.15.57:0/216364976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece4100e80 0x7fece4104e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 --2- 172.21.15.57:0/216364976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece41004e0 0x7fece41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 -- 172.21.15.57:0/216364976 >> 172.21.15.57:0/216364976 conn(0x7fece40fbda0 msgr2=0x7fece40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 -- 172.21.15.57:0/216364976 shutdown_connections 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.346+0000 7fece9e00700 1 -- 172.21.15.57:0/216364976 wait complete. 2024-02-28T23:12:20.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.347+0000 7fece9e00700 1 Processor -- start 2024-02-28T23:12:20.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.347+0000 7fece9e00700 1 -- start start 2024-02-28T23:12:20.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:20.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece9e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4100e80 0x7fece4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece9e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4198ca0 con 0x7fece41004e0 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece9e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4198de0 con 0x7fece4100e80 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54428/0 (socket says 172.21.15.57:54428) 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece2a00700 1 -- 172.21.15.57:0/1592069570 learned_addr learned my addr 172.21.15.57:0/1592069570 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.348+0000 7fece2000700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4100e80 0x7fece4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece2a00700 1 -- 172.21.15.57:0/1592069570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4100e80 msgr2=0x7fece4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece2a00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4100e80 0x7fece4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece2a00700 1 -- 172.21.15.57:0/1592069570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00095f0 con 0x7fece41004e0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece2a00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c930 tx=0x7fecd400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd4007ad0 con 0x7fece41004e0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecd4007c30 con 0x7fece41004e0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece419cc30 con 0x7fece41004e0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd4018610 con 0x7fece41004e0 2024-02-28T23:12:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.349+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece419d180 con 0x7fece41004e0 2024-02-28T23:12:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.351+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece41045d0 con 0x7fece41004e0 2024-02-28T23:12:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.352+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fecd400f450 con 0x7fece41004e0 2024-02-28T23:12:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.353+0000 7fece8a00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.353+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fecd408d230 con 0x7fece41004e0 2024-02-28T23:12:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.353+0000 7fece2000700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.354+0000 7fece2000700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fece4199730 tx=0x7fecd001a040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.358+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fecd4056e70 con 0x7fece41004e0 2024-02-28T23:12:20.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.597+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fece404f2a0 con 0x7fece41004e0 2024-02-28T23:12:20.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.598+0000 7fece8a00700 1 -- 172.21.15.57:0/1592069570 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7fece404f2a0 con 0x7fece41004e0 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 msgr2=0x7fecc806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fece4199730 tx=0x7fecd001a040 comp rx=0 tx=0).stop 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 msgr2=0x7fece4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c930 tx=0x7fecd400ccf0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 shutdown_connections 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fecc806d520 0x7fecc806f9d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fece41004e0 0x7fece4198110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 --2- 172.21.15.57:0/1592069570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4100e80 0x7fece4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:20.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 >> 172.21.15.57:0/1592069570 conn(0x7fece40fbda0 msgr2=0x7fece40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:20.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 shutdown_connections 2024-02-28T23:12:20.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:20.601+0000 7fece9e00700 1 -- 172.21.15.57:0/1592069570 wait complete. 2024-02-28T23:12:21.574 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-28T23:12:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:21 smithi057 ceph-mon[29096]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:22.052 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:21 smithi084 ceph-mon[34826]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:23.919 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:23 smithi057 ceph-mon[29096]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:23 smithi084 ceph-mon[34826]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.243+0000 7effe7a00700 1 -- 172.21.15.57:0/938442423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe01004e0 msgr2=0x7effe01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.243+0000 7effe7a00700 1 --2- 172.21.15.57:0/938442423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe01004e0 0x7effe01008b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7effd4009910 tx=0x7effd4009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 -- 172.21.15.57:0/938442423 shutdown_connections 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 --2- 172.21.15.57:0/938442423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe0100e80 0x7effe0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 --2- 172.21.15.57:0/938442423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe01004e0 0x7effe01008b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 -- 172.21.15.57:0/938442423 >> 172.21.15.57:0/938442423 conn(0x7effe00fbda0 msgr2=0x7effe00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 -- 172.21.15.57:0/938442423 shutdown_connections 2024-02-28T23:12:24.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.244+0000 7effe7a00700 1 -- 172.21.15.57:0/938442423 wait complete. 2024-02-28T23:12:24.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.245+0000 7effe7a00700 1 Processor -- start 2024-02-28T23:12:24.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.245+0000 7effe7a00700 1 -- start start 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effe7a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe01004e0 0x7effe0195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effe7a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effe7a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0196b10 con 0x7effe0100e80 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effe7a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe019a8a0 con 0x7effe01004e0 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effdfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effdfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54454/0 (socket says 172.21.15.57:54454) 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.246+0000 7effdfe00700 1 -- 172.21.15.57:0/1796046744 learned_addr learned my addr 172.21.15.57:0/1796046744 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effe5000700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe01004e0 0x7effe0195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdfe00700 1 -- 172.21.15.57:0/1796046744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe01004e0 msgr2=0x7effe0195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdfe00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe01004e0 0x7effe0195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdfe00700 1 -- 172.21.15.57:0/1796046744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40095f0 con 0x7effe0100e80 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdfe00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7effcc00e8f0 tx=0x7effcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effcc009790 con 0x7effe0100e80 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.247+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effcc004cf0 con 0x7effe0100e80 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.248+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe019ab80 con 0x7effe0100e80 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.248+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effcc005690 con 0x7effe0100e80 2024-02-28T23:12:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.248+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe019b0d0 con 0x7effe0100e80 2024-02-28T23:12:24.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.249+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe01087f0 con 0x7effe0100e80 2024-02-28T23:12:24.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.250+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7effcc004750 con 0x7effe0100e80 2024-02-28T23:12:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.251+0000 7effdd600700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 0x7effc806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.251+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7effcc014070 con 0x7effe0100e80 2024-02-28T23:12:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.252+0000 7effe5000700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 0x7effc806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.252+0000 7effe5000700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 0x7effc806f9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7effd400b760 tx=0x7effd4005fd0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:24.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.256+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7effcc057070 con 0x7effe0100e80 2024-02-28T23:12:24.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.504+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}) v1 -- 0x7effe019b350 con 0x7effc806d520 2024-02-28T23:12:24.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.513+0000 7effdd600700 1 -- 172.21.15.57:0/1796046744 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7effe019b350 con 0x7effc806d520 2024-02-28T23:12:24.513 INFO:teuthology.orchestra.run.smithi057.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:12:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.516+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 msgr2=0x7effc806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.516+0000 7effe7a00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 0x7effc806f9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7effd400b760 tx=0x7effd4005fd0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 msgr2=0x7effe0196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7effcc00e8f0 tx=0x7effcc00ecb0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 shutdown_connections 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7effc806d520 0x7effc806f9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effe01004e0 0x7effe0195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 --2- 172.21.15.57:0/1796046744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe0100e80 0x7effe0196430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 >> 172.21.15.57:0/1796046744 conn(0x7effe00fbda0 msgr2=0x7effe00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 shutdown_connections 2024-02-28T23:12:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:24.517+0000 7effe7a00700 1 -- 172.21.15.57:0/1796046744 wait complete. 2024-02-28T23:12:25.324 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-28T23:12:25.325 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:12:25.325 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-28T23:12:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:12:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:12:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:12:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:25 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:12:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:25 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:26.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:26 smithi057 ceph-mon[29096]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:12:26.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:26 smithi084 ceph-mon[34826]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-02-28T23:12:27.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:27 smithi057 ceph-mon[29096]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:27 smithi084 ceph-mon[34826]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:28.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.210+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3886027762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab010c8f0 msgr2=0x7f9ab010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.210+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3886027762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab010c8f0 0x7f9ab010ccc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009910 tx=0x7f9aa0009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3886027762 shutdown_connections 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3886027762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab0106b80 0x7f9ab0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3886027762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab010c8f0 0x7f9ab010ccc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3886027762 >> 172.21.15.57:0/3886027762 conn(0x7f9ab0076060 msgr2=0x7f9ab0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3886027762 shutdown_connections 2024-02-28T23:12:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.211+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3886027762 wait complete. 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 Processor -- start 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 -- start start 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab010c8f0 0x7f9ab019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab019d120 con 0x7f9ab0106b80 2024-02-28T23:12:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.212+0000 7f9ab6e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab01a0eb0 con 0x7f9ab010c8f0 2024-02-28T23:12:28.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60156/0 (socket says 172.21.15.57:60156) 2024-02-28T23:12:28.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 -- 172.21.15.57:0/3585079894 learned_addr learned my addr 172.21.15.57:0/3585079894 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aaf400700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab010c8f0 0x7f9ab019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 -- 172.21.15.57:0/3585079894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab010c8f0 msgr2=0x7f9ab019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab010c8f0 0x7f9ab019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.213+0000 7f9aafe00700 1 -- 172.21.15.57:0/3585079894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00095f0 con 0x7f9ab0106b80 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9aafe00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000b5c0 tx=0x7f9aa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aa001d070 con 0x7f9ab0106b80 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab01a1130 con 0x7f9ab0106b80 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aa0005e00 con 0x7f9ab0106b80 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aa0017400 con 0x7f9ab0106b80 2024-02-28T23:12:28.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.214+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab01a1620 con 0x7f9ab0106b80 2024-02-28T23:12:28.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.216+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab0109b20 con 0x7f9ab0106b80 2024-02-28T23:12:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.217+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f9aa000f460 con 0x7f9ab0106b80 2024-02-28T23:12:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.218+0000 7f9aacc00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 0x7f9a9406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.218+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9aa008e450 con 0x7f9ab0106b80 2024-02-28T23:12:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.218+0000 7f9aaf400700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 0x7f9a9406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.219+0000 7f9aaf400700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 0x7f9a9406f9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005fd0 tx=0x7f9a9c005f60 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:28.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.223+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9aa0058110 con 0x7f9ab0106b80 2024-02-28T23:12:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.482+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ab01a1890 con 0x7f9a9406d520 2024-02-28T23:12:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.484+0000 7f9aacc00700 1 -- 172.21.15.57:0/3585079894 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9ab01a1890 con 0x7f9a9406d520 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 msgr2=0x7f9a9406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 0x7f9a9406f9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005fd0 tx=0x7f9a9c005f60 comp rx=0 tx=0).stop 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 msgr2=0x7f9ab019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000b5c0 tx=0x7f9aa000be20 comp rx=0 tx=0).stop 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 shutdown_connections 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f9a9406d520 0x7f9a9406f9d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab0106b80 0x7f9ab019c500 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 --2- 172.21.15.57:0/3585079894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ab010c8f0 0x7f9ab019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.487+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 >> 172.21.15.57:0/3585079894 conn(0x7f9ab0076060 msgr2=0x7f9ab0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.488+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 shutdown_connections 2024-02-28T23:12:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.488+0000 7f9ab6e00700 1 -- 172.21.15.57:0/3585079894 wait complete. 2024-02-28T23:12:28.504 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:12:28.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 -- 172.21.15.57:0/2688274536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260102470 msgr2=0x7fd2601028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 --2- 172.21.15.57:0/2688274536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260102470 0x7fd2601028e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009910 tx=0x7fd24c009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:28.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 -- 172.21.15.57:0/2688274536 shutdown_connections 2024-02-28T23:12:28.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 --2- 172.21.15.57:0/2688274536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260102470 0x7fd2601028e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 --2- 172.21.15.57:0/2688274536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260108470 0x7fd260108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.652+0000 7fd267600700 1 -- 172.21.15.57:0/2688274536 >> 172.21.15.57:0/2688274536 conn(0x7fd2600fdf90 msgr2=0x7fd2601003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.653+0000 7fd267600700 1 -- 172.21.15.57:0/2688274536 shutdown_connections 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.653+0000 7fd267600700 1 -- 172.21.15.57:0/2688274536 wait complete. 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.654+0000 7fd267600700 1 Processor -- start 2024-02-28T23:12:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.654+0000 7fd267600700 1 -- start start 2024-02-28T23:12:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.654+0000 7fd267600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260102470 0x7fd260198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.654+0000 7fd267600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.654+0000 7fd267600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260198e00 con 0x7fd260108470 2024-02-28T23:12:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd267600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260198f40 con 0x7fd260102470 2024-02-28T23:12:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60172/0 (socket says 172.21.15.57:60172) 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 -- 172.21.15.57:0/1618338588 learned_addr learned my addr 172.21.15.57:0/1618338588 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd264c00700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260102470 0x7fd260198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 -- 172.21.15.57:0/1618338588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260102470 msgr2=0x7fd260198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260102470 0x7fd260198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.655+0000 7fd25fe00700 1 -- 172.21.15.57:0/1618338588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0095f0 con 0x7fd260108470 2024-02-28T23:12:28.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd25fe00700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00b5c0 tx=0x7fd24c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:28.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c01d070 con 0x7fd260108470 2024-02-28T23:12:28.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24c00f460 con 0x7fd260108470 2024-02-28T23:12:28.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26019cce0 con 0x7fd260108470 2024-02-28T23:12:28.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c017610 con 0x7fd260108470 2024-02-28T23:12:28.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.656+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26019d1d0 con 0x7fd260108470 2024-02-28T23:12:28.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.658+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240005320 con 0x7fd260108470 2024-02-28T23:12:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.659+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fd24c003a60 con 0x7fd260108470 2024-02-28T23:12:28.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.660+0000 7fd25d600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 0x7fd24806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.660+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd24c08e490 con 0x7fd260108470 2024-02-28T23:12:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.660+0000 7fd264c00700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 0x7fd24806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.661+0000 7fd264c00700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 0x7fd24806f9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd254005ea0 tx=0x7fd254005e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.665+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd24c058150 con 0x7fd260108470 2024-02-28T23:12:28.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.936+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd240000bf0 con 0x7fd24806d520 2024-02-28T23:12:28.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.938+0000 7fd25d600700 1 -- 172.21.15.57:0/1618338588 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd240000bf0 con 0x7fd24806d520 2024-02-28T23:12:28.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.940+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 msgr2=0x7fd24806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.940+0000 7fd267600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 0x7fd24806f9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd254005ea0 tx=0x7fd254005e30 comp rx=0 tx=0).stop 2024-02-28T23:12:28.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.940+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 msgr2=0x7fd2601987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:28.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.940+0000 7fd267600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00b5c0 tx=0x7fd24c00be00 comp rx=0 tx=0).stop 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 shutdown_connections 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fd24806d520 0x7fd24806f9d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd260102470 0x7fd260198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 --2- 172.21.15.57:0/1618338588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd260108470 0x7fd2601987b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 >> 172.21.15.57:0/1618338588 conn(0x7fd2600fdf90 msgr2=0x7fd2600ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 shutdown_connections 2024-02-28T23:12:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:28.941+0000 7fd267600700 1 -- 172.21.15.57:0/1618338588 wait complete. 2024-02-28T23:12:29.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.100+0000 7f5697e00700 1 -- 172.21.15.57:0/1859911095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 msgr2=0x7f5690110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 --2- 172.21.15.57:0/1859911095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f5690110a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f567c009940 tx=0x7f567c009c50 comp rx=0 tx=0).stop 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 -- 172.21.15.57:0/1859911095 shutdown_connections 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 --2- 172.21.15.57:0/1859911095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f5690110a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 --2- 172.21.15.57:0/1859911095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 0x7f5690068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 -- 172.21.15.57:0/1859911095 >> 172.21.15.57:0/1859911095 conn(0x7f5690075130 msgr2=0x7f5690075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 -- 172.21.15.57:0/1859911095 shutdown_connections 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.101+0000 7f5697e00700 1 -- 172.21.15.57:0/1859911095 wait complete. 2024-02-28T23:12:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.102+0000 7f5697e00700 1 Processor -- start 2024-02-28T23:12:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.102+0000 7f5697e00700 1 -- start start 2024-02-28T23:12:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5697e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 0x7f569019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5697e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5697e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569019d080 con 0x7f5690068ee0 2024-02-28T23:12:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5697e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56901a0e10 con 0x7f5690068540 2024-02-28T23:12:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5694a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5694a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60188/0 (socket says 172.21.15.57:60188) 2024-02-28T23:12:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5694a00700 1 -- 172.21.15.57:0/1985941533 learned_addr learned my addr 172.21.15.57:0/1985941533 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.103+0000 7f5695400700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 0x7f569019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f5694a00700 1 -- 172.21.15.57:0/1985941533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 msgr2=0x7f569019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f5694a00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 0x7f569019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f5694a00700 1 -- 172.21.15.57:0/1985941533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c0095f0 con 0x7f5690068ee0 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f5694a00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f567c0055f0 tx=0x7f567c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f567c01d070 con 0x7f5690068ee0 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f567c00f460 con 0x7f5690068ee0 2024-02-28T23:12:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f567c021640 con 0x7f5690068ee0 2024-02-28T23:12:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.104+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56901a1090 con 0x7f5690068ee0 2024-02-28T23:12:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.105+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56901a1590 con 0x7f5690068ee0 2024-02-28T23:12:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.106+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5690101410 con 0x7f5690068ee0 2024-02-28T23:12:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.107+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f567c00f5d0 con 0x7f5690068ee0 2024-02-28T23:12:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.108+0000 7f568e000700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 0x7f567806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.108+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f567c08e2f0 con 0x7f5690068ee0 2024-02-28T23:12:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.108+0000 7f5695400700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 0x7f567806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.109+0000 7f5695400700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 0x7f567806f9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5684005950 tx=0x7f56840058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:29.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.113+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f567c057f30 con 0x7f5690068ee0 2024-02-28T23:12:29.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.358+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f56901a18c0 con 0x7f567806d520 2024-02-28T23:12:29.371 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (2m) 82s ago 15m 13.9M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (12m) 82s ago 16m 8144k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (14m) 2m ago 14m 7952k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (16m) 82s ago 16m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (14m) 2m ago 14m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (90s) 82s ago 15m 50.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (8m) 82s ago 8m 57.1M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (8m) 82s ago 8m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (8m) 2m ago 8m 64.5M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (8m) 2m ago 8m 116M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (4m) 82s ago 17m 501M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:12:29.372 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (3m) 2m ago 14m 446M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (17m) 82s ago 18m 54.0M 2048M 18.2.0 dc2bc1663786 687a52a7799e 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (14m) 2m ago 14m 45.1M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (3m) 82s ago 16m 7633k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (3m) 2m ago 14m 2965k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (13m) 82s ago 13m 227M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (12m) 82s ago 12m 240M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (12m) 82s ago 12m 201M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (11m) 2m ago 11m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (11m) 2m ago 11m 236M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (10m) 2m ago 10m 229M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:12:29.373 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (2m) 82s ago 15m 47.7M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:12:29.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.371+0000 7f568e000700 1 -- 172.21.15.57:0/1985941533 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f56901a18c0 con 0x7f567806d520 2024-02-28T23:12:29.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 msgr2=0x7f567806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 0x7f567806f9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5684005950 tx=0x7f56840058e0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 msgr2=0x7f569019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f567c0055f0 tx=0x7f567c0056d0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 shutdown_connections 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f567806d520 0x7f567806f9d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5690068540 0x7f569019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 --2- 172.21.15.57:0/1985941533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5690068ee0 0x7f569019c9a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.374+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 >> 172.21.15.57:0/1985941533 conn(0x7f5690075130 msgr2=0x7f569010eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.375+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 shutdown_connections 2024-02-28T23:12:29.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.375+0000 7f5697e00700 1 -- 172.21.15.57:0/1985941533 wait complete. 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 -- 172.21.15.57:0/1377102059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e8106b80 msgr2=0x7fb1e8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 --2- 172.21.15.57:0/1377102059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e8106b80 0x7fb1e8106ff0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009910 tx=0x7fb1d4009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 -- 172.21.15.57:0/1377102059 shutdown_connections 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 --2- 172.21.15.57:0/1377102059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e8106b80 0x7fb1e8106ff0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 --2- 172.21.15.57:0/1377102059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e810c8f0 0x7fb1e810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 -- 172.21.15.57:0/1377102059 >> 172.21.15.57:0/1377102059 conn(0x7fb1e8076060 msgr2=0x7fb1e8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:29.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.550+0000 7fb1efc00700 1 -- 172.21.15.57:0/1377102059 shutdown_connections 2024-02-28T23:12:29.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.551+0000 7fb1efc00700 1 -- 172.21.15.57:0/1377102059 wait complete. 2024-02-28T23:12:29.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 Processor -- start 2024-02-28T23:12:29.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 -- start start 2024-02-28T23:12:29.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e8106b80 0x7fb1e819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e819d060 con 0x7fb1e810c8f0 2024-02-28T23:12:29.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1efc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e81a0df0 con 0x7fb1e8106b80 2024-02-28T23:12:29.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.552+0000 7fb1e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60198/0 (socket says 172.21.15.57:60198) 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1e7e00700 1 -- 172.21.15.57:0/3644068022 learned_addr learned my addr 172.21.15.57:0/3644068022 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1ed200700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e8106b80 0x7fb1e819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1e7e00700 1 -- 172.21.15.57:0/3644068022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e8106b80 msgr2=0x7fb1e819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1e7e00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e8106b80 0x7fb1e819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1e7e00700 1 -- 172.21.15.57:0/3644068022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40095f0 con 0x7fb1e810c8f0 2024-02-28T23:12:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.553+0000 7fb1e7e00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.554+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d401d070 con 0x7fb1e810c8f0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.554+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1d400f460 con 0x7fb1e810c8f0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.554+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e81a1070 con 0x7fb1e810c8f0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.554+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d4017630 con 0x7fb1e810c8f0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.554+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e81a1560 con 0x7fb1e810c8f0 2024-02-28T23:12:29.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.555+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e810ecc0 con 0x7fb1e810c8f0 2024-02-28T23:12:29.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.556+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fb1d400f5d0 con 0x7fb1e810c8f0 2024-02-28T23:12:29.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.557+0000 7fb1e5600700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 0x7fb1d006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:29.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.557+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb1d408e450 con 0x7fb1e810c8f0 2024-02-28T23:12:29.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.557+0000 7fb1ed200700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 0x7fb1d006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:29.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.558+0000 7fb1ed200700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 0x7fb1d006f9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005950 tx=0x7fb1dc0058e0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.562+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb1d4058110 con 0x7fb1e810c8f0 2024-02-28T23:12:29.787 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:29 smithi057 ceph-mon[29096]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:29.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:29 smithi057 ceph-mon[29096]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:29.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:29 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:29 smithi084 ceph-mon[34826]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:29 smithi084 ceph-mon[34826]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:29 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:29.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.910+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb1e804f2a0 con 0x7fb1e810c8f0 2024-02-28T23:12:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.910+0000 7fb1e5600700 1 -- 172.21.15.57:0/3644068022 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fb1d405b7d0 con 0x7fb1e810c8f0 2024-02-28T23:12:29.910 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:12:29.910 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:12:29.910 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:12:29.911 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 msgr2=0x7fb1d006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 0x7fb1d006f9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005950 tx=0x7fb1dc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 msgr2=0x7fb1e819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d400be20 comp rx=0 tx=0).stop 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 shutdown_connections 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb1d006d520 0x7fb1d006f9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e8106b80 0x7fb1e819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 --2- 172.21.15.57:0/3644068022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1e810c8f0 0x7fb1e819c980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 >> 172.21.15.57:0/3644068022 conn(0x7fb1e8076060 msgr2=0x7fb1e8077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.914+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 shutdown_connections 2024-02-28T23:12:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:29.915+0000 7fb1efc00700 1 -- 172.21.15.57:0/3644068022 wait complete. 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 -- 172.21.15.57:0/3792301099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 msgr2=0x7fb48010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 --2- 172.21.15.57:0/3792301099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb48010c5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009940 tx=0x7fb46c009c50 comp rx=0 tx=0).stop 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 -- 172.21.15.57:0/3792301099 shutdown_connections 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 --2- 172.21.15.57:0/3792301099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb48010c5c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 --2- 172.21.15.57:0/3792301099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 0x7fb480073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 -- 172.21.15.57:0/3792301099 >> 172.21.15.57:0/3792301099 conn(0x7fb4800fbda0 msgr2=0x7fb4800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 -- 172.21.15.57:0/3792301099 shutdown_connections 2024-02-28T23:12:30.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.087+0000 7fb487a00700 1 -- 172.21.15.57:0/3792301099 wait complete. 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.088+0000 7fb487a00700 1 Processor -- start 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.088+0000 7fb487a00700 1 -- start start 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb487a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 0x7fb480198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb487a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb487a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480198e30 con 0x7fb4800739c0 2024-02-28T23:12:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb487a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48019cbc0 con 0x7fb4800730b0 2024-02-28T23:12:30.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60226/0 (socket says 172.21.15.57:60226) 2024-02-28T23:12:30.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.089+0000 7fb47fe00700 1 -- 172.21.15.57:0/1081596540 learned_addr learned my addr 172.21.15.57:0/1081596540 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:30.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb485000700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 0x7fb480198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47fe00700 1 -- 172.21.15.57:0/1081596540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 msgr2=0x7fb480198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47fe00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 0x7fb480198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47fe00700 1 -- 172.21.15.57:0/1081596540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0095f0 con 0x7fb4800739c0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47fe00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb46c01d070 con 0x7fb4800739c0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb46c003d50 con 0x7fb4800739c0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48019ce40 con 0x7fb4800739c0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.090+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb46c021410 con 0x7fb4800739c0 2024-02-28T23:12:30.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.091+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48019d2d0 con 0x7fb4800739c0 2024-02-28T23:12:30.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.092+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb480109cc0 con 0x7fb4800739c0 2024-02-28T23:12:30.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.093+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7fb46c00f810 con 0x7fb4800739c0 2024-02-28T23:12:30.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.094+0000 7fb47d600700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 0x7fb46806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.095+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb46c08e2b0 con 0x7fb4800739c0 2024-02-28T23:12:30.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.095+0000 7fb485000700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 0x7fb46806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.096+0000 7fb485000700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 0x7fb46806f9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb474009bf0 tx=0x7fb474009260 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.099+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb46c057ef0 con 0x7fb4800739c0 2024-02-28T23:12:30.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.380+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb48004f2a0 con 0x7fb4800739c0 2024-02-28T23:12:30.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.380+0000 7fb47d600700 1 -- 172.21.15.57:0/1081596540 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1847 (secure 0 0 0) 0x7fb46c05b5b0 con 0x7fb4800739c0 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:12:30.381 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:12:30.382 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:30.383 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:12:30.384 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:12:30.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 msgr2=0x7fb46806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 0x7fb46806f9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb474009bf0 tx=0x7fb474009260 comp rx=0 tx=0).stop 2024-02-28T23:12:30.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 msgr2=0x7fb480198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c00bbc0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 shutdown_connections 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7fb46806d520 0x7fb46806f9d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4800730b0 0x7fb480198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 --2- 172.21.15.57:0/1081596540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4800739c0 0x7fb480198750 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 >> 172.21.15.57:0/1081596540 conn(0x7fb4800fbda0 msgr2=0x7fb480106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.384+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 shutdown_connections 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.385+0000 7fb487a00700 1 -- 172.21.15.57:0/1081596540 wait complete. 2024-02-28T23:12:30.385 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:12:30.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 -- 172.21.15.57:0/150168600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100530 msgr2=0x7f03bc100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 --2- 172.21.15.57:0/150168600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100530 0x7f03bc100900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009910 tx=0x7f03b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 -- 172.21.15.57:0/150168600 shutdown_connections 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 --2- 172.21.15.57:0/150168600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100ed0 0x7f03bc104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 --2- 172.21.15.57:0/150168600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100530 0x7f03bc100900 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 -- 172.21.15.57:0/150168600 >> 172.21.15.57:0/150168600 conn(0x7f03bc0fbe10 msgr2=0x7f03bc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 -- 172.21.15.57:0/150168600 shutdown_connections 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.545+0000 7f03c3600700 1 -- 172.21.15.57:0/150168600 wait complete. 2024-02-28T23:12:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.546+0000 7f03c3600700 1 Processor -- start 2024-02-28T23:12:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.546+0000 7f03c3600700 1 -- start start 2024-02-28T23:12:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03c3600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100530 0x7f03bc195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03c3600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03c3600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc196b70 con 0x7f03bc100ed0 2024-02-28T23:12:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03c3600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc19a900 con 0x7f03bc100530 2024-02-28T23:12:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60244/0 (socket says 172.21.15.57:60244) 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 -- 172.21.15.57:0/3358037111 learned_addr learned my addr 172.21.15.57:0/3358037111 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03c0c00700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100530 0x7f03bc195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 -- 172.21.15.57:0/3358037111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100530 msgr2=0x7f03bc195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100530 0x7f03bc195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.547+0000 7f03bbe00700 1 -- 172.21.15.57:0/3358037111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00095f0 con 0x7f03bc100ed0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03bbe00700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03a800b480 tx=0x7f03a800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a80118c0 con 0x7f03bc100ed0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc19abe0 con 0x7f03bc100ed0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03a8010380 con 0x7f03bc100ed0 2024-02-28T23:12:30.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a800f4f0 con 0x7f03bc100ed0 2024-02-28T23:12:30.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.548+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc19b130 con 0x7f03bc100ed0 2024-02-28T23:12:30.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.549+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc108840 con 0x7f03bc100ed0 2024-02-28T23:12:30.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.551+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f03a80104f0 con 0x7f03bc100ed0 2024-02-28T23:12:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.552+0000 7f03b9600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 0x7f03a4073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.552+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f03a808cad0 con 0x7f03bc100ed0 2024-02-28T23:12:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.552+0000 7f03c0c00700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 0x7f03a4073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.553+0000 7f03c0c00700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 0x7f03a4073de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006010 tx=0x7f03b001a040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.557+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f03a8056830 con 0x7f03bc100ed0 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3644068022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-28T23:12:30.778 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:30 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/1081596540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:12:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.816+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 --> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03bc19b470 con 0x7f03a4071930 2024-02-28T23:12:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.818+0000 7f03b9600700 1 -- 172.21.15.57:0/3358037111 <== mgr.14676 v2:172.21.15.57:6800/3833211904 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f03bc19b470 con 0x7f03a4071930 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "2/23 daemons upgraded", 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:12:30.818 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:12:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 msgr2=0x7f03a4073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 0x7f03a4073de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006010 tx=0x7f03b001a040 comp rx=0 tx=0).stop 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 msgr2=0x7f03bc196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03a800b480 tx=0x7f03a800b840 comp rx=0 tx=0).stop 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 shutdown_connections 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f03a4071930 0x7f03a4073de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03bc100530 0x7f03bc195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 --2- 172.21.15.57:0/3358037111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03bc100ed0 0x7f03bc196490 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.821+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 >> 172.21.15.57:0/3358037111 conn(0x7f03bc0fbe10 msgr2=0x7f03bc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.822+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 shutdown_connections 2024-02-28T23:12:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.822+0000 7f03c3600700 1 -- 172.21.15.57:0/3358037111 wait complete. 2024-02-28T23:12:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3644068022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-28T23:12:30.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:30 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/1081596540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:12:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 -- 172.21.15.57:0/2695738854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 msgr2=0x7f94840689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 --2- 172.21.15.57:0/2695738854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f94840689b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9470009910 tx=0x7f9470009c20 comp rx=0 tx=0).stop 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 -- 172.21.15.57:0/2695738854 shutdown_connections 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 --2- 172.21.15.57:0/2695738854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f94840689b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 --2- 172.21.15.57:0/2695738854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 0x7f94841066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 -- 172.21.15.57:0/2695738854 >> 172.21.15.57:0/2695738854 conn(0x7f9484075130 msgr2=0x7f9484075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 -- 172.21.15.57:0/2695738854 shutdown_connections 2024-02-28T23:12:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.982+0000 7f9489e00700 1 -- 172.21.15.57:0/2695738854 wait complete. 2024-02-28T23:12:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 Processor -- start 2024-02-28T23:12:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 -- start start 2024-02-28T23:12:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 0x7f9484198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9484198c40 con 0x7f9484068540 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.983+0000 7f9489e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419c9d0 con 0x7f9484106320 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60266/0 (socket says 172.21.15.57:60266) 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 -- 172.21.15.57:0/3906549764 learned_addr learned my addr 172.21.15.57:0/3906549764 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:12:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482000700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 0x7f9484198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 -- 172.21.15.57:0/3906549764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 msgr2=0x7f9484198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 0x7f9484198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.984+0000 7f9482a00700 1 -- 172.21.15.57:0/3906549764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700095f0 con 0x7f9484068540 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9482a00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f947400d8d0 tx=0x7f947400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9474009940 con 0x7f9484068540 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948419ccb0 con 0x7f9484068540 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9474010460 con 0x7f9484068540 2024-02-28T23:12:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947400f5f0 con 0x7f9484068540 2024-02-28T23:12:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.985+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948419d200 con 0x7f9484068540 2024-02-28T23:12:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.987+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948404f2a0 con 0x7f9484068540 2024-02-28T23:12:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.987+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 32) v1 ==== 91338+0+0 (secure 0 0 0) 0x7f9474009aa0 con 0x7f9484068540 2024-02-28T23:12:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.989+0000 7f9488a00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 0x7f946806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:12:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.989+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f947408cfe0 con 0x7f9484068540 2024-02-28T23:12:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.989+0000 7f9482000700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 0x7f946806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:12:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.990+0000 7f9482000700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 0x7f946806f9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9484199640 tx=0x7f947001a040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:12:30.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:30.994+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9474056c60 con 0x7f9484068540 2024-02-28T23:12:31.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.304+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f948419d4c0 con 0x7f9484068540 2024-02-28T23:12:31.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.304+0000 7f9488a00700 1 -- 172.21.15.57:0/3906549764 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f947405a320 con 0x7f9484068540 2024-02-28T23:12:31.304 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 msgr2=0x7f946806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 0x7f946806f9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9484199640 tx=0x7f947001a040 comp rx=0 tx=0).stop 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 msgr2=0x7f9484198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f947400d8d0 tx=0x7f947400dc90 comp rx=0 tx=0).stop 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 shutdown_connections 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:6800/3833211904,v1:172.21.15.57:6801/3833211904] conn(0x7f946806d520 0x7f946806f9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9484068540 0x7f9484198020 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 --2- 172.21.15.57:0/3906549764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9484106320 0x7f9484198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 >> 172.21.15.57:0/3906549764 conn(0x7f9484075130 msgr2=0x7f94840ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:12:31.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.307+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 shutdown_connections 2024-02-28T23:12:31.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:12:31.308+0000 7f9489e00700 1 -- 172.21.15.57:0/3906549764 wait complete. 2024-02-28T23:12:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:31 smithi057 ceph-mon[29096]: Upgrade: Target is version 19.0.0-1670-g1fb5d106 (squid) 2024-02-28T23:12:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:31 smithi057 ceph-mon[29096]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc'] 2024-02-28T23:12:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:31 smithi057 ceph-mon[29096]: Upgrade: Setting container_image for all mgr 2024-02-28T23:12:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:31 smithi057 ceph-mon[29096]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:31.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:31 smithi057 ceph-mon[29096]: from='client.? 172.21.15.57:0/3906549764' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:12:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:31 smithi084 ceph-mon[34826]: Upgrade: Target is version 19.0.0-1670-g1fb5d106 (squid) 2024-02-28T23:12:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:31 smithi084 ceph-mon[34826]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc'] 2024-02-28T23:12:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:31 smithi084 ceph-mon[34826]: Upgrade: Setting container_image for all mgr 2024-02-28T23:12:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:31 smithi084 ceph-mon[34826]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:31 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3906549764' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:12:32.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:32 smithi057 ceph-mon[29096]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:32 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:32 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:12:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:32 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:12:32.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:32 smithi057 ceph-mon[29096]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:32 smithi084 ceph-mon[34826]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:12:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:32 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:32 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:12:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:32 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:12:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:32 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:12:33.682 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:33 smithi057 ceph-mon[29096]: Upgrade: Updating mon.smithi057 2024-02-28T23:12:33.682 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:33 smithi057 ceph-mon[29096]: Deploying daemon mon.smithi057 on smithi057 2024-02-28T23:12:33.682 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:33 smithi057 ceph-mon[29096]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:33 smithi084 ceph-mon[34826]: Upgrade: Updating mon.smithi057 2024-02-28T23:12:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:33 smithi084 ceph-mon[34826]: Deploying daemon mon.smithi057 on smithi057 2024-02-28T23:12:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:33 smithi084 ceph-mon[34826]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:12:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:34 smithi057 systemd[1]: Stopping Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:12:35.402 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:35 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[29092]: 2024-02-28T23:12:35.123+0000 7f6c48c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi057 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:12:35.402 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:35 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[29092]: 2024-02-28T23:12:35.123+0000 7f6c48c00700 -1 mon.smithi057@0(leader) e2 *** Got Signal Terminated *** 2024-02-28T23:12:35.705 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:35 smithi057 podman[89317]: 2024-02-28 23:12:35.402983207 +0000 UTC m=+0.400073068 container died 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , ceph=True) 2024-02-28T23:12:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:35 smithi057 podman[89317]: 2024-02-28 23:12:35.908740987 +0000 UTC m=+0.905830835 container cleanup 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0) 2024-02-28T23:12:36.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:35 smithi057 bash[89317]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057 2024-02-28T23:12:36.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:36 smithi057 podman[89340]: 2024-02-28 23:12:36.235676914 +0000 UTC m=+0.818831527 container remove 687a52a7799eab13267b3fac469d370a003c900c7419c1bee88c9d6ee8d8d1dc (image=quay.io/ceph/ceph:v18.2.0, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True) 2024-02-28T23:12:37.193 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:36 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service: Deactivated successfully. 2024-02-28T23:12:37.193 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:36 smithi057 systemd[1]: Stopped Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:12:37.193 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:36 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service: Consumed 33.411s CPU time. 2024-02-28T23:12:37.557 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 systemd[1]: Starting Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:12:37.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 podman[89472]: 2024-02-28 23:12:37.462930608 +0000 UTC m=+0.032990933 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:12:37.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 podman[89472]: 2024-02-28 23:12:37.570080631 +0000 UTC m=+0.140140979 container create f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-28T23:12:38.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 podman[89472]: 2024-02-28 23:12:37.826584333 +0000 UTC m=+0.396644688 container init f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, GIT_CLEAN=True, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-28T23:12:38.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 podman[89472]: 2024-02-28 23:12:37.838582834 +0000 UTC m=+0.408643168 container start f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, ceph=True, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:12:38.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: set uid:gid to 167:167 (ceph:ceph) 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev), process ceph-mon, pid 2 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: pidfile_write: ignore empty --pid-file 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: load: jerasure load: lrc 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: RocksDB version: 7.9.2 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Git sha 0 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Compile date 2024-02-23 16:20:17 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: DB SUMMARY 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: DB Session ID: AIL15MP72FIDAOJLBRPM 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: CURRENT file: CURRENT 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: IDENTITY file: IDENTITY 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1732 Bytes 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi057/store.db dir, Total Num: 1, files: 000035.sst 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi057/store.db: 000033.log size: 1013313 ; 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.error_if_exists: 0 2024-02-28T23:12:38.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.create_if_missing: 0 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.paranoid_checks: 1 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.env: 0x555d38870b20 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.fs: PosixFileSystem 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.info_log: 0x555d3a3df7c0 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_file_opening_threads: 16 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.statistics: (nil) 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.use_fsync: 0 2024-02-28T23:12:38.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_log_file_size: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.keep_log_file_num: 1000 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.recycle_log_file_num: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_fallocate: 1 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_mmap_reads: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_mmap_writes: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.use_direct_reads: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.create_missing_column_families: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.db_log_dir: 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.wal_dir: 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-28T23:12:38.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.advise_random_on_open: 1 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.db_write_buffer_size: 0 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.write_buffer_manager: 0x555d3a3b3400 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.rate_limiter: (nil) 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.wal_recovery_mode: 2 2024-02-28T23:12:38.324 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enable_thread_tracking: 0 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enable_pipelined_write: 0 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.unordered_write: 0 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.row_cache: None 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.wal_filter: None 2024-02-28T23:12:38.325 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_ingest_behind: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.two_write_queues: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.manual_wal_flush: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.wal_compression: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.atomic_flush: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.log_readahead_size: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.best_efforts_recovery: 0 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-28T23:12:38.326 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.allow_data_in_errors: 0 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.db_host_id: __hostname__ 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_background_jobs: 2 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_background_compactions: -1 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_subcompactions: 1 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_total_wal_size: 0 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-28T23:12:38.327 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_open_files: -1 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bytes_per_sync: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_readahead_size: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_background_flushes: -1 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Compression algorithms supported: 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kZSTD supported: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kXpressCompression supported: 0 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kLZ4HCCompression supported: 1 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kZlibCompression supported: 1 2024-02-28T23:12:38.328 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kSnappyCompression supported: 1 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kLZ4Compression supported: 1 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: kBZip2Compression supported: 0 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi057/store.db/MANIFEST-000015 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.merge_operator: 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_filter: None 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_filter_factory: None 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.sst_partitioner_factory: None 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555d3a3df8e0) 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: cache_index_and_filter_blocks: 1 2024-02-28T23:12:38.329 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: pin_top_level_index_and_filter: 1 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_type: 0 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: data_block_index_type: 0 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_shortening: 1 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: checksum: 4 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: no_block_cache: 0 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache: 0x555d3a3e3090 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_name: BinnedLRUCache 2024-02-28T23:12:38.330 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_options: 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: capacity : 536870912 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: num_shard_bits : 4 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: strict_capacity_limit : 0 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: high_pri_pool_ratio: 0.000 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_cache_compressed: (nil) 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: persistent_cache: (nil) 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_size: 4096 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_size_deviation: 10 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_restart_interval: 16 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: index_block_restart_interval: 1 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: metadata_block_size: 4096 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: partition_filters: 0 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: use_delta_encoding: 1 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: filter_policy: bloomfilter 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: whole_key_filtering: 1 2024-02-28T23:12:38.331 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: verify_compression: 0 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: read_amp_bytes_per_bit: 0 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: format_version: 5 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: enable_index_compression: 1 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: block_align: 0 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: max_auto_readahead_size: 262144 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: prepopulate_block_cache: 0 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: initial_auto_readahead_size: 8192 2024-02-28T23:12:38.332 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.write_buffer_size: 33554432 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_write_buffer_number: 2 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression: NoCompression 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression: Disabled 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.prefix_extractor: nullptr 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-28T23:12:38.333 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.num_levels: 7 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.level: 32767 2024-02-28T23:12:38.334 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.strategy: 0 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.enabled: false 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.target_file_size_base: 67108864 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-28T23:12:38.335 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.arena_block_size: 1048576 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.disable_auto_compactions: 0 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-28T23:12:38.336 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.table_properties_collectors: 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.inplace_update_support: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.bloom_locality: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.max_successive_merges: 0 2024-02-28T23:12:38.337 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.paranoid_file_checks: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.force_consistency_checks: 1 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.report_bg_io_stats: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.ttl: 2592000 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enable_blob_files: false 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.min_blob_size: 0 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_file_size: 268435456 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-28T23:12:38.338 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.blob_file_starting_level: 0 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi057/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16517, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c66d57c8-b843-4c5a-aecf-6c5cd94005f7 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161957896697, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161957904141, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 941765, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16518, "largest_seqno": 17869, "table_properties": {"data_size": 936068, "index_size": 3274, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1413, "raw_key_size": 12738, "raw_average_key_size": 23, "raw_value_size": 924620, "raw_average_value_size": 1712, "num_data_blocks": 149, "num_entries": 540, "num_filter_entries": 540, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709161957, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c66d57c8-b843-4c5a-aecf-6c5cd94005f7", "db_session_id": "AIL15MP72FIDAOJLBRPM", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161957904376, "job": 1, "event": "recovery_finished"} 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi057/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555d3a4d2000 2024-02-28T23:12:38.339 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: DB pointer 0x555d3a4be000 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** DB Stats ** 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** Compaction Stats [default] ** 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: L0 1/0 919.69 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 220.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-28T23:12:38.340 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: L6 1/0 7.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Sum 2/0 8.74 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 220.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 220.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** Compaction Stats [default] ** 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 220.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-28T23:12:38.341 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Cumulative compaction: 0.00 GB write, 51.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Interval compaction: 0.00 GB write, 51.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Block cache BinnedLRUCache@0x555d3a3e3090#2 capacity: 512.00 MB usage: 4.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.53 KB,0.000292063%) IndexBlock(1,3.33 KB,0.000634789%) Misc(1,0.00 KB,0%) 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: starting mon.smithi057 rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi057 fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???) e2 preinit fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).mds e21 new map 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 bash[89472]: f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).mds e21 print_map 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: e21 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:12:38.342 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: legacy client fscid: 1 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: Filesystem 'cephfs' (1) 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: fs_name cephfs 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: epoch 21 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: tableserver 0 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: root 0 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: session_timeout 60 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: session_autoclose 300 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: max_file_size 1099511627776 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: max_xattr_size 65536 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: required_client_features {} 2024-02-28T23:12:38.343 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: last_failure 0 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: last_failure_osd_epoch 47 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: max_mds 2 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: in 0,1 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: up {0=14480,1=24281} 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: failed 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: damaged 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: stopped 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: data_pools [3] 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: metadata_pool 2 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: inline_data disabled 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: balancer 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: bal_rank_mask -1 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: standby_count_wanted 1 2024-02-28T23:12:38.344 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: [mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: [mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: [mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: [mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout: 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 ceph-mon[89491]: mon.smithi057@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-02-28T23:12:38.345 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:37 smithi057 systemd[1]: Started Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: pgmap v134: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: mon.smithi057 calling monitor election 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T23:12:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: monmap e2: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: osdmap e49: 6 total, 6 up, 6 in 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: mgrmap e32: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: overall HEALTH_OK 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: mgrmap e33: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:39.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:38 smithi057 ceph-mon[89491]: from='mgr.14676 ' entity='' 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: from='mgr.14676 172.21.15.57:0/1878628989' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: pgmap v134: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: mon.smithi057 calling monitor election 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: monmap e2: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-28T23:12:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:12:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: osdmap e49: 6 total, 6 up, 6 in 2024-02-28T23:12:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: mgrmap e32: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: overall HEALTH_OK 2024-02-28T23:12:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: mgrmap e33: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:38 smithi084 ceph-mon[34826]: from='mgr.14676 ' entity='' 2024-02-28T23:12:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:12:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:12:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:12:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:12:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:12:47.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:47 smithi057 ceph-mon[89491]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:12:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis restarted 2024-02-28T23:12:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: Standby manager daemon smithi084.fhkzis started 2024-02-28T23:12:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/crt"}]: dispatch 2024-02-28T23:12:47.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-28T23:12:47.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi084.fhkzis/key"}]: dispatch 2024-02-28T23:12:47.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:47 smithi084 ceph-mon[34826]: from='mgr.? 172.21.15.84:0/796795483' entity='mgr.smithi084.fhkzis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: mgrmap e34: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: Active manager daemon smithi057.zrrldj restarted 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: Activating manager daemon smithi057.zrrldj 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: osdmap e50: 6 total, 6 up, 6 in 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: mgrmap e35: smithi057.zrrldj(active, starting, since 0.0198191s), standbys: smithi084.fhkzis 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:12:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:12:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: Manager daemon smithi057.zrrldj is now available 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:12:48.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: mgrmap e34: smithi057.zrrldj(active, since 4m), standbys: smithi084.fhkzis 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: Active manager daemon smithi057.zrrldj restarted 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: Activating manager daemon smithi057.zrrldj 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: osdmap e50: 6 total, 6 up, 6 in 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: mgrmap e35: smithi057.zrrldj(active, starting, since 0.0198191s), standbys: smithi084.fhkzis 2024-02-28T23:12:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi057.zrrldj", "id": "smithi057.zrrldj"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr metadata", "who": "smithi084.fhkzis", "id": "smithi084.fhkzis"}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:12:48.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: Manager daemon smithi057.zrrldj is now available 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/mirror_snapshot_schedule"}]: dispatch 2024-02-28T23:12:48.833 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:48 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi057.zrrldj/trash_purge_schedule"}]: dispatch 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: [28/Feb/2024:23:12:48] ENGINE Bus STARTING 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: mgrmap e36: smithi057.zrrldj(active, since 1.03117s), standbys: smithi084.fhkzis 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: [28/Feb/2024:23:12:48] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: [28/Feb/2024:23:12:49] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T23:12:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:49 smithi057 ceph-mon[89491]: [28/Feb/2024:23:12:49] ENGINE Bus STARTED 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: [28/Feb/2024:23:12:48] ENGINE Bus STARTING 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: mgrmap e36: smithi057.zrrldj(active, since 1.03117s), standbys: smithi084.fhkzis 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: [28/Feb/2024:23:12:48] ENGINE Serving on http://172.21.15.57:8765 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: [28/Feb/2024:23:12:49] ENGINE Serving on https://172.21.15.57:7150 2024-02-28T23:12:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:49 smithi084 ceph-mon[34826]: [28/Feb/2024:23:12:49] ENGINE Bus STARTED 2024-02-28T23:12:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:50 smithi084 ceph-mon[34826]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:50 smithi084 ceph-mon[34826]: mgrmap e37: smithi057.zrrldj(active, since 2s), standbys: smithi084.fhkzis 2024-02-28T23:12:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:50 smithi057 ceph-mon[89491]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:50 smithi057 ceph-mon[89491]: mgrmap e37: smithi057.zrrldj(active, since 2s), standbys: smithi084.fhkzis 2024-02-28T23:12:53.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:53 smithi057 ceph-mon[89491]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:53.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:53 smithi057 ceph-mon[89491]: mgrmap e38: smithi057.zrrldj(active, since 4s), standbys: smithi084.fhkzis 2024-02-28T23:12:53.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:53.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:53 smithi084 ceph-mon[34826]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:53 smithi084 ceph-mon[34826]: mgrmap e38: smithi057.zrrldj(active, since 4s), standbys: smithi084.fhkzis 2024-02-28T23:12:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:53 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:53 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:55.869 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:55 smithi084 ceph-mon[34826]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:55.870 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:55 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:55.870 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:55 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:55 smithi057 ceph-mon[89491]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:12:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:57.198 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:56 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:57.199 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:56 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:57.199 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:56 smithi084 ceph-mon[34826]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-28T23:12:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:56 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:56 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:56 smithi057 ceph-mon[89491]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-28T23:12:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:59 smithi084 ceph-mon[34826]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-02-28T23:12:59.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:59 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:59.084 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:12:59 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:59.298 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:59 smithi057 ceph-mon[89491]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-02-28T23:12:59.298 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:12:59.298 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:12:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:00 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi084 2024-02-28T23:13:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:00 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:00 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:00 smithi084 ceph-mon[34826]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-28T23:13:01.096 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:00 smithi057 ceph-mon[89491]: Detected new or changed devices on smithi084 2024-02-28T23:13:01.096 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:00 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:01.096 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:00 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:01.096 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:00 smithi057 ceph-mon[89491]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.482+0000 7f4356000700 1 -- 172.21.15.57:0/1556862115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 msgr2=0x7f4350104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.482+0000 7f4356000700 1 --2- 172.21.15.57:0/1556862115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f4350104eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f433c009910 tx=0x7f433c009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 -- 172.21.15.57:0/1556862115 shutdown_connections 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 --2- 172.21.15.57:0/1556862115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f4350104eb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 --2- 172.21.15.57:0/1556862115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 0x7f4350068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 -- 172.21.15.57:0/1556862115 >> 172.21.15.57:0/1556862115 conn(0x7f4350075130 msgr2=0x7f4350075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 -- 172.21.15.57:0/1556862115 shutdown_connections 2024-02-28T23:13:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.483+0000 7f4356000700 1 -- 172.21.15.57:0/1556862115 wait complete. 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.484+0000 7f4356000700 1 Processor -- start 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.484+0000 7f4356000700 1 -- start start 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f4356000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 0x7f43500ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f4356000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f4356000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350100720 con 0x7f4350069140 2024-02-28T23:13:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f4356000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350100860 con 0x7f43500687a0 2024-02-28T23:13:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f434ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f434ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57116/0 (socket says 172.21.15.57:57116) 2024-02-28T23:13:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f434ea00700 1 -- 172.21.15.57:0/3096489292 learned_addr learned my addr 172.21.15.57:0/3096489292 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.485+0000 7f434f400700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 0x7f43500ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.486+0000 7f434ea00700 1 -- 172.21.15.57:0/3096489292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 msgr2=0x7f43500ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.486+0000 7f434ea00700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 0x7f43500ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.486+0000 7f434ea00700 1 -- 172.21.15.57:0/3096489292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0095f0 con 0x7f4350069140 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.486+0000 7f434ea00700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f433c006010 tx=0x7f433c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.487+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433c01d070 con 0x7f4350069140 2024-02-28T23:13:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.487+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f433c00f460 con 0x7f4350069140 2024-02-28T23:13:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.487+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433c017610 con 0x7f4350069140 2024-02-28T23:13:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.487+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350100ae0 con 0x7f4350069140 2024-02-28T23:13:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.487+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501a22e0 con 0x7f4350069140 2024-02-28T23:13:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.488+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350108800 con 0x7f4350069140 2024-02-28T23:13:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.490+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f433c00f5d0 con 0x7f4350069140 2024-02-28T23:13:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.491+0000 7f4347e00700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 0x7f433406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.491+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f433c08ec50 con 0x7f4350069140 2024-02-28T23:13:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.492+0000 7f434f400700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 0x7f433406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.493+0000 7f434f400700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 0x7f433406fe90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4350077bc0 tx=0x7f434000a3b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.498+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f433c058270 con 0x7f4350069140 2024-02-28T23:13:01.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.770+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f435002cca0 con 0x7f433406d9e0 2024-02-28T23:13:01.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.772+0000 7f4347e00700 1 -- 172.21.15.57:0/3096489292 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f435002cca0 con 0x7f433406d9e0 2024-02-28T23:13:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.775+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 msgr2=0x7f433406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.775+0000 7f4356000700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 0x7f433406fe90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4350077bc0 tx=0x7f434000a3b0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.775+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 msgr2=0x7f43501001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.775+0000 7f4356000700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f433c006010 tx=0x7f433c00bd70 comp rx=0 tx=0).stop 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 shutdown_connections 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f433406d9e0 0x7f433406fe90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43500687a0 0x7f43500ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 --2- 172.21.15.57:0/3096489292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4350069140 0x7f43501001e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 >> 172.21.15.57:0/3096489292 conn(0x7f4350075130 msgr2=0x7f43500fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 shutdown_connections 2024-02-28T23:13:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.776+0000 7f4356000700 1 -- 172.21.15.57:0/3096489292 wait complete. 2024-02-28T23:13:01.792 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 -- 172.21.15.57:0/2236064729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 msgr2=0x7f79d81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 --2- 172.21.15.57:0/2236064729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d81008b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009910 tx=0x7f79c8009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 -- 172.21.15.57:0/2236064729 shutdown_connections 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 --2- 172.21.15.57:0/2236064729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 --2- 172.21.15.57:0/2236064729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d81008b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 -- 172.21.15.57:0/2236064729 >> 172.21.15.57:0/2236064729 conn(0x7f79d80fbda0 msgr2=0x7f79d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 -- 172.21.15.57:0/2236064729 shutdown_connections 2024-02-28T23:13:01.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.941+0000 7f79dee00700 1 -- 172.21.15.57:0/2236064729 wait complete. 2024-02-28T23:13:01.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.942+0000 7f79dee00700 1 Processor -- start 2024-02-28T23:13:01.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.942+0000 7f79dee00700 1 -- start start 2024-02-28T23:13:01.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79dee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79dee00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79dee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8074e00 con 0x7f79d81004e0 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79dee00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8074f70 con 0x7f79d8100e80 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79d7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79d7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57136/0 (socket says 172.21.15.57:57136) 2024-02-28T23:13:01.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.943+0000 7f79d7e00700 1 -- 172.21.15.57:0/35322795 learned_addr learned my addr 172.21.15.57:0/35322795 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:01.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7400700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7e00700 1 -- 172.21.15.57:0/35322795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 msgr2=0x7f79d8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:01.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7e00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:01.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7e00700 1 -- 172.21.15.57:0/35322795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c80095f0 con 0x7f79d81004e0 2024-02-28T23:13:01.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7400700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:13:01.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d7e00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79c800b5c0 tx=0x7f79c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:01.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.944+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c801d070 con 0x7f79d81004e0 2024-02-28T23:13:01.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.945+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79c8005e00 con 0x7f79d81004e0 2024-02-28T23:13:01.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.945+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8073b30 con 0x7f79d81004e0 2024-02-28T23:13:01.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.945+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c8017400 con 0x7f79d81004e0 2024-02-28T23:13:01.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.945+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d81a6890 con 0x7f79d81004e0 2024-02-28T23:13:01.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.946+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d81087f0 con 0x7f79d81004e0 2024-02-28T23:13:01.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.947+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f79c800f460 con 0x7f79d81004e0 2024-02-28T23:13:01.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.948+0000 7f79d4c00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 0x7f79bc06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:01.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.949+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f79c808ebc0 con 0x7f79d81004e0 2024-02-28T23:13:01.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.949+0000 7f79d7400700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 0x7f79bc06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:01.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.950+0000 7f79d7400700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 0x7f79bc06fe80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005fd0 tx=0x7f79c4005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:01.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:01.954+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79c80581e0 con 0x7f79d81004e0 2024-02-28T23:13:02.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.264+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79d8074640 con 0x7f79bc06d9d0 2024-02-28T23:13:02.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.266+0000 7f79d4c00700 1 -- 172.21.15.57:0/35322795 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f79d8074640 con 0x7f79bc06d9d0 2024-02-28T23:13:02.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.270+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 msgr2=0x7f79bc06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.270+0000 7f79dee00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 0x7f79bc06fe80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005fd0 tx=0x7f79c4005ee0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.270+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 msgr2=0x7f79d8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.270+0000 7f79dee00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79c800b5c0 tx=0x7f79c800be20 comp rx=0 tx=0).stop 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.270+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 shutdown_connections 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79bc06d9d0 0x7f79bc06fe80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79d81004e0 0x7f79d8073050 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 --2- 172.21.15.57:0/35322795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d8100e80 0x7f79d8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 >> 172.21.15.57:0/35322795 conn(0x7f79d80fbda0 msgr2=0x7f79d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 shutdown_connections 2024-02-28T23:13:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.271+0000 7f79dee00700 1 -- 172.21.15.57:0/35322795 wait complete. 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 -- 172.21.15.57:0/2331382753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f70103040 msgr2=0x7f8f70107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 --2- 172.21.15.57:0/2331382753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f70103040 0x7f8f70107010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009910 tx=0x7f8f5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 -- 172.21.15.57:0/2331382753 shutdown_connections 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 --2- 172.21.15.57:0/2331382753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f70103040 0x7f8f70107010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 --2- 172.21.15.57:0/2331382753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f701026a0 0x7f8f70102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 -- 172.21.15.57:0/2331382753 >> 172.21.15.57:0/2331382753 conn(0x7f8f700fdf40 msgr2=0x7f8f70100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 -- 172.21.15.57:0/2331382753 shutdown_connections 2024-02-28T23:13:02.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.432+0000 7f8f77200700 1 -- 172.21.15.57:0/2331382753 wait complete. 2024-02-28T23:13:02.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 Processor -- start 2024-02-28T23:13:02.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 -- start start 2024-02-28T23:13:02.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70103040 0x7f8f70198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70198c70 con 0x7f8f701026a0 2024-02-28T23:13:02.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.434+0000 7f8f77200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7019ca00 con 0x7f8f70103040 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57146/0 (socket says 172.21.15.57:57146) 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 -- 172.21.15.57:0/2042839594 learned_addr learned my addr 172.21.15.57:0/2042839594 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6f400700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70103040 0x7f8f70198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 -- 172.21.15.57:0/2042839594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70103040 msgr2=0x7f8f70198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70103040 0x7f8f70198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.435+0000 7f8f6fe00700 1 -- 172.21.15.57:0/2042839594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0095f0 con 0x7f8f701026a0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f6fe00700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c8a0 tx=0x7f8f6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6000cea0 con 0x7f8f701026a0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f60004830 con 0x7f8f701026a0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f600055b0 con 0x7f8f701026a0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7019cce0 con 0x7f8f701026a0 2024-02-28T23:13:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.436+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7019d230 con 0x7f8f701026a0 2024-02-28T23:13:02.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.438+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7010aa10 con 0x7f8f701026a0 2024-02-28T23:13:02.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.439+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f8f600049a0 con 0x7f8f701026a0 2024-02-28T23:13:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.440+0000 7f8f6cc00700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 0x7f8f5406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.440+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8f6008d2d0 con 0x7f8f701026a0 2024-02-28T23:13:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.440+0000 7f8f6f400700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 0x7f8f5406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.441+0000 7f8f6f400700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 0x7f8f5406fe80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f70199670 tx=0x7f8f5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:02.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.445+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f60056990 con 0x7f8f701026a0 2024-02-28T23:13:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.688+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f7019d500 con 0x7f8f5406d9d0 2024-02-28T23:13:02.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.701+0000 7f8f6cc00700 1 -- 172.21.15.57:0/2042839594 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8f7019d500 con 0x7f8f5406d9d0 2024-02-28T23:13:02.700 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:13:02.700 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (2m) 6s ago 16m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:13:02.700 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (12m) 6s ago 16m 8292k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:13:02.700 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (15m) 10s ago 15m 8216k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (16m) 6s ago 16m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (15m) 10s ago 15m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (2m) 6s ago 16m 75.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (9m) 6s ago 9m 56.8M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (9m) 6s ago 9m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (9m) 10s ago 9m 57.9M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (9m) 10s ago 9m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (5m) 6s ago 18m 477M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (4m) 10s ago 15m 447M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (25s) 6s ago 18m 29.4M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (15m) 10s ago 14m 48.9M 2048M 18.2.0 dc2bc1663786 89931f9db573 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (3m) 6s ago 16m 8010k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (3m) 10s ago 15m 7768k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (14m) 6s ago 14m 229M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:13:02.701 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (13m) 6s ago 13m 241M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:13:02.702 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (12m) 6s ago 12m 202M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:13:02.702 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (12m) 10s ago 12m 272M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:13:02.702 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (11m) 10s ago 11m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:13:02.702 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (11m) 10s ago 11m 232M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:13:02.702 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (3m) 6s ago 15m 58.6M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:13:02.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 msgr2=0x7f8f5406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 0x7f8f5406fe80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f70199670 tx=0x7f8f5c01a040 comp rx=0 tx=0).stop 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 msgr2=0x7f8f70198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c8a0 tx=0x7f8f6000cc60 comp rx=0 tx=0).stop 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 shutdown_connections 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d9d0 0x7f8f5406fe80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f701026a0 0x7f8f70198050 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 --2- 172.21.15.57:0/2042839594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70103040 0x7f8f70198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 >> 172.21.15.57:0/2042839594 conn(0x7f8f700fdf40 msgr2=0x7f8f70100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 shutdown_connections 2024-02-28T23:13:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.705+0000 7f8f77200700 1 -- 172.21.15.57:0/2042839594 wait complete. 2024-02-28T23:13:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.883+0000 7f0ae4000700 1 -- 172.21.15.57:0/1660525534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 msgr2=0x7f0adc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.883+0000 7f0ae4000700 1 --2- 172.21.15.57:0/1660525534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc106ff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009910 tx=0x7f0ac8009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 -- 172.21.15.57:0/1660525534 shutdown_connections 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 --2- 172.21.15.57:0/1660525534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc106ff0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 --2- 172.21.15.57:0/1660525534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc10c8f0 0x7f0adc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 -- 172.21.15.57:0/1660525534 >> 172.21.15.57:0/1660525534 conn(0x7f0adc076060 msgr2=0x7f0adc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 -- 172.21.15.57:0/1660525534 shutdown_connections 2024-02-28T23:13:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.884+0000 7f0ae4000700 1 -- 172.21.15.57:0/1660525534 wait complete. 2024-02-28T23:13:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 Processor -- start 2024-02-28T23:13:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 -- start start 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc10c8f0 0x7f0adc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc19d060 con 0x7f0adc106b80 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.885+0000 7f0ae4000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1a0df0 con 0x7f0adc10c8f0 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57152/0 (socket says 172.21.15.57:57152) 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 -- 172.21.15.57:0/3955476655 learned_addr learned my addr 172.21.15.57:0/3955476655 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 -- 172.21.15.57:0/3955476655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc10c8f0 msgr2=0x7f0adc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc10c8f0 0x7f0adc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 -- 172.21.15.57:0/3955476655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80095f0 con 0x7f0adc106b80 2024-02-28T23:13:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ae1600700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c8a0 tx=0x7f0ad000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad000cea0 con 0x7f0adc106b80 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ad0004830 con 0x7f0adc106b80 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.886+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad00055b0 con 0x7f0adc106b80 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.887+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc1a10d0 con 0x7f0adc106b80 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.887+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc1a1620 con 0x7f0adc106b80 2024-02-28T23:13:02.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.888+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc10ecc0 con 0x7f0adc106b80 2024-02-28T23:13:02.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.889+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f0ad00049a0 con 0x7f0adc106b80 2024-02-28T23:13:02.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.890+0000 7f0ada000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 0x7f0ac406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:02.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.890+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0ad008d370 con 0x7f0adc106b80 2024-02-28T23:13:02.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.890+0000 7f0ae0c00700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 0x7f0ac406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:02.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.891+0000 7f0ae0c00700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 0x7f0ac406fe80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8006010 tx=0x7f0ac800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:02.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:02.894+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ad0056990 con 0x7f0adc106b80 2024-02-28T23:13:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.230+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0adc1a18f0 con 0x7f0adc106b80 2024-02-28T23:13:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.231+0000 7f0ada000700 1 -- 172.21.15.57:0/3955476655 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7f0ad001d320 con 0x7f0adc106b80 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:13:03.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 11, 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:13:03.231 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:13:03.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 msgr2=0x7f0ac406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 0x7f0ac406fe80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8006010 tx=0x7f0ac800b540 comp rx=0 tx=0).stop 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 msgr2=0x7f0adc19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c8a0 tx=0x7f0ad000cc60 comp rx=0 tx=0).stop 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 shutdown_connections 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ac406d9d0 0x7f0ac406fe80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0adc106b80 0x7f0adc19c440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 --2- 172.21.15.57:0/3955476655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc10c8f0 0x7f0adc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 >> 172.21.15.57:0/3955476655 conn(0x7f0adc076060 msgr2=0x7f0adc077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 shutdown_connections 2024-02-28T23:13:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.234+0000 7f0ae4000700 1 -- 172.21.15.57:0/3955476655 wait complete. 2024-02-28T23:13:03.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:03 smithi057 ceph-mon[89491]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:03 smithi057 ceph-mon[89491]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-28T23:13:03.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:03 smithi057 ceph-mon[89491]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:03 smithi057 ceph-mon[89491]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:03 smithi084 ceph-mon[34826]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:03 smithi084 ceph-mon[34826]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-28T23:13:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:03 smithi084 ceph-mon[34826]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:03 smithi084 ceph-mon[34826]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 -- 172.21.15.57:0/1620016179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c100e80 msgr2=0x7fcb2c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 --2- 172.21.15.57:0/1620016179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c100e80 0x7fcb2c104e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009910 tx=0x7fcb18009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 -- 172.21.15.57:0/1620016179 shutdown_connections 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 --2- 172.21.15.57:0/1620016179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c100e80 0x7fcb2c104e50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 --2- 172.21.15.57:0/1620016179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 -- 172.21.15.57:0/1620016179 >> 172.21.15.57:0/1620016179 conn(0x7fcb2c0fbda0 msgr2=0x7fcb2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 -- 172.21.15.57:0/1620016179 shutdown_connections 2024-02-28T23:13:03.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.383+0000 7fcb34000700 1 -- 172.21.15.57:0/1620016179 wait complete. 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.384+0000 7fcb34000700 1 Processor -- start 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.384+0000 7fcb34000700 1 -- start start 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb34000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb34000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 0x7fcb2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb34000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c198e70 con 0x7fcb2c1004e0 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb34000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19cc00 con 0x7fcb2c100e80 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57174/0 (socket says 172.21.15.57:57174) 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 -- 172.21.15.57:0/2874210162 learned_addr learned my addr 172.21.15.57:0/2874210162 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb30c00700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 0x7fcb2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 -- 172.21.15.57:0/2874210162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 msgr2=0x7fcb2c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 0x7fcb2c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb31600700 1 -- 172.21.15.57:0/2874210162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180095f0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.385+0000 7fcb30c00700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 0x7fcb2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:13:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb31600700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000ba50 tx=0x7fcb2000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb2000ca00 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb2000cb60 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb200075a0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c19cee0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.386+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c19d430 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.388+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c1045d0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.388+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fcb2000ccd0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.389+0000 7fcb2a000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 0x7fcb1406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.389+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcb20006e40 con 0x7fcb2c1004e0 2024-02-28T23:13:03.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.390+0000 7fcb30c00700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 0x7fcb1406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.390+0000 7fcb30c00700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 0x7fcb1406fe80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb18006010 tx=0x7fcb1800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:03.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.394+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb200564f0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.670+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcb2c04f2a0 con 0x7fcb2c1004e0 2024-02-28T23:13:03.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.671+0000 7fcb2a000700 1 -- 172.21.15.57:0/2874210162 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fcb20019070 con 0x7fcb2c1004e0 2024-02-28T23:13:03.669 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:13:03.670 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:13:03.671 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:03.672 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.673+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 msgr2=0x7fcb1406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.673+0000 7fcb34000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 0x7fcb1406fe80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb18006010 tx=0x7fcb1800b540 comp rx=0 tx=0).stop 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.673+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 msgr2=0x7fcb2c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.673+0000 7fcb34000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000ba50 tx=0x7fcb2000be10 comp rx=0 tx=0).stop 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 shutdown_connections 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb1406d9d0 0x7fcb1406fe80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb2c1004e0 0x7fcb2c198250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 --2- 172.21.15.57:0/2874210162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c100e80 0x7fcb2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 >> 172.21.15.57:0/2874210162 conn(0x7fcb2c0fbda0 msgr2=0x7fcb2c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 shutdown_connections 2024-02-28T23:13:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.674+0000 7fcb34000700 1 -- 172.21.15.57:0/2874210162 wait complete. 2024-02-28T23:13:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 -- 172.21.15.57:0/41963242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88840730f0 msgr2=0x7f88840734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 --2- 172.21.15.57:0/41963242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88840730f0 0x7f88840734c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8874009910 tx=0x7f8874009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 -- 172.21.15.57:0/41963242 shutdown_connections 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 --2- 172.21.15.57:0/41963242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8884073a00 0x7f8884110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 --2- 172.21.15.57:0/41963242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88840730f0 0x7f88840734c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 -- 172.21.15.57:0/41963242 >> 172.21.15.57:0/41963242 conn(0x7f88840fbda0 msgr2=0x7f88840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 -- 172.21.15.57:0/41963242 shutdown_connections 2024-02-28T23:13:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.838+0000 7f8888e00700 1 -- 172.21.15.57:0/41963242 wait complete. 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.839+0000 7f8888e00700 1 Processor -- start 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.839+0000 7f8888e00700 1 -- start start 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f8888e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88840730f0 0x7f888419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f8888e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f8888e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419d000 con 0x7f8884073a00 2024-02-28T23:13:03.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f8888e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88841a0d90 con 0x7f88840730f0 2024-02-28T23:13:03.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f887be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f887be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57188/0 (socket says 172.21.15.57:57188) 2024-02-28T23:13:03.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f887be00700 1 -- 172.21.15.57:0/1031967914 learned_addr learned my addr 172.21.15.57:0/1031967914 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.840+0000 7f8881e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88840730f0 0x7f888419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887be00700 1 -- 172.21.15.57:0/1031967914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88840730f0 msgr2=0x7f888419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887be00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88840730f0 0x7f888419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887be00700 1 -- 172.21.15.57:0/1031967914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740095f0 con 0x7f8884073a00 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887be00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d8d0 tx=0x7f886c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f886c0098a0 con 0x7f8884073a00 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f886c010460 con 0x7f8884073a00 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88841a1070 con 0x7f8884073a00 2024-02-28T23:13:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f886c00f5f0 con 0x7f8884073a00 2024-02-28T23:13:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.841+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88841a15c0 con 0x7f8884073a00 2024-02-28T23:13:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.843+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888410e4d0 con 0x7f8884073a00 2024-02-28T23:13:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.844+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f886c009a00 con 0x7f8884073a00 2024-02-28T23:13:03.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.845+0000 7f887aa00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 0x7f886806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:03.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.845+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f886c08d5b0 con 0x7f8884073a00 2024-02-28T23:13:03.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.845+0000 7f8881e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 0x7f886806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:03.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.846+0000 7f8881e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 0x7f886806fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8874006010 tx=0x7f887401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:03.850+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f886c056bd0 con 0x7f8884073a00 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/3955476655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/2874210162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:04 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:13:04.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.103+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88841a18a0 con 0x7f886806d9d0 2024-02-28T23:13:04.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.105+0000 7f887aa00700 1 -- 172.21.15.57:0/1031967914 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f88841a18a0 con 0x7f886806d9d0 2024-02-28T23:13:04.104 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:13:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:13:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "3/23 daemons upgraded", 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2024-02-28T23:13:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:13:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 msgr2=0x7f886806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 0x7f886806fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8874006010 tx=0x7f887401a040 comp rx=0 tx=0).stop 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 msgr2=0x7f888419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d8d0 tx=0x7f886c00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 shutdown_connections 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f886806d9d0 0x7f886806fe80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88840730f0 0x7f888419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 --2- 172.21.15.57:0/1031967914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8884073a00 0x7f888419c920 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 >> 172.21.15.57:0/1031967914 conn(0x7f88840fbda0 msgr2=0x7f8884102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 shutdown_connections 2024-02-28T23:13:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.108+0000 7f8888e00700 1 -- 172.21.15.57:0/1031967914 wait complete. 2024-02-28T23:13:04.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 -- 172.21.15.57:0/2541669182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f00739a0 msgr2=0x7f50f010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 --2- 172.21.15.57:0/2541669182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f00739a0 0x7f50f010c5e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009940 tx=0x7f50dc009c50 comp rx=0 tx=0).stop 2024-02-28T23:13:04.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 -- 172.21.15.57:0/2541669182 shutdown_connections 2024-02-28T23:13:04.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 --2- 172.21.15.57:0/2541669182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f00739a0 0x7f50f010c5e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 --2- 172.21.15.57:0/2541669182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f0073090 0x7f50f0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.257+0000 7f50f6c00700 1 -- 172.21.15.57:0/2541669182 >> 172.21.15.57:0/2541669182 conn(0x7f50f00fbda0 msgr2=0x7f50f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:04.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.258+0000 7f50f6c00700 1 -- 172.21.15.57:0/2541669182 shutdown_connections 2024-02-28T23:13:04.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.258+0000 7f50f6c00700 1 -- 172.21.15.57:0/2541669182 wait complete. 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 Processor -- start 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 -- start start 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00739a0 0x7f50f01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f0198dd0 con 0x7f50f0073090 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50f6c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f019cb60 con 0x7f50f00739a0 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.259+0000 7f50efe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57204/0 (socket says 172.21.15.57:57204) 2024-02-28T23:13:04.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 -- 172.21.15.57:0/391090210 learned_addr learned my addr 172.21.15.57:0/391090210 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:04.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50ef400700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00739a0 0x7f50f01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:04.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 -- 172.21.15.57:0/391090210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00739a0 msgr2=0x7f50f01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00739a0 0x7f50f01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 -- 172.21.15.57:0/391090210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0095f0 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.260+0000 7f50efe00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50e000d8d0 tx=0x7f50e000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.261+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50e00098a0 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.261+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50e0010460 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.261+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50e000f5f0 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.261+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f019ce40 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.261+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f019d360 con 0x7f50f0073090 2024-02-28T23:13:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.262+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0109ce0 con 0x7f50f0073090 2024-02-28T23:13:04.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.263+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f50e0009a00 con 0x7f50f0073090 2024-02-28T23:13:04.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.264+0000 7f50ecc00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 0x7f50d406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:04.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.264+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f50e008d510 con 0x7f50f0073090 2024-02-28T23:13:04.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.264+0000 7f50ef400700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 0x7f50d406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:04.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.265+0000 7f50ef400700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 0x7f50d406fe80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009910 tx=0x7f50dc005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.269+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50e0056bd0 con 0x7f50f0073090 2024-02-28T23:13:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3955476655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2874210162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:13:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:04.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:04.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:04.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:13:04.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.612+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f50f004f2a0 con 0x7f50f0073090 2024-02-28T23:13:04.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.613+0000 7f50ecc00700 1 -- 172.21.15.57:0/391090210 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f50e005a290 con 0x7f50f0073090 2024-02-28T23:13:04.612 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:13:04.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 msgr2=0x7f50d406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 0x7f50d406fe80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009910 tx=0x7f50dc005ab0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 msgr2=0x7f50f01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:04.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50e000d8d0 tx=0x7f50e000dbe0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 shutdown_connections 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f50d406d9d0 0x7f50d406fe80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f0073090 0x7f50f01981b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 --2- 172.21.15.57:0/391090210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00739a0 0x7f50f01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 >> 172.21.15.57:0/391090210 conn(0x7f50f00fbda0 msgr2=0x7f50f0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 shutdown_connections 2024-02-28T23:13:04.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:04.616+0000 7f50f6c00700 1 -- 172.21.15.57:0/391090210 wait complete. 2024-02-28T23:13:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: Detected new or changed devices on smithi057 2024-02-28T23:13:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:13:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:13:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-28T23:13:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:13:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:13:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:05 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/391090210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: Detected new or changed devices on smithi057 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.conf 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.conf 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-28T23:13:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:05 smithi084 ceph-mon[34826]: from='client.? 172.21.15.57:0/391090210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:13:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:06 smithi084 ceph-mon[34826]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:13:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:06 smithi084 ceph-mon[34826]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:13:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:06 smithi057 ceph-mon[89491]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:13:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:06 smithi057 ceph-mon[89491]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: Updating smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: Updating smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:07 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:08 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:13:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:08 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:08 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-28T23:13:08.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:13:08.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:08.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-28T23:13:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:09 smithi084 ceph-mon[34826]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:09.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:09 smithi057 ceph-mon[89491]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:10.657 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: Upgrade: Updating mon.smithi084 2024-02-28T23:13:10.658 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:10.658 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:10.658 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:10.658 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:10.658 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:10 smithi084 ceph-mon[34826]: Deploying daemon mon.smithi084 on smithi084 2024-02-28T23:13:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: Upgrade: Updating mon.smithi084 2024-02-28T23:13:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:10.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:10.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:10 smithi057 ceph-mon[89491]: Deploying daemon mon.smithi084 on smithi084 2024-02-28T23:13:11.670 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:11 smithi084 ceph-mon[34826]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:11 smithi057 ceph-mon[89491]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:12 smithi084 systemd[1]: Stopping Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:13:13.022 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:12 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084[34816]: 2024-02-28T23:13:12.617+0000 7f614d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi084 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:13:13.022 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:12 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084[34816]: 2024-02-28T23:13:12.617+0000 7f614d800700 -1 mon.smithi084@1(peon) e2 *** Got Signal Terminated *** 2024-02-28T23:13:13.022 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:12 smithi084 podman[70280]: 2024-02-28 23:13:12.699807936 +0000 UTC m=+0.224775309 container died 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1) 2024-02-28T23:13:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 podman[70280]: 2024-02-28 23:13:13.136126538 +0000 UTC m=+0.661093915 container cleanup 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.build-date=20231212) 2024-02-28T23:13:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 bash[70280]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084 2024-02-28T23:13:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 podman[70294]: 2024-02-28 23:13:13.419725606 +0000 UTC m=+0.704704950 container remove 89931f9db5733a434b8bebb26a15ca60851330c3b3d1014599d9483e126927ae (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, RELEASE=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-28T23:13:14.260 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi084.service: Deactivated successfully. 2024-02-28T23:13:14.260 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 systemd[1]: Stopped Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:13:14.260 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:13 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi084.service: Consumed 19.920s CPU time. 2024-02-28T23:13:14.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 systemd[1]: Starting Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:13:14.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 podman[70422]: 2024-02-28 23:13:14.531265234 +0000 UTC m=+0.036836619 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:13:14.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 podman[70422]: 2024-02-28 23:13:14.66257872 +0000 UTC m=+0.168150125 container create b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-28T23:13:14.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 podman[70422]: 2024-02-28 23:13:14.924642531 +0000 UTC m=+0.430213982 container init b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 podman[70422]: 2024-02-28 23:13:14.93491493 +0000 UTC m=+0.440486341 container start b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: set uid:gid to 167:167 (ceph:ceph) 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev), process ceph-mon, pid 2 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: pidfile_write: ignore empty --pid-file 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: load: jerasure load: lrc 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: RocksDB version: 7.9.2 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Git sha 0 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Compile date 2024-02-23 16:20:17 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: DB SUMMARY 2024-02-28T23:13:15.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: DB Session ID: IG4CXPTF6QIA3LXC19KG 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: CURRENT file: CURRENT 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: IDENTITY file: IDENTITY 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1626 Bytes 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi084/store.db dir, Total Num: 1, files: 000030.sst 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi084/store.db: 000028.log size: 4582318 ; 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.error_if_exists: 0 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.create_if_missing: 0 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.paranoid_checks: 1 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.env: 0x5626fb9bab20 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.fs: PosixFileSystem 2024-02-28T23:13:15.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.info_log: 0x5626fdc8f7c0 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_file_opening_threads: 16 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.statistics: (nil) 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.use_fsync: 0 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_log_file_size: 0 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.keep_log_file_num: 1000 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.recycle_log_file_num: 0 2024-02-28T23:13:15.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_fallocate: 1 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_mmap_reads: 0 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_mmap_writes: 0 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.use_direct_reads: 0 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.create_missing_column_families: 0 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.db_log_dir: 2024-02-28T23:13:15.335 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.wal_dir: 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.advise_random_on_open: 1 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.db_write_buffer_size: 0 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.write_buffer_manager: 0x5626fdc63400 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-28T23:13:15.336 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.rate_limiter: (nil) 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.wal_recovery_mode: 2 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enable_thread_tracking: 0 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enable_pipelined_write: 0 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.unordered_write: 0 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-28T23:13:15.337 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.row_cache: None 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.wal_filter: None 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_ingest_behind: 0 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.two_write_queues: 0 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.manual_wal_flush: 0 2024-02-28T23:13:15.338 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.wal_compression: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.atomic_flush: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.log_readahead_size: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.best_efforts_recovery: 0 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-28T23:13:15.340 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.allow_data_in_errors: 0 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.db_host_id: __hostname__ 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_background_jobs: 2 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_background_compactions: -1 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_subcompactions: 1 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_total_wal_size: 0 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-28T23:13:15.341 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_open_files: -1 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bytes_per_sync: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_readahead_size: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_background_flushes: -1 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Compression algorithms supported: 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kZSTD supported: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kXpressCompression supported: 0 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kLZ4HCCompression supported: 1 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kZlibCompression supported: 1 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kSnappyCompression supported: 1 2024-02-28T23:13:15.342 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kLZ4Compression supported: 1 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: kBZip2Compression supported: 0 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi084/store.db/MANIFEST-000010 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.merge_operator: 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_filter: None 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_filter_factory: None 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.sst_partitioner_factory: None 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5626fdc8f8e0) 2024-02-28T23:13:15.343 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_type: 0 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: data_block_index_type: 0 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_shortening: 1 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: checksum: 4 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: no_block_cache: 0 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache: 0x5626fdc93090 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_options: 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: capacity : 536870912 2024-02-28T23:13:15.344 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: num_shard_bits : 4 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: strict_capacity_limit : 0 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_cache_compressed: (nil) 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: persistent_cache: (nil) 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_size: 4096 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_size_deviation: 10 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_restart_interval: 16 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: index_block_restart_interval: 1 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: metadata_block_size: 4096 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: partition_filters: 0 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: use_delta_encoding: 1 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: filter_policy: bloomfilter 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: whole_key_filtering: 1 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: verify_compression: 0 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-02-28T23:13:15.345 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: format_version: 5 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: enable_index_compression: 1 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: block_align: 0 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: max_auto_readahead_size: 262144 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: prepopulate_block_cache: 0 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: initial_auto_readahead_size: 8192 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.write_buffer_size: 33554432 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_write_buffer_number: 2 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression: NoCompression 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression: Disabled 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.prefix_extractor: nullptr 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.num_levels: 7 2024-02-28T23:13:15.346 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-28T23:13:15.347 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-28T23:13:15.348 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.level: 32767 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.strategy: 0 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.enabled: false 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.target_file_size_base: 67108864 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-28T23:13:15.349 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.arena_block_size: 1048576 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-28T23:13:15.350 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.disable_auto_compactions: 0 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.table_properties_collectors: 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.inplace_update_support: 0 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-28T23:13:15.351 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.bloom_locality: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.max_successive_merges: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.paranoid_file_checks: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.force_consistency_checks: 1 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.report_bg_io_stats: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.ttl: 2592000 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-28T23:13:15.352 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enable_blob_files: false 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.min_blob_size: 0 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_file_size: 268435456 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.blob_file_starting_level: 0 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi084/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16841, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ea462b4a-01a8-46aa-8522-ef4f18e31a70 2024-02-28T23:13:15.353 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161994990259, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:14 smithi084 ceph-mon[70440]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161995008588, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2872170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16846, "largest_seqno": 18947, "table_properties": {"data_size": 2864105, "index_size": 5061, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1989, "raw_key_size": 18595, "raw_average_key_size": 23, "raw_value_size": 2847224, "raw_average_value_size": 3594, "num_data_blocks": 230, "num_entries": 792, "num_filter_entries": 792, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709161995, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ea462b4a-01a8-46aa-8522-ef4f18e31a70", "db_session_id": "IG4CXPTF6QIA3LXC19KG", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709161995008931, "job": 1, "event": "recovery_finished"} 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi084/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5626fdd82000 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: DB pointer 0x5626fdd6e000 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** DB Stats ** 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-28T23:13:15.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** Compaction Stats [default] ** 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: L0 1/0 2.74 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 505.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: L6 1/0 7.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Sum 2/0 10.58 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 505.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 505.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-28T23:13:15.355 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** Compaction Stats [default] ** 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 505.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Cumulative compaction: 0.00 GB write, 105.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T23:13:15.356 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Interval compaction: 0.00 GB write, 105.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Block cache BinnedLRUCache@0x5626fdc93090#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: starting mon.smithi084 rank 1 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi084 fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???) e2 preinit fsid 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).mds e21 new map 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).mds e21 print_map 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: e21 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:15.357 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: legacy client fscid: 1 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: Filesystem 'cephfs' (1) 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: fs_name cephfs 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: epoch 21 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: tableserver 0 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: root 0 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: session_timeout 60 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: session_autoclose 300 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: max_file_size 1099511627776 2024-02-28T23:13:15.358 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: max_xattr_size 65536 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: required_client_features {} 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: last_failure 0 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: last_failure_osd_epoch 47 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: max_mds 2 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: in 0,1 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: up {0=14480,1=24281} 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: failed 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: damaged 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: stopped 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: data_pools [3] 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: metadata_pool 2 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: inline_data disabled 2024-02-28T23:13:15.359 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: balancer 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: bal_rank_mask -1 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: standby_count_wanted 1 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: [mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: [mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: [mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: [mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout: 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 ceph-mon[70440]: mon.smithi084@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-02-28T23:13:15.360 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 bash[70422]: b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c 2024-02-28T23:13:15.361 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:15 smithi084 systemd[1]: Started Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:13:16.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:16 smithi057 ceph-mon[89491]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:13:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:16 smithi084 ceph-mon[70440]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: mon.smithi057 calling monitor election 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: mon.smithi084 calling monitor election 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: monmap e3: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:13:17.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: osdmap e50: 6 total, 6 up, 6 in 2024-02-28T23:13:17.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: mgrmap e38: smithi057.zrrldj(active, since 28s), standbys: smithi084.fhkzis 2024-02-28T23:13:17.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: overall HEALTH_OK 2024-02-28T23:13:17.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:17.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:17.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:17 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:13:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi057"}]: dispatch 2024-02-28T23:13:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mon metadata", "id": "smithi084"}]: dispatch 2024-02-28T23:13:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: mon.smithi057 calling monitor election 2024-02-28T23:13:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: mon.smithi084 calling monitor election 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: mon.smithi057 is new leader, mons smithi057,smithi084 in quorum (ranks 0,1) 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: monmap e3: 2 mons at {smithi057=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],smithi084=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby-replay 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: osdmap e50: 6 total, 6 up, 6 in 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: mgrmap e38: smithi057.zrrldj(active, since 28s), standbys: smithi084.fhkzis 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: overall HEALTH_OK 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:18.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:17 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:13:18.851 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:19.877 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:19 smithi084 ceph-mon[70440]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:13:20.026 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:19 smithi057 ceph-mon[89491]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:13:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:21 smithi057 ceph-mon[89491]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:22.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:22.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:21 smithi084 ceph-mon[70440]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:13:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:24.005 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:23 smithi084 ceph-mon[70440]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:24.005 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:24.005 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:23 smithi057 ceph-mon[89491]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:25.734 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:25 smithi084 ceph-mon[70440]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:13:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:25 smithi057 ceph-mon[89491]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:13:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:27 smithi057 ceph-mon[89491]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:27 smithi084 ceph-mon[70440]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:29.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:29.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:29.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: Reconfiguring mon.smithi057 (monmap changed)... 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:29.489 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:29 smithi057 ceph-mon[89491]: Reconfiguring daemon mon.smithi057 on smithi057 2024-02-28T23:13:29.540 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: Reconfiguring mon.smithi057 (monmap changed)... 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:29.541 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:29 smithi084 ceph-mon[70440]: Reconfiguring daemon mon.smithi057 on smithi057 2024-02-28T23:13:31.369 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:31.369 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:31.369 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:31.370 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:13:31.370 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:13:31.370 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi057.zrrldj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:13:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:32.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:32 smithi057 ceph-mon[89491]: Reconfiguring mgr.smithi057.zrrldj (monmap changed)... 2024-02-28T23:13:32.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:32 smithi057 ceph-mon[89491]: Reconfiguring daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T23:13:32.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:32 smithi084 ceph-mon[70440]: Reconfiguring mgr.smithi057.zrrldj (monmap changed)... 2024-02-28T23:13:32.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:32 smithi084 ceph-mon[70440]: Reconfiguring daemon mgr.smithi057.zrrldj on smithi057 2024-02-28T23:13:33.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:33 smithi084 ceph-mon[70440]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:33.482 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:33.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:33 smithi057 ceph-mon[89491]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:33.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: Reconfiguring ceph-exporter.smithi057 (monmap changed)... 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T23:13:34.544 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: Unable to update caps for client.ceph-exporter.smithi057 2024-02-28T23:13:34.545 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi057"}]: dispatch 2024-02-28T23:13:34.545 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:34.545 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:34 smithi057 ceph-mon[89491]: Reconfiguring daemon ceph-exporter.smithi057 on smithi057 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 -- 172.21.15.57:0/2790091939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 msgr2=0x7fee34102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 --2- 172.21.15.57:0/2790091939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34102a70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee28009910 tx=0x7fee28009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 -- 172.21.15.57:0/2790091939 shutdown_connections 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 --2- 172.21.15.57:0/2790091939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 --2- 172.21.15.57:0/2790091939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34102a70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 -- 172.21.15.57:0/2790091939 >> 172.21.15.57:0/2790091939 conn(0x7fee340fdf40 msgr2=0x7fee34100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 -- 172.21.15.57:0/2790091939 shutdown_connections 2024-02-28T23:13:34.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.793+0000 7fee3c000700 1 -- 172.21.15.57:0/2790091939 wait complete. 2024-02-28T23:13:34.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 Processor -- start 2024-02-28T23:13:34.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 -- start start 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34198d50 con 0x7fee341026a0 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee3c000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3419cae0 con 0x7fee34103040 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.795+0000 7fee39600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40448/0 (socket says 172.21.15.57:40448) 2024-02-28T23:13:34.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 -- 172.21.15.57:0/3009608333 learned_addr learned my addr 172.21.15.57:0/3009608333 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:34.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee38c00700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:34.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 -- 172.21.15.57:0/3009608333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 msgr2=0x7fee34198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:34.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 -- 172.21.15.57:0/3009608333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280095f0 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee38c00700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.796+0000 7fee39600700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b5c0 tx=0x7fee2800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.797+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2801d070 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.797+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2800f460 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.797+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3419cd60 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.797+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee28017610 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.797+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3419d250 con 0x7fee341026a0 2024-02-28T23:13:34.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.798+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3410a9b0 con 0x7fee341026a0 2024-02-28T23:13:34.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.799+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fee2800f5d0 con 0x7fee341026a0 2024-02-28T23:13:34.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.800+0000 7fee32000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 0x7fee1c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:34.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.800+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fee2808ed30 con 0x7fee341026a0 2024-02-28T23:13:34.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.800+0000 7fee38c00700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 0x7fee1c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:34.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.801+0000 7fee38c00700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 0x7fee1c06fe80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:34.806+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee28058350 con 0x7fee341026a0 2024-02-28T23:13:34.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:34.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:34.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: Reconfiguring ceph-exporter.smithi057 (monmap changed)... 2024-02-28T23:13:34.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: Unable to update caps for client.ceph-exporter.smithi057 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi057"}]: dispatch 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:34.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:34 smithi084 ceph-mon[70440]: Reconfiguring daemon ceph-exporter.smithi057 on smithi057 2024-02-28T23:13:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.076+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee3419d4c0 con 0x7fee1c06d9d0 2024-02-28T23:13:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.079+0000 7fee32000700 1 -- 172.21.15.57:0/3009608333 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fee3419d4c0 con 0x7fee1c06d9d0 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.079+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 msgr2=0x7fee1c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.079+0000 7fee3c000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 0x7fee1c06fe80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20005dc0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.079+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 msgr2=0x7fee34198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.079+0000 7fee3c000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b5c0 tx=0x7fee2800be00 comp rx=0 tx=0).stop 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 shutdown_connections 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee1c06d9d0 0x7fee1c06fe80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341026a0 0x7fee34198130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 --2- 172.21.15.57:0/3009608333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee34103040 0x7fee34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 >> 172.21.15.57:0/3009608333 conn(0x7fee340fdf40 msgr2=0x7fee340ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 shutdown_connections 2024-02-28T23:13:35.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.083+0000 7fee3c000700 1 -- 172.21.15.57:0/3009608333 wait complete. 2024-02-28T23:13:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 -- 172.21.15.57:0/3967479845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 msgr2=0x7fbce41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 --2- 172.21.15.57:0/3967479845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41028e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009910 tx=0x7fbcd0009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 -- 172.21.15.57:0/3967479845 shutdown_connections 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 --2- 172.21.15.57:0/3967479845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41028e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 --2- 172.21.15.57:0/3967479845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 0x7fbce4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 -- 172.21.15.57:0/3967479845 >> 172.21.15.57:0/3967479845 conn(0x7fbce40fdf90 msgr2=0x7fbce41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:35.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 -- 172.21.15.57:0/3967479845 shutdown_connections 2024-02-28T23:13:35.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.252+0000 7fbce8a00700 1 -- 172.21.15.57:0/3967479845 wait complete. 2024-02-28T23:13:35.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.253+0000 7fbce8a00700 1 Processor -- start 2024-02-28T23:13:35.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.253+0000 7fbce8a00700 1 -- start start 2024-02-28T23:13:35.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce8a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce8a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 0x7fbce4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce8a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce4198e10 con 0x7fbce4102470 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce8a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce419cba0 con 0x7fbce4108470 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48038/0 (socket says 172.21.15.57:48038) 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce1e00700 1 -- 172.21.15.57:0/2959323514 learned_addr learned my addr 172.21.15.57:0/2959323514 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.254+0000 7fbce1400700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 0x7fbce4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbce1e00700 1 -- 172.21.15.57:0/2959323514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 msgr2=0x7fbce4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbce1e00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 0x7fbce4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbce1e00700 1 -- 172.21.15.57:0/2959323514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00095f0 con 0x7fbce4102470 2024-02-28T23:13:35.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbce1e00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c8a0 tx=0x7fbcd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd400cea0 con 0x7fbce4102470 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce419ce80 con 0x7fbce4102470 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcd4004830 con 0x7fbce4102470 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.255+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd40055b0 con 0x7fbce4102470 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.256+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce419d3d0 con 0x7fbce4102470 2024-02-28T23:13:35.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.257+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce410a970 con 0x7fbce4102470 2024-02-28T23:13:35.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.258+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fbcd40049a0 con 0x7fbce4102470 2024-02-28T23:13:35.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.259+0000 7fbcdaa00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 0x7fbcc806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.259+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbcd408d2d0 con 0x7fbce4102470 2024-02-28T23:13:35.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.259+0000 7fbce1400700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 0x7fbcc806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.260+0000 7fbce1400700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 0x7fbcc806fe80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbce4199810 tx=0x7fbcd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.263+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcd4056990 con 0x7fbce4102470 2024-02-28T23:13:35.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.525+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbce4199690 con 0x7fbcc806d9d0 2024-02-28T23:13:35.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.527+0000 7fbcdaa00700 1 -- 172.21.15.57:0/2959323514 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbcd0005dc0 con 0x7fbcc806d9d0 2024-02-28T23:13:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 msgr2=0x7fbcc806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 0x7fbcc806fe80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbce4199810 tx=0x7fbcd001a040 comp rx=0 tx=0).stop 2024-02-28T23:13:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 msgr2=0x7fbce41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c8a0 tx=0x7fbcd400cc60 comp rx=0 tx=0).stop 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 shutdown_connections 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbcc806d9d0 0x7fbcc806fe80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4102470 0x7fbce41981f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 --2- 172.21.15.57:0/2959323514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbce4108470 0x7fbce4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 >> 172.21.15.57:0/2959323514 conn(0x7fbce40fdf90 msgr2=0x7fbce40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 shutdown_connections 2024-02-28T23:13:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.530+0000 7fbce8a00700 1 -- 172.21.15.57:0/2959323514 wait complete. 2024-02-28T23:13:35.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 -- 172.21.15.57:0/2181047062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 msgr2=0x7f45dc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2181047062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc104eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009910 tx=0x7f45c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:35.701 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:35 smithi057 ceph-mon[89491]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 -- 172.21.15.57:0/2181047062 shutdown_connections 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2181047062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc104eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2181047062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 0x7f45dc068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.701+0000 7f45e0c00700 1 -- 172.21.15.57:0/2181047062 >> 172.21.15.57:0/2181047062 conn(0x7f45dc075130 msgr2=0x7f45dc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.702+0000 7f45e0c00700 1 -- 172.21.15.57:0/2181047062 shutdown_connections 2024-02-28T23:13:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.702+0000 7f45e0c00700 1 -- 172.21.15.57:0/2181047062 wait complete. 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.703+0000 7f45e0c00700 1 Processor -- start 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.703+0000 7f45e0c00700 1 -- start start 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45e0c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 0x7f45dc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45e0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45e0c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc198da0 con 0x7f45dc069140 2024-02-28T23:13:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45e0c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19cb30 con 0x7f45dc0687a0 2024-02-28T23:13:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48052/0 (socket says 172.21.15.57:48052) 2024-02-28T23:13:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45d3e00700 1 -- 172.21.15.57:0/2768893375 learned_addr learned my addr 172.21.15.57:0/2768893375 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.704+0000 7f45d9e00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 0x7f45dc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.705+0000 7f45d3e00700 1 -- 172.21.15.57:0/2768893375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 msgr2=0x7f45dc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.705+0000 7f45d3e00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 0x7f45dc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.705+0000 7f45d3e00700 1 -- 172.21.15.57:0/2768893375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c40095f0 con 0x7f45dc069140 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.705+0000 7f45d3e00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45c400bd70 tx=0x7f45c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.706+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c401d070 con 0x7f45dc069140 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.706+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45c4005dc0 con 0x7f45dc069140 2024-02-28T23:13:35.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.706+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc19cdb0 con 0x7f45dc069140 2024-02-28T23:13:35.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.706+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc19d2a0 con 0x7f45dc069140 2024-02-28T23:13:35.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.706+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c4017400 con 0x7f45dc069140 2024-02-28T23:13:35.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.707+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b8005320 con 0x7f45dc069140 2024-02-28T23:13:35.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.708+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f45c4003a60 con 0x7f45dc069140 2024-02-28T23:13:35.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.709+0000 7f45d2a00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 0x7f45c006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:35.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.709+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f45c408eaf0 con 0x7f45dc069140 2024-02-28T23:13:35.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.710+0000 7f45d9e00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 0x7f45c006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:35.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.710+0000 7f45d9e00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 0x7f45c006fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005950 tx=0x7f45cc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:35.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.714+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45c4058110 con 0x7f45dc069140 2024-02-28T23:13:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:35 smithi084 ceph-mon[70440]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:13:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.965+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f45b8000bf0 con 0x7f45c006d9d0 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:35.999+0000 7f45d2a00700 1 -- 172.21.15.57:0/2768893375 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f45b8000bf0 con 0x7f45c006d9d0 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (3m) 40s ago 17m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (13m) 40s ago 17m 8292k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (15m) 15s ago 15m 8258k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:13:35.999 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (17m) 40s ago 17m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (15m) 15s ago 15m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (2m) 40s ago 16m 75.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (9m) 40s ago 9m 56.8M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (9m) 40s ago 9m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (9m) 15s ago 9m 57.9M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (9m) 15s ago 9m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (5m) 40s ago 18m 477M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (4m) 15s ago 15m 448M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (58s) 40s ago 19m 29.4M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (21s) 15s ago 15m 20.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (4m) 40s ago 17m 8010k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (4m) 15s ago 15m 7759k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (14m) 40s ago 14m 229M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:13:36.000 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (13m) 40s ago 13m 241M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:13:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (13m) 40s ago 13m 202M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:13:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (12m) 15s ago 12m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:13:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (12m) 15s ago 12m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:13:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (11m) 15s ago 11m 232M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:13:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (3m) 40s ago 16m 58.6M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 msgr2=0x7f45c006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 0x7f45c006fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005950 tx=0x7f45cc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 msgr2=0x7f45dc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45c400bd70 tx=0x7f45c400be50 comp rx=0 tx=0).stop 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 shutdown_connections 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f45c006d9d0 0x7f45c006fe80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45dc0687a0 0x7f45dc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 --2- 172.21.15.57:0/2768893375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45dc069140 0x7f45dc1986c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 >> 172.21.15.57:0/2768893375 conn(0x7f45dc075130 msgr2=0x7f45dc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 shutdown_connections 2024-02-28T23:13:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.003+0000 7f45e0c00700 1 -- 172.21.15.57:0/2768893375 wait complete. 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 -- 172.21.15.57:0/1548048733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4100c60 msgr2=0x7f41f4104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 --2- 172.21.15.57:0/1548048733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4100c60 0x7f41f4104c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009910 tx=0x7f41e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 -- 172.21.15.57:0/1548048733 shutdown_connections 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 --2- 172.21.15.57:0/1548048733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4100c60 0x7f41f4104c30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 --2- 172.21.15.57:0/1548048733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f41002c0 0x7f41f4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.166+0000 7f41f3e00700 1 -- 172.21.15.57:0/1548048733 >> 172.21.15.57:0/1548048733 conn(0x7f41f40fbb60 msgr2=0x7f41f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.167+0000 7f41f3e00700 1 -- 172.21.15.57:0/1548048733 shutdown_connections 2024-02-28T23:13:36.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.167+0000 7f41f3e00700 1 -- 172.21.15.57:0/1548048733 wait complete. 2024-02-28T23:13:36.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 Processor -- start 2024-02-28T23:13:36.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 -- start start 2024-02-28T23:13:36.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f4100c60 0x7f41f4196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4196930 con 0x7f41f41002c0 2024-02-28T23:13:36.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.168+0000 7f41f3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f419a6c0 con 0x7f41f4100c60 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48058/0 (socket says 172.21.15.57:48058) 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 -- 172.21.15.57:0/345459944 learned_addr learned my addr 172.21.15.57:0/345459944 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2000700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f4100c60 0x7f41f4196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 -- 172.21.15.57:0/345459944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f4100c60 msgr2=0x7f41f4196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f4100c60 0x7f41f4196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.169+0000 7f41f2a00700 1 -- 172.21.15.57:0/345459944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00095f0 con 0x7f41f41002c0 2024-02-28T23:13:36.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41f2a00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b4b0 tx=0x7f41e400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:36.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e4011800 con 0x7f41f41002c0 2024-02-28T23:13:36.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f419a9a0 con 0x7f41f41002c0 2024-02-28T23:13:36.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41e4011e20 con 0x7f41f41002c0 2024-02-28T23:13:36.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e400f2e0 con 0x7f41f41002c0 2024-02-28T23:13:36.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.170+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f419aef0 con 0x7f41f41002c0 2024-02-28T23:13:36.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.172+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d4005320 con 0x7f41f41002c0 2024-02-28T23:13:36.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.172+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f41e4011960 con 0x7f41f41002c0 2024-02-28T23:13:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.173+0000 7f41eb400700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 0x7f41d806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.174+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f41e408d1b0 con 0x7f41f41002c0 2024-02-28T23:13:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.174+0000 7f41f2000700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 0x7f41d806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.175+0000 7f41f2000700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 0x7f41d806fe80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41f4197330 tx=0x7f41e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.178+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41e4056900 con 0x7f41f41002c0 2024-02-28T23:13:36.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.530+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f41d4005cc0 con 0x7f41f41002c0 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.531+0000 7f41eb400700 1 -- 172.21.15.57:0/345459944 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f41e4059fc0 con 0x7f41f41002c0 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:13:36.531 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:13:36.532 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 msgr2=0x7f41d806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 0x7f41d806fe80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41f4197330 tx=0x7f41e001a040 comp rx=0 tx=0).stop 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 msgr2=0x7f41f4195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b4b0 tx=0x7f41e400b870 comp rx=0 tx=0).stop 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 shutdown_connections 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f41d806d9d0 0x7f41d806fe80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f41002c0 0x7f41f4195d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.534+0000 7f41f3e00700 1 --2- 172.21.15.57:0/345459944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41f4100c60 0x7f41f4196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.535+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 >> 172.21.15.57:0/345459944 conn(0x7f41f40fbb60 msgr2=0x7f41f40fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:36.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.535+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 shutdown_connections 2024-02-28T23:13:36.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.535+0000 7f41f3e00700 1 -- 172.21.15.57:0/345459944 wait complete. 2024-02-28T23:13:36.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 -- 172.21.15.57:0/878459825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044102470 msgr2=0x7f80441028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 --2- 172.21.15.57:0/878459825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044102470 0x7f80441028e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8034009910 tx=0x7f8034009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 -- 172.21.15.57:0/878459825 shutdown_connections 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 --2- 172.21.15.57:0/878459825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044102470 0x7f80441028e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 --2- 172.21.15.57:0/878459825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044108470 0x7f8044108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 -- 172.21.15.57:0/878459825 >> 172.21.15.57:0/878459825 conn(0x7f80440fdf90 msgr2=0x7f80441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 -- 172.21.15.57:0/878459825 shutdown_connections 2024-02-28T23:13:36.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.699+0000 7f804c800700 1 -- 172.21.15.57:0/878459825 wait complete. 2024-02-28T23:13:36.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.700+0000 7f804c800700 1 Processor -- start 2024-02-28T23:13:36.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.700+0000 7f804c800700 1 -- start start 2024-02-28T23:13:36.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f804c800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044102470 0x7f80441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f804c800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f804c800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198e10 con 0x7f8044108470 2024-02-28T23:13:36.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f804c800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804419cba0 con 0x7f8044102470 2024-02-28T23:13:36.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48072/0 (socket says 172.21.15.57:48072) 2024-02-28T23:13:36.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 -- 172.21.15.57:0/3325301737 learned_addr learned my addr 172.21.15.57:0/3325301737 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049e00700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044102470 0x7f80441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 -- 172.21.15.57:0/3325301737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044102470 msgr2=0x7f80441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044102470 0x7f80441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.701+0000 7f8049400700 1 -- 172.21.15.57:0/3325301737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340095f0 con 0x7f8044108470 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f8049400700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f803400b5c0 tx=0x7f803400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803401d070 con 0x7f8044108470 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f803400f460 con 0x7f8044108470 2024-02-28T23:13:36.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804419ce20 con 0x7f8044108470 2024-02-28T23:13:36.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8034017610 con 0x7f8044108470 2024-02-28T23:13:36.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.702+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804419d310 con 0x7f8044108470 2024-02-28T23:13:36.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.703+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804410a970 con 0x7f8044108470 2024-02-28T23:13:36.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.704+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f803400f5d0 con 0x7f8044108470 2024-02-28T23:13:36.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.705+0000 7f803ea00700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 0x7f802c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:36.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.705+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f803408edc0 con 0x7f8044108470 2024-02-28T23:13:36.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.706+0000 7f8049e00700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 0x7f802c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:36.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.706+0000 7f8049e00700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 0x7f802c06fe80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8038005fd0 tx=0x7f803800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:36.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:36.710+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8034092030 con 0x7f8044108470 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: Reconfiguring crash.smithi057 (monmap changed)... 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: Reconfiguring daemon crash.smithi057 on smithi057 2024-02-28T23:13:36.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/345459944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: Reconfiguring crash.smithi057 (monmap changed)... 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: Reconfiguring daemon crash.smithi057 on smithi057 2024-02-28T23:13:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/345459944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:13:37.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.036+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8044199550 con 0x7f8044108470 2024-02-28T23:13:37.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.037+0000 7f803ea00700 1 -- 172.21.15.57:0/3325301737 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f803405be90 con 0x7f8044108470 2024-02-28T23:13:37.037 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:13:37.038 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:13:37.039 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:37.040 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 msgr2=0x7f802c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 0x7f802c06fe80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8038005fd0 tx=0x7f803800a560 comp rx=0 tx=0).stop 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 msgr2=0x7f8044198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f803400b5c0 tx=0x7f803400be00 comp rx=0 tx=0).stop 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 shutdown_connections 2024-02-28T23:13:37.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f802c06d9d0 0x7f802c06fe80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8044102470 0x7f80441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 --2- 172.21.15.57:0/3325301737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8044108470 0x7f8044198730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 >> 172.21.15.57:0/3325301737 conn(0x7f80440fdf90 msgr2=0x7f80440ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.041+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 shutdown_connections 2024-02-28T23:13:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.042+0000 7f804c800700 1 -- 172.21.15.57:0/3325301737 wait complete. 2024-02-28T23:13:37.043 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 -- 172.21.15.57:0/2599176783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 msgr2=0x7f0210108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 --2- 172.21.15.57:0/2599176783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210108840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0204009910 tx=0x7f0204009c20 comp rx=0 tx=0).stop 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 -- 172.21.15.57:0/2599176783 shutdown_connections 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 --2- 172.21.15.57:0/2599176783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f02101028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 --2- 172.21.15.57:0/2599176783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210108840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 -- 172.21.15.57:0/2599176783 >> 172.21.15.57:0/2599176783 conn(0x7f02100fdf90 msgr2=0x7f02101003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 -- 172.21.15.57:0/2599176783 shutdown_connections 2024-02-28T23:13:37.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.220+0000 7f0218000700 1 -- 172.21.15.57:0/2599176783 wait complete. 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.221+0000 7f0218000700 1 Processor -- start 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.221+0000 7f0218000700 1 -- start start 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.221+0000 7f0218000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f0210198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0218000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0218000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210198e30 con 0x7f0210108470 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0218000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019cbc0 con 0x7f0210102470 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48086/0 (socket says 172.21.15.57:48086) 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 -- 172.21.15.57:0/2730155462 learned_addr learned my addr 172.21.15.57:0/2730155462 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0215600700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f0210198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 -- 172.21.15.57:0/2730155462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 msgr2=0x7f0210198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f0210198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0214c00700 1 -- 172.21.15.57:0/2730155462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040095f0 con 0x7f0210108470 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.222+0000 7f0215600700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f0210198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f0214c00700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b4b0 tx=0x7f01fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01fc011800 con 0x7f0210108470 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01fc011e20 con 0x7f0210108470 2024-02-28T23:13:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021019cea0 con 0x7f0210108470 2024-02-28T23:13:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01fc00f2e0 con 0x7f0210108470 2024-02-28T23:13:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.223+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021019d3f0 con 0x7f0210108470 2024-02-28T23:13:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.224+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021010a9e0 con 0x7f0210108470 2024-02-28T23:13:37.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.225+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f01fc010380 con 0x7f0210108470 2024-02-28T23:13:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.226+0000 7f020e000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 0x7f01f806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.226+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f01fc08d120 con 0x7f0210108470 2024-02-28T23:13:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.227+0000 7f0215600700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 0x7f01f806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.227+0000 7f0215600700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 0x7f01f806fe80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f020400b5c0 tx=0x7f020401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.231+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01fc0569a0 con 0x7f0210108470 2024-02-28T23:13:37.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.507+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f021019d6c0 con 0x7f01f806d9d0 2024-02-28T23:13:37.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.509+0000 7f020e000700 1 -- 172.21.15.57:0/2730155462 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f021019d6c0 con 0x7f01f806d9d0 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:13:37.509 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:13:37.510 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:13:37.510 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "4/23 daemons upgraded", 2024-02-28T23:13:37.510 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2024-02-28T23:13:37.510 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:13:37.510 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 msgr2=0x7f01f806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 0x7f01f806fe80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f020400b5c0 tx=0x7f020401a040 comp rx=0 tx=0).stop 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 msgr2=0x7f0210198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b4b0 tx=0x7f01fc00b870 comp rx=0 tx=0).stop 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 shutdown_connections 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f01f806d9d0 0x7f01f806fe80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0210102470 0x7f0210198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 --2- 172.21.15.57:0/2730155462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210108470 0x7f0210198750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.512+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 >> 172.21.15.57:0/2730155462 conn(0x7f02100fdf90 msgr2=0x7f02100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.513+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 shutdown_connections 2024-02-28T23:13:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.513+0000 7f0218000700 1 -- 172.21.15.57:0/2730155462 wait complete. 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 -- 172.21.15.57:0/830896363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 msgr2=0x7efeb0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 --2- 172.21.15.57:0/830896363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0104eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe98009870 tx=0x7efe98009b80 comp rx=0 tx=0).stop 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 -- 172.21.15.57:0/830896363 shutdown_connections 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 --2- 172.21.15.57:0/830896363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0104eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 --2- 172.21.15.57:0/830896363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 0x7efeb0068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 -- 172.21.15.57:0/830896363 >> 172.21.15.57:0/830896363 conn(0x7efeb0075130 msgr2=0x7efeb0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 -- 172.21.15.57:0/830896363 shutdown_connections 2024-02-28T23:13:37.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.682+0000 7efeb7200700 1 -- 172.21.15.57:0/830896363 wait complete. 2024-02-28T23:13:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 Processor -- start 2024-02-28T23:13:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 -- start start 2024-02-28T23:13:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 0x7efeb0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0198c70 con 0x7efeb0069140 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.684+0000 7efeb7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019ca00 con 0x7efeb00687a0 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48106/0 (socket says 172.21.15.57:48106) 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 -- 172.21.15.57:0/928974835 learned_addr learned my addr 172.21.15.57:0/928974835 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efeafe00700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 0x7efeb0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 -- 172.21.15.57:0/928974835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 msgr2=0x7efeb0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 0x7efeb0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 -- 172.21.15.57:0/928974835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98009520 con 0x7efeb0069140 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.685+0000 7efea7e00700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efe98004700 tx=0x7efe9800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.686+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9801c070 con 0x7efeb0069140 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.686+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe9800fbf0 con 0x7efeb0069140 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.686+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb019cc80 con 0x7efeb0069140 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.686+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe98022df0 con 0x7efeb0069140 2024-02-28T23:13:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.686+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb019d170 con 0x7efeb0069140 2024-02-28T23:13:37.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.687+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb004f2a0 con 0x7efeb0069140 2024-02-28T23:13:37.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.688+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7efe98020070 con 0x7efeb0069140 2024-02-28T23:13:37.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.689+0000 7efead600700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 0x7efe9406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:13:37.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.689+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efe9808e760 con 0x7efeb0069140 2024-02-28T23:13:37.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.689+0000 7efeafe00700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 0x7efe9406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:13:37.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.690+0000 7efeafe00700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 0x7efe9406fe80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efea0005fd0 tx=0x7efea0009310 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:13:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:37.694+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe98057ed0 con 0x7efeb0069140 2024-02-28T23:13:37.805 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:37 smithi057 ceph-mon[89491]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:37.805 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:37 smithi057 ceph-mon[89491]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:37.805 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3325301737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:13:38.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.026+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efeb019d450 con 0x7efeb0069140 2024-02-28T23:13:38.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.027+0000 7efead600700 1 -- 172.21.15.57:0/928974835 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efe9805b590 con 0x7efeb0069140 2024-02-28T23:13:38.026 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:13:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.029+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 msgr2=0x7efe9406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.029+0000 7efeb7200700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 0x7efe9406fe80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efea0005fd0 tx=0x7efea0009310 comp rx=0 tx=0).stop 2024-02-28T23:13:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.029+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 msgr2=0x7efeb0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:13:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.029+0000 7efeb7200700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efe98004700 tx=0x7efe9800b640 comp rx=0 tx=0).stop 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 shutdown_connections 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d9d0 0x7efe9406fe80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb00687a0 0x7efeb0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 --2- 172.21.15.57:0/928974835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb0069140 0x7efeb0198590 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 >> 172.21.15.57:0/928974835 conn(0x7efeb0075130 msgr2=0x7efeb0101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 shutdown_connections 2024-02-28T23:13:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:13:38.030+0000 7efeb7200700 1 -- 172.21.15.57:0/928974835 wait complete. 2024-02-28T23:13:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:37 smithi084 ceph-mon[70440]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:37 smithi084 ceph-mon[70440]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3325301737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:13:38.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:38.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:38.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:38.965 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T23:13:38.966 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:38.966 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:38 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/928974835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:38 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/928974835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:13:40.021 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:39 smithi057 ceph-mon[89491]: Reconfiguring osd.0 (monmap changed)... 2024-02-28T23:13:40.022 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:39 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.0 on smithi057 2024-02-28T23:13:40.022 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:39 smithi057 ceph-mon[89491]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:39 smithi084 ceph-mon[70440]: Reconfiguring osd.0 (monmap changed)... 2024-02-28T23:13:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:39 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.0 on smithi057 2024-02-28T23:13:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:39 smithi084 ceph-mon[70440]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: Reconfiguring osd.1 (monmap changed)... 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:41.559 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:41 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.1 on smithi057 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: Reconfiguring osd.1 (monmap changed)... 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:41 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.1 on smithi057 2024-02-28T23:13:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:43 smithi084 ceph-mon[70440]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:43 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:43 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:43 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:13:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:43 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:43.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:43 smithi057 ceph-mon[89491]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:43.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:43 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:43.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:43 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:43.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:43 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:13:43.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:43 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:44.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:44 smithi057 ceph-mon[89491]: Reconfiguring osd.2 (monmap changed)... 2024-02-28T23:13:44.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:44 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.2 on smithi057 2024-02-28T23:13:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:44 smithi084 ceph-mon[70440]: Reconfiguring osd.2 (monmap changed)... 2024-02-28T23:13:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:44 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.2 on smithi057 2024-02-28T23:13:45.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:45 smithi057 ceph-mon[89491]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:45 smithi084 ceph-mon[70440]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: Reconfiguring mds.cephfs.smithi057.bgmggd (monmap changed)... 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:46.689 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:46 smithi057 ceph-mon[89491]: Reconfiguring daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:13:46.819 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:46.820 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:46.820 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: Reconfiguring mds.cephfs.smithi057.bgmggd (monmap changed)... 2024-02-28T23:13:46.820 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:13:46.820 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:46.820 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:46 smithi084 ceph-mon[70440]: Reconfiguring daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:13:47.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:47 smithi057 ceph-mon[89491]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:47 smithi084 ceph-mon[70440]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:49.006 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:49.006 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:49.006 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:13:49.006 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:49.006 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:13:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:13:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:49 smithi057 ceph-mon[89491]: Reconfiguring mds.cephfs.smithi057.sbkaml (monmap changed)... 2024-02-28T23:13:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:49 smithi057 ceph-mon[89491]: Reconfiguring daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:13:49.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:49 smithi057 ceph-mon[89491]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:49 smithi084 ceph-mon[70440]: Reconfiguring mds.cephfs.smithi057.sbkaml (monmap changed)... 2024-02-28T23:13:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:49 smithi084 ceph-mon[70440]: Reconfiguring daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:13:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:49 smithi084 ceph-mon[70440]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: Reconfiguring ceph-exporter.smithi084 (monmap changed)... 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: Unable to update caps for client.ceph-exporter.smithi084 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi084"}]: dispatch 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:51 smithi084 ceph-mon[70440]: Reconfiguring daemon ceph-exporter.smithi084 on smithi084 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: Reconfiguring ceph-exporter.smithi084 (monmap changed)... 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:13:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-28T23:13:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: Unable to update caps for client.ceph-exporter.smithi084 2024-02-28T23:13:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi084"}]: dispatch 2024-02-28T23:13:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:51 smithi057 ceph-mon[89491]: Reconfiguring daemon ceph-exporter.smithi084 on smithi084 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: Reconfiguring crash.smithi084 (monmap changed)... 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:53.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:53 smithi084 ceph-mon[70440]: Reconfiguring daemon crash.smithi084 on smithi084 2024-02-28T23:13:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: Reconfiguring crash.smithi084 (monmap changed)... 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:53 smithi057 ceph-mon[89491]: Reconfiguring daemon crash.smithi084 on smithi084 2024-02-28T23:13:55.712 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:55.712 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:55.712 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:55.712 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:13:55.712 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:13:55.713 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:55.718 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:13:55.718 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:55.718 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:55.718 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi084.fhkzis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-28T23:13:55.718 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-28T23:13:55.719 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:56.748 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:56 smithi084 ceph-mon[70440]: Reconfiguring mgr.smithi084.fhkzis (monmap changed)... 2024-02-28T23:13:56.748 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:56 smithi084 ceph-mon[70440]: Reconfiguring daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:13:56.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:56 smithi057 ceph-mon[89491]: Reconfiguring mgr.smithi084.fhkzis (monmap changed)... 2024-02-28T23:13:56.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:56 smithi057 ceph-mon[89491]: Reconfiguring daemon mgr.smithi084.fhkzis on smithi084 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:57.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:57 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-28T23:13:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:57 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:58.759 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:58 smithi084 ceph-mon[70440]: Reconfiguring mon.smithi084 (monmap changed)... 2024-02-28T23:13:58.759 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:58 smithi084 ceph-mon[70440]: Reconfiguring daemon mon.smithi084 on smithi084 2024-02-28T23:13:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:58 smithi057 ceph-mon[89491]: Reconfiguring mon.smithi084 (monmap changed)... 2024-02-28T23:13:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:58 smithi057 ceph-mon[89491]: Reconfiguring daemon mon.smithi084 on smithi084 2024-02-28T23:13:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:59 smithi057 ceph-mon[89491]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:13:59.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:13:59 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:13:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:59 smithi084 ceph-mon[70440]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:13:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:59 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:59 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:13:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:59 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:13:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:13:59 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:00.707 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:00 smithi084 ceph-mon[70440]: Reconfiguring osd.3 (monmap changed)... 2024-02-28T23:14:00.708 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:00 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.3 on smithi084 2024-02-28T23:14:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:00 smithi057 ceph-mon[89491]: Reconfiguring osd.3 (monmap changed)... 2024-02-28T23:14:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:00 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.3 on smithi084 2024-02-28T23:14:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:01 smithi057 ceph-mon[89491]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:01 smithi084 ceph-mon[70440]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: Reconfiguring osd.4 (monmap changed)... 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:02.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:02 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.4 on smithi084 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: Reconfiguring osd.4 (monmap changed)... 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:02 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.4 on smithi084 2024-02-28T23:14:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:03 smithi057 ceph-mon[89491]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:03 smithi084 ceph-mon[70440]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: Reconfiguring osd.5 (monmap changed)... 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: Reconfiguring daemon osd.5 on smithi084 2024-02-28T23:14:04.998 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:04 smithi084 ceph-mon[70440]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:05.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: Reconfiguring osd.5 (monmap changed)... 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: Reconfiguring daemon osd.5 on smithi084 2024-02-28T23:14:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:04 smithi057 ceph-mon[89491]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: Reconfiguring mds.cephfs.smithi084.ztfxpc (monmap changed)... 2024-02-28T23:14:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:14:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:07 smithi084 ceph-mon[70440]: Reconfiguring daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:14:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: Reconfiguring mds.cephfs.smithi084.ztfxpc (monmap changed)... 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:07 smithi057 ceph-mon[89491]: Reconfiguring daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.209+0000 7fc4d6600700 1 -- 172.21.15.57:0/1822919275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 msgr2=0x7fc4d01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.209+0000 7fc4d6600700 1 --2- 172.21.15.57:0/1822919275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01028e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009910 tx=0x7fc4bc009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 -- 172.21.15.57:0/1822919275 shutdown_connections 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 --2- 172.21.15.57:0/1822919275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01028e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 --2- 172.21.15.57:0/1822919275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 0x7fc4d0108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 -- 172.21.15.57:0/1822919275 >> 172.21.15.57:0/1822919275 conn(0x7fc4d00fdf90 msgr2=0x7fc4d01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 -- 172.21.15.57:0/1822919275 shutdown_connections 2024-02-28T23:14:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.210+0000 7fc4d6600700 1 -- 172.21.15.57:0/1822919275 wait complete. 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 Processor -- start 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 -- start start 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 0x7fc4d0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0198d80 con 0x7fc4d0102470 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.211+0000 7fc4d6600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0198ec0 con 0x7fc4d0108470 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55830/0 (socket says 172.21.15.57:55830) 2024-02-28T23:14:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 -- 172.21.15.57:0/2806127154 learned_addr learned my addr 172.21.15.57:0/2806127154 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cea00700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 0x7fc4d0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 -- 172.21.15.57:0/2806127154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 msgr2=0x7fc4d0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 0x7fc4d0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.212+0000 7fc4cf400700 1 -- 172.21.15.57:0/2806127154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0095f0 con 0x7fc4d0102470 2024-02-28T23:14:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4cf400700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c930 tx=0x7fc4c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c0007ad0 con 0x7fc4d0102470 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4c0007c30 con 0x7fc4d0102470 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d019cd10 con 0x7fc4d0102470 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c0018610 con 0x7fc4d0102470 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.213+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d019d260 con 0x7fc4d0102470 2024-02-28T23:14:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.214+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d010a970 con 0x7fc4d0102470 2024-02-28T23:14:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.216+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fc4c000f450 con 0x7fc4d0102470 2024-02-28T23:14:08.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.216+0000 7fc4c7e00700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 0x7fc4b406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.217+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc4c008d9f0 con 0x7fc4d0102470 2024-02-28T23:14:08.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.217+0000 7fc4cea00700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 0x7fc4b406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.218+0000 7fc4cea00700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 0x7fc4b406fe80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0199810 tx=0x7fc4bc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:08.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.222+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4c0056f90 con 0x7fc4d0102470 2024-02-28T23:14:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.469+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4d019d4e0 con 0x7fc4b406d9d0 2024-02-28T23:14:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.472+0000 7fc4c7e00700 1 -- 172.21.15.57:0/2806127154 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc4d019d4e0 con 0x7fc4b406d9d0 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 msgr2=0x7fc4b406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 0x7fc4b406fe80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0199810 tx=0x7fc4bc01a040 comp rx=0 tx=0).stop 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 msgr2=0x7fc4d01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c930 tx=0x7fc4c000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 shutdown_connections 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc4b406d9d0 0x7fc4b406fe80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4d0102470 0x7fc4d01981f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 --2- 172.21.15.57:0/2806127154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4d0108470 0x7fc4d0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 >> 172.21.15.57:0/2806127154 conn(0x7fc4d00fdf90 msgr2=0x7fc4d00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.474+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 shutdown_connections 2024-02-28T23:14:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.475+0000 7fc4d6600700 1 -- 172.21.15.57:0/2806127154 wait complete. 2024-02-28T23:14:08.489 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:14:08.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.644+0000 7f9b03e00700 1 -- 172.21.15.57:0/170014885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b041024d0 msgr2=0x7f9b041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.644+0000 7f9b03e00700 1 --2- 172.21.15.57:0/170014885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b041024d0 0x7f9b041028a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009910 tx=0x7f9af4009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 -- 172.21.15.57:0/170014885 shutdown_connections 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 --2- 172.21.15.57:0/170014885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b04102e70 0x7f9b04106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 --2- 172.21.15.57:0/170014885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b041024d0 0x7f9b041028a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 -- 172.21.15.57:0/170014885 >> 172.21.15.57:0/170014885 conn(0x7f9b040fdd90 msgr2=0x7f9b041001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 -- 172.21.15.57:0/170014885 shutdown_connections 2024-02-28T23:14:08.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.645+0000 7f9b03e00700 1 -- 172.21.15.57:0/170014885 wait complete. 2024-02-28T23:14:08.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 Processor -- start 2024-02-28T23:14:08.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 -- start start 2024-02-28T23:14:08.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b041024d0 0x7f9b04197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04198a80 con 0x7f9b04102e70 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.646+0000 7f9b03e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0419c810 con 0x7f9b041024d0 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55834/0 (socket says 172.21.15.57:55834) 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 -- 172.21.15.57:0/435447845 learned_addr learned my addr 172.21.15.57:0/435447845 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02a00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b041024d0 0x7f9b04197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 -- 172.21.15.57:0/435447845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b041024d0 msgr2=0x7f9b04197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b041024d0 0x7f9b04197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 -- 172.21.15.57:0/435447845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40095f0 con 0x7f9b04102e70 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.647+0000 7f9b02000700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d8d0 tx=0x7f9af000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.648+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af00041f0 con 0x7f9b04102e70 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.648+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9af0007620 con 0x7f9b04102e70 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.648+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af00056b0 con 0x7f9b04102e70 2024-02-28T23:14:08.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.648+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0419caf0 con 0x7f9b04102e70 2024-02-28T23:14:08.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.648+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0419cff0 con 0x7f9b04102e70 2024-02-28T23:14:08.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.649+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0410a7e0 con 0x7f9b04102e70 2024-02-28T23:14:08.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.650+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f9af0004750 con 0x7f9b04102e70 2024-02-28T23:14:08.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.651+0000 7f9afb400700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 0x7f9ae806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:08.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.651+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9af008d530 con 0x7f9b04102e70 2024-02-28T23:14:08.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.651+0000 7f9b02a00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 0x7f9ae806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:08.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.652+0000 7f9b02a00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 0x7f9ae806fe90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9af4006010 tx=0x7f9af401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:08.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.656+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9af0056b70 con 0x7f9b04102e70 2024-02-28T23:14:08.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.900+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b0419d320 con 0x7f9ae806d9e0 2024-02-28T23:14:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.902+0000 7f9afb400700 1 -- 172.21.15.57:0/435447845 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9b0419d320 con 0x7f9ae806d9e0 2024-02-28T23:14:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.905+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 msgr2=0x7f9ae806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.905+0000 7f9b03e00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 0x7f9ae806fe90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9af4006010 tx=0x7f9af401a040 comp rx=0 tx=0).stop 2024-02-28T23:14:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.905+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 msgr2=0x7f9b041983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.905+0000 7f9b03e00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d8d0 tx=0x7f9af000dbe0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 shutdown_connections 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ae806d9e0 0x7f9ae806fe90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b041024d0 0x7f9b04197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 --2- 172.21.15.57:0/435447845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b04102e70 0x7f9b041983a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 >> 172.21.15.57:0/435447845 conn(0x7f9b040fdd90 msgr2=0x7f9b04100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 shutdown_connections 2024-02-28T23:14:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:08.906+0000 7f9b03e00700 1 -- 172.21.15.57:0/435447845 wait complete. 2024-02-28T23:14:09.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.068+0000 7f9695800700 1 -- 172.21.15.57:0/2404994432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690102470 msgr2=0x7f96901028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.068+0000 7f9695800700 1 --2- 172.21.15.57:0/2404994432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690102470 0x7f96901028e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f967800b0a0 tx=0x7f967800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 -- 172.21.15.57:0/2404994432 shutdown_connections 2024-02-28T23:14:09.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 --2- 172.21.15.57:0/2404994432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690102470 0x7f96901028e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 --2- 172.21.15.57:0/2404994432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690108470 0x7f9690108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 -- 172.21.15.57:0/2404994432 >> 172.21.15.57:0/2404994432 conn(0x7f96900fdf90 msgr2=0x7f96901003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:09.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 -- 172.21.15.57:0/2404994432 shutdown_connections 2024-02-28T23:14:09.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.069+0000 7f9695800700 1 -- 172.21.15.57:0/2404994432 wait complete. 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 Processor -- start 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 -- start start 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690102470 0x7f96901981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9690198e10 con 0x7f9690108470 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.070+0000 7f9695800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969019cba0 con 0x7f9690102470 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55846/0 (socket says 172.21.15.57:55846) 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 -- 172.21.15.57:0/2962203898 learned_addr learned my addr 172.21.15.57:0/2962203898 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f968ea00700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690102470 0x7f96901981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 -- 172.21.15.57:0/2962203898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690102470 msgr2=0x7f96901981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690102470 0x7f96901981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.071+0000 7f9686000700 1 -- 172.21.15.57:0/2962203898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009d00 con 0x7f9690108470 2024-02-28T23:14:09.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9686000700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9678000f80 tx=0x7f967800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967800e040 con 0x7f9690108470 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9678003e60 con 0x7f9690108470 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969019ce20 con 0x7f9690108470 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967801d910 con 0x7f9690108470 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.072+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969019d310 con 0x7f9690108470 2024-02-28T23:14:09.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.073+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969010a970 con 0x7f9690108470 2024-02-28T23:14:09.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.074+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f9678004820 con 0x7f9690108470 2024-02-28T23:14:09.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.075+0000 7f9687e00700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 0x7f967406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.075+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f967808ef40 con 0x7f9690108470 2024-02-28T23:14:09.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.075+0000 7f968ea00700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 0x7f967406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.076+0000 7f968ea00700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 0x7f967406fe90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f968000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.080+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96780c2d20 con 0x7f9690108470 2024-02-28T23:14:09.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.321+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f96901995c0 con 0x7f967406d9e0 2024-02-28T23:14:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.334+0000 7f9687e00700 1 -- 172.21.15.57:0/2962203898 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f96901995c0 con 0x7f967406d9e0 2024-02-28T23:14:09.333 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:14:09.333 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (3m) 73s ago 17m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:14:09.333 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (13m) 73s ago 18m 8292k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:14:09.333 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (16m) 48s ago 16m 8258k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:14:09.333 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (17m) 73s ago 17m 6669k - 18.2.0 dc2bc1663786 dcc43cde880c 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (16m) 48s ago 16m 6665k - 18.2.0 dc2bc1663786 74e652bd1cdc 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (3m) 73s ago 17m 75.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (10m) 73s ago 10m 56.8M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (10m) 73s ago 10m 115M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (10m) 48s ago 10m 57.9M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (10m) 48s ago 10m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (6m) 73s ago 19m 477M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (5m) 48s ago 16m 448M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (91s) 73s ago 19m 29.4M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (54s) 48s ago 16m 20.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (4m) 73s ago 17m 8010k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (4m) 48s ago 16m 7759k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (15m) 73s ago 15m 229M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:14:09.334 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (14m) 73s ago 14m 241M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:14:09.335 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (13m) 73s ago 13m 202M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:14:09.335 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (13m) 48s ago 13m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:14:09.335 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (12m) 48s ago 12m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:14:09.335 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (12m) 48s ago 12m 232M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:14:09.335 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (4m) 73s ago 16m 58.6M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 msgr2=0x7f967406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 0x7f967406fe90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f968000a560 comp rx=0 tx=0).stop 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 msgr2=0x7f9690198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9678000f80 tx=0x7f967800cdd0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 shutdown_connections 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f967406d9e0 0x7f967406fe90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9690102470 0x7f96901981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 --2- 172.21.15.57:0/2962203898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9690108470 0x7f9690198730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 >> 172.21.15.57:0/2962203898 conn(0x7f96900fdf90 msgr2=0x7f96900ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 shutdown_connections 2024-02-28T23:14:09.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.338+0000 7f9695800700 1 -- 172.21.15.57:0/2962203898 wait complete. 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 -- 172.21.15.57:0/3594175155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec10c8f0 msgr2=0x7f64ec10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 --2- 172.21.15.57:0/3594175155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec10c8f0 0x7f64ec10ccc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009910 tx=0x7f64dc009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 -- 172.21.15.57:0/3594175155 shutdown_connections 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 --2- 172.21.15.57:0/3594175155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec106b80 0x7f64ec106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 --2- 172.21.15.57:0/3594175155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec10c8f0 0x7f64ec10ccc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 -- 172.21.15.57:0/3594175155 >> 172.21.15.57:0/3594175155 conn(0x7f64ec076060 msgr2=0x7f64ec078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 -- 172.21.15.57:0/3594175155 shutdown_connections 2024-02-28T23:14:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.495+0000 7f64f3000700 1 -- 172.21.15.57:0/3594175155 wait complete. 2024-02-28T23:14:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.496+0000 7f64f3000700 1 Processor -- start 2024-02-28T23:14:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.496+0000 7f64f3000700 1 -- start start 2024-02-28T23:14:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64f3000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64f3000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec10c8f0 0x7f64ec19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64f3000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec19d120 con 0x7f64ec106b80 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64f3000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1a0eb0 con 0x7f64ec10c8f0 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55872/0 (socket says 172.21.15.57:55872) 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 -- 172.21.15.57:0/3714977571 learned_addr learned my addr 172.21.15.57:0/3714977571 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:09.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64eb400700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec10c8f0 0x7f64ec19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 -- 172.21.15.57:0/3714977571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec10c8f0 msgr2=0x7f64ec19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec10c8f0 0x7f64ec19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.497+0000 7f64ebe00700 1 -- 172.21.15.57:0/3714977571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0095f0 con 0x7f64ec106b80 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64ebe00700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b5c0 tx=0x7f64dc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc01d070 con 0x7f64ec106b80 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64dc00f460 con 0x7f64ec106b80 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec1a1130 con 0x7f64ec106b80 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc017610 con 0x7f64ec106b80 2024-02-28T23:14:09.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.498+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec1a1620 con 0x7f64ec106b80 2024-02-28T23:14:09.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.499+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec109b20 con 0x7f64ec106b80 2024-02-28T23:14:09.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.500+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f64dc00f5d0 con 0x7f64ec106b80 2024-02-28T23:14:09.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.501+0000 7f64e8c00700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 0x7f64d006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.501+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f64dc08ec70 con 0x7f64ec106b80 2024-02-28T23:14:09.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.501+0000 7f64eb400700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 0x7f64d006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.502+0000 7f64eb400700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 0x7f64d006fe80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005fd0 tx=0x7f64d800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.506+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64dc058330 con 0x7f64ec106b80 2024-02-28T23:14:09.704 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:09.704 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: Reconfiguring mds.cephfs.smithi084.dadqfu (monmap changed)... 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:09.705 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:09 smithi084 ceph-mon[70440]: Reconfiguring daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:14:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.817+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f64ec19d960 con 0x7f64ec106b80 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: Reconfiguring mds.cephfs.smithi084.dadqfu (monmap changed)... 2024-02-28T23:14:09.816 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:14:09.817 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:09.817 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:09 smithi057 ceph-mon[89491]: Reconfiguring daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:14:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.818+0000 7f64e8c00700 1 -- 172.21.15.57:0/3714977571 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f64dc05b9f0 con 0x7f64ec106b80 2024-02-28T23:14:09.817 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:14:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:14:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:14:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:14:09.818 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 msgr2=0x7f64d006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 0x7f64d006fe80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005fd0 tx=0x7f64d800a560 comp rx=0 tx=0).stop 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 msgr2=0x7f64ec19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b5c0 tx=0x7f64dc00be00 comp rx=0 tx=0).stop 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 shutdown_connections 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f64d006d9d0 0x7f64d006fe80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64ec106b80 0x7f64ec19c500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 --2- 172.21.15.57:0/3714977571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64ec10c8f0 0x7f64ec19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 >> 172.21.15.57:0/3714977571 conn(0x7f64ec076060 msgr2=0x7f64ec077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 shutdown_connections 2024-02-28T23:14:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.821+0000 7f64f3000700 1 -- 172.21.15.57:0/3714977571 wait complete. 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 -- 172.21.15.57:0/3238739911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54101004e0 msgr2=0x7f54101008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 --2- 172.21.15.57:0/3238739911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54101004e0 0x7f54101008b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5404009910 tx=0x7f5404009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 -- 172.21.15.57:0/3238739911 shutdown_connections 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 --2- 172.21.15.57:0/3238739911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5410100e80 0x7f5410104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 --2- 172.21.15.57:0/3238739911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54101004e0 0x7f54101008b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 -- 172.21.15.57:0/3238739911 >> 172.21.15.57:0/3238739911 conn(0x7f54100fbda0 msgr2=0x7f54100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 -- 172.21.15.57:0/3238739911 shutdown_connections 2024-02-28T23:14:09.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.982+0000 7f5417e00700 1 -- 172.21.15.57:0/3238739911 wait complete. 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.983+0000 7f5417e00700 1 Processor -- start 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.983+0000 7f5417e00700 1 -- start start 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5417e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54101004e0 0x7f5410073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5417e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5417e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410073ad0 con 0x7f5410100e80 2024-02-28T23:14:09.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5417e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410073c10 con 0x7f54101004e0 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5414a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5414a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55898/0 (socket says 172.21.15.57:55898) 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5414a00700 1 -- 172.21.15.57:0/3533490419 learned_addr learned my addr 172.21.15.57:0/3533490419 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.984+0000 7f5415400700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54101004e0 0x7f5410073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5414a00700 1 -- 172.21.15.57:0/3533490419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54101004e0 msgr2=0x7f5410073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5414a00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54101004e0 0x7f5410073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5414a00700 1 -- 172.21.15.57:0/3533490419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040095f0 con 0x7f5410100e80 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5414a00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b480 tx=0x7f53fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53fc0118c0 con 0x7f5410100e80 2024-02-28T23:14:09.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53fc010380 con 0x7f5410100e80 2024-02-28T23:14:09.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54101a2160 con 0x7f5410100e80 2024-02-28T23:14:09.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53fc00f4f0 con 0x7f5410100e80 2024-02-28T23:14:09.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.985+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101a2570 con 0x7f5410100e80 2024-02-28T23:14:09.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.986+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54101087f0 con 0x7f5410100e80 2024-02-28T23:14:09.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.988+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f53fc0104f0 con 0x7f5410100e80 2024-02-28T23:14:09.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.988+0000 7f540e000700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 0x7f53f806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:09.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.989+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f53fc08d200 con 0x7f5410100e80 2024-02-28T23:14:09.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.989+0000 7f5415400700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 0x7f53f806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:09.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.990+0000 7f5415400700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 0x7f53f806fe80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5404006010 tx=0x7f540401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:09.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:09.993+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53fc056930 con 0x7f5410100e80 2024-02-28T23:14:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.276+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f54101081d0 con 0x7f5410100e80 2024-02-28T23:14:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.277+0000 7f540e000700 1 -- 172.21.15.57:0/3533490419 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f53fc059ff0 con 0x7f5410100e80 2024-02-28T23:14:10.276 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:14:10.276 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:14:10.276 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:14:10.277 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:14:10.278 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:10.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 msgr2=0x7f53f806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 0x7f53f806fe80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5404006010 tx=0x7f540401a040 comp rx=0 tx=0).stop 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 msgr2=0x7f5410073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b480 tx=0x7f53fc00b840 comp rx=0 tx=0).stop 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 shutdown_connections 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f53f806d9d0 0x7f53f806fe80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54101004e0 0x7f5410073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 --2- 172.21.15.57:0/3533490419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5410100e80 0x7f5410073590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 >> 172.21.15.57:0/3533490419 conn(0x7f54100fbda0 msgr2=0x7f54100fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 shutdown_connections 2024-02-28T23:14:10.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.280+0000 7f5417e00700 1 -- 172.21.15.57:0/3533490419 wait complete. 2024-02-28T23:14:10.281 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:14:10.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.446+0000 7f7a13800700 1 -- 172.21.15.57:0/3830447921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c10c8f0 msgr2=0x7f7a0c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.446+0000 7f7a13800700 1 --2- 172.21.15.57:0/3830447921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c10ccc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009910 tx=0x7f7a00009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:10.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 -- 172.21.15.57:0/3830447921 shutdown_connections 2024-02-28T23:14:10.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 --2- 172.21.15.57:0/3830447921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c106b80 0x7f7a0c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 --2- 172.21.15.57:0/3830447921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c10ccc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 -- 172.21.15.57:0/3830447921 >> 172.21.15.57:0/3830447921 conn(0x7f7a0c076060 msgr2=0x7f7a0c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:10.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 -- 172.21.15.57:0/3830447921 shutdown_connections 2024-02-28T23:14:10.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.447+0000 7f7a13800700 1 -- 172.21.15.57:0/3830447921 wait complete. 2024-02-28T23:14:10.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.448+0000 7f7a13800700 1 Processor -- start 2024-02-28T23:14:10.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.448+0000 7f7a13800700 1 -- start start 2024-02-28T23:14:10.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a13800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a13800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a13800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c19d120 con 0x7f7a0c106b80 2024-02-28T23:14:10.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a13800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1a0eb0 con 0x7f7a0c10c8f0 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a10e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a10e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55920/0 (socket says 172.21.15.57:55920) 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.449+0000 7f7a10e00700 1 -- 172.21.15.57:0/497538068 learned_addr learned my addr 172.21.15.57:0/497538068 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a0be00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a10e00700 1 -- 172.21.15.57:0/497538068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 msgr2=0x7f7a0c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a10e00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a10e00700 1 -- 172.21.15.57:0/497538068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000095f0 con 0x7f7a0c106b80 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a0be00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a10e00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000b5c0 tx=0x7f7a0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0001d070 con 0x7f7a0c106b80 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a00005e00 con 0x7f7a0c106b80 2024-02-28T23:14:10.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a00017400 con 0x7f7a0c106b80 2024-02-28T23:14:10.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.450+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c1a1130 con 0x7f7a0c106b80 2024-02-28T23:14:10.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.451+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c1a1620 con 0x7f7a0c106b80 2024-02-28T23:14:10.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.452+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c109b20 con 0x7f7a0c106b80 2024-02-28T23:14:10.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.453+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f7a0000f460 con 0x7f7a0c106b80 2024-02-28T23:14:10.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.454+0000 7f7a09600700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 0x7f79f406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.454+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7a0008ebf0 con 0x7f7a0c106b80 2024-02-28T23:14:10.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.454+0000 7f7a0be00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 0x7f79f406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.455+0000 7f7a0be00700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 0x7f79f406fe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005fd0 tx=0x7f79f8005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:10.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.459+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a00058210 con 0x7f7a0c106b80 2024-02-28T23:14:10.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:10 smithi057 ceph-mon[89491]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:10.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:10 smithi057 ceph-mon[89491]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:10.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3714977571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3533490419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:14:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.713+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a0c1a1950 con 0x7f79f406d9d0 2024-02-28T23:14:10.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.715+0000 7f7a09600700 1 -- 172.21.15.57:0/497538068 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7a0c1a1950 con 0x7f79f406d9d0 2024-02-28T23:14:10.714 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:14:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:14:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:14:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "4/23 daemons upgraded", 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mon daemons", 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:14:10.715 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 msgr2=0x7f79f406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 0x7f79f406fe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005fd0 tx=0x7f79f8005ee0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 msgr2=0x7f7a0c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000b5c0 tx=0x7f7a0000be20 comp rx=0 tx=0).stop 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 shutdown_connections 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79f406d9d0 0x7f79f406fe80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a0c106b80 0x7f7a0c19c500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 --2- 172.21.15.57:0/497538068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a0c10c8f0 0x7f7a0c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 >> 172.21.15.57:0/497538068 conn(0x7f7a0c076060 msgr2=0x7f7a0c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:10.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.718+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 shutdown_connections 2024-02-28T23:14:10.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.719+0000 7f7a13800700 1 -- 172.21.15.57:0/497538068 wait complete. 2024-02-28T23:14:10.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:10 smithi084 ceph-mon[70440]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:10.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:10 smithi084 ceph-mon[70440]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:10.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3714977571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:10.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3533490419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:14:10.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 -- 172.21.15.57:0/3437734129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00687a0 msgr2=0x7f0ad0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/3437734129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00687a0 0x7f0ad0068b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009910 tx=0x7f0ac0009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:10.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 -- 172.21.15.57:0/3437734129 shutdown_connections 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/3437734129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0069140 0x7f0ad0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/3437734129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00687a0 0x7f0ad0068b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 -- 172.21.15.57:0/3437734129 >> 172.21.15.57:0/3437734129 conn(0x7f0ad0075130 msgr2=0x7f0ad0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 -- 172.21.15.57:0/3437734129 shutdown_connections 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.896+0000 7f0ad4e00700 1 -- 172.21.15.57:0/3437734129 wait complete. 2024-02-28T23:14:10.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.897+0000 7f0ad4e00700 1 Processor -- start 2024-02-28T23:14:10.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0ad4e00700 1 -- start start 2024-02-28T23:14:10.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0ad4e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad00687a0 0x7f0ad01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0ad4e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0ad4e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0198cd0 con 0x7f0ad0069140 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0ad4e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad019ca60 con 0x7f0ad00687a0 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0acd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0acd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55938/0 (socket says 172.21.15.57:55938) 2024-02-28T23:14:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.898+0000 7f0acd400700 1 -- 172.21.15.57:0/2156975878 learned_addr learned my addr 172.21.15.57:0/2156975878 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0acde00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad00687a0 0x7f0ad01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0acd400700 1 -- 172.21.15.57:0/2156975878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad00687a0 msgr2=0x7f0ad01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0acd400700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad00687a0 0x7f0ad01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0acd400700 1 -- 172.21.15.57:0/2156975878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00095f0 con 0x7f0ad0069140 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0acd400700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c8f0 tx=0x7f0abc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0abc007a50 con 0x7f0ad0069140 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0abc007bb0 con 0x7f0ad0069140 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0abc018630 con 0x7f0ad0069140 2024-02-28T23:14:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.899+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad019cd40 con 0x7f0ad0069140 2024-02-28T23:14:10.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.900+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad019d290 con 0x7f0ad0069140 2024-02-28T23:14:10.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.901+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad004f2a0 con 0x7f0ad0069140 2024-02-28T23:14:10.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.902+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f0abc007d20 con 0x7f0ad0069140 2024-02-28T23:14:10.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.903+0000 7f0ac6a00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 0x7f0ab406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:10.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.903+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0abc08d780 con 0x7f0ad0069140 2024-02-28T23:14:10.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.903+0000 7f0acde00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 0x7f0ab406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:10.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.904+0000 7f0acde00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 0x7f0ab406fe80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006010 tx=0x7f0ac001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:10.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:10.909+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0abc056d20 con 0x7f0ad0069140 2024-02-28T23:14:11.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.241+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0ad0199410 con 0x7f0ad0069140 2024-02-28T23:14:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.242+0000 7f0ac6a00700 1 -- 172.21.15.57:0/2156975878 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f0abc05a3e0 con 0x7f0ad0069140 2024-02-28T23:14:11.240 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.245+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 msgr2=0x7f0ab406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 0x7f0ab406fe80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0006010 tx=0x7f0ac001a040 comp rx=0 tx=0).stop 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 msgr2=0x7f0ad01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c8f0 tx=0x7f0abc00ccb0 comp rx=0 tx=0).stop 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 shutdown_connections 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab406d9d0 0x7f0ab406fe80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad00687a0 0x7f0ad01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 --2- 172.21.15.57:0/2156975878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0069140 0x7f0ad01985f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 >> 172.21.15.57:0/2156975878 conn(0x7f0ad0075130 msgr2=0x7f0ad0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 shutdown_connections 2024-02-28T23:14:11.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:11.246+0000 7f0ad4e00700 1 -- 172.21.15.57:0/2156975878 wait complete. 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi057"}]: dispatch 2024-02-28T23:14:12.014 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi057"}]': finished 2024-02-28T23:14:12.015 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi084"}]: dispatch 2024-02-28T23:14:12.015 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi084"}]': finished 2024-02-28T23:14:12.015 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:11 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2156975878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:14:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi057"}]: dispatch 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi057"}]': finished 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi084"}]: dispatch 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi084"}]': finished 2024-02-28T23:14:12.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:11 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2156975878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:14:12.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:12 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all mon 2024-02-28T23:14:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:12 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all mon 2024-02-28T23:14:14.048 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:14.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: Upgrade: Updating crash.smithi057 (1/2) 2024-02-28T23:14:14.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:14.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:14:14.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:14.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:13 smithi057 ceph-mon[89491]: Deploying daemon crash.smithi057 on smithi057 2024-02-28T23:14:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: Upgrade: Updating crash.smithi057 (1/2) 2024-02-28T23:14:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi057", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:14:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:13 smithi084 ceph-mon[70440]: Deploying daemon crash.smithi057 on smithi057 2024-02-28T23:14:15.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:14 smithi057 ceph-mon[89491]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:14 smithi084 ceph-mon[70440]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:17 smithi057 ceph-mon[89491]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:17 smithi084 ceph-mon[70440]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:19 smithi084 ceph-mon[70440]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:19 smithi057 ceph-mon[89491]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:21 smithi057 ceph-mon[89491]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:21 smithi057 ceph-mon[89491]: mgrmap e39: smithi057.zrrldj(active, since 92s), standbys: smithi084.fhkzis 2024-02-28T23:14:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:14:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:21.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:21 smithi084 ceph-mon[70440]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:21 smithi084 ceph-mon[70440]: mgrmap e39: smithi057.zrrldj(active, since 92s), standbys: smithi084.fhkzis 2024-02-28T23:14:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi084", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-28T23:14:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:22 smithi057 ceph-mon[89491]: Upgrade: Updating crash.smithi084 (2/2) 2024-02-28T23:14:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:22 smithi057 ceph-mon[89491]: Deploying daemon crash.smithi084 on smithi084 2024-02-28T23:14:22.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:22 smithi084 ceph-mon[70440]: Upgrade: Updating crash.smithi084 (2/2) 2024-02-28T23:14:22.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:22 smithi084 ceph-mon[70440]: Deploying daemon crash.smithi084 on smithi084 2024-02-28T23:14:23.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:23 smithi084 ceph-mon[70440]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:23 smithi057 ceph-mon[89491]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:25 smithi084 ceph-mon[70440]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:25.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:25 smithi057 ceph-mon[89491]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:27.560 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:27 smithi084 ceph-mon[70440]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:27.560 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:27.560 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:27.560 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:27.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:27 smithi057 ceph-mon[89491]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:27.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:28 smithi057 ceph-mon[89491]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:28 smithi084 ceph-mon[70440]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 3 op/s 2024-02-28T23:14:31.743 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:31 smithi057 ceph-mon[89491]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:31.743 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:31.743 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:31 smithi084 ceph-mon[70440]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-28T23:14:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.688 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:33.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:33.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:35.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:35 smithi084 ceph-mon[70440]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:35 smithi057 ceph-mon[89491]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:36.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:37.609 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:37 smithi057 ceph-mon[89491]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:37.794 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:37 smithi084 ceph-mon[70440]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:38 smithi057 ceph-mon[89491]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:38 smithi084 ceph-mon[70440]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:41.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:41 smithi057 ceph-mon[89491]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:41 smithi084 ceph-mon[70440]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 -- 172.21.15.57:0/427384647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101330 msgr2=0x7fc56c101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 --2- 172.21.15.57:0/427384647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101330 0x7fc56c101700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009910 tx=0x7fc55c009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 -- 172.21.15.57:0/427384647 shutdown_connections 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 --2- 172.21.15.57:0/427384647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101cd0 0x7fc56c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 --2- 172.21.15.57:0/427384647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101330 0x7fc56c101700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 -- 172.21.15.57:0/427384647 >> 172.21.15.57:0/427384647 conn(0x7fc56c0faad0 msgr2=0x7fc56c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 -- 172.21.15.57:0/427384647 shutdown_connections 2024-02-28T23:14:41.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.439+0000 7fc56b200700 1 -- 172.21.15.57:0/427384647 wait complete. 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.440+0000 7fc56b200700 1 Processor -- start 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc56b200700 1 -- start start 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc56b200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101330 0x7fc56c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc56b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc56b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c198bf0 con 0x7fc56c101cd0 2024-02-28T23:14:41.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc56b200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c19c980 con 0x7fc56c101330 2024-02-28T23:14:41.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.441+0000 7fc569400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51742/0 (socket says 172.21.15.57:51742) 2024-02-28T23:14:41.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 -- 172.21.15.57:0/160581004 learned_addr learned my addr 172.21.15.57:0/160581004 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:41.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569e00700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101330 0x7fc56c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 -- 172.21.15.57:0/160581004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101330 msgr2=0x7fc56c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101330 0x7fc56c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 -- 172.21.15.57:0/160581004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c0095f0 con 0x7fc56c101cd0 2024-02-28T23:14:41.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.442+0000 7fc569400700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc55800c8f0 tx=0x7fc55800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:41.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.445+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc558007a50 con 0x7fc56c101cd0 2024-02-28T23:14:41.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.445+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c19cc60 con 0x7fc56c101cd0 2024-02-28T23:14:41.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.445+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc558007bb0 con 0x7fc56c101cd0 2024-02-28T23:14:41.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.445+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc558018630 con 0x7fc56c101cd0 2024-02-28T23:14:41.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.445+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c19d1b0 con 0x7fc56c101cd0 2024-02-28T23:14:41.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.447+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c109ca0 con 0x7fc56c101cd0 2024-02-28T23:14:41.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.448+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc558004830 con 0x7fc56c101cd0 2024-02-28T23:14:41.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.449+0000 7fc562a00700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 0x7fc55006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.449+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc55808d480 con 0x7fc56c101cd0 2024-02-28T23:14:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.449+0000 7fc569e00700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 0x7fc55006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.450+0000 7fc569e00700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 0x7fc55006fd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc55c006010 tx=0x7fc55c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.455+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc558056d30 con 0x7fc56c101cd0 2024-02-28T23:14:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.712+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc56c19d480 con 0x7fc55006d8a0 2024-02-28T23:14:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.715+0000 7fc562a00700 1 -- 172.21.15.57:0/160581004 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fc56c19d480 con 0x7fc55006d8a0 2024-02-28T23:14:41.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 msgr2=0x7fc55006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 0x7fc55006fd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc55c006010 tx=0x7fc55c01a040 comp rx=0 tx=0).stop 2024-02-28T23:14:41.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 msgr2=0x7fc56c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc55800c8f0 tx=0x7fc55800ccb0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 shutdown_connections 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc55006d8a0 0x7fc55006fd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc56c101330 0x7fc56c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 --2- 172.21.15.57:0/160581004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc56c101cd0 0x7fc56c198510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 >> 172.21.15.57:0/160581004 conn(0x7fc56c0faad0 msgr2=0x7fc56c0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 shutdown_connections 2024-02-28T23:14:41.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.718+0000 7fc56b200700 1 -- 172.21.15.57:0/160581004 wait complete. 2024-02-28T23:14:41.735 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:14:41.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.885+0000 7f0391600700 1 -- 172.21.15.57:0/2600673621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 msgr2=0x7f038c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.885+0000 7f0391600700 1 --2- 172.21.15.57:0/2600673621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c073460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f037c009940 tx=0x7f037c009c50 comp rx=0 tx=0).stop 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 -- 172.21.15.57:0/2600673621 shutdown_connections 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 --2- 172.21.15.57:0/2600673621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 0x7f038c10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 --2- 172.21.15.57:0/2600673621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c073460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 -- 172.21.15.57:0/2600673621 >> 172.21.15.57:0/2600673621 conn(0x7f038c0fbd80 msgr2=0x7f038c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 -- 172.21.15.57:0/2600673621 shutdown_connections 2024-02-28T23:14:41.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.886+0000 7f0391600700 1 -- 172.21.15.57:0/2600673621 wait complete. 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 Processor -- start 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 -- start start 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 0x7f038c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c198de0 con 0x7f038c073090 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.887+0000 7f0391600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c19cb70 con 0x7f038c0739a0 2024-02-28T23:14:41.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51768/0 (socket says 172.21.15.57:51768) 2024-02-28T23:14:41.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 -- 172.21.15.57:0/4264363301 learned_addr learned my addr 172.21.15.57:0/4264363301 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038a000700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 0x7f038c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 -- 172.21.15.57:0/4264363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 msgr2=0x7f038c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 0x7f038c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.888+0000 7f038aa00700 1 -- 172.21.15.57:0/4264363301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0095f0 con 0x7f038c073090 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f038aa00700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b870 tx=0x7f037c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c01d070 con 0x7f038c073090 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f037c00f460 con 0x7f038c073090 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c19cdf0 con 0x7f038c073090 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c021640 con 0x7f038c073090 2024-02-28T23:14:41.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.889+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c19d2e0 con 0x7f038c073090 2024-02-28T23:14:41.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.891+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c109cc0 con 0x7f038c073090 2024-02-28T23:14:41.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.892+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f037c00fa60 con 0x7f038c073090 2024-02-28T23:14:41.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.893+0000 7f0383400700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 0x7f037006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:41.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.893+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f037c08e8f0 con 0x7f038c073090 2024-02-28T23:14:41.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.893+0000 7f038a000700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 0x7f037006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:41.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.894+0000 7f038a000700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 0x7f037006fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0378006fd0 tx=0x7f0378008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:41.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:41.899+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f037c0580b0 con 0x7f038c073090 2024-02-28T23:14:42.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.163+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f038c19d500 con 0x7f037006d8a0 2024-02-28T23:14:42.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.165+0000 7f0383400700 1 -- 172.21.15.57:0/4264363301 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f038c19d500 con 0x7f037006d8a0 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 msgr2=0x7f037006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 0x7f037006fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0378006fd0 tx=0x7f0378008040 comp rx=0 tx=0).stop 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 msgr2=0x7f038c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b870 tx=0x7f037c0056a0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 shutdown_connections 2024-02-28T23:14:42.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f037006d8a0 0x7f037006fd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f038c073090 0x7f038c1981c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 --2- 172.21.15.57:0/4264363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c0739a0 0x7f038c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.167+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 >> 172.21.15.57:0/4264363301 conn(0x7f038c0fbd80 msgr2=0x7f038c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:42.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.168+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 shutdown_connections 2024-02-28T23:14:42.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.168+0000 7f0391600700 1 -- 172.21.15.57:0/4264363301 wait complete. 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 -- 172.21.15.57:0/2346041020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0140739a0 msgr2=0x7fe01410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 --2- 172.21.15.57:0/2346041020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0140739a0 0x7fe01410c5e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe000009940 tx=0x7fe000009c50 comp rx=0 tx=0).stop 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 -- 172.21.15.57:0/2346041020 shutdown_connections 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 --2- 172.21.15.57:0/2346041020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0140739a0 0x7fe01410c5e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 --2- 172.21.15.57:0/2346041020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe014073090 0x7fe014073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 -- 172.21.15.57:0/2346041020 >> 172.21.15.57:0/2346041020 conn(0x7fe0140fbda0 msgr2=0x7fe0140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 -- 172.21.15.57:0/2346041020 shutdown_connections 2024-02-28T23:14:42.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.337+0000 7fe01c400700 1 -- 172.21.15.57:0/2346041020 wait complete. 2024-02-28T23:14:42.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.338+0000 7fe01c400700 1 Processor -- start 2024-02-28T23:14:42.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe01c400700 1 -- start start 2024-02-28T23:14:42.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe01c400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe01c400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0140739a0 0x7fe0141986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe01c400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe014198dd0 con 0x7fe014073090 2024-02-28T23:14:42.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe01c400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01419cb60 con 0x7fe0140739a0 2024-02-28T23:14:42.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe019a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe019a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51792/0 (socket says 172.21.15.57:51792) 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.339+0000 7fe019a00700 1 -- 172.21.15.57:0/523580509 learned_addr learned my addr 172.21.15.57:0/523580509 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe019000700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0140739a0 0x7fe0141986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe019a00700 1 -- 172.21.15.57:0/523580509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0140739a0 msgr2=0x7fe0141986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe019a00700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0140739a0 0x7fe0141986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe019a00700 1 -- 172.21.15.57:0/523580509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000095f0 con 0x7fe014073090 2024-02-28T23:14:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe019a00700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe00800d8d0 tx=0x7fe00800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.340+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe008009940 con 0x7fe014073090 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.341+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe008010460 con 0x7fe014073090 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.341+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01419ce40 con 0x7fe014073090 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.341+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00800f5f0 con 0x7fe014073090 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.341+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01419d390 con 0x7fe014073090 2024-02-28T23:14:42.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.342+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe014109ce0 con 0x7fe014073090 2024-02-28T23:14:42.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.343+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe0080105d0 con 0x7fe014073090 2024-02-28T23:14:42.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.344+0000 7fe012000700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 0x7fdffc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.344+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe00808d480 con 0x7fe014073090 2024-02-28T23:14:42.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.344+0000 7fe019000700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 0x7fdffc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.345+0000 7fe019000700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 0x7fdffc06fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe000005b40 tx=0x7fe000005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:42.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.349+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe008056ce0 con 0x7fe014073090 2024-02-28T23:14:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.593+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe014199650 con 0x7fdffc06d8a0 2024-02-28T23:14:42.593 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi057"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi057"}]': finished 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi084"}]: dispatch 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi084"}]': finished 2024-02-28T23:14:42.594 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-28T23:14:42.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.607+0000 7fe012000700 1 -- 172.21.15.57:0/523580509 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe00001e040 con 0x7fdffc06d8a0 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (4m) 9s ago 18m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (14m) 9s ago 18m 8434k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (17m) 12s ago 17m 8342k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (24s) 9s ago 18m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (16s) 12s ago 16m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (3m) 9s ago 17m 75.2M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (10m) 9s ago 10m 57.2M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (10m) 9s ago 10m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:14:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (10m) 12s ago 10m 57.7M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (10m) 12s ago 10m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (6m) 9s ago 19m 495M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (5m) 12s ago 16m 450M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (2m) 9s ago 20m 44.7M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (88s) 12s ago 16m 37.1M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (5m) 9s ago 18m 7902k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (5m) 12s ago 16m 7662k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (15m) 9s ago 15m 229M 1218M 18.2.0 dc2bc1663786 de79b804e269 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (15m) 9s ago 15m 242M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (14m) 9s ago 14m 202M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (13m) 12s ago 13m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (13m) 12s ago 13m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (12m) 12s ago 12m 233M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:14:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (4m) 9s ago 17m 57.5M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 msgr2=0x7fdffc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 0x7fdffc06fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe000005b40 tx=0x7fe000005a90 comp rx=0 tx=0).stop 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 msgr2=0x7fe0141981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe00800d8d0 tx=0x7fe00800dc90 comp rx=0 tx=0).stop 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 shutdown_connections 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdffc06d8a0 0x7fdffc06fd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014073090 0x7fe0141981b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 --2- 172.21.15.57:0/523580509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0140739a0 0x7fe0141986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 >> 172.21.15.57:0/523580509 conn(0x7fe0140fbda0 msgr2=0x7fe014106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 shutdown_connections 2024-02-28T23:14:42.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.610+0000 7fe01c400700 1 -- 172.21.15.57:0/523580509 wait complete. 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 -- 172.21.15.57:0/2956150289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64740732f0 msgr2=0x7f64740736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 --2- 172.21.15.57:0/2956150289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64740732f0 0x7f64740736c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6464009910 tx=0x7f6464009c20 comp rx=0 tx=0).stop 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 -- 172.21.15.57:0/2956150289 shutdown_connections 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 --2- 172.21.15.57:0/2956150289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474073c00 0x7f647410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 --2- 172.21.15.57:0/2956150289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64740732f0 0x7f64740736c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.760+0000 7f6473e00700 1 -- 172.21.15.57:0/2956150289 >> 172.21.15.57:0/2956150289 conn(0x7f64740fbbc0 msgr2=0x7f64740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.761+0000 7f6473e00700 1 -- 172.21.15.57:0/2956150289 shutdown_connections 2024-02-28T23:14:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.761+0000 7f6473e00700 1 -- 172.21.15.57:0/2956150289 wait complete. 2024-02-28T23:14:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 Processor -- start 2024-02-28T23:14:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 -- start start 2024-02-28T23:14:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64740732f0 0x7f6474198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474198c40 con 0x7f6474073c00 2024-02-28T23:14:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.762+0000 7f6473e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647419c9d0 con 0x7f64740732f0 2024-02-28T23:14:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51804/0 (socket says 172.21.15.57:51804) 2024-02-28T23:14:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 -- 172.21.15.57:0/2813138357 learned_addr learned my addr 172.21.15.57:0/2813138357 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472a00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64740732f0 0x7f6474198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 -- 172.21.15.57:0/2813138357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64740732f0 msgr2=0x7f6474198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64740732f0 0x7f6474198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.763+0000 7f6472000700 1 -- 172.21.15.57:0/2813138357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640095f0 con 0x7f6474073c00 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6472000700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f646000b4b0 tx=0x7f646000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6460011800 con 0x7f6474073c00 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6460011e20 con 0x7f6474073c00 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f646000f2e0 con 0x7f6474073c00 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647419ccb0 con 0x7f6474073c00 2024-02-28T23:14:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.764+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647419d200 con 0x7f6474073c00 2024-02-28T23:14:42.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.766+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6474109b20 con 0x7f6474073c00 2024-02-28T23:14:42.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.766+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6460010380 con 0x7f6474073c00 2024-02-28T23:14:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.767+0000 7f6478a00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 0x7f645806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:42.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.767+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f646008cfe0 con 0x7f6474073c00 2024-02-28T23:14:42.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.768+0000 7f6472a00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 0x7f645806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:42.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.768+0000 7f6472a00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 0x7f645806fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6464005b40 tx=0x7f6464005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:42.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:42.772+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6460056930 con 0x7f6474073c00 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:42.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi057"}]: dispatch 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi057"}]': finished 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi084"}]: dispatch 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi084"}]': finished 2024-02-28T23:14:42.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-28T23:14:43.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.102+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6474062380 con 0x7f6474073c00 2024-02-28T23:14:43.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.103+0000 7f6478a00700 1 -- 172.21.15.57:0/2813138357 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f6460059ff0 con 0x7f6474073c00 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:43.103 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:14:43.104 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.107+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 msgr2=0x7f645806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.107+0000 7f6473e00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 0x7f645806fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6464005b40 tx=0x7f6464005a90 comp rx=0 tx=0).stop 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.107+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 msgr2=0x7f6474198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.107+0000 7f6473e00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f646000b4b0 tx=0x7f646000b870 comp rx=0 tx=0).stop 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 shutdown_connections 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645806d8a0 0x7f645806fd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64740732f0 0x7f6474198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 --2- 172.21.15.57:0/2813138357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6474073c00 0x7f6474198560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 >> 172.21.15.57:0/2813138357 conn(0x7f64740fbbc0 msgr2=0x7f6474106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 shutdown_connections 2024-02-28T23:14:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.108+0000 7f6473e00700 1 -- 172.21.15.57:0/2813138357 wait complete. 2024-02-28T23:14:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 -- 172.21.15.57:0/913225186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c073090 msgr2=0x7ff91c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 --2- 172.21.15.57:0/913225186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c073090 0x7ff91c073460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009940 tx=0x7ff90c009c50 comp rx=0 tx=0).stop 2024-02-28T23:14:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 -- 172.21.15.57:0/913225186 shutdown_connections 2024-02-28T23:14:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 --2- 172.21.15.57:0/913225186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c0739a0 0x7ff91c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 --2- 172.21.15.57:0/913225186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c073090 0x7ff91c073460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.295+0000 7ff922800700 1 -- 172.21.15.57:0/913225186 >> 172.21.15.57:0/913225186 conn(0x7ff91c0fbda0 msgr2=0x7ff91c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.296+0000 7ff922800700 1 -- 172.21.15.57:0/913225186 shutdown_connections 2024-02-28T23:14:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.296+0000 7ff922800700 1 -- 172.21.15.57:0/913225186 wait complete. 2024-02-28T23:14:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.297+0000 7ff922800700 1 Processor -- start 2024-02-28T23:14:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.298+0000 7ff922800700 1 -- start start 2024-02-28T23:14:43.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.298+0000 7ff922800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c073090 0x7ff91c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.298+0000 7ff922800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.298+0000 7ff922800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c198de0 con 0x7ff91c0739a0 2024-02-28T23:14:43.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.298+0000 7ff922800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c19cb70 con 0x7ff91c073090 2024-02-28T23:14:43.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51832/0 (socket says 172.21.15.57:51832) 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 -- 172.21.15.57:0/1272363883 learned_addr learned my addr 172.21.15.57:0/1272363883 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff91b400700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c073090 0x7ff91c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 -- 172.21.15.57:0/1272363883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c073090 msgr2=0x7ff91c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c073090 0x7ff91c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.299+0000 7ff912a00700 1 -- 172.21.15.57:0/1272363883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0095f0 con 0x7ff91c0739a0 2024-02-28T23:14:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff912a00700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b4b0 tx=0x7ff90400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff904011800 con 0x7ff91c0739a0 2024-02-28T23:14:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff904011e20 con 0x7ff91c0739a0 2024-02-28T23:14:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c19ce50 con 0x7ff91c0739a0 2024-02-28T23:14:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c19d3a0 con 0x7ff91c0739a0 2024-02-28T23:14:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.300+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90400f2e0 con 0x7ff91c0739a0 2024-02-28T23:14:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.301+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c109ce0 con 0x7ff91c0739a0 2024-02-28T23:14:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.302+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff904010380 con 0x7ff91c0739a0 2024-02-28T23:14:43.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.304+0000 7ff918c00700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 0x7ff8fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.304+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff90408cf20 con 0x7ff91c0739a0 2024-02-28T23:14:43.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.304+0000 7ff91b400700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 0x7ff8fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.305+0000 7ff91b400700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 0x7ff8fc06fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005b40 tx=0x7ff90c005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:43.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.308+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9040567a0 con 0x7ff91c0739a0 2024-02-28T23:14:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all crash 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: Upgrade: osd.0 is safe to restart 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2813138357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:43.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.594+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff91c04f2a0 con 0x7ff91c0739a0 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.595+0000 7ff918c00700 1 -- 172.21.15.57:0/1272363883 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7ff904015760 con 0x7ff91c0739a0 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:14:43.596 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:14:43.597 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:14:43.598 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:14:43.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 msgr2=0x7ff8fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 0x7ff8fc06fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005b40 tx=0x7ff90c005ab0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 msgr2=0x7ff91c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b4b0 tx=0x7ff90400b7c0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 shutdown_connections 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff8fc06d8a0 0x7ff8fc06fd50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91c073090 0x7ff91c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 --2- 172.21.15.57:0/1272363883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91c0739a0 0x7ff91c198700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 >> 172.21.15.57:0/1272363883 conn(0x7ff91c0fbda0 msgr2=0x7ff91c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 shutdown_connections 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.599+0000 7ff922800700 1 -- 172.21.15.57:0/1272363883 wait complete. 2024-02-28T23:14:43.600 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all crash 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: Upgrade: osd.0 is safe to restart 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:43.742 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2813138357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:14:43.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 -- 172.21.15.57:0/3949733719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c073090 msgr2=0x7f5c8c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 --2- 172.21.15.57:0/3949733719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c073090 0x7f5c8c073460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009940 tx=0x7f5c7c009c50 comp rx=0 tx=0).stop 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 -- 172.21.15.57:0/3949733719 shutdown_connections 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 --2- 172.21.15.57:0/3949733719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 --2- 172.21.15.57:0/3949733719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c073090 0x7f5c8c073460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 -- 172.21.15.57:0/3949733719 >> 172.21.15.57:0/3949733719 conn(0x7f5c8c0fbda0 msgr2=0x7f5c8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 -- 172.21.15.57:0/3949733719 shutdown_connections 2024-02-28T23:14:43.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.755+0000 7f5c92600700 1 -- 172.21.15.57:0/3949733719 wait complete. 2024-02-28T23:14:43.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.757+0000 7f5c92600700 1 Processor -- start 2024-02-28T23:14:43.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.757+0000 7f5c92600700 1 -- start start 2024-02-28T23:14:43.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.757+0000 7f5c92600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c073090 0x7f5c8c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.757+0000 7f5c92600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c92600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c198da0 con 0x7f5c8c0739a0 2024-02-28T23:14:43.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c92600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c19cb30 con 0x7f5c8c073090 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51844/0 (socket says 172.21.15.57:51844) 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 -- 172.21.15.57:0/831122049 learned_addr learned my addr 172.21.15.57:0/831122049 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8b400700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c073090 0x7f5c8c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 -- 172.21.15.57:0/831122049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c073090 msgr2=0x7f5c8c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c073090 0x7f5c8c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.758+0000 7f5c8aa00700 1 -- 172.21.15.57:0/831122049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0095f0 con 0x7f5c8c0739a0 2024-02-28T23:14:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c8aa00700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d8d0 tx=0x7f5c7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c780098a0 con 0x7f5c8c0739a0 2024-02-28T23:14:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c78010460 con 0x7f5c8c0739a0 2024-02-28T23:14:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c19ce10 con 0x7f5c8c0739a0 2024-02-28T23:14:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7800f5f0 con 0x7f5c8c0739a0 2024-02-28T23:14:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.759+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c19d360 con 0x7f5c8c0739a0 2024-02-28T23:14:43.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.760+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c109ce0 con 0x7f5c8c0739a0 2024-02-28T23:14:43.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.762+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5c78009a00 con 0x7f5c8c0739a0 2024-02-28T23:14:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.763+0000 7f5c83e00700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 0x7f5c7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.763+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c7808d410 con 0x7f5c8c0739a0 2024-02-28T23:14:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.763+0000 7f5c8b400700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 0x7f5c7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.764+0000 7f5c8b400700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 0x7f5c7006fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006010 tx=0x7f5c7c005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:43.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:43.768+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c78056bd0 con 0x7f5c8c0739a0 2024-02-28T23:14:44.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.025+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c8c19d580 con 0x7f5c7006d8a0 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.027+0000 7f5c83e00700 1 -- 172.21.15.57:0/831122049 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5c8c19d580 con 0x7f5c7006d8a0 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:14:44.027 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:14:44.028 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:14:44.028 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "6/23 daemons upgraded", 2024-02-28T23:14:44.028 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:14:44.028 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:14:44.028 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:14:44.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 msgr2=0x7f5c7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 0x7f5c7006fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006010 tx=0x7f5c7c005ab0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 msgr2=0x7f5c8c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d8d0 tx=0x7f5c7800dbe0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 shutdown_connections 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.030+0000 7f5c92600700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c7006d8a0 0x7f5c7006fd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.031+0000 7f5c92600700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8c073090 0x7f5c8c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.031+0000 7f5c92600700 1 --2- 172.21.15.57:0/831122049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c8c0739a0 0x7f5c8c1986c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.031+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 >> 172.21.15.57:0/831122049 conn(0x7f5c8c0fbda0 msgr2=0x7f5c8c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.031+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 shutdown_connections 2024-02-28T23:14:44.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.031+0000 7f5c92600700 1 -- 172.21.15.57:0/831122049 wait complete. 2024-02-28T23:14:44.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 -- 172.21.15.57:0/703765820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 msgr2=0x7f728010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 --2- 172.21.15.57:0/703765820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f728010c5e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f726c009940 tx=0x7f726c009c50 comp rx=0 tx=0).stop 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 -- 172.21.15.57:0/703765820 shutdown_connections 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 --2- 172.21.15.57:0/703765820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f728010c5e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 --2- 172.21.15.57:0/703765820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f7280073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 -- 172.21.15.57:0/703765820 >> 172.21.15.57:0/703765820 conn(0x7f72800fbda0 msgr2=0x7f72800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 -- 172.21.15.57:0/703765820 shutdown_connections 2024-02-28T23:14:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.197+0000 7f7286000700 1 -- 172.21.15.57:0/703765820 wait complete. 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.198+0000 7f7286000700 1 Processor -- start 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.198+0000 7f7286000700 1 -- start start 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f7286000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f72801981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f7286000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f7286000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280198d40 con 0x7f72800739a0 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f7286000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280198e80 con 0x7f7280073090 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f727ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f727ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51866/0 (socket says 172.21.15.57:51866) 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f727ea00700 1 -- 172.21.15.57:0/1676188004 learned_addr learned my addr 172.21.15.57:0/1676188004 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.199+0000 7f727f400700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f72801981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f727ea00700 1 -- 172.21.15.57:0/1676188004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 msgr2=0x7f72801981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f727ea00700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f72801981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f727ea00700 1 -- 172.21.15.57:0/1676188004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0095f0 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f727f400700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f72801981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f727ea00700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b870 tx=0x7f726c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c01d070 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f726c00f460 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c021640 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.200+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728019cc70 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.201+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728019d160 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.202+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280109ce0 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.203+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f726c00fa60 con 0x7f72800739a0 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.205+0000 7f7277e00700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:14:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.205+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f726c08eb80 con 0x7f72800739a0 2024-02-28T23:14:44.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.205+0000 7f727f400700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:14:44.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.206+0000 7f727f400700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 0x7f726406fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7270005fd0 tx=0x7f7270005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:14:44.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.209+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f726c058580 con 0x7f72800739a0 2024-02-28T23:14:44.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.531+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f728004f2a0 con 0x7f72800739a0 2024-02-28T23:14:44.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.531+0000 7f7277e00700 1 -- 172.21.15.57:0/1676188004 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f726c05bc40 con 0x7f72800739a0 2024-02-28T23:14:44.531 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:14:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 msgr2=0x7f726406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 0x7f726406fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7270005fd0 tx=0x7f7270005e30 comp rx=0 tx=0).stop 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 msgr2=0x7f72801986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b870 tx=0x7f726c0056a0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 shutdown_connections 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7280073090 0x7f72801981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 --2- 172.21.15.57:0/1676188004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72800739a0 0x7f72801986f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 >> 172.21.15.57:0/1676188004 conn(0x7f72800fbda0 msgr2=0x7f7280106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 shutdown_connections 2024-02-28T23:14:44.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:14:44.534+0000 7f7286000700 1 -- 172.21.15.57:0/1676188004 wait complete. 2024-02-28T23:14:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:44 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1272363883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:14:44.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:44 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:44.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:44 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T23:14:44.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:44 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:44 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1272363883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:14:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:44 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:44 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-28T23:14:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:44 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:14:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:45 smithi057 ceph-mon[89491]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:45 smithi057 ceph-mon[89491]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:45 smithi057 ceph-mon[89491]: Upgrade: Updating osd.0 2024-02-28T23:14:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:45 smithi057 ceph-mon[89491]: Deploying daemon osd.0 on smithi057 2024-02-28T23:14:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:45 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1676188004' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:14:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:45 smithi084 ceph-mon[70440]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:45 smithi084 ceph-mon[70440]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:14:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:45 smithi084 ceph-mon[70440]: Upgrade: Updating osd.0 2024-02-28T23:14:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:45 smithi084 ceph-mon[70440]: Deploying daemon osd.0 on smithi057 2024-02-28T23:14:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:45 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1676188004' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:14:47.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:47 smithi057 ceph-mon[89491]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:47.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:47 smithi084 ceph-mon[70440]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:48.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:48.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:14:49.460 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:49 smithi057 systemd[1]: Stopping Ceph osd.0 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:14:49.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:49 smithi057 ceph-mon[89491]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:49.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:49 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[43912]: 2024-02-28T23:14:49.516+0000 7fd190e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:14:49.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:49 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[43912]: 2024-02-28T23:14:49.516+0000 7fd190e00700 -1 osd.0 50 *** Got signal Terminated *** 2024-02-28T23:14:49.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:49 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[43912]: 2024-02-28T23:14:49.516+0000 7fd190e00700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:14:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:49 smithi084 ceph-mon[70440]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:14:50.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:50 smithi057 ceph-mon[89491]: osd.0 marked itself down and dead 2024-02-28T23:14:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:50 smithi084 ceph-mon[70440]: osd.0 marked itself down and dead 2024-02-28T23:14:51.231 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:50 smithi057 podman[101358]: 2024-02-28 23:14:50.937674257 +0000 UTC m=+1.552501169 container stop de79b804e269669b8fa10115ca3ebee565a74f82b691a47e5454fff6e8da8f58 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_CLEAN=True, ceph=True, RELEASE=HEAD) 2024-02-28T23:14:51.232 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:50 smithi057 podman[101358]: 2024-02-28 23:14:50.964259 +0000 UTC m=+1.579085878 container died de79b804e269669b8fa10115ca3ebee565a74f82b691a47e5454fff6e8da8f58 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:14:51.484 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:51 smithi057 ceph-mon[89491]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:51.484 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:51 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:14:51.484 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:51 smithi057 ceph-mon[89491]: osdmap e51: 6 total, 5 up, 6 in 2024-02-28T23:14:51.484 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:51 smithi057 podman[101358]: 2024-02-28 23:14:51.485140676 +0000 UTC m=+2.099967581 container cleanup de79b804e269669b8fa10115ca3ebee565a74f82b691a47e5454fff6e8da8f58 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD) 2024-02-28T23:14:51.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:51 smithi057 bash[101358]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0 2024-02-28T23:14:51.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:51 smithi057 podman[101376]: 2024-02-28 23:14:51.728422517 +0000 UTC m=+0.769999495 container remove de79b804e269669b8fa10115ca3ebee565a74f82b691a47e5454fff6e8da8f58 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, ceph=True) 2024-02-28T23:14:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:51 smithi084 ceph-mon[70440]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:14:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:51 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:14:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:51 smithi084 ceph-mon[70440]: osdmap e51: 6 total, 5 up, 6 in 2024-02-28T23:14:52.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:52 smithi057 ceph-mon[89491]: osdmap e52: 6 total, 5 up, 6 in 2024-02-28T23:14:52.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:52 smithi084 ceph-mon[70440]: osdmap e52: 6 total, 5 up, 6 in 2024-02-28T23:14:52.858 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:52 smithi057 podman[101465]: 2024-02-28 23:14:52.480510148 +0000 UTC m=+0.033746430 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:14:52.858 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:52 smithi057 podman[101465]: 2024-02-28 23:14:52.613028883 +0000 UTC m=+0.166265200 container create 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-28T23:14:52.858 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:52 smithi057 podman[101465]: 2024-02-28 23:14:52.858883565 +0000 UTC m=+0.412119885 container init 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True) 2024-02-28T23:14:53.318 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:52 smithi057 podman[101465]: 2024-02-28 23:14:52.871839243 +0000 UTC m=+0.425075589 container start 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-28T23:14:53.318 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:52 smithi057 podman[101465]: 2024-02-28 23:14:52.982422491 +0000 UTC m=+0.535658834 container attach 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-28T23:14:53.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:53 smithi057 ceph-mon[89491]: pgmap v67: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-28T23:14:53.569 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:53 smithi057 podman[101465]: 2024-02-28 23:14:53.320251387 +0000 UTC m=+0.873487727 container died 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-02-28T23:14:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:53 smithi084 ceph-mon[70440]: pgmap v67: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-28T23:14:54.544 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 podman[101491]: 2024-02-28 23:14:54.144946915 +0000 UTC m=+0.810914051 container remove 02189103eaf803be2a5f93973fbd5c9732c0fb0244c1b55e923dc63f69ddc075 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-28T23:14:54.544 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-28T23:14:54.545 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 systemd[1]: Stopped Ceph osd.0 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:14:54.545 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.0.service: Consumed 1min 50.425s CPU time. 2024-02-28T23:14:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:54 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 63/321 objects degraded (19.626%), 19 pgs degraded (PG_DEGRADED) 2024-02-28T23:14:54.819 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 systemd[1]: Starting Ceph osd.0 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:14:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:54 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 63/321 objects degraded (19.626%), 19 pgs degraded (PG_DEGRADED) 2024-02-28T23:14:55.282 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:54 smithi057 podman[101590]: 2024-02-28 23:14:54.896816605 +0000 UTC m=+0.036249906 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:14:55.283 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:55 smithi057 podman[101590]: 2024-02-28 23:14:55.004603284 +0000 UTC m=+0.144036620 container create fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:14:55.283 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:55 smithi057 podman[101590]: 2024-02-28 23:14:55.283463617 +0000 UTC m=+0.422896997 container init fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-02-28T23:14:55.546 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:55 smithi057 podman[101590]: 2024-02-28 23:14:55.294958906 +0000 UTC m=+0.434392259 container start fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:14:55.546 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:55 smithi057 podman[101590]: 2024-02-28 23:14:55.373779922 +0000 UTC m=+0.513213238 container attach fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:14:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:55 smithi057 ceph-mon[89491]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:14:55.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:55 smithi084 ceph-mon[70440]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 bash[101590]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-28T23:14:56.571 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-28T23:14:56.572 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-28T23:14:56.572 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate[101604]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-28T23:14:56.572 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:56 smithi057 podman[101590]: 2024-02-28 23:14:56.257622962 +0000 UTC m=+1.397056272 container died fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:14:57.453 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 podman[101745]: 2024-02-28 23:14:57.086398727 +0000 UTC m=+0.806954795 container remove fa513e2d67b21058a5d4690858c72f7deac4fdb4966c9b65b0439703a98671ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-activate, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-28T23:14:57.714 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:57 smithi057 ceph-mon[89491]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:14:57.715 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 podman[101805]: 2024-02-28 23:14:57.360247582 +0000 UTC m=+0.034941784 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:14:57.715 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 podman[101805]: 2024-02-28 23:14:57.462698408 +0000 UTC m=+0.137392664 container create a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-28T23:14:57.715 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 podman[101805]: 2024-02-28 23:14:57.716088946 +0000 UTC m=+0.390783232 container init a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, GIT_CLEAN=True) 2024-02-28T23:14:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:57 smithi084 ceph-mon[70440]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:14:58.068 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 podman[101805]: 2024-02-28 23:14:57.727860913 +0000 UTC m=+0.402555176 container start a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-28T23:14:58.069 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 ceph-osd[101824]: -- 172.21.15.57:0/3682412311 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557978af7680 con 0x557978ac1000 2024-02-28T23:14:58.069 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 bash[101805]: a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df 2024-02-28T23:14:58.069 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:57 smithi057 systemd[1]: Started Ceph osd.0 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:14:58.568 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:14:58 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:14:58.301+0000 7fbd9ebce7c0 -1 Falling back to public interface 2024-02-28T23:14:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:14:58 smithi084 ceph-mon[70440]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:14:59.248 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:59.248 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:14:59.248 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:14:59.248 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:14:58 smithi057 ceph-mon[89491]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:15:01.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:01 smithi057 ceph-mon[89491]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:15:01.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:01 smithi084 ceph-mon[70440]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:15:03.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:03 smithi057 ceph-mon[89491]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:15:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:03.383 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:03 smithi084 ceph-mon[70440]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63/321 objects degraded (19.626%) 2024-02-28T23:15:03.383 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:03.383 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:03.818 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:15:03 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:15:03.421+0000 7fbd9ebce7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-28T23:15:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:04 smithi057 ceph-mon[89491]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-28T23:15:04.319 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:15:04 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:15:04.043+0000 7fbd9ebce7c0 -1 osd.0 50 log_to_monitors true 2024-02-28T23:15:04.319 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:15:04 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:15:04.128+0000 7fbd8e000700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:15:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:04 smithi084 ceph-mon[70440]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-28T23:15:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:05 smithi084 ceph-mon[70440]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:05 smithi084 ceph-mon[70440]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-28T23:15:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:05 smithi084 ceph-mon[70440]: osdmap e53: 6 total, 5 up, 6 in 2024-02-28T23:15:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:05 smithi084 ceph-mon[70440]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:15:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:05 smithi057 ceph-mon[89491]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:05 smithi057 ceph-mon[89491]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-28T23:15:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:05 smithi057 ceph-mon[89491]: osdmap e53: 6 total, 5 up, 6 in 2024-02-28T23:15:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:05 smithi057 ceph-mon[89491]: from='osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026] boot 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: osdmap e54: 6 total, 6 up, 6 in 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:06 smithi084 ceph-mon[70440]: osdmap e55: 6 total, 6 up, 6 in 2024-02-28T23:15:06.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:15:06.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: osd.0 [v2:172.21.15.57:6802/1424360026,v1:172.21.15.57:6803/1424360026] boot 2024-02-28T23:15:06.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: osdmap e54: 6 total, 6 up, 6 in 2024-02-28T23:15:06.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-28T23:15:06.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:06.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:06.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:06 smithi057 ceph-mon[89491]: osdmap e55: 6 total, 6 up, 6 in 2024-02-28T23:15:07.504 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:07 smithi057 ceph-mon[89491]: pgmap v76: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:07 smithi084 ceph-mon[70440]: pgmap v76: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:08.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:08.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:08 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:08 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:09.482 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:09 smithi057 ceph-mon[89491]: pgmap v78: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:09 smithi084 ceph-mon[70440]: pgmap v78: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 63/321 objects degraded (19.626%) 2024-02-28T23:15:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:10 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 63/321 objects degraded (19.626%), 19 pgs degraded) 2024-02-28T23:15:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:10 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:15:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:10 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 63/321 objects degraded (19.626%), 19 pgs degraded) 2024-02-28T23:15:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:10 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:15:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:11 smithi057 ceph-mon[89491]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-28T23:15:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:11 smithi084 ceph-mon[70440]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:15:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:15:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:14.562 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:14.562 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.562 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.563 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.563 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-28T23:15:14.563 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-28T23:15:14.563 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:14 smithi057 ceph-mon[89491]: Upgrade: osd.1 is safe to restart 2024-02-28T23:15:14.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:14.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:14.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-28T23:15:14.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-28T23:15:14.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:14 smithi084 ceph-mon[70440]: Upgrade: osd.1 is safe to restart 2024-02-28T23:15:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.705+0000 7fa58dc00700 1 -- 172.21.15.57:0/931370836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 msgr2=0x7fa5881008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.705+0000 7fa58dc00700 1 --2- 172.21.15.57:0/931370836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa5881008b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa578009910 tx=0x7fa578009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 -- 172.21.15.57:0/931370836 shutdown_connections 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 --2- 172.21.15.57:0/931370836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 --2- 172.21.15.57:0/931370836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa5881008b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 -- 172.21.15.57:0/931370836 >> 172.21.15.57:0/931370836 conn(0x7fa5880fbda0 msgr2=0x7fa5880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 -- 172.21.15.57:0/931370836 shutdown_connections 2024-02-28T23:15:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.706+0000 7fa58dc00700 1 -- 172.21.15.57:0/931370836 wait complete. 2024-02-28T23:15:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.707+0000 7fa58dc00700 1 Processor -- start 2024-02-28T23:15:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.707+0000 7fa58dc00700 1 -- start start 2024-02-28T23:15:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa58dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa58dc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa58dc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa588196a80 con 0x7fa5881004e0 2024-02-28T23:15:14.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa58dc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa588196bc0 con 0x7fa588100e80 2024-02-28T23:15:14.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa586a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:14.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa586a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34196/0 (socket says 172.21.15.57:34196) 2024-02-28T23:15:14.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa586a00700 1 -- 172.21.15.57:0/3856812479 learned_addr learned my addr 172.21.15.57:0/3856812479 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa586000700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.708+0000 7fa586a00700 1 -- 172.21.15.57:0/3856812479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 msgr2=0x7fa588196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa586a00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa586a00700 1 -- 172.21.15.57:0/3856812479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780095f0 con 0x7fa5881004e0 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa586000700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:15:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa586a00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa578006010 tx=0x7fa57800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57801d070 con 0x7fa5881004e0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa578004fb0 con 0x7fa5881004e0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57800f690 con 0x7fa5881004e0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.709+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58819a9b0 con 0x7fa5881004e0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.710+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58819aea0 con 0x7fa5881004e0 2024-02-28T23:15:14.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.711+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5881087f0 con 0x7fa5881004e0 2024-02-28T23:15:14.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.712+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa5780228a0 con 0x7fa5881004e0 2024-02-28T23:15:14.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.713+0000 7fa57f400700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 0x7fa56c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:14.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.713+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa57808ed30 con 0x7fa5881004e0 2024-02-28T23:15:14.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.714+0000 7fa586000700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 0x7fa56c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:14.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.714+0000 7fa586000700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 0x7fa56c06fd50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa574005fd0 tx=0x7fa574005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:14.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.719+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa578058570 con 0x7fa5881004e0 2024-02-28T23:15:14.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.977+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa588197360 con 0x7fa56c06d8a0 2024-02-28T23:15:14.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.979+0000 7fa57f400700 1 -- 172.21.15.57:0/3856812479 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa588197360 con 0x7fa56c06d8a0 2024-02-28T23:15:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 msgr2=0x7fa56c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 0x7fa56c06fd50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa574005fd0 tx=0x7fa574005e20 comp rx=0 tx=0).stop 2024-02-28T23:15:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 msgr2=0x7fa588195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa578006010 tx=0x7fa57800be20 comp rx=0 tx=0).stop 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 shutdown_connections 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa56c06d8a0 0x7fa56c06fd50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5881004e0 0x7fa588195ef0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 --2- 172.21.15.57:0/3856812479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa588100e80 0x7fa588196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 >> 172.21.15.57:0/3856812479 conn(0x7fa5880fbda0 msgr2=0x7fa5880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 shutdown_connections 2024-02-28T23:15:14.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:14.983+0000 7fa58dc00700 1 -- 172.21.15.57:0/3856812479 wait complete. 2024-02-28T23:15:15.006 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.154+0000 7f9b92800700 1 -- 172.21.15.57:0/3833079787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c100e80 msgr2=0x7f9b8c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.154+0000 7f9b92800700 1 --2- 172.21.15.57:0/3833079787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c100e80 0x7f9b8c104e50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009910 tx=0x7f9b78009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 -- 172.21.15.57:0/3833079787 shutdown_connections 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 --2- 172.21.15.57:0/3833079787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c100e80 0x7f9b8c104e50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 --2- 172.21.15.57:0/3833079787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 -- 172.21.15.57:0/3833079787 >> 172.21.15.57:0/3833079787 conn(0x7f9b8c0fbda0 msgr2=0x7f9b8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 -- 172.21.15.57:0/3833079787 shutdown_connections 2024-02-28T23:15:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.155+0000 7f9b92800700 1 -- 172.21.15.57:0/3833079787 wait complete. 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.156+0000 7f9b92800700 1 Processor -- start 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b92800700 1 -- start start 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b92800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b92800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c100e80 0x7f9b8c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b92800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c073ad0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b92800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c073c10 con 0x7f9b8c100e80 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53918/0 (socket says 172.21.15.57:53918) 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.157+0000 7f9b8b400700 1 -- 172.21.15.57:0/1652160274 learned_addr learned my addr 172.21.15.57:0/1652160274 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.158+0000 7f9b8aa00700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c100e80 0x7f9b8c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.158+0000 7f9b8b400700 1 -- 172.21.15.57:0/1652160274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c100e80 msgr2=0x7f9b8c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.158+0000 7f9b8b400700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c100e80 0x7f9b8c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.158+0000 7f9b8b400700 1 -- 172.21.15.57:0/1652160274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b780095f0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.158+0000 7f9b8b400700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c8a0 tx=0x7f9b7c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:15.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.159+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c00cea0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.159+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b7c004830 con 0x7f9b8c1004e0 2024-02-28T23:15:15.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.159+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c0055b0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.159+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c1a2100 con 0x7f9b8c1004e0 2024-02-28T23:15:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.159+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c1a2510 con 0x7f9b8c1004e0 2024-02-28T23:15:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.160+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c108850 con 0x7f9b8c1004e0 2024-02-28T23:15:15.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.161+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9b7c0049a0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.162+0000 7f9b83e00700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 0x7f9b7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.162+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9b7c08d1b0 con 0x7f9b8c1004e0 2024-02-28T23:15:15.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.162+0000 7f9b8aa00700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 0x7f9b7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.163+0000 7f9b8aa00700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 0x7f9b7006fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c074840 tx=0x7f9b7801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:15.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.167+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b7c056970 con 0x7f9b8c1004e0 2024-02-28T23:15:15.410 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:15 smithi057 ceph-mon[89491]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:15:15.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.440+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b8c061190 con 0x7f9b7006d8a0 2024-02-28T23:15:15.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.442+0000 7f9b83e00700 1 -- 172.21.15.57:0/1652160274 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9b8c061190 con 0x7f9b7006d8a0 2024-02-28T23:15:15.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 msgr2=0x7f9b7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 0x7f9b7006fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c074840 tx=0x7f9b7801a040 comp rx=0 tx=0).stop 2024-02-28T23:15:15.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 msgr2=0x7f9b8c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c8a0 tx=0x7f9b7c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:15:15.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 shutdown_connections 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9b7006d8a0 0x7f9b7006fd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b8c1004e0 0x7f9b8c073050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 --2- 172.21.15.57:0/1652160274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b8c100e80 0x7f9b8c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.444+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 >> 172.21.15.57:0/1652160274 conn(0x7f9b8c0fbda0 msgr2=0x7f9b8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.445+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 shutdown_connections 2024-02-28T23:15:15.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.445+0000 7f9b92800700 1 -- 172.21.15.57:0/1652160274 wait complete. 2024-02-28T23:15:15.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:15 smithi084 ceph-mon[70440]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 -- 172.21.15.57:0/2894290073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 msgr2=0x7f600c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 --2- 172.21.15.57:0/2894290073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c068b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009910 tx=0x7f5ffc009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 -- 172.21.15.57:0/2894290073 shutdown_connections 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 --2- 172.21.15.57:0/2894290073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 0x7f600c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 --2- 172.21.15.57:0/2894290073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c068b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.604+0000 7f6010c00700 1 -- 172.21.15.57:0/2894290073 >> 172.21.15.57:0/2894290073 conn(0x7f600c075130 msgr2=0x7f600c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.605+0000 7f6010c00700 1 -- 172.21.15.57:0/2894290073 shutdown_connections 2024-02-28T23:15:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.605+0000 7f6010c00700 1 -- 172.21.15.57:0/2894290073 wait complete. 2024-02-28T23:15:15.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.606+0000 7f6010c00700 1 Processor -- start 2024-02-28T23:15:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.607+0000 7f6010c00700 1 -- start start 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.607+0000 7f6010c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.607+0000 7f6010c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 0x7f600c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.607+0000 7f6010c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c196b00 con 0x7f600c0687a0 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.607+0000 7f6010c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c19a890 con 0x7f600c069140 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53928/0 (socket says 172.21.15.57:53928) 2024-02-28T23:15:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 -- 172.21.15.57:0/1864521052 learned_addr learned my addr 172.21.15.57:0/1864521052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009400700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 0x7f600c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 -- 172.21.15.57:0/1864521052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 msgr2=0x7f600c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 0x7f600c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.608+0000 7f6009e00700 1 -- 172.21.15.57:0/1864521052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0095f0 con 0x7f600c0687a0 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6009e00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005f50 tx=0x7f5ffc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:15.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc01d070 con 0x7f600c0687a0 2024-02-28T23:15:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ffc00b610 con 0x7f600c0687a0 2024-02-28T23:15:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c19ab10 con 0x7f600c0687a0 2024-02-28T23:15:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c19b000 con 0x7f600c0687a0 2024-02-28T23:15:15.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.609+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc022470 con 0x7f600c0687a0 2024-02-28T23:15:15.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.610+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c108790 con 0x7f600c0687a0 2024-02-28T23:15:15.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.612+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5ffc00fca0 con 0x7f600c0687a0 2024-02-28T23:15:15.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.613+0000 7f6002a00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 0x7f5ff006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:15.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.613+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5ffc08e9b0 con 0x7f600c0687a0 2024-02-28T23:15:15.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.614+0000 7f6009400700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 0x7f5ff006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:15.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.614+0000 7f6009400700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 0x7f5ff006fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f600c197500 tx=0x7f5ff8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:15.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.623+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ffc058170 con 0x7f600c0687a0 2024-02-28T23:15:15.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.872+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f600c19b270 con 0x7f5ff006d8a0 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.911+0000 7f6002a00700 1 -- 172.21.15.57:0/1864521052 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f600c19b270 con 0x7f5ff006d8a0 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (4m) 10s ago 18m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (15m) 10s ago 19m 8464k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (17m) 45s ago 17m 8342k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:15:15.910 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (57s) 10s ago 19m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (50s) 45s ago 17m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (4m) 10s ago 18m 75.2M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (11m) 10s ago 11m 57.2M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (11m) 10s ago 11m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (11m) 45s ago 11m 57.7M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (11m) 45s ago 11m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (7m) 10s ago 20m 498M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (6m) 45s ago 17m 450M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (2m) 10s ago 20m 45.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (2m) 45s ago 17m 37.1M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (5m) 10s ago 18m 7948k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (5m) 45s ago 17m 7662k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (18s) 10s ago 16m 11.1M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (15m) 10s ago 15m 241M 1218M 18.2.0 dc2bc1663786 183ec4fb27f3 2024-02-28T23:15:15.911 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (15m) 10s ago 15m 203M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:15:15.912 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (14m) 45s ago 14m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:15:15.912 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (13m) 45s ago 13m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:15:15.912 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (13m) 45s ago 13m 233M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:15:15.912 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (5m) 10s ago 18m 60.9M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:15:15.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.914+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 msgr2=0x7f5ff006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.914+0000 7f6010c00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 0x7f5ff006fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f600c197500 tx=0x7f5ff8009260 comp rx=0 tx=0).stop 2024-02-28T23:15:15.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.914+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 msgr2=0x7f600c195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:15.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.914+0000 7f6010c00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005f50 tx=0x7f5ffc00f740 comp rx=0 tx=0).stop 2024-02-28T23:15:15.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 shutdown_connections 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ff006d8a0 0x7f5ff006fd50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f600c0687a0 0x7f600c195ee0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 --2- 172.21.15.57:0/1864521052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c069140 0x7f600c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 >> 172.21.15.57:0/1864521052 conn(0x7f600c075130 msgr2=0x7f600c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 shutdown_connections 2024-02-28T23:15:15.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:15.915+0000 7f6010c00700 1 -- 172.21.15.57:0/1864521052 wait complete. 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 -- 172.21.15.57:0/3458162134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf8100e80 msgr2=0x7fbdf8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 --2- 172.21.15.57:0/3458162134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf8100e80 0x7fbdf8104e50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009910 tx=0x7fbde4009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 -- 172.21.15.57:0/3458162134 shutdown_connections 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 --2- 172.21.15.57:0/3458162134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf8100e80 0x7fbdf8104e50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 --2- 172.21.15.57:0/3458162134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf81004e0 0x7fbdf81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 -- 172.21.15.57:0/3458162134 >> 172.21.15.57:0/3458162134 conn(0x7fbdf80fbda0 msgr2=0x7fbdf80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 -- 172.21.15.57:0/3458162134 shutdown_connections 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.066+0000 7fbdff800700 1 -- 172.21.15.57:0/3458162134 wait complete. 2024-02-28T23:15:16.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.067+0000 7fbdff800700 1 Processor -- start 2024-02-28T23:15:16.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdff800700 1 -- start start 2024-02-28T23:15:16.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdff800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdff800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf8100e80 0x7fbdf8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdff800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8073ad0 con 0x7fbdf81004e0 2024-02-28T23:15:16.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdff800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8073c10 con 0x7fbdf8100e80 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdfce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdfce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53956/0 (socket says 172.21.15.57:53956) 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.068+0000 7fbdfce00700 1 -- 172.21.15.57:0/4193011278 learned_addr learned my addr 172.21.15.57:0/4193011278 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.069+0000 7fbdf7e00700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf8100e80 0x7fbdf8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.069+0000 7fbdfce00700 1 -- 172.21.15.57:0/4193011278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf8100e80 msgr2=0x7fbdf8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.069+0000 7fbdfce00700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf8100e80 0x7fbdf8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.069+0000 7fbdfce00700 1 -- 172.21.15.57:0/4193011278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40095f0 con 0x7fbdf81004e0 2024-02-28T23:15:16.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.069+0000 7fbdfce00700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c8a0 tx=0x7fbdec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:16.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.070+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec00cea0 con 0x7fbdf81004e0 2024-02-28T23:15:16.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.070+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdec004830 con 0x7fbdf81004e0 2024-02-28T23:15:16.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.070+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf81a2100 con 0x7fbdf81004e0 2024-02-28T23:15:16.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.070+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf81a2510 con 0x7fbdf81004e0 2024-02-28T23:15:16.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.070+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec0055b0 con 0x7fbdf81004e0 2024-02-28T23:15:16.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.071+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8108850 con 0x7fbdf81004e0 2024-02-28T23:15:16.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.072+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbdec0049a0 con 0x7fbdf81004e0 2024-02-28T23:15:16.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.073+0000 7fbdf5600700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.074+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbdec08d130 con 0x7fbdf81004e0 2024-02-28T23:15:16.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.074+0000 7fbdf7e00700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.075+0000 7fbdf7e00700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 0x7fbde006fd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:16.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.079+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbdec056990 con 0x7fbdf81004e0 2024-02-28T23:15:16.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:16 smithi057 ceph-mon[89491]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:16.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:16 smithi057 ceph-mon[89491]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:16.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:16.323 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T23:15:16.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.415+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbdf8108230 con 0x7fbdf81004e0 2024-02-28T23:15:16.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.416+0000 7fbdf5600700 1 -- 172.21.15.57:0/4193011278 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fbdec004c50 con 0x7fbdf81004e0 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:16.415 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 5 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:15:16.416 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:15:16.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 msgr2=0x7fbde006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 0x7fbde006fd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4005fd0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 msgr2=0x7fbdf8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c8a0 tx=0x7fbdec00cc60 comp rx=0 tx=0).stop 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 shutdown_connections 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf81004e0 0x7fbdf8073050 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 --2- 172.21.15.57:0/4193011278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdf8100e80 0x7fbdf8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 >> 172.21.15.57:0/4193011278 conn(0x7fbdf80fbda0 msgr2=0x7fbdf80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 shutdown_connections 2024-02-28T23:15:16.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.419+0000 7fbdff800700 1 -- 172.21.15.57:0/4193011278 wait complete. 2024-02-28T23:15:16.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 -- 172.21.15.57:0/2918801487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 msgr2=0x7fbdec1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 --2- 172.21.15.57:0/2918801487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec1008b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009910 tx=0x7fbddc009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:16.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:16.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 -- 172.21.15.57:0/2918801487 shutdown_connections 2024-02-28T23:15:16.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 --2- 172.21.15.57:0/2918801487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 0x7fbdec104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 --2- 172.21.15.57:0/2918801487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec1008b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 -- 172.21.15.57:0/2918801487 >> 172.21.15.57:0/2918801487 conn(0x7fbdec0fbda0 msgr2=0x7fbdec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:16.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.575+0000 7fbdf1600700 1 -- 172.21.15.57:0/2918801487 shutdown_connections 2024-02-28T23:15:16.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.576+0000 7fbdf1600700 1 -- 172.21.15.57:0/2918801487 wait complete. 2024-02-28T23:15:16.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.577+0000 7fbdf1600700 1 Processor -- start 2024-02-28T23:15:16.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.577+0000 7fbdf1600700 1 -- start start 2024-02-28T23:15:16.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdf1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdf1600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 0x7fbdec073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdf1600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec073ad0 con 0x7fbdec1004e0 2024-02-28T23:15:16.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdf1600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec073c10 con 0x7fbdec100e80 2024-02-28T23:15:16.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdeaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdeaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53974/0 (socket says 172.21.15.57:53974) 2024-02-28T23:15:16.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.578+0000 7fbdeaa00700 1 -- 172.21.15.57:0/3069833418 learned_addr learned my addr 172.21.15.57:0/3069833418 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:16.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.579+0000 7fbdea000700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 0x7fbdec073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.579+0000 7fbdeaa00700 1 -- 172.21.15.57:0/3069833418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 msgr2=0x7fbdec073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.579+0000 7fbdeaa00700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 0x7fbdec073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.579+0000 7fbdeaa00700 1 -- 172.21.15.57:0/3069833418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbdec1004e0 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.579+0000 7fbdeaa00700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b5c0 tx=0x7fbddc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.580+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc01d070 con 0x7fbdec1004e0 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.580+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbddc00f460 con 0x7fbdec1004e0 2024-02-28T23:15:16.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.580+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc017610 con 0x7fbdec1004e0 2024-02-28T23:15:16.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.580+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec073e90 con 0x7fbdec1004e0 2024-02-28T23:15:16.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.580+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec1a23f0 con 0x7fbdec1004e0 2024-02-28T23:15:16.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.581+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec1087f0 con 0x7fbdec1004e0 2024-02-28T23:15:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:16 smithi084 ceph-mon[70440]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:16 smithi084 ceph-mon[70440]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-28T23:15:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:16.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.582+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbddc00f5d0 con 0x7fbdec1004e0 2024-02-28T23:15:16.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.583+0000 7fbde3400700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 0x7fbdd006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:16.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.583+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbddc08e9d0 con 0x7fbdec1004e0 2024-02-28T23:15:16.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.584+0000 7fbdea000700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 0x7fbdd006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:16.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.584+0000 7fbdea000700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 0x7fbdd006fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8005fd0 tx=0x7fbdd800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:16.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.588+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbddc0582b0 con 0x7fbdec1004e0 2024-02-28T23:15:16.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.875+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbdec04f2a0 con 0x7fbdec1004e0 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.876+0000 7fbde3400700 1 -- 172.21.15.57:0/3069833418 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fbddc05b970 con 0x7fbdec1004e0 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:15:16.875 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:15:16.876 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:15:16.877 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:16.879 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.880+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 msgr2=0x7fbdd006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.880+0000 7fbdf1600700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 0x7fbdd006fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8005fd0 tx=0x7fbdd800a560 comp rx=0 tx=0).stop 2024-02-28T23:15:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.880+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 msgr2=0x7fbdec073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.880+0000 7fbdf1600700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b5c0 tx=0x7fbddc00be00 comp rx=0 tx=0).stop 2024-02-28T23:15:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 shutdown_connections 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbdd006d8a0 0x7fbdd006fd50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdec1004e0 0x7fbdec073050 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 --2- 172.21.15.57:0/3069833418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdec100e80 0x7fbdec073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 >> 172.21.15.57:0/3069833418 conn(0x7fbdec0fbda0 msgr2=0x7fbdec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 shutdown_connections 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:16.881+0000 7fbdf1600700 1 -- 172.21.15.57:0/3069833418 wait complete. 2024-02-28T23:15:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:15:17.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 -- 172.21.15.57:0/4094601418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 msgr2=0x7f0fec1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 --2- 172.21.15.57:0/4094601418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec1028e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009910 tx=0x7f0fd8009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:17.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 -- 172.21.15.57:0/4094601418 shutdown_connections 2024-02-28T23:15:17.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 --2- 172.21.15.57:0/4094601418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec1028e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 --2- 172.21.15.57:0/4094601418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 -- 172.21.15.57:0/4094601418 >> 172.21.15.57:0/4094601418 conn(0x7f0fec0fdf90 msgr2=0x7f0fec1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:17.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 -- 172.21.15.57:0/4094601418 shutdown_connections 2024-02-28T23:15:17.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.037+0000 7f0ff2200700 1 -- 172.21.15.57:0/4094601418 wait complete. 2024-02-28T23:15:17.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 Processor -- start 2024-02-28T23:15:17.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 -- start start 2024-02-28T23:15:17.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec198db0 con 0x7f0fec102470 2024-02-28T23:15:17.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.038+0000 7f0ff2200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec19cb40 con 0x7f0fec108470 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53986/0 (socket says 172.21.15.57:53986) 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 -- 172.21.15.57:0/91139390 learned_addr learned my addr 172.21.15.57:0/91139390 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feaa00700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 -- 172.21.15.57:0/91139390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 msgr2=0x7f0fec1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 -- 172.21.15.57:0/91139390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80095f0 con 0x7f0fec102470 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feaa00700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.039+0000 7f0feb400700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00ba50 tx=0x7f0fdc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.040+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc00ca00 con 0x7f0fec102470 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.040+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec19ce20 con 0x7f0fec102470 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.040+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fdc00cb60 con 0x7f0fec102470 2024-02-28T23:15:17.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.040+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc0075a0 con 0x7f0fec102470 2024-02-28T23:15:17.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.040+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec19d370 con 0x7f0fec102470 2024-02-28T23:15:17.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.041+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec10aa00 con 0x7f0fec102470 2024-02-28T23:15:17.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.042+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0fdc00ccd0 con 0x7f0fec102470 2024-02-28T23:15:17.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.043+0000 7f0fe3e00700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 0x7f0fd006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.043+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0fdc006e40 con 0x7f0fec102470 2024-02-28T23:15:17.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.043+0000 7f0feaa00700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 0x7f0fd006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.044+0000 7f0feaa00700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 0x7f0fd006fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006010 tx=0x7f0fd800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.047+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fdc056560 con 0x7f0fec102470 2024-02-28T23:15:17.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.304+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fec19d660 con 0x7f0fd006d8a0 2024-02-28T23:15:17.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.307+0000 7f0fe3e00700 1 -- 172.21.15.57:0/91139390 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0fec19d660 con 0x7f0fd006d8a0 2024-02-28T23:15:17.306 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:15:17.306 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:15:17.306 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:15:17.306 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:15:17.306 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "7/23 daemons upgraded", 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:15:17.307 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:15:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 msgr2=0x7f0fd006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 0x7f0fd006fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006010 tx=0x7f0fd800b540 comp rx=0 tx=0).stop 2024-02-28T23:15:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 msgr2=0x7f0fec198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00ba50 tx=0x7f0fdc00be10 comp rx=0 tx=0).stop 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 shutdown_connections 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0fd006d8a0 0x7f0fd006fd50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fec102470 0x7f0fec198190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 --2- 172.21.15.57:0/91139390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fec108470 0x7f0fec1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 >> 172.21.15.57:0/91139390 conn(0x7f0fec0fdf90 msgr2=0x7f0fec0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 shutdown_connections 2024-02-28T23:15:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.310+0000 7f0ff2200700 1 -- 172.21.15.57:0/91139390 wait complete. 2024-02-28T23:15:17.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.475+0000 7f0956c00700 1 -- 172.21.15.57:0/700077938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09501024d0 msgr2=0x7f09501028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.475+0000 7f0956c00700 1 --2- 172.21.15.57:0/700077938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09501024d0 0x7f09501028a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0944009910 tx=0x7f0944009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:17.474 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:17.474 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: Upgrade: Updating osd.1 2024-02-28T23:15:17.474 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: Deploying daemon osd.1 on smithi057 2024-02-28T23:15:17.474 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:15:17.475 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4193011278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:17.475 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3069833418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:15:17.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 -- 172.21.15.57:0/700077938 shutdown_connections 2024-02-28T23:15:17.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 --2- 172.21.15.57:0/700077938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0950102e70 0x7f0950106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 --2- 172.21.15.57:0/700077938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09501024d0 0x7f09501028a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 -- 172.21.15.57:0/700077938 >> 172.21.15.57:0/700077938 conn(0x7f09500fdd90 msgr2=0x7f09501001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 -- 172.21.15.57:0/700077938 shutdown_connections 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.476+0000 7f0956c00700 1 -- 172.21.15.57:0/700077938 wait complete. 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.477+0000 7f0956c00700 1 Processor -- start 2024-02-28T23:15:17.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.477+0000 7f0956c00700 1 -- start start 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.477+0000 7f0956c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09501024d0 0x7f0950197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0956c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0956c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950198a80 con 0x7f0950102e70 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0956c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095019c810 con 0x7f09501024d0 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0954e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0954e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54000/0 (socket says 172.21.15.57:54000) 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0954e00700 1 -- 172.21.15.57:0/843776080 learned_addr learned my addr 172.21.15.57:0/843776080 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.478+0000 7f0955800700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09501024d0 0x7f0950197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f0954e00700 1 -- 172.21.15.57:0/843776080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09501024d0 msgr2=0x7f0950197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f0954e00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09501024d0 0x7f0950197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f0954e00700 1 -- 172.21.15.57:0/843776080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440095f0 con 0x7f0950102e70 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f0954e00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c8f0 tx=0x7f093c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c0078a0 con 0x7f0950102e70 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f093c00f450 con 0x7f0950102e70 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095019caf0 con 0x7f0950102e70 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.480+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095019d040 con 0x7f0950102e70 2024-02-28T23:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.479+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c018610 con 0x7f0950102e70 2024-02-28T23:15:17.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.481+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095010a7e0 con 0x7f0950102e70 2024-02-28T23:15:17.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.482+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f093c00f5c0 con 0x7f0950102e70 2024-02-28T23:15:17.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.483+0000 7f094e000700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 0x7f093806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:17.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.483+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f093c08d3e0 con 0x7f0950102e70 2024-02-28T23:15:17.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.483+0000 7f0955800700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 0x7f093806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:17.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.484+0000 7f0955800700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 0x7f093806fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0944005f50 tx=0x7f094401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:17.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.488+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f093c056c40 con 0x7f0950102e70 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: Upgrade: Updating osd.1 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: Deploying daemon osd.1 on smithi057 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4193011278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3069833418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:15:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.817+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f095019d310 con 0x7f0950102e70 2024-02-28T23:15:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.818+0000 7f094e000700 1 -- 172.21.15.57:0/843776080 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f093c05a300 con 0x7f0950102e70 2024-02-28T23:15:17.817 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 msgr2=0x7f093806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 0x7f093806fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0944005f50 tx=0x7f094401a040 comp rx=0 tx=0).stop 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 msgr2=0x7f09501983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c8f0 tx=0x7f093c00cc00 comp rx=0 tx=0).stop 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 shutdown_connections 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f093806d8a0 0x7f093806fd50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09501024d0 0x7f0950197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 --2- 172.21.15.57:0/843776080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0950102e70 0x7f09501983a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.822+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 >> 172.21.15.57:0/843776080 conn(0x7f09500fdd90 msgr2=0x7f0950100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:17.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.823+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 shutdown_connections 2024-02-28T23:15:17.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:17.823+0000 7f0956c00700 1 -- 172.21.15.57:0/843776080 wait complete. 2024-02-28T23:15:18.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:18 smithi057 ceph-mon[89491]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:18.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:18 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/843776080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:15:18.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:18.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:18.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:18 smithi084 ceph-mon[70440]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:18 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/843776080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:15:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:18.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:18.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:19 smithi057 ceph-mon[89491]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-02-28T23:15:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:19 smithi084 ceph-mon[70440]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-02-28T23:15:21.319 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:20 smithi057 systemd[1]: Stopping Ceph osd.1 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:15:21.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:21 smithi057 ceph-mon[89491]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:15:21.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:21 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[49423]: 2024-02-28T23:15:21.449+0000 7f5a79400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:15:21.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:21 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[49423]: 2024-02-28T23:15:21.449+0000 7f5a79400700 -1 osd.1 55 *** Got signal Terminated *** 2024-02-28T23:15:21.819 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:21 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[49423]: 2024-02-28T23:15:21.449+0000 7f5a79400700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:15:21.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:21 smithi084 ceph-mon[70440]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:15:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:22 smithi057 ceph-mon[89491]: osd.1 marked itself down and dead 2024-02-28T23:15:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:22 smithi084 ceph-mon[70440]: osd.1 marked itself down and dead 2024-02-28T23:15:23.241 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:22 smithi057 podman[106944]: 2024-02-28 23:15:22.933366043 +0000 UTC m=+1.647462184 container died 183ec4fb27f3dd56d08d8c8530cc1b900904123a93e12284d52f1c80a993af72 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:15:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:23 smithi057 ceph-mon[89491]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:15:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:23 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:15:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:23 smithi057 ceph-mon[89491]: osdmap e56: 6 total, 5 up, 6 in 2024-02-28T23:15:23.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:23 smithi057 podman[106944]: 2024-02-28 23:15:23.350037699 +0000 UTC m=+2.064133887 container cleanup 183ec4fb27f3dd56d08d8c8530cc1b900904123a93e12284d52f1c80a993af72 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:15:23.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:23 smithi057 bash[106944]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1 2024-02-28T23:15:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:23 smithi084 ceph-mon[70440]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:15:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:23 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:15:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:23 smithi084 ceph-mon[70440]: osdmap e56: 6 total, 5 up, 6 in 2024-02-28T23:15:24.068 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:23 smithi057 podman[106963]: 2024-02-28 23:15:23.734639834 +0000 UTC m=+0.780813680 container remove 183ec4fb27f3dd56d08d8c8530cc1b900904123a93e12284d52f1c80a993af72 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, ceph=True) 2024-02-28T23:15:24.745 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:24 smithi057 ceph-mon[89491]: osdmap e57: 6 total, 5 up, 6 in 2024-02-28T23:15:24.745 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:24 smithi057 podman[107051]: 2024-02-28 23:15:24.477461546 +0000 UTC m=+0.033247213 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:15:24.745 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:24 smithi057 podman[107051]: 2024-02-28 23:15:24.58545407 +0000 UTC m=+0.141239774 container create 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-28T23:15:24.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:24 smithi084 ceph-mon[70440]: osdmap e57: 6 total, 5 up, 6 in 2024-02-28T23:15:25.012 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:24 smithi057 podman[107051]: 2024-02-28 23:15:24.872732193 +0000 UTC m=+0.428517903 container init 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-28T23:15:25.012 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:24 smithi057 podman[107051]: 2024-02-28 23:15:24.88509466 +0000 UTC m=+0.440880391 container start 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-1fb5d10) 2024-02-28T23:15:25.012 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:25 smithi057 podman[107051]: 2024-02-28 23:15:25.012984263 +0000 UTC m=+0.568769936 container attach 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222) 2024-02-28T23:15:25.706 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:25 smithi057 ceph-mon[89491]: pgmap v88: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43/321 objects degraded (13.396%) 2024-02-28T23:15:25.706 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:25 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 14 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:25.707 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:25 smithi057 podman[107051]: 2024-02-28 23:15:25.339104626 +0000 UTC m=+0.894890336 container died 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-28T23:15:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:25 smithi084 ceph-mon[70440]: pgmap v88: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43/321 objects degraded (13.396%) 2024-02-28T23:15:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:25 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 14 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:26.507 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 podman[107077]: 2024-02-28 23:15:26.074943001 +0000 UTC m=+0.715063675 container remove 15cbd62fe2fb2b12a12090391a7dee4f84728faf9fa596dad6905254ef871f23 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-02-28T23:15:26.507 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-28T23:15:26.507 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 systemd[1]: Stopped Ceph osd.1 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:15:26.507 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.1.service: Consumed 2min 856ms CPU time. 2024-02-28T23:15:26.818 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 systemd[1]: Starting Ceph osd.1 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:15:27.144 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 podman[107173]: 2024-02-28 23:15:26.779617376 +0000 UTC m=+0.037234879 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:15:27.144 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:26 smithi057 podman[107173]: 2024-02-28 23:15:26.926189618 +0000 UTC m=+0.183807147 container create 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-02-28T23:15:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:27 smithi057 ceph-mon[89491]: pgmap v89: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:27.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:27 smithi057 podman[107173]: 2024-02-28 23:15:27.221793667 +0000 UTC m=+0.479411239 container init 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-28T23:15:27.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:27 smithi057 podman[107173]: 2024-02-28 23:15:27.236058985 +0000 UTC m=+0.493676485 container start 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-28T23:15:27.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:27 smithi057 podman[107173]: 2024-02-28 23:15:27.320385781 +0000 UTC m=+0.578003350 container attach 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-28T23:15:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:27 smithi084 ceph-mon[70440]: pgmap v89: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:28.319 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 bash[107173]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-28T23:15:28.320 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-28T23:15:28.321 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-28T23:15:28.321 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate[107188]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-28T23:15:28.321 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 podman[107173]: 2024-02-28 23:15:28.195064358 +0000 UTC m=+1.452681910 container died 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-28T23:15:29.263 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:28 smithi057 podman[107317]: 2024-02-28 23:15:28.907129829 +0000 UTC m=+0.690027233 container remove 2f2f8ba243410e4be280c10ecd8c3ce76f597265d1adfa2f219e10886e4e35ab (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-activate, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-28T23:15:29.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:29 smithi057 ceph-mon[89491]: pgmap v90: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:29.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 podman[107376]: 2024-02-28 23:15:29.174205352 +0000 UTC m=+0.042575402 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:15:29.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 podman[107376]: 2024-02-28 23:15:29.282946511 +0000 UTC m=+0.151316590 container create 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:15:29.709 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:29 smithi084 ceph-mon[70440]: pgmap v90: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:30.068 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 podman[107376]: 2024-02-28 23:15:29.572171077 +0000 UTC m=+0.440541160 container init 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-28T23:15:30.069 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 podman[107376]: 2024-02-28 23:15:29.583976919 +0000 UTC m=+0.452347032 container start 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:15:30.069 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 ceph-osd[107394]: -- 172.21.15.57:0/994027615 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bd59709680 con 0x55bd596d2c00 2024-02-28T23:15:30.069 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 bash[107376]: 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 2024-02-28T23:15:30.069 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:29 smithi057 systemd[1]: Started Ceph osd.1 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:15:30.568 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:30 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:15:30.150+0000 7fdcb187c7c0 -1 Falling back to public interface 2024-02-28T23:15:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:30 smithi057 ceph-mon[89491]: pgmap v91: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:30 smithi084 ceph-mon[70440]: pgmap v91: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:33 smithi057 ceph-mon[89491]: pgmap v92: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:33 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 49/321 objects degraded (15.265%), 16 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:33 smithi084 ceph-mon[70440]: pgmap v92: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:33 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 49/321 objects degraded (15.265%), 16 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:35.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:35 smithi057 ceph-mon[89491]: pgmap v93: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:35 smithi084 ceph-mon[70440]: pgmap v93: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/321 objects degraded (15.265%) 2024-02-28T23:15:35.818 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:35 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:15:35.364+0000 7fdcb187c7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-28T23:15:36.195 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:36 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:15:36.033+0000 7fdcb187c7c0 -1 osd.1 55 log_to_monitors true 2024-02-28T23:15:36.196 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:36 smithi057 ceph-mon[89491]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-28T23:15:36.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:36 smithi084 ceph-mon[70440]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-28T23:15:37.568 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:15:37.156+0000 7fdc9bc00700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: pgmap v94: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/321 objects degraded (15.265%) 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: osdmap e58: 6 total, 5 up, 6 in 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: pgmap v94: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/321 objects degraded (15.265%) 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: osdmap e58: 6 total, 5 up, 6 in 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:38.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:38 smithi057 ceph-mon[89491]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' 2024-02-28T23:15:38.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:38 smithi084 ceph-mon[70440]: from='osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969]' entity='osd.1' 2024-02-28T23:15:39.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: pgmap v96: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 49/321 objects degraded (15.265%) 2024-02-28T23:15:39.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:15:39.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969] boot 2024-02-28T23:15:39.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: osdmap e59: 6 total, 6 up, 6 in 2024-02-28T23:15:39.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:15:39.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:39.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: pgmap v96: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 49/321 objects degraded (15.265%) 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: osd.1 [v2:172.21.15.57:6810/3847484969,v1:172.21.15.57:6811/3847484969] boot 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: osdmap e59: 6 total, 6 up, 6 in 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:40.284 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:40 smithi057 ceph-mon[89491]: osdmap e60: 6 total, 6 up, 6 in 2024-02-28T23:15:40.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:40 smithi084 ceph-mon[70440]: osdmap e60: 6 total, 6 up, 6 in 2024-02-28T23:15:41.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:41 smithi057 ceph-mon[89491]: pgmap v99: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 21/321 objects degraded (6.542%) 2024-02-28T23:15:41.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:41 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:41 smithi084 ceph-mon[70440]: pgmap v99: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 21/321 objects degraded (6.542%) 2024-02-28T23:15:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:41 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded (PG_DEGRADED) 2024-02-28T23:15:42.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:42 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded) 2024-02-28T23:15:42.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:42 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:15:42.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:42 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded) 2024-02-28T23:15:42.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:42 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:15:43.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:43 smithi057 ceph-mon[89491]: pgmap v100: 65 pgs: 22 peering, 43 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-28T23:15:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:43 smithi084 ceph-mon[70440]: pgmap v100: 65 pgs: 22 peering, 43 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-28T23:15:46.023 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: pgmap v101: 65 pgs: 2 peering, 63 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-28T23:15:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: pgmap v101: 65 pgs: 2 peering, 63 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-28T23:15:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:46.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-28T23:15:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:46 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-28T23:15:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:46 smithi057 ceph-mon[89491]: Upgrade: osd.2 is safe to restart 2024-02-28T23:15:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:46 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-28T23:15:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:46 smithi084 ceph-mon[70440]: Upgrade: osd.2 is safe to restart 2024-02-28T23:15:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:47 smithi057 ceph-mon[89491]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-28T23:15:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:47 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:47 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:15:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:47 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:48.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.001+0000 7f80ea600700 1 -- 172.21.15.57:0/479562526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ffd60 msgr2=0x7f80e4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.001+0000 7f80ea600700 1 --2- 172.21.15.57:0/479562526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ffd60 0x7f80e4103d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009910 tx=0x7f80d0009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.001+0000 7f80ea600700 1 -- 172.21.15.57:0/479562526 shutdown_connections 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.002+0000 7f80ea600700 1 --2- 172.21.15.57:0/479562526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ffd60 0x7f80e4103d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.002+0000 7f80ea600700 1 --2- 172.21.15.57:0/479562526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ff3c0 0x7f80e40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.002+0000 7f80ea600700 1 -- 172.21.15.57:0/479562526 >> 172.21.15.57:0/479562526 conn(0x7f80e40fac80 msgr2=0x7f80e40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.002+0000 7f80ea600700 1 -- 172.21.15.57:0/479562526 shutdown_connections 2024-02-28T23:15:48.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.002+0000 7f80ea600700 1 -- 172.21.15.57:0/479562526 wait complete. 2024-02-28T23:15:48.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.003+0000 7f80ea600700 1 Processor -- start 2024-02-28T23:15:48.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.003+0000 7f80ea600700 1 -- start start 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80ea600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80ea600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ffd60 0x7f80e4101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80ea600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4101850 con 0x7f80e40ff3c0 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80ea600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4101990 con 0x7f80e40ffd60 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60196/0 (socket says 172.21.15.57:60196) 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80e3400700 1 -- 172.21.15.57:0/366030221 learned_addr learned my addr 172.21.15.57:0/366030221 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.004+0000 7f80e2a00700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ffd60 0x7f80e4101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80e3400700 1 -- 172.21.15.57:0/366030221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ffd60 msgr2=0x7f80e4101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80e3400700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ffd60 0x7f80e4101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80e3400700 1 -- 172.21.15.57:0/366030221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d00095f0 con 0x7f80e40ff3c0 2024-02-28T23:15:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80e3400700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c8a0 tx=0x7f80d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d400cea0 con 0x7f80e40ff3c0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80d4004830 con 0x7f80e40ff3c0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d40055b0 con 0x7f80e40ff3c0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.005+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e41a2060 con 0x7f80e40ff3c0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.006+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e41a2470 con 0x7f80e40ff3c0 2024-02-28T23:15:48.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.007+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4108740 con 0x7f80e40ff3c0 2024-02-28T23:15:48.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.008+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f80d40049a0 con 0x7f80e40ff3c0 2024-02-28T23:15:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.009+0000 7f80dbe00700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 0x7f80c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.009+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f80d408d040 con 0x7f80e40ff3c0 2024-02-28T23:15:48.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.009+0000 7f80e2a00700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 0x7f80c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.010+0000 7f80e2a00700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 0x7f80c806fd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80e41025c0 tx=0x7f80d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.014+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80d4056920 con 0x7f80e40ff3c0 2024-02-28T23:15:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:47 smithi084 ceph-mon[70440]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-28T23:15:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:47 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:47 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-28T23:15:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:47 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:15:48.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.273+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80e4061190 con 0x7f80c806d8a0 2024-02-28T23:15:48.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.275+0000 7f80dbe00700 1 -- 172.21.15.57:0/366030221 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f80e4061190 con 0x7f80c806d8a0 2024-02-28T23:15:48.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.280+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 msgr2=0x7f80c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.280+0000 7f80ea600700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 0x7f80c806fd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80e41025c0 tx=0x7f80d001a040 comp rx=0 tx=0).stop 2024-02-28T23:15:48.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.280+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 msgr2=0x7f80e4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.280+0000 7f80ea600700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c8a0 tx=0x7f80d400cc60 comp rx=0 tx=0).stop 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 shutdown_connections 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80c806d8a0 0x7f80c806fd50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80e40ff3c0 0x7f80e4100dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 --2- 172.21.15.57:0/366030221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e40ffd60 0x7f80e4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 >> 172.21.15.57:0/366030221 conn(0x7f80e40fac80 msgr2=0x7f80e40690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 shutdown_connections 2024-02-28T23:15:48.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.281+0000 7f80ea600700 1 -- 172.21.15.57:0/366030221 wait complete. 2024-02-28T23:15:48.304 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 -- 172.21.15.57:0/802478463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 msgr2=0x7f82e81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 --2- 172.21.15.57:0/802478463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e81008b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009910 tx=0x7f82dc009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 -- 172.21.15.57:0/802478463 shutdown_connections 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 --2- 172.21.15.57:0/802478463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 --2- 172.21.15.57:0/802478463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e81008b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 -- 172.21.15.57:0/802478463 >> 172.21.15.57:0/802478463 conn(0x7f82e80fbda0 msgr2=0x7f82e80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 -- 172.21.15.57:0/802478463 shutdown_connections 2024-02-28T23:15:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.463+0000 7f82f0400700 1 -- 172.21.15.57:0/802478463 wait complete. 2024-02-28T23:15:48.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.465+0000 7f82f0400700 1 Processor -- start 2024-02-28T23:15:48.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.465+0000 7f82f0400700 1 -- start start 2024-02-28T23:15:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.465+0000 7f82f0400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.465+0000 7f82f0400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82f0400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8073ad0 con 0x7f82e81004e0 2024-02-28T23:15:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82f0400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8073c10 con 0x7f82e8100e80 2024-02-28T23:15:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82eda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82eda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60220/0 (socket says 172.21.15.57:60220) 2024-02-28T23:15:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82eda00700 1 -- 172.21.15.57:0/168533243 learned_addr learned my addr 172.21.15.57:0/168533243 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.466+0000 7f82ed000700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82eda00700 1 -- 172.21.15.57:0/168533243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 msgr2=0x7f82e8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82eda00700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82eda00700 1 -- 172.21.15.57:0/168533243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0095f0 con 0x7f82e81004e0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82ed000700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82eda00700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0092e0 tx=0x7f82dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82dc01d070 con 0x7f82e81004e0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82dc005e00 con 0x7f82e81004e0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82dc017400 con 0x7f82e81004e0 2024-02-28T23:15:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.467+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e8073e90 con 0x7f82e81004e0 2024-02-28T23:15:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.468+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81a23f0 con 0x7f82e81004e0 2024-02-28T23:15:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.469+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e81087f0 con 0x7f82e81004e0 2024-02-28T23:15:48.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.471+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82dc00f460 con 0x7f82e81004e0 2024-02-28T23:15:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.471+0000 7f82e6000700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 0x7f82d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.472+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f82dc08e980 con 0x7f82e81004e0 2024-02-28T23:15:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.472+0000 7f82ed000700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 0x7f82d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.473+0000 7f82ed000700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 0x7f82d006fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82d4005950 tx=0x7f82d40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.477+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82dc0581c0 con 0x7f82e81004e0 2024-02-28T23:15:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.741+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82e8074620 con 0x7f82d006d8a0 2024-02-28T23:15:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.743+0000 7f82e6000700 1 -- 172.21.15.57:0/168533243 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f82e8074620 con 0x7f82d006d8a0 2024-02-28T23:15:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 msgr2=0x7f82d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 0x7f82d006fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82d4005950 tx=0x7f82d40058e0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 msgr2=0x7f82e8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0092e0 tx=0x7f82dc00be20 comp rx=0 tx=0).stop 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 shutdown_connections 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d006d8a0 0x7f82d006fd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81004e0 0x7f82e8073050 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 --2- 172.21.15.57:0/168533243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82e8100e80 0x7f82e8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 >> 172.21.15.57:0/168533243 conn(0x7f82e80fbda0 msgr2=0x7f82e80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 shutdown_connections 2024-02-28T23:15:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.746+0000 7f82f0400700 1 -- 172.21.15.57:0/168533243 wait complete. 2024-02-28T23:15:48.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:48 smithi057 ceph-mon[89491]: Upgrade: Updating osd.2 2024-02-28T23:15:48.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:48 smithi057 ceph-mon[89491]: Deploying daemon osd.2 on smithi057 2024-02-28T23:15:48.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:48.788 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 -- 172.21.15.57:0/2386201789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 msgr2=0x7fda10068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 --2- 172.21.15.57:0/2386201789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda10068b70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda00009910 tx=0x7fda00009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 -- 172.21.15.57:0/2386201789 shutdown_connections 2024-02-28T23:15:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 --2- 172.21.15.57:0/2386201789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10069140 0x7fda10104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 --2- 172.21.15.57:0/2386201789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda10068b70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 -- 172.21.15.57:0/2386201789 >> 172.21.15.57:0/2386201789 conn(0x7fda10075130 msgr2=0x7fda10075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:48.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 -- 172.21.15.57:0/2386201789 shutdown_connections 2024-02-28T23:15:48.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.903+0000 7fda16c00700 1 -- 172.21.15.57:0/2386201789 wait complete. 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.904+0000 7fda16c00700 1 Processor -- start 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.904+0000 7fda16c00700 1 -- start start 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda16c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda16c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10069140 0x7fda101985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda16c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10198cd0 con 0x7fda100687a0 2024-02-28T23:15:48.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda16c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1019ca60 con 0x7fda10069140 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda0fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda0fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60232/0 (socket says 172.21.15.57:60232) 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.905+0000 7fda0fe00700 1 -- 172.21.15.57:0/2337533910 learned_addr learned my addr 172.21.15.57:0/2337533910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0fe00700 1 -- 172.21.15.57:0/2337533910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10069140 msgr2=0x7fda101985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0fe00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10069140 0x7fda101985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0fe00700 1 -- 172.21.15.57:0/2337533910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda000095f0 con 0x7fda100687a0 2024-02-28T23:15:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0fe00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda0000b5c0 tx=0x7fda0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda0001d070 con 0x7fda100687a0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda00005e00 con 0x7fda100687a0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.906+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda00017400 con 0x7fda100687a0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.907+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1019cce0 con 0x7fda100687a0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.907+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1019d1d0 con 0x7fda100687a0 2024-02-28T23:15:48.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.908+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1004f2a0 con 0x7fda100687a0 2024-02-28T23:15:48.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.909+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fda0000f460 con 0x7fda100687a0 2024-02-28T23:15:48.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.910+0000 7fda0cc00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:48.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.910+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fda0008e8c0 con 0x7fda100687a0 2024-02-28T23:15:48.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.911+0000 7fda0f400700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:48.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.911+0000 7fda0f400700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc005950 tx=0x7fd9fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:48.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:48.916+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda000581a0 con 0x7fda100687a0 2024-02-28T23:15:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:48 smithi084 ceph-mon[70440]: Upgrade: Updating osd.2 2024-02-28T23:15:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:48 smithi084 ceph-mon[70440]: Deploying daemon osd.2 on smithi057 2024-02-28T23:15:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:15:49.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.150+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda1019d550 con 0x7fd9f406d8a0 2024-02-28T23:15:49.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.182+0000 7fda0cc00700 1 -- 172.21.15.57:0/2337533910 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fda1019d550 con 0x7fd9f406d8a0 2024-02-28T23:15:49.182 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:15:49.182 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (5m) 12s ago 19m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:15:49.182 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (15m) 12s ago 19m 8698k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:15:49.182 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (18m) 78s ago 18m 8342k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (91s) 12s ago 19m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (83s) 78s ago 18m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (4m) 12s ago 18m 75.2M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (12m) 12s ago 12m 57.2M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (11m) 12s ago 11m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (11m) 78s ago 11m 57.7M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (12m) 78s ago 12m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (7m) 12s ago 21m 499M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (6m) 78s ago 17m 450M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (3m) 12s ago 21m 47.1M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (2m) 78s ago 17m 37.1M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:15:49.183 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (6m) 12s ago 19m 7646k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (6m) 78s ago 17m 7662k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (51s) 12s ago 16m 87.5M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (19s) 12s ago 16m 11.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (15m) 12s ago 15m 202M 1218M 18.2.0 dc2bc1663786 69f473d71531 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (15m) 78s ago 14m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (14m) 78s ago 14m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (13m) 78s ago 13m 233M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:15:49.184 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (5m) 12s ago 18m 61.7M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:15:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.185+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 msgr2=0x7fd9f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.185+0000 7fda16c00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc005950 tx=0x7fd9fc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.185+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 msgr2=0x7fda101980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.185+0000 7fda16c00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda0000b5c0 tx=0x7fda0000be20 comp rx=0 tx=0).stop 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 shutdown_connections 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda100687a0 0x7fda101980b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 --2- 172.21.15.57:0/2337533910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10069140 0x7fda101985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 >> 172.21.15.57:0/2337533910 conn(0x7fda10075130 msgr2=0x7fda10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 shutdown_connections 2024-02-28T23:15:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.186+0000 7fda16c00700 1 -- 172.21.15.57:0/2337533910 wait complete. 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 -- 172.21.15.57:0/2900129710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd8106b80 msgr2=0x7f6bd8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 --2- 172.21.15.57:0/2900129710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd8106b80 0x7f6bd8106ff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009910 tx=0x7f6bc4009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 -- 172.21.15.57:0/2900129710 shutdown_connections 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 --2- 172.21.15.57:0/2900129710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd8106b80 0x7f6bd8106ff0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 --2- 172.21.15.57:0/2900129710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd810c8f0 0x7f6bd810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 -- 172.21.15.57:0/2900129710 >> 172.21.15.57:0/2900129710 conn(0x7f6bd8076020 msgr2=0x7f6bd8078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 -- 172.21.15.57:0/2900129710 shutdown_connections 2024-02-28T23:15:49.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.349+0000 7f6be0400700 1 -- 172.21.15.57:0/2900129710 wait complete. 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.350+0000 7f6be0400700 1 Processor -- start 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.350+0000 7f6be0400700 1 -- start start 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6be0400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd8106b80 0x7f6bd819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6be0400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6be0400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd819d070 con 0x7f6bd810c8f0 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6be0400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a0e00 con 0x7f6bd8106b80 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6bdd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6bdd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60242/0 (socket says 172.21.15.57:60242) 2024-02-28T23:15:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6bdd000700 1 -- 172.21.15.57:0/1428339290 learned_addr learned my addr 172.21.15.57:0/1428339290 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.351+0000 7f6bdda00700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd8106b80 0x7f6bd819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bdd000700 1 -- 172.21.15.57:0/1428339290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd8106b80 msgr2=0x7f6bd819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bdd000700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd8106b80 0x7f6bd819c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bdd000700 1 -- 172.21.15.57:0/1428339290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40095f0 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bdd000700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400b5c0 tx=0x7f6bc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc401d070 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bc4005e00 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc4017400 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd81a1080 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.352+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd81a1570 con 0x7f6bd810c8f0 2024-02-28T23:15:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.353+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd804f2a0 con 0x7f6bd810c8f0 2024-02-28T23:15:49.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.355+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6bc400f460 con 0x7f6bd810c8f0 2024-02-28T23:15:49.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.356+0000 7f6bd6000700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 0x7f6bc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.356+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6bc408ea50 con 0x7f6bd810c8f0 2024-02-28T23:15:49.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.356+0000 7f6bdda00700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 0x7f6bc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.357+0000 7f6bdda00700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 0x7f6bc006fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005950 tx=0x7f6bcc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:49.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.361+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6bc4058210 con 0x7f6bd810c8f0 2024-02-28T23:15:49.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.705+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6bd819d7b0 con 0x7f6bd810c8f0 2024-02-28T23:15:49.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.706+0000 7f6bd6000700 1 -- 172.21.15.57:0/1428339290 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f6bc405b8d0 con 0x7f6bd810c8f0 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:15:49.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:15:49.707 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:15:49.707 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-02-28T23:15:49.707 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:15:49.707 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:15:49.707 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:15:49.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.708+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 msgr2=0x7f6bc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.708+0000 7f6be0400700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 0x7f6bc006fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005950 tx=0x7f6bcc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 msgr2=0x7f6bd819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400b5c0 tx=0x7f6bc400be20 comp rx=0 tx=0).stop 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 shutdown_connections 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6bc006d8a0 0x7f6bc006fd50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd8106b80 0x7f6bd819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 --2- 172.21.15.57:0/1428339290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bd810c8f0 0x7f6bd819c990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 >> 172.21.15.57:0/1428339290 conn(0x7f6bd8076020 msgr2=0x7f6bd8077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 shutdown_connections 2024-02-28T23:15:49.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.709+0000 7f6be0400700 1 -- 172.21.15.57:0/1428339290 wait complete. 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 -- 172.21.15.57:0/2460663866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82840730f0 msgr2=0x7f82840734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 --2- 172.21.15.57:0/2460663866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82840730f0 0x7f82840734c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8274009910 tx=0x7f8274009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 -- 172.21.15.57:0/2460663866 shutdown_connections 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 --2- 172.21.15.57:0/2460663866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8284073a00 0x7f8284110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 --2- 172.21.15.57:0/2460663866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82840730f0 0x7f82840734c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 -- 172.21.15.57:0/2460663866 >> 172.21.15.57:0/2460663866 conn(0x7f82840fbb10 msgr2=0x7f82840fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 -- 172.21.15.57:0/2460663866 shutdown_connections 2024-02-28T23:15:49.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.869+0000 7f8289800700 1 -- 172.21.15.57:0/2460663866 wait complete. 2024-02-28T23:15:49.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.871+0000 7f8289800700 1 Processor -- start 2024-02-28T23:15:49.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.871+0000 7f8289800700 1 -- start start 2024-02-28T23:15:49.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8289800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82840730f0 0x7f828419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8289800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8289800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828419cd80 con 0x7f8284073a00 2024-02-28T23:15:49.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8289800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82841a0b10 con 0x7f82840730f0 2024-02-28T23:15:49.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8283400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8283400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60248/0 (socket says 172.21.15.57:60248) 2024-02-28T23:15:49.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.872+0000 7f8283400700 1 -- 172.21.15.57:0/4232577589 learned_addr learned my addr 172.21.15.57:0/4232577589 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8283e00700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82840730f0 0x7f828419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8283400700 1 -- 172.21.15.57:0/4232577589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82840730f0 msgr2=0x7f828419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8283400700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82840730f0 0x7f828419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8283400700 1 -- 172.21.15.57:0/4232577589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740095f0 con 0x7f8284073a00 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8283400700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f827000d8d0 tx=0x7f827000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8270009940 con 0x7f8284073a00 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82841a0df0 con 0x7f8284073a00 2024-02-28T23:15:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8270010460 con 0x7f8284073a00 2024-02-28T23:15:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.873+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f827000f5f0 con 0x7f8284073a00 2024-02-28T23:15:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.874+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82841a1340 con 0x7f8284073a00 2024-02-28T23:15:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.875+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828410e280 con 0x7f8284073a00 2024-02-28T23:15:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.876+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82700105d0 con 0x7f8284073a00 2024-02-28T23:15:49.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.877+0000 7f8280c00700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 0x7f826806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:49.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.877+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f827008d3c0 con 0x7f8284073a00 2024-02-28T23:15:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.877+0000 7f8283e00700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 0x7f826806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.878+0000 7f8283e00700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 0x7f826806fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8274006010 tx=0x7f827401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:49.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:49.886+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8270056ca0 con 0x7f8284073a00 2024-02-28T23:15:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:49 smithi057 ceph-mon[89491]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-02-28T23:15:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:49 smithi057 ceph-mon[89491]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:49 smithi057 ceph-mon[89491]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:49 smithi084 ceph-mon[70440]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-02-28T23:15:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:49 smithi084 ceph-mon[70440]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:49 smithi084 ceph-mon[70440]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.177+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f82841a0f80 con 0x7f8284073a00 2024-02-28T23:15:50.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.178+0000 7f8280c00700 1 -- 172.21.15.57:0/4232577589 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f8270016760 con 0x7f8284073a00 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:15:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:15:50.179 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:50.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:50.181 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:15:50.181 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:50.181 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 msgr2=0x7f826806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 0x7f826806fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8274006010 tx=0x7f827401a040 comp rx=0 tx=0).stop 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 msgr2=0x7f828419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f827000d8d0 tx=0x7f827000dc90 comp rx=0 tx=0).stop 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 shutdown_connections 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f826806d8a0 0x7f826806fd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82840730f0 0x7f828419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 --2- 172.21.15.57:0/4232577589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8284073a00 0x7f828419c6a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 >> 172.21.15.57:0/4232577589 conn(0x7f82840fbb10 msgr2=0x7f8284102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 shutdown_connections 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.181+0000 7f8289800700 1 -- 172.21.15.57:0/4232577589 wait complete. 2024-02-28T23:15:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:15:50.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 -- 172.21.15.57:0/352640202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda10106b80 msgr2=0x7fda10106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 --2- 172.21.15.57:0/352640202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda10106b80 0x7fda10106ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009910 tx=0x7fd9fc009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 -- 172.21.15.57:0/352640202 shutdown_connections 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 --2- 172.21.15.57:0/352640202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda10106b80 0x7fda10106ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 --2- 172.21.15.57:0/352640202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda1010c8f0 0x7fda1010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 -- 172.21.15.57:0/352640202 >> 172.21.15.57:0/352640202 conn(0x7fda10076060 msgr2=0x7fda10078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 -- 172.21.15.57:0/352640202 shutdown_connections 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.345+0000 7fda15e00700 1 -- 172.21.15.57:0/352640202 wait complete. 2024-02-28T23:15:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.346+0000 7fda15e00700 1 Processor -- start 2024-02-28T23:15:50.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.346+0000 7fda15e00700 1 -- start start 2024-02-28T23:15:50.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda15e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10106b80 0x7fda1019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda15e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda15e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1019d060 con 0x7fda1010c8f0 2024-02-28T23:15:50.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda15e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101a0df0 con 0x7fda10106b80 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda0e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda0e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60266/0 (socket says 172.21.15.57:60266) 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.347+0000 7fda0e000700 1 -- 172.21.15.57:0/3592965616 learned_addr learned my addr 172.21.15.57:0/3592965616 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda0ea00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10106b80 0x7fda1019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda0e000700 1 -- 172.21.15.57:0/3592965616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10106b80 msgr2=0x7fda1019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda0e000700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10106b80 0x7fda1019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda0e000700 1 -- 172.21.15.57:0/3592965616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc0095f0 con 0x7fda1010c8f0 2024-02-28T23:15:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda0e000700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc006010 tx=0x7fd9fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.348+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc01d070 con 0x7fda1010c8f0 2024-02-28T23:15:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.349+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9fc004fb0 con 0x7fda1010c8f0 2024-02-28T23:15:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.349+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda101a1070 con 0x7fda1010c8f0 2024-02-28T23:15:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.349+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc022ea0 con 0x7fda1010c8f0 2024-02-28T23:15:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.349+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda101a1560 con 0x7fda1010c8f0 2024-02-28T23:15:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.350+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1010ecc0 con 0x7fda1010c8f0 2024-02-28T23:15:50.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.351+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd9fc022470 con 0x7fda1010c8f0 2024-02-28T23:15:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.352+0000 7fda14a00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.353+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd9fc08ebf0 con 0x7fda1010c8f0 2024-02-28T23:15:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.353+0000 7fda0ea00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.354+0000 7fda0ea00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda0000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:50.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.357+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9fc058580 con 0x7fda1010c8f0 2024-02-28T23:15:50.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.605+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda101a1840 con 0x7fd9f406d8a0 2024-02-28T23:15:50.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.608+0000 7fda14a00700 1 -- 172.21.15.57:0/3592965616 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fda101a1840 con 0x7fd9f406d8a0 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "8/23 daemons upgraded", 2024-02-28T23:15:50.608 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:15:50.609 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:15:50.609 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 msgr2=0x7fd9f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda0000a560 comp rx=0 tx=0).stop 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 msgr2=0x7fda1019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc006010 tx=0x7fd9fc00be20 comp rx=0 tx=0).stop 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 shutdown_connections 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd9f406d8a0 0x7fd9f406fd50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda10106b80 0x7fda1019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.611+0000 7fda15e00700 1 --2- 172.21.15.57:0/3592965616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda1010c8f0 0x7fda1019c980 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.612+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 >> 172.21.15.57:0/3592965616 conn(0x7fda10076060 msgr2=0x7fda10077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:50.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.612+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 shutdown_connections 2024-02-28T23:15:50.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.612+0000 7fda15e00700 1 -- 172.21.15.57:0/3592965616 wait complete. 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 -- 172.21.15.57:0/1664669466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100c30 msgr2=0x7f5c44104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 --2- 172.21.15.57:0/1664669466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100c30 0x7f5c44104c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009910 tx=0x7f5c30009c20 comp rx=0 tx=0).stop 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 -- 172.21.15.57:0/1664669466 shutdown_connections 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 --2- 172.21.15.57:0/1664669466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100c30 0x7f5c44104c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 --2- 172.21.15.57:0/1664669466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100290 0x7f5c44100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 -- 172.21.15.57:0/1664669466 >> 172.21.15.57:0/1664669466 conn(0x7f5c440fbb10 msgr2=0x7f5c440fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 -- 172.21.15.57:0/1664669466 shutdown_connections 2024-02-28T23:15:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.775+0000 7f5c4a400700 1 -- 172.21.15.57:0/1664669466 wait complete. 2024-02-28T23:15:50.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.777+0000 7f5c4a400700 1 Processor -- start 2024-02-28T23:15:50.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.777+0000 7f5c4a400700 1 -- start start 2024-02-28T23:15:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c4a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c4a400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100c30 0x7f5c44073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c4a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44073ad0 con 0x7f5c44100290 2024-02-28T23:15:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c4a400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44073c10 con 0x7f5c44100c30 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c49000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c49000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60292/0 (socket says 172.21.15.57:60292) 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c49000700 1 -- 172.21.15.57:0/2968455314 learned_addr learned my addr 172.21.15.57:0/2968455314 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.778+0000 7f5c43e00700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100c30 0x7f5c44073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c49000700 1 -- 172.21.15.57:0/2968455314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100c30 msgr2=0x7f5c44073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c49000700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100c30 0x7f5c44073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c49000700 1 -- 172.21.15.57:0/2968455314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300095f0 con 0x7f5c44100290 2024-02-28T23:15:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c49000700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d8d0 tx=0x7f5c3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c38009940 con 0x7f5c44100290 2024-02-28T23:15:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c38010460 con 0x7f5c44100290 2024-02-28T23:15:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3800f5f0 con 0x7f5c44100290 2024-02-28T23:15:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.779+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c441a1e70 con 0x7f5c44100290 2024-02-28T23:15:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.780+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c441a2280 con 0x7f5c44100290 2024-02-28T23:15:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.781+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c44108600 con 0x7f5c44100290 2024-02-28T23:15:50.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.782+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5c380105d0 con 0x7f5c44100290 2024-02-28T23:15:50.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.783+0000 7f5c41600700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 0x7f5c2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:15:50.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.783+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c3808d410 con 0x7f5c44100290 2024-02-28T23:15:50.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.784+0000 7f5c43e00700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 0x7f5c2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:15:50.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.784+0000 7f5c43e00700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 0x7f5c2c06fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c44074840 tx=0x7f5c3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:15:50.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:50.787+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c38056c50 con 0x7f5c44100290 2024-02-28T23:15:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:50 smithi057 ceph-mon[89491]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:50 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1428339290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:50 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4232577589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:15:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:50 smithi084 ceph-mon[70440]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:50 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1428339290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:15:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:50 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4232577589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:15:51.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.128+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c44107fe0 con 0x7f5c44100290 2024-02-28T23:15:51.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.129+0000 7f5c41600700 1 -- 172.21.15.57:0/2968455314 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5c3805a310 con 0x7f5c44100290 2024-02-28T23:15:51.128 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:15:51.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 msgr2=0x7f5c2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 0x7f5c2c06fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c44074840 tx=0x7f5c3001a040 comp rx=0 tx=0).stop 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 msgr2=0x7f5c44073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d8d0 tx=0x7f5c3800dc90 comp rx=0 tx=0).stop 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 shutdown_connections 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5c2c06d8a0 0x7f5c2c06fd50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:51.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c44100290 0x7f5c44073050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:51.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 --2- 172.21.15.57:0/2968455314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c44100c30 0x7f5c44073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:15:51.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.131+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 >> 172.21.15.57:0/2968455314 conn(0x7f5c440fbb10 msgr2=0x7f5c440fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:15:51.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.132+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 shutdown_connections 2024-02-28T23:15:51.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:15:51.132+0000 7f5c4a400700 1 -- 172.21.15.57:0/2968455314 wait complete. 2024-02-28T23:15:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:51 smithi057 ceph-mon[89491]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-28T23:15:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:51 smithi057 ceph-mon[89491]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:51 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2968455314' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:15:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:51 smithi084 ceph-mon[70440]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-28T23:15:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:51 smithi084 ceph-mon[70440]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:15:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:51 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2968455314' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:15:52.570 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:52 smithi057 systemd[1]: Stopping Ceph osd.2 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:15:53.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[55064]: 2024-02-28T23:15:52.800+0000 7f475f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:15:53.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[55064]: 2024-02-28T23:15:52.800+0000 7f475f200700 -1 osd.2 60 *** Got signal Terminated *** 2024-02-28T23:15:53.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[55064]: 2024-02-28T23:15:52.800+0000 7f475f200700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:15:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:53 smithi057 ceph-mon[89491]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-02-28T23:15:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:53 smithi057 ceph-mon[89491]: osd.2 marked itself down and dead 2024-02-28T23:15:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:53 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:53 smithi084 ceph-mon[70440]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-02-28T23:15:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:53 smithi084 ceph-mon[70440]: osd.2 marked itself down and dead 2024-02-28T23:15:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:53 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:15:54.640 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:54 smithi057 podman[112800]: 2024-02-28 23:15:54.33608299 +0000 UTC m=+1.676316180 container died 69f473d715313cdc238108c81f1f5afc6175eaa37ed2cf86688707fff7ee449b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=HEAD, org.label-schema.build-date=20231212) 2024-02-28T23:15:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:54 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:15:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:54 smithi057 ceph-mon[89491]: osdmap e61: 6 total, 5 up, 6 in 2024-02-28T23:15:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:54 smithi057 ceph-mon[89491]: osdmap e62: 6 total, 5 up, 6 in 2024-02-28T23:15:55.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:54 smithi057 podman[112800]: 2024-02-28 23:15:54.86077932 +0000 UTC m=+2.201012580 container cleanup 69f473d715313cdc238108c81f1f5afc6175eaa37ed2cf86688707fff7ee449b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:15:55.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:54 smithi057 bash[112800]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2 2024-02-28T23:15:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:54 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:15:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:54 smithi084 ceph-mon[70440]: osdmap e61: 6 total, 5 up, 6 in 2024-02-28T23:15:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:54 smithi084 ceph-mon[70440]: osdmap e62: 6 total, 5 up, 6 in 2024-02-28T23:15:55.568 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:55 smithi057 podman[112818]: 2024-02-28 23:15:55.111639732 +0000 UTC m=+0.757439887 container remove 69f473d715313cdc238108c81f1f5afc6175eaa37ed2cf86688707fff7ee449b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True) 2024-02-28T23:15:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:55 smithi057 ceph-mon[89491]: pgmap v107: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-28T23:15:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:55 smithi057 ceph-mon[89491]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-02-28T23:15:56.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:55 smithi057 podman[112910]: 2024-02-28 23:15:55.830354889 +0000 UTC m=+0.038988893 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:15:56.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:55 smithi057 podman[112910]: 2024-02-28 23:15:55.954091469 +0000 UTC m=+0.162725515 container create d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS) 2024-02-28T23:15:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:55 smithi084 ceph-mon[70440]: pgmap v107: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-28T23:15:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:55 smithi084 ceph-mon[70440]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-02-28T23:15:56.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:56 smithi057 podman[112910]: 2024-02-28 23:15:56.308144176 +0000 UTC m=+0.516778208 container init d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-02-28T23:15:56.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:56 smithi057 podman[112910]: 2024-02-28 23:15:56.319581139 +0000 UTC m=+0.528215171 container start d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-28T23:15:56.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:56 smithi057 podman[112910]: 2024-02-28 23:15:56.406426997 +0000 UTC m=+0.615061050 container attach d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-28T23:15:57.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:56 smithi057 podman[112910]: 2024-02-28 23:15:56.777842983 +0000 UTC m=+0.986477042 container died d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-28T23:15:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:57 smithi057 ceph-mon[89491]: pgmap v109: 65 pgs: 20 peering, 4 stale+active+clean, 41 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:15:57.819 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:57 smithi057 podman[112934]: 2024-02-28 23:15:57.563815267 +0000 UTC m=+0.769886094 container remove d9e94f76562468005d5a245302a98f75a414d3d2ed0bd65a2ff612f5d0359204 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-28T23:15:57.819 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:57 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-28T23:15:57.819 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:57 smithi057 systemd[1]: Stopped Ceph osd.2 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:15:57.819 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:57 smithi057 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.2.service: Consumed 1min 41.114s CPU time. 2024-02-28T23:15:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:57 smithi084 ceph-mon[70440]: pgmap v109: 65 pgs: 20 peering, 4 stale+active+clean, 41 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-28T23:15:58.318 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 systemd[1]: Starting Ceph osd.2 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:15:58.713 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 podman[113033]: 2024-02-28 23:15:58.347599285 +0000 UTC m=+0.038096801 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:15:58.713 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 podman[113033]: 2024-02-28 23:15:58.461247886 +0000 UTC m=+0.151745436 container create 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-28T23:15:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:58 smithi057 ceph-mon[89491]: pgmap v110: 65 pgs: 3 active+undersized+degraded, 20 peering, 4 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/321 objects degraded (3.738%) 2024-02-28T23:15:59.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 podman[113033]: 2024-02-28 23:15:58.714876123 +0000 UTC m=+0.405373690 container init 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:15:59.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 podman[113033]: 2024-02-28 23:15:58.729820894 +0000 UTC m=+0.420318421 container start 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-28T23:15:59.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:58 smithi057 podman[113033]: 2024-02-28 23:15:58.838872464 +0000 UTC m=+0.529369996 container attach 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:15:59.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:58 smithi084 ceph-mon[70440]: pgmap v110: 65 pgs: 3 active+undersized+degraded, 20 peering, 4 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/321 objects degraded (3.738%) 2024-02-28T23:16:00.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 12/321 objects degraded (3.738%), 3 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:00.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 bash[113033]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-28T23:16:00.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-28T23:16:00.071 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-28T23:16:00.071 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-28T23:16:00.071 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate[113048]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-28T23:16:00.071 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:15:59 smithi057 podman[113033]: 2024-02-28 23:15:59.715042719 +0000 UTC m=+1.405540336 container died 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-28T23:16:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:15:59 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 12/321 objects degraded (3.738%), 3 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:00.773 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:00 smithi057 podman[113183]: 2024-02-28 23:16:00.425670576 +0000 UTC m=+0.690324166 container remove 71bd7e36e74829a51e6ad5783447fa2ed78fb1b36b88529ec1745cccb39f6226 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:16:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:00 smithi057 ceph-mon[89491]: pgmap v111: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:01.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:00 smithi057 podman[113244]: 2024-02-28 23:16:00.682701718 +0000 UTC m=+0.035842699 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:16:01.069 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:00 smithi057 podman[113244]: 2024-02-28 23:16:00.809890247 +0000 UTC m=+0.163031256 container create 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-28T23:16:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:00 smithi084 ceph-mon[70440]: pgmap v111: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:01.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 podman[113244]: 2024-02-28 23:16:01.09525574 +0000 UTC m=+0.448396752 container init 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:16:01.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 podman[113244]: 2024-02-28 23:16:01.108322001 +0000 UTC m=+0.461463014 container start 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-28T23:16:01.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-osd[113262]: -- 172.21.15.57:0/1875580797 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a35365680 con 0x558a3532ec00 2024-02-28T23:16:01.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 bash[113244]: 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a 2024-02-28T23:16:01.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 systemd[1]: Started Ceph osd.2 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:16:02.068 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:16:01.680+0000 7fdc8bba27c0 -1 Falling back to public interface 2024-02-28T23:16:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-mon[89491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-02-28T23:16:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:01 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:02.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:01 smithi084 ceph-mon[70440]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-02-28T23:16:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:01 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:01 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:01 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:03.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:03 smithi057 ceph-mon[89491]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:03.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:03.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:03 smithi084 ceph-mon[70440]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:05 smithi057 ceph-mon[89491]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:05.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:05 smithi084 ceph-mon[70440]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:16:06.568 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:06 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:16:06.284+0000 7fdc8bba27c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-28T23:16:07.068 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:06 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:16:06.816+0000 7fdc8bba27c0 -1 osd.2 60 log_to_monitors true 2024-02-28T23:16:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:07 smithi084 ceph-mon[70440]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 39/321 objects degraded (12.150%) 2024-02-28T23:16:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:07 smithi084 ceph-mon[70440]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-28T23:16:07.592 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:07 smithi057 ceph-mon[89491]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 39/321 objects degraded (12.150%) 2024-02-28T23:16:07.593 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:07 smithi057 ceph-mon[89491]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-28T23:16:07.593 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:16:07 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:16:07.332+0000 7fdc7b000700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:16:08.695 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:08 smithi057 ceph-mon[89491]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-28T23:16:08.695 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:08 smithi057 ceph-mon[89491]: osdmap e63: 6 total, 5 up, 6 in 2024-02-28T23:16:08.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:08 smithi057 ceph-mon[89491]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:16:08.696 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:08 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 39/321 objects degraded (12.150%), 12 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:08 smithi084 ceph-mon[70440]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-28T23:16:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:08 smithi084 ceph-mon[70440]: osdmap e63: 6 total, 5 up, 6 in 2024-02-28T23:16:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:08 smithi084 ceph-mon[70440]: from='osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-02-28T23:16:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:08 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 39/321 objects degraded (12.150%), 12 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: pgmap v116: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 39/321 objects degraded (12.150%) 2024-02-28T23:16:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:16:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626] boot 2024-02-28T23:16:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: osdmap e64: 6 total, 6 up, 6 in 2024-02-28T23:16:09.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:16:09.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:09 smithi057 ceph-mon[89491]: osdmap e65: 6 total, 6 up, 6 in 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: pgmap v116: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 39/321 objects degraded (12.150%) 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: osd.2 [v2:172.21.15.57:6818/3954650626,v1:172.21.15.57:6819/3954650626] boot 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: osdmap e64: 6 total, 6 up, 6 in 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-28T23:16:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:09 smithi084 ceph-mon[70440]: osdmap e65: 6 total, 6 up, 6 in 2024-02-28T23:16:10.934 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:10.934 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:11.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:11.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:12.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:11 smithi057 ceph-mon[89491]: pgmap v119: 65 pgs: 15 peering, 5 active+undersized+degraded, 7 active+undersized, 38 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 17/321 objects degraded (5.296%) 2024-02-28T23:16:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:11 smithi084 ceph-mon[70440]: pgmap v119: 65 pgs: 15 peering, 5 active+undersized+degraded, 7 active+undersized, 38 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 17/321 objects degraded (5.296%) 2024-02-28T23:16:13.034 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:12 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:13.034 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:12 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:13.034 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:12 smithi057 ceph-mon[89491]: pgmap v120: 65 pgs: 15 peering, 4 active+undersized+degraded, 5 active+undersized, 41 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 14/321 objects degraded (4.361%) 2024-02-28T23:16:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:12 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:12 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:12 smithi084 ceph-mon[70440]: pgmap v120: 65 pgs: 15 peering, 4 active+undersized+degraded, 5 active+undersized, 41 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 14/321 objects degraded (4.361%) 2024-02-28T23:16:14.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:13 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:13 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:14 smithi057 ceph-mon[89491]: pgmap v121: 65 pgs: 5 peering, 60 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:16:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:14 smithi084 ceph-mon[70440]: pgmap v121: 65 pgs: 5 peering, 60 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:16:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:15 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded) 2024-02-28T23:16:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:15 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:16:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:15 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded) 2024-02-28T23:16:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:15 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:16:17.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:16 smithi057 ceph-mon[89491]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:16:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:16 smithi084 ceph-mon[70440]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:19.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:19.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-28T23:16:20.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:19 smithi057 ceph-mon[89491]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:20.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:19 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-28T23:16:20.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:19 smithi057 ceph-mon[89491]: Upgrade: osd.3 is safe to restart 2024-02-28T23:16:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:19 smithi084 ceph-mon[70440]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:19 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-28T23:16:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:19 smithi084 ceph-mon[70440]: Upgrade: osd.3 is safe to restart 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 -- 172.21.15.57:0/1040784131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 msgr2=0x7f58f4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 --2- 172.21.15.57:0/1040784131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f4106ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009870 tx=0x7f58e0009b80 comp rx=0 tx=0).stop 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 -- 172.21.15.57:0/1040784131 shutdown_connections 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 --2- 172.21.15.57:0/1040784131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f4106ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 --2- 172.21.15.57:0/1040784131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 0x7f58f410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 -- 172.21.15.57:0/1040784131 >> 172.21.15.57:0/1040784131 conn(0x7f58f4076020 msgr2=0x7f58f4078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 -- 172.21.15.57:0/1040784131 shutdown_connections 2024-02-28T23:16:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.310+0000 7f58fa200700 1 -- 172.21.15.57:0/1040784131 wait complete. 2024-02-28T23:16:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.311+0000 7f58fa200700 1 Processor -- start 2024-02-28T23:16:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.311+0000 7f58fa200700 1 -- start start 2024-02-28T23:16:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58fa200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58fa200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 0x7f58f419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58fa200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f419d070 con 0x7f58f4106b80 2024-02-28T23:16:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58fa200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41a0e00 con 0x7f58f410c8f0 2024-02-28T23:16:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44498/0 (socket says 172.21.15.57:44498) 2024-02-28T23:16:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58f3400700 1 -- 172.21.15.57:0/3997983329 learned_addr learned my addr 172.21.15.57:0/3997983329 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.312+0000 7f58f2a00700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 0x7f58f419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58f3400700 1 -- 172.21.15.57:0/3997983329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 msgr2=0x7f58f419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58f3400700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 0x7f58f419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58f3400700 1 -- 172.21.15.57:0/3997983329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e40095f0 con 0x7f58f4106b80 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58f3400700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58e400eb90 tx=0x7f58e400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e400cd70 con 0x7f58f4106b80 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e40044e0 con 0x7f58f4106b80 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e0009520 con 0x7f58f4106b80 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e4010660 con 0x7f58f4106b80 2024-02-28T23:16:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.313+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f41a14a0 con 0x7f58f4106b80 2024-02-28T23:16:21.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.314+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f404f2a0 con 0x7f58f4106b80 2024-02-28T23:16:21.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.316+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f58e40038a0 con 0x7f58f4106b80 2024-02-28T23:16:21.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.316+0000 7f58ebe00700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 0x7f58d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.317+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f58e4014070 con 0x7f58f4106b80 2024-02-28T23:16:21.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.317+0000 7f58f2a00700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 0x7f58d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.318+0000 7f58f2a00700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 0x7f58d806fd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009f50 tx=0x7f58e0019040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:21.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.321+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58e4057470 con 0x7f58f4106b80 2024-02-28T23:16:21.548 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:16:21.548 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: Upgrade: Updating osd.3 2024-02-28T23:16:21.549 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:21.549 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:16:21.549 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:21.549 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:21 smithi084 ceph-mon[70440]: Deploying daemon osd.3 on smithi084 2024-02-28T23:16:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:16:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: Upgrade: Updating osd.3 2024-02-28T23:16:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:21.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-28T23:16:21.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:21.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:21 smithi057 ceph-mon[89491]: Deploying daemon osd.3 on smithi084 2024-02-28T23:16:21.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.580+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58f41a1880 con 0x7f58d806d8a0 2024-02-28T23:16:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.582+0000 7f58ebe00700 1 -- 172.21.15.57:0/3997983329 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f58f41a1880 con 0x7f58d806d8a0 2024-02-28T23:16:21.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 msgr2=0x7f58d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 0x7f58d806fd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009f50 tx=0x7f58e0019040 comp rx=0 tx=0).stop 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 msgr2=0x7f58f419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58e400eb90 tx=0x7f58e400ef50 comp rx=0 tx=0).stop 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 shutdown_connections 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58d806d8a0 0x7f58d806fd50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58f4106b80 0x7f58f419c450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 --2- 172.21.15.57:0/3997983329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f410c8f0 0x7f58f419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 >> 172.21.15.57:0/3997983329 conn(0x7f58f4076020 msgr2=0x7f58f4077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 shutdown_connections 2024-02-28T23:16:21.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.585+0000 7f58fa200700 1 -- 172.21.15.57:0/3997983329 wait complete. 2024-02-28T23:16:21.600 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 -- 172.21.15.57:0/3574309702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c8106b80 msgr2=0x7f26c8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 --2- 172.21.15.57:0/3574309702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c8106b80 0x7f26c8106ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009910 tx=0x7f26b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 -- 172.21.15.57:0/3574309702 shutdown_connections 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 --2- 172.21.15.57:0/3574309702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c8106b80 0x7f26c8106ff0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 --2- 172.21.15.57:0/3574309702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c810c8f0 0x7f26c810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.754+0000 7f26cf800700 1 -- 172.21.15.57:0/3574309702 >> 172.21.15.57:0/3574309702 conn(0x7f26c8076060 msgr2=0x7f26c8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.755+0000 7f26cf800700 1 -- 172.21.15.57:0/3574309702 shutdown_connections 2024-02-28T23:16:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.755+0000 7f26cf800700 1 -- 172.21.15.57:0/3574309702 wait complete. 2024-02-28T23:16:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 Processor -- start 2024-02-28T23:16:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 -- start start 2024-02-28T23:16:21.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c8106b80 0x7f26c819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c819d060 con 0x7f26c810c8f0 2024-02-28T23:16:21.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.756+0000 7f26cf800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c81a0df0 con 0x7f26c8106b80 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44526/0 (socket says 172.21.15.57:44526) 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 -- 172.21.15.57:0/1487958470 learned_addr learned my addr 172.21.15.57:0/1487958470 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26cce00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c8106b80 0x7f26c819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 -- 172.21.15.57:0/1487958470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c8106b80 msgr2=0x7f26c819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c8106b80 0x7f26c819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:21.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.757+0000 7f26bfe00700 1 -- 172.21.15.57:0/1487958470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00095f0 con 0x7f26c810c8f0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26bfe00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b5c0 tx=0x7f26b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26b001d070 con 0x7f26c810c8f0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26b0005e00 con 0x7f26c810c8f0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26cf800700 1 -- 172.21.15.57:0/1487958470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c81a1070 con 0x7f26c810c8f0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26b0017400 con 0x7f26c810c8f0 2024-02-28T23:16:21.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.758+0000 7f26cf800700 1 -- 172.21.15.57:0/1487958470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c81a1560 con 0x7f26c810c8f0 2024-02-28T23:16:21.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.760+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a00052f0 con 0x7f26c810c8f0 2024-02-28T23:16:21.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.764+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f26b000f6f0 con 0x7f26c810c8f0 2024-02-28T23:16:21.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.765+0000 7f26c6000700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 0x7f26ac06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:21.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.765+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f26b008e910 con 0x7f26c810c8f0 2024-02-28T23:16:21.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.766+0000 7f26cce00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 0x7f26ac06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:21.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.767+0000 7f26cce00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 0x7f26ac06fda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26b8005950 tx=0x7f26b800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:21.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:21.771+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26b0058150 con 0x7f26c810c8f0 2024-02-28T23:16:22.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.022+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26a0000bc0 con 0x7f26ac06d8f0 2024-02-28T23:16:22.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.025+0000 7f26c6000700 1 -- 172.21.15.57:0/1487958470 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f26a0000bc0 con 0x7f26ac06d8f0 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 msgr2=0x7f26ac06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 0x7f26ac06fda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26b8005950 tx=0x7f26b800b410 comp rx=0 tx=0).stop 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 msgr2=0x7f26c819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b5c0 tx=0x7f26b000bd70 comp rx=0 tx=0).stop 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 shutdown_connections 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f26ac06d8f0 0x7f26ac06fda0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26c8106b80 0x7f26c819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 --2- 172.21.15.57:0/1487958470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26c810c8f0 0x7f26c819c980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.028+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 >> 172.21.15.57:0/1487958470 conn(0x7f26c8076060 msgr2=0x7f26c8077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:22.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.029+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 shutdown_connections 2024-02-28T23:16:22.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.029+0000 7f26bea00700 1 -- 172.21.15.57:0/1487958470 wait complete. 2024-02-28T23:16:22.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.198+0000 7f2a30800700 1 -- 172.21.15.57:0/3745046641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 msgr2=0x7f2a28104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.198+0000 7f2a30800700 1 --2- 172.21.15.57:0/3745046641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28104e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009910 tx=0x7f2a18009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 -- 172.21.15.57:0/3745046641 shutdown_connections 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 --2- 172.21.15.57:0/3745046641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28104e50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 --2- 172.21.15.57:0/3745046641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a281008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 -- 172.21.15.57:0/3745046641 >> 172.21.15.57:0/3745046641 conn(0x7f2a280fbda0 msgr2=0x7f2a280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 -- 172.21.15.57:0/3745046641 shutdown_connections 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.199+0000 7f2a30800700 1 -- 172.21.15.57:0/3745046641 wait complete. 2024-02-28T23:16:22.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.200+0000 7f2a30800700 1 Processor -- start 2024-02-28T23:16:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.200+0000 7f2a30800700 1 -- start start 2024-02-28T23:16:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a30800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a28195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a30800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a30800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28196b50 con 0x7f2a28100e80 2024-02-28T23:16:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a30800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2819a8e0 con 0x7f2a281004e0 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a2d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a2d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44536/0 (socket says 172.21.15.57:44536) 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a2d400700 1 -- 172.21.15.57:0/3786849819 learned_addr learned my addr 172.21.15.57:0/3786849819 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.201+0000 7f2a2de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a28195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.202+0000 7f2a2d400700 1 -- 172.21.15.57:0/3786849819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 msgr2=0x7f2a28195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.202+0000 7f2a2d400700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a28195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.202+0000 7f2a2d400700 1 -- 172.21.15.57:0/3786849819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180095f0 con 0x7f2a28100e80 2024-02-28T23:16:22.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.202+0000 7f2a2de00700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a28195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:16:22.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.202+0000 7f2a2d400700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a18006010 tx=0x7f2a1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:22.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.203+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1801d070 con 0x7f2a28100e80 2024-02-28T23:16:22.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.203+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a180050d0 con 0x7f2a28100e80 2024-02-28T23:16:22.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.203+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a18017400 con 0x7f2a28100e80 2024-02-28T23:16:22.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.203+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2819ab60 con 0x7f2a28100e80 2024-02-28T23:16:22.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.203+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2819b050 con 0x7f2a28100e80 2024-02-28T23:16:22.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.204+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a28108850 con 0x7f2a28100e80 2024-02-28T23:16:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.205+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a1800faf0 con 0x7f2a28100e80 2024-02-28T23:16:22.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.206+0000 7f2a22a00700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.206+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2a1808e740 con 0x7f2a28100e80 2024-02-28T23:16:22.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.206+0000 7f2a2de00700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.207+0000 7f2a2de00700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c011490 tx=0x7f2a1c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.211+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a18057f80 con 0x7f2a28100e80 2024-02-28T23:16:22.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.461+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a281973a0 con 0x7f2a1006d8a0 2024-02-28T23:16:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.483+0000 7f2a22a00700 1 -- 172.21.15.57:0/3786849819 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2a281973a0 con 0x7f2a1006d8a0 2024-02-28T23:16:22.482 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:16:22.482 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (6m) 12s ago 19m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:16:22.482 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (16m) 12s ago 20m 9000k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (18m) 111s ago 18m 8342k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (2m) 12s ago 20m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (116s) 111s ago 18m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (5m) 12s ago 19m 75.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (12m) 12s ago 12m 56.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (12m) 12s ago 12m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (12m) 111s ago 12m 57.7M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (12m) 111s ago 12m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (8m) 12s ago 21m 500M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (7m) 111s ago 18m 450M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (3m) 12s ago 21m 48.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (3m) 111s ago 18m 37.1M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (7m) 12s ago 20m 7982k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (6m) 111s ago 18m 7662k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:16:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (85s) 12s ago 17m 87.7M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (53s) 12s ago 16m 99.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (21s) 12s ago 16m 11.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (15m) 111s ago 15m 273M 2241M 18.2.0 dc2bc1663786 84d8887a8903 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (14m) 111s ago 14m 239M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (14m) 111s ago 14m 233M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:16:22.484 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (6m) 12s ago 19m 63.0M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:16:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 msgr2=0x7f2a1006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c011490 tx=0x7f2a1c009260 comp rx=0 tx=0).stop 2024-02-28T23:16:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 msgr2=0x7f2a28196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a18006010 tx=0x7f2a1800f690 comp rx=0 tx=0).stop 2024-02-28T23:16:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 shutdown_connections 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a281004e0 0x7f2a28195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 --2- 172.21.15.57:0/3786849819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a28100e80 0x7f2a28196470 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.487+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 >> 172.21.15.57:0/3786849819 conn(0x7f2a280fbda0 msgr2=0x7f2a280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.488+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 shutdown_connections 2024-02-28T23:16:22.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.488+0000 7f2a30800700 1 -- 172.21.15.57:0/3786849819 wait complete. 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 -- 172.21.15.57:0/3826731922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff764103040 msgr2=0x7ff764107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3826731922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff764103040 0x7ff764107010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff750009910 tx=0x7ff750009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 -- 172.21.15.57:0/3826731922 shutdown_connections 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3826731922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff764103040 0x7ff764107010 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3826731922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7641026a0 0x7ff764102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 -- 172.21.15.57:0/3826731922 >> 172.21.15.57:0/3826731922 conn(0x7ff7640fdf40 msgr2=0x7ff764100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 -- 172.21.15.57:0/3826731922 shutdown_connections 2024-02-28T23:16:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.652+0000 7ff76aa00700 1 -- 172.21.15.57:0/3826731922 wait complete. 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.653+0000 7ff76aa00700 1 Processor -- start 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.653+0000 7ff76aa00700 1 -- start start 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff76aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff76aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff764103040 0x7ff764198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff76aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764198c70 con 0x7ff7641026a0 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff76aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76419c990 con 0x7ff764103040 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff763e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff763e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44548/0 (socket says 172.21.15.57:44548) 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff763e00700 1 -- 172.21.15.57:0/3361177459 learned_addr learned my addr 172.21.15.57:0/3361177459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.654+0000 7ff763400700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff764103040 0x7ff764198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff763e00700 1 -- 172.21.15.57:0/3361177459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff764103040 msgr2=0x7ff764198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff763e00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff764103040 0x7ff764198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff763e00700 1 -- 172.21.15.57:0/3361177459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500095f0 con 0x7ff7641026a0 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff763e00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e920 tx=0x7ff75400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff754009790 con 0x7ff7641026a0 2024-02-28T23:16:22.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff75400cba0 con 0x7ff7641026a0 2024-02-28T23:16:22.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff754018650 con 0x7ff7641026a0 2024-02-28T23:16:22.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76419cc70 con 0x7ff7641026a0 2024-02-28T23:16:22.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.655+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76419d1c0 con 0x7ff7641026a0 2024-02-28T23:16:22.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.656+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76410aa10 con 0x7ff7641026a0 2024-02-28T23:16:22.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.657+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff75400cd10 con 0x7ff7641026a0 2024-02-28T23:16:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.658+0000 7ff760c00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 0x7ff74806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.659+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff754014070 con 0x7ff7641026a0 2024-02-28T23:16:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.659+0000 7ff763400700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 0x7ff74806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.659+0000 7ff763400700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 0x7ff74806fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff750006010 tx=0x7ff750005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.663+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff754052d20 con 0x7ff7641026a0 2024-02-28T23:16:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:22 smithi057 ceph-mon[89491]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:22 smithi084 ceph-mon[70440]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:22.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.987+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff76419d3e0 con 0x7ff7641026a0 2024-02-28T23:16:22.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.988+0000 7ff760c00700 1 -- 172.21.15.57:0/3361177459 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7ff75405ac50 con 0x7ff7641026a0 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:16:22.987 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-02-28T23:16:22.988 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 7 2024-02-28T23:16:22.989 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:16:22.989 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:16:22.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.991+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 msgr2=0x7ff74806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.991+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 0x7ff74806fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff750006010 tx=0x7ff750005a90 comp rx=0 tx=0).stop 2024-02-28T23:16:22.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 msgr2=0x7ff764198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:22.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e920 tx=0x7ff75400ece0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 shutdown_connections 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff74806d8a0 0x7ff74806fd50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641026a0 0x7ff764198050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 --2- 172.21.15.57:0/3361177459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff764103040 0x7ff764198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 >> 172.21.15.57:0/3361177459 conn(0x7ff7640fdf40 msgr2=0x7ff764100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 shutdown_connections 2024-02-28T23:16:22.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:22.992+0000 7ff76aa00700 1 -- 172.21.15.57:0/3361177459 wait complete. 2024-02-28T23:16:23.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.159+0000 7fc3bde00700 1 -- 172.21.15.57:0/267184276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8108470 msgr2=0x7fc3b8108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.159+0000 7fc3bde00700 1 --2- 172.21.15.57:0/267184276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8108470 0x7fc3b8108840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009870 tx=0x7fc3a8009b80 comp rx=0 tx=0).stop 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 -- 172.21.15.57:0/267184276 shutdown_connections 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 --2- 172.21.15.57:0/267184276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8102470 0x7fc3b81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 --2- 172.21.15.57:0/267184276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8108470 0x7fc3b8108840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 -- 172.21.15.57:0/267184276 >> 172.21.15.57:0/267184276 conn(0x7fc3b80fdf90 msgr2=0x7fc3b81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 -- 172.21.15.57:0/267184276 shutdown_connections 2024-02-28T23:16:23.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.160+0000 7fc3bde00700 1 -- 172.21.15.57:0/267184276 wait complete. 2024-02-28T23:16:23.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.161+0000 7fc3bde00700 1 Processor -- start 2024-02-28T23:16:23.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.161+0000 7fc3bde00700 1 -- start start 2024-02-28T23:16:23.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3bde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3bde00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 0x7fc3b8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3bde00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8198e10 con 0x7fc3b8102470 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3bde00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b819cba0 con 0x7fc3b8108470 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44552/0 (socket says 172.21.15.57:44552) 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 -- 172.21.15.57:0/947504234 learned_addr learned my addr 172.21.15.57:0/947504234 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6000700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 0x7fc3b8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 -- 172.21.15.57:0/947504234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 msgr2=0x7fc3b8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 0x7fc3b8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6a00700 1 -- 172.21.15.57:0/947504234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009520 con 0x7fc3b8102470 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.162+0000 7fc3b6000700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 0x7fc3b8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3b6a00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009230 tx=0x7fc3a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3a801c070 con 0x7fc3b8102470 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3a800fbf0 con 0x7fc3b8102470 2024-02-28T23:16:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b819ce20 con 0x7fc3b8102470 2024-02-28T23:16:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3a8022e80 con 0x7fc3b8102470 2024-02-28T23:16:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.163+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b819d310 con 0x7fc3b8102470 2024-02-28T23:16:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.164+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b810a9e0 con 0x7fc3b8102470 2024-02-28T23:16:23.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.165+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc3a8020070 con 0x7fc3b8102470 2024-02-28T23:16:23.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.166+0000 7fc3bca00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 0x7fc39c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.167+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc3a808e5d0 con 0x7fc3b8102470 2024-02-28T23:16:23.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.167+0000 7fc3b6000700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 0x7fc39c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.167+0000 7fc3b6000700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 0x7fc39c06fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8199810 tx=0x7fc3a4009310 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:23.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.172+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3a8057e40 con 0x7fc3b8102470 2024-02-28T23:16:23.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.447+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc3b804f2a0 con 0x7fc3b8102470 2024-02-28T23:16:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.448+0000 7fc3bca00700 1 -- 172.21.15.57:0/947504234 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fc3a805b500 con 0x7fc3b8102470 2024-02-28T23:16:23.446 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:16:23.446 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:16:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:16:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:23.449 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:23.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.451+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 msgr2=0x7fc39c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.451+0000 7fc3bde00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 0x7fc39c06fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8199810 tx=0x7fc3a4009310 comp rx=0 tx=0).stop 2024-02-28T23:16:23.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.451+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 msgr2=0x7fc3b81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.451+0000 7fc3bde00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009230 tx=0x7fc3a80047b0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 shutdown_connections 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc39c06d8a0 0x7fc39c06fd50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3b8102470 0x7fc3b81981f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 --2- 172.21.15.57:0/947504234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3b8108470 0x7fc3b8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 >> 172.21.15.57:0/947504234 conn(0x7fc3b80fdf90 msgr2=0x7fc3b81002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 shutdown_connections 2024-02-28T23:16:23.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.452+0000 7fc3bde00700 1 -- 172.21.15.57:0/947504234 wait complete. 2024-02-28T23:16:23.452 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:16:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 -- 172.21.15.57:0/2711314396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 msgr2=0x7fe32010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 --2- 172.21.15.57:0/2711314396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32010ccc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe310009910 tx=0x7fe310009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 -- 172.21.15.57:0/2711314396 shutdown_connections 2024-02-28T23:16:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 --2- 172.21.15.57:0/2711314396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe320106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 --2- 172.21.15.57:0/2711314396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32010ccc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 -- 172.21.15.57:0/2711314396 >> 172.21.15.57:0/2711314396 conn(0x7fe320076060 msgr2=0x7fe320078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:23.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 -- 172.21.15.57:0/2711314396 shutdown_connections 2024-02-28T23:16:23.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.617+0000 7fe325c00700 1 -- 172.21.15.57:0/2711314396 wait complete. 2024-02-28T23:16:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.618+0000 7fe325c00700 1 Processor -- start 2024-02-28T23:16:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe325c00700 1 -- start start 2024-02-28T23:16:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe325c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe32019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe325c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe325c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32019d120 con 0x7fe32010c8f0 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe325c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3201a0eb0 con 0x7fe320106b80 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe31e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe31e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44576/0 (socket says 172.21.15.57:44576) 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.619+0000 7fe31e000700 1 -- 172.21.15.57:0/1014416845 learned_addr learned my addr 172.21.15.57:0/1014416845 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31ea00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe32019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31e000700 1 -- 172.21.15.57:0/1014416845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 msgr2=0x7fe32019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31e000700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe32019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31e000700 1 -- 172.21.15.57:0/1014416845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100095f0 con 0x7fe32010c8f0 2024-02-28T23:16:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31ea00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe32019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe31e000700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00d8d0 tx=0x7fe30c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.620+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe30c009940 con 0x7fe32010c8f0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.621+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe30c010460 con 0x7fe32010c8f0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.621+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3201a1190 con 0x7fe32010c8f0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.621+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe30c00f5f0 con 0x7fe32010c8f0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.621+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3201a16e0 con 0x7fe32010c8f0 2024-02-28T23:16:23.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.622+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe320109b20 con 0x7fe32010c8f0 2024-02-28T23:16:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.623+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe30c0105d0 con 0x7fe32010c8f0 2024-02-28T23:16:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.624+0000 7fe317400700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.625+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe30c08d410 con 0x7fe32010c8f0 2024-02-28T23:16:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.625+0000 7fe31ea00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:23.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.626+0000 7fe31ea00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe310005b40 tx=0x7fe31001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:23.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.629+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe30c056c50 con 0x7fe32010c8f0 2024-02-28T23:16:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:16:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3361177459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/947504234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:16:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:16:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3361177459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:23.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/947504234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:16:23.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.879+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3201a19c0 con 0x7fe30406d8a0 2024-02-28T23:16:23.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.882+0000 7fe317400700 1 -- 172.21.15.57:0/1014416845 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe3201a19c0 con 0x7fe30406d8a0 2024-02-28T23:16:23.881 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:16:23.881 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:16:23.881 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:16:23.881 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "9/23 daemons upgraded", 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:16:23.882 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:16:23.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.885+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 msgr2=0x7fe30406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.885+0000 7fe325c00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe310005b40 tx=0x7fe31001a040 comp rx=0 tx=0).stop 2024-02-28T23:16:23.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.885+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 msgr2=0x7fe32019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.885+0000 7fe325c00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00d8d0 tx=0x7fe30c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 shutdown_connections 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe320106b80 0x7fe32019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 --2- 172.21.15.57:0/1014416845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe32010c8f0 0x7fe32019ca40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 >> 172.21.15.57:0/1014416845 conn(0x7fe320076060 msgr2=0x7fe320077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 shutdown_connections 2024-02-28T23:16:23.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:23.886+0000 7fe325c00700 1 -- 172.21.15.57:0/1014416845 wait complete. 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 -- 172.21.15.57:0/2647612330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75581004e0 msgr2=0x7f75581008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 --2- 172.21.15.57:0/2647612330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75581004e0 0x7f75581008b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f754c00b0a0 tx=0x7f754c00b3b0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 -- 172.21.15.57:0/2647612330 shutdown_connections 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 --2- 172.21.15.57:0/2647612330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7558100e80 0x7f7558104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 --2- 172.21.15.57:0/2647612330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75581004e0 0x7f75581008b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 -- 172.21.15.57:0/2647612330 >> 172.21.15.57:0/2647612330 conn(0x7f75580fbda0 msgr2=0x7f75580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.048+0000 7f755f800700 1 -- 172.21.15.57:0/2647612330 shutdown_connections 2024-02-28T23:16:24.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.049+0000 7f755f800700 1 -- 172.21.15.57:0/2647612330 wait complete. 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.049+0000 7f755f800700 1 Processor -- start 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f755f800700 1 -- start start 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f755f800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75581004e0 0x7f7558198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f755f800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f755f800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558198cd0 con 0x7f7558100e80 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f755f800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558198e10 con 0x7f75581004e0 2024-02-28T23:16:24.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f7557e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f7557e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44596/0 (socket says 172.21.15.57:44596) 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.050+0000 7f7557e00700 1 -- 172.21.15.57:0/3176440198 learned_addr learned my addr 172.21.15.57:0/3176440198 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f755ce00700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75581004e0 0x7f7558198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7557e00700 1 -- 172.21.15.57:0/3176440198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75581004e0 msgr2=0x7f7558198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7557e00700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75581004e0 0x7f7558198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7557e00700 1 -- 172.21.15.57:0/3176440198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009d00 con 0x7f7558100e80 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7557e00700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f754400c960 tx=0x7f754400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7544007a70 con 0x7f7558100e80 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7544007bd0 con 0x7f7558100e80 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75440186c0 con 0x7f7558100e80 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.051+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755819cc60 con 0x7f7558100e80 2024-02-28T23:16:24.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.052+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755819d1b0 con 0x7f7558100e80 2024-02-28T23:16:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.053+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75581045d0 con 0x7f7558100e80 2024-02-28T23:16:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.054+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f754400faa0 con 0x7f7558100e80 2024-02-28T23:16:24.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.055+0000 7f7555600700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 0x7f754006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:24.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.056+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f754408d410 con 0x7f7558100e80 2024-02-28T23:16:24.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.056+0000 7f755ce00700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 0x7f754006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:24.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.056+0000 7f755ce00700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 0x7f754006fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f754c009480 tx=0x7f754c00b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:24.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.060+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f754405a820 con 0x7f7558100e80 2024-02-28T23:16:24.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.381+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7558103fb0 con 0x7f7558100e80 2024-02-28T23:16:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.382+0000 7f7555600700 1 -- 172.21.15.57:0/3176440198 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f754405a3b0 con 0x7f7558100e80 2024-02-28T23:16:24.381 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 msgr2=0x7f754006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 0x7f754006fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f754c009480 tx=0x7f754c00b7f0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 msgr2=0x7f7558198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f754400c960 tx=0x7f754400cd20 comp rx=0 tx=0).stop 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 shutdown_connections 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f754006d8a0 0x7f754006fd50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75581004e0 0x7f7558198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 --2- 172.21.15.57:0/3176440198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7558100e80 0x7f7558198680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:24.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 >> 172.21.15.57:0/3176440198 conn(0x7f75580fbda0 msgr2=0x7f75580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:24.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.386+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 shutdown_connections 2024-02-28T23:16:24.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:24.387+0000 7f755f800700 1 -- 172.21.15.57:0/3176440198 wait complete. 2024-02-28T23:16:24.817 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:24 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3176440198' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:16:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:24 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3176440198' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:16:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:25 smithi084 ceph-mon[70440]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:25 smithi084 ceph-mon[70440]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:16:25.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:25 smithi084 systemd[1]: Stopping Ceph osd.3 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:16:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:25 smithi057 ceph-mon[89491]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:25.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:25 smithi057 ceph-mon[89491]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:16:26.080 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:25 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[40743]: 2024-02-28T23:16:25.716+0000 7f2b4e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:16:26.081 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:25 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[40743]: 2024-02-28T23:16:25.716+0000 7f2b4e000700 -1 osd.3 65 *** Got signal Terminated *** 2024-02-28T23:16:26.081 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:25 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[40743]: 2024-02-28T23:16:25.716+0000 7f2b4e000700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:16:26.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:26 smithi084 ceph-mon[70440]: osd.3 marked itself down and dead 2024-02-28T23:16:26.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:26 smithi057 ceph-mon[89491]: osd.3 marked itself down and dead 2024-02-28T23:16:27.276 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:27 smithi084 podman[79555]: 2024-02-28 23:16:27.002486687 +0000 UTC m=+1.414456061 container died 84d8887a89034007ce05c480c3d0f520381266dff0cd01fa903bc5290fc0ba4d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=HEAD) 2024-02-28T23:16:27.526 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:27 smithi084 ceph-mon[70440]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:27.526 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:27 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:16:27.526 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:27 smithi084 ceph-mon[70440]: osdmap e66: 6 total, 5 up, 6 in 2024-02-28T23:16:27.526 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:27 smithi084 podman[79555]: 2024-02-28 23:16:27.402629569 +0000 UTC m=+1.814598954 container cleanup 84d8887a89034007ce05c480c3d0f520381266dff0cd01fa903bc5290fc0ba4d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:16:27.526 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:27 smithi084 bash[79555]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3 2024-02-28T23:16:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:27 smithi057 ceph-mon[89491]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:27.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:27 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:16:27.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:27 smithi057 ceph-mon[89491]: osdmap e66: 6 total, 5 up, 6 in 2024-02-28T23:16:27.830 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:27 smithi084 podman[79571]: 2024-02-28 23:16:27.677814386 +0000 UTC m=+0.653824398 container remove 84d8887a89034007ce05c480c3d0f520381266dff0cd01fa903bc5290fc0ba4d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:16:28.807 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:28 smithi084 ceph-mon[70440]: osdmap e67: 6 total, 5 up, 6 in 2024-02-28T23:16:28.808 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:28 smithi084 podman[79658]: 2024-02-28 23:16:28.397795908 +0000 UTC m=+0.035596003 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:16:28.808 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:28 smithi084 podman[79658]: 2024-02-28 23:16:28.546977257 +0000 UTC m=+0.184777419 container create b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:16:28.808 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:28 smithi084 podman[79658]: 2024-02-28 23:16:28.80909602 +0000 UTC m=+0.446896159 container init b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-28T23:16:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:28 smithi057 ceph-mon[89491]: osdmap e67: 6 total, 5 up, 6 in 2024-02-28T23:16:29.080 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:28 smithi084 podman[79658]: 2024-02-28 23:16:28.820170499 +0000 UTC m=+0.457970611 container start b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-02-28T23:16:29.080 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:28 smithi084 podman[79658]: 2024-02-28 23:16:28.949286641 +0000 UTC m=+0.587086796 container attach b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-28T23:16:29.551 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:29 smithi084 podman[79658]: 2024-02-28 23:16:29.253864459 +0000 UTC m=+0.891664574 container died b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-28T23:16:29.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:29 smithi057 ceph-mon[89491]: pgmap v130: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-02-28T23:16:29.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:29 smithi057 ceph-mon[89491]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-28T23:16:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:29 smithi084 ceph-mon[70440]: pgmap v130: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-02-28T23:16:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:29 smithi084 ceph-mon[70440]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-28T23:16:30.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:29 smithi084 podman[79683]: 2024-02-28 23:16:29.994718649 +0000 UTC m=+0.722726312 container remove b5f55a98d41e26c13e729a11227497e2d9ca325aa2d69a9cc65f3934dd416b29 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-28T23:16:30.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-28T23:16:30.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 systemd[1]: Stopped Ceph osd.3 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:16:30.331 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.3.service: Consumed 2min 6.470s CPU time. 2024-02-28T23:16:30.737 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:30 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 45/321 objects degraded (14.019%), 12 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:30.737 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 systemd[1]: Starting Ceph osd.3 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:16:31.006 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 podman[79782]: 2024-02-28 23:16:30.643798011 +0000 UTC m=+0.032923312 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:16:31.006 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:30 smithi084 podman[79782]: 2024-02-28 23:16:30.762197851 +0000 UTC m=+0.151323185 container create 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:16:31.006 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 podman[79782]: 2024-02-28 23:16:31.007056488 +0000 UTC m=+0.396181832 container init 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=) 2024-02-28T23:16:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:30 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 45/321 objects degraded (14.019%), 12 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:31.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 podman[79782]: 2024-02-28 23:16:31.01786905 +0000 UTC m=+0.406994393 container start 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-02-28T23:16:31.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 podman[79782]: 2024-02-28 23:16:31.13969594 +0000 UTC m=+0.528821324 container attach 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-28T23:16:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-mon[70440]: pgmap v131: 65 pgs: 12 active+undersized+degraded, 4 peering, 2 stale+active+clean, 13 active+undersized, 34 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/321 objects degraded (14.019%) 2024-02-28T23:16:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:31 smithi057 ceph-mon[89491]: pgmap v131: 65 pgs: 12 active+undersized+degraded, 4 peering, 2 stale+active+clean, 13 active+undersized, 34 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/321 objects degraded (14.019%) 2024-02-28T23:16:32.186 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-28T23:16:32.186 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-28T23:16:32.186 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-28T23:16:32.186 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 bash[79782]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate[79797]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-28T23:16:32.187 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:31 smithi084 podman[79782]: 2024-02-28 23:16:31.887505255 +0000 UTC m=+1.276630599 container died 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-02-28T23:16:32.830 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:32 smithi084 podman[79922]: 2024-02-28 23:16:32.542944391 +0000 UTC m=+0.636681893 container remove 9041cb9f60617bff34061b20790de63971806c9a8168bec186a5574a92d52d32 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-28T23:16:33.188 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:32 smithi084 podman[79982]: 2024-02-28 23:16:32.794295782 +0000 UTC m=+0.033830631 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:16:33.188 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:32 smithi084 podman[79982]: 2024-02-28 23:16:32.927606337 +0000 UTC m=+0.167141184 container create c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-02-28T23:16:33.580 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 podman[79982]: 2024-02-28 23:16:33.189767359 +0000 UTC m=+0.429302266 container init c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:16:33.580 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 podman[79982]: 2024-02-28 23:16:33.20044922 +0000 UTC m=+0.439984077 container start c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-28T23:16:33.580 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-osd[80000]: -- 172.21.15.84:0/1001752235 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b50fdab680 con 0x55b50fd74c00 2024-02-28T23:16:33.580 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 bash[79982]: c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 2024-02-28T23:16:33.580 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 systemd[1]: Started Ceph osd.3 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:16:33.947 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:16:33.774+0000 7fa06c91a7c0 -1 Falling back to public interface 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 4 peering, 16 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 64/321 objects degraded (19.938%) 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:33.947 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 4 peering, 16 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 64/321 objects degraded (19.938%) 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:34 smithi057 ceph-mon[89491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-28T23:16:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:34 smithi084 ceph-mon[70440]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-28T23:16:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:35 smithi057 ceph-mon[89491]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:35 smithi084 ceph-mon[70440]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:37 smithi057 ceph-mon[89491]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:37 smithi084 ceph-mon[70440]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:38.961 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:38.961 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:38.961 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:38 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 67/321 objects degraded (20.872%), 19 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:39.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:38 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 67/321 objects degraded (20.872%), 19 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:39.330 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:38 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:16:38.960+0000 7fa06c91a7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-28T23:16:39.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:39 smithi084 ceph-mon[70440]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:39.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:39 smithi084 ceph-mon[70440]: from='osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:16:39.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:39 smithi084 ceph-mon[70440]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:16:39.830 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:39 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:16:39.569+0000 7fa06c91a7c0 -1 osd.3 65 log_to_monitors true 2024-02-28T23:16:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:39 smithi057 ceph-mon[89491]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-02-28T23:16:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:39 smithi057 ceph-mon[89491]: from='osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:16:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:39 smithi057 ceph-mon[89491]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-28T23:16:41.023 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:16:40.984+0000 7fa056c00700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: from='osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: osdmap e68: 6 total, 5 up, 6 in 2024-02-28T23:16:41.024 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:40 smithi084 ceph-mon[70440]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:16:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-28T23:16:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: from='osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:16:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: osdmap e68: 6 total, 5 up, 6 in 2024-02-28T23:16:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:40 smithi057 ceph-mon[89491]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:16:42.054 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:41 smithi084 ceph-mon[70440]: pgmap v137: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 67/321 objects degraded (20.872%) 2024-02-28T23:16:42.054 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:41 smithi084 ceph-mon[70440]: from='osd.3 ' entity='osd.3' 2024-02-28T23:16:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:41 smithi057 ceph-mon[89491]: pgmap v137: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 67/321 objects degraded (20.872%) 2024-02-28T23:16:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:41 smithi057 ceph-mon[89491]: from='osd.3 ' entity='osd.3' 2024-02-28T23:16:43.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:42 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:16:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:42 smithi057 ceph-mon[89491]: osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848] boot 2024-02-28T23:16:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:42 smithi057 ceph-mon[89491]: osdmap e69: 6 total, 6 up, 6 in 2024-02-28T23:16:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:42 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:16:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:42 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:16:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:42 smithi084 ceph-mon[70440]: osd.3 [v2:172.21.15.84:6800/2190333848,v1:172.21.15.84:6801/2190333848] boot 2024-02-28T23:16:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:42 smithi084 ceph-mon[70440]: osdmap e69: 6 total, 6 up, 6 in 2024-02-28T23:16:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:42 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-28T23:16:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:43 smithi057 ceph-mon[89491]: pgmap v139: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 67/321 objects degraded (20.872%) 2024-02-28T23:16:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:43 smithi057 ceph-mon[89491]: osdmap e70: 6 total, 6 up, 6 in 2024-02-28T23:16:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:43 smithi084 ceph-mon[70440]: pgmap v139: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 67/321 objects degraded (20.872%) 2024-02-28T23:16:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:43 smithi084 ceph-mon[70440]: osdmap e70: 6 total, 6 up, 6 in 2024-02-28T23:16:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:44 smithi057 ceph-mon[89491]: pgmap v141: 65 pgs: 3 peering, 15 active+undersized+degraded, 12 active+undersized, 35 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 55/321 objects degraded (17.134%) 2024-02-28T23:16:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:44 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 55/321 objects degraded (17.134%), 15 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:44 smithi084 ceph-mon[70440]: pgmap v141: 65 pgs: 3 peering, 15 active+undersized+degraded, 12 active+undersized, 35 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 55/321 objects degraded (17.134%) 2024-02-28T23:16:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:44 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 55/321 objects degraded (17.134%), 15 pgs degraded (PG_DEGRADED) 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:16:46.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:46 smithi057 ceph-mon[89491]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:16:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:46 smithi084 ceph-mon[70440]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-28T23:16:47.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:47 smithi057 ceph-mon[89491]: pgmap v142: 65 pgs: 3 peering, 4 active+undersized+degraded, 3 active+undersized, 55 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 12/321 objects degraded (3.738%) 2024-02-28T23:16:47.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:47 smithi084 ceph-mon[70440]: pgmap v142: 65 pgs: 3 peering, 4 active+undersized+degraded, 3 active+undersized, 55 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 12/321 objects degraded (3.738%) 2024-02-28T23:16:49.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:49 smithi057 ceph-mon[89491]: pgmap v143: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 2 op/s 2024-02-28T23:16:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:49 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/321 objects degraded (3.738%), 4 pgs degraded) 2024-02-28T23:16:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:49 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:16:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:49 smithi084 ceph-mon[70440]: pgmap v143: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 2 op/s 2024-02-28T23:16:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:16:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:49 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/321 objects degraded (3.738%), 4 pgs degraded) 2024-02-28T23:16:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:49 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:16:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:51 smithi057 ceph-mon[89491]: pgmap v144: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:16:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:51 smithi084 ceph-mon[70440]: pgmap v144: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:16:53.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:53 smithi057 ceph-mon[89491]: pgmap v145: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:53.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:53 smithi084 ceph-mon[70440]: pgmap v145: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:54.563 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:54 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 -- 172.21.15.57:0/1188017997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 msgr2=0x7f01581028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 --2- 172.21.15.57:0/1188017997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581028e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0144009910 tx=0x7f0144009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 -- 172.21.15.57:0/1188017997 shutdown_connections 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 --2- 172.21.15.57:0/1188017997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581028e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 --2- 172.21.15.57:0/1188017997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 0x7f0158108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 -- 172.21.15.57:0/1188017997 >> 172.21.15.57:0/1188017997 conn(0x7f01580fdf90 msgr2=0x7f01581003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 -- 172.21.15.57:0/1188017997 shutdown_connections 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.563+0000 7f015da00700 1 -- 172.21.15.57:0/1188017997 wait complete. 2024-02-28T23:16:54.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 Processor -- start 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 -- start start 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 0x7f0158198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158198e10 con 0x7f0158102470 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.565+0000 7f015da00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015819cba0 con 0x7f0158108470 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52098/0 (socket says 172.21.15.57:52098) 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 -- 172.21.15.57:0/3141175034 learned_addr learned my addr 172.21.15.57:0/3141175034 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156000700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 0x7f0158198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 -- 172.21.15.57:0/3141175034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 msgr2=0x7f0158198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 0x7f0158198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.566+0000 7f0156a00700 1 -- 172.21.15.57:0/3141175034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440095f0 con 0x7f0158102470 2024-02-28T23:16:54.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f0156a00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f014800c8a0 tx=0x7f014800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:54.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f014800cea0 con 0x7f0158102470 2024-02-28T23:16:54.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0148004830 con 0x7f0158102470 2024-02-28T23:16:54.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015819ce80 con 0x7f0158102470 2024-02-28T23:16:54.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01480055b0 con 0x7f0158102470 2024-02-28T23:16:54.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.567+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015819d3d0 con 0x7f0158102470 2024-02-28T23:16:54.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.568+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015810a970 con 0x7f0158102470 2024-02-28T23:16:54.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.569+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f01480049a0 con 0x7f0158102470 2024-02-28T23:16:54.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.571+0000 7f014f400700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 0x7f013c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:54.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.571+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f014808d0d0 con 0x7f0158102470 2024-02-28T23:16:54.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.571+0000 7f0156000700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 0x7f013c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:54.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.572+0000 7f0156000700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 0x7f013c06fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0158199810 tx=0x7f014401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:54.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.576+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0148056910 con 0x7f0158102470 2024-02-28T23:16:54.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:54 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:16:54.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.829+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0158199690 con 0x7f013c06d8a0 2024-02-28T23:16:54.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.832+0000 7f014f400700 1 -- 172.21.15.57:0/3141175034 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f01440056c0 con 0x7f013c06d8a0 2024-02-28T23:16:54.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.834+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 msgr2=0x7f013c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.834+0000 7f015da00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 0x7f013c06fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0158199810 tx=0x7f014401a040 comp rx=0 tx=0).stop 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.834+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 msgr2=0x7f01581981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.834+0000 7f015da00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f014800c8a0 tx=0x7f014800cc60 comp rx=0 tx=0).stop 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 shutdown_connections 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f013c06d8a0 0x7f013c06fd50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0158102470 0x7f01581981f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 --2- 172.21.15.57:0/3141175034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0158108470 0x7f0158198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 >> 172.21.15.57:0/3141175034 conn(0x7f01580fdf90 msgr2=0x7f01580ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 shutdown_connections 2024-02-28T23:16:54.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.835+0000 7f015da00700 1 -- 172.21.15.57:0/3141175034 wait complete. 2024-02-28T23:16:54.851 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:16:54.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 -- 172.21.15.57:0/1843544367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454108470 msgr2=0x7f7454108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 --2- 172.21.15.57:0/1843544367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454108470 0x7f7454108840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7444009910 tx=0x7f7444009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:54.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 -- 172.21.15.57:0/1843544367 shutdown_connections 2024-02-28T23:16:54.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 --2- 172.21.15.57:0/1843544367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454102470 0x7f74541028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 --2- 172.21.15.57:0/1843544367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454108470 0x7f7454108840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.994+0000 7f7459a00700 1 -- 172.21.15.57:0/1843544367 >> 172.21.15.57:0/1843544367 conn(0x7f74540fdf90 msgr2=0x7f74541003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:54.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.995+0000 7f7459a00700 1 -- 172.21.15.57:0/1843544367 shutdown_connections 2024-02-28T23:16:54.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.995+0000 7f7459a00700 1 -- 172.21.15.57:0/1843544367 wait complete. 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 Processor -- start 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 -- start start 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 0x7f7454198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454198e10 con 0x7f7454102470 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.996+0000 7f7459a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745419cba0 con 0x7f7454108470 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41134/0 (socket says 172.21.15.57:41134) 2024-02-28T23:16:54.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 -- 172.21.15.57:0/3451920368 learned_addr learned my addr 172.21.15.57:0/3451920368 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f744be00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 0x7f7454198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 -- 172.21.15.57:0/3451920368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 msgr2=0x7f7454198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 0x7f7454198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 -- 172.21.15.57:0/3451920368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440095f0 con 0x7f7454102470 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f744be00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 0x7f7454198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.997+0000 7f7452a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f744400b5c0 tx=0x7f744400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.998+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744401d070 con 0x7f7454102470 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.998+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7444005e00 con 0x7f7454102470 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.998+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7444017400 con 0x7f7454102470 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.998+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745419ce20 con 0x7f7454102470 2024-02-28T23:16:54.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.998+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745419d310 con 0x7f7454102470 2024-02-28T23:16:54.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:54.999+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745410a9e0 con 0x7f7454102470 2024-02-28T23:16:54.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.000+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f744400f460 con 0x7f7454102470 2024-02-28T23:16:55.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.001+0000 7f744b400700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 0x7f743806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.001+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f744408e960 con 0x7f7454102470 2024-02-28T23:16:55.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.001+0000 7f744be00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 0x7f743806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.002+0000 7f744be00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 0x7f743806fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f743c005fd0 tx=0x7f743c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:55.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.006+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74440581a0 con 0x7f7454102470 2024-02-28T23:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.262+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7454199660 con 0x7f743806d8a0 2024-02-28T23:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.263+0000 7f744b400700 1 -- 172.21.15.57:0/3451920368 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7454199660 con 0x7f743806d8a0 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.266+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 msgr2=0x7f743806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.266+0000 7f7459a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 0x7f743806fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f743c005fd0 tx=0x7f743c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.266+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 msgr2=0x7f74541981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.266+0000 7f7459a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f744400b5c0 tx=0x7f744400be20 comp rx=0 tx=0).stop 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 shutdown_connections 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f743806d8a0 0x7f743806fd50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7454102470 0x7f74541981f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 --2- 172.21.15.57:0/3451920368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7454108470 0x7f7454198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 >> 172.21.15.57:0/3451920368 conn(0x7f74540fdf90 msgr2=0x7f74541002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 shutdown_connections 2024-02-28T23:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.267+0000 7f7459a00700 1 -- 172.21.15.57:0/3451920368 wait complete. 2024-02-28T23:16:55.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.426+0000 7f29b4400700 1 -- 172.21.15.57:0/2025361638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 msgr2=0x7f29ac100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.426+0000 7f29b4400700 1 --2- 172.21.15.57:0/2025361638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac100040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2998009910 tx=0x7f2998009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.426+0000 7f29b4400700 1 -- 172.21.15.57:0/2025361638 shutdown_connections 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.427+0000 7f29b4400700 1 --2- 172.21.15.57:0/2025361638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac100040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.427+0000 7f29b4400700 1 --2- 172.21.15.57:0/2025361638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.427+0000 7f29b4400700 1 -- 172.21.15.57:0/2025361638 >> 172.21.15.57:0/2025361638 conn(0x7f29ac075130 msgr2=0x7f29ac075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.427+0000 7f29b4400700 1 -- 172.21.15.57:0/2025361638 shutdown_connections 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.427+0000 7f29b4400700 1 -- 172.21.15.57:0/2025361638 wait complete. 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.428+0000 7f29b4400700 1 Processor -- start 2024-02-28T23:16:55.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.428+0000 7f29b4400700 1 -- start start 2024-02-28T23:16:55.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b4400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b4400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b4400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac196b60 con 0x7f29ac104130 2024-02-28T23:16:55.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b4400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19a8f0 con 0x7f29ac0ffbd0 2024-02-28T23:16:55.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41154/0 (socket says 172.21.15.57:41154) 2024-02-28T23:16:55.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.429+0000 7f29b1000700 1 -- 172.21.15.57:0/3288312030 learned_addr learned my addr 172.21.15.57:0/3288312030 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b1a00700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b1000700 1 -- 172.21.15.57:0/3288312030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac0ffbd0 msgr2=0x7f29ac195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b1000700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b1000700 1 -- 172.21.15.57:0/3288312030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980095f0 con 0x7f29ac104130 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b1000700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29980092e0 tx=0x7f299800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:55.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299801d070 con 0x7f29ac104130 2024-02-28T23:16:55.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f299800f460 con 0x7f29ac104130 2024-02-28T23:16:55.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.430+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac19ab70 con 0x7f29ac104130 2024-02-28T23:16:55.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.431+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2998017630 con 0x7f29ac104130 2024-02-28T23:16:55.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.431+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac19b060 con 0x7f29ac104130 2024-02-28T23:16:55.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.432+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac04f2a0 con 0x7f29ac104130 2024-02-28T23:16:55.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.433+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f299800f5d0 con 0x7f29ac104130 2024-02-28T23:16:55.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.433+0000 7f29aa000700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 0x7f299406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.434+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f299808eb00 con 0x7f29ac104130 2024-02-28T23:16:55.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.434+0000 7f29b1a00700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 0x7f299406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.435+0000 7f29b1a00700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 0x7f299406fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006fd0 tx=0x7f29a0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:55.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.438+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29980582c0 con 0x7f29ac104130 2024-02-28T23:16:55.475 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:55 smithi057 ceph-mon[89491]: pgmap v146: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:16:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:55 smithi084 ceph-mon[70440]: pgmap v146: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:16:55.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.685+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29ac19b360 con 0x7f299406d8a0 2024-02-28T23:16:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.702+0000 7f29aa000700 1 -- 172.21.15.57:0/3288312030 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f29ac19b360 con 0x7f299406d8a0 2024-02-28T23:16:55.701 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (6m) 46s ago 20m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (16m) 46s ago 20m 9000k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (19m) 18s ago 19m 8510k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (2m) 46s ago 20m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (2m) 18s ago 19m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (5m) 46s ago 20m 75.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (13m) 46s ago 13m 56.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (13m) 46s ago 13m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (12m) 18s ago 12m 58.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (13m) 18s ago 13m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (8m) 46s ago 22m 500M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (7m) 18s ago 18m 451M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:16:55.702 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (4m) 46s ago 22m 48.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (3m) 18s ago 18m 38.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (7m) 46s ago 20m 7982k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (7m) 18s ago 19m 7944k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (118s) 46s ago 17m 87.7M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (86s) 46s ago 17m 99.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (55s) 46s ago 16m 11.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (22s) 18s ago 16m 11.1M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (15m) 18s ago 15m 238M 2241M 18.2.0 dc2bc1663786 16295d100027 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (14m) 18s ago 14m 232M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:16:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (6m) 46s ago 19m 63.0M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:16:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.704+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 msgr2=0x7f299406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.704+0000 7f29b4400700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 0x7f299406fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006fd0 tx=0x7f29a0008040 comp rx=0 tx=0).stop 2024-02-28T23:16:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.704+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 msgr2=0x7f29ac196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.704+0000 7f29b4400700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29980092e0 tx=0x7f299800be00 comp rx=0 tx=0).stop 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 shutdown_connections 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299406d8a0 0x7f299406fd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 --2- 172.21.15.57:0/3288312030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac104130 0x7f29ac196480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 >> 172.21.15.57:0/3288312030 conn(0x7f29ac075130 msgr2=0x7f29ac0685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 shutdown_connections 2024-02-28T23:16:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.705+0000 7f29b4400700 1 -- 172.21.15.57:0/3288312030 wait complete. 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 -- 172.21.15.57:0/1211246315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 msgr2=0x7f6c401008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 --2- 172.21.15.57:0/1211246315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c401008b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009910 tx=0x7f6c30009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 -- 172.21.15.57:0/1211246315 shutdown_connections 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 --2- 172.21.15.57:0/1211246315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 0x7f6c40104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 --2- 172.21.15.57:0/1211246315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c401008b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 -- 172.21.15.57:0/1211246315 >> 172.21.15.57:0/1211246315 conn(0x7f6c400fbda0 msgr2=0x7f6c400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 -- 172.21.15.57:0/1211246315 shutdown_connections 2024-02-28T23:16:55.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.859+0000 7f6c45800700 1 -- 172.21.15.57:0/1211246315 wait complete. 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.860+0000 7f6c45800700 1 Processor -- start 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c45800700 1 -- start start 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c45800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c45800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 0x7f6c401964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c45800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40196bb0 con 0x7f6c401004e0 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c45800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4019a940 con 0x7f6c40100e80 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c3ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c3ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41176/0 (socket says 172.21.15.57:41176) 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.861+0000 7f6c3ea00700 1 -- 172.21.15.57:0/3798988513 learned_addr learned my addr 172.21.15.57:0/3798988513 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:55.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c3ea00700 1 -- 172.21.15.57:0/3798988513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 msgr2=0x7f6c401964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:55.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c3e000700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 0x7f6c401964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c3ea00700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 0x7f6c401964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:55.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c3ea00700 1 -- 172.21.15.57:0/3798988513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300095f0 con 0x7f6c401004e0 2024-02-28T23:16:55.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c3ea00700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b5c0 tx=0x7f6c3000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3001d070 con 0x7f6c401004e0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3000f460 con 0x7f6c401004e0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4019abc0 con 0x7f6c401004e0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c30017610 con 0x7f6c401004e0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.862+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4019b0b0 con 0x7f6c401004e0 2024-02-28T23:16:55.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.864+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c401087f0 con 0x7f6c401004e0 2024-02-28T23:16:55.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.865+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6c3000f5d0 con 0x7f6c401004e0 2024-02-28T23:16:55.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.866+0000 7f6c37400700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 0x7f6c2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:55.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.866+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c3008eb30 con 0x7f6c401004e0 2024-02-28T23:16:55.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.866+0000 7f6c3e000700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 0x7f6c2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:55.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.867+0000 7f6c3e000700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 0x7f6c2406fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:55.871+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c300582f0 con 0x7f6c401004e0 2024-02-28T23:16:56.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.176+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c4002cc30 con 0x7f6c401004e0 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.178+0000 7f6c37400700 1 -- 172.21.15.57:0/3798988513 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f6c3005b9b0 con 0x7f6c401004e0 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:56.177 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 8 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:16:56.178 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:16:56.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.180+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 msgr2=0x7f6c2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.180+0000 7f6c45800700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 0x7f6c2406fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2024-02-28T23:16:56.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.180+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 msgr2=0x7f6c40195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.180+0000 7f6c45800700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b5c0 tx=0x7f6c3000be00 comp rx=0 tx=0).stop 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 shutdown_connections 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c2406d8a0 0x7f6c2406fd50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c401004e0 0x7f6c40195f90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 --2- 172.21.15.57:0/3798988513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c40100e80 0x7f6c401964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 >> 172.21.15.57:0/3798988513 conn(0x7f6c400fbda0 msgr2=0x7f6c400fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 shutdown_connections 2024-02-28T23:16:56.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.181+0000 7f6c45800700 1 -- 172.21.15.57:0/3798988513 wait complete. 2024-02-28T23:16:56.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.341+0000 7f4ce2a00700 1 -- 172.21.15.57:0/924143737 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4007c80 con 0x7f4ce41004e0 2024-02-28T23:16:56.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 -- 172.21.15.57:0/924143737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce41004e0 msgr2=0x7f4ce41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 --2- 172.21.15.57:0/924143737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce41004e0 0x7f4ce41008b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400b0a0 tx=0x7f4cd400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 -- 172.21.15.57:0/924143737 shutdown_connections 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 --2- 172.21.15.57:0/924143737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce4100e80 0x7f4ce4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 --2- 172.21.15.57:0/924143737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce41004e0 0x7f4ce41008b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 -- 172.21.15.57:0/924143737 >> 172.21.15.57:0/924143737 conn(0x7f4ce40fbda0 msgr2=0x7f4ce40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 -- 172.21.15.57:0/924143737 shutdown_connections 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.342+0000 7f4cea800700 1 -- 172.21.15.57:0/924143737 wait complete. 2024-02-28T23:16:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.343+0000 7f4cea800700 1 Processor -- start 2024-02-28T23:16:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.343+0000 7f4cea800700 1 -- start start 2024-02-28T23:16:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cea800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce41004e0 0x7f4ce41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cea800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cea800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce4198d10 con 0x7f4ce4100e80 2024-02-28T23:16:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cea800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce419caa0 con 0x7f4ce41004e0 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cdaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cdaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41192/0 (socket says 172.21.15.57:41192) 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.344+0000 7f4cdaa00700 1 -- 172.21.15.57:0/3018451268 learned_addr learned my addr 172.21.15.57:0/3018451268 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4cdaa00700 1 -- 172.21.15.57:0/3018451268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce41004e0 msgr2=0x7f4ce41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4cdaa00700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce41004e0 0x7f4ce41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4cdaa00700 1 -- 172.21.15.57:0/3018451268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd4009d00 con 0x7f4ce4100e80 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4cdaa00700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00d8d0 tx=0x7f4ccc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ccc0041f0 con 0x7f4ce4100e80 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ccc007620 con 0x7f4ce4100e80 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ccc003c50 con 0x7f4ce4100e80 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.345+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce419cd80 con 0x7f4ce4100e80 2024-02-28T23:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.346+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce419d2d0 con 0x7f4ce4100e80 2024-02-28T23:16:56.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.347+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce41045d0 con 0x7f4ce4100e80 2024-02-28T23:16:56.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.348+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4ccc004350 con 0x7f4ce4100e80 2024-02-28T23:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.349+0000 7f4ce0c00700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 0x7f4cc806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.349+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4ccc08cf80 con 0x7f4ce4100e80 2024-02-28T23:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.349+0000 7f4ce3400700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 0x7f4cc806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.350+0000 7f4ce3400700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 0x7f4cc806fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009480 tx=0x7f4cd4006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:56.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.354+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ccc056860 con 0x7f4ce4100e80 2024-02-28T23:16:56.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:56 smithi057 ceph-mon[89491]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:56 smithi057 ceph-mon[89491]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:56 smithi057 ceph-mon[89491]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:56 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3798988513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:56 smithi084 ceph-mon[70440]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:56 smithi084 ceph-mon[70440]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:56 smithi084 ceph-mon[70440]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:56 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3798988513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:16:56.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.631+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ce419d540 con 0x7f4ce4100e80 2024-02-28T23:16:56.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.632+0000 7f4ce0c00700 1 -- 172.21.15.57:0/3018451268 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f4ccc01f760 con 0x7f4ce4100e80 2024-02-28T23:16:56.632 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:16:56.632 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:16:56.632 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:16:56.632 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:16:56.633 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:16:56.634 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:56.635 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.635+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 msgr2=0x7f4cc806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.635+0000 7f4cea800700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 0x7f4cc806fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009480 tx=0x7f4cd4006000 comp rx=0 tx=0).stop 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.635+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 msgr2=0x7f4ce4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.635+0000 7f4cea800700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00d8d0 tx=0x7f4ccc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 shutdown_connections 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cc806d8a0 0x7f4cc806fd50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce41004e0 0x7f4ce41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 --2- 172.21.15.57:0/3018451268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ce4100e80 0x7f4ce4198630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 >> 172.21.15.57:0/3018451268 conn(0x7f4ce40fbda0 msgr2=0x7f4ce40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 shutdown_connections 2024-02-28T23:16:56.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.636+0000 7f4cea800700 1 -- 172.21.15.57:0/3018451268 wait complete. 2024-02-28T23:16:56.637 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 -- 172.21.15.57:0/1620908090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 msgr2=0x7f3d38106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 --2- 172.21.15.57:0/1620908090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d38106ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009910 tx=0x7f3d24009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 -- 172.21.15.57:0/1620908090 shutdown_connections 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 --2- 172.21.15.57:0/1620908090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d38106ff0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 --2- 172.21.15.57:0/1620908090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 0x7f3d3810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 -- 172.21.15.57:0/1620908090 >> 172.21.15.57:0/1620908090 conn(0x7f3d38076060 msgr2=0x7f3d38078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 -- 172.21.15.57:0/1620908090 shutdown_connections 2024-02-28T23:16:56.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.787+0000 7f3d3de00700 1 -- 172.21.15.57:0/1620908090 wait complete. 2024-02-28T23:16:56.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.788+0000 7f3d3de00700 1 Processor -- start 2024-02-28T23:16:56.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.788+0000 7f3d3de00700 1 -- start start 2024-02-28T23:16:56.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d3de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d3de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 0x7f3d3819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d3de00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819d100 con 0x7f3d38106b80 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d3de00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d381a0e90 con 0x7f3d3810c8f0 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d36a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d36a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41206/0 (socket says 172.21.15.57:41206) 2024-02-28T23:16:56.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.789+0000 7f3d36a00700 1 -- 172.21.15.57:0/652295282 learned_addr learned my addr 172.21.15.57:0/652295282 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d36000700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 0x7f3d3819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d36a00700 1 -- 172.21.15.57:0/652295282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 msgr2=0x7f3d3819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d36a00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 0x7f3d3819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d36a00700 1 -- 172.21.15.57:0/652295282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240095f0 con 0x7f3d38106b80 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d36a00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b4b0 tx=0x7f3d2800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d28011800 con 0x7f3d38106b80 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d28011e20 con 0x7f3d38106b80 2024-02-28T23:16:56.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2800f2e0 con 0x7f3d38106b80 2024-02-28T23:16:56.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d381a1170 con 0x7f3d38106b80 2024-02-28T23:16:56.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.790+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381a16c0 con 0x7f3d38106b80 2024-02-28T23:16:56.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.792+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d38109b20 con 0x7f3d38106b80 2024-02-28T23:16:56.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.793+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3d28010380 con 0x7f3d38106b80 2024-02-28T23:16:56.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.794+0000 7f3d3ca00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 0x7f3d1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:56.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.794+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3d2808cff0 con 0x7f3d38106b80 2024-02-28T23:16:56.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.794+0000 7f3d36000700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 0x7f3d1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:56.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.795+0000 7f3d36000700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 0x7f3d1c06fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b5c0 tx=0x7f3d24005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:56.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:56.798+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3d280c0d20 con 0x7f3d38106b80 2024-02-28T23:16:57.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.045+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d381a19a0 con 0x7f3d1c06d8a0 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.047+0000 7f3d3ca00700 1 -- 172.21.15.57:0/652295282 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3d381a19a0 con 0x7f3d1c06d8a0 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:16:57.047 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:16:57.048 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:16:57.048 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "10/23 daemons upgraded", 2024-02-28T23:16:57.048 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:16:57.048 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:16:57.048 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:16:57.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 msgr2=0x7f3d1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 0x7f3d1c06fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b5c0 tx=0x7f3d24005960 comp rx=0 tx=0).stop 2024-02-28T23:16:57.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 msgr2=0x7f3d3819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b4b0 tx=0x7f3d2800b870 comp rx=0 tx=0).stop 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 shutdown_connections 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3d1c06d8a0 0x7f3d1c06fd50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d38106b80 0x7f3d3819c4e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 --2- 172.21.15.57:0/652295282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d3810c8f0 0x7f3d3819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 >> 172.21.15.57:0/652295282 conn(0x7f3d38076060 msgr2=0x7f3d38077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.050+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 shutdown_connections 2024-02-28T23:16:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.051+0000 7f3d3de00700 1 -- 172.21.15.57:0/652295282 wait complete. 2024-02-28T23:16:57.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.216+0000 7ffa21600700 1 -- 172.21.15.57:0/772056115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 msgr2=0x7ffa1c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.216+0000 7ffa21600700 1 --2- 172.21.15.57:0/772056115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c100040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009910 tx=0x7ffa04009c20 comp rx=0 tx=0).stop 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 -- 172.21.15.57:0/772056115 shutdown_connections 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 --2- 172.21.15.57:0/772056115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c100040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 --2- 172.21.15.57:0/772056115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 0x7ffa1c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 -- 172.21.15.57:0/772056115 >> 172.21.15.57:0/772056115 conn(0x7ffa1c075130 msgr2=0x7ffa1c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 -- 172.21.15.57:0/772056115 shutdown_connections 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.217+0000 7ffa21600700 1 -- 172.21.15.57:0/772056115 wait complete. 2024-02-28T23:16:57.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.218+0000 7ffa21600700 1 Processor -- start 2024-02-28T23:16:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.218+0000 7ffa21600700 1 -- start start 2024-02-28T23:16:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa21600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa21600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 0x7ffa1c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa21600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c196a70 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa21600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19a800 con 0x7ffa1c104130 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa1aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa1aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41230/0 (socket says 172.21.15.57:41230) 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa1aa00700 1 -- 172.21.15.57:0/1657031249 learned_addr learned my addr 172.21.15.57:0/1657031249 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.219+0000 7ffa13e00700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 0x7ffa1c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.220+0000 7ffa1aa00700 1 -- 172.21.15.57:0/1657031249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 msgr2=0x7ffa1c196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.220+0000 7ffa1aa00700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 0x7ffa1c196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.220+0000 7ffa1aa00700 1 -- 172.21.15.57:0/1657031249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040095f0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.220+0000 7ffa1aa00700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c8a0 tx=0x7ffa0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:57.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.221+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0c00cea0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.221+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa0c004830 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.221+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0c0055b0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.221+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c19aae0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.221+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19b030 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.222+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c1086a0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.223+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ffa0c0049a0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.224+0000 7ffa13400700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 0x7ffa0006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:16:57.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.224+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffa0c08d0e0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.225+0000 7ffa13e00700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 0x7ffa0006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:16:57.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.225+0000 7ffa13e00700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 0x7ffa0006fd50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005100 tx=0x7ffa0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:16:57.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.229+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa0c056920 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:57 smithi057 ceph-mon[89491]: pgmap v147: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-02-28T23:16:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:57 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3018451268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:16:57.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.545+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffa1c04f2a0 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.546+0000 7ffa13400700 1 -- 172.21.15.57:0/1657031249 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ffa0c01d090 con 0x7ffa1c0ffbd0 2024-02-28T23:16:57.546 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:16:57.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 msgr2=0x7ffa0006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 0x7ffa0006fd50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005100 tx=0x7ffa0401a040 comp rx=0 tx=0).stop 2024-02-28T23:16:57.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 msgr2=0x7ffa1c195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:16:57.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c8a0 tx=0x7ffa0c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 shutdown_connections 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa0006d8a0 0x7ffa0006fd50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa1c0ffbd0 0x7ffa1c195e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 --2- 172.21.15.57:0/1657031249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa1c104130 0x7ffa1c196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 >> 172.21.15.57:0/1657031249 conn(0x7ffa1c075130 msgr2=0x7ffa1c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 shutdown_connections 2024-02-28T23:16:57.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:16:57.550+0000 7ffa21600700 1 -- 172.21.15.57:0/1657031249 wait complete. 2024-02-28T23:16:57.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:57 smithi084 ceph-mon[70440]: pgmap v147: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-02-28T23:16:57.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:57 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3018451268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:16:58.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:58 smithi057 ceph-mon[89491]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:58.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:58 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1657031249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:16:58.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:58 smithi084 ceph-mon[70440]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:16:58.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:58 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1657031249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:16:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:16:59 smithi057 ceph-mon[89491]: pgmap v148: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:16:59.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:16:59 smithi084 ceph-mon[70440]: pgmap v148: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:01.512 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:01 smithi084 ceph-mon[70440]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:17:01.512 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:01 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:17:01.513 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:01 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:17:01.513 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:01 smithi084 ceph-mon[70440]: Upgrade: osd.4 is safe to restart 2024-02-28T23:17:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:01 smithi057 ceph-mon[89491]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:17:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:01 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:17:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:01 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-28T23:17:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:01 smithi057 ceph-mon[89491]: Upgrade: osd.4 is safe to restart 2024-02-28T23:17:03.922 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: Upgrade: Updating osd.4 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: Deploying daemon osd.4 on smithi084 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:03.923 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: Upgrade: Updating osd.4 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: Deploying daemon osd.4 on smithi084 2024-02-28T23:17:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:05 smithi057 ceph-mon[89491]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:17:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:05 smithi084 ceph-mon[70440]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:17:07.747 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:07 smithi084 systemd[1]: Stopping Ceph osd.4 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:17:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:07 smithi057 ceph-mon[89491]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:17:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:07 smithi084 ceph-mon[70440]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:17:08.080 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:07 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[45444]: 2024-02-28T23:17:07.886+0000 7f9e6ee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:17:08.080 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:07 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[45444]: 2024-02-28T23:17:07.886+0000 7f9e6ee00700 -1 osd.4 70 *** Got signal Terminated *** 2024-02-28T23:17:08.080 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:07 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[45444]: 2024-02-28T23:17:07.886+0000 7f9e6ee00700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:17:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:08 smithi057 ceph-mon[89491]: osd.4 marked itself down and dead 2024-02-28T23:17:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:08 smithi084 ceph-mon[70440]: osd.4 marked itself down and dead 2024-02-28T23:17:09.709 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:09 smithi084 podman[84913]: 2024-02-28 23:17:09.440849862 +0000 UTC m=+1.696909205 container died 16295d1000278fe1dc38b071d9493a625034f4ddaca21ef2bcc80ba51f56e0c7 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-28T23:17:10.038 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:09 smithi084 ceph-mon[70440]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:17:10.038 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:09 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:17:10.038 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:09 smithi084 ceph-mon[70440]: osdmap e71: 6 total, 5 up, 6 in 2024-02-28T23:17:10.038 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:09 smithi084 podman[84913]: 2024-02-28 23:17:09.838139297 +0000 UTC m=+2.094198577 container cleanup 16295d1000278fe1dc38b071d9493a625034f4ddaca21ef2bcc80ba51f56e0c7 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-28T23:17:10.038 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:09 smithi084 bash[84913]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4 2024-02-28T23:17:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:09 smithi057 ceph-mon[89491]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:17:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:09 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:17:10.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:09 smithi057 ceph-mon[89491]: osdmap e71: 6 total, 5 up, 6 in 2024-02-28T23:17:10.329 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:10 smithi084 podman[84931]: 2024-02-28 23:17:10.039295482 +0000 UTC m=+0.584587940 container remove 16295d1000278fe1dc38b071d9493a625034f4ddaca21ef2bcc80ba51f56e0c7 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-28T23:17:11.039 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:10 smithi084 ceph-mon[70440]: osdmap e72: 6 total, 5 up, 6 in 2024-02-28T23:17:11.039 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:10 smithi084 podman[85019]: 2024-02-28 23:17:10.767699277 +0000 UTC m=+0.040098962 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:11.039 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:10 smithi084 podman[85019]: 2024-02-28 23:17:10.882057048 +0000 UTC m=+0.154456729 container create 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-28T23:17:11.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:10 smithi057 ceph-mon[89491]: osdmap e72: 6 total, 5 up, 6 in 2024-02-28T23:17:11.330 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:11 smithi084 podman[85019]: 2024-02-28 23:17:11.152265681 +0000 UTC m=+0.424665387 container init 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10) 2024-02-28T23:17:11.330 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:11 smithi084 podman[85019]: 2024-02-28 23:17:11.163010149 +0000 UTC m=+0.435409836 container start 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:17:11.330 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:11 smithi084 podman[85019]: 2024-02-28 23:17:11.234404337 +0000 UTC m=+0.506804025 container attach 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-02-28T23:17:11.940 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:11 smithi084 ceph-mon[70440]: pgmap v156: 65 pgs: 6 active+undersized+degraded, 13 peering, 2 stale+active+clean, 5 active+undersized, 39 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/321 objects degraded (7.477%) 2024-02-28T23:17:11.940 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:11 smithi084 ceph-mon[70440]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-28T23:17:11.941 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:11 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 24/321 objects degraded (7.477%), 6 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:11.941 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:11 smithi084 podman[85019]: 2024-02-28 23:17:11.597822654 +0000 UTC m=+0.870222306 container died 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:17:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:11 smithi057 ceph-mon[89491]: pgmap v156: 65 pgs: 6 active+undersized+degraded, 13 peering, 2 stale+active+clean, 5 active+undersized, 39 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/321 objects degraded (7.477%) 2024-02-28T23:17:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:11 smithi057 ceph-mon[89491]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-28T23:17:12.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:11 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 24/321 objects degraded (7.477%), 6 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:12.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:12 smithi084 podman[85045]: 2024-02-28 23:17:12.254613465 +0000 UTC m=+0.638284746 container remove 6504e0f3ebc34161a0113f62bff89dfa64a7e1f7606bb970ececaeeef2f3dd38 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:17:12.581 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:12 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-28T23:17:12.581 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:12 smithi084 systemd[1]: Stopped Ceph osd.4 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:17:12.581 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:12 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.4.service: Consumed 1min 47.853s CPU time. 2024-02-28T23:17:12.839 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:12 smithi084 systemd[1]: Starting Ceph osd.4 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:17:13.213 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:12 smithi084 ceph-mon[70440]: pgmap v157: 65 pgs: 10 active+undersized+degraded, 13 peering, 1 stale+active+clean, 9 active+undersized, 32 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/321 objects degraded (10.592%) 2024-02-28T23:17:13.213 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:13 smithi084 podman[85146]: 2024-02-28 23:17:12.944536748 +0000 UTC m=+0.035271583 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:13.213 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:13 smithi084 podman[85146]: 2024-02-28 23:17:13.064021535 +0000 UTC m=+0.154756383 container create 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-28T23:17:13.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:12 smithi057 ceph-mon[89491]: pgmap v157: 65 pgs: 10 active+undersized+degraded, 13 peering, 1 stale+active+clean, 9 active+undersized, 32 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/321 objects degraded (10.592%) 2024-02-28T23:17:13.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:13 smithi084 podman[85146]: 2024-02-28 23:17:13.309422652 +0000 UTC m=+0.400157541 container init 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-28T23:17:13.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:13 smithi084 podman[85146]: 2024-02-28 23:17:13.321047621 +0000 UTC m=+0.411782433 container start 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:17:13.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:13 smithi084 podman[85146]: 2024-02-28 23:17:13.399712062 +0000 UTC m=+0.490446957 container attach 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, org.label-schema.build-date=20240222) 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-28T23:17:14.574 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 bash[85146]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate[85161]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-28T23:17:14.575 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 podman[85146]: 2024-02-28 23:17:14.24042661 +0000 UTC m=+1.331161492 container died 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-28T23:17:15.139 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:14 smithi084 podman[85281]: 2024-02-28 23:17:14.936762576 +0000 UTC m=+0.678934187 container remove 3b1ff3ea1a5caf17e619586bc6744a4a0f5496c8d6772679d249f3db31dd42e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-02-28T23:17:15.467 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:15 smithi084 ceph-mon[70440]: pgmap v158: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:17:15.467 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 podman[85341]: 2024-02-28 23:17:15.212873261 +0000 UTC m=+0.035226964 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:15.468 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 podman[85341]: 2024-02-28 23:17:15.312445334 +0000 UTC m=+0.134799062 container create f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-28T23:17:15.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:15 smithi057 ceph-mon[89491]: pgmap v158: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 39/321 objects degraded (12.150%) 2024-02-28T23:17:15.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 podman[85341]: 2024-02-28 23:17:15.607936933 +0000 UTC m=+0.430290705 container init f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-28T23:17:15.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 podman[85341]: 2024-02-28 23:17:15.618033976 +0000 UTC m=+0.440387711 container start f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-28T23:17:15.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 ceph-osd[85359]: -- 172.21.15.84:0/2834225792 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602d4a37680 con 0x5602d4a01000 2024-02-28T23:17:15.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 bash[85341]: f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 2024-02-28T23:17:15.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:15 smithi084 systemd[1]: Started Ceph osd.4 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:17:16.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:16 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:17:16.185+0000 7f7216bad7c0 -1 Falling back to public interface 2024-02-28T23:17:17.144 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:17.144 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:17.144 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:17.144 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:16 smithi084 ceph-mon[70440]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-02-28T23:17:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:16 smithi057 ceph-mon[89491]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-02-28T23:17:17.992 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:17 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 52/321 objects degraded (16.199%), 16 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:17.992 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:17 smithi084 ceph-mon[70440]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-28T23:17:18.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:17 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 52/321 objects degraded (16.199%), 16 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:18.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:17 smithi057 ceph-mon[89491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-28T23:17:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:19 smithi084 ceph-mon[70440]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-02-28T23:17:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:19.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:19 smithi057 ceph-mon[89491]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-02-28T23:17:19.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:21.381 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:21 smithi057 ceph-mon[89491]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:21.381 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:21.381 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:21.580 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:21 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:17:21.314+0000 7f7216bad7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-28T23:17:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:21 smithi084 ceph-mon[70440]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:22.330 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:21 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:17:21.900+0000 7f7216bad7c0 -1 osd.4 70 log_to_monitors true 2024-02-28T23:17:22.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:22 smithi084 ceph-mon[70440]: from='osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:17:22.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:22 smithi084 ceph-mon[70440]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:17:22.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:22.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:22.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:22 smithi057 ceph-mon[89491]: from='osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:17:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:22 smithi057 ceph-mon[89491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-28T23:17:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:23.386 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:17:23.162+0000 7f7201000700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:17:23.386 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-mon[70440]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:23.386 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-mon[70440]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-28T23:17:23.386 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-mon[70440]: osdmap e73: 6 total, 5 up, 6 in 2024-02-28T23:17:23.386 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-mon[70440]: from='osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:17:23.386 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:23 smithi084 ceph-mon[70440]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:17:23.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:23 smithi057 ceph-mon[89491]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:23 smithi057 ceph-mon[89491]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-28T23:17:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:23 smithi057 ceph-mon[89491]: osdmap e73: 6 total, 5 up, 6 in 2024-02-28T23:17:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:23 smithi057 ceph-mon[89491]: from='osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:17:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:23 smithi057 ceph-mon[89491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:17:24.559 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:24 smithi084 ceph-mon[70440]: from='osd.4 ' entity='osd.4' 2024-02-28T23:17:24.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:24 smithi057 ceph-mon[89491]: from='osd.4 ' entity='osd.4' 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: pgmap v164: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486] boot 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: osdmap e74: 6 total, 6 up, 6 in 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:17:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:25 smithi057 ceph-mon[89491]: osdmap e75: 6 total, 6 up, 6 in 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: pgmap v164: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: osd.4 [v2:172.21.15.84:6808/890451486,v1:172.21.15.84:6809/890451486] boot 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: osdmap e74: 6 total, 6 up, 6 in 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-28T23:17:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:25 smithi084 ceph-mon[70440]: osdmap e75: 6 total, 6 up, 6 in 2024-02-28T23:17:26.565 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:26 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 30/321 objects degraded (9.346%), 11 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:26.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:26 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 30/321 objects degraded (9.346%), 11 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:27.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:27 smithi084 ceph-mon[70440]: pgmap v167: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 30/321 objects degraded (9.346%) 2024-02-28T23:17:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:27 smithi057 ceph-mon[89491]: pgmap v167: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 30/321 objects degraded (9.346%) 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 -- 172.21.15.57:0/3513489464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 msgr2=0x7f1afc1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 --2- 172.21.15.57:0/3513489464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc1028e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009910 tx=0x7f1ae8009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 -- 172.21.15.57:0/3513489464 shutdown_connections 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 --2- 172.21.15.57:0/3513489464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc1028e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 --2- 172.21.15.57:0/3513489464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 0x7f1afc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 -- 172.21.15.57:0/3513489464 >> 172.21.15.57:0/3513489464 conn(0x7f1afc0fdf90 msgr2=0x7f1afc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 -- 172.21.15.57:0/3513489464 shutdown_connections 2024-02-28T23:17:27.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.730+0000 7f1b03800700 1 -- 172.21.15.57:0/3513489464 wait complete. 2024-02-28T23:17:27.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.731+0000 7f1b03800700 1 Processor -- start 2024-02-28T23:17:27.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b03800700 1 -- start start 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b03800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b03800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 0x7f1afc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b03800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc0793f0 con 0x7f1afc102470 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b03800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc075ce0 con 0x7f1afc108470 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.732+0000 7f1b00e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37026/0 (socket says 172.21.15.57:37026) 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 -- 172.21.15.57:0/349859568 learned_addr learned my addr 172.21.15.57:0/349859568 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1afbe00700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 0x7f1afc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:27.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 -- 172.21.15.57:0/349859568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 msgr2=0x7f1afc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:27.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 0x7f1afc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 -- 172.21.15.57:0/349859568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80095f0 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.733+0000 7f1b00e00700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d900 tx=0x7f1af000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.734+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1af00041f0 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.734+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1af0004cf0 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.734+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc075fc0 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.734+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1af000b750 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.734+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc1a6a20 con 0x7f1afc102470 2024-02-28T23:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.735+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1afc10a970 con 0x7f1afc102470 2024-02-28T23:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.736+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1af0004750 con 0x7f1afc102470 2024-02-28T23:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.737+0000 7f1af9600700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 0x7f1ae406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.737+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1af008d2e0 con 0x7f1afc102470 2024-02-28T23:17:27.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.737+0000 7f1afbe00700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 0x7f1ae406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:27.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.738+0000 7f1afbe00700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 0x7f1ae406fd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1afc076a50 tx=0x7f1ae801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:27.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.742+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1af0056b20 con 0x7f1afc102470 2024-02-28T23:17:27.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.994+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1afc061190 con 0x7f1ae406d8a0 2024-02-28T23:17:27.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:27.998+0000 7f1af9600700 1 -- 172.21.15.57:0/349859568 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1afc061190 con 0x7f1ae406d8a0 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 msgr2=0x7f1ae406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 0x7f1ae406fd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1afc076a50 tx=0x7f1ae801a040 comp rx=0 tx=0).stop 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 msgr2=0x7f1afc075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d900 tx=0x7f1af000dc10 comp rx=0 tx=0).stop 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 shutdown_connections 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ae406d8a0 0x7f1ae406fd50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1afc102470 0x7f1afc075260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 --2- 172.21.15.57:0/349859568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1afc108470 0x7f1afc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 >> 172.21.15.57:0/349859568 conn(0x7f1afc0fdf90 msgr2=0x7f1afc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 shutdown_connections 2024-02-28T23:17:28.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.001+0000 7f1b03800700 1 -- 172.21.15.57:0/349859568 wait complete. 2024-02-28T23:17:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 -- 172.21.15.57:0/2206075229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22881026a0 msgr2=0x7f2288102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 --2- 172.21.15.57:0/2206075229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22881026a0 0x7f2288102a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2278009910 tx=0x7f2278009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 -- 172.21.15.57:0/2206075229 shutdown_connections 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 --2- 172.21.15.57:0/2206075229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2288103040 0x7f2288107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 --2- 172.21.15.57:0/2206075229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22881026a0 0x7f2288102a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.171+0000 7f228d400700 1 -- 172.21.15.57:0/2206075229 >> 172.21.15.57:0/2206075229 conn(0x7f22880fdf40 msgr2=0x7f2288100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:28.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.172+0000 7f228d400700 1 -- 172.21.15.57:0/2206075229 shutdown_connections 2024-02-28T23:17:28.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.172+0000 7f228d400700 1 -- 172.21.15.57:0/2206075229 wait complete. 2024-02-28T23:17:28.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.173+0000 7f228d400700 1 Processor -- start 2024-02-28T23:17:28.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.173+0000 7f228d400700 1 -- start start 2024-02-28T23:17:28.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.173+0000 7f228d400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22881026a0 0x7f22881981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f228d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f228d400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288198dd0 con 0x7f2288103040 2024-02-28T23:17:28.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f228d400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228819cb60 con 0x7f22881026a0 2024-02-28T23:17:28.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37040/0 (socket says 172.21.15.57:37040) 2024-02-28T23:17:28.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 -- 172.21.15.57:0/3996352996 learned_addr learned my addr 172.21.15.57:0/3996352996 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:28.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2286800700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22881026a0 0x7f22881981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 -- 172.21.15.57:0/3996352996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22881026a0 msgr2=0x7f22881981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22881026a0 0x7f22881981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.174+0000 7f2285e00700 1 -- 172.21.15.57:0/3996352996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780095f0 con 0x7f2288103040 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f2285e00700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f227400d8d0 tx=0x7f227400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2274004a00 con 0x7f2288103040 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22740054e0 con 0x7f2288103040 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2274009d40 con 0x7f2288103040 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228819ce40 con 0x7f2288103040 2024-02-28T23:17:28.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.175+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228819d390 con 0x7f2288103040 2024-02-28T23:17:28.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.176+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228810a9b0 con 0x7f2288103040 2024-02-28T23:17:28.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.177+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2274005020 con 0x7f2288103040 2024-02-28T23:17:28.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.178+0000 7f227f400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 0x7f226c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.178+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f227408ccd0 con 0x7f2288103040 2024-02-28T23:17:28.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.179+0000 7f2286800700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 0x7f226c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.179+0000 7f2286800700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 0x7f226c06fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f227801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:28.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.184+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22740565b0 con 0x7f2288103040 2024-02-28T23:17:28.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.437+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f228819d600 con 0x7f226c06d8a0 2024-02-28T23:17:28.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.439+0000 7f227f400700 1 -- 172.21.15.57:0/3996352996 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f228819d600 con 0x7f226c06d8a0 2024-02-28T23:17:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.442+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 msgr2=0x7f226c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.442+0000 7f228d400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 0x7f226c06fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f227801a040 comp rx=0 tx=0).stop 2024-02-28T23:17:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.442+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 msgr2=0x7f22881986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f227400d8d0 tx=0x7f227400dc90 comp rx=0 tx=0).stop 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 shutdown_connections 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f226c06d8a0 0x7f226c06fd50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22881026a0 0x7f22881981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 --2- 172.21.15.57:0/3996352996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2288103040 0x7f22881986f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 >> 172.21.15.57:0/3996352996 conn(0x7f22880fdf40 msgr2=0x7f22880ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 shutdown_connections 2024-02-28T23:17:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.443+0000 7f228d400700 1 -- 172.21.15.57:0/3996352996 wait complete. 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 -- 172.21.15.57:0/2792152665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 msgr2=0x7ff68410c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 --2- 172.21.15.57:0/2792152665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff68410c5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff670009940 tx=0x7ff670009c50 comp rx=0 tx=0).stop 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 -- 172.21.15.57:0/2792152665 shutdown_connections 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 --2- 172.21.15.57:0/2792152665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff68410c5c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 --2- 172.21.15.57:0/2792152665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 0x7ff684073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 -- 172.21.15.57:0/2792152665 >> 172.21.15.57:0/2792152665 conn(0x7ff6840fbda0 msgr2=0x7ff6840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 -- 172.21.15.57:0/2792152665 shutdown_connections 2024-02-28T23:17:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.610+0000 7ff68b800700 1 -- 172.21.15.57:0/2792152665 wait complete. 2024-02-28T23:17:28.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 Processor -- start 2024-02-28T23:17:28.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 -- start start 2024-02-28T23:17:28.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 0x7ff684198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684198da0 con 0x7ff6840739c0 2024-02-28T23:17:28.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.612+0000 7ff68b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684198ee0 con 0x7ff6840730b0 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37054/0 (socket says 172.21.15.57:37054) 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 -- 172.21.15.57:0/1548262362 learned_addr learned my addr 172.21.15.57:0/1548262362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff688e00700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 0x7ff684198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 -- 172.21.15.57:0/1548262362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 msgr2=0x7ff684198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 0x7ff684198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 -- 172.21.15.57:0/1548262362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700095f0 con 0x7ff6840739c0 2024-02-28T23:17:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.613+0000 7ff683e00700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff670005620 tx=0x7ff670005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.614+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff67001d070 con 0x7ff6840739c0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.614+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff67000f460 con 0x7ff6840739c0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.614+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68419ccd0 con 0x7ff6840739c0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.614+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff670021640 con 0x7ff6840739c0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.614+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68419d1c0 con 0x7ff6840739c0 2024-02-28T23:17:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.615+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff684109cc0 con 0x7ff6840739c0 2024-02-28T23:17:28.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.616+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff67000fa40 con 0x7ff6840739c0 2024-02-28T23:17:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.617+0000 7ff681600700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 0x7ff66c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.617+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff67008e760 con 0x7ff6840739c0 2024-02-28T23:17:28.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.618+0000 7ff688e00700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 0x7ff66c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:28.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.618+0000 7ff688e00700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 0x7ff66c06fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff678005ea0 tx=0x7ff678005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:28.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.622+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff670057fa0 con 0x7ff6840739c0 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:28.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:28.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:28.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.867+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff68419d430 con 0x7ff66c06d8a0 2024-02-28T23:17:28.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.899+0000 7ff681600700 1 -- 172.21.15.57:0/1548262362 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff68419d430 con 0x7ff66c06d8a0 2024-02-28T23:17:28.898 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (7m) 79s ago 20m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (17m) 79s ago 21m 9000k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (19m) 8s ago 19m 8782k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (3m) 79s ago 21m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (3m) 8s ago 19m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (6m) 79s ago 20m 75.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (13m) 79s ago 13m 56.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (13m) 79s ago 13m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (13m) 8s ago 13m 57.7M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (13m) 8s ago 13m 109M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (9m) 79s ago 22m 500M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (8m) 8s ago 19m 451M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:17:28.899 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (4m) 79s ago 23m 48.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (4m) 8s ago 19m 38.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (8m) 79s ago 21m 7982k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (8m) 8s ago 19m 8070k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (2m) 79s ago 18m 87.7M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (119s) 79s ago 17m 99.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (88s) 79s ago 17m 11.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (56s) 8s ago 16m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (13s) 8s ago 16m 11.2M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (15m) 8s ago 15m 231M 2241M 18.2.0 dc2bc1663786 0a11bedfc9f1 2024-02-28T23:17:28.900 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (7m) 79s ago 20m 63.0M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 msgr2=0x7ff66c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 0x7ff66c06fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff678005ea0 tx=0x7ff678005e30 comp rx=0 tx=0).stop 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 msgr2=0x7ff684198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff670005620 tx=0x7ff670005700 comp rx=0 tx=0).stop 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 shutdown_connections 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff66c06d8a0 0x7ff66c06fd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6840730b0 0x7ff684198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 --2- 172.21.15.57:0/1548262362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6840739c0 0x7ff684198750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 >> 172.21.15.57:0/1548262362 conn(0x7ff6840fbda0 msgr2=0x7ff684106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 shutdown_connections 2024-02-28T23:17:28.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:28.903+0000 7ff68b800700 1 -- 172.21.15.57:0/1548262362 wait complete. 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 -- 172.21.15.57:0/183817697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca01024d0 msgr2=0x7f6ca01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 --2- 172.21.15.57:0/183817697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca01024d0 0x7f6ca01028a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009910 tx=0x7f6c90009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 -- 172.21.15.57:0/183817697 shutdown_connections 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 --2- 172.21.15.57:0/183817697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca0102e70 0x7f6ca0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 --2- 172.21.15.57:0/183817697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca01024d0 0x7f6ca01028a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 -- 172.21.15.57:0/183817697 >> 172.21.15.57:0/183817697 conn(0x7f6ca00fdd90 msgr2=0x7f6ca01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 -- 172.21.15.57:0/183817697 shutdown_connections 2024-02-28T23:17:29.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.070+0000 7f6c9f200700 1 -- 172.21.15.57:0/183817697 wait complete. 2024-02-28T23:17:29.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.071+0000 7f6c9f200700 1 Processor -- start 2024-02-28T23:17:29.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.071+0000 7f6c9f200700 1 -- start start 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c9f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca01024d0 0x7f6ca0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c9f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c9f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00793f0 con 0x7f6ca0102e70 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c9f200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0075ce0 con 0x7f6ca01024d0 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c97e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37070/0 (socket says 172.21.15.57:37070) 2024-02-28T23:17:29.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.072+0000 7f6c97e00700 1 -- 172.21.15.57:0/623811398 learned_addr learned my addr 172.21.15.57:0/623811398 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:29.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.073+0000 7f6c9de00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca01024d0 0x7f6ca0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.073+0000 7f6c97e00700 1 -- 172.21.15.57:0/623811398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca01024d0 msgr2=0x7f6ca0075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.073+0000 7f6c97e00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca01024d0 0x7f6ca0075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.073+0000 7f6c97e00700 1 -- 172.21.15.57:0/623811398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c900095f0 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.073+0000 7f6c97e00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b480 tx=0x7f6c8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.074+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c880118c0 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.074+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0075fc0 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.074+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c88010380 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.074+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01a6760 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.074+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8801a5d0 con 0x7f6ca0102e70 2024-02-28T23:17:29.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.075+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca010a7e0 con 0x7f6ca0102e70 2024-02-28T23:17:29.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.076+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6c880104f0 con 0x7f6ca0102e70 2024-02-28T23:17:29.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.077+0000 7f6c96a00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 0x7f6c8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.077+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c8808d0b0 con 0x7f6ca0102e70 2024-02-28T23:17:29.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.077+0000 7f6c9de00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 0x7f6c8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.078+0000 7f6c9de00700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 0x7f6c8406fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c9001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.082+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c88056870 con 0x7f6ca0102e70 2024-02-28T23:17:29.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.398+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ca0062380 con 0x7f6ca0102e70 2024-02-28T23:17:29.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.399+0000 7f6c96a00700 1 -- 172.21.15.57:0/623811398 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f6c88059f30 con 0x7f6ca0102e70 2024-02-28T23:17:29.398 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:17:29.398 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:17:29.398 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:17:29.398 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 5 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 9 2024-02-28T23:17:29.399 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:17:29.400 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 msgr2=0x7f6c8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 0x7f6c8406fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c9001a040 comp rx=0 tx=0).stop 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 msgr2=0x7f6ca00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b480 tx=0x7f6c8800b840 comp rx=0 tx=0).stop 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 shutdown_connections 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c8406d8a0 0x7f6c8406fd50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ca01024d0 0x7f6ca0075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 --2- 172.21.15.57:0/623811398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca0102e70 0x7f6ca00757a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.402+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 >> 172.21.15.57:0/623811398 conn(0x7f6ca00fdd90 msgr2=0x7f6ca00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:29.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.403+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 shutdown_connections 2024-02-28T23:17:29.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.404+0000 7f6c9f200700 1 -- 172.21.15.57:0/623811398 wait complete. 2024-02-28T23:17:29.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 -- 172.21.15.57:0/1961258798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 msgr2=0x7f0f441008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 --2- 172.21.15.57:0/1961258798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f441008b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009910 tx=0x7f0f34009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:29.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 -- 172.21.15.57:0/1961258798 shutdown_connections 2024-02-28T23:17:29.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 --2- 172.21.15.57:0/1961258798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 --2- 172.21.15.57:0/1961258798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f441008b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.568+0000 7f0f4b400700 1 -- 172.21.15.57:0/1961258798 >> 172.21.15.57:0/1961258798 conn(0x7f0f440fbda0 msgr2=0x7f0f440fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:29.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.569+0000 7f0f4b400700 1 -- 172.21.15.57:0/1961258798 shutdown_connections 2024-02-28T23:17:29.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.569+0000 7f0f4b400700 1 -- 172.21.15.57:0/1961258798 wait complete. 2024-02-28T23:17:29.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.570+0000 7f0f4b400700 1 Processor -- start 2024-02-28T23:17:29.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.570+0000 7f0f4b400700 1 -- start start 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f4b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f4b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44193140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f4b400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44196d90 con 0x7f0f441004e0 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f4b400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44193680 con 0x7f0f44100e80 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f48a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f48a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37082/0 (socket says 172.21.15.57:37082) 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f48a00700 1 -- 172.21.15.57:0/3562621794 learned_addr learned my addr 172.21.15.57:0/3562621794 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.571+0000 7f0f3be00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44193140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.572+0000 7f0f48a00700 1 -- 172.21.15.57:0/3562621794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 msgr2=0x7f0f44193140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.572+0000 7f0f48a00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44193140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.572+0000 7f0f48a00700 1 -- 172.21.15.57:0/3562621794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340095f0 con 0x7f0f441004e0 2024-02-28T23:17:29.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.572+0000 7f0f3be00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44193140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:17:29.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.572+0000 7f0f48a00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f340092e0 tx=0x7f0f3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:29.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.573+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f3401d070 con 0x7f0f441004e0 2024-02-28T23:17:29.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.573+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f34005e00 con 0x7f0f441004e0 2024-02-28T23:17:29.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.573+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f34017400 con 0x7f0f441004e0 2024-02-28T23:17:29.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.573+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f44193900 con 0x7f0f441004e0 2024-02-28T23:17:29.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.573+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441a6870 con 0x7f0f441004e0 2024-02-28T23:17:29.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.575+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f441045d0 con 0x7f0f441004e0 2024-02-28T23:17:29.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.576+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0f3400f460 con 0x7f0f441004e0 2024-02-28T23:17:29.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.577+0000 7f0f42000700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 0x7f0f2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:29.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.577+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0f3408e8e0 con 0x7f0f441004e0 2024-02-28T23:17:29.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.577+0000 7f0f3be00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 0x7f0f2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:29.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.578+0000 7f0f3be00700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 0x7f0f2806fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:29.577 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:29.578 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-28T23:17:29.578 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:29.578 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: pgmap v168: 65 pgs: 10 peering, 4 active+undersized+degraded, 6 active+undersized, 45 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 14/321 objects degraded (4.361%) 2024-02-28T23:17:29.578 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:29.578 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/623811398' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:29.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.582+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f340581c0 con 0x7f0f441004e0 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: pgmap v168: 65 pgs: 10 peering, 4 active+undersized+degraded, 6 active+undersized, 45 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 14/321 objects degraded (4.361%) 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/623811398' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:17:29.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.864+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f44194240 con 0x7f0f441004e0 2024-02-28T23:17:29.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.865+0000 7f0f42000700 1 -- 172.21.15.57:0/3562621794 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f0f3405b880 con 0x7f0f441004e0 2024-02-28T23:17:29.863 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:17:29.864 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:17:29.865 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:17:29.866 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 msgr2=0x7f0f2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 0x7f0f2806fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 msgr2=0x7f0f44192c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f340092e0 tx=0x7f0f3400be20 comp rx=0 tx=0).stop 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 shutdown_connections 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f2806d8a0 0x7f0f2806fd50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f441004e0 0x7f0f44192c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 --2- 172.21.15.57:0/3562621794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f44100e80 0x7f0f44193140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:29.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 >> 172.21.15.57:0/3562621794 conn(0x7f0f440fbda0 msgr2=0x7f0f440fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:29.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 shutdown_connections 2024-02-28T23:17:29.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:29.868+0000 7f0f4b400700 1 -- 172.21.15.57:0/3562621794 wait complete. 2024-02-28T23:17:29.868 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:17:30.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 -- 172.21.15.57:0/4145689014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44106b80 msgr2=0x7f7a44106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 --2- 172.21.15.57:0/4145689014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44106b80 0x7f7a44106ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009910 tx=0x7f7a30009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:30.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 -- 172.21.15.57:0/4145689014 shutdown_connections 2024-02-28T23:17:30.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 --2- 172.21.15.57:0/4145689014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44106b80 0x7f7a44106ff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 --2- 172.21.15.57:0/4145689014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4410c660 0x7f7a4410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.035+0000 7f7a49800700 1 -- 172.21.15.57:0/4145689014 >> 172.21.15.57:0/4145689014 conn(0x7f7a44075e90 msgr2=0x7f7a440782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:30.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.036+0000 7f7a49800700 1 -- 172.21.15.57:0/4145689014 shutdown_connections 2024-02-28T23:17:30.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.036+0000 7f7a49800700 1 -- 172.21.15.57:0/4145689014 wait complete. 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 Processor -- start 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 -- start start 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a44106b80 0x7f7a4419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4419ce00 con 0x7f7a4410c660 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.037+0000 7f7a49800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441a0b90 con 0x7f7a44106b80 2024-02-28T23:17:30.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37090/0 (socket says 172.21.15.57:37090) 2024-02-28T23:17:30.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 -- 172.21.15.57:0/260829772 learned_addr learned my addr 172.21.15.57:0/260829772 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:30.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43e00700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a44106b80 0x7f7a4419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 -- 172.21.15.57:0/260829772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a44106b80 msgr2=0x7f7a4419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a44106b80 0x7f7a4419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 -- 172.21.15.57:0/260829772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300095f0 con 0x7f7a4410c660 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.038+0000 7f7a43400700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b5c0 tx=0x7f7a3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.039+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3001d070 con 0x7f7a4410c660 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.039+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a30005e00 con 0x7f7a4410c660 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.039+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a441a0e10 con 0x7f7a4410c660 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.039+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a30017400 con 0x7f7a4410c660 2024-02-28T23:17:30.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.039+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a441a1300 con 0x7f7a4410c660 2024-02-28T23:17:30.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.040+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4410eb40 con 0x7f7a4410c660 2024-02-28T23:17:30.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.041+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7a3000f460 con 0x7f7a4410c660 2024-02-28T23:17:30.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.042+0000 7f7a40c00700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 0x7f7a2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.042+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7a3008e960 con 0x7f7a4410c660 2024-02-28T23:17:30.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.042+0000 7f7a43e00700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 0x7f7a2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.043+0000 7f7a43e00700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 0x7f7a2806fd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005950 tx=0x7f7a340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:30.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.047+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a300581a0 con 0x7f7a4410c660 2024-02-28T23:17:30.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.298+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a4402cca0 con 0x7f7a2806d8a0 2024-02-28T23:17:30.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.300+0000 7f7a40c00700 1 -- 172.21.15.57:0/260829772 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7a4402cca0 con 0x7f7a2806d8a0 2024-02-28T23:17:30.299 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:17:30.299 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:17:30.300 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:17:30.300 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:17:30.300 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:17:30.300 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:17:30.300 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout: "crash" 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "11/23 daemons upgraded", 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:17:30.301 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:17:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 msgr2=0x7f7a2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 0x7f7a2806fd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005950 tx=0x7f7a340058e0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 msgr2=0x7f7a4419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b5c0 tx=0x7f7a3000be20 comp rx=0 tx=0).stop 2024-02-28T23:17:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 shutdown_connections 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a2806d8a0 0x7f7a2806fd50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a44106b80 0x7f7a4419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.303+0000 7f7a49800700 1 --2- 172.21.15.57:0/260829772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a4410c660 0x7f7a4419c720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.304+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 >> 172.21.15.57:0/260829772 conn(0x7f7a44075e90 msgr2=0x7f7a440778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.304+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 shutdown_connections 2024-02-28T23:17:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.304+0000 7f7a49800700 1 -- 172.21.15.57:0/260829772 wait complete. 2024-02-28T23:17:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.466+0000 7f022fe00700 1 -- 172.21.15.57:0/1731672559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 msgr2=0x7f02301028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.466+0000 7f022fe00700 1 --2- 172.21.15.57:0/1731672559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f02301028a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0220009910 tx=0x7f0220009c20 comp rx=0 tx=0).stop 2024-02-28T23:17:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 -- 172.21.15.57:0/1731672559 shutdown_connections 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 --2- 172.21.15.57:0/1731672559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 0x7f0230106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 --2- 172.21.15.57:0/1731672559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f02301028a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 -- 172.21.15.57:0/1731672559 >> 172.21.15.57:0/1731672559 conn(0x7f02300fdd90 msgr2=0x7f02301001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 -- 172.21.15.57:0/1731672559 shutdown_connections 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.467+0000 7f022fe00700 1 -- 172.21.15.57:0/1731672559 wait complete. 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 Processor -- start 2024-02-28T23:17:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 -- start start 2024-02-28T23:17:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 0x7f02301983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0230198a80 con 0x7f02301024d0 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.468+0000 7f022fe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023019c810 con 0x7f0230102e70 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37108/0 (socket says 172.21.15.57:37108) 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 -- 172.21.15.57:0/790879662 learned_addr learned my addr 172.21.15.57:0/790879662 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f0226000700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 0x7f02301983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 -- 172.21.15.57:0/790879662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 msgr2=0x7f02301983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 0x7f02301983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.469+0000 7f022ea00700 1 -- 172.21.15.57:0/790879662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200095f0 con 0x7f02301024d0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f022ea00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0220006010 tx=0x7f022000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022001d070 con 0x7f02301024d0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f022000f460 con 0x7f02301024d0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023019ca90 con 0x7f02301024d0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0220017630 con 0x7f02301024d0 2024-02-28T23:17:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.470+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023019cf80 con 0x7f02301024d0 2024-02-28T23:17:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.471+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023010a7e0 con 0x7f02301024d0 2024-02-28T23:17:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.472+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f022000f5d0 con 0x7f02301024d0 2024-02-28T23:17:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.473+0000 7f0234a00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 0x7f021406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:17:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.474+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f022008e960 con 0x7f02301024d0 2024-02-28T23:17:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.474+0000 7f0226000700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 0x7f021406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:17:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.475+0000 7f0226000700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 0x7f021406fd50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0218005950 tx=0x7f02180058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:17:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.478+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f02200581a0 con 0x7f02301024d0 2024-02-28T23:17:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.809+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0230062380 con 0x7f02301024d0 2024-02-28T23:17:30.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.810+0000 7f0234a00700 1 -- 172.21.15.57:0/790879662 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f022005b860 con 0x7f02301024d0 2024-02-28T23:17:30.809 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:17:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.814+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 msgr2=0x7f021406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.814+0000 7f022fe00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 0x7f021406fd50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0218005950 tx=0x7f02180058e0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.814+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 msgr2=0x7f0230197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:17:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.814+0000 7f022fe00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0220006010 tx=0x7f022000be20 comp rx=0 tx=0).stop 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 shutdown_connections 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f021406d8a0 0x7f021406fd50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02301024d0 0x7f0230197e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 --2- 172.21.15.57:0/790879662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0230102e70 0x7f02301983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 >> 172.21.15.57:0/790879662 conn(0x7f02300fdd90 msgr2=0x7f0230100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 shutdown_connections 2024-02-28T23:17:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:17:30.815+0000 7f022fe00700 1 -- 172.21.15.57:0/790879662 wait complete. 2024-02-28T23:17:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:30 smithi084 ceph-mon[70440]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3562621794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:17:30.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:30 smithi057 ceph-mon[89491]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:30.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3562621794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:17:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:31 smithi057 ceph-mon[89491]: pgmap v169: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:17:32.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:31 smithi057 ceph-mon[89491]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:32.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:31 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded) 2024-02-28T23:17:32.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:31 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:17:32.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:31 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/790879662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:17:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:31 smithi084 ceph-mon[70440]: pgmap v169: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:17:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:31 smithi084 ceph-mon[70440]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:17:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:31 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/321 objects degraded (4.361%), 4 pgs degraded) 2024-02-28T23:17:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:31 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:17:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:31 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/790879662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:17:34.007 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:33 smithi084 ceph-mon[70440]: pgmap v170: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-02-28T23:17:34.007 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:34.007 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:33 smithi057 ceph-mon[89491]: pgmap v170: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-02-28T23:17:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:35 smithi057 ceph-mon[89491]: pgmap v171: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:17:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:35 smithi084 ceph-mon[70440]: pgmap v171: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:17:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:37 smithi057 ceph-mon[89491]: pgmap v172: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:37 smithi084 ceph-mon[70440]: pgmap v172: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:39 smithi057 ceph-mon[89491]: pgmap v173: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:17:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:39 smithi084 ceph-mon[70440]: pgmap v173: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:17:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:41.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:41 smithi057 ceph-mon[89491]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:17:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:41 smithi084 ceph-mon[70440]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:17:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:43 smithi057 ceph-mon[89491]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:43 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:43.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:43 smithi084 ceph-mon[70440]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:17:43.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:43 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:44.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:44 smithi057 ceph-mon[89491]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:44.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:44 smithi057 ceph-mon[89491]: Upgrade: osd.5 is safe to restart 2024-02-28T23:17:44.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:44 smithi084 ceph-mon[70440]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-28T23:17:44.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:44 smithi084 ceph-mon[70440]: Upgrade: osd.5 is safe to restart 2024-02-28T23:17:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:45 smithi057 ceph-mon[89491]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:17:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:17:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:45 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:45 smithi084 ceph-mon[70440]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:17:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-28T23:17:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:45 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:17:46.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:46 smithi057 ceph-mon[89491]: Upgrade: Updating osd.5 2024-02-28T23:17:46.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:46 smithi057 ceph-mon[89491]: Deploying daemon osd.5 on smithi084 2024-02-28T23:17:46.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:46 smithi084 ceph-mon[70440]: Upgrade: Updating osd.5 2024-02-28T23:17:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:46 smithi084 ceph-mon[70440]: Deploying daemon osd.5 on smithi084 2024-02-28T23:17:47.533 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:47 smithi084 ceph-mon[70440]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:17:47.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:47 smithi057 ceph-mon[89491]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:17:48.496 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:48.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:17:49.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:49 smithi057 ceph-mon[89491]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:17:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:49 smithi084 ceph-mon[70440]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:17:50.080 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:49 smithi084 systemd[1]: Stopping Ceph osd.5 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:17:50.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:50 smithi057 ceph-mon[89491]: osd.5 marked itself down and dead 2024-02-28T23:17:50.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:50 smithi084 ceph-mon[70440]: osd.5 marked itself down and dead 2024-02-28T23:17:50.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:50 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[50238]: 2024-02-28T23:17:50.194+0000 7f9875600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:17:50.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:50 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[50238]: 2024-02-28T23:17:50.194+0000 7f9875600700 -1 osd.5 75 *** Got signal Terminated *** 2024-02-28T23:17:50.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:50 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[50238]: 2024-02-28T23:17:50.194+0000 7f9875600700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:17:51.077 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:50 smithi084 podman[89816]: 2024-02-28 23:17:50.792588938 +0000 UTC m=+0.743007851 container died 0a11bedfc9f19724fdaeddefc530cbc6d1c5b010949aea9a787b165048666831 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-28T23:17:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:51 smithi084 ceph-mon[70440]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:17:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:51 smithi084 ceph-mon[70440]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:17:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:51 smithi084 ceph-mon[70440]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-28T23:17:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:51 smithi084 ceph-mon[70440]: osdmap e76: 6 total, 5 up, 6 in 2024-02-28T23:17:51.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:51 smithi084 podman[89816]: 2024-02-28 23:17:51.204444109 +0000 UTC m=+1.154863050 container cleanup 0a11bedfc9f19724fdaeddefc530cbc6d1c5b010949aea9a787b165048666831 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2) 2024-02-28T23:17:51.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:51 smithi084 bash[89816]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5 2024-02-28T23:17:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:51 smithi057 ceph-mon[89491]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:17:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:51 smithi057 ceph-mon[89491]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-28T23:17:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:51 smithi057 ceph-mon[89491]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-28T23:17:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:51 smithi057 ceph-mon[89491]: osdmap e76: 6 total, 5 up, 6 in 2024-02-28T23:17:51.830 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:51 smithi084 podman[89837]: 2024-02-28 23:17:51.497384832 +0000 UTC m=+0.691657727 container remove 0a11bedfc9f19724fdaeddefc530cbc6d1c5b010949aea9a787b165048666831 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD) 2024-02-28T23:17:52.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:52 smithi084 ceph-mon[70440]: osdmap e77: 6 total, 5 up, 6 in 2024-02-28T23:17:52.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:52 smithi084 podman[89923]: 2024-02-28 23:17:52.262087448 +0000 UTC m=+0.033938695 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:52.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:52 smithi084 podman[89923]: 2024-02-28 23:17:52.364891351 +0000 UTC m=+0.136742615 container create dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-28T23:17:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:52 smithi057 ceph-mon[89491]: osdmap e77: 6 total, 5 up, 6 in 2024-02-28T23:17:53.064 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:52 smithi084 podman[89923]: 2024-02-28 23:17:52.627082842 +0000 UTC m=+0.398934132 container init dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:17:53.064 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:52 smithi084 podman[89923]: 2024-02-28 23:17:52.63828753 +0000 UTC m=+0.410138802 container start dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-02-28T23:17:53.064 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:52 smithi084 podman[89923]: 2024-02-28 23:17:52.758994164 +0000 UTC m=+0.530845464 container attach dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-28T23:17:53.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:53 smithi084 podman[89923]: 2024-02-28 23:17:53.066225077 +0000 UTC m=+0.838076377 container died dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-02-28T23:17:53.662 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:53 smithi084 ceph-mon[70440]: pgmap v182: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-02-28T23:17:53.662 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:53 smithi084 ceph-mon[70440]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-28T23:17:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:53 smithi057 ceph-mon[89491]: pgmap v182: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-02-28T23:17:53.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:53 smithi057 ceph-mon[89491]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-28T23:17:54.080 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:53 smithi084 podman[89948]: 2024-02-28 23:17:53.787746113 +0000 UTC m=+0.703288038 container remove dd1df6725d879089477a76376a387f9861247c4618858f05bf268ae932c5e556 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-28T23:17:54.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:53 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-28T23:17:54.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:53 smithi084 systemd[1]: Stopped Ceph osd.5 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:17:54.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:53 smithi084 systemd[1]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.5.service: Consumed 1min 52.771s CPU time. 2024-02-28T23:17:54.545 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 systemd[1]: Starting Ceph osd.5 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:17:54.546 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:54 smithi084 ceph-mon[70440]: Health check failed: Degraded data redundancy: 14/321 objects degraded (4.361%), 6 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:54 smithi057 ceph-mon[89491]: Health check failed: Degraded data redundancy: 14/321 objects degraded (4.361%), 6 pgs degraded (PG_DEGRADED) 2024-02-28T23:17:54.830 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 podman[90052]: 2024-02-28 23:17:54.446924271 +0000 UTC m=+0.032118795 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:54.830 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 podman[90052]: 2024-02-28 23:17:54.571923169 +0000 UTC m=+0.157117734 container create 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-28T23:17:55.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 podman[90052]: 2024-02-28 23:17:54.836356429 +0000 UTC m=+0.421551034 container init 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-28T23:17:55.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 podman[90052]: 2024-02-28 23:17:54.847768728 +0000 UTC m=+0.432963296 container start 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-28T23:17:55.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:54 smithi084 podman[90052]: 2024-02-28 23:17:54.924345272 +0000 UTC m=+0.509539856 container attach 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:17:55.706 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-mon[70440]: pgmap v183: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/321 objects degraded (4.361%) 2024-02-28T23:17:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:55 smithi057 ceph-mon[89491]: pgmap v183: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/321 objects degraded (4.361%) 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-28T23:17:56.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 bash[90052]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate[90067]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-28T23:17:56.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:55 smithi084 podman[90052]: 2024-02-28 23:17:55.769016198 +0000 UTC m=+1.354210731 container died 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-02-28T23:17:56.830 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:56 smithi084 podman[90198]: 2024-02-28 23:17:56.527893332 +0000 UTC m=+0.739574443 container remove 1fc6611f0fa2addcc115c93afe8a9ded03022c9a96ee4dc8a442b9d9fc808cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-activate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-28T23:17:57.164 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:56 smithi084 podman[90259]: 2024-02-28 23:17:56.794030939 +0000 UTC m=+0.032475838 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:17:57.164 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:56 smithi084 podman[90259]: 2024-02-28 23:17:56.929062903 +0000 UTC m=+0.167507838 container create bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-28T23:17:57.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:57 smithi084 ceph-mon[70440]: pgmap v184: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/321 objects degraded (13.707%) 2024-02-28T23:17:57.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 podman[90259]: 2024-02-28 23:17:57.165758768 +0000 UTC m=+0.404203763 container init bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-28T23:17:57.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 podman[90259]: 2024-02-28 23:17:57.176193579 +0000 UTC m=+0.414638521 container start bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-02-28T23:17:57.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 ceph-osd[90277]: -- 172.21.15.84:0/2904198226 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fb126f7680 con 0x55fb126c0c00 2024-02-28T23:17:57.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 bash[90259]: bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 2024-02-28T23:17:57.580 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 systemd[1]: Started Ceph osd.5 for 3640fe00-d68c-11ee-95c3-87774f69a715. 2024-02-28T23:17:57.655 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:57 smithi057 ceph-mon[89491]: pgmap v184: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/321 objects degraded (13.707%) 2024-02-28T23:17:58.080 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:17:57 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:17:57.749+0000 7f78389b67c0 -1 Falling back to public interface 2024-02-28T23:17:58.744 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:58.745 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:58.745 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:58.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:17:58.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:17:59.809 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:59 smithi084 ceph-mon[70440]: pgmap v185: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:17:59.809 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:17:59 smithi084 ceph-mon[70440]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-28T23:17:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:59 smithi057 ceph-mon[89491]: pgmap v185: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:17:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:17:59 smithi057 ceph-mon[89491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 -- 172.21.15.57:0/4258597052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd624100e80 msgr2=0x7fd624104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 --2- 172.21.15.57:0/4258597052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd624100e80 0x7fd624104e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd610009870 tx=0x7fd610009b80 comp rx=0 tx=0).stop 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 -- 172.21.15.57:0/4258597052 shutdown_connections 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 --2- 172.21.15.57:0/4258597052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd624100e80 0x7fd624104e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 --2- 172.21.15.57:0/4258597052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6241004e0 0x7fd6241008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 -- 172.21.15.57:0/4258597052 >> 172.21.15.57:0/4258597052 conn(0x7fd6240fbda0 msgr2=0x7fd6240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 -- 172.21.15.57:0/4258597052 shutdown_connections 2024-02-28T23:18:00.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.993+0000 7fd62b800700 1 -- 172.21.15.57:0/4258597052 wait complete. 2024-02-28T23:18:00.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.994+0000 7fd62b800700 1 Processor -- start 2024-02-28T23:18:00.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.994+0000 7fd62b800700 1 -- start start 2024-02-28T23:18:00.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd62b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd62b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100e80 0x7fd624196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd62b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624196b30 con 0x7fd6241004e0 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd62b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62419a8c0 con 0x7fd624100e80 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd628e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd628e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58910/0 (socket says 172.21.15.57:58910) 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd628e00700 1 -- 172.21.15.57:0/566119912 learned_addr learned my addr 172.21.15.57:0/566119912 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.995+0000 7fd623e00700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100e80 0x7fd624196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd628e00700 1 -- 172.21.15.57:0/566119912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100e80 msgr2=0x7fd624196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd628e00700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100e80 0x7fd624196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd628e00700 1 -- 172.21.15.57:0/566119912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180095f0 con 0x7fd6241004e0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd628e00700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eb90 tx=0x7fd61800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd61800ccc0 con 0x7fd6241004e0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.996+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd61800ce20 con 0x7fd6241004e0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.997+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd610009520 con 0x7fd6241004e0 2024-02-28T23:18:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.997+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd618010660 con 0x7fd6241004e0 2024-02-28T23:18:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.997+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62419af60 con 0x7fd6241004e0 2024-02-28T23:18:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.998+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624108850 con 0x7fd6241004e0 2024-02-28T23:18:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:00.999+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd618003e50 con 0x7fd6241004e0 2024-02-28T23:18:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.000+0000 7fd621600700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 0x7fd60c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.000+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd618014070 con 0x7fd6241004e0 2024-02-28T23:18:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.000+0000 7fd623e00700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 0x7fd60c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.001+0000 7fd623e00700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 0x7fd60c06fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd624197530 tx=0x7fd610019040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:01.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.005+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd618057340 con 0x7fd6241004e0 2024-02-28T23:18:01.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.257+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd62419b1e0 con 0x7fd60c06d8a0 2024-02-28T23:18:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.259+0000 7fd621600700 1 -- 172.21.15.57:0/566119912 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd62419b1e0 con 0x7fd60c06d8a0 2024-02-28T23:18:01.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.262+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 msgr2=0x7fd60c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.262+0000 7fd62b800700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 0x7fd60c06fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd624197530 tx=0x7fd610019040 comp rx=0 tx=0).stop 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.262+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 msgr2=0x7fd624195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.262+0000 7fd62b800700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eb90 tx=0x7fd61800eea0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 shutdown_connections 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd60c06d8a0 0x7fd60c06fd50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6241004e0 0x7fd624195f10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 --2- 172.21.15.57:0/566119912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100e80 0x7fd624196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 >> 172.21.15.57:0/566119912 conn(0x7fd6240fbda0 msgr2=0x7fd6240fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 shutdown_connections 2024-02-28T23:18:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.263+0000 7fd62b800700 1 -- 172.21.15.57:0/566119912 wait complete. 2024-02-28T23:18:01.279 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:18:01.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 -- 172.21.15.57:0/435794590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 msgr2=0x7fc51c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 --2- 172.21.15.57:0/435794590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c108840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00b0a0 tx=0x7fc50c00b3b0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 -- 172.21.15.57:0/435794590 shutdown_connections 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 --2- 172.21.15.57:0/435794590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 --2- 172.21.15.57:0/435794590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c108840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 -- 172.21.15.57:0/435794590 >> 172.21.15.57:0/435794590 conn(0x7fc51c0fdf90 msgr2=0x7fc51c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 -- 172.21.15.57:0/435794590 shutdown_connections 2024-02-28T23:18:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.434+0000 7fc522c00700 1 -- 172.21.15.57:0/435794590 wait complete. 2024-02-28T23:18:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.435+0000 7fc522c00700 1 Processor -- start 2024-02-28T23:18:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc522c00700 1 -- start start 2024-02-28T23:18:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc522c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc522c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc522c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c198e30 con 0x7fc51c108470 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc522c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19cbc0 con 0x7fc51c102470 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc513400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc513400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58924/0 (socket says 172.21.15.57:58924) 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc513400700 1 -- 172.21.15.57:0/2311963203 learned_addr learned my addr 172.21.15.57:0/2311963203 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.436+0000 7fc51be00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.437+0000 7fc513400700 1 -- 172.21.15.57:0/2311963203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 msgr2=0x7fc51c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.437+0000 7fc513400700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.437+0000 7fc513400700 1 -- 172.21.15.57:0/2311963203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c009d00 con 0x7fc51c108470 2024-02-28T23:18:01.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.437+0000 7fc51be00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.437+0000 7fc513400700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b4b0 tx=0x7fc50400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.438+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc504011800 con 0x7fc51c108470 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.438+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc504011e20 con 0x7fc51c108470 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.438+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c19cea0 con 0x7fc51c108470 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.438+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc50400f2e0 con 0x7fc51c108470 2024-02-28T23:18:01.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.438+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c19d3f0 con 0x7fc51c108470 2024-02-28T23:18:01.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.439+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c10a9e0 con 0x7fc51c108470 2024-02-28T23:18:01.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.440+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc504010380 con 0x7fc51c108470 2024-02-28T23:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.441+0000 7fc519600700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 0x7fc50006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.441+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc50408cf80 con 0x7fc51c108470 2024-02-28T23:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.442+0000 7fc51be00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 0x7fc50006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.442+0000 7fc51be00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 0x7fc50006fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000f80 tx=0x7fc50c0097e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:01.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.450+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc5040567c0 con 0x7fc51c108470 2024-02-28T23:18:01.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:01 smithi057 ceph-mon[89491]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:18:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.705+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc51c19d730 con 0x7fc50006d8a0 2024-02-28T23:18:01.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.708+0000 7fc519600700 1 -- 172.21.15.57:0/2311963203 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc51c19d730 con 0x7fc50006d8a0 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 msgr2=0x7fc50006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 0x7fc50006fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000f80 tx=0x7fc50c0097e0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 msgr2=0x7fc51c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b4b0 tx=0x7fc50400b7c0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 shutdown_connections 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc50006d8a0 0x7fc50006fd50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc51c102470 0x7fc51c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 --2- 172.21.15.57:0/2311963203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c108470 0x7fc51c198750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.711+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 >> 172.21.15.57:0/2311963203 conn(0x7fc51c0fdf90 msgr2=0x7fc51c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:01.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.712+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 shutdown_connections 2024-02-28T23:18:01.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.712+0000 7fc522c00700 1 -- 172.21.15.57:0/2311963203 wait complete. 2024-02-28T23:18:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:01 smithi084 ceph-mon[70440]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:18:01.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 -- 172.21.15.57:0/894727561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 msgr2=0x7f7024073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 --2- 172.21.15.57:0/894727561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024073460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7014009940 tx=0x7f7014009c50 comp rx=0 tx=0).stop 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 -- 172.21.15.57:0/894727561 shutdown_connections 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 --2- 172.21.15.57:0/894727561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f702410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 --2- 172.21.15.57:0/894727561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024073460 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 -- 172.21.15.57:0/894727561 >> 172.21.15.57:0/894727561 conn(0x7f70240fbda0 msgr2=0x7f70240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 -- 172.21.15.57:0/894727561 shutdown_connections 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.879+0000 7f702a000700 1 -- 172.21.15.57:0/894727561 wait complete. 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.880+0000 7f702a000700 1 Processor -- start 2024-02-28T23:18:01.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.880+0000 7f702a000700 1 -- start start 2024-02-28T23:18:01.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.880+0000 7f702a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f702a000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f70241986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f702a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7024198da0 con 0x7f7024073090 2024-02-28T23:18:01.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f702a000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702419cb30 con 0x7f70240739a0 2024-02-28T23:18:01.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7023400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7023400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58942/0 (socket says 172.21.15.57:58942) 2024-02-28T23:18:01.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7023400700 1 -- 172.21.15.57:0/4261732918 learned_addr learned my addr 172.21.15.57:0/4261732918 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:01.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7022a00700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f70241986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7022a00700 1 -- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 msgr2=0x7f7024198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7022a00700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7022a00700 1 -- 172.21.15.57:0/4261732918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140095f0 con 0x7f70240739a0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.881+0000 7f7023400700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f7022a00700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f70241986c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f701000e970 tx=0x7f701000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701000c9d0 con 0x7f70240739a0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7010010430 con 0x7f70240739a0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7010018630 con 0x7f70240739a0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702419ce10 con 0x7f70240739a0 2024-02-28T23:18:01.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.882+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702419d360 con 0x7f70240739a0 2024-02-28T23:18:01.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.883+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7024109ce0 con 0x7f70240739a0 2024-02-28T23:18:01.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.885+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f70100105a0 con 0x7f70240739a0 2024-02-28T23:18:01.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.885+0000 7f701be00700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 0x7f700806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:01.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.886+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7010014070 con 0x7f70240739a0 2024-02-28T23:18:01.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.886+0000 7f7023400700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 0x7f700806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:01.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.887+0000 7f7023400700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 0x7f700806fd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f701400b780 tx=0x7f7014005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:01.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:01.894+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7010052d20 con 0x7f70240739a0 2024-02-28T23:18:02.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.138+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f702419d580 con 0x7f700806d8a0 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.157+0000 7f701be00700 1 -- 172.21.15.57:0/4261732918 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f702419d580 con 0x7f700806d8a0 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (7m) 112s ago 21m 13.7M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (17m) 112s ago 21m 9000k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (20m) 0s ago 20m 9038k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (3m) 112s ago 21m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:18:02.157 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (3m) 0s ago 20m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (7m) 112s ago 21m 75.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (14m) 112s ago 14m 56.9M - 18.2.0 dc2bc1663786 a9935010f465 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (14m) 112s ago 14m 114M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (14m) 0s ago 14m 58.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (14m) 0s ago 14m 108M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (10m) 112s ago 23m 500M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (9m) 0s ago 20m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (5m) 112s ago 23m 48.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (4m) 0s ago 19m 39.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (8m) 112s ago 21m 7982k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (8m) 0s ago 20m 8174k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (3m) 112s ago 19m 87.7M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (2m) 112s ago 18m 99.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (2m) 112s ago 17m 11.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:18:02.158 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (89s) 0s ago 17m 105M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:18:02.159 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (46s) 0s ago 16m 99.0M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:18:02.159 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (5s) 0s ago 16m 11.1M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:18:02.159 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (8m) 112s ago 20m 63.0M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:18:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 msgr2=0x7f700806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 0x7f700806fd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f701400b780 tx=0x7f7014005ab0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 msgr2=0x7f70241986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f70241986c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f701000e970 tx=0x7f701000ec80 comp rx=0 tx=0).stop 2024-02-28T23:18:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 shutdown_connections 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700806d8a0 0x7f700806fd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.160+0000 7f702a000700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7024073090 0x7f7024198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.161+0000 7f702a000700 1 --2- 172.21.15.57:0/4261732918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70240739a0 0x7f70241986c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.161+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 >> 172.21.15.57:0/4261732918 conn(0x7f70240fbda0 msgr2=0x7f7024106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.161+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 shutdown_connections 2024-02-28T23:18:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.161+0000 7f702a000700 1 -- 172.21.15.57:0/4261732918 wait complete. 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 -- 172.21.15.57:0/1490530383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 msgr2=0x7f0e680ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 --2- 172.21.15.57:0/1490530383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e680ff790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009910 tx=0x7f0e5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 -- 172.21.15.57:0/1490530383 shutdown_connections 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 --2- 172.21.15.57:0/1490530383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 0x7f0e68103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 --2- 172.21.15.57:0/1490530383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e680ff790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 -- 172.21.15.57:0/1490530383 >> 172.21.15.57:0/1490530383 conn(0x7f0e680fac80 msgr2=0x7f0e680fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 -- 172.21.15.57:0/1490530383 shutdown_connections 2024-02-28T23:18:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.326+0000 7f0e70600700 1 -- 172.21.15.57:0/1490530383 wait complete. 2024-02-28T23:18:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.327+0000 7f0e70600700 1 Processor -- start 2024-02-28T23:18:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e70600700 1 -- start start 2024-02-28T23:18:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e70600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e70600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 0x7f0e68196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e70600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68196b50 con 0x7f0e680ff3c0 2024-02-28T23:18:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e70600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6819a8e0 con 0x7f0e680ffd60 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e6dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e6dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58950/0 (socket says 172.21.15.57:58950) 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.328+0000 7f0e6dc00700 1 -- 172.21.15.57:0/4244593200 learned_addr learned my addr 172.21.15.57:0/4244593200 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.329+0000 7f0e6d200700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 0x7f0e68196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.329+0000 7f0e6dc00700 1 -- 172.21.15.57:0/4244593200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 msgr2=0x7f0e68196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.329+0000 7f0e6dc00700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 0x7f0e68196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.329+0000 7f0e6dc00700 1 -- 172.21.15.57:0/4244593200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0095f0 con 0x7f0e680ff3c0 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.329+0000 7f0e6dc00700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005b40 tx=0x7f0e5c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.330+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c01d070 con 0x7f0e680ff3c0 2024-02-28T23:18:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.330+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e5c00bed0 con 0x7f0e680ff3c0 2024-02-28T23:18:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.330+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6819ab60 con 0x7f0e680ff3c0 2024-02-28T23:18:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.330+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c022470 con 0x7f0e680ff3c0 2024-02-28T23:18:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.330+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6819b050 con 0x7f0e680ff3c0 2024-02-28T23:18:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.331+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6804f2a0 con 0x7f0e680ff3c0 2024-02-28T23:18:02.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.332+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0e5c0050d0 con 0x7f0e680ff3c0 2024-02-28T23:18:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.333+0000 7f0e66000700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 0x7f0e5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.333+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0e5c08e970 con 0x7f0e680ff3c0 2024-02-28T23:18:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.333+0000 7f0e6d200700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 0x7f0e5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.334+0000 7f0e6d200700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 0x7f0e5006fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e68197550 tx=0x7f0e54009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.338+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e5c058130 con 0x7f0e680ff3c0 2024-02-28T23:18:02.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.664+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0e6819b270 con 0x7f0e680ff3c0 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.665+0000 7f0e66000700 1 -- 172.21.15.57:0/4244593200 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7f0e5c05b7f0 con 0x7f0e680ff3c0 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 5 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 9 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:18:02.666 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:18:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.668+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 msgr2=0x7f0e5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.668+0000 7f0e70600700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 0x7f0e5006fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e68197550 tx=0x7f0e54009260 comp rx=0 tx=0).stop 2024-02-28T23:18:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.668+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 msgr2=0x7f0e68195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.668+0000 7f0e70600700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005b40 tx=0x7f0e5c00f740 comp rx=0 tx=0).stop 2024-02-28T23:18:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 shutdown_connections 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e5006d8a0 0x7f0e5006fd50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e680ff3c0 0x7f0e68195f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 --2- 172.21.15.57:0/4244593200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e680ffd60 0x7f0e68196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 >> 172.21.15.57:0/4244593200 conn(0x7f0e680fac80 msgr2=0x7f0e68103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 shutdown_connections 2024-02-28T23:18:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.669+0000 7f0e70600700 1 -- 172.21.15.57:0/4244593200 wait complete. 2024-02-28T23:18:02.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:02 smithi057 ceph-mon[89491]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:02 smithi057 ceph-mon[89491]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:02 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:02 smithi084 ceph-mon[70440]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:02 smithi084 ceph-mon[70440]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:02 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:02.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 -- 172.21.15.57:0/3067943207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0140102e60 msgr2=0x7f0140106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 --2- 172.21.15.57:0/3067943207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0140102e60 0x7f0140106e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0130009910 tx=0x7f0130009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 -- 172.21.15.57:0/3067943207 shutdown_connections 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 --2- 172.21.15.57:0/3067943207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0140102e60 0x7f0140106e30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 --2- 172.21.15.57:0/3067943207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01401024c0 0x7f0140102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 -- 172.21.15.57:0/3067943207 >> 172.21.15.57:0/3067943207 conn(0x7f01400fdd20 msgr2=0x7f0140100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 -- 172.21.15.57:0/3067943207 shutdown_connections 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.833+0000 7f0146600700 1 -- 172.21.15.57:0/3067943207 wait complete. 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.834+0000 7f0146600700 1 Processor -- start 2024-02-28T23:18:02.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.834+0000 7f0146600700 1 -- start start 2024-02-28T23:18:02.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0146600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0146600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140102e60 0x7f0140198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0146600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01401989e0 con 0x7f01401024c0 2024-02-28T23:18:02.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0146600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0140198b20 con 0x7f0140102e60 2024-02-28T23:18:02.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58966/0 (socket says 172.21.15.57:58966) 2024-02-28T23:18:02.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 -- 172.21.15.57:0/1863299809 learned_addr learned my addr 172.21.15.57:0/1863299809 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f013fe00700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140102e60 0x7f0140198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 -- 172.21.15.57:0/1863299809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140102e60 msgr2=0x7f0140198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140102e60 0x7f0140198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.835+0000 7f0145200700 1 -- 172.21.15.57:0/1863299809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300095f0 con 0x7f01401024c0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f0145200700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f013400c930 tx=0x7f013400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0134007ad0 con 0x7f01401024c0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0134007c30 con 0x7f01401024c0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0134018610 con 0x7f01401024c0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014019c970 con 0x7f01401024c0 2024-02-28T23:18:02.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.836+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014019cec0 con 0x7f01401024c0 2024-02-28T23:18:02.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.837+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014010a830 con 0x7f01401024c0 2024-02-28T23:18:02.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.838+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f013400f450 con 0x7f01401024c0 2024-02-28T23:18:02.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.839+0000 7f013d600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 0x7f012806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:02.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.840+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f013408d5d0 con 0x7f01401024c0 2024-02-28T23:18:02.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.840+0000 7f013fe00700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 0x7f012806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:02.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.841+0000 7f013fe00700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 0x7f012806fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0140199470 tx=0x7f013001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:02.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:02.844+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0134056eb0 con 0x7f01401024c0 2024-02-28T23:18:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.131+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f014010a210 con 0x7f01401024c0 2024-02-28T23:18:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.132+0000 7f013d600700 1 -- 172.21.15.57:0/1863299809 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f013405a570 con 0x7f01401024c0 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:e21 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 21 2024-02-28T23:18:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:11:42.560463+0000 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 47 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:18:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=14480,1=24281} 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.57:6828/2848794823,v1:172.21.15.57:6829/2848794823] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{0:24291} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.84:6826/2979507340,v1:172.21.15.84:6827/2979507340] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{1:24281} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.84:6824/2141462155,v1:172.21.15.84:6825/2141462155] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{1:24299} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/3659715660,v1:172.21.15.57:6827/3659715660] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 msgr2=0x7f012806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 0x7f012806fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0140199470 tx=0x7f013001a040 comp rx=0 tx=0).stop 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 msgr2=0x7f0140197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f013400c930 tx=0x7f013400ccf0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 shutdown_connections 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f012806d8a0 0x7f012806fd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01401024c0 0x7f0140197e50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 --2- 172.21.15.57:0/1863299809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140102e60 0x7f0140198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 >> 172.21.15.57:0/1863299809 conn(0x7f01400fdd20 msgr2=0x7f01401000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 shutdown_connections 2024-02-28T23:18:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.136+0000 7f0146600700 1 -- 172.21.15.57:0/1863299809 wait complete. 2024-02-28T23:18:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:18:03.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 -- 172.21.15.57:0/547833570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 msgr2=0x7f2a2c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 --2- 172.21.15.57:0/547833570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c073480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009940 tx=0x7f2a1c009c50 comp rx=0 tx=0).stop 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 -- 172.21.15.57:0/547833570 shutdown_connections 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 --2- 172.21.15.57:0/547833570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 0x7f2a2c10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 --2- 172.21.15.57:0/547833570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c073480 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 -- 172.21.15.57:0/547833570 >> 172.21.15.57:0/547833570 conn(0x7f2a2c0fbda0 msgr2=0x7f2a2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 -- 172.21.15.57:0/547833570 shutdown_connections 2024-02-28T23:18:03.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.295+0000 7f2a32a00700 1 -- 172.21.15.57:0/547833570 wait complete. 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.296+0000 7f2a32a00700 1 Processor -- start 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.296+0000 7f2a32a00700 1 -- start start 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.296+0000 7f2a32a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.296+0000 7f2a32a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 0x7f2a2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a32a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c198e70 con 0x7f2a2c0730b0 2024-02-28T23:18:03.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a32a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19cc00 con 0x7f2a2c0739c0 2024-02-28T23:18:03.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58970/0 (socket says 172.21.15.57:58970) 2024-02-28T23:18:03.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 -- 172.21.15.57:0/3844224688 learned_addr learned my addr 172.21.15.57:0/3844224688 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:03.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2b400700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 0x7f2a2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 -- 172.21.15.57:0/3844224688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 msgr2=0x7f2a2c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 0x7f2a2c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.297+0000 7f2a2be00700 1 -- 172.21.15.57:0/3844224688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0095f0 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a2be00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00bb10 tx=0x7f2a1c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c01d070 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a1c003d10 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c19ce80 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c021410 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.298+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c19d370 con 0x7f2a2c0730b0 2024-02-28T23:18:03.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.299+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c109cc0 con 0x7f2a2c0730b0 2024-02-28T23:18:03.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.300+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a1c00f460 con 0x7f2a2c0730b0 2024-02-28T23:18:03.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.301+0000 7f2a28c00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.301+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2a1c08e5b0 con 0x7f2a2c0730b0 2024-02-28T23:18:03.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.301+0000 7f2a2b400700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.302+0000 7f2a2b400700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c199870 tx=0x7f2a18009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:03.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.306+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a1c057d70 con 0x7f2a2c0730b0 2024-02-28T23:18:03.329 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:18:02 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:18:02.893+0000 7f78389b67c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-28T23:18:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.559+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a2c1996c0 con 0x7f2a1006d8a0 2024-02-28T23:18:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.561+0000 7f2a28c00700 1 -- 172.21.15.57:0/3844224688 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a2c1996c0 con 0x7f2a1006d8a0 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:18:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "12/23 daemons upgraded", 2024-02-28T23:18:03.562 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading osd daemons", 2024-02-28T23:18:03.562 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:18:03.562 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:18:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 msgr2=0x7f2a1006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c199870 tx=0x7f2a18009260 comp rx=0 tx=0).stop 2024-02-28T23:18:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 msgr2=0x7f2a2c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00bb10 tx=0x7f2a1c00bb40 comp rx=0 tx=0).stop 2024-02-28T23:18:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 shutdown_connections 2024-02-28T23:18:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a1006d8a0 0x7f2a1006fd50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0730b0 0x7f2a2c198250 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 --2- 172.21.15.57:0/3844224688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a2c0739c0 0x7f2a2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.564+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 >> 172.21.15.57:0/3844224688 conn(0x7f2a2c0fbda0 msgr2=0x7f2a2c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:03.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.565+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 shutdown_connections 2024-02-28T23:18:03.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.565+0000 7f2a32a00700 1 -- 172.21.15.57:0/3844224688 wait complete. 2024-02-28T23:18:03.677 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:18:03.405+0000 7f78389b67c0 -1 osd.5 75 log_to_monitors true 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='client.44263 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4244593200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 51/321 objects degraded (15.888%), 14 pgs degraded (PG_DEGRADED) 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1863299809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:18:03.678 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:03 smithi084 ceph-mon[70440]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 -- 172.21.15.57:0/688588093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 msgr2=0x7f66dc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 --2- 172.21.15.57:0/688588093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc0689b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009870 tx=0x7f66c4009b80 comp rx=0 tx=0).stop 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 -- 172.21.15.57:0/688588093 shutdown_connections 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 --2- 172.21.15.57:0/688588093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc0689b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 --2- 172.21.15.57:0/688588093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 0x7f66dc108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 -- 172.21.15.57:0/688588093 >> 172.21.15.57:0/688588093 conn(0x7f66dc075130 msgr2=0x7f66dc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 -- 172.21.15.57:0/688588093 shutdown_connections 2024-02-28T23:18:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.729+0000 7f66e2c00700 1 -- 172.21.15.57:0/688588093 wait complete. 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/321 objects degraded (15.888%) 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='client.44263 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4244593200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 51/321 objects degraded (15.888%), 14 pgs degraded (PG_DEGRADED) 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1863299809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:18:03.730 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:03 smithi057 ceph-mon[89491]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.730+0000 7f66e2c00700 1 Processor -- start 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.730+0000 7f66e2c00700 1 -- start start 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66e2c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66e2c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 0x7f66dc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66e2c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc198cd0 con 0x7f66dc068540 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66e2c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc19ca60 con 0x7f66dc1085b0 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58996/0 (socket says 172.21.15.57:58996) 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66dbe00700 1 -- 172.21.15.57:0/676189521 learned_addr learned my addr 172.21.15.57:0/676189521 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.731+0000 7f66d3e00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 0x7f66dc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66dbe00700 1 -- 172.21.15.57:0/676189521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 msgr2=0x7f66dc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66dbe00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 0x7f66dc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66dbe00700 1 -- 172.21.15.57:0/676189521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009520 con 0x7f66dc068540 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66dbe00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e970 tx=0x7f66cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc00c9d0 con 0x7f66dc068540 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66cc004020 con 0x7f66dc068540 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.732+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc003d30 con 0x7f66dc068540 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.733+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc19cd40 con 0x7f66dc068540 2024-02-28T23:18:03.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.733+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc19d290 con 0x7f66dc068540 2024-02-28T23:18:03.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.734+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc10aa00 con 0x7f66dc068540 2024-02-28T23:18:03.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.735+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f66cc004190 con 0x7f66dc068540 2024-02-28T23:18:03.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.736+0000 7f66d9600700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 0x7f66c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:03.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.736+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f66cc014070 con 0x7f66dc068540 2024-02-28T23:18:03.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.736+0000 7f66d3e00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 0x7f66c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:03.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.737+0000 7f66d3e00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 0x7f66c006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009f50 tx=0x7f66c4019040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:03.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:03.741+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66cc056fb0 con 0x7f66dc068540 2024-02-28T23:18:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.067+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66dc04f2a0 con 0x7f66dc068540 2024-02-28T23:18:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.068+0000 7f66d9600700 1 -- 172.21.15.57:0/676189521 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1290 (secure 0 0 0) 0x7f66cc05a670 con 0x7f66dc068540 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 51/321 objects degraded (15.888%), 14 pgs degraded 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: osd.5 (root=default,host=smithi084) is down 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/321 objects degraded (15.888%), 14 pgs degraded 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-02-28T23:18:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-02-28T23:18:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 msgr2=0x7f66c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 0x7f66c006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009f50 tx=0x7f66c4019040 comp rx=0 tx=0).stop 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 msgr2=0x7f66dc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e970 tx=0x7f66cc00ed30 comp rx=0 tx=0).stop 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 shutdown_connections 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f66c006d8a0 0x7f66c006fd50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66dc068540 0x7f66dc1980b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 --2- 172.21.15.57:0/676189521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66dc1085b0 0x7f66dc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 >> 172.21.15.57:0/676189521 conn(0x7f66dc075130 msgr2=0x7f66dc106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 shutdown_connections 2024-02-28T23:18:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:04.071+0000 7f66e2c00700 1 -- 172.21.15.57:0/676189521 wait complete. 2024-02-28T23:18:04.330 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:18:04.114+0000 7f7827e00700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-28T23:18:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:05.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/676189521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:18:05.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-28T23:18:05.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:18:05.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: osdmap e78: 6 total, 5 up, 6 in 2024-02-28T23:18:05.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:04 smithi057 ceph-mon[89491]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:18:05.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:05.071 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:05.072 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:05.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/676189521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:18:05.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-28T23:18:05.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:18:05.073 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: osdmap e78: 6 total, 5 up, 6 in 2024-02-28T23:18:05.074 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:04 smithi084 ceph-mon[70440]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-02-28T23:18:06.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:05 smithi057 ceph-mon[89491]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 51/321 objects degraded (15.888%) 2024-02-28T23:18:06.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:05 smithi057 ceph-mon[89491]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:18:06.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:05 smithi057 ceph-mon[89491]: osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677] boot 2024-02-28T23:18:06.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:05 smithi057 ceph-mon[89491]: osdmap e79: 6 total, 6 up, 6 in 2024-02-28T23:18:06.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:05 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:18:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:05 smithi084 ceph-mon[70440]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 51/321 objects degraded (15.888%) 2024-02-28T23:18:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:05 smithi084 ceph-mon[70440]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-28T23:18:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:05 smithi084 ceph-mon[70440]: osd.5 [v2:172.21.15.84:6816/2020554677,v1:172.21.15.84:6817/2020554677] boot 2024-02-28T23:18:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:05 smithi084 ceph-mon[70440]: osdmap e79: 6 total, 6 up, 6 in 2024-02-28T23:18:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:05 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-28T23:18:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:07 smithi057 ceph-mon[89491]: osdmap e80: 6 total, 6 up, 6 in 2024-02-28T23:18:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:07 smithi057 ceph-mon[89491]: pgmap v192: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 51/321 objects degraded (15.888%) 2024-02-28T23:18:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:07 smithi084 ceph-mon[70440]: osdmap e80: 6 total, 6 up, 6 in 2024-02-28T23:18:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:07 smithi084 ceph-mon[70440]: pgmap v192: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 51/321 objects degraded (15.888%) 2024-02-28T23:18:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:08 smithi057 ceph-mon[89491]: Health check update: Degraded data redundancy: 41/321 objects degraded (12.773%), 10 pgs degraded (PG_DEGRADED) 2024-02-28T23:18:08.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:08 smithi084 ceph-mon[70440]: Health check update: Degraded data redundancy: 41/321 objects degraded (12.773%), 10 pgs degraded (PG_DEGRADED) 2024-02-28T23:18:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:09 smithi084 ceph-mon[70440]: pgmap v193: 65 pgs: 10 active+undersized+degraded, 11 active+undersized, 44 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 41/321 objects degraded (12.773%) 2024-02-28T23:18:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:09 smithi057 ceph-mon[89491]: pgmap v193: 65 pgs: 10 active+undersized+degraded, 11 active+undersized, 44 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 41/321 objects degraded (12.773%) 2024-02-28T23:18:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all osd 2024-02-28T23:18:10.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: Upgrade: Setting require_osd_release to 19 squid 2024-02-28T23:18:10.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all osd 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-28T23:18:10.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-28T23:18:10.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-28T23:18:10.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: Upgrade: Setting require_osd_release to 19 squid 2024-02-28T23:18:10.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/321 objects degraded (12.773%), 10 pgs degraded) 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: osdmap e81: 6 total, 6 up, 6 in 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/321 objects degraded (12.773%), 10 pgs degraded) 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: osdmap e81: 6 total, 6 up, 6 in 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-28T23:18:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-28T23:18:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:12 smithi057 ceph-mon[89491]: osdmap e82: 6 total, 6 up, 6 in 2024-02-28T23:18:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:12 smithi057 ceph-mon[89491]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-28T23:18:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:12 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-28T23:18:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:12 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2024-02-28T23:18:12.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:12 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:12 smithi084 ceph-mon[70440]: osdmap e82: 6 total, 6 up, 6 in 2024-02-28T23:18:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:12 smithi084 ceph-mon[70440]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-28T23:18:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:12 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-28T23:18:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:12 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2024-02-28T23:18:12.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:12 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:18:13.071+0000 7f864a200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:18:13.969 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-02-28T23:18:13.969 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:13.969 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] up:boot 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/3579754700,v1:172.21.15.57:6827/3579754700] up:boot 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:13.970 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: osdmap e83: 6 total, 6 up, 6 in 2024-02-28T23:18:13.971 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:13 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-28T23:18:14.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] up:boot 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/3579754700,v1:172.21.15.57:6827/3579754700] up:boot 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.sbkaml=up:active,1=cephfs.smithi084.ztfxpc=up:active} 2 up:standby 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-28T23:18:14.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-28T23:18:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: osdmap e83: 6 total, 6 up, 6 in 2024-02-28T23:18:14.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:13 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.84:6824/3824521120,v1:172.21.15.84:6825/3824521120] up:boot 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/3389712306,v1:172.21.15.57:6829/3389712306] up:boot 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:18:14.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:14 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.84:6824/3824521120,v1:172.21.15.84:6825/3824521120] up:boot 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/3389712306,v1:172.21.15.57:6829/3389712306] up:boot 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:18:15.001 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:14 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:18:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:15 smithi057 ceph-mon[89491]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-28T23:18:15.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:15 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:15.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:15 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:18:15.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:15 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:15 smithi084 ceph-mon[70440]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-28T23:18:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:15 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:15 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.bgmggd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:18:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:15 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:16.768 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:16 smithi057 ceph-mon[89491]: Upgrade: Updating mds.cephfs.smithi057.bgmggd 2024-02-28T23:18:16.769 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:16 smithi057 ceph-mon[89491]: Deploying daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:18:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:16 smithi084 ceph-mon[70440]: Upgrade: Updating mds.cephfs.smithi057.bgmggd 2024-02-28T23:18:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:16 smithi084 ceph-mon[70440]: Deploying daemon mds.cephfs.smithi057.bgmggd on smithi057 2024-02-28T23:18:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:17 smithi057 ceph-mon[89491]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-02-28T23:18:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:17 smithi084 ceph-mon[70440]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-02-28T23:18:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:19 smithi057 ceph-mon[89491]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 699 B/s rd, 1 op/s 2024-02-28T23:18:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:19 smithi084 ceph-mon[70440]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 699 B/s rd, 1 op/s 2024-02-28T23:18:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:20.271 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:20 smithi057 ceph-mon[89491]: osdmap e84: 6 total, 6 up, 6 in 2024-02-28T23:18:20.272 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:20 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:18:20.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:20 smithi084 ceph-mon[70440]: osdmap e84: 6 total, 6 up, 6 in 2024-02-28T23:18:20.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:20 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:18:21.517 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:21 smithi057 ceph-mon[89491]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-02-28T23:18:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:21 smithi084 ceph-mon[70440]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 566 B/s rd, 0 op/s 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:boot 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:23.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:23 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 566 B/s rd, 0 op/s 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:boot 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:23 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:18:25.800 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:25 smithi057 ceph-mon[89491]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-02-28T23:18:25.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:25 smithi084 ceph-mon[70440]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-02-28T23:18:27.721 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:27 smithi057 ceph-mon[89491]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-28T23:18:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:27 smithi084 ceph-mon[70440]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-28T23:18:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:29 smithi057 ceph-mon[89491]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:29 smithi084 ceph-mon[70440]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:30.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:30.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:31 smithi057 ceph-mon[89491]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:31 smithi084 ceph-mon[70440]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:33.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:32 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:33.024 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:32 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:32 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:32 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:33 smithi057 ceph-mon[89491]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:33 smithi084 ceph-mon[70440]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 -- 172.21.15.57:0/2173544156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8068540 msgr2=0x7f70b80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 --2- 172.21.15.57:0/2173544156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8068540 0x7f70b80689b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009870 tx=0x7f70a4009b80 comp rx=0 tx=0).stop 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 -- 172.21.15.57:0/2173544156 shutdown_connections 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 --2- 172.21.15.57:0/2173544156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8068540 0x7f70b80689b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 --2- 172.21.15.57:0/2173544156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8101000 0x7f70b81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 -- 172.21.15.57:0/2173544156 >> 172.21.15.57:0/2173544156 conn(0x7f70b8075130 msgr2=0x7f70b8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 -- 172.21.15.57:0/2173544156 shutdown_connections 2024-02-28T23:18:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.255+0000 7f70bde00700 1 -- 172.21.15.57:0/2173544156 wait complete. 2024-02-28T23:18:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.256+0000 7f70bde00700 1 Processor -- start 2024-02-28T23:18:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70bde00700 1 -- start start 2024-02-28T23:18:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70bde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70bde00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8101000 0x7f70b8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70bde00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8198c60 con 0x7f70b8068540 2024-02-28T23:18:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70bde00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b819c9f0 con 0x7f70b8101000 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.257+0000 7f70b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33806/0 (socket says 172.21.15.57:33806) 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 -- 172.21.15.57:0/687088463 learned_addr learned my addr 172.21.15.57:0/687088463 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6000700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8101000 0x7f70b8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 -- 172.21.15.57:0/687088463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8101000 msgr2=0x7f70b8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8101000 0x7f70b8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 -- 172.21.15.57:0/687088463 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4009520 con 0x7f70b8068540 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.258+0000 7f70b6a00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70a800e940 tx=0x7f70a800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.259+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a800eee0 con 0x7f70b8068540 2024-02-28T23:18:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.259+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b819ccd0 con 0x7f70b8068540 2024-02-28T23:18:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.259+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70a8004cf0 con 0x7f70b8068540 2024-02-28T23:18:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.259+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a8005650 con 0x7f70b8068540 2024-02-28T23:18:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.259+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b819d220 con 0x7f70b8068540 2024-02-28T23:18:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.260+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b804f2a0 con 0x7f70b8068540 2024-02-28T23:18:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.262+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f70a8004750 con 0x7f70b8068540 2024-02-28T23:18:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.264+0000 7f70bca00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 0x7f709c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.264+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f70a8014070 con 0x7f70b8068540 2024-02-28T23:18:34.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.264+0000 7f70b6000700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 0x7f709c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.265+0000 7f70b6000700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 0x7f709c06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009f50 tx=0x7f70a4019040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.269+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70a8057300 con 0x7f70b8068540 2024-02-28T23:18:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.527+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70b819d650 con 0x7f709c06d8a0 2024-02-28T23:18:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.530+0000 7f70bca00700 1 -- 172.21.15.57:0/687088463 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f70b819d650 con 0x7f709c06d8a0 2024-02-28T23:18:34.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 msgr2=0x7f709c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 0x7f709c06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009f50 tx=0x7f70a4019040 comp rx=0 tx=0).stop 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 msgr2=0x7f70b8198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70a800e940 tx=0x7f70a800ed00 comp rx=0 tx=0).stop 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 shutdown_connections 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f709c06d8a0 0x7f709c06fd50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70b8068540 0x7f70b8198040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 --2- 172.21.15.57:0/687088463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b8101000 0x7f70b8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 >> 172.21.15.57:0/687088463 conn(0x7f70b8075130 msgr2=0x7f70b80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 shutdown_connections 2024-02-28T23:18:34.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.533+0000 7f70bde00700 1 -- 172.21.15.57:0/687088463 wait complete. 2024-02-28T23:18:34.548 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 -- 172.21.15.57:0/1826001505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 msgr2=0x7f0d880689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/1826001505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d880689b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009910 tx=0x7f0d74009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 -- 172.21.15.57:0/1826001505 shutdown_connections 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/1826001505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d880689b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/1826001505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 0x7f0d881066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.699+0000 7f0d8ea00700 1 -- 172.21.15.57:0/1826001505 >> 172.21.15.57:0/1826001505 conn(0x7f0d88075130 msgr2=0x7f0d88075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.700+0000 7f0d8ea00700 1 -- 172.21.15.57:0/1826001505 shutdown_connections 2024-02-28T23:18:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.700+0000 7f0d8ea00700 1 -- 172.21.15.57:0/1826001505 wait complete. 2024-02-28T23:18:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.701+0000 7f0d8ea00700 1 Processor -- start 2024-02-28T23:18:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.701+0000 7f0d8ea00700 1 -- start start 2024-02-28T23:18:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d8ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d8ea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 0x7f0d88198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d8ea00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88198c40 con 0x7f0d88068540 2024-02-28T23:18:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d8ea00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819c9d0 con 0x7f0d88106320 2024-02-28T23:18:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33826/0 (socket says 172.21.15.57:33826) 2024-02-28T23:18:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.702+0000 7f0d87e00700 1 -- 172.21.15.57:0/4156811233 learned_addr learned my addr 172.21.15.57:0/4156811233 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d87400700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 0x7f0d88198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d87e00700 1 -- 172.21.15.57:0/4156811233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 msgr2=0x7f0d88198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d87e00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 0x7f0d88198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d87e00700 1 -- 172.21.15.57:0/4156811233 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740095f0 con 0x7f0d88068540 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d87e00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e920 tx=0x7f0d7800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d78009790 con 0x7f0d88068540 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d7800cba0 con 0x7f0d88068540 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8819ccb0 con 0x7f0d88068540 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.703+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d78018650 con 0x7f0d88068540 2024-02-28T23:18:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.704+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8819d200 con 0x7f0d88068540 2024-02-28T23:18:34.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.705+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8804f2a0 con 0x7f0d88068540 2024-02-28T23:18:34.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.706+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0d7800cd10 con 0x7f0d88068540 2024-02-28T23:18:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.707+0000 7f0d84c00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 0x7f0d6c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.707+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d78014070 con 0x7f0d88068540 2024-02-28T23:18:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.707+0000 7f0d87400700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 0x7f0d6c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.708+0000 7f0d87400700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 0x7f0d6c06fd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d74006010 tx=0x7f0d74009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.712+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d78052d20 con 0x7f0d88068540 2024-02-28T23:18:34.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.970+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d8819d530 con 0x7f0d6c06d8a0 2024-02-28T23:18:34.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.972+0000 7f0d84c00700 1 -- 172.21.15.57:0/4156811233 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0d8819d530 con 0x7f0d6c06d8a0 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 msgr2=0x7f0d6c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 0x7f0d6c06fd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d74006010 tx=0x7f0d74009f90 comp rx=0 tx=0).stop 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 msgr2=0x7f0d88198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e920 tx=0x7f0d7800ece0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 shutdown_connections 2024-02-28T23:18:34.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d6c06d8a0 0x7f0d6c06fd50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d88068540 0x7f0d88198020 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 --2- 172.21.15.57:0/4156811233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d88106320 0x7f0d88198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:34.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.975+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 >> 172.21.15.57:0/4156811233 conn(0x7f0d88075130 msgr2=0x7f0d880ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:34.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.976+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 shutdown_connections 2024-02-28T23:18:34.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:34.976+0000 7f0d8ea00700 1 -- 172.21.15.57:0/4156811233 wait complete. 2024-02-28T23:18:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 -- 172.21.15.57:0/3427889932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 msgr2=0x7f4a8c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 --2- 172.21.15.57:0/3427889932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c104e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009910 tx=0x7f4a78009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 -- 172.21.15.57:0/3427889932 shutdown_connections 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 --2- 172.21.15.57:0/3427889932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c104e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 --2- 172.21.15.57:0/3427889932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.160+0000 7f4a92c00700 1 -- 172.21.15.57:0/3427889932 >> 172.21.15.57:0/3427889932 conn(0x7f4a8c0fbda0 msgr2=0x7f4a8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.161+0000 7f4a92c00700 1 -- 172.21.15.57:0/3427889932 shutdown_connections 2024-02-28T23:18:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.161+0000 7f4a92c00700 1 -- 172.21.15.57:0/3427889932 wait complete. 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 Processor -- start 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 -- start start 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c073ad0 con 0x7f4a8c100e80 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.162+0000 7f4a92c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c073c10 con 0x7f4a8c1004e0 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50678/0 (socket says 172.21.15.57:50678) 2024-02-28T23:18:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 -- 172.21.15.57:0/919735863 learned_addr learned my addr 172.21.15.57:0/919735863 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8be00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 -- 172.21.15.57:0/919735863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 msgr2=0x7f4a8c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8b400700 1 -- 172.21.15.57:0/919735863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780095f0 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.163+0000 7f4a8be00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a8b400700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800bda0 tx=0x7f4a7800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a7801d070 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a7800f460 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a78017630 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c073e90 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.164+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c1a2430 con 0x7f4a8c100e80 2024-02-28T23:18:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.165+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c108850 con 0x7f4a8c100e80 2024-02-28T23:18:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.166+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a7800fa40 con 0x7f4a8c100e80 2024-02-28T23:18:35.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.167+0000 7f4a88c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 0x7f4a7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.168+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a7808e900 con 0x7f4a8c100e80 2024-02-28T23:18:35.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.168+0000 7f4a8be00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 0x7f4a7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.168+0000 7f4a8be00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 0x7f4a7006fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005950 tx=0x7f4a7c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.172+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a78058130 con 0x7f4a8c100e80 2024-02-28T23:18:35.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.420+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a8c061190 con 0x7f4a7006d8a0 2024-02-28T23:18:35.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.440+0000 7f4a88c00700 1 -- 172.21.15.57:0/919735863 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4a8c061190 con 0x7f4a7006d8a0 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (8m) 5s ago 22m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (18m) 5s ago 22m 9369k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (20m) 33s ago 20m 9038k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (4m) 5s ago 22m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (4m) 33s ago 20m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (7m) 5s ago 21m 73.5M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (13s) 5s ago 14m 10.8M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (14m) 5s ago 14m 11.3M - 18.2.0 dc2bc1663786 5e7fefec34bc 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (14m) 33s ago 14m 58.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (14m) 33s ago 14m 108M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:18:35.440 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (10m) 5s ago 23m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (9m) 33s ago 20m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (5m) 5s ago 24m 54.3M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (5m) 33s ago 20m 39.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (9m) 5s ago 22m 7884k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (9m) 33s ago 20m 8174k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (3m) 5s ago 19m 92.3M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (3m) 5s ago 18m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (2m) 5s ago 18m 82.8M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (2m) 33s ago 17m 105M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (80s) 33s ago 17m 99.0M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (38s) 33s ago 16m 11.1M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:18:35.441 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (8m) 5s ago 21m 69.5M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.443+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 msgr2=0x7f4a7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.443+0000 7f4a92c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 0x7f4a7006fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005950 tx=0x7f4a7c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.443+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 msgr2=0x7f4a8c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.443+0000 7f4a92c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800bda0 tx=0x7f4a7800be80 comp rx=0 tx=0).stop 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 shutdown_connections 2024-02-28T23:18:35.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a7006d8a0 0x7f4a7006fd50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c1004e0 0x7f4a8c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 --2- 172.21.15.57:0/919735863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a8c100e80 0x7f4a8c073590 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 >> 172.21.15.57:0/919735863 conn(0x7f4a8c0fbda0 msgr2=0x7f4a8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 shutdown_connections 2024-02-28T23:18:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.444+0000 7f4a92c00700 1 -- 172.21.15.57:0/919735863 wait complete. 2024-02-28T23:18:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 -- 172.21.15.57:0/1515728559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101420 msgr2=0x7ff0901017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 --2- 172.21.15.57:0/1515728559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101420 0x7ff0901017f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff084009910 tx=0x7ff084009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 -- 172.21.15.57:0/1515728559 shutdown_connections 2024-02-28T23:18:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 --2- 172.21.15.57:0/1515728559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101d30 0x7ff09010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 --2- 172.21.15.57:0/1515728559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101420 0x7ff0901017f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.610+0000 7ff097400700 1 -- 172.21.15.57:0/1515728559 >> 172.21.15.57:0/1515728559 conn(0x7ff0900fac80 msgr2=0x7ff0900fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:35.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.611+0000 7ff097400700 1 -- 172.21.15.57:0/1515728559 shutdown_connections 2024-02-28T23:18:35.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.611+0000 7ff097400700 1 -- 172.21.15.57:0/1515728559 wait complete. 2024-02-28T23:18:35.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 Processor -- start 2024-02-28T23:18:35.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 -- start start 2024-02-28T23:18:35.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101420 0x7ff090198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090198bf0 con 0x7ff090101d30 2024-02-28T23:18:35.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.612+0000 7ff097400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090198d30 con 0x7ff090101420 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50704/0 (socket says 172.21.15.57:50704) 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 -- 172.21.15.57:0/2103725820 learned_addr learned my addr 172.21.15.57:0/2103725820 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff094a00700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101420 0x7ff090198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 -- 172.21.15.57:0/2103725820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101420 msgr2=0x7ff090198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101420 0x7ff090198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:35.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.613+0000 7ff08fe00700 1 -- 172.21.15.57:0/2103725820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840095f0 con 0x7ff090101d30 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff08fe00700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c930 tx=0x7ff07c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff07c007ad0 con 0x7ff090101d30 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff07c007c30 con 0x7ff090101d30 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff07c018610 con 0x7ff090101d30 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09019cb80 con 0x7ff090101d30 2024-02-28T23:18:35.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.614+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09019d030 con 0x7ff090101d30 2024-02-28T23:18:35.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.616+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff090109d60 con 0x7ff090101d30 2024-02-28T23:18:35.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.617+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff07c00f450 con 0x7ff090101d30 2024-02-28T23:18:35.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.618+0000 7ff08d600700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 0x7ff07806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:35.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.618+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff07c08d700 con 0x7ff090101d30 2024-02-28T23:18:35.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.618+0000 7ff094a00700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 0x7ff07806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:35.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.619+0000 7ff094a00700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 0x7ff07806fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff084006010 tx=0x7ff08401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:35.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:35.623+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff07c056eb0 con 0x7ff090101d30 2024-02-28T23:18:36.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.003+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff09004f2a0 con 0x7ff090101d30 2024-02-28T23:18:36.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.004+0000 7ff08d600700 1 -- 172.21.15.57:0/2103725820 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff07c05a570 con 0x7ff090101d30 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:18:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 1 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 11 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:18:36.004 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:18:36.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.007+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 msgr2=0x7ff07806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.007+0000 7ff097400700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 0x7ff07806fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff084006010 tx=0x7ff08401a040 comp rx=0 tx=0).stop 2024-02-28T23:18:36.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.007+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 msgr2=0x7ff0901985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.007+0000 7ff097400700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c930 tx=0x7ff07c00ccf0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 shutdown_connections 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff07806d8a0 0x7ff07806fd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff090101420 0x7ff090198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 --2- 172.21.15.57:0/2103725820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff090101d30 0x7ff0901985a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 >> 172.21.15.57:0/2103725820 conn(0x7ff0900fac80 msgr2=0x7ff0900ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 shutdown_connections 2024-02-28T23:18:36.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.008+0000 7ff097400700 1 -- 172.21.15.57:0/2103725820 wait complete. 2024-02-28T23:18:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:35 smithi057 ceph-mon[89491]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:35 smithi057 ceph-mon[89491]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:35 smithi084 ceph-mon[70440]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:35 smithi084 ceph-mon[70440]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 -- 172.21.15.57:0/544966619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5808100e80 msgr2=0x7f5808104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 --2- 172.21.15.57:0/544966619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5808100e80 0x7f5808104e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009910 tx=0x7f57f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 -- 172.21.15.57:0/544966619 shutdown_connections 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 --2- 172.21.15.57:0/544966619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5808100e80 0x7f5808104e50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 --2- 172.21.15.57:0/544966619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58081004e0 0x7f58081008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 -- 172.21.15.57:0/544966619 >> 172.21.15.57:0/544966619 conn(0x7f58080fbda0 msgr2=0x7f58080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 -- 172.21.15.57:0/544966619 shutdown_connections 2024-02-28T23:18:36.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.175+0000 7f580fc00700 1 -- 172.21.15.57:0/544966619 wait complete. 2024-02-28T23:18:36.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 Processor -- start 2024-02-28T23:18:36.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 -- start start 2024-02-28T23:18:36.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5808100e80 0x7f5808194070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808196cb0 con 0x7f58081004e0 2024-02-28T23:18:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.177+0000 7f580fc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081945b0 con 0x7f5808100e80 2024-02-28T23:18:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50734/0 (socket says 172.21.15.57:50734) 2024-02-28T23:18:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 -- 172.21.15.57:0/2055155405 learned_addr learned my addr 172.21.15.57:0/2055155405 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f5807e00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5808100e80 0x7f5808194070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 -- 172.21.15.57:0/2055155405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5808100e80 msgr2=0x7f5808194070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5808100e80 0x7f5808194070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.178+0000 7f580d200700 1 -- 172.21.15.57:0/2055155405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40095f0 con 0x7f58081004e0 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f580d200700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d8d0 tx=0x7f57fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57fc0098a0 con 0x7f58081004e0 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57fc010460 con 0x7f58081004e0 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5808194890 con 0x7f58081004e0 2024-02-28T23:18:36.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57fc00f5f0 con 0x7f58081004e0 2024-02-28T23:18:36.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.179+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58081a6950 con 0x7f58081004e0 2024-02-28T23:18:36.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.181+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58081045d0 con 0x7f58081004e0 2024-02-28T23:18:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.182+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f57fc009a00 con 0x7f58081004e0 2024-02-28T23:18:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.183+0000 7f5805600700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 0x7f57f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.183+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f57fc08d430 con 0x7f58081004e0 2024-02-28T23:18:36.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.184+0000 7f5807e00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 0x7f57f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.184+0000 7f5807e00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 0x7f57f006fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006010 tx=0x7f57f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:36.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.190+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f57fc056be0 con 0x7f58081004e0 2024-02-28T23:18:36.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.476+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f580804f2a0 con 0x7f58081004e0 2024-02-28T23:18:36.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.477+0000 7f5805600700 1 -- 172.21.15.57:0/2055155405 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f57fc05a2a0 con 0x7f58081004e0 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:e27 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:18:36.476 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:18:36.477 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/3389712306,v1:172.21.15.57:6829/3389712306] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:36.478 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44289} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/3824521120,v1:172.21.15.84:6825/3824521120] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 msgr2=0x7f57f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 0x7f57f006fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006010 tx=0x7f57f401a040 comp rx=0 tx=0).stop 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 msgr2=0x7f5808193b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d8d0 tx=0x7f57fc00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 shutdown_connections 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f57f006d8a0 0x7f57f006fd50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58081004e0 0x7f5808193b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 --2- 172.21.15.57:0/2055155405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5808100e80 0x7f5808194070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.481+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 >> 172.21.15.57:0/2055155405 conn(0x7f58080fbda0 msgr2=0x7f58080fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.482+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 shutdown_connections 2024-02-28T23:18:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.482+0000 7f580fc00700 1 -- 172.21.15.57:0/2055155405 wait complete. 2024-02-28T23:18:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 -- 172.21.15.57:0/3032661731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 msgr2=0x7f2e84108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 --2- 172.21.15.57:0/3032661731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e84108840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b0a0 tx=0x7f2e7400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 -- 172.21.15.57:0/3032661731 shutdown_connections 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 --2- 172.21.15.57:0/3032661731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 0x7f2e841028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 --2- 172.21.15.57:0/3032661731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e84108840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 -- 172.21.15.57:0/3032661731 >> 172.21.15.57:0/3032661731 conn(0x7f2e840fdf90 msgr2=0x7f2e841003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 -- 172.21.15.57:0/3032661731 shutdown_connections 2024-02-28T23:18:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.644+0000 7f2e89c00700 1 -- 172.21.15.57:0/3032661731 wait complete. 2024-02-28T23:18:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.645+0000 7f2e89c00700 1 Processor -- start 2024-02-28T23:18:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.645+0000 7f2e89c00700 1 -- start start 2024-02-28T23:18:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e89c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 0x7f2e84198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e89c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e89c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84198e90 con 0x7f2e84108470 2024-02-28T23:18:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e89c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419cc20 con 0x7f2e84102470 2024-02-28T23:18:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e82000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e82000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50738/0 (socket says 172.21.15.57:50738) 2024-02-28T23:18:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.646+0000 7f2e82000700 1 -- 172.21.15.57:0/1776334607 learned_addr learned my addr 172.21.15.57:0/1776334607 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e82a00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 0x7f2e84198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e82000700 1 -- 172.21.15.57:0/1776334607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 msgr2=0x7f2e84198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e82000700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 0x7f2e84198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e82000700 1 -- 172.21.15.57:0/1776334607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74009d00 con 0x7f2e84108470 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e82000700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00d900 tx=0x7f2e6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6c004a00 con 0x7f2e84108470 2024-02-28T23:18:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.647+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e6c0054e0 con 0x7f2e84108470 2024-02-28T23:18:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.648+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8419cf00 con 0x7f2e84108470 2024-02-28T23:18:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.648+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6c009d60 con 0x7f2e84108470 2024-02-28T23:18:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.648+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8419d450 con 0x7f2e84108470 2024-02-28T23:18:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.649+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8410a9e0 con 0x7f2e84108470 2024-02-28T23:18:36.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.650+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2e6c005020 con 0x7f2e84108470 2024-02-28T23:18:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.651+0000 7f2e7b400700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 0x7f2e6806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.651+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2e6c08d680 con 0x7f2e84108470 2024-02-28T23:18:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.651+0000 7f2e82a00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 0x7f2e6806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:36.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.652+0000 7f2e82a00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 0x7f2e6806fd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009480 tx=0x7f2e7400b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:36.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.656+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e6c056eb0 con 0x7f2e84108470 2024-02-28T23:18:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.919+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e841996e0 con 0x7f2e6806d8a0 2024-02-28T23:18:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.921+0000 7f2e7b400700 1 -- 172.21.15.57:0/1776334607 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2e841996e0 con 0x7f2e6806d8a0 2024-02-28T23:18:36.920 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:18:36.920 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:18:36.920 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "13/23 daemons upgraded", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:18:36.921 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 msgr2=0x7f2e6806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 0x7f2e6806fd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009480 tx=0x7f2e7400b7f0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 msgr2=0x7f2e841987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00d900 tx=0x7f2e6c00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 shutdown_connections 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2e6806d8a0 0x7f2e6806fd50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e84102470 0x7f2e84198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 --2- 172.21.15.57:0/1776334607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e84108470 0x7f2e841987b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.924+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 >> 172.21.15.57:0/1776334607 conn(0x7f2e840fdf90 msgr2=0x7f2e840ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.925+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 shutdown_connections 2024-02-28T23:18:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:36.925+0000 7f2e89c00700 1 -- 172.21.15.57:0/1776334607 wait complete. 2024-02-28T23:18:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:36 smithi057 ceph-mon[89491]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:36 smithi057 ceph-mon[89491]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2103725820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2055155405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:18:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:36 smithi084 ceph-mon[70440]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:36 smithi084 ceph-mon[70440]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2103725820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2055155405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 -- 172.21.15.57:0/3063291935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0100e80 msgr2=0x7f6de0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 --2- 172.21.15.57:0/3063291935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0100e80 0x7f6de0104e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009910 tx=0x7f6dc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 -- 172.21.15.57:0/3063291935 shutdown_connections 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 --2- 172.21.15.57:0/3063291935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0100e80 0x7f6de0104e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 --2- 172.21.15.57:0/3063291935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de01004e0 0x7f6de01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 -- 172.21.15.57:0/3063291935 >> 172.21.15.57:0/3063291935 conn(0x7f6de00fbda0 msgr2=0x7f6de00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.084+0000 7f6de5400700 1 -- 172.21.15.57:0/3063291935 shutdown_connections 2024-02-28T23:18:37.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.085+0000 7f6de5400700 1 -- 172.21.15.57:0/3063291935 wait complete. 2024-02-28T23:18:37.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.085+0000 7f6de5400700 1 Processor -- start 2024-02-28T23:18:37.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6de5400700 1 -- start start 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6de5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6de5400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de0100e80 0x7f6de0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6de5400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0198d50 con 0x7f6de01004e0 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6de5400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de019cae0 con 0x7f6de0100e80 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6dde800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6dde800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50758/0 (socket says 172.21.15.57:50758) 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.086+0000 7f6dde800700 1 -- 172.21.15.57:0/2649746847 learned_addr learned my addr 172.21.15.57:0/2649746847 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:18:37.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dd7e00700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de0100e80 0x7f6de0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dde800700 1 -- 172.21.15.57:0/2649746847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de0100e80 msgr2=0x7f6de0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dde800700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de0100e80 0x7f6de0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dde800700 1 -- 172.21.15.57:0/2649746847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80095f0 con 0x7f6de01004e0 2024-02-28T23:18:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dde800700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00060b0 tx=0x7f6dd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.087+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd000dda0 con 0x7f6de01004e0 2024-02-28T23:18:37.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.088+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd000f040 con 0x7f6de01004e0 2024-02-28T23:18:37.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.088+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de019cdc0 con 0x7f6de01004e0 2024-02-28T23:18:37.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.088+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd0014790 con 0x7f6de01004e0 2024-02-28T23:18:37.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.088+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de019d310 con 0x7f6de01004e0 2024-02-28T23:18:37.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.089+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de01045d0 con 0x7f6de01004e0 2024-02-28T23:18:37.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.090+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6dd0014980 con 0x7f6de01004e0 2024-02-28T23:18:37.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.091+0000 7f6dd7400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 0x7f6dc406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:18:37.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.091+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6dd008d070 con 0x7f6de01004e0 2024-02-28T23:18:37.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.092+0000 7f6dd7e00700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 0x7f6dc406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:18:37.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.092+0000 7f6dd7e00700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 0x7f6dc406fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6de0199750 tx=0x7f6dc801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:18:37.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.096+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dd0056830 con 0x7f6de01004e0 2024-02-28T23:18:37.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.412+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6de0199560 con 0x7f6de01004e0 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.413+0000 7f6dd7400700 1 -- 172.21.15.57:0/2649746847 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6dd0059ef0 con 0x7f6de01004e0 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:18:37.412 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 msgr2=0x7f6dc406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 0x7f6dc406fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6de0199750 tx=0x7f6dc801a040 comp rx=0 tx=0).stop 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 msgr2=0x7f6de0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00060b0 tx=0x7f6dd000d3b0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 shutdown_connections 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dc406d8a0 0x7f6dc406fd50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de01004e0 0x7f6de0198130 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 --2- 172.21.15.57:0/2649746847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de0100e80 0x7f6de0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.417+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 >> 172.21.15.57:0/2649746847 conn(0x7f6de00fbda0 msgr2=0x7f6de00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:18:37.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.418+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 shutdown_connections 2024-02-28T23:18:37.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:18:37.418+0000 7f6de5400700 1 -- 172.21.15.57:0/2649746847 wait complete. 2024-02-28T23:18:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2649746847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2649746847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:18:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:38.987 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:38.987 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:38.988 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:38.988 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:38.988 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:38.988 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:18:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:18:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:39 smithi057 ceph-mon[89491]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:39 smithi084 ceph-mon[70440]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: Upgrade: Updating mds.cephfs.smithi057.sbkaml 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:41.488 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:41 smithi057 ceph-mon[89491]: Deploying daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: Upgrade: Updating mds.cephfs.smithi057.sbkaml 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi057.sbkaml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:18:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:41 smithi084 ceph-mon[70440]: Deploying daemon mds.cephfs.smithi057.sbkaml on smithi057 2024-02-28T23:18:43.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:43 smithi057 ceph-mon[89491]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:43 smithi084 ceph-mon[70440]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:45.553 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:45 smithi057 ceph-mon[89491]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:45.553 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:45 smithi057 ceph-mon[89491]: osdmap e85: 6 total, 6 up, 6 in 2024-02-28T23:18:45.553 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:45 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:18:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:45 smithi084 ceph-mon[70440]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:45 smithi084 ceph-mon[70440]: osdmap e85: 6 total, 6 up, 6 in 2024-02-28T23:18:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:45 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:18:47.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:47 smithi057 ceph-mon[89491]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:47.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:47 smithi084 ceph-mon[70440]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:boot 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:boot 2024-02-28T23:18:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:18:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:18:51.508 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:51 smithi057 ceph-mon[89491]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:51 smithi084 ceph-mon[70440]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:53.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:53 smithi057 ceph-mon[89491]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:53.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:53 smithi084 ceph-mon[70440]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:55.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:55 smithi057 ceph-mon[89491]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:55 smithi084 ceph-mon[70440]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:56.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:56 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:56.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:56 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:56 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:56.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:56 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:57.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:57 smithi057 ceph-mon[89491]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:57.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:57 smithi084 ceph-mon[70440]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:58.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:58.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:58 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:58.664 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:58.664 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:58 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:18:59.398 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:18:59 smithi057 ceph-mon[89491]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:18:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:18:59 smithi084 ceph-mon[70440]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:01 smithi057 ceph-mon[89491]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:01 smithi084 ceph-mon[70440]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:19:03.633 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:19:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:04.645 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:19:04.645 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.645 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.645 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.645 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:19:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:05.715 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:05 smithi084 ceph-mon[70440]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:05 smithi057 ceph-mon[89491]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: Upgrade: Updating mds.cephfs.smithi084.ztfxpc 2024-02-28T23:19:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:19:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:19:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: Deploying daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:19:07.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:06 smithi084 ceph-mon[70440]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: Upgrade: Updating mds.cephfs.smithi084.ztfxpc 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.ztfxpc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: Deploying daemon mds.cephfs.smithi084.ztfxpc on smithi084 2024-02-28T23:19:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:06 smithi057 ceph-mon[89491]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 -- 172.21.15.57:0/1375819446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9744073a00 msgr2=0x7f9744110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 --2- 172.21.15.57:0/1375819446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9744073a00 0x7f9744110d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9730009910 tx=0x7f9730009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 -- 172.21.15.57:0/1375819446 shutdown_connections 2024-02-28T23:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 --2- 172.21.15.57:0/1375819446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9744073a00 0x7f9744110d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 --2- 172.21.15.57:0/1375819446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97440730f0 0x7f97440734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 -- 172.21.15.57:0/1375819446 >> 172.21.15.57:0/1375819446 conn(0x7f97440fbda0 msgr2=0x7f97440fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 -- 172.21.15.57:0/1375819446 shutdown_connections 2024-02-28T23:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.594+0000 7f9749400700 1 -- 172.21.15.57:0/1375819446 wait complete. 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.595+0000 7f9749400700 1 Processor -- start 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9749400700 1 -- start start 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9749400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9749400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744073a00 0x7f974419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9749400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419d050 con 0x7f97440730f0 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9749400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97441a0de0 con 0x7f9744073a00 2024-02-28T23:19:07.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9742800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9742800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55212/0 (socket says 172.21.15.57:55212) 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.596+0000 7f9742800700 1 -- 172.21.15.57:0/3146844324 learned_addr learned my addr 172.21.15.57:0/3146844324 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f9741e00700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744073a00 0x7f974419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f9742800700 1 -- 172.21.15.57:0/3146844324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744073a00 msgr2=0x7f974419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f9742800700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744073a00 0x7f974419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f9742800700 1 -- 172.21.15.57:0/3146844324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300095f0 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f9742800700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f973400c8a0 tx=0x7f973400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.597+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f973400cea0 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.598+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9734004830 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.598+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97441a10c0 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.598+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97340055b0 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.598+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441a1610 con 0x7f97440730f0 2024-02-28T23:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.599+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974410e4d0 con 0x7f97440730f0 2024-02-28T23:19:07.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.600+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f97340049a0 con 0x7f97440730f0 2024-02-28T23:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.601+0000 7f973b400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 0x7f972806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.601+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f973408d260 con 0x7f97440730f0 2024-02-28T23:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.601+0000 7f9741e00700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 0x7f972806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.602+0000 7f9741e00700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 0x7f972806fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97300059d0 tx=0x7f973001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:07.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.606+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9734056970 con 0x7f97440730f0 2024-02-28T23:19:07.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.857+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97441a1890 con 0x7f972806d8a0 2024-02-28T23:19:07.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.859+0000 7f973b400700 1 -- 172.21.15.57:0/3146844324 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f97441a1890 con 0x7f972806d8a0 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.862+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 msgr2=0x7f972806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.862+0000 7f9749400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 0x7f972806fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97300059d0 tx=0x7f973001a040 comp rx=0 tx=0).stop 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.862+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 msgr2=0x7f974419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.862+0000 7f9749400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f973400c8a0 tx=0x7f973400cc60 comp rx=0 tx=0).stop 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 shutdown_connections 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f972806d8a0 0x7f972806fd50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97440730f0 0x7f974419c430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 --2- 172.21.15.57:0/3146844324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744073a00 0x7f974419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:07.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 >> 172.21.15.57:0/3146844324 conn(0x7f97440fbda0 msgr2=0x7f9744102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:07.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 shutdown_connections 2024-02-28T23:19:07.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:07.863+0000 7f9749400700 1 -- 172.21.15.57:0/3146844324 wait complete. 2024-02-28T23:19:07.879 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:19:08.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.029+0000 7f840f600700 1 -- 172.21.15.57:0/3590682204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8408100e80 msgr2=0x7f8408104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.029+0000 7f840f600700 1 --2- 172.21.15.57:0/3590682204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8408100e80 0x7f8408104e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009910 tx=0x7f83f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:08.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 -- 172.21.15.57:0/3590682204 shutdown_connections 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 --2- 172.21.15.57:0/3590682204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8408100e80 0x7f8408104e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 --2- 172.21.15.57:0/3590682204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84081004e0 0x7f84081008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 -- 172.21.15.57:0/3590682204 >> 172.21.15.57:0/3590682204 conn(0x7f84080fbda0 msgr2=0x7f84080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 -- 172.21.15.57:0/3590682204 shutdown_connections 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.030+0000 7f840f600700 1 -- 172.21.15.57:0/3590682204 wait complete. 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.031+0000 7f840f600700 1 Processor -- start 2024-02-28T23:19:08.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.031+0000 7f840f600700 1 -- start start 2024-02-28T23:19:08.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840f600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840f600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8408100e80 0x7f84081985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840f600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408198ca0 con 0x7f84081004e0 2024-02-28T23:19:08.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840f600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840819ca30 con 0x7f8408100e80 2024-02-28T23:19:08.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55236/0 (socket says 172.21.15.57:55236) 2024-02-28T23:19:08.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 -- 172.21.15.57:0/733867066 learned_addr learned my addr 172.21.15.57:0/733867066 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f8407e00700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8408100e80 0x7f84081985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 -- 172.21.15.57:0/733867066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8408100e80 msgr2=0x7f84081985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8408100e80 0x7f84081985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.032+0000 7f840cc00700 1 -- 172.21.15.57:0/733867066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40095f0 con 0x7f84081004e0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f840cc00700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e920 tx=0x7f83fc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc009790 con 0x7f84081004e0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83fc00cba0 con 0x7f84081004e0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840819cd10 con 0x7f84081004e0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc018650 con 0x7f84081004e0 2024-02-28T23:19:08.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.033+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840819d260 con 0x7f84081004e0 2024-02-28T23:19:08.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.035+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84081045d0 con 0x7f84081004e0 2024-02-28T23:19:08.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.036+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f83fc00cd10 con 0x7f84081004e0 2024-02-28T23:19:08.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.037+0000 7f8405600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 0x7f83f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.037+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f83fc014070 con 0x7f84081004e0 2024-02-28T23:19:08.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.037+0000 7f8407e00700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 0x7f83f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.038+0000 7f8407e00700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 0x7f83f006fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84081996a0 tx=0x7f83f4009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.042+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83fc052d20 con 0x7f84081004e0 2024-02-28T23:19:08.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.292+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f840819d4f0 con 0x7f83f006d8a0 2024-02-28T23:19:08.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.295+0000 7f8405600700 1 -- 172.21.15.57:0/733867066 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f840819d4f0 con 0x7f83f006d8a0 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.297+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 msgr2=0x7f83f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.297+0000 7f840f600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 0x7f83f006fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84081996a0 tx=0x7f83f4009f70 comp rx=0 tx=0).stop 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.297+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 msgr2=0x7f8408198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.297+0000 7f840f600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e920 tx=0x7f83fc00ece0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 shutdown_connections 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83f006d8a0 0x7f83f006fd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84081004e0 0x7f8408198080 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 --2- 172.21.15.57:0/733867066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8408100e80 0x7f84081985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 >> 172.21.15.57:0/733867066 conn(0x7f84080fbda0 msgr2=0x7f84080fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:08.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 shutdown_connections 2024-02-28T23:19:08.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.298+0000 7f840f600700 1 -- 172.21.15.57:0/733867066 wait complete. 2024-02-28T23:19:08.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 -- 172.21.15.57:0/1647457419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050102470 msgr2=0x7f10501028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 --2- 172.21.15.57:0/1647457419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050102470 0x7f10501028e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f103c009910 tx=0x7f103c009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 -- 172.21.15.57:0/1647457419 shutdown_connections 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 --2- 172.21.15.57:0/1647457419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050102470 0x7f10501028e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 --2- 172.21.15.57:0/1647457419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050108470 0x7f1050108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 -- 172.21.15.57:0/1647457419 >> 172.21.15.57:0/1647457419 conn(0x7f10500fdf90 msgr2=0x7f10501003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 -- 172.21.15.57:0/1647457419 shutdown_connections 2024-02-28T23:19:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.460+0000 7f1054c00700 1 -- 172.21.15.57:0/1647457419 wait complete. 2024-02-28T23:19:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.461+0000 7f1054c00700 1 Processor -- start 2024-02-28T23:19:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.461+0000 7f1054c00700 1 -- start start 2024-02-28T23:19:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.461+0000 7f1054c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050102470 0x7f10501981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f1054c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f1054c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050198d80 con 0x7f1050108470 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f1054c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050198ec0 con 0x7f1050102470 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55248/0 (socket says 172.21.15.57:55248) 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 -- 172.21.15.57:0/2904387720 learned_addr learned my addr 172.21.15.57:0/2904387720 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104de00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050102470 0x7f10501981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 -- 172.21.15.57:0/2904387720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050102470 msgr2=0x7f10501981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050102470 0x7f10501981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.462+0000 7f104d400700 1 -- 172.21.15.57:0/2904387720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0095f0 con 0x7f1050108470 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f104d400700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b5c0 tx=0x7f103c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c01d070 con 0x7f1050108470 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f103c00f460 con 0x7f1050108470 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105019ccb0 con 0x7f1050108470 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c017630 con 0x7f1050108470 2024-02-28T23:19:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.463+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105019d1a0 con 0x7f1050108470 2024-02-28T23:19:08.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.465+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105004f2a0 con 0x7f1050108470 2024-02-28T23:19:08.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.466+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f103c00f5d0 con 0x7f1050108470 2024-02-28T23:19:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.466+0000 7f1046a00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 0x7f103406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.467+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f103c08eb50 con 0x7f1050108470 2024-02-28T23:19:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.467+0000 7f104de00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 0x7f103406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.468+0000 7f104de00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 0x7f103406fd50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f104000ac50 tx=0x7f104000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.472+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f103c058260 con 0x7f1050108470 2024-02-28T23:19:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.715+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f105010a3c0 con 0x7f103406d8a0 2024-02-28T23:19:08.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.734+0000 7f1046a00700 1 -- 172.21.15.57:0/2904387720 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f105010a3c0 con 0x7f103406d8a0 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (8m) 13s ago 22m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (18m) 13s ago 22m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (21m) 66s ago 21m 9038k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (4m) 13s ago 22m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (4m) 66s ago 21m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (8m) 13s ago 22m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (46s) 13s ago 15m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (21s) 13s ago 15m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (15m) 66s ago 15m 58.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (15m) 66s ago 15m 108M - 18.2.0 dc2bc1663786 d4c8ee8651b1 2024-02-28T23:19:08.734 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (11m) 13s ago 24m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (10m) 66s ago 21m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (6m) 13s ago 24m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (5m) 66s ago 21m 39.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (9m) 13s ago 22m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (9m) 66s ago 21m 8174k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (4m) 13s ago 20m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (3m) 13s ago 19m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (3m) 13s ago 18m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (2m) 66s ago 18m 105M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (113s) 66s ago 17m 99.0M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (71s) 66s ago 17m 11.1M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:19:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (9m) 13s ago 21m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 msgr2=0x7f103406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 0x7f103406fd50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f104000ac50 tx=0x7f104000a380 comp rx=0 tx=0).stop 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 msgr2=0x7f1050198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b5c0 tx=0x7f103c00bd50 comp rx=0 tx=0).stop 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 shutdown_connections 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f103406d8a0 0x7f103406fd50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1050102470 0x7f10501981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 --2- 172.21.15.57:0/2904387720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1050108470 0x7f1050198730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.737+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 >> 172.21.15.57:0/2904387720 conn(0x7f10500fdf90 msgr2=0x7f10500ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.738+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 shutdown_connections 2024-02-28T23:19:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.738+0000 7f1054c00700 1 -- 172.21.15.57:0/2904387720 wait complete. 2024-02-28T23:19:08.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.900+0000 7f7dde400700 1 -- 172.21.15.57:0/2262263045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 msgr2=0x7f7dd80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 --2- 172.21.15.57:0/2262263045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd80734c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009910 tx=0x7f7dc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:08.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 -- 172.21.15.57:0/2262263045 shutdown_connections 2024-02-28T23:19:08.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 --2- 172.21.15.57:0/2262263045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 --2- 172.21.15.57:0/2262263045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd80734c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 -- 172.21.15.57:0/2262263045 >> 172.21.15.57:0/2262263045 conn(0x7f7dd80fbda0 msgr2=0x7f7dd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 -- 172.21.15.57:0/2262263045 shutdown_connections 2024-02-28T23:19:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.901+0000 7f7dde400700 1 -- 172.21.15.57:0/2262263045 wait complete. 2024-02-28T23:19:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.902+0000 7f7dde400700 1 Processor -- start 2024-02-28T23:19:08.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.902+0000 7f7dde400700 1 -- start start 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dde400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dde400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dde400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd819cb10 con 0x7f7dd80730f0 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dde400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd819cc80 con 0x7f7dd8073a00 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55266/0 (socket says 172.21.15.57:55266) 2024-02-28T23:19:08.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 -- 172.21.15.57:0/1493539249 learned_addr learned my addr 172.21.15.57:0/1493539249 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd6a00700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 -- 172.21.15.57:0/1493539249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 msgr2=0x7f7dd8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd7400700 1 -- 172.21.15.57:0/1493539249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc80095f0 con 0x7f7dd80730f0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.903+0000 7f7dd6a00700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dd7400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc801d070 con 0x7f7dd80730f0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dc8004fb0 con 0x7f7dd80730f0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc8022ea0 con 0x7f7dd80730f0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd819cf60 con 0x7f7dd80730f0 2024-02-28T23:19:08.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.904+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd819d4b0 con 0x7f7dd80730f0 2024-02-28T23:19:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.905+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd8109ce0 con 0x7f7dd80730f0 2024-02-28T23:19:08.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.906+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7dc8022470 con 0x7f7dd80730f0 2024-02-28T23:19:08.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.907+0000 7f7dcfe00700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 0x7f7dbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:08.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.907+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7dc808eb50 con 0x7f7dd80730f0 2024-02-28T23:19:08.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.907+0000 7f7dd6a00700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 0x7f7dbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:08.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.908+0000 7f7dd6a00700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 0x7f7dbc06fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4007900 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:08.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:08.912+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7dc80584b0 con 0x7f7dd80730f0 2024-02-28T23:19:09.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.249+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7dd81994c0 con 0x7f7dd80730f0 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.250+0000 7f7dcfe00700 1 -- 172.21.15.57:0/1493539249 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f7dc8027340 con 0x7f7dd80730f0 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:19:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 12 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:19:09.251 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 msgr2=0x7f7dbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 0x7f7dbc06fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4007900 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 msgr2=0x7f7dd81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc800be20 comp rx=0 tx=0).stop 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 shutdown_connections 2024-02-28T23:19:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dbc06d8a0 0x7f7dbc06fd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dd80730f0 0x7f7dd81981b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 --2- 172.21.15.57:0/1493539249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dd8073a00 0x7f7dd8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.253+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 >> 172.21.15.57:0/1493539249 conn(0x7f7dd80fbda0 msgr2=0x7f7dd8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:09.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.254+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 shutdown_connections 2024-02-28T23:19:09.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.254+0000 7f7dde400700 1 -- 172.21.15.57:0/1493539249 wait complete. 2024-02-28T23:19:09.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 -- 172.21.15.57:0/314932978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0081004e0 msgr2=0x7fe0081008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 --2- 172.21.15.57:0/314932978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0081004e0 0x7fe0081008b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009910 tx=0x7fdff8009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 -- 172.21.15.57:0/314932978 shutdown_connections 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 --2- 172.21.15.57:0/314932978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe008100e80 0x7fe008104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 --2- 172.21.15.57:0/314932978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0081004e0 0x7fe0081008b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 -- 172.21.15.57:0/314932978 >> 172.21.15.57:0/314932978 conn(0x7fe0080fbda0 msgr2=0x7fe0080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 -- 172.21.15.57:0/314932978 shutdown_connections 2024-02-28T23:19:09.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.418+0000 7fe00ea00700 1 -- 172.21.15.57:0/314932978 wait complete. 2024-02-28T23:19:09.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.419+0000 7fe00ea00700 1 Processor -- start 2024-02-28T23:19:09.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.419+0000 7fe00ea00700 1 -- start start 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe00ea00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0081004e0 0x7fe008195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe00ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe00ea00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008196b10 con 0x7fe008100e80 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe00ea00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00819a8a0 con 0x7fe0081004e0 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe007400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe007400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55284/0 (socket says 172.21.15.57:55284) 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe007400700 1 -- 172.21.15.57:0/3196877578 learned_addr learned my addr 172.21.15.57:0/3196877578 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.420+0000 7fe007e00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0081004e0 0x7fe008195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe007400700 1 -- 172.21.15.57:0/3196877578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0081004e0 msgr2=0x7fe008195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe007400700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0081004e0 0x7fe008195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe007400700 1 -- 172.21.15.57:0/3196877578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80095f0 con 0x7fe008100e80 2024-02-28T23:19:09.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe007400700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdff400d900 tx=0x7fdff400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:09.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff4009900 con 0x7fe008100e80 2024-02-28T23:19:09.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdff4010460 con 0x7fe008100e80 2024-02-28T23:19:09.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff400f5f0 con 0x7fe008100e80 2024-02-28T23:19:09.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00819ab80 con 0x7fe008100e80 2024-02-28T23:19:09.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.421+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00819b0d0 con 0x7fe008100e80 2024-02-28T23:19:09.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.423+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0081087f0 con 0x7fe008100e80 2024-02-28T23:19:09.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.424+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdff40105d0 con 0x7fe008100e80 2024-02-28T23:19:09.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.425+0000 7fe004c00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 0x7fdfec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.425+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdff408d390 con 0x7fe008100e80 2024-02-28T23:19:09.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.425+0000 7fe007e00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 0x7fdfec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.426+0000 7fe007e00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 0x7fdfec06fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006010 tx=0x7fdff801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.430+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdff4056b20 con 0x7fe008100e80 2024-02-28T23:19:09.508 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:09 smithi057 ceph-mon[89491]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.508 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:09 smithi057 ceph-mon[89491]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:09.508 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:09 smithi057 ceph-mon[89491]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.508 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:09 smithi057 ceph-mon[89491]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:09 smithi084 ceph-mon[70440]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:09 smithi084 ceph-mon[70440]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:09 smithi084 ceph-mon[70440]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:09 smithi084 ceph-mon[70440]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.735+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe00804f2a0 con 0x7fe008100e80 2024-02-28T23:19:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.736+0000 7fe004c00700 1 -- 172.21.15.57:0/3196877578 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1657 (secure 0 0 0) 0x7fdff4017690 con 0x7fe008100e80 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:e30 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:19:09.737 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:19:09.738 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:09.739 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:19:09.740 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:19:09.740 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:19:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.739+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 msgr2=0x7fdfec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.739+0000 7fe00ea00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 0x7fdfec06fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006010 tx=0x7fdff801a040 comp rx=0 tx=0).stop 2024-02-28T23:19:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.739+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 msgr2=0x7fe008196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.739+0000 7fe00ea00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdff400d900 tx=0x7fdff400dcc0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 shutdown_connections 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdfec06d8a0 0x7fdfec06fd50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0081004e0 0x7fe008195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 --2- 172.21.15.57:0/3196877578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe008100e80 0x7fe008196430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 >> 172.21.15.57:0/3196877578 conn(0x7fe0080fbda0 msgr2=0x7fe0080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 shutdown_connections 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.740+0000 7fe00ea00700 1 -- 172.21.15.57:0/3196877578 wait complete. 2024-02-28T23:19:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 30 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 -- 172.21.15.57:0/911513298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 msgr2=0x7f120c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 --2- 172.21.15.57:0/911513298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1028e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009910 tx=0x7f11f8009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 -- 172.21.15.57:0/911513298 shutdown_connections 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 --2- 172.21.15.57:0/911513298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1028e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 --2- 172.21.15.57:0/911513298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 -- 172.21.15.57:0/911513298 >> 172.21.15.57:0/911513298 conn(0x7f120c0fdf90 msgr2=0x7f120c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 -- 172.21.15.57:0/911513298 shutdown_connections 2024-02-28T23:19:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.893+0000 7f1212200700 1 -- 172.21.15.57:0/911513298 wait complete. 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.894+0000 7f1212200700 1 Processor -- start 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.894+0000 7f1212200700 1 -- start start 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f1212200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f1212200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f1212200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c198e10 con 0x7f120c102470 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f1212200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c19cba0 con 0x7f120c108470 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55296/0 (socket says 172.21.15.57:55296) 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 -- 172.21.15.57:0/3525895059 learned_addr learned my addr 172.21.15.57:0/3525895059 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120aa00700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 -- 172.21.15.57:0/3525895059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 msgr2=0x7f120c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:09.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120b400700 1 -- 172.21.15.57:0/3525895059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80095f0 con 0x7f120c102470 2024-02-28T23:19:09.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.895+0000 7f120aa00700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f120b400700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c8a0 tx=0x7f11fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc00cea0 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11fc004830 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c19ce80 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc0055b0 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.896+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c19d3d0 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.897+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c10a970 con 0x7f120c102470 2024-02-28T23:19:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.898+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f11fc0049a0 con 0x7f120c102470 2024-02-28T23:19:09.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.899+0000 7f1203e00700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 0x7f11f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:09.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.899+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11fc08d180 con 0x7f120c102470 2024-02-28T23:19:09.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.900+0000 7f120aa00700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 0x7f11f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:09.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.900+0000 7f120aa00700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 0x7f11f006fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11f8006010 tx=0x7f11f800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:09.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:09.904+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11fc056910 con 0x7f120c102470 2024-02-28T23:19:10.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.145+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f120c199690 con 0x7f11f006d8a0 2024-02-28T23:19:10.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.148+0000 7f1203e00700 1 -- 172.21.15.57:0/3525895059 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11f8005560 con 0x7f11f006d8a0 2024-02-28T23:19:10.147 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:19:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "14/23 daemons upgraded", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:19:10.148 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 msgr2=0x7f11f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 0x7f11f006fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11f8006010 tx=0x7f11f800b540 comp rx=0 tx=0).stop 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 msgr2=0x7f120c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c8a0 tx=0x7f11fc00cc60 comp rx=0 tx=0).stop 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 shutdown_connections 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11f006d8a0 0x7f11f006fd50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f120c102470 0x7f120c1981f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 --2- 172.21.15.57:0/3525895059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f120c108470 0x7f120c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 >> 172.21.15.57:0/3525895059 conn(0x7f120c0fdf90 msgr2=0x7f120c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 shutdown_connections 2024-02-28T23:19:10.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.151+0000 7f1212200700 1 -- 172.21.15.57:0/3525895059 wait complete. 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 -- 172.21.15.57:0/1081049324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc073090 msgr2=0x7f5efc073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 --2- 172.21.15.57:0/1081049324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc073090 0x7f5efc073460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009940 tx=0x7f5eec009c50 comp rx=0 tx=0).stop 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 -- 172.21.15.57:0/1081049324 shutdown_connections 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 --2- 172.21.15.57:0/1081049324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc0739a0 0x7f5efc10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 --2- 172.21.15.57:0/1081049324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc073090 0x7f5efc073460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 -- 172.21.15.57:0/1081049324 >> 172.21.15.57:0/1081049324 conn(0x7f5efc0fbda0 msgr2=0x7f5efc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 -- 172.21.15.57:0/1081049324 shutdown_connections 2024-02-28T23:19:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.314+0000 7f5f02400700 1 -- 172.21.15.57:0/1081049324 wait complete. 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.315+0000 7f5f02400700 1 Processor -- start 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.315+0000 7f5f02400700 1 -- start start 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5f02400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc073090 0x7f5efc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5f02400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5f02400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc198da0 con 0x7f5efc0739a0 2024-02-28T23:19:10.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5f02400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc19cb30 con 0x7f5efc073090 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5efaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5efaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55316/0 (socket says 172.21.15.57:55316) 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.316+0000 7f5efaa00700 1 -- 172.21.15.57:0/2177843798 learned_addr learned my addr 172.21.15.57:0/2177843798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5efb400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc073090 0x7f5efc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5efaa00700 1 -- 172.21.15.57:0/2177843798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc073090 msgr2=0x7f5efc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5efaa00700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc073090 0x7f5efc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5efaa00700 1 -- 172.21.15.57:0/2177843798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0095f0 con 0x7f5efc0739a0 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5efaa00700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d900 tx=0x7f5ee800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee80041f0 con 0x7f5efc0739a0 2024-02-28T23:19:10.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ee8004cf0 con 0x7f5efc0739a0 2024-02-28T23:19:10.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.317+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc19ce10 con 0x7f5efc0739a0 2024-02-28T23:19:10.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.318+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800b750 con 0x7f5efc0739a0 2024-02-28T23:19:10.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.318+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc19d360 con 0x7f5efc0739a0 2024-02-28T23:19:10.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.319+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc109ce0 con 0x7f5efc0739a0 2024-02-28T23:19:10.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.320+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5ee8004350 con 0x7f5efc0739a0 2024-02-28T23:19:10.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.321+0000 7f5ef3e00700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 0x7f5ee006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:10.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.321+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5ee808d640 con 0x7f5efc0739a0 2024-02-28T23:19:10.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.321+0000 7f5efb400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 0x7f5ee006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:10.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.322+0000 7f5efb400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 0x7f5ee006fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006010 tx=0x7f5eec005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:10.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.325+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ee8056dd0 con 0x7f5efc0739a0 2024-02-28T23:19:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:10 smithi057 ceph-mon[89491]: osdmap e86: 6 total, 6 up, 6 in 2024-02-28T23:19:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:10 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:19:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1493539249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:10.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3196877578' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:19:10.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:10 smithi084 ceph-mon[70440]: osdmap e86: 6 total, 6 up, 6 in 2024-02-28T23:19:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:10 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:19:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1493539249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:10.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3196877578' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:19:10.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.630+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5efc04f2a0 con 0x7f5efc0739a0 2024-02-28T23:19:10.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.631+0000 7f5ef3e00700 1 -- 172.21.15.57:0/2177843798 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5ee805a490 con 0x7f5efc0739a0 2024-02-28T23:19:10.631 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:19:10.631 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:19:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:19:10.631 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:19:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:19:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 msgr2=0x7f5ee006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 0x7f5ee006fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006010 tx=0x7f5eec005a90 comp rx=0 tx=0).stop 2024-02-28T23:19:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 msgr2=0x7f5efc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d900 tx=0x7f5ee800dcc0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 shutdown_connections 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5ee006d8a0 0x7f5ee006fd50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5efc073090 0x7f5efc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 --2- 172.21.15.57:0/2177843798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5efc0739a0 0x7f5efc1986c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.634+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 >> 172.21.15.57:0/2177843798 conn(0x7f5efc0fbda0 msgr2=0x7f5efc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.635+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 shutdown_connections 2024-02-28T23:19:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:10.635+0000 7f5f02400700 1 -- 172.21.15.57:0/2177843798 wait complete. 2024-02-28T23:19:11.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:11 smithi057 ceph-mon[89491]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:11 smithi057 ceph-mon[89491]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:11 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2177843798' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:19:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:11 smithi084 ceph-mon[70440]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:11 smithi084 ceph-mon[70440]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:11 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2177843798' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:19:13.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:13 smithi057 ceph-mon[89491]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:13 smithi084 ceph-mon[70440]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:15 smithi084 ceph-mon[70440]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:15.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:15 smithi057 ceph-mon[89491]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] up:boot 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:16 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] up:boot 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:16 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:19:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:17 smithi057 ceph-mon[89491]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:17.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:17 smithi084 ceph-mon[70440]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:19 smithi084 ceph-mon[70440]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:19.802 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:19 smithi057 ceph-mon[89491]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:21.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:21.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:21.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:21.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:21 smithi057 ceph-mon[89491]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:21 smithi084 ceph-mon[70440]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:23.043 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:23.043 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:19:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:23 smithi057 ceph-mon[89491]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:23 smithi084 ceph-mon[70440]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:25 smithi057 ceph-mon[89491]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:26.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:25 smithi084 ceph-mon[70440]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:27 smithi057 ceph-mon[89491]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:27 smithi084 ceph-mon[70440]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:29 smithi084 ceph-mon[70440]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:29 smithi057 ceph-mon[89491]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:31.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:31 smithi057 ceph-mon[89491]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:31.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:31 smithi084 ceph-mon[70440]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:33 smithi057 ceph-mon[89491]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:33 smithi084 ceph-mon[70440]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:35 smithi084 ceph-mon[70440]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:35.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:35 smithi057 ceph-mon[89491]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:37.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:37 smithi057 ceph-mon[89491]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:37 smithi084 ceph-mon[70440]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:39.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:39 smithi057 ceph-mon[89491]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:39 smithi084 ceph-mon[70440]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 -- 172.21.15.57:0/313634901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a4073090 msgr2=0x7f60a4073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 --2- 172.21.15.57:0/313634901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a4073090 0x7f60a4073460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6094009940 tx=0x7f6094009c50 comp rx=0 tx=0).stop 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 -- 172.21.15.57:0/313634901 shutdown_connections 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 --2- 172.21.15.57:0/313634901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a40739a0 0x7f60a410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 --2- 172.21.15.57:0/313634901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a4073090 0x7f60a4073460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 -- 172.21.15.57:0/313634901 >> 172.21.15.57:0/313634901 conn(0x7f60a40fbda0 msgr2=0x7f60a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 -- 172.21.15.57:0/313634901 shutdown_connections 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.812+0000 7f60a9a00700 1 -- 172.21.15.57:0/313634901 wait complete. 2024-02-28T23:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.813+0000 7f60a9a00700 1 Processor -- start 2024-02-28T23:19:40.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a9a00700 1 -- start start 2024-02-28T23:19:40.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:40.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a9a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 0x7f60a419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a9a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4198ee0 con 0x7f60a40739a0 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a9a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4199050 con 0x7f60a41988d0 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.814+0000 7f60a2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36830/0 (socket says 172.21.15.57:36830) 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f60a2a00700 1 -- 172.21.15.57:0/1907138176 learned_addr learned my addr 172.21.15.57:0/1907138176 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f60a2a00700 1 -- 172.21.15.57:0/1907138176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 msgr2=0x7f60a419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f609be00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 0x7f60a419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f60a2a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 0x7f60a419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f60a2a00700 1 -- 172.21.15.57:0/1907138176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940095f0 con 0x7f60a40739a0 2024-02-28T23:19:40.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f609be00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 0x7f60a419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.815+0000 7f60a2a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f609400b580 tx=0x7f60940056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.816+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609401d070 con 0x7f60a40739a0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.816+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6094005e00 con 0x7f60a40739a0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.816+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a419d2e0 con 0x7f60a40739a0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.816+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6094021410 con 0x7f60a40739a0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.816+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a419d830 con 0x7f60a40739a0 2024-02-28T23:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.817+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a4109e20 con 0x7f60a40739a0 2024-02-28T23:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.818+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f609400f880 con 0x7f60a40739a0 2024-02-28T23:19:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.820+0000 7f609b400700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 0x7f608806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.820+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f609408e7c0 con 0x7f60a40739a0 2024-02-28T23:19:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.820+0000 7f609be00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 0x7f608806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.821+0000 7f609be00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 0x7f608806fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f608c005fd0 tx=0x7f608c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:40.825+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6094057ed0 con 0x7f60a40739a0 2024-02-28T23:19:41.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.075+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60a419db80 con 0x7f608806d8a0 2024-02-28T23:19:41.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.079+0000 7f609b400700 1 -- 172.21.15.57:0/1907138176 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f60a419db80 con 0x7f608806d8a0 2024-02-28T23:19:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.081+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 msgr2=0x7f608806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.081+0000 7f60a9a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 0x7f608806fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f608c005fd0 tx=0x7f608c005dc0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.081+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 msgr2=0x7f60a4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.081+0000 7f60a9a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f609400b580 tx=0x7f60940056a0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 shutdown_connections 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f608806d8a0 0x7f608806fd50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60a40739a0 0x7f60a4198390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 --2- 172.21.15.57:0/1907138176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a41988d0 0x7f60a419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 >> 172.21.15.57:0/1907138176 conn(0x7f60a40fbda0 msgr2=0x7f60a4106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 shutdown_connections 2024-02-28T23:19:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.082+0000 7f60a9a00700 1 -- 172.21.15.57:0/1907138176 wait complete. 2024-02-28T23:19:41.097 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 -- 172.21.15.57:0/2986587194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf24106b80 msgr2=0x7faf24106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 --2- 172.21.15.57:0/2986587194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf24106b80 0x7faf24106ff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009910 tx=0x7faf0c009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 -- 172.21.15.57:0/2986587194 shutdown_connections 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 --2- 172.21.15.57:0/2986587194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf24106b80 0x7faf24106ff0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 --2- 172.21.15.57:0/2986587194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf2410c8f0 0x7faf2410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 -- 172.21.15.57:0/2986587194 >> 172.21.15.57:0/2986587194 conn(0x7faf24076060 msgr2=0x7faf24078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 -- 172.21.15.57:0/2986587194 shutdown_connections 2024-02-28T23:19:41.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.250+0000 7faf28c00700 1 -- 172.21.15.57:0/2986587194 wait complete. 2024-02-28T23:19:41.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.251+0000 7faf28c00700 1 Processor -- start 2024-02-28T23:19:41.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.251+0000 7faf28c00700 1 -- start start 2024-02-28T23:19:41.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf28c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 0x7faf2419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf28c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf28c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2419cfd0 con 0x7faf2410c8f0 2024-02-28T23:19:41.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf28c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2419d110 con 0x7faf24106b80 2024-02-28T23:19:41.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf19400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf19400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36836/0 (socket says 172.21.15.57:36836) 2024-02-28T23:19:41.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.252+0000 7faf19400700 1 -- 172.21.15.57:0/1544286086 learned_addr learned my addr 172.21.15.57:0/1544286086 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf21e00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 0x7faf2419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf19400700 1 -- 172.21.15.57:0/1544286086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 msgr2=0x7faf2419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf19400700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 0x7faf2419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf19400700 1 -- 172.21.15.57:0/1544286086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0095f0 con 0x7faf2410c8f0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf21e00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 0x7faf2419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf19400700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b5c0 tx=0x7faf0c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0c01d070 con 0x7faf2410c8f0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf0c00f460 con 0x7faf2410c8f0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0c017610 con 0x7faf2410c8f0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.253+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf241a0f00 con 0x7faf2410c8f0 2024-02-28T23:19:41.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.254+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf241a13f0 con 0x7faf2410c8f0 2024-02-28T23:19:41.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.255+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2410ecc0 con 0x7faf2410c8f0 2024-02-28T23:19:41.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.256+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7faf0c00f5d0 con 0x7faf2410c8f0 2024-02-28T23:19:41.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.257+0000 7faf1b400700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 0x7faf0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.257+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faf0c08eb30 con 0x7faf2410c8f0 2024-02-28T23:19:41.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.258+0000 7faf21e00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 0x7faf0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.258+0000 7faf21e00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 0x7faf0806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf1400d340 tx=0x7faf14009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.262+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf0c0582c0 con 0x7faf2410c8f0 2024-02-28T23:19:41.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.517+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf241a1720 con 0x7faf0806d8a0 2024-02-28T23:19:41.517 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:41 smithi057 ceph-mon[89491]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:41.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.520+0000 7faf1b400700 1 -- 172.21.15.57:0/1544286086 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faf241a1720 con 0x7faf0806d8a0 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 msgr2=0x7faf0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 0x7faf0806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf1400d340 tx=0x7faf14009040 comp rx=0 tx=0).stop 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 msgr2=0x7faf2419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b5c0 tx=0x7faf0c00be00 comp rx=0 tx=0).stop 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 shutdown_connections 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faf0806d8a0 0x7faf0806fd50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24106b80 0x7faf2419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 --2- 172.21.15.57:0/1544286086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf2410c8f0 0x7faf2419c980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.523+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 >> 172.21.15.57:0/1544286086 conn(0x7faf24076060 msgr2=0x7faf24077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:41.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.524+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 shutdown_connections 2024-02-28T23:19:41.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.524+0000 7faf28c00700 1 -- 172.21.15.57:0/1544286086 wait complete. 2024-02-28T23:19:41.573 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:41 smithi084 ceph-mon[70440]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:41.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 -- 172.21.15.57:0/1244992939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6374073a00 msgr2=0x7f637410c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 --2- 172.21.15.57:0/1244992939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6374073a00 0x7f637410c360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6360009910 tx=0x7f6360009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:41.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 -- 172.21.15.57:0/1244992939 shutdown_connections 2024-02-28T23:19:41.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 --2- 172.21.15.57:0/1244992939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6374073a00 0x7f637410c360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 --2- 172.21.15.57:0/1244992939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63740730f0 0x7f63740734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.702+0000 7f637a000700 1 -- 172.21.15.57:0/1244992939 >> 172.21.15.57:0/1244992939 conn(0x7f63740fbb60 msgr2=0x7f63740fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.703+0000 7f637a000700 1 -- 172.21.15.57:0/1244992939 shutdown_connections 2024-02-28T23:19:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.703+0000 7f637a000700 1 -- 172.21.15.57:0/1244992939 wait complete. 2024-02-28T23:19:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 Processor -- start 2024-02-28T23:19:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 -- start start 2024-02-28T23:19:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6374073a00 0x7f63741984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637419c810 con 0x7f63740730f0 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.704+0000 7f637a000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637419c980 con 0x7f6374073a00 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36844/0 (socket says 172.21.15.57:36844) 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 -- 172.21.15.57:0/2775463080 learned_addr learned my addr 172.21.15.57:0/2775463080 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6373e00700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6374073a00 0x7f63741984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 -- 172.21.15.57:0/2775463080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6374073a00 msgr2=0x7f63741984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6374073a00 0x7f63741984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.705+0000 7f6378c00700 1 -- 172.21.15.57:0/2775463080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600095f0 con 0x7f63740730f0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f6378c00700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f636800b4b0 tx=0x7f636800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6368011800 con 0x7f63740730f0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6368011e20 con 0x7f63740730f0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636800f2e0 con 0x7f63740730f0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637419cc60 con 0x7f63740730f0 2024-02-28T23:19:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.706+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637419d1b0 con 0x7f63740730f0 2024-02-28T23:19:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.707+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374109a60 con 0x7f63740730f0 2024-02-28T23:19:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.708+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6368010380 con 0x7f63740730f0 2024-02-28T23:19:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.709+0000 7f6371600700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 0x7f635c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.709+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f636808d0e0 con 0x7f63740730f0 2024-02-28T23:19:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.710+0000 7f6373e00700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 0x7f635c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.710+0000 7f6373e00700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 0x7f635c06fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.714+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6368056890 con 0x7f63740730f0 2024-02-28T23:19:41.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.953+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6374199210 con 0x7f635c06d8a0 2024-02-28T23:19:41.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.987+0000 7f6371600700 1 -- 172.21.15.57:0/2775463080 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6374199210 con 0x7f635c06d8a0 2024-02-28T23:19:41.986 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (9m) 46s ago 23m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (19m) 46s ago 23m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (22m) 22s ago 22m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (5m) 46s ago 23m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (5m) 22s ago 21m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (8m) 46s ago 22m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (80s) 46s ago 15m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (54s) 46s ago 15m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (15m) 22s ago 15m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (27s) 22s ago 15m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (11m) 46s ago 24m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:19:41.987 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (10m) 22s ago 21m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (7m) 46s ago 25m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (6m) 22s ago 21m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (10m) 46s ago 23m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (10m) 22s ago 21m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (4m) 46s ago 20m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (4m) 46s ago 20m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (3m) 46s ago 19m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (3m) 22s ago 18m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (2m) 22s ago 18m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (105s) 22s ago 17m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:19:41.988 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (9m) 46s ago 22m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 msgr2=0x7f635c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 0x7f635c06fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636001a040 comp rx=0 tx=0).stop 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 msgr2=0x7f6374197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f636800b4b0 tx=0x7f636800b870 comp rx=0 tx=0).stop 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 shutdown_connections 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f635c06d8a0 0x7f635c06fd50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63740730f0 0x7f6374197e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 --2- 172.21.15.57:0/2775463080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6374073a00 0x7f63741984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 >> 172.21.15.57:0/2775463080 conn(0x7f63740fbb60 msgr2=0x7f6374106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.991+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 shutdown_connections 2024-02-28T23:19:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:41.992+0000 7f637a000700 1 -- 172.21.15.57:0/2775463080 wait complete. 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 -- 172.21.15.57:0/2925989851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c108470 msgr2=0x7f4c5c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 --2- 172.21.15.57:0/2925989851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c108470 0x7f4c5c108840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009910 tx=0x7f4c4c009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 -- 172.21.15.57:0/2925989851 shutdown_connections 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 --2- 172.21.15.57:0/2925989851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c102470 0x7f4c5c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 --2- 172.21.15.57:0/2925989851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c108470 0x7f4c5c108840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 -- 172.21.15.57:0/2925989851 >> 172.21.15.57:0/2925989851 conn(0x7f4c5c0fdf90 msgr2=0x7f4c5c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:42.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 -- 172.21.15.57:0/2925989851 shutdown_connections 2024-02-28T23:19:42.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.153+0000 7f4c63000700 1 -- 172.21.15.57:0/2925989851 wait complete. 2024-02-28T23:19:42.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.154+0000 7f4c63000700 1 Processor -- start 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c63000700 1 -- start start 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c63000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c63000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c108470 0x7f4c5c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c63000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c198e30 con 0x7f4c5c102470 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c63000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c19cbc0 con 0x7f4c5c108470 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c5be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c5be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36868/0 (socket says 172.21.15.57:36868) 2024-02-28T23:19:42.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.155+0000 7f4c5be00700 1 -- 172.21.15.57:0/3945864408 learned_addr learned my addr 172.21.15.57:0/3945864408 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c5b400700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c108470 0x7f4c5c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c5be00700 1 -- 172.21.15.57:0/3945864408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c108470 msgr2=0x7f4c5c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c5be00700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c108470 0x7f4c5c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c5be00700 1 -- 172.21.15.57:0/3945864408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c0095f0 con 0x7f4c5c102470 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c5be00700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006010 tx=0x7f4c4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c01d070 con 0x7f4c5c102470 2024-02-28T23:19:42.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c4c004fb0 con 0x7f4c5c102470 2024-02-28T23:19:42.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c00f690 con 0x7f4c5c102470 2024-02-28T23:19:42.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.156+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c19ce40 con 0x7f4c5c102470 2024-02-28T23:19:42.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.157+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c19d330 con 0x7f4c5c102470 2024-02-28T23:19:42.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.158+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c10a9e0 con 0x7f4c5c102470 2024-02-28T23:19:42.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.158+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4c4c0228a0 con 0x7f4c5c102470 2024-02-28T23:19:42.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.159+0000 7f4c58c00700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 0x7f4c4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.159+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4c4c08ede0 con 0x7f4c5c102470 2024-02-28T23:19:42.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.160+0000 7f4c5b400700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 0x7f4c4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.160+0000 7f4c5b400700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 0x7f4c4006fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:42.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.164+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c4c058570 con 0x7f4c5c102470 2024-02-28T23:19:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.481+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c5c04f2a0 con 0x7f4c5c102470 2024-02-28T23:19:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.483+0000 7f4c58c00700 1 -- 172.21.15.57:0/3945864408 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f4c4c027340 con 0x7f4c5c102470 2024-02-28T23:19:42.481 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:19:42.481 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:19:42.481 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:19:42.482 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:19:42.483 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:19:42.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.485+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 msgr2=0x7f4c4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.485+0000 7f4c63000700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 0x7f4c4006fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48008040 comp rx=0 tx=0).stop 2024-02-28T23:19:42.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.485+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 msgr2=0x7f4c5c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.485+0000 7f4c63000700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006010 tx=0x7f4c4c00be20 comp rx=0 tx=0).stop 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 shutdown_connections 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c4006d8a0 0x7f4c4006fd50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c102470 0x7f4c5c198210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 --2- 172.21.15.57:0/3945864408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5c108470 0x7f4c5c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 >> 172.21.15.57:0/3945864408 conn(0x7f4c5c0fdf90 msgr2=0x7f4c5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 shutdown_connections 2024-02-28T23:19:42.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.486+0000 7f4c63000700 1 -- 172.21.15.57:0/3945864408 wait complete. 2024-02-28T23:19:42.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:42 smithi057 ceph-mon[89491]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:42.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:42 smithi057 ceph-mon[89491]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:42.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:42 smithi084 ceph-mon[70440]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:42.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:42 smithi084 ceph-mon[70440]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:42.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 -- 172.21.15.57:0/2874109463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e94106b80 msgr2=0x7f1e94106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/2874109463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e94106b80 0x7f1e94106ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009910 tx=0x7f1e80009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:42.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 -- 172.21.15.57:0/2874109463 shutdown_connections 2024-02-28T23:19:42.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/2874109463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e94106b80 0x7f1e94106ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/2874109463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9410c660 0x7f1e9410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.643+0000 7f1e9ac00700 1 -- 172.21.15.57:0/2874109463 >> 172.21.15.57:0/2874109463 conn(0x7f1e94075e90 msgr2=0x7f1e940782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:42.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.644+0000 7f1e9ac00700 1 -- 172.21.15.57:0/2874109463 shutdown_connections 2024-02-28T23:19:42.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.644+0000 7f1e9ac00700 1 -- 172.21.15.57:0/2874109463 wait complete. 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 Processor -- start 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 -- start start 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e94106b80 0x7f1e9419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9419cee0 con 0x7f1e9410c660 2024-02-28T23:19:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e9ac00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941a0c70 con 0x7f1e94106b80 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.645+0000 7f1e98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36884/0 (socket says 172.21.15.57:36884) 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 -- 172.21.15.57:0/1174873199 learned_addr learned my addr 172.21.15.57:0/1174873199 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e99800700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e94106b80 0x7f1e9419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 -- 172.21.15.57:0/1174873199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e94106b80 msgr2=0x7f1e9419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e94106b80 0x7f1e9419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 -- 172.21.15.57:0/1174873199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800095f0 con 0x7f1e9410c660 2024-02-28T23:19:42.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.646+0000 7f1e98e00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000f690 tx=0x7f1e8000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:42.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.647+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e8001d070 con 0x7f1e9410c660 2024-02-28T23:19:42.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.647+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e800050d0 con 0x7f1e9410c660 2024-02-28T23:19:42.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.647+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e80022470 con 0x7f1e9410c660 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.647+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e941a0ef0 con 0x7f1e9410c660 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.647+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941a13e0 con 0x7f1e9410c660 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.648+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9410eb40 con 0x7f1e9410c660 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.649+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1e80003680 con 0x7f1e9410c660 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.650+0000 7f1e92000700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 0x7f1e7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:42.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.650+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e8008e950 con 0x7f1e9410c660 2024-02-28T23:19:42.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.650+0000 7f1e99800700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 0x7f1e7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:42.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.651+0000 7f1e99800700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 0x7f1e7c06fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009bf0 tx=0x7f1e88009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.654+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e800580e0 con 0x7f1e9410c660 2024-02-28T23:19:42.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.927+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1e94062380 con 0x7f1e9410c660 2024-02-28T23:19:42.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.928+0000 7f1e92000700 1 -- 172.21.15.57:0/1174873199 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f1e8005b7a0 con 0x7f1e9410c660 2024-02-28T23:19:42.927 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:19:42.927 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:19:42.928 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:19:42.929 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:19:42.930 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 msgr2=0x7f1e7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 0x7f1e7c06fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009bf0 tx=0x7f1e88009260 comp rx=0 tx=0).stop 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 msgr2=0x7f1e9419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000f690 tx=0x7f1e8000f770 comp rx=0 tx=0).stop 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 shutdown_connections 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e7c06d8a0 0x7f1e7c06fd50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e94106b80 0x7f1e9419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 --2- 172.21.15.57:0/1174873199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9410c660 0x7f1e9419c800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.931+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 >> 172.21.15.57:0/1174873199 conn(0x7f1e94075e90 msgr2=0x7f1e940779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.932+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 shutdown_connections 2024-02-28T23:19:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:42.932+0000 7f1e9ac00700 1 -- 172.21.15.57:0/1174873199 wait complete. 2024-02-28T23:19:42.932 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:19:43.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 -- 172.21.15.57:0/701747070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674108470 msgr2=0x7fd674108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 --2- 172.21.15.57:0/701747070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674108470 0x7fd674108840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b0a0 tx=0x7fd66400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 -- 172.21.15.57:0/701747070 shutdown_connections 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 --2- 172.21.15.57:0/701747070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674102470 0x7fd6741028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 --2- 172.21.15.57:0/701747070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674108470 0x7fd674108840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 -- 172.21.15.57:0/701747070 >> 172.21.15.57:0/701747070 conn(0x7fd6740fdf90 msgr2=0x7fd6741003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 -- 172.21.15.57:0/701747070 shutdown_connections 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.082+0000 7fd67a200700 1 -- 172.21.15.57:0/701747070 wait complete. 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.083+0000 7fd67a200700 1 Processor -- start 2024-02-28T23:19:43.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.083+0000 7fd67a200700 1 -- start start 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd67a200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd67a200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674108470 0x7fd674198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd67a200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674198dc0 con 0x7fd674102470 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd67a200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674198f00 con 0x7fd674108470 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36912/0 (socket says 172.21.15.57:36912) 2024-02-28T23:19:43.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 -- 172.21.15.57:0/2370610105 learned_addr learned my addr 172.21.15.57:0/2370610105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:43.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd66aa00700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674108470 0x7fd674198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 -- 172.21.15.57:0/2370610105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674108470 msgr2=0x7fd674198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674108470 0x7fd674198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.084+0000 7fd673400700 1 -- 172.21.15.57:0/2370610105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009d00 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd673400700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b070 tx=0x7fd66400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66400e040 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd664003e60 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67419ccf0 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66401d910 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.085+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67419d1e0 con 0x7fd674102470 2024-02-28T23:19:43.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.086+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67410a9e0 con 0x7fd674102470 2024-02-28T23:19:43.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.087+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd664004820 con 0x7fd674102470 2024-02-28T23:19:43.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.088+0000 7fd670c00700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 0x7fd65806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.088+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd66408ee20 con 0x7fd674102470 2024-02-28T23:19:43.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.089+0000 7fd66aa00700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 0x7fd65806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.089+0000 7fd66aa00700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 0x7fd65806fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd674199850 tx=0x7fd65c005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:43.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.093+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6640585d0 con 0x7fd674102470 2024-02-28T23:19:43.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.350+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd674199600 con 0x7fd65806d8a0 2024-02-28T23:19:43.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.352+0000 7fd670c00700 1 -- 172.21.15.57:0/2370610105 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd674199600 con 0x7fd65806d8a0 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:19:43.351 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:19:43.352 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:19:43.352 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:19:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 msgr2=0x7fd65806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 0x7fd65806fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd674199850 tx=0x7fd65c005d50 comp rx=0 tx=0).stop 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 msgr2=0x7fd674198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b070 tx=0x7fd66400cdd0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 shutdown_connections 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd65806d8a0 0x7fd65806fd50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd674102470 0x7fd674198230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 --2- 172.21.15.57:0/2370610105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd674108470 0x7fd674198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.355+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 >> 172.21.15.57:0/2370610105 conn(0x7fd6740fdf90 msgr2=0x7fd674100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:43.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.356+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 shutdown_connections 2024-02-28T23:19:43.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.356+0000 7fd67a200700 1 -- 172.21.15.57:0/2370610105 wait complete. 2024-02-28T23:19:43.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 -- 172.21.15.57:0/2264567781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c068540 msgr2=0x7f1d5c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 --2- 172.21.15.57:0/2264567781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c068540 0x7f1d5c0689b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009910 tx=0x7f1d48009c20 comp rx=0 tx=0).stop 2024-02-28T23:19:43.507 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:43 smithi057 ceph-mon[89491]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:43.507 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:43 smithi057 ceph-mon[89491]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:43.507 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3945864408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:43.507 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1174873199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:19:43.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 -- 172.21.15.57:0/2264567781 shutdown_connections 2024-02-28T23:19:43.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 --2- 172.21.15.57:0/2264567781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c068540 0x7f1d5c0689b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 --2- 172.21.15.57:0/2264567781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.508+0000 7f1d61a00700 1 -- 172.21.15.57:0/2264567781 >> 172.21.15.57:0/2264567781 conn(0x7f1d5c075130 msgr2=0x7f1d5c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:43.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.509+0000 7f1d61a00700 1 -- 172.21.15.57:0/2264567781 shutdown_connections 2024-02-28T23:19:43.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.509+0000 7f1d61a00700 1 -- 172.21.15.57:0/2264567781 wait complete. 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.510+0000 7f1d61a00700 1 Processor -- start 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.510+0000 7f1d61a00700 1 -- start start 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d61a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c068540 0x7f1d5c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d61a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d61a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c198cd0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d61a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c19ca60 con 0x7f1d5c068540 2024-02-28T23:19:43.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d5a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d5a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36926/0 (socket says 172.21.15.57:36926) 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d5a000700 1 -- 172.21.15.57:0/513126665 learned_addr learned my addr 172.21.15.57:0/513126665 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.511+0000 7f1d5aa00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c068540 0x7f1d5c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.512+0000 7f1d5a000700 1 -- 172.21.15.57:0/513126665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c068540 msgr2=0x7f1d5c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.512+0000 7f1d5a000700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c068540 0x7f1d5c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.512+0000 7f1d5a000700 1 -- 172.21.15.57:0/513126665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480095f0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.512+0000 7f1d5a000700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d480092e0 tx=0x7f1d4800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:43.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.513+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d4801d070 con 0x7f1d5c1085b0 2024-02-28T23:19:43.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.513+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d480052a0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.513+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c19cce0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.513+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d48017400 con 0x7f1d5c1085b0 2024-02-28T23:19:43.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.513+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c19d1d0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.514+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c10aa00 con 0x7f1d5c1085b0 2024-02-28T23:19:43.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.515+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1d48017680 con 0x7f1d5c1085b0 2024-02-28T23:19:43.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.516+0000 7f1d53400700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 0x7f1d4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:19:43.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.516+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d4808e880 con 0x7f1d5c1085b0 2024-02-28T23:19:43.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.517+0000 7f1d5aa00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 0x7f1d4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:19:43.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.517+0000 7f1d5aa00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 0x7f1d4006fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c101db0 tx=0x7f1d4c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:19:43.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.525+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d48058030 con 0x7f1d5c1085b0 2024-02-28T23:19:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:43 smithi084 ceph-mon[70440]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:43 smithi084 ceph-mon[70440]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3945864408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:19:43.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1174873199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:19:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.843+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1d5c04f2a0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.843+0000 7f1d53400700 1 -- 172.21.15.57:0/513126665 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1d4805b6f0 con 0x7f1d5c1085b0 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:19:43.842 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 msgr2=0x7f1d4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 0x7f1d4006fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c101db0 tx=0x7f1d4c009260 comp rx=0 tx=0).stop 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 msgr2=0x7f1d5c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d480092e0 tx=0x7f1d4800b6c0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 shutdown_connections 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1d4006d8a0 0x7f1d4006fd50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c068540 0x7f1d5c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 --2- 172.21.15.57:0/513126665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d5c1085b0 0x7f1d5c1985f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 >> 172.21.15.57:0/513126665 conn(0x7f1d5c075130 msgr2=0x7f1d5c106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 shutdown_connections 2024-02-28T23:19:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:19:43.846+0000 7f1d61a00700 1 -- 172.21.15.57:0/513126665 wait complete. 2024-02-28T23:19:44.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:44 smithi057 ceph-mon[89491]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:44.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:44 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/513126665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:19:44.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:44 smithi084 ceph-mon[70440]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:19:44.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:44 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/513126665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:19:45.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:45 smithi057 ceph-mon[89491]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:45.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:45 smithi084 ceph-mon[70440]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:47.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:47 smithi057 ceph-mon[89491]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:47.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:47 smithi084 ceph-mon[70440]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:48.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:19:49.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:49 smithi057 ceph-mon[89491]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:49 smithi084 ceph-mon[70440]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:51.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:51 smithi057 ceph-mon[89491]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:51 smithi084 ceph-mon[70440]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:53.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:53 smithi057 ceph-mon[89491]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:53.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:53 smithi084 ceph-mon[70440]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:55.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:55 smithi057 ceph-mon[89491]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:55.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:55 smithi084 ceph-mon[70440]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:57 smithi057 ceph-mon[89491]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:57.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:57 smithi084 ceph-mon[70440]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:19:59 smithi057 ceph-mon[89491]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:19:59.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:19:59 smithi084 ceph-mon[70440]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:19:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:20:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:19:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:20:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:19:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-28T23:20:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:19:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:20:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:19:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-28T23:20:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-mon[89491]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:20:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-mon[89491]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:20:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-mon[89491]: fs cephfs is degraded 2024-02-28T23:20:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-mon[89491]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:20:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:00 smithi057 ceph-mon[89491]: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:20:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:00 smithi084 ceph-mon[70440]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:20:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:00 smithi084 ceph-mon[70440]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:20:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:00 smithi084 ceph-mon[70440]: fs cephfs is degraded 2024-02-28T23:20:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:00 smithi084 ceph-mon[70440]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:20:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:00 smithi084 ceph-mon[70440]: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:20:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:01 smithi057 ceph-mon[89491]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:01 smithi084 ceph-mon[70440]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:03 smithi057 ceph-mon[89491]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:03 smithi084 ceph-mon[70440]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:05 smithi057 ceph-mon[89491]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:05.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:05 smithi084 ceph-mon[70440]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:07 smithi057 ceph-mon[89491]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:07.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:07 smithi084 ceph-mon[70440]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:09 smithi057 ceph-mon[89491]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:09.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:09 smithi084 ceph-mon[70440]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:11 smithi057 ceph-mon[89491]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:11.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:11 smithi084 ceph-mon[70440]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:13 smithi057 ceph-mon[89491]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:13.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:13 smithi084 ceph-mon[70440]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 -- 172.21.15.57:0/4244968926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 msgr2=0x7f1688100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 --2- 172.21.15.57:0/4244968926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688100760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1678009910 tx=0x7f1678009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 -- 172.21.15.57:0/4244968926 shutdown_connections 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 --2- 172.21.15.57:0/4244968926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 0x7f1688104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 --2- 172.21.15.57:0/4244968926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688100760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 -- 172.21.15.57:0/4244968926 >> 172.21.15.57:0/4244968926 conn(0x7f16880fbc30 msgr2=0x7f16880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 -- 172.21.15.57:0/4244968926 shutdown_connections 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.022+0000 7f168f000700 1 -- 172.21.15.57:0/4244968926 wait complete. 2024-02-28T23:20:14.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 Processor -- start 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 -- start start 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 0x7f1688073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688073ad0 con 0x7f1688100390 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f168f000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688073c10 con 0x7f1688100d30 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.024+0000 7f1687e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36184/0 (socket says 172.21.15.57:36184) 2024-02-28T23:20:14.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 -- 172.21.15.57:0/455945455 learned_addr learned my addr 172.21.15.57:0/455945455 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:14.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687400700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 0x7f1688073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 -- 172.21.15.57:0/455945455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 msgr2=0x7f1688073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 0x7f1688073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 -- 172.21.15.57:0/455945455 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780095f0 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.025+0000 7f1687e00700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f167800b5c0 tx=0x7f167800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.026+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f167801d070 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.026+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f167800f460 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.026+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1678017610 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.026+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1688073e90 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.026+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16881a2270 con 0x7f1688100390 2024-02-28T23:20:14.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.027+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16881086a0 con 0x7f1688100390 2024-02-28T23:20:14.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.028+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f167800f5d0 con 0x7f1688100390 2024-02-28T23:20:14.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.029+0000 7f1684c00700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 0x7f166c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.029+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f167808eb10 con 0x7f1688100390 2024-02-28T23:20:14.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.029+0000 7f1687400700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 0x7f166c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.030+0000 7f1687400700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 0x7f166c06fd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1674005fd0 tx=0x7f167400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.034+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16780582a0 con 0x7f1688100390 2024-02-28T23:20:14.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.290+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f168802cc80 con 0x7f166c06d8a0 2024-02-28T23:20:14.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.293+0000 7f1684c00700 1 -- 172.21.15.57:0/455945455 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f168802cc80 con 0x7f166c06d8a0 2024-02-28T23:20:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 msgr2=0x7f166c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 0x7f166c06fd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1674005fd0 tx=0x7f167400a560 comp rx=0 tx=0).stop 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 msgr2=0x7f1688073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f167800b5c0 tx=0x7f167800be00 comp rx=0 tx=0).stop 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 shutdown_connections 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f166c06d8a0 0x7f166c06fd50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1688100390 0x7f1688073050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 --2- 172.21.15.57:0/455945455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1688100d30 0x7f1688073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 >> 172.21.15.57:0/455945455 conn(0x7f16880fbc30 msgr2=0x7f16880fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 shutdown_connections 2024-02-28T23:20:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.296+0000 7f168f000700 1 -- 172.21.15.57:0/455945455 wait complete. 2024-02-28T23:20:14.313 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 -- 172.21.15.57:0/2114235178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4102470 msgr2=0x7f1ac41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 --2- 172.21.15.57:0/2114235178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4102470 0x7f1ac41028e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009910 tx=0x7f1ab0009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 -- 172.21.15.57:0/2114235178 shutdown_connections 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 --2- 172.21.15.57:0/2114235178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4102470 0x7f1ac41028e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 --2- 172.21.15.57:0/2114235178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4108470 0x7f1ac4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 -- 172.21.15.57:0/2114235178 >> 172.21.15.57:0/2114235178 conn(0x7f1ac40fdf90 msgr2=0x7f1ac41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 -- 172.21.15.57:0/2114235178 shutdown_connections 2024-02-28T23:20:14.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.459+0000 7f1acb200700 1 -- 172.21.15.57:0/2114235178 wait complete. 2024-02-28T23:20:14.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.461+0000 7f1acb200700 1 Processor -- start 2024-02-28T23:20:14.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.461+0000 7f1acb200700 1 -- start start 2024-02-28T23:20:14.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.461+0000 7f1acb200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 0x7f1ac41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1acb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1acb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4198e10 con 0x7f1ac4108470 2024-02-28T23:20:14.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1acb200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419cba0 con 0x7f1ac4102470 2024-02-28T23:20:14.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36214/0 (socket says 172.21.15.57:36214) 2024-02-28T23:20:14.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 -- 172.21.15.57:0/3392006631 learned_addr learned my addr 172.21.15.57:0/3392006631 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:14.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3e00700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 0x7f1ac41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 -- 172.21.15.57:0/3392006631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 msgr2=0x7f1ac41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 0x7f1ac41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3400700 1 -- 172.21.15.57:0/3392006631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00095f0 con 0x7f1ac4108470 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.462+0000 7f1ac3e00700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 0x7f1ac41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1ac3400700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b5c0 tx=0x7f1ab000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab001d070 con 0x7f1ac4108470 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ab0005e00 con 0x7f1ac4108470 2024-02-28T23:20:14.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac419ce20 con 0x7f1ac4108470 2024-02-28T23:20:14.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab0017400 con 0x7f1ac4108470 2024-02-28T23:20:14.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.463+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac419d310 con 0x7f1ac4108470 2024-02-28T23:20:14.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.465+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac410a970 con 0x7f1ac4108470 2024-02-28T23:20:14.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.466+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1ab000f460 con 0x7f1ac4108470 2024-02-28T23:20:14.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.466+0000 7f1ac0c00700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 0x7f1aa806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.467+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ab008ea10 con 0x7f1ac4108470 2024-02-28T23:20:14.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.467+0000 7f1ac3e00700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 0x7f1aa806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.468+0000 7f1ac3e00700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 0x7f1aa806fd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005950 tx=0x7f1ab40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.471+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ab00581a0 con 0x7f1ac4108470 2024-02-28T23:20:14.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.720+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac41995c0 con 0x7f1aa806d8a0 2024-02-28T23:20:14.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.723+0000 7f1ac0c00700 1 -- 172.21.15.57:0/3392006631 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1ac41995c0 con 0x7f1aa806d8a0 2024-02-28T23:20:14.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.725+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 msgr2=0x7f1aa806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.725+0000 7f1acb200700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 0x7f1aa806fd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005950 tx=0x7f1ab40058e0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.725+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 msgr2=0x7f1ac4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.725+0000 7f1acb200700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b5c0 tx=0x7f1ab000be20 comp rx=0 tx=0).stop 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 shutdown_connections 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1aa806d8a0 0x7f1aa806fd50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ac4102470 0x7f1ac41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 --2- 172.21.15.57:0/3392006631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac4108470 0x7f1ac4198730 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 >> 172.21.15.57:0/3392006631 conn(0x7f1ac40fdf90 msgr2=0x7f1ac40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 shutdown_connections 2024-02-28T23:20:14.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.726+0000 7f1acb200700 1 -- 172.21.15.57:0/3392006631 wait complete. 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 -- 172.21.15.57:0/2847176509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0739e0 msgr2=0x7fda5c10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 --2- 172.21.15.57:0/2847176509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0739e0 0x7fda5c10c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fda48009910 tx=0x7fda48009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 -- 172.21.15.57:0/2847176509 shutdown_connections 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 --2- 172.21.15.57:0/2847176509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0739e0 0x7fda5c10c5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 --2- 172.21.15.57:0/2847176509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0730d0 0x7fda5c0734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 -- 172.21.15.57:0/2847176509 >> 172.21.15.57:0/2847176509 conn(0x7fda5c0fbda0 msgr2=0x7fda5c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:14.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.888+0000 7fda63e00700 1 -- 172.21.15.57:0/2847176509 shutdown_connections 2024-02-28T23:20:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.889+0000 7fda63e00700 1 -- 172.21.15.57:0/2847176509 wait complete. 2024-02-28T23:20:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 Processor -- start 2024-02-28T23:20:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 -- start start 2024-02-28T23:20:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0739e0 0x7fda5c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c198d90 con 0x7fda5c0730d0 2024-02-28T23:20:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda63e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c198ed0 con 0x7fda5c0739e0 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.890+0000 7fda61400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36240/0 (socket says 172.21.15.57:36240) 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 -- 172.21.15.57:0/204000403 learned_addr learned my addr 172.21.15.57:0/204000403 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda60a00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0739e0 0x7fda5c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 -- 172.21.15.57:0/204000403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0739e0 msgr2=0x7fda5c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0739e0 0x7fda5c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:14.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 -- 172.21.15.57:0/204000403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda480095f0 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.891+0000 7fda61400700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c930 tx=0x7fda5000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.892+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda50007ad0 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.892+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda50007c30 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.892+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda50018610 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.892+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c19cd20 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.892+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c19d270 con 0x7fda5c0730d0 2024-02-28T23:20:14.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.893+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c109ca0 con 0x7fda5c0730d0 2024-02-28T23:20:14.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.894+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fda5000f450 con 0x7fda5c0730d0 2024-02-28T23:20:14.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.895+0000 7fda5a000700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 0x7fda4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:14.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.895+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda5008d720 con 0x7fda5c0730d0 2024-02-28T23:20:14.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.895+0000 7fda60a00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 0x7fda4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:14.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.896+0000 7fda60a00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 0x7fda4406fd50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda5c199820 tx=0x7fda4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:14.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:14.900+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda50056eb0 con 0x7fda5c0730d0 2024-02-28T23:20:15.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.137+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda5c199670 con 0x7fda4406d8a0 2024-02-28T23:20:15.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.169+0000 7fda5a000700 1 -- 172.21.15.57:0/204000403 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fda5c199670 con 0x7fda4406d8a0 2024-02-28T23:20:15.168 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (9m) 79s ago 23m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (20m) 79s ago 24m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (22m) 55s ago 22m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (5m) 79s ago 24m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (5m) 55s ago 22m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (9m) 79s ago 23m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (113s) 79s ago 16m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (87s) 79s ago 16m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (16m) 55s ago 16m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (60s) 55s ago 16m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (12m) 79s ago 25m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (11m) 55s ago 22m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (7m) 79s ago 25m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:20:15.169 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (7m) 55s ago 22m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (10m) 79s ago 23m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (10m) 55s ago 22m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (5m) 79s ago 21m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (4m) 79s ago 20m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (4m) 79s ago 19m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (3m) 55s ago 19m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (2m) 55s ago 18m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (2m) 55s ago 18m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:20:15.170 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (10m) 79s ago 23m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:20:15.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.171+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 msgr2=0x7fda4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.171+0000 7fda63e00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 0x7fda4406fd50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda5c199820 tx=0x7fda4801a040 comp rx=0 tx=0).stop 2024-02-28T23:20:15.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 msgr2=0x7fda5c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c930 tx=0x7fda5000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 shutdown_connections 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda4406d8a0 0x7fda4406fd50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda5c0730d0 0x7fda5c198200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 --2- 172.21.15.57:0/204000403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda5c0739e0 0x7fda5c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 >> 172.21.15.57:0/204000403 conn(0x7fda5c0fbda0 msgr2=0x7fda5c106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 shutdown_connections 2024-02-28T23:20:15.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.172+0000 7fda63e00700 1 -- 172.21.15.57:0/204000403 wait complete. 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 -- 172.21.15.57:0/645403692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 msgr2=0x7f1cbc110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 --2- 172.21.15.57:0/645403692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc110a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009940 tx=0x7f1ca8009c50 comp rx=0 tx=0).stop 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 -- 172.21.15.57:0/645403692 shutdown_connections 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 --2- 172.21.15.57:0/645403692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc110a10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 --2- 172.21.15.57:0/645403692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 -- 172.21.15.57:0/645403692 >> 172.21.15.57:0/645403692 conn(0x7f1cbc075130 msgr2=0x7f1cbc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 -- 172.21.15.57:0/645403692 shutdown_connections 2024-02-28T23:20:15.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.328+0000 7f1cc4200700 1 -- 172.21.15.57:0/645403692 wait complete. 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.329+0000 7f1cc4200700 1 Processor -- start 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.329+0000 7f1cc4200700 1 -- start start 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc4200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc4200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc4200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc19d080 con 0x7f1cbc068ee0 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc4200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1a0e10 con 0x7f1cbc068540 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37408/0 (socket says 172.21.15.57:37408) 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 -- 172.21.15.57:0/3997535182 learned_addr learned my addr 172.21.15.57:0/3997535182 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc1800700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 -- 172.21.15.57:0/3997535182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 msgr2=0x7f1cbc19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.330+0000 7f1cc0e00700 1 -- 172.21.15.57:0/3997535182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca80095f0 con 0x7f1cbc068ee0 2024-02-28T23:20:15.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cc1800700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:20:15.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cc0e00700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80055f0 tx=0x7f1ca80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca801d070 con 0x7f1cbc068ee0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca800f460 con 0x7f1cbc068ee0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc1a1090 con 0x7f1cbc068ee0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca8021640 con 0x7f1cbc068ee0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.331+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc1a15e0 con 0x7f1cbc068ee0 2024-02-28T23:20:15.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.332+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc101410 con 0x7f1cbc068ee0 2024-02-28T23:20:15.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.334+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1ca800f5d0 con 0x7f1cbc068ee0 2024-02-28T23:20:15.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.334+0000 7f1cba000700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 0x7f1ca406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.334+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ca808e860 con 0x7f1cbc068ee0 2024-02-28T23:20:15.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.335+0000 7f1cc1800700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 0x7f1ca406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.335+0000 7f1cc1800700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 0x7f1ca406fd50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00094f0 tx=0x7f1cb0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:15.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.339+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ca8057f70 con 0x7f1cbc068ee0 2024-02-28T23:20:15.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.649+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1cbc04f2a0 con 0x7f1cbc068ee0 2024-02-28T23:20:15.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.650+0000 7f1cba000700 1 -- 172.21.15.57:0/3997535182 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f1ca805b630 con 0x7f1cbc068ee0 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:15.650 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:20:15.651 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:20:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 msgr2=0x7f1ca406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 0x7f1ca406fd50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00094f0 tx=0x7f1cb0009040 comp rx=0 tx=0).stop 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 msgr2=0x7f1cbc19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80055f0 tx=0x7f1ca80056d0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 shutdown_connections 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1ca406d8a0 0x7f1ca406fd50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc068540 0x7f1cbc19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 --2- 172.21.15.57:0/3997535182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cbc068ee0 0x7f1cbc19c9a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 >> 172.21.15.57:0/3997535182 conn(0x7f1cbc075130 msgr2=0x7f1cbc10eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 shutdown_connections 2024-02-28T23:20:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.653+0000 7f1cc4200700 1 -- 172.21.15.57:0/3997535182 wait complete. 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 -- 172.21.15.57:0/4020947786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 msgr2=0x7f2574108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 --2- 172.21.15.57:0/4020947786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574108840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2564009910 tx=0x7f2564009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 -- 172.21.15.57:0/4020947786 shutdown_connections 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 --2- 172.21.15.57:0/4020947786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 0x7f25741028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 --2- 172.21.15.57:0/4020947786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574108840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.811+0000 7f2579a00700 1 -- 172.21.15.57:0/4020947786 >> 172.21.15.57:0/4020947786 conn(0x7f25740fdf90 msgr2=0x7f25741003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:15.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.812+0000 7f2579a00700 1 -- 172.21.15.57:0/4020947786 shutdown_connections 2024-02-28T23:20:15.812 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:15 smithi057 ceph-mon[89491]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:15.812 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:15 smithi057 ceph-mon[89491]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:15.812 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:15 smithi057 ceph-mon[89491]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.812+0000 7f2579a00700 1 -- 172.21.15.57:0/4020947786 wait complete. 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 Processor -- start 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 -- start start 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 0x7f25741981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2574198e10 con 0x7f2574108470 2024-02-28T23:20:15.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.813+0000 7f2579a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257419cba0 con 0x7f2574102470 2024-02-28T23:20:15.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37422/0 (socket says 172.21.15.57:37422) 2024-02-28T23:20:15.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 -- 172.21.15.57:0/173298206 learned_addr learned my addr 172.21.15.57:0/173298206 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:15.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 0x7f25741981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 -- 172.21.15.57:0/173298206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 msgr2=0x7f25741981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 0x7f25741981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.814+0000 7f2572000700 1 -- 172.21.15.57:0/173298206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640095f0 con 0x7f2574108470 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f2572000700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f256000d8d0 tx=0x7f256000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2560009940 con 0x7f2574108470 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2560010460 con 0x7f2574108470 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257419ce80 con 0x7f2574108470 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256000f5f0 con 0x7f2574108470 2024-02-28T23:20:15.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.815+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257419d3d0 con 0x7f2574108470 2024-02-28T23:20:15.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.816+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257410a9e0 con 0x7f2574108470 2024-02-28T23:20:15.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.818+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f25600105d0 con 0x7f2574108470 2024-02-28T23:20:15.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.819+0000 7f256b400700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:15.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.819+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f256008d510 con 0x7f2574108470 2024-02-28T23:20:15.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.819+0000 7f2572a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:15.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.820+0000 7f2572a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 0x7f255806fd50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f256400b5c0 tx=0x7f256401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:15.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:15.824+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2560056ca0 con 0x7f2574108470 2024-02-28T23:20:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:15 smithi084 ceph-mon[70440]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:15 smithi084 ceph-mon[70440]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:15.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:15 smithi084 ceph-mon[70440]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:16.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.103+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f257404f2a0 con 0x7f2574108470 2024-02-28T23:20:16.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.104+0000 7f256b400700 1 -- 172.21.15.57:0/173298206 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f2560016740 con 0x7f2574108470 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:20:16.104 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:20:16.105 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:20:16.106 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 msgr2=0x7f255806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 0x7f255806fd50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f256400b5c0 tx=0x7f256401a040 comp rx=0 tx=0).stop 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 msgr2=0x7f2574198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f256000d8d0 tx=0x7f256000dc90 comp rx=0 tx=0).stop 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 shutdown_connections 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2574102470 0x7f25741981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 --2- 172.21.15.57:0/173298206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2574108470 0x7f2574198730 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 >> 172.21.15.57:0/173298206 conn(0x7f25740fdf90 msgr2=0x7f25741002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 shutdown_connections 2024-02-28T23:20:16.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.108+0000 7f2579a00700 1 -- 172.21.15.57:0/173298206 wait complete. 2024-02-28T23:20:16.109 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:20:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 -- 172.21.15.57:0/3080169743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 msgr2=0x7fb444108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 --2- 172.21.15.57:0/3080169743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444108840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb438009910 tx=0x7fb438009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 -- 172.21.15.57:0/3080169743 shutdown_connections 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 --2- 172.21.15.57:0/3080169743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 0x7fb4441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 --2- 172.21.15.57:0/3080169743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444108840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 -- 172.21.15.57:0/3080169743 >> 172.21.15.57:0/3080169743 conn(0x7fb4440fdf90 msgr2=0x7fb4441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 -- 172.21.15.57:0/3080169743 shutdown_connections 2024-02-28T23:20:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.267+0000 7fb44c600700 1 -- 172.21.15.57:0/3080169743 wait complete. 2024-02-28T23:20:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.268+0000 7fb44c600700 1 Processor -- start 2024-02-28T23:20:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.268+0000 7fb44c600700 1 -- start start 2024-02-28T23:20:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb44c600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 0x7fb4441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb44c600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb44c600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444198e10 con 0x7fb444108470 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb44c600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44419cba0 con 0x7fb444102470 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37436/0 (socket says 172.21.15.57:37436) 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 -- 172.21.15.57:0/3751981850 learned_addr learned my addr 172.21.15.57:0/3751981850 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449c00700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 0x7fb4441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 -- 172.21.15.57:0/3751981850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 msgr2=0x7fb4441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 0x7fb4441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.269+0000 7fb449200700 1 -- 172.21.15.57:0/3751981850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4380095f0 con 0x7fb444108470 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb449200700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c8a0 tx=0x7fb43000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb43000cea0 con 0x7fb444108470 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb430004830 con 0x7fb444108470 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4300055b0 con 0x7fb444108470 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44419ce80 con 0x7fb444108470 2024-02-28T23:20:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.270+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44419d3d0 con 0x7fb444108470 2024-02-28T23:20:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.271+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44410a9e0 con 0x7fb444108470 2024-02-28T23:20:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.272+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb4300049a0 con 0x7fb444108470 2024-02-28T23:20:16.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.273+0000 7fb442000700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.273+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb43008d190 con 0x7fb444108470 2024-02-28T23:20:16.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.273+0000 7fb449c00700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.274+0000 7fb449c00700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 0x7fb42c06fd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb438006010 tx=0x7fb43801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.278+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb430056920 con 0x7fb444108470 2024-02-28T23:20:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.514+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb444199660 con 0x7fb42c06d8a0 2024-02-28T23:20:16.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.516+0000 7fb442000700 1 -- 172.21.15.57:0/3751981850 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb444199660 con 0x7fb42c06d8a0 2024-02-28T23:20:16.516 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:20:16.516 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:20:16.516 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:20:16.517 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:20:16.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.519+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 msgr2=0x7fb42c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.519+0000 7fb44c600700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 0x7fb42c06fd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb438006010 tx=0x7fb43801a040 comp rx=0 tx=0).stop 2024-02-28T23:20:16.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 msgr2=0x7fb444198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c8a0 tx=0x7fb43000cc60 comp rx=0 tx=0).stop 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 shutdown_connections 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb444102470 0x7fb4441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 --2- 172.21.15.57:0/3751981850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444108470 0x7fb444198730 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 >> 172.21.15.57:0/3751981850 conn(0x7fb4440fdf90 msgr2=0x7fb4441002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 shutdown_connections 2024-02-28T23:20:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.520+0000 7fb44c600700 1 -- 172.21.15.57:0/3751981850 wait complete. 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 -- 172.21.15.57:0/778088290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ffd60 msgr2=0x7fb6ac103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 --2- 172.21.15.57:0/778088290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac103d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb698009910 tx=0x7fb698009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 -- 172.21.15.57:0/778088290 shutdown_connections 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 --2- 172.21.15.57:0/778088290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac103d30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 --2- 172.21.15.57:0/778088290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 -- 172.21.15.57:0/778088290 >> 172.21.15.57:0/778088290 conn(0x7fb6ac0fac80 msgr2=0x7fb6ac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 -- 172.21.15.57:0/778088290 shutdown_connections 2024-02-28T23:20:16.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.669+0000 7fb6b1200700 1 -- 172.21.15.57:0/778088290 wait complete. 2024-02-28T23:20:16.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.671+0000 7fb6b1200700 1 Processor -- start 2024-02-28T23:20:16.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.671+0000 7fb6b1200700 1 -- start start 2024-02-28T23:20:16.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6b1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6b1200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6b1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac197850 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6b1200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac197990 con 0x7fb6ac0ffd60 2024-02-28T23:20:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6a9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6a9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37446/0 (socket says 172.21.15.57:37446) 2024-02-28T23:20:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.672+0000 7fb6a9e00700 1 -- 172.21.15.57:0/2660223859 learned_addr learned my addr 172.21.15.57:0/2660223859 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9400700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9e00700 1 -- 172.21.15.57:0/2660223859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 msgr2=0x7fb6ac1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9e00700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9e00700 1 -- 172.21.15.57:0/2660223859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6980095f0 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9400700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:20:16.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a9e00700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c930 tx=0x7fb69c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb69c007ad0 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb69c007c30 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac19fdf0 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.673+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb69c018610 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.674+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac1a0200 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.675+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac107730 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:16 smithi057 ceph-mon[89491]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:16.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3997535182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:20:16.675 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/173298206' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:20:16.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.676+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb69c00f450 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.677+0000 7fb6a2a00700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 0x7fb69006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:16.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.677+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb69c08d810 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.677+0000 7fb6a9400700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 0x7fb69006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:16.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.678+0000 7fb6a9400700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 0x7fb69006fd50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb698006010 tx=0x7fb69800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:16.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.685+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb69c056f20 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:16 smithi084 ceph-mon[70440]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3997535182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:20:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/173298206' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:20:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.991+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb6ac107110 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.992+0000 7fb6a2a00700 1 -- 172.21.15.57:0/2660223859 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb69c05a5e0 con 0x7fb6ac0ff3c0 2024-02-28T23:20:16.991 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:20:16.991 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:20:16.992 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:20:16.992 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:20:16.992 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 msgr2=0x7fb69006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 0x7fb69006fd50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb698006010 tx=0x7fb69800b540 comp rx=0 tx=0).stop 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 msgr2=0x7fb6ac196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c930 tx=0x7fb69c00ccf0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 shutdown_connections 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb69006d8a0 0x7fb69006fd50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6ac0ff3c0 0x7fb6ac196d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 --2- 172.21.15.57:0/2660223859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6ac0ffd60 0x7fb6ac1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.995+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 >> 172.21.15.57:0/2660223859 conn(0x7fb6ac0fac80 msgr2=0x7fb6ac0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.996+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 shutdown_connections 2024-02-28T23:20:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:16.996+0000 7fb6b1200700 1 -- 172.21.15.57:0/2660223859 wait complete. 2024-02-28T23:20:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:17 smithi057 ceph-mon[89491]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:17 smithi057 ceph-mon[89491]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:17.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2660223859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:20:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:17 smithi084 ceph-mon[70440]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:17 smithi084 ceph-mon[70440]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2660223859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:20:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:19 smithi057 ceph-mon[89491]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:19.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:19 smithi084 ceph-mon[70440]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:21 smithi057 ceph-mon[89491]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:21.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:21 smithi084 ceph-mon[70440]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:23 smithi057 ceph-mon[89491]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:23 smithi084 ceph-mon[70440]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:25 smithi057 ceph-mon[89491]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:25.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:25 smithi084 ceph-mon[70440]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:27 smithi057 ceph-mon[89491]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:27 smithi084 ceph-mon[70440]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:29 smithi057 ceph-mon[89491]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:29 smithi084 ceph-mon[70440]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:31 smithi057 ceph-mon[89491]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:31 smithi084 ceph-mon[70440]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:33 smithi057 ceph-mon[89491]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:33.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:33 smithi084 ceph-mon[70440]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:35 smithi057 ceph-mon[89491]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:35 smithi084 ceph-mon[70440]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:37.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:37 smithi057 ceph-mon[89491]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:37 smithi084 ceph-mon[70440]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:39.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:39 smithi057 ceph-mon[89491]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:39.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:39 smithi084 ceph-mon[70440]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:41 smithi057 ceph-mon[89491]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:41.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:41 smithi084 ceph-mon[70440]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:43.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:43 smithi057 ceph-mon[89491]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:43.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:43 smithi084 ceph-mon[70440]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:45.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:45 smithi057 ceph-mon[89491]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:45.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:45 smithi084 ceph-mon[70440]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 -- 172.21.15.57:0/490582762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 msgr2=0x7f31d4108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 --2- 172.21.15.57:0/490582762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4108840 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009910 tx=0x7f31c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 -- 172.21.15.57:0/490582762 shutdown_connections 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 --2- 172.21.15.57:0/490582762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 0x7f31d41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 --2- 172.21.15.57:0/490582762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4108840 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 -- 172.21.15.57:0/490582762 >> 172.21.15.57:0/490582762 conn(0x7f31d40fdf90 msgr2=0x7f31d41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 -- 172.21.15.57:0/490582762 shutdown_connections 2024-02-28T23:20:47.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.172+0000 7f31d9800700 1 -- 172.21.15.57:0/490582762 wait complete. 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.173+0000 7f31d9800700 1 Processor -- start 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.173+0000 7f31d9800700 1 -- start start 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d9800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 0x7f31d41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d9800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d9800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4198e10 con 0x7f31d4108470 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d9800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d419cba0 con 0x7f31d4102470 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54716/0 (socket says 172.21.15.57:54716) 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d2000700 1 -- 172.21.15.57:0/2369535711 learned_addr learned my addr 172.21.15.57:0/2369535711 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.174+0000 7f31d2a00700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 0x7f31d41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31d2000700 1 -- 172.21.15.57:0/2369535711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 msgr2=0x7f31d41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31d2000700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 0x7f31d41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31d2000700 1 -- 172.21.15.57:0/2369535711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c40095f0 con 0x7f31d4108470 2024-02-28T23:20:47.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31d2000700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f31c000d8d0 tx=0x7f31c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c0009940 con 0x7f31d4108470 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31c0010460 con 0x7f31d4108470 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d419ce80 con 0x7f31d4108470 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.175+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c000f5f0 con 0x7f31d4108470 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.176+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d419d3d0 con 0x7f31d4108470 2024-02-28T23:20:47.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.177+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d410a9e0 con 0x7f31d4108470 2024-02-28T23:20:47.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.178+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f31c00105d0 con 0x7f31d4108470 2024-02-28T23:20:47.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.179+0000 7f31cb400700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.179+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f31c008d4c0 con 0x7f31d4108470 2024-02-28T23:20:47.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.179+0000 7f31d2a00700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.180+0000 7f31d2a00700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 0x7f31b806fd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006010 tx=0x7f31c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:47.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.184+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31c0056c50 con 0x7f31d4108470 2024-02-28T23:20:47.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.433+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d4199660 con 0x7f31b806d8a0 2024-02-28T23:20:47.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.436+0000 7f31cb400700 1 -- 172.21.15.57:0/2369535711 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31d4199660 con 0x7f31b806d8a0 2024-02-28T23:20:47.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 msgr2=0x7f31b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 0x7f31b806fd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006010 tx=0x7f31c401a040 comp rx=0 tx=0).stop 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 msgr2=0x7f31d4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f31c000d8d0 tx=0x7f31c000dc90 comp rx=0 tx=0).stop 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 shutdown_connections 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d4102470 0x7f31d41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 --2- 172.21.15.57:0/2369535711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31d4108470 0x7f31d4198730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 >> 172.21.15.57:0/2369535711 conn(0x7f31d40fdf90 msgr2=0x7f31d41002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 shutdown_connections 2024-02-28T23:20:47.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.439+0000 7f31d9800700 1 -- 172.21.15.57:0/2369535711 wait complete. 2024-02-28T23:20:47.454 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 -- 172.21.15.57:0/1704497247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 msgr2=0x7f0f24104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1704497247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24104e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009910 tx=0x7f0f10009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 -- 172.21.15.57:0/1704497247 shutdown_connections 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1704497247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24104e50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1704497247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 0x7f0f241008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.602+0000 7f0f2b400700 1 -- 172.21.15.57:0/1704497247 >> 172.21.15.57:0/1704497247 conn(0x7f0f240fbda0 msgr2=0x7f0f240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.603+0000 7f0f2b400700 1 -- 172.21.15.57:0/1704497247 shutdown_connections 2024-02-28T23:20:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.603+0000 7f0f2b400700 1 -- 172.21.15.57:0/1704497247 wait complete. 2024-02-28T23:20:47.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 Processor -- start 2024-02-28T23:20:47.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 -- start start 2024-02-28T23:20:47.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 0x7f0f241980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24198d00 con 0x7f0f24100e80 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.604+0000 7f0f2b400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2419ca90 con 0x7f0f241004e0 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54736/0 (socket says 172.21.15.57:54736) 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 -- 172.21.15.57:0/1993016458 learned_addr learned my addr 172.21.15.57:0/1993016458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f28a00700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 0x7f0f241980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 -- 172.21.15.57:0/1993016458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 msgr2=0x7f0f241980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 0x7f0f241980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.605+0000 7f0f23e00700 1 -- 172.21.15.57:0/1993016458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100095f0 con 0x7f0f24100e80 2024-02-28T23:20:47.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f23e00700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000bda0 tx=0x7f0f1000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:47.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1001d070 con 0x7f0f24100e80 2024-02-28T23:20:47.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f1000f460 con 0x7f0f24100e80 2024-02-28T23:20:47.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f10017610 con 0x7f0f24100e80 2024-02-28T23:20:47.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2419cd10 con 0x7f0f24100e80 2024-02-28T23:20:47.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.606+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2419d200 con 0x7f0f24100e80 2024-02-28T23:20:47.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.607+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f241045d0 con 0x7f0f24100e80 2024-02-28T23:20:47.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.608+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0f1000fa40 con 0x7f0f24100e80 2024-02-28T23:20:47.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.609+0000 7f0f21600700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 0x7f0f0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:47.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.609+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f1008e9a0 con 0x7f0f24100e80 2024-02-28T23:20:47.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.610+0000 7f0f28a00700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 0x7f0f0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:47.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.610+0000 7f0f28a00700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 0x7f0f0c06fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005950 tx=0x7f0f180058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:47.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.614+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f10058130 con 0x7f0f24100e80 2024-02-28T23:20:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:47 smithi057 ceph-mon[89491]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:47.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:47 smithi084 ceph-mon[70440]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:47.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.871+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f2402cca0 con 0x7f0f0c06d8a0 2024-02-28T23:20:47.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.874+0000 7f0f21600700 1 -- 172.21.15.57:0/1993016458 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0f2402cca0 con 0x7f0f0c06d8a0 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 msgr2=0x7f0f0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 0x7f0f0c06fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005950 tx=0x7f0f180058e0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 msgr2=0x7f0f24198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000bda0 tx=0x7f0f1000be80 comp rx=0 tx=0).stop 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 shutdown_connections 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f0c06d8a0 0x7f0f0c06fd50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f241004e0 0x7f0f241980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 --2- 172.21.15.57:0/1993016458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f24100e80 0x7f0f24198620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 >> 172.21.15.57:0/1993016458 conn(0x7f0f240fbda0 msgr2=0x7f0f240fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 shutdown_connections 2024-02-28T23:20:47.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:47.877+0000 7f0f2b400700 1 -- 172.21.15.57:0/1993016458 wait complete. 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 -- 172.21.15.57:0/691675022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 msgr2=0x7f304c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 --2- 172.21.15.57:0/691675022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c10ca30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f304000b0a0 tx=0x7f304000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 -- 172.21.15.57:0/691675022 shutdown_connections 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 --2- 172.21.15.57:0/691675022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 --2- 172.21.15.57:0/691675022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c10ca30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 -- 172.21.15.57:0/691675022 >> 172.21.15.57:0/691675022 conn(0x7f304c075e90 msgr2=0x7f304c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 -- 172.21.15.57:0/691675022 shutdown_connections 2024-02-28T23:20:48.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.038+0000 7f3052400700 1 -- 172.21.15.57:0/691675022 wait complete. 2024-02-28T23:20:48.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.039+0000 7f3052400700 1 Processor -- start 2024-02-28T23:20:48.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f3052400700 1 -- start start 2024-02-28T23:20:48.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f3052400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f3052400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f3052400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c19cea0 con 0x7f304c10c660 2024-02-28T23:20:48.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f3052400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1a0c30 con 0x7f304c106b80 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f304be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f304be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54754/0 (socket says 172.21.15.57:54754) 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.040+0000 7f304be00700 1 -- 172.21.15.57:0/636757843 learned_addr learned my addr 172.21.15.57:0/636757843 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3051000700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f304be00700 1 -- 172.21.15.57:0/636757843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 msgr2=0x7f304c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f304be00700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f304be00700 1 -- 172.21.15.57:0/636757843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3040009d00 con 0x7f304c10c660 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3051000700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f304be00700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f303800d8d0 tx=0x7f303800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3038009940 con 0x7f304c10c660 2024-02-28T23:20:48.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3038010460 con 0x7f304c10c660 2024-02-28T23:20:48.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c1a0f10 con 0x7f304c10c660 2024-02-28T23:20:48.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.041+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303800f5f0 con 0x7f304c10c660 2024-02-28T23:20:48.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.042+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1a1460 con 0x7f304c10c660 2024-02-28T23:20:48.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.043+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c10eb40 con 0x7f304c10c660 2024-02-28T23:20:48.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.044+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f30380105d0 con 0x7f304c10c660 2024-02-28T23:20:48.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.044+0000 7f3049600700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 0x7f303406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.045+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f303808d4c0 con 0x7f304c10c660 2024-02-28T23:20:48.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.045+0000 7f3051000700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 0x7f303406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.045+0000 7f3051000700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 0x7f303406fd50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3040009480 tx=0x7f3040000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.049+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3038056c50 con 0x7f304c10c660 2024-02-28T23:20:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.294+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f304c1a1740 con 0x7f303406d8a0 2024-02-28T23:20:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.307+0000 7f3049600700 1 -- 172.21.15.57:0/636757843 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f304c1a1740 con 0x7f303406d8a0 2024-02-28T23:20:48.305 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:20:48.305 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (10m) 113s ago 24m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:20:48.305 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (20m) 113s ago 24m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (23m) 88s ago 23m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (6m) 113s ago 24m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (6m) 88s ago 23m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (9m) 113s ago 23m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (2m) 113s ago 17m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (2m) 113s ago 16m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (16m) 88s ago 16m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (93s) 88s ago 17m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (12m) 113s ago 26m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (11m) 88s ago 22m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (8m) 113s ago 26m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (7m) 88s ago 22m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:20:48.306 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (11m) 113s ago 24m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (11m) 88s ago 22m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (5m) 113s ago 21m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (5m) 113s ago 21m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (4m) 113s ago 20m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (4m) 88s ago 19m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (3m) 88s ago 19m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (2m) 88s ago 18m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:20:48.307 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (10m) 113s ago 23m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:20:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.310+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 msgr2=0x7f303406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.310+0000 7f3052400700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 0x7f303406fd50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3040009480 tx=0x7f3040000f40 comp rx=0 tx=0).stop 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.310+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 msgr2=0x7f304c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.310+0000 7f3052400700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f303800d8d0 tx=0x7f303800dc90 comp rx=0 tx=0).stop 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 shutdown_connections 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f303406d8a0 0x7f303406fd50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c106b80 0x7f304c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 --2- 172.21.15.57:0/636757843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f304c10c660 0x7f304c19c7c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 >> 172.21.15.57:0/636757843 conn(0x7f304c075e90 msgr2=0x7f304c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 shutdown_connections 2024-02-28T23:20:48.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.311+0000 7f3052400700 1 -- 172.21.15.57:0/636757843 wait complete. 2024-02-28T23:20:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 -- 172.21.15.57:0/1910296020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 msgr2=0x7fe0a4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/1910296020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4068b70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe09400b0a0 tx=0x7fe09400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 -- 172.21.15.57:0/1910296020 shutdown_connections 2024-02-28T23:20:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/1910296020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 0x7fe0a4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/1910296020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4068b70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.466+0000 7fe0a8c00700 1 -- 172.21.15.57:0/1910296020 >> 172.21.15.57:0/1910296020 conn(0x7fe0a4075130 msgr2=0x7fe0a4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.467+0000 7fe0a8c00700 1 -- 172.21.15.57:0/1910296020 shutdown_connections 2024-02-28T23:20:48.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.467+0000 7fe0a8c00700 1 -- 172.21.15.57:0/1910296020 wait complete. 2024-02-28T23:20:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 Processor -- start 2024-02-28T23:20:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 -- start start 2024-02-28T23:20:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 0x7fe0a4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4198d30 con 0x7fe0a40687a0 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a8c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a419cac0 con 0x7fe0a4069140 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54786/0 (socket says 172.21.15.57:54786) 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.468+0000 7fe0a1e00700 1 -- 172.21.15.57:0/2627131433 learned_addr learned my addr 172.21.15.57:0/2627131433 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe099400700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 0x7fe0a4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe0a1e00700 1 -- 172.21.15.57:0/2627131433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 msgr2=0x7fe0a4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe0a1e00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 0x7fe0a4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe0a1e00700 1 -- 172.21.15.57:0/2627131433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009d00 con 0x7fe0a40687a0 2024-02-28T23:20:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe0a1e00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe09400b070 tx=0x7fe09400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09400e040 con 0x7fe0a40687a0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe094003e60 con 0x7fe0a40687a0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.469+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a419cd40 con 0x7fe0a40687a0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.470+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09401d910 con 0x7fe0a40687a0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.470+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a419d230 con 0x7fe0a40687a0 2024-02-28T23:20:48.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.471+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a404f2a0 con 0x7fe0a40687a0 2024-02-28T23:20:48.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.472+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe094004820 con 0x7fe0a40687a0 2024-02-28T23:20:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.473+0000 7fe09b400700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 0x7fe08806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.473+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe09408eec0 con 0x7fe0a40687a0 2024-02-28T23:20:48.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.473+0000 7fe099400700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 0x7fe08806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.474+0000 7fe099400700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 0x7fe08806fd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005fd0 tx=0x7fe08c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.478+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0940585d0 con 0x7fe0a40687a0 2024-02-28T23:20:48.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.787+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0a419d510 con 0x7fe0a40687a0 2024-02-28T23:20:48.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.788+0000 7fe09b400700 1 -- 172.21.15.57:0/2627131433 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fe09405bc90 con 0x7fe0a40687a0 2024-02-28T23:20:48.786 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:20:48.787 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:20:48.788 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:20:48.788 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:20:48.788 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:20:48.788 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:20:48.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.790+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 msgr2=0x7fe08806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.790+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 0x7fe08806fd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005fd0 tx=0x7fe08c00a560 comp rx=0 tx=0).stop 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.790+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 msgr2=0x7fe0a4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.790+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe09400b070 tx=0x7fe09400cdd0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 shutdown_connections 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe08806d8a0 0x7fe08806fd50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0a40687a0 0x7fe0a4198110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 --2- 172.21.15.57:0/2627131433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4069140 0x7fe0a4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 >> 172.21.15.57:0/2627131433 conn(0x7fe0a4075130 msgr2=0x7fe0a4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 shutdown_connections 2024-02-28T23:20:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.791+0000 7fe0a8c00700 1 -- 172.21.15.57:0/2627131433 wait complete. 2024-02-28T23:20:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:48 smithi057 ceph-mon[89491]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:48.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:48 smithi084 ceph-mon[70440]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:48.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:20:48.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 -- 172.21.15.57:0/1481780024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 msgr2=0x7fa51c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 --2- 172.21.15.57:0/1481780024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c104e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa508009910 tx=0x7fa508009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:48.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 -- 172.21.15.57:0/1481780024 shutdown_connections 2024-02-28T23:20:48.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 --2- 172.21.15.57:0/1481780024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c104e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 --2- 172.21.15.57:0/1481780024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 0x7fa51c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.942+0000 7fa524600700 1 -- 172.21.15.57:0/1481780024 >> 172.21.15.57:0/1481780024 conn(0x7fa51c0fbda0 msgr2=0x7fa51c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:48.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.943+0000 7fa524600700 1 -- 172.21.15.57:0/1481780024 shutdown_connections 2024-02-28T23:20:48.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.943+0000 7fa524600700 1 -- 172.21.15.57:0/1481780024 wait complete. 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 Processor -- start 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 -- start start 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 0x7fa51c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c198ca0 con 0x7fa51c100e80 2024-02-28T23:20:48.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa524600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c198de0 con 0x7fa51c1004e0 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa521200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa521200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54802/0 (socket says 172.21.15.57:54802) 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.944+0000 7fa521200700 1 -- 172.21.15.57:0/4129723020 learned_addr learned my addr 172.21.15.57:0/4129723020 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.945+0000 7fa521c00700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 0x7fa51c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.945+0000 7fa521200700 1 -- 172.21.15.57:0/4129723020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 msgr2=0x7fa51c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.945+0000 7fa521200700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 0x7fa51c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.945+0000 7fa521200700 1 -- 172.21.15.57:0/4129723020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080095f0 con 0x7fa51c100e80 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.945+0000 7fa521200700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa508005f50 tx=0x7fa50800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.946+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa50801d070 con 0x7fa51c100e80 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.946+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa50800f460 con 0x7fa51c100e80 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.946+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c19cbd0 con 0x7fa51c100e80 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.946+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa508017630 con 0x7fa51c100e80 2024-02-28T23:20:48.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.946+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c19d0c0 con 0x7fa51c100e80 2024-02-28T23:20:48.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.947+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c1045d0 con 0x7fa51c100e80 2024-02-28T23:20:48.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.948+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa50800f5d0 con 0x7fa51c100e80 2024-02-28T23:20:48.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.949+0000 7fa51a000700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:48.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.949+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa50808ebb0 con 0x7fa51c100e80 2024-02-28T23:20:48.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.949+0000 7fa521c00700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:48.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.950+0000 7fa521c00700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 0x7fa50406fd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e270 tx=0x7fa510009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:48.954+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5080582c0 con 0x7fa51c100e80 2024-02-28T23:20:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.221+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa51c04f2a0 con 0x7fa51c100e80 2024-02-28T23:20:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.222+0000 7fa51a000700 1 -- 172.21.15.57:0/4129723020 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fa50805b980 con 0x7fa51c100e80 2024-02-28T23:20:49.222 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:20:49.222 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:20:49.222 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:20:49.223 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:20:49.224 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:20:49.225 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:20:49.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 msgr2=0x7fa50406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 0x7fa50406fd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e270 tx=0x7fa510009040 comp rx=0 tx=0).stop 2024-02-28T23:20:49.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 msgr2=0x7fa51c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa508005f50 tx=0x7fa50800be20 comp rx=0 tx=0).stop 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 shutdown_connections 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa51c1004e0 0x7fa51c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 --2- 172.21.15.57:0/4129723020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa51c100e80 0x7fa51c198650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.225+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 >> 172.21.15.57:0/4129723020 conn(0x7fa51c0fbda0 msgr2=0x7fa51c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.226+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 shutdown_connections 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.226+0000 7fa524600700 1 -- 172.21.15.57:0/4129723020 wait complete. 2024-02-28T23:20:49.227 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.375+0000 7fdb75400700 1 -- 172.21.15.57:0/114725345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 msgr2=0x7fdb70068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.375+0000 7fdb75400700 1 --2- 172.21.15.57:0/114725345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb70068b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009910 tx=0x7fdb60009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 -- 172.21.15.57:0/114725345 shutdown_connections 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 --2- 172.21.15.57:0/114725345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 0x7fdb70104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 --2- 172.21.15.57:0/114725345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb70068b70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 -- 172.21.15.57:0/114725345 >> 172.21.15.57:0/114725345 conn(0x7fdb70075130 msgr2=0x7fdb70075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 -- 172.21.15.57:0/114725345 shutdown_connections 2024-02-28T23:20:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.376+0000 7fdb75400700 1 -- 172.21.15.57:0/114725345 wait complete. 2024-02-28T23:20:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 Processor -- start 2024-02-28T23:20:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 -- start start 2024-02-28T23:20:49.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 0x7fdb701001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70100720 con 0x7fdb700687a0 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.377+0000 7fdb75400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70100860 con 0x7fdb70069140 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54806/0 (socket says 172.21.15.57:54806) 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 -- 172.21.15.57:0/965921301 learned_addr learned my addr 172.21.15.57:0/965921301 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb67e00700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 0x7fdb701001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 -- 172.21.15.57:0/965921301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 msgr2=0x7fdb701001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 0x7fdb701001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 -- 172.21.15.57:0/965921301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600095f0 con 0x7fdb700687a0 2024-02-28T23:20:49.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.378+0000 7fdb6e800700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb60006010 tx=0x7fdb6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.379+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb6001d070 con 0x7fdb700687a0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.379+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb6000f460 con 0x7fdb700687a0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.379+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb70100ae0 con 0x7fdb700687a0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.379+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb60017630 con 0x7fdb700687a0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.379+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb701a23e0 con 0x7fdb700687a0 2024-02-28T23:20:49.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.380+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70108790 con 0x7fdb700687a0 2024-02-28T23:20:49.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.381+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdb6000f5d0 con 0x7fdb700687a0 2024-02-28T23:20:49.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.382+0000 7fdb67400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 0x7fdb5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.382+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb6008e9c0 con 0x7fdb700687a0 2024-02-28T23:20:49.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.382+0000 7fdb67e00700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 0x7fdb5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.383+0000 7fdb67e00700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 0x7fdb5406fd50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005950 tx=0x7fdb580058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:49.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.387+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb60058170 con 0x7fdb700687a0 2024-02-28T23:20:49.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.634+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb70061190 con 0x7fdb5406d8a0 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.636+0000 7fdb67400700 1 -- 172.21.15.57:0/965921301 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdb70061190 con 0x7fdb5406d8a0 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:20:49.635 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:20:49.636 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:20:49.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 msgr2=0x7fdb5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 0x7fdb5406fd50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005950 tx=0x7fdb580058e0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 msgr2=0x7fdb700ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb60006010 tx=0x7fdb6000be20 comp rx=0 tx=0).stop 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 shutdown_connections 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb5406d8a0 0x7fdb5406fd50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb700687a0 0x7fdb700ffca0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 --2- 172.21.15.57:0/965921301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb70069140 0x7fdb701001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 >> 172.21.15.57:0/965921301 conn(0x7fdb70075130 msgr2=0x7fdb700fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.639+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 shutdown_connections 2024-02-28T23:20:49.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.640+0000 7fdb75400700 1 -- 172.21.15.57:0/965921301 wait complete. 2024-02-28T23:20:49.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.800+0000 7efc40800700 1 -- 172.21.15.57:0/1854690240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc38106b80 msgr2=0x7efc38106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.800+0000 7efc40800700 1 --2- 172.21.15.57:0/1854690240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc38106b80 0x7efc38106ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc28009910 tx=0x7efc28009c20 comp rx=0 tx=0).stop 2024-02-28T23:20:49.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 -- 172.21.15.57:0/1854690240 shutdown_connections 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 --2- 172.21.15.57:0/1854690240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc38106b80 0x7efc38106ff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 --2- 172.21.15.57:0/1854690240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3810c8f0 0x7efc3810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 -- 172.21.15.57:0/1854690240 >> 172.21.15.57:0/1854690240 conn(0x7efc38076060 msgr2=0x7efc38078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 -- 172.21.15.57:0/1854690240 shutdown_connections 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.801+0000 7efc40800700 1 -- 172.21.15.57:0/1854690240 wait complete. 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 Processor -- start 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 -- start start 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc38106b80 0x7efc3819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3819d060 con 0x7efc3810c8f0 2024-02-28T23:20:49.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.802+0000 7efc40800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381a0df0 con 0x7efc38106b80 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54822/0 (socket says 172.21.15.57:54822) 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 -- 172.21.15.57:0/3937522015 learned_addr learned my addr 172.21.15.57:0/3937522015 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3de00700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc38106b80 0x7efc3819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 -- 172.21.15.57:0/3937522015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc38106b80 msgr2=0x7efc3819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc38106b80 0x7efc3819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 -- 172.21.15.57:0/3937522015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280095f0 con 0x7efc3810c8f0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.803+0000 7efc3d400700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc280092e0 tx=0x7efc2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.804+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2801d070 con 0x7efc3810c8f0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.804+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc28005e00 con 0x7efc3810c8f0 2024-02-28T23:20:49.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.804+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc28017400 con 0x7efc3810c8f0 2024-02-28T23:20:49.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.804+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc381a1070 con 0x7efc3810c8f0 2024-02-28T23:20:49.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.804+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc381a1560 con 0x7efc3810c8f0 2024-02-28T23:20:49.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.805+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3810ecc0 con 0x7efc3810c8f0 2024-02-28T23:20:49.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.806+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efc2800f460 con 0x7efc3810c8f0 2024-02-28T23:20:49.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.807+0000 7efc32a00700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 0x7efc2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:20:49.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.808+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc2808e990 con 0x7efc3810c8f0 2024-02-28T23:20:49.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.808+0000 7efc3de00700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 0x7efc2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:20:49.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.808+0000 7efc3de00700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 0x7efc2006fd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005950 tx=0x7efc2c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:20:49.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:49.812+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc280581c0 con 0x7efc3810c8f0 2024-02-28T23:20:49.811 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:49 smithi057 ceph-mon[89491]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:49.811 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:49 smithi057 ceph-mon[89491]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:49.811 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:49 smithi057 ceph-mon[89491]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:49.811 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:49 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2627131433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:20:49.811 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:49 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4129723020' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:20:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:49 smithi084 ceph-mon[70440]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:49 smithi084 ceph-mon[70440]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:49 smithi084 ceph-mon[70440]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:49 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2627131433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:20:49.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:49 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4129723020' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:20:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.130+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc3802cc30 con 0x7efc3810c8f0 2024-02-28T23:20:50.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.131+0000 7efc32a00700 1 -- 172.21.15.57:0/3937522015 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7efc2805b880 con 0x7efc3810c8f0 2024-02-28T23:20:50.129 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:20:50.129 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:20:50.129 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:20:50.129 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:20:50.130 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:20:50.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 msgr2=0x7efc2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:50.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 0x7efc2006fd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005950 tx=0x7efc2c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:20:50.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 msgr2=0x7efc3819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:20:50.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc280092e0 tx=0x7efc2800be20 comp rx=0 tx=0).stop 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 shutdown_connections 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2006d8a0 0x7efc2006fd50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc38106b80 0x7efc3819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 --2- 172.21.15.57:0/3937522015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc3810c8f0 0x7efc3819c980 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 >> 172.21.15.57:0/3937522015 conn(0x7efc38076060 msgr2=0x7efc38077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 shutdown_connections 2024-02-28T23:20:50.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:20:50.134+0000 7efc40800700 1 -- 172.21.15.57:0/3937522015 wait complete. 2024-02-28T23:20:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:50 smithi084 ceph-mon[70440]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:50 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3937522015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:20:50.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:50 smithi057 ceph-mon[89491]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:20:50.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:50 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3937522015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:20:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:51 smithi084 ceph-mon[70440]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:51 smithi057 ceph-mon[89491]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:53 smithi057 ceph-mon[89491]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:53 smithi084 ceph-mon[70440]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:55 smithi057 ceph-mon[89491]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:55 smithi084 ceph-mon[70440]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:57 smithi057 ceph-mon[89491]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:20:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:57 smithi084 ceph-mon[70440]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:20:59 smithi057 ceph-mon[89491]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:20:59 smithi084 ceph-mon[70440]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:01 smithi057 ceph-mon[89491]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:01 smithi084 ceph-mon[70440]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:03.883 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:03 smithi084 ceph-mon[70440]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:03.883 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:03 smithi057 ceph-mon[89491]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:05 smithi057 ceph-mon[89491]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:05 smithi084 ceph-mon[70440]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:07 smithi057 ceph-mon[89491]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:08.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:07 smithi084 ceph-mon[70440]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:09 smithi057 ceph-mon[89491]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:09 smithi084 ceph-mon[70440]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:11 smithi057 ceph-mon[89491]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:11 smithi084 ceph-mon[70440]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:13 smithi057 ceph-mon[89491]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:13 smithi084 ceph-mon[70440]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:15 smithi057 ceph-mon[89491]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:15 smithi084 ceph-mon[70440]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:17 smithi057 ceph-mon[89491]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:17 smithi084 ceph-mon[70440]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:19 smithi084 ceph-mon[70440]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:19 smithi057 ceph-mon[89491]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.309+0000 7fb062000700 1 -- 172.21.15.57:0/2087991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c100e80 msgr2=0x7fb05c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.309+0000 7fb062000700 1 --2- 172.21.15.57:0/2087991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c100e80 0x7fb05c104e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb048009910 tx=0x7fb048009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 -- 172.21.15.57:0/2087991 shutdown_connections 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 --2- 172.21.15.57:0/2087991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c100e80 0x7fb05c104e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 --2- 172.21.15.57:0/2087991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c1004e0 0x7fb05c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 -- 172.21.15.57:0/2087991 >> 172.21.15.57:0/2087991 conn(0x7fb05c0fbda0 msgr2=0x7fb05c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 -- 172.21.15.57:0/2087991 shutdown_connections 2024-02-28T23:21:20.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.310+0000 7fb062000700 1 -- 172.21.15.57:0/2087991 wait complete. 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 Processor -- start 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 -- start start 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c100e80 0x7fb05c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c073ad0 con 0x7fb05c1004e0 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.311+0000 7fb062000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c073c10 con 0x7fb05c100e80 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36080/0 (socket says 172.21.15.57:36080) 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 -- 172.21.15.57:0/2913469611 learned_addr learned my addr 172.21.15.57:0/2913469611 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05aa00700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c100e80 0x7fb05c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 -- 172.21.15.57:0/2913469611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c100e80 msgr2=0x7fb05c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c100e80 0x7fb05c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.312+0000 7fb05b400700 1 -- 172.21.15.57:0/2913469611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480095f0 con 0x7fb05c1004e0 2024-02-28T23:21:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb05b400700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00d8d0 tx=0x7fb04c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04c009940 con 0x7fb05c1004e0 2024-02-28T23:21:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb04c010460 con 0x7fb05c1004e0 2024-02-28T23:21:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04c00f5f0 con 0x7fb05c1004e0 2024-02-28T23:21:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c1a2100 con 0x7fb05c1004e0 2024-02-28T23:21:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.313+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c1a2510 con 0x7fb05c1004e0 2024-02-28T23:21:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.314+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c108850 con 0x7fb05c1004e0 2024-02-28T23:21:20.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.316+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb04c0105d0 con 0x7fb05c1004e0 2024-02-28T23:21:20.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.316+0000 7fb053e00700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 0x7fb04006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.317+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb04c08d510 con 0x7fb05c1004e0 2024-02-28T23:21:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.317+0000 7fb05aa00700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 0x7fb04006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.317+0000 7fb05aa00700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 0x7fb04006fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb05c074840 tx=0x7fb04801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.322+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb04c056ca0 con 0x7fb05c1004e0 2024-02-28T23:21:20.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.572+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb05c061190 con 0x7fb04006d8a0 2024-02-28T23:21:20.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.575+0000 7fb053e00700 1 -- 172.21.15.57:0/2913469611 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb05c061190 con 0x7fb04006d8a0 2024-02-28T23:21:20.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 msgr2=0x7fb04006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 0x7fb04006fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb05c074840 tx=0x7fb04801a040 comp rx=0 tx=0).stop 2024-02-28T23:21:20.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 msgr2=0x7fb05c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00d8d0 tx=0x7fb04c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:21:20.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 shutdown_connections 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb04006d8a0 0x7fb04006fd50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb05c1004e0 0x7fb05c073050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 --2- 172.21.15.57:0/2913469611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb05c100e80 0x7fb05c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 >> 172.21.15.57:0/2913469611 conn(0x7fb05c0fbda0 msgr2=0x7fb05c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 shutdown_connections 2024-02-28T23:21:20.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.578+0000 7fb062000700 1 -- 172.21.15.57:0/2913469611 wait complete. 2024-02-28T23:21:20.593 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 -- 172.21.15.57:0/2731938390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c106b80 msgr2=0x7fa07c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 --2- 172.21.15.57:0/2731938390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c106b80 0x7fa07c106ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa068009910 tx=0x7fa068009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 -- 172.21.15.57:0/2731938390 shutdown_connections 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 --2- 172.21.15.57:0/2731938390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c106b80 0x7fa07c106ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 --2- 172.21.15.57:0/2731938390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c10c8f0 0x7fa07c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 -- 172.21.15.57:0/2731938390 >> 172.21.15.57:0/2731938390 conn(0x7fa07c076060 msgr2=0x7fa07c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:20.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 -- 172.21.15.57:0/2731938390 shutdown_connections 2024-02-28T23:21:20.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.742+0000 7fa083400700 1 -- 172.21.15.57:0/2731938390 wait complete. 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.743+0000 7fa083400700 1 Processor -- start 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa083400700 1 -- start start 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa083400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c106b80 0x7fa07c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa083400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa083400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c19d060 con 0x7fa07c10c8f0 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa083400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a0df0 con 0x7fa07c106b80 2024-02-28T23:21:20.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa07be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa07be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36100/0 (socket says 172.21.15.57:36100) 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.744+0000 7fa07be00700 1 -- 172.21.15.57:0/222679206 learned_addr learned my addr 172.21.15.57:0/222679206 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa080a00700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c106b80 0x7fa07c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa07be00700 1 -- 172.21.15.57:0/222679206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c106b80 msgr2=0x7fa07c19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa07be00700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c106b80 0x7fa07c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:20.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa07be00700 1 -- 172.21.15.57:0/222679206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0680095f0 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa07be00700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b5c0 tx=0x7fa06800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa06801d070 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa068005e00 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.745+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c1a1070 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.746+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa068017400 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.746+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c1a1560 con 0x7fa07c10c8f0 2024-02-28T23:21:20.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.747+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c10ecc0 con 0x7fa07c10c8f0 2024-02-28T23:21:20.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.748+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa06800f460 con 0x7fa07c10c8f0 2024-02-28T23:21:20.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.749+0000 7fa079600700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 0x7fa06406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:20.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.749+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa06808eb00 con 0x7fa07c10c8f0 2024-02-28T23:21:20.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.749+0000 7fa080a00700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 0x7fa06406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:20.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.750+0000 7fa080a00700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 0x7fa06406fd50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa070005950 tx=0x7fa0700058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:20.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:20.753+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa068058210 con 0x7fa07c10c8f0 2024-02-28T23:21:21.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.006+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa07c1a1840 con 0x7fa06406d8a0 2024-02-28T23:21:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.007+0000 7fa079600700 1 -- 172.21.15.57:0/222679206 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa07c1a1840 con 0x7fa06406d8a0 2024-02-28T23:21:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 msgr2=0x7fa06406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 0x7fa06406fd50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa070005950 tx=0x7fa0700058e0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 msgr2=0x7fa07c19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b5c0 tx=0x7fa06800be20 comp rx=0 tx=0).stop 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 shutdown_connections 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa06406d8a0 0x7fa06406fd50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa07c106b80 0x7fa07c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 --2- 172.21.15.57:0/222679206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa07c10c8f0 0x7fa07c19c980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 >> 172.21.15.57:0/222679206 conn(0x7fa07c076060 msgr2=0x7fa07c077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 shutdown_connections 2024-02-28T23:21:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.011+0000 7fa083400700 1 -- 172.21.15.57:0/222679206 wait complete. 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.173+0000 7fae09e00700 1 -- 172.21.15.57:0/1811453270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 msgr2=0x7fae04073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.173+0000 7fae09e00700 1 --2- 172.21.15.57:0/1811453270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04073440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009940 tx=0x7fadf4009c50 comp rx=0 tx=0).stop 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 -- 172.21.15.57:0/1811453270 shutdown_connections 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 --2- 172.21.15.57:0/1811453270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae0410c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 --2- 172.21.15.57:0/1811453270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04073440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 -- 172.21.15.57:0/1811453270 >> 172.21.15.57:0/1811453270 conn(0x7fae040fbd40 msgr2=0x7fae040fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:21.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 -- 172.21.15.57:0/1811453270 shutdown_connections 2024-02-28T23:21:21.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.174+0000 7fae09e00700 1 -- 172.21.15.57:0/1811453270 wait complete. 2024-02-28T23:21:21.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 Processor -- start 2024-02-28T23:21:21.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 -- start start 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae04198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04198c40 con 0x7fae04073070 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.175+0000 7fae09e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419c9d0 con 0x7fae04073980 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36120/0 (socket says 172.21.15.57:36120) 2024-02-28T23:21:21.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 -- 172.21.15.57:0/2498690206 learned_addr learned my addr 172.21.15.57:0/2498690206 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:21.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 -- 172.21.15.57:0/2498690206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 msgr2=0x7fae04198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02000700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae04198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae04198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02a00700 1 -- 172.21.15.57:0/2498690206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40095f0 con 0x7fae04073070 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.176+0000 7fae02000700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae04198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae02a00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b870 tx=0x7fadf40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf401d070 con 0x7fae04073070 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf400f460 con 0x7fae04073070 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf4021640 con 0x7fae04073070 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0419cc50 con 0x7fae04073070 2024-02-28T23:21:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.177+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0419d140 con 0x7fae04073070 2024-02-28T23:21:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.179+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae04109cc0 con 0x7fae04073070 2024-02-28T23:21:21.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.180+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fadf400fa40 con 0x7fae04073070 2024-02-28T23:21:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.181+0000 7fae08a00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 0x7fade806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.181+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fadf408e800 con 0x7fae04073070 2024-02-28T23:21:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.181+0000 7fae02000700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 0x7fade806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.182+0000 7fae02000700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 0x7fade806fd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf0005fd0 tx=0x7fadf0005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.185+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadf4057f10 con 0x7fae04073070 2024-02-28T23:21:21.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.426+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae0419d470 con 0x7fade806d8a0 2024-02-28T23:21:21.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.460+0000 7fae08a00700 1 -- 172.21.15.57:0/2498690206 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fae0419d470 con 0x7fade806d8a0 2024-02-28T23:21:21.460 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:21:21.460 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (11m) 2m ago 24m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (21m) 2m ago 25m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (23m) 2m ago 23m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (7m) 2m ago 25m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (6m) 2m ago 23m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (10m) 2m ago 24m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (2m) 2m ago 17m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (2m) 2m ago 17m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (17m) 2m ago 17m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (2m) 2m ago 17m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (13m) 2m ago 26m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (12m) 2m ago 23m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (8m) 2m ago 26m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (8m) 2m ago 23m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:21:21.461 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (12m) 2m ago 25m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (11m) 2m ago 23m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 2m ago 22m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (5m) 2m ago 21m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (5m) 2m ago 21m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (4m) 2m ago 20m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (4m) 2m ago 19m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (3m) 2m ago 19m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:21:21.462 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (11m) 2m ago 24m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 msgr2=0x7fade806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 0x7fade806fd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf0005fd0 tx=0x7fadf0005dc0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 msgr2=0x7fae04198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b870 tx=0x7fadf40056a0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 shutdown_connections 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fade806d8a0 0x7fade806fd50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae04073070 0x7fae04198020 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 --2- 172.21.15.57:0/2498690206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae04073980 0x7fae04198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 >> 172.21.15.57:0/2498690206 conn(0x7fae040fbd40 msgr2=0x7fae04106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 shutdown_connections 2024-02-28T23:21:21.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.463+0000 7fae09e00700 1 -- 172.21.15.57:0/2498690206 wait complete. 2024-02-28T23:21:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.624+0000 7f2b95000700 1 -- 172.21.15.57:0/1085144771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90100e30 msgr2=0x7f2b90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.624+0000 7f2b95000700 1 --2- 172.21.15.57:0/1085144771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90100e30 0x7f2b90101200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009910 tx=0x7f2b80009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 -- 172.21.15.57:0/1085144771 shutdown_connections 2024-02-28T23:21:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 --2- 172.21.15.57:0/1085144771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90068540 0x7f2b900689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 --2- 172.21.15.57:0/1085144771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90100e30 0x7f2b90101200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 -- 172.21.15.57:0/1085144771 >> 172.21.15.57:0/1085144771 conn(0x7f2b90074fa0 msgr2=0x7f2b900753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 -- 172.21.15.57:0/1085144771 shutdown_connections 2024-02-28T23:21:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.625+0000 7f2b95000700 1 -- 172.21.15.57:0/1085144771 wait complete. 2024-02-28T23:21:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.626+0000 7f2b95000700 1 Processor -- start 2024-02-28T23:21:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.626+0000 7f2b95000700 1 -- start start 2024-02-28T23:21:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b95000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b95000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90100e30 0x7f2b90198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b95000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b901989f0 con 0x7f2b90068540 2024-02-28T23:21:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b95000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9019c780 con 0x7f2b90100e30 2024-02-28T23:21:21.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b8f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b8f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36148/0 (socket says 172.21.15.57:36148) 2024-02-28T23:21:21.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b8f400700 1 -- 172.21.15.57:0/3310741060 learned_addr learned my addr 172.21.15.57:0/3310741060 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b8f400700 1 -- 172.21.15.57:0/3310741060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90100e30 msgr2=0x7f2b90198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.627+0000 7f2b8ea00700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90100e30 0x7f2b90198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b8f400700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90100e30 0x7f2b90198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b8f400700 1 -- 172.21.15.57:0/3310741060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800095f0 con 0x7f2b90068540 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b8f400700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000b5c0 tx=0x7f2b8000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b8001d070 con 0x7f2b90068540 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9019ca00 con 0x7f2b90068540 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b8000f460 con 0x7f2b90068540 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b80017610 con 0x7f2b90068540 2024-02-28T23:21:21.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.628+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9019cf70 con 0x7f2b90068540 2024-02-28T23:21:21.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.630+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b90062380 con 0x7f2b90068540 2024-02-28T23:21:21.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.631+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2b8000f5d0 con 0x7f2b90068540 2024-02-28T23:21:21.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.632+0000 7f2b87e00700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 0x7f2b7406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:21.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.632+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b8008ebd0 con 0x7f2b90068540 2024-02-28T23:21:21.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.632+0000 7f2b8ea00700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 0x7f2b7406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:21.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.633+0000 7f2b8ea00700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 0x7f2b7406fd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b901993f0 tx=0x7f2b7c005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:21.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.641+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b800582e0 con 0x7f2b90068540 2024-02-28T23:21:21.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.964+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2b9019d360 con 0x7f2b90068540 2024-02-28T23:21:21.964 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:21 smithi057 ceph-mon[89491]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:21.964 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:21 smithi057 ceph-mon[89491]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.965+0000 7f2b87e00700 1 -- 172.21.15.57:0/3310741060 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f2b8005b9a0 con 0x7f2b90068540 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:21.965 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:21:21.966 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:21:21.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 msgr2=0x7f2b7406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 0x7f2b7406fd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b901993f0 tx=0x7f2b7c005cf0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 msgr2=0x7f2b90197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:21.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000b5c0 tx=0x7f2b8000be00 comp rx=0 tx=0).stop 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 shutdown_connections 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b7406d8a0 0x7f2b7406fd50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b90068540 0x7f2b90197dd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 --2- 172.21.15.57:0/3310741060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b90100e30 0x7f2b90198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 >> 172.21.15.57:0/3310741060 conn(0x7f2b90074fa0 msgr2=0x7f2b900fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 shutdown_connections 2024-02-28T23:21:21.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:21.968+0000 7f2b95000700 1 -- 172.21.15.57:0/3310741060 wait complete. 2024-02-28T23:21:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:21 smithi084 ceph-mon[70440]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:21 smithi084 ceph-mon[70440]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 -- 172.21.15.57:0/4173017642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 msgr2=0x7f44a4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 --2- 172.21.15.57:0/4173017642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a4106ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f448c009910 tx=0x7f448c009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 -- 172.21.15.57:0/4173017642 shutdown_connections 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 --2- 172.21.15.57:0/4173017642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a4106ff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 --2- 172.21.15.57:0/4173017642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 0x7f44a410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 -- 172.21.15.57:0/4173017642 >> 172.21.15.57:0/4173017642 conn(0x7f44a4075e90 msgr2=0x7f44a40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 -- 172.21.15.57:0/4173017642 shutdown_connections 2024-02-28T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.129+0000 7f44a3200700 1 -- 172.21.15.57:0/4173017642 wait complete. 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.130+0000 7f44a3200700 1 Processor -- start 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a3200700 1 -- start start 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 0x7f44a419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a419ce00 con 0x7f44a4106b80 2024-02-28T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41a0b90 con 0x7f44a410c660 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36174/0 (socket says 172.21.15.57:36174) 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.131+0000 7f44a1e00700 1 -- 172.21.15.57:0/1807545494 learned_addr learned my addr 172.21.15.57:0/1807545494 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f449be00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 0x7f44a419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f44a1e00700 1 -- 172.21.15.57:0/1807545494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 msgr2=0x7f44a419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f44a1e00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 0x7f44a419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f44a1e00700 1 -- 172.21.15.57:0/1807545494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0095f0 con 0x7f44a4106b80 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f44a1e00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f449400c8a0 tx=0x7f449400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:22.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449400cea0 con 0x7f44a4106b80 2024-02-28T23:21:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4494004830 con 0x7f44a4106b80 2024-02-28T23:21:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44940055b0 con 0x7f44a4106b80 2024-02-28T23:21:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.132+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a41a0e70 con 0x7f44a4106b80 2024-02-28T23:21:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.133+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a41a1370 con 0x7f44a4106b80 2024-02-28T23:21:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.134+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a410eb40 con 0x7f44a4106b80 2024-02-28T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.135+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f44940049a0 con 0x7f44a4106b80 2024-02-28T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.136+0000 7f449aa00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 0x7f448806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.136+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f449408d260 con 0x7f44a4106b80 2024-02-28T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.136+0000 7f449be00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 0x7f448806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.137+0000 7f449be00700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 0x7f448806fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44a419d800 tx=0x7f448c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.141+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4494056970 con 0x7f44a4106b80 2024-02-28T23:21:22.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.419+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f44a41a1640 con 0x7f44a4106b80 2024-02-28T23:21:22.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.420+0000 7f449aa00700 1 -- 172.21.15.57:0/1807545494 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f4494004c50 con 0x7f44a4106b80 2024-02-28T23:21:22.419 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:21:22.419 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:21:22.419 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:21:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.422+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 msgr2=0x7f448806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.422+0000 7f44a3200700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 0x7f448806fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44a419d800 tx=0x7f448c01a040 comp rx=0 tx=0).stop 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.422+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 msgr2=0x7f44a419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.422+0000 7f44a3200700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f449400c8a0 tx=0x7f449400cc60 comp rx=0 tx=0).stop 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 shutdown_connections 2024-02-28T23:21:22.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f448806d8a0 0x7f448806fd50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44a4106b80 0x7f44a419c1e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 --2- 172.21.15.57:0/1807545494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44a410c660 0x7f44a419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 >> 172.21.15.57:0/1807545494 conn(0x7f44a4075e90 msgr2=0x7f44a40778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 shutdown_connections 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.423+0000 7f44a3200700 1 -- 172.21.15.57:0/1807545494 wait complete. 2024-02-28T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:21:22.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.585+0000 7f7a20e00700 1 -- 172.21.15.57:0/2842591497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 msgr2=0x7f7a1c0ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.585+0000 7f7a20e00700 1 --2- 172.21.15.57:0/2842591497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c0ff3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009910 tx=0x7f7a0c009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:22.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 -- 172.21.15.57:0/2842591497 shutdown_connections 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 --2- 172.21.15.57:0/2842591497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 0x7f7a1c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 --2- 172.21.15.57:0/2842591497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c0ff3b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 -- 172.21.15.57:0/2842591497 >> 172.21.15.57:0/2842591497 conn(0x7f7a1c0faa50 msgr2=0x7f7a1c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 -- 172.21.15.57:0/2842591497 shutdown_connections 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.586+0000 7f7a20e00700 1 -- 172.21.15.57:0/2842591497 wait complete. 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.587+0000 7f7a20e00700 1 Processor -- start 2024-02-28T23:21:22.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.587+0000 7f7a20e00700 1 -- start start 2024-02-28T23:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a20e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a20e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 0x7f7a1c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a20e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c19cdc0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a20e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c1a0b50 con 0x7f7a1c0ff8f0 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36196/0 (socket says 172.21.15.57:36196) 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 -- 172.21.15.57:0/4159945427 learned_addr learned my addr 172.21.15.57:0/4159945427 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1aa00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 0x7f7a1c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 -- 172.21.15.57:0/4159945427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 msgr2=0x7f7a1c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 0x7f7a1c19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.588+0000 7f7a1b400700 1 -- 172.21.15.57:0/4159945427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0095f0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a1b400700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c006010 tx=0x7f7a0c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0c01d070 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c1a0dd0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a0c00f460 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0c017630 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.589+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c1a12c0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.590+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1c101500 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.591+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7a0c00f5d0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.592+0000 7f7a13e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 0x7f7a0006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.592+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7a0c08ea90 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.592+0000 7f7a1aa00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 0x7f7a0006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.593+0000 7f7a1aa00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 0x7f7a0006fd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a08005950 tx=0x7f7a080058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.597+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a0c0581a0 con 0x7f7a1c0fefe0 2024-02-28T23:21:22.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.838+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a1c1a15a0 con 0x7f7a0006d8a0 2024-02-28T23:21:22.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.841+0000 7f7a13e00700 1 -- 172.21.15.57:0/4159945427 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7a1c1a15a0 con 0x7f7a0006d8a0 2024-02-28T23:21:22.841 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:21:22.841 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:21:22.841 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:21:22.841 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:21:22.842 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 msgr2=0x7f7a0006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 0x7f7a0006fd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a08005950 tx=0x7f7a080058e0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 msgr2=0x7f7a1c19c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c006010 tx=0x7f7a0c00be20 comp rx=0 tx=0).stop 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 shutdown_connections 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.844+0000 7f7a20e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a0006d8a0 0x7f7a0006fd50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.845+0000 7f7a20e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a1c0fefe0 0x7f7a1c19c1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.845+0000 7f7a20e00700 1 --2- 172.21.15.57:0/4159945427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a1c0ff8f0 0x7f7a1c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:22.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.845+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 >> 172.21.15.57:0/4159945427 conn(0x7f7a1c0faa50 msgr2=0x7f7a1c110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:22.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.845+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 shutdown_connections 2024-02-28T23:21:22.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.845+0000 7f7a20e00700 1 -- 172.21.15.57:0/4159945427 wait complete. 2024-02-28T23:21:22.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 -- 172.21.15.57:0/1766087517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1380739a0 msgr2=0x7ff13810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:22.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 --2- 172.21.15.57:0/1766087517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1380739a0 0x7ff13810c5e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff124009940 tx=0x7ff124009c50 comp rx=0 tx=0).stop 2024-02-28T23:21:22.999 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:22 smithi057 ceph-mon[89491]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:22.999 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:22 smithi057 ceph-mon[89491]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:22.999 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3310741060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:21:22.999 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1807545494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:21:23.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 -- 172.21.15.57:0/1766087517 shutdown_connections 2024-02-28T23:21:23.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 --2- 172.21.15.57:0/1766087517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1380739a0 0x7ff13810c5e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 --2- 172.21.15.57:0/1766087517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff138073090 0x7ff138073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:22.999+0000 7ff140600700 1 -- 172.21.15.57:0/1766087517 >> 172.21.15.57:0/1766087517 conn(0x7ff1380fbda0 msgr2=0x7ff1380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:23.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.000+0000 7ff140600700 1 -- 172.21.15.57:0/1766087517 shutdown_connections 2024-02-28T23:21:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.000+0000 7ff140600700 1 -- 172.21.15.57:0/1766087517 wait complete. 2024-02-28T23:21:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.001+0000 7ff140600700 1 Processor -- start 2024-02-28T23:21:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.001+0000 7ff140600700 1 -- start start 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.001+0000 7ff140600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff140600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1380739a0 0x7ff1381986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff140600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff138198dd0 con 0x7ff138073090 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff140600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819cb60 con 0x7ff1380739a0 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36218/0 (socket says 172.21.15.57:36218) 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 -- 172.21.15.57:0/1661678375 learned_addr learned my addr 172.21.15.57:0/1661678375 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13d200700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1380739a0 0x7ff1381986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 -- 172.21.15.57:0/1661678375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1380739a0 msgr2=0x7ff1381986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1380739a0 0x7ff1381986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.002+0000 7ff13dc00700 1 -- 172.21.15.57:0/1661678375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1240095f0 con 0x7ff138073090 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff13dc00700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c8a0 tx=0x7ff12c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff12c00cea0 con 0x7ff138073090 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff12c004830 con 0x7ff138073090 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13819ce40 con 0x7ff138073090 2024-02-28T23:21:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff12c0055b0 con 0x7ff138073090 2024-02-28T23:21:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.003+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13819d360 con 0x7ff138073090 2024-02-28T23:21:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.004+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff138109ce0 con 0x7ff138073090 2024-02-28T23:21:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.006+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff12c0049a0 con 0x7ff138073090 2024-02-28T23:21:23.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.007+0000 7ff136000700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 0x7ff12006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:23.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.007+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff12c08d180 con 0x7ff138073090 2024-02-28T23:21:23.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.007+0000 7ff13d200700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 0x7ff12006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:23.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.008+0000 7ff13d200700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 0x7ff12006fd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff124006010 tx=0x7ff124005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.012+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff12c056910 con 0x7ff138073090 2024-02-28T23:21:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:22 smithi084 ceph-mon[70440]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:22 smithi084 ceph-mon[70440]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3310741060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:21:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1807545494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:21:23.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.320+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff13804f2a0 con 0x7ff138073090 2024-02-28T23:21:23.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.321+0000 7ff136000700 1 -- 172.21.15.57:0/1661678375 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff12c01d100 con 0x7ff138073090 2024-02-28T23:21:23.320 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:21:23.320 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:21:23.320 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:21:23.320 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:21:23.321 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:21:23.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.323+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 msgr2=0x7ff12006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:23.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 0x7ff12006fd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff124006010 tx=0x7ff124005a90 comp rx=0 tx=0).stop 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 msgr2=0x7ff1381981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c8a0 tx=0x7ff12c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 shutdown_connections 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff12006d8a0 0x7ff12006fd50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff138073090 0x7ff1381981b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 --2- 172.21.15.57:0/1661678375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1380739a0 0x7ff1381986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 >> 172.21.15.57:0/1661678375 conn(0x7ff1380fbda0 msgr2=0x7ff138106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 shutdown_connections 2024-02-28T23:21:23.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:23.324+0000 7ff140600700 1 -- 172.21.15.57:0/1661678375 wait complete. 2024-02-28T23:21:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:23 smithi057 ceph-mon[89491]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1661678375' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:21:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:23 smithi084 ceph-mon[70440]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1661678375' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:21:25.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:24 smithi057 ceph-mon[89491]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:24 smithi084 ceph-mon[70440]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:25 smithi057 ceph-mon[89491]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:25 smithi084 ceph-mon[70440]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:27 smithi057 ceph-mon[89491]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:27 smithi084 ceph-mon[70440]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:30.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:29 smithi057 ceph-mon[89491]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:29 smithi084 ceph-mon[70440]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:31 smithi057 ceph-mon[89491]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:32.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:31 smithi084 ceph-mon[70440]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:33 smithi057 ceph-mon[89491]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:33 smithi084 ceph-mon[70440]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:35 smithi057 ceph-mon[89491]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:36.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:35 smithi084 ceph-mon[70440]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:37 smithi057 ceph-mon[89491]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:37 smithi084 ceph-mon[70440]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:39 smithi057 ceph-mon[89491]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:39 smithi084 ceph-mon[70440]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:41 smithi057 ceph-mon[89491]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:41 smithi084 ceph-mon[70440]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:43 smithi057 ceph-mon[89491]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:43 smithi084 ceph-mon[70440]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:45 smithi057 ceph-mon[89491]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:45 smithi084 ceph-mon[70440]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:47 smithi057 ceph-mon[89491]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:47 smithi084 ceph-mon[70440]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:21:50.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:49 smithi057 ceph-mon[89491]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:49 smithi084 ceph-mon[70440]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:52.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:51 smithi057 ceph-mon[89491]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:52.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:51 smithi084 ceph-mon[70440]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:53.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 -- 172.21.15.57:0/2780852121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 msgr2=0x7fd154108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 --2- 172.21.15.57:0/2780852121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154108840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd144009910 tx=0x7fd144009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 -- 172.21.15.57:0/2780852121 shutdown_connections 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 --2- 172.21.15.57:0/2780852121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 0x7fd1541028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 --2- 172.21.15.57:0/2780852121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154108840 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.501+0000 7fd15a200700 1 -- 172.21.15.57:0/2780852121 >> 172.21.15.57:0/2780852121 conn(0x7fd1540fdf90 msgr2=0x7fd1541003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.502+0000 7fd15a200700 1 -- 172.21.15.57:0/2780852121 shutdown_connections 2024-02-28T23:21:53.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.502+0000 7fd15a200700 1 -- 172.21.15.57:0/2780852121 wait complete. 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 Processor -- start 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 -- start start 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 0x7fd1541981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154198e10 con 0x7fd154108470 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.503+0000 7fd15a200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15419cba0 con 0x7fd154102470 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37174/0 (socket says 172.21.15.57:37174) 2024-02-28T23:21:53.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 -- 172.21.15.57:0/3287511198 learned_addr learned my addr 172.21.15.57:0/3287511198 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd153400700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 0x7fd1541981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 -- 172.21.15.57:0/3287511198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 msgr2=0x7fd1541981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 0x7fd1541981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.504+0000 7fd152a00700 1 -- 172.21.15.57:0/3287511198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440095f0 con 0x7fd154108470 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.505+0000 7fd152a00700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d8d0 tx=0x7fd14000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.505+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd140009940 con 0x7fd154108470 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.505+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15419ce80 con 0x7fd154108470 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.505+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd140010460 con 0x7fd154108470 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.505+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15419d3d0 con 0x7fd154108470 2024-02-28T23:21:53.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.506+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14000f5f0 con 0x7fd154108470 2024-02-28T23:21:53.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.506+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15410a9e0 con 0x7fd154108470 2024-02-28T23:21:53.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.507+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd1400105d0 con 0x7fd154108470 2024-02-28T23:21:53.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.508+0000 7fd14be00700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 0x7fd13806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.508+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd14008d5d0 con 0x7fd154108470 2024-02-28T23:21:53.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.509+0000 7fd153400700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 0x7fd13806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.509+0000 7fd153400700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 0x7fd13806fd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd14400b5c0 tx=0x7fd1440058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:53.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.513+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd140056ce0 con 0x7fd154108470 2024-02-28T23:21:53.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.763+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1541995c0 con 0x7fd13806d8a0 2024-02-28T23:21:53.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.765+0000 7fd14be00700 1 -- 172.21.15.57:0/3287511198 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd1541995c0 con 0x7fd13806d8a0 2024-02-28T23:21:53.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 msgr2=0x7fd13806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 0x7fd13806fd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd14400b5c0 tx=0x7fd1440058e0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 msgr2=0x7fd154198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d8d0 tx=0x7fd14000dc90 comp rx=0 tx=0).stop 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 shutdown_connections 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd13806d8a0 0x7fd13806fd50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd154102470 0x7fd1541981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 --2- 172.21.15.57:0/3287511198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd154108470 0x7fd154198730 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 >> 172.21.15.57:0/3287511198 conn(0x7fd1540fdf90 msgr2=0x7fd1541002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 shutdown_connections 2024-02-28T23:21:53.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.768+0000 7fd15a200700 1 -- 172.21.15.57:0/3287511198 wait complete. 2024-02-28T23:21:53.783 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:21:53.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.939+0000 7fea7c400700 1 -- 172.21.15.57:0/1650381956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 msgr2=0x7fea74106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.939+0000 7fea7c400700 1 --2- 172.21.15.57:0/1650381956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea74106ff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea60009910 tx=0x7fea60009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 -- 172.21.15.57:0/1650381956 shutdown_connections 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 --2- 172.21.15.57:0/1650381956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea74106ff0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 --2- 172.21.15.57:0/1650381956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 0x7fea7410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 -- 172.21.15.57:0/1650381956 >> 172.21.15.57:0/1650381956 conn(0x7fea74076060 msgr2=0x7fea74078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 -- 172.21.15.57:0/1650381956 shutdown_connections 2024-02-28T23:21:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.940+0000 7fea7c400700 1 -- 172.21.15.57:0/1650381956 wait complete. 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.941+0000 7fea7c400700 1 Processor -- start 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.941+0000 7fea7c400700 1 -- start start 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea7c400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea7c400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 0x7fea7419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea7c400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7419d060 con 0x7fea74106b80 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea7c400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741a0df0 con 0x7fea7410c8f0 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37184/0 (socket says 172.21.15.57:37184) 2024-02-28T23:21:53.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79a00700 1 -- 172.21.15.57:0/2807787079 learned_addr learned my addr 172.21.15.57:0/2807787079 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:53.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79000700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 0x7fea7419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79a00700 1 -- 172.21.15.57:0/2807787079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 msgr2=0x7fea7419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.942+0000 7fea79a00700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 0x7fea7419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea79a00700 1 -- 172.21.15.57:0/2807787079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600095f0 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea79a00700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea6800d8d0 tx=0x7fea6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea68009940 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea68010460 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6800f5f0 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea741a10d0 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.943+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea741a1620 con 0x7fea74106b80 2024-02-28T23:21:53.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.944+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7410ecc0 con 0x7fea74106b80 2024-02-28T23:21:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.945+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fea680105d0 con 0x7fea74106b80 2024-02-28T23:21:53.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.946+0000 7fea72000700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 0x7fea5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:53.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.946+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fea6808d4c0 con 0x7fea74106b80 2024-02-28T23:21:53.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.947+0000 7fea79000700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 0x7fea5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:53.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.947+0000 7fea79000700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 0x7fea5c06fd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea60005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:53.951+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea68056c50 con 0x7fea74106b80 2024-02-28T23:21:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:53 smithi057 ceph-mon[89491]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:54.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.205+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea741a1900 con 0x7fea5c06d8a0 2024-02-28T23:21:54.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.208+0000 7fea72000700 1 -- 172.21.15.57:0/2807787079 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fea741a1900 con 0x7fea5c06d8a0 2024-02-28T23:21:54.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.210+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 msgr2=0x7fea5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.210+0000 7fea7c400700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 0x7fea5c06fd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea60005fd0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.210+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 msgr2=0x7fea7419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.210+0000 7fea7c400700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea6800d8d0 tx=0x7fea6800dc90 comp rx=0 tx=0).stop 2024-02-28T23:21:54.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 shutdown_connections 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fea5c06d8a0 0x7fea5c06fd50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea74106b80 0x7fea7419c440 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 --2- 172.21.15.57:0/2807787079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7410c8f0 0x7fea7419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 >> 172.21.15.57:0/2807787079 conn(0x7fea74076060 msgr2=0x7fea74077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 shutdown_connections 2024-02-28T23:21:54.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.211+0000 7fea7c400700 1 -- 172.21.15.57:0/2807787079 wait complete. 2024-02-28T23:21:54.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:53 smithi084 ceph-mon[70440]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 -- 172.21.15.57:0/4292043562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 msgr2=0x7f11e4073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 --2- 172.21.15.57:0/4292043562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4073460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009940 tx=0x7f11d4009c50 comp rx=0 tx=0).stop 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 -- 172.21.15.57:0/4292043562 shutdown_connections 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 --2- 172.21.15.57:0/4292043562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 0x7f11e410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 --2- 172.21.15.57:0/4292043562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4073460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 -- 172.21.15.57:0/4292043562 >> 172.21.15.57:0/4292043562 conn(0x7f11e40fbda0 msgr2=0x7f11e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.378+0000 7f11eae00700 1 -- 172.21.15.57:0/4292043562 shutdown_connections 2024-02-28T23:21:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.379+0000 7f11eae00700 1 -- 172.21.15.57:0/4292043562 wait complete. 2024-02-28T23:21:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.379+0000 7f11eae00700 1 Processor -- start 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11eae00700 1 -- start start 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11eae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11eae00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 0x7f11e41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11eae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4198da0 con 0x7f11e4073090 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11eae00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e419cb30 con 0x7f11e40739a0 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11e3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11e3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37198/0 (socket says 172.21.15.57:37198) 2024-02-28T23:21:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.380+0000 7f11e3e00700 1 -- 172.21.15.57:0/3261999389 learned_addr learned my addr 172.21.15.57:0/3261999389 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.381+0000 7f11db400700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 0x7f11e41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.381+0000 7f11e3e00700 1 -- 172.21.15.57:0/3261999389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 msgr2=0x7f11e41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.381+0000 7f11e3e00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 0x7f11e41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.381+0000 7f11e3e00700 1 -- 172.21.15.57:0/3261999389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40095f0 con 0x7f11e4073090 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.381+0000 7f11e3e00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005620 tx=0x7f11d4005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:54.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.382+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d401d070 con 0x7f11e4073090 2024-02-28T23:21:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.382+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11d400f460 con 0x7f11e4073090 2024-02-28T23:21:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.382+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e419cdb0 con 0x7f11e4073090 2024-02-28T23:21:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.382+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d4021640 con 0x7f11e4073090 2024-02-28T23:21:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.382+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e419d2a0 con 0x7f11e4073090 2024-02-28T23:21:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.383+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4109ce0 con 0x7f11e4073090 2024-02-28T23:21:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.384+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f11d400fa40 con 0x7f11e4073090 2024-02-28T23:21:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.385+0000 7f11e1600700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 0x7f11c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.385+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11d408e810 con 0x7f11e4073090 2024-02-28T23:21:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.385+0000 7f11db400700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 0x7f11c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.386+0000 7f11db400700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 0x7f11c806fd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11cc005fd0 tx=0x7f11cc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:54.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.390+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11d4057fa0 con 0x7f11e4073090 2024-02-28T23:21:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.635+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e41995c0 con 0x7f11c806d8a0 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.655+0000 7f11e1600700 1 -- 172.21.15.57:0/3261999389 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f11e41995c0 con 0x7f11c806d8a0 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (11m) 2m ago 25m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (21m) 2m ago 25m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (24m) 2m ago 24m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:21:54.654 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (7m) 2m ago 25m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (7m) 2m ago 24m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (10m) 2m ago 24m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (3m) 2m ago 18m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (3m) 2m ago 18m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (17m) 2m ago 17m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (2m) 2m ago 18m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (13m) 2m ago 27m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (12m) 2m ago 23m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (9m) 2m ago 27m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (8m) 2m ago 23m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (12m) 2m ago 25m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (12m) 2m ago 24m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 2m ago 22m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:21:54.655 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 2m ago 22m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:21:54.656 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (5m) 2m ago 21m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:21:54.656 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (5m) 2m ago 21m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:21:54.656 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (4m) 2m ago 20m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:21:54.656 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (3m) 2m ago 19m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:21:54.656 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (11m) 2m ago 24m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 msgr2=0x7f11c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 0x7f11c806fd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11cc005fd0 tx=0x7f11cc00a560 comp rx=0 tx=0).stop 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 msgr2=0x7f11e4198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005620 tx=0x7f11d4005700 comp rx=0 tx=0).stop 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 shutdown_connections 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11c806d8a0 0x7f11c806fd50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4073090 0x7f11e4198180 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 --2- 172.21.15.57:0/3261999389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11e40739a0 0x7f11e41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.658+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 >> 172.21.15.57:0/3261999389 conn(0x7f11e40fbda0 msgr2=0x7f11e4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:54.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.659+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 shutdown_connections 2024-02-28T23:21:54.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.659+0000 7f11eae00700 1 -- 172.21.15.57:0/3261999389 wait complete. 2024-02-28T23:21:54.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 -- 172.21.15.57:0/278577781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801004e0 msgr2=0x7f94801008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 --2- 172.21.15.57:0/278577781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801004e0 0x7f94801008b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9474009910 tx=0x7f9474009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:54.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 -- 172.21.15.57:0/278577781 shutdown_connections 2024-02-28T23:21:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 --2- 172.21.15.57:0/278577781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9480100e80 0x7f9480104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 --2- 172.21.15.57:0/278577781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801004e0 0x7f94801008b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 -- 172.21.15.57:0/278577781 >> 172.21.15.57:0/278577781 conn(0x7f94800fbda0 msgr2=0x7f94800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 -- 172.21.15.57:0/278577781 shutdown_connections 2024-02-28T23:21:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.811+0000 7f9487600700 1 -- 172.21.15.57:0/278577781 wait complete. 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.812+0000 7f9487600700 1 Processor -- start 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.812+0000 7f9487600700 1 -- start start 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.812+0000 7f9487600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94801004e0 0x7f94801981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.812+0000 7f9487600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f9487600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480198dc0 con 0x7f9480100e80 2024-02-28T23:21:54.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f9487600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948019cb50 con 0x7f94801004e0 2024-02-28T23:21:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37214/0 (socket says 172.21.15.57:37214) 2024-02-28T23:21:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 -- 172.21.15.57:0/409859703 learned_addr learned my addr 172.21.15.57:0/409859703 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f9484c00700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94801004e0 0x7f94801981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 -- 172.21.15.57:0/409859703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94801004e0 msgr2=0x7f94801981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94801004e0 0x7f94801981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.813+0000 7f947fe00700 1 -- 172.21.15.57:0/409859703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740095f0 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f947fe00700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b480 tx=0x7f946c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c0118c0 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946c010380 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948019ce30 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c00f4f0 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.814+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948019d380 con 0x7f9480100e80 2024-02-28T23:21:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.815+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94801045d0 con 0x7f9480100e80 2024-02-28T23:21:54.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.816+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f946c0104f0 con 0x7f9480100e80 2024-02-28T23:21:54.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.817+0000 7f947d600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 0x7f946806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:54.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.817+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f946c08d210 con 0x7f9480100e80 2024-02-28T23:21:54.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.817+0000 7f9484c00700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 0x7f946806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:54.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.818+0000 7f9484c00700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 0x7f946806fd50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9474006010 tx=0x7f947401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:54.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:54.822+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f946c056920 con 0x7f9480100e80 2024-02-28T23:21:55.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.143+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f948004f2a0 con 0x7f9480100e80 2024-02-28T23:21:55.142 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:54 smithi057 ceph-mon[89491]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.144+0000 7f947d600700 1 -- 172.21.15.57:0/409859703 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f946c059fe0 con 0x7f9480100e80 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:21:55.143 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:21:55.144 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:21:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 msgr2=0x7f946806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 0x7f946806fd50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9474006010 tx=0x7f947401a040 comp rx=0 tx=0).stop 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 msgr2=0x7f94801986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b480 tx=0x7f946c00b840 comp rx=0 tx=0).stop 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 shutdown_connections 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f946806d8a0 0x7f946806fd50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94801004e0 0x7f94801981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 --2- 172.21.15.57:0/409859703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480100e80 0x7f94801986e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.148+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 >> 172.21.15.57:0/409859703 conn(0x7f94800fbda0 msgr2=0x7f94800fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.149+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 shutdown_connections 2024-02-28T23:21:55.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.149+0000 7f9487600700 1 -- 172.21.15.57:0/409859703 wait complete. 2024-02-28T23:21:55.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 -- 172.21.15.57:0/3514482707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f4106b80 msgr2=0x7f82f4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 --2- 172.21.15.57:0/3514482707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f4106b80 0x7f82f4106ff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009910 tx=0x7f82e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 -- 172.21.15.57:0/3514482707 shutdown_connections 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 --2- 172.21.15.57:0/3514482707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f4106b80 0x7f82f4106ff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 --2- 172.21.15.57:0/3514482707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f410c8f0 0x7f82f410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 -- 172.21.15.57:0/3514482707 >> 172.21.15.57:0/3514482707 conn(0x7f82f4076060 msgr2=0x7f82f4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 -- 172.21.15.57:0/3514482707 shutdown_connections 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.311+0000 7f82fa800700 1 -- 172.21.15.57:0/3514482707 wait complete. 2024-02-28T23:21:55.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.312+0000 7f82fa800700 1 Processor -- start 2024-02-28T23:21:55.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82fa800700 1 -- start start 2024-02-28T23:21:55.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82fa800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f4106b80 0x7f82f419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82fa800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82fa800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419d060 con 0x7f82f410c8f0 2024-02-28T23:21:55.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82fa800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41a0df0 con 0x7f82f4106b80 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49834/0 (socket says 172.21.15.57:49834) 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.313+0000 7f82f2a00700 1 -- 172.21.15.57:0/1943025157 learned_addr learned my addr 172.21.15.57:0/1943025157 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.314+0000 7f82f3400700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f4106b80 0x7f82f419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.314+0000 7f82f2a00700 1 -- 172.21.15.57:0/1943025157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f4106b80 msgr2=0x7f82f419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.314+0000 7f82f2a00700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f4106b80 0x7f82f419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.314+0000 7f82f2a00700 1 -- 172.21.15.57:0/1943025157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00095f0 con 0x7f82f410c8f0 2024-02-28T23:21:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.314+0000 7f82f2a00700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f82e0006010 tx=0x7f82e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.315+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e001d070 con 0x7f82f410c8f0 2024-02-28T23:21:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.315+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82e000f460 con 0x7f82f410c8f0 2024-02-28T23:21:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.315+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e0017630 con 0x7f82f410c8f0 2024-02-28T23:21:55.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.315+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f41a1070 con 0x7f82f410c8f0 2024-02-28T23:21:55.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.315+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f41a1560 con 0x7f82f410c8f0 2024-02-28T23:21:55.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.316+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f410ecc0 con 0x7f82f410c8f0 2024-02-28T23:21:55.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.317+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82e000f5d0 con 0x7f82f410c8f0 2024-02-28T23:21:55.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.318+0000 7f82ebe00700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 0x7f82d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.319+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f82e008ea90 con 0x7f82f410c8f0 2024-02-28T23:21:55.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.319+0000 7f82f3400700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 0x7f82d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:55.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.320+0000 7f82f3400700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 0x7f82d806fd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005950 tx=0x7f82e40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:55.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.323+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82e00581a0 con 0x7f82f410c8f0 2024-02-28T23:21:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:54 smithi084 ceph-mon[70440]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:55.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.591+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f82f410e6a0 con 0x7f82f410c8f0 2024-02-28T23:21:55.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.592+0000 7f82ebe00700 1 -- 172.21.15.57:0/1943025157 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f82e005b860 con 0x7f82f410c8f0 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:21:55.591 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:21:55.592 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:21:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:21:55.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 msgr2=0x7f82d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 0x7f82d806fd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005950 tx=0x7f82e40058e0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 msgr2=0x7f82f419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f82e0006010 tx=0x7f82e000be20 comp rx=0 tx=0).stop 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 shutdown_connections 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82d806d8a0 0x7f82d806fd50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82f4106b80 0x7f82f419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 --2- 172.21.15.57:0/1943025157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82f410c8f0 0x7f82f419c980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 >> 172.21.15.57:0/1943025157 conn(0x7f82f4076060 msgr2=0x7f82f4077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 shutdown_connections 2024-02-28T23:21:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.596+0000 7f82fa800700 1 -- 172.21.15.57:0/1943025157 wait complete. 2024-02-28T23:21:55.596 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 -- 172.21.15.57:0/3406359855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c10c8f0 msgr2=0x7fe31c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 --2- 172.21.15.57:0/3406359855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c10c8f0 0x7fe31c10ccc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe310009910 tx=0x7fe310009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 -- 172.21.15.57:0/3406359855 shutdown_connections 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 --2- 172.21.15.57:0/3406359855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe31c106b80 0x7fe31c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 --2- 172.21.15.57:0/3406359855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c10c8f0 0x7fe31c10ccc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 -- 172.21.15.57:0/3406359855 >> 172.21.15.57:0/3406359855 conn(0x7fe31c076060 msgr2=0x7fe31c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 -- 172.21.15.57:0/3406359855 shutdown_connections 2024-02-28T23:21:55.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.756+0000 7fe323400700 1 -- 172.21.15.57:0/3406359855 wait complete. 2024-02-28T23:21:55.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.757+0000 7fe323400700 1 Processor -- start 2024-02-28T23:21:55.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe323400700 1 -- start start 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe323400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe323400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe31c10c8f0 0x7fe31c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe323400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c19d120 con 0x7fe31c106b80 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe323400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1a0eb0 con 0x7fe31c10c8f0 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe320a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe320a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49844/0 (socket says 172.21.15.57:49844) 2024-02-28T23:21:55.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.758+0000 7fe320a00700 1 -- 172.21.15.57:0/2562911030 learned_addr learned my addr 172.21.15.57:0/2562911030 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe320a00700 1 -- 172.21.15.57:0/2562911030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe31c10c8f0 msgr2=0x7fe31c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe320a00700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe31c10c8f0 0x7fe31c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe320a00700 1 -- 172.21.15.57:0/2562911030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100095f0 con 0x7fe31c106b80 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe320a00700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe31000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31001d070 con 0x7fe31c106b80 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe31000f460 con 0x7fe31c106b80 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1a1130 con 0x7fe31c106b80 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.759+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe310017630 con 0x7fe31c106b80 2024-02-28T23:21:55.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.760+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1a1620 con 0x7fe31c106b80 2024-02-28T23:21:55.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.761+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c109b20 con 0x7fe31c106b80 2024-02-28T23:21:55.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.762+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe31000f5d0 con 0x7fe31c106b80 2024-02-28T23:21:55.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.763+0000 7fe319600700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:55.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.763+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe31008ea10 con 0x7fe31c106b80 2024-02-28T23:21:55.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.763+0000 7fe31be00700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:55.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.764+0000 7fe31be00700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe308005950 tx=0x7fe3080058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:55.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:55.768+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3100581a0 con 0x7fe31c106b80 2024-02-28T23:21:56.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.010+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe31c1a1840 con 0x7fe30406d8a0 2024-02-28T23:21:56.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.013+0000 7fe319600700 1 -- 172.21.15.57:0/2562911030 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe31c1a1840 con 0x7fe30406d8a0 2024-02-28T23:21:56.011 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:21:56.011 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:21:56.011 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:21:56.012 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 msgr2=0x7fe30406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe308005950 tx=0x7fe3080058e0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 msgr2=0x7fe31c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe31000be20 comp rx=0 tx=0).stop 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 shutdown_connections 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe30406d8a0 0x7fe30406fd50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe31c106b80 0x7fe31c19c500 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 --2- 172.21.15.57:0/2562911030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe31c10c8f0 0x7fe31c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 >> 172.21.15.57:0/2562911030 conn(0x7fe31c076060 msgr2=0x7fe31c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.016+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 shutdown_connections 2024-02-28T23:21:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.017+0000 7fe323400700 1 -- 172.21.15.57:0/2562911030 wait complete. 2024-02-28T23:21:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:55 smithi084 ceph-mon[70440]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:55 smithi084 ceph-mon[70440]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:55 smithi084 ceph-mon[70440]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:55 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/409859703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:21:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:55 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1943025157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 -- 172.21.15.57:0/2776796888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 msgr2=0x7f3f3810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 --2- 172.21.15.57:0/2776796888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f3810c580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009910 tx=0x7f3f20009c20 comp rx=0 tx=0).stop 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 -- 172.21.15.57:0/2776796888 shutdown_connections 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 --2- 172.21.15.57:0/2776796888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f3810c580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 --2- 172.21.15.57:0/2776796888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 0x7f3f380734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 -- 172.21.15.57:0/2776796888 >> 172.21.15.57:0/2776796888 conn(0x7f3f380fbda0 msgr2=0x7f3f380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:56.163 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:55 smithi057 ceph-mon[89491]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:56.163 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:55 smithi057 ceph-mon[89491]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:56.163 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:55 smithi057 ceph-mon[89491]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:56.163 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:55 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/409859703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:21:56.164 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:55 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1943025157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:21:56.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 -- 172.21.15.57:0/2776796888 shutdown_connections 2024-02-28T23:21:56.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.164+0000 7f3f3d000700 1 -- 172.21.15.57:0/2776796888 wait complete. 2024-02-28T23:21:56.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 Processor -- start 2024-02-28T23:21:56.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 -- start start 2024-02-28T23:21:56.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 0x7f3f38197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:56.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:56.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3819c970 con 0x7f3f38073a00 2024-02-28T23:21:56.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.166+0000 7f3f3d000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3819cae0 con 0x7f3f380730f0 2024-02-28T23:21:56.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49854/0 (socket says 172.21.15.57:49854) 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 -- 172.21.15.57:0/1393568891 learned_addr learned my addr 172.21.15.57:0/1393568891 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f35e00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 0x7f3f38197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 -- 172.21.15.57:0/1393568891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 msgr2=0x7f3f38197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 0x7f3f38197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 -- 172.21.15.57:0/1393568891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200095f0 con 0x7f3f38073a00 2024-02-28T23:21:56.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.167+0000 7f3f2fe00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b5c0 tx=0x7f3f2000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.168+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2001d070 con 0x7f3f38073a00 2024-02-28T23:21:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.168+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f2000f460 con 0x7f3f38073a00 2024-02-28T23:21:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.168+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f20017630 con 0x7f3f38073a00 2024-02-28T23:21:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.168+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3819cdc0 con 0x7f3f38073a00 2024-02-28T23:21:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.168+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3819d310 con 0x7f3f38073a00 2024-02-28T23:21:56.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.169+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38109c80 con 0x7f3f38073a00 2024-02-28T23:21:56.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.170+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3f2000f5d0 con 0x7f3f38073a00 2024-02-28T23:21:56.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.171+0000 7f3f2ea00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 0x7f3f1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:21:56.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.171+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3f2008eab0 con 0x7f3f38073a00 2024-02-28T23:21:56.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.172+0000 7f3f35e00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 0x7f3f1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:21:56.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.172+0000 7f3f35e00700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 0x7f3f1c06fd50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005fd0 tx=0x7f3f2800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:21:56.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.176+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f20090e60 con 0x7f3f38073a00 2024-02-28T23:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.481+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3f38199480 con 0x7f3f38073a00 2024-02-28T23:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.482+0000 7f3f2ea00700 1 -- 172.21.15.57:0/1393568891 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3f2005bd70 con 0x7f3f38073a00 2024-02-28T23:21:56.481 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:21:56.481 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:21:56.481 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:21:56.481 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:21:56.481 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:21:56.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 msgr2=0x7f3f1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 0x7f3f1c06fd50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005fd0 tx=0x7f3f2800a560 comp rx=0 tx=0).stop 2024-02-28T23:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 msgr2=0x7f3f38198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b5c0 tx=0x7f3f2000be00 comp rx=0 tx=0).stop 2024-02-28T23:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 shutdown_connections 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3f1c06d8a0 0x7f3f1c06fd50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f380730f0 0x7f3f38197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 --2- 172.21.15.57:0/1393568891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f38073a00 0x7f3f38198630 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.485+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 >> 172.21.15.57:0/1393568891 conn(0x7f3f380fbda0 msgr2=0x7f3f38106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.486+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 shutdown_connections 2024-02-28T23:21:56.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:21:56.486+0000 7f3f3d000700 1 -- 172.21.15.57:0/1393568891 wait complete. 2024-02-28T23:21:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:56 smithi057 ceph-mon[89491]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:56 smithi057 ceph-mon[89491]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:56 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1393568891' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:21:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:56 smithi084 ceph-mon[70440]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:21:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:56 smithi084 ceph-mon[70440]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:56 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1393568891' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:21:59.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:21:59 smithi057 ceph-mon[89491]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:21:59.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:21:59 smithi084 ceph-mon[70440]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:01 smithi057 ceph-mon[89491]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:01.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:01 smithi084 ceph-mon[70440]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:03.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:03 smithi057 ceph-mon[89491]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:03.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:03.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:03 smithi084 ceph-mon[70440]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:03.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:05.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:05 smithi057 ceph-mon[89491]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:05.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:05 smithi084 ceph-mon[70440]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:07.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:07 smithi057 ceph-mon[89491]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:07 smithi084 ceph-mon[70440]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:09.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:09 smithi057 ceph-mon[89491]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:09 smithi084 ceph-mon[70440]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:11.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:11 smithi057 ceph-mon[89491]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:11.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:11 smithi084 ceph-mon[70440]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:13.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:13 smithi057 ceph-mon[89491]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:13.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:13 smithi084 ceph-mon[70440]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:15.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:15 smithi057 ceph-mon[89491]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:15.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:15 smithi084 ceph-mon[70440]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:17 smithi057 ceph-mon[89491]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:17.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:17 smithi084 ceph-mon[70440]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:19 smithi084 ceph-mon[70440]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:19.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:19 smithi057 ceph-mon[89491]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:21.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:21 smithi057 ceph-mon[89491]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:21.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:21 smithi084 ceph-mon[70440]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:23 smithi057 ceph-mon[89491]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:23 smithi084 ceph-mon[70440]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:25 smithi057 ceph-mon[89491]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:25.832 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:25 smithi084 ceph-mon[70440]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.660+0000 7f0db4000700 1 -- 172.21.15.57:0/1950675053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac1004e0 msgr2=0x7f0dac1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.660+0000 7f0db4000700 1 --2- 172.21.15.57:0/1950675053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac1004e0 0x7f0dac1008b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009910 tx=0x7f0da0009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 -- 172.21.15.57:0/1950675053 shutdown_connections 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 --2- 172.21.15.57:0/1950675053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac100e80 0x7f0dac104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 --2- 172.21.15.57:0/1950675053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac1004e0 0x7f0dac1008b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 -- 172.21.15.57:0/1950675053 >> 172.21.15.57:0/1950675053 conn(0x7f0dac0fbda0 msgr2=0x7f0dac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 -- 172.21.15.57:0/1950675053 shutdown_connections 2024-02-28T23:22:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.661+0000 7f0db4000700 1 -- 172.21.15.57:0/1950675053 wait complete. 2024-02-28T23:22:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 Processor -- start 2024-02-28T23:22:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 -- start start 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac1004e0 0x7f0dac073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac073ad0 con 0x7f0dac100e80 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.662+0000 7f0db4000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac073c10 con 0x7f0dac1004e0 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46494/0 (socket says 172.21.15.57:46494) 2024-02-28T23:22:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 -- 172.21.15.57:0/1257993100 learned_addr learned my addr 172.21.15.57:0/1257993100 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:26.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db1600700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac1004e0 0x7f0dac073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 -- 172.21.15.57:0/1257993100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac1004e0 msgr2=0x7f0dac073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac1004e0 0x7f0dac073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.663+0000 7f0db0c00700 1 -- 172.21.15.57:0/1257993100 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da00095f0 con 0x7f0dac100e80 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0db0c00700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800e8f0 tx=0x7f0d9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d98009790 con 0x7f0dac100e80 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d98004cf0 con 0x7f0dac100e80 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac1a20c0 con 0x7f0dac100e80 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d98005690 con 0x7f0dac100e80 2024-02-28T23:22:26.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.664+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac1a24d0 con 0x7f0dac100e80 2024-02-28T23:22:26.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.665+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac1087f0 con 0x7f0dac100e80 2024-02-28T23:22:26.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.667+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0d98004750 con 0x7f0dac100e80 2024-02-28T23:22:26.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.667+0000 7f0daa000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 0x7f0d9406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:26.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.668+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d98014070 con 0x7f0dac100e80 2024-02-28T23:22:26.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.668+0000 7f0db1600700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 0x7f0d9406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:26.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.669+0000 7f0db1600700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 0x7f0d9406fd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0da000b760 tx=0x7f0da0009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:26.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.673+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d980570d0 con 0x7f0dac100e80 2024-02-28T23:22:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.922+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dac061190 con 0x7f0d9406d8a0 2024-02-28T23:22:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.924+0000 7f0daa000700 1 -- 172.21.15.57:0/1257993100 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0dac061190 con 0x7f0d9406d8a0 2024-02-28T23:22:26.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.926+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 msgr2=0x7f0d9406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:26.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.926+0000 7f0db4000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 0x7f0d9406fd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0da000b760 tx=0x7f0da0009f90 comp rx=0 tx=0).stop 2024-02-28T23:22:26.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.926+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 msgr2=0x7f0dac073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:26.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.926+0000 7f0db4000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800e8f0 tx=0x7f0d9800ecb0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 shutdown_connections 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d9406d8a0 0x7f0d9406fd50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dac1004e0 0x7f0dac073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 --2- 172.21.15.57:0/1257993100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dac100e80 0x7f0dac073590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 >> 172.21.15.57:0/1257993100 conn(0x7f0dac0fbda0 msgr2=0x7f0dac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 shutdown_connections 2024-02-28T23:22:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:26.927+0000 7f0db4000700 1 -- 172.21.15.57:0/1257993100 wait complete. 2024-02-28T23:22:26.943 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 -- 172.21.15.57:0/2100338182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 msgr2=0x7f47ac106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 --2- 172.21.15.57:0/2100338182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac106ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4794009910 tx=0x7f4794009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 -- 172.21.15.57:0/2100338182 shutdown_connections 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 --2- 172.21.15.57:0/2100338182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac106ff0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 --2- 172.21.15.57:0/2100338182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 -- 172.21.15.57:0/2100338182 >> 172.21.15.57:0/2100338182 conn(0x7f47ac076060 msgr2=0x7f47ac078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 -- 172.21.15.57:0/2100338182 shutdown_connections 2024-02-28T23:22:27.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.096+0000 7f47b1200700 1 -- 172.21.15.57:0/2100338182 wait complete. 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.097+0000 7f47b1200700 1 Processor -- start 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47b1200700 1 -- start start 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47b1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47b1200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47b1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac19d060 con 0x7f47ac106b80 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47b1200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a0df0 con 0x7f47ac10c8f0 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47a9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47a9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46516/0 (socket says 172.21.15.57:46516) 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.098+0000 7f47a9e00700 1 -- 172.21.15.57:0/552195728 learned_addr learned my addr 172.21.15.57:0/552195728 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:27.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a3e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a9e00700 1 -- 172.21.15.57:0/552195728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 msgr2=0x7f47ac19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a9e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a9e00700 1 -- 172.21.15.57:0/552195728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940095f0 con 0x7f47ac106b80 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a3e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a9e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d8d0 tx=0x7f479c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c009940 con 0x7f47ac106b80 2024-02-28T23:22:27.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac1a10d0 con 0x7f47ac106b80 2024-02-28T23:22:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f479c010460 con 0x7f47ac106b80 2024-02-28T23:22:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.099+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c00f5f0 con 0x7f47ac106b80 2024-02-28T23:22:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.100+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1a1620 con 0x7f47ac106b80 2024-02-28T23:22:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.101+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac10ecc0 con 0x7f47ac106b80 2024-02-28T23:22:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.102+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f479c0105d0 con 0x7f47ac106b80 2024-02-28T23:22:27.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.103+0000 7f47a2a00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 0x7f479006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.103+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f479c08d4c0 con 0x7f47ac106b80 2024-02-28T23:22:27.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.103+0000 7f47a3e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 0x7f479006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.104+0000 7f47a3e00700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 0x7f479006fd50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f479400b5c0 tx=0x7f4794005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.108+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f479c056c50 con 0x7f47ac106b80 2024-02-28T23:22:27.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.353+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47ac1a1900 con 0x7f479006d8a0 2024-02-28T23:22:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.356+0000 7f47a2a00700 1 -- 172.21.15.57:0/552195728 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47ac1a1900 con 0x7f479006d8a0 2024-02-28T23:22:27.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 msgr2=0x7f479006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 0x7f479006fd50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f479400b5c0 tx=0x7f4794005960 comp rx=0 tx=0).stop 2024-02-28T23:22:27.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 msgr2=0x7f47ac19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d8d0 tx=0x7f479c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 shutdown_connections 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f479006d8a0 0x7f479006fd50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47ac106b80 0x7f47ac19c440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 --2- 172.21.15.57:0/552195728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47ac10c8f0 0x7f47ac19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 >> 172.21.15.57:0/552195728 conn(0x7f47ac076060 msgr2=0x7f47ac077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 shutdown_connections 2024-02-28T23:22:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.359+0000 7f47b1200700 1 -- 172.21.15.57:0/552195728 wait complete. 2024-02-28T23:22:27.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 -- 172.21.15.57:0/527060469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 msgr2=0x7fda3c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 --2- 172.21.15.57:0/527060469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c1008b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009910 tx=0x7fda2c009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 -- 172.21.15.57:0/527060469 shutdown_connections 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 --2- 172.21.15.57:0/527060469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 --2- 172.21.15.57:0/527060469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c1008b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 -- 172.21.15.57:0/527060469 >> 172.21.15.57:0/527060469 conn(0x7fda3c0fbda0 msgr2=0x7fda3c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 -- 172.21.15.57:0/527060469 shutdown_connections 2024-02-28T23:22:27.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.513+0000 7fda40c00700 1 -- 172.21.15.57:0/527060469 wait complete. 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.514+0000 7fda40c00700 1 Processor -- start 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.514+0000 7fda40c00700 1 -- start start 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda40c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda40c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda40c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c196b10 con 0x7fda3c1004e0 2024-02-28T23:22:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda40c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c19a8a0 con 0x7fda3c100e80 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda39e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda39e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46532/0 (socket says 172.21.15.57:46532) 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda39e00700 1 -- 172.21.15.57:0/3590895751 learned_addr learned my addr 172.21.15.57:0/3590895751 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.515+0000 7fda39400700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda39e00700 1 -- 172.21.15.57:0/3590895751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 msgr2=0x7fda3c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda39e00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda39e00700 1 -- 172.21.15.57:0/3590895751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0095f0 con 0x7fda3c1004e0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda39400700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda39e00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b5c0 tx=0x7fda2c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda2c01d070 con 0x7fda3c1004e0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda2c00f460 con 0x7fda3c1004e0 2024-02-28T23:22:27.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c19ab20 con 0x7fda3c1004e0 2024-02-28T23:22:27.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda2c017610 con 0x7fda3c1004e0 2024-02-28T23:22:27.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.516+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c19b010 con 0x7fda3c1004e0 2024-02-28T23:22:27.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.517+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c1087f0 con 0x7fda3c1004e0 2024-02-28T23:22:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.519+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fda2c00f5d0 con 0x7fda3c1004e0 2024-02-28T23:22:27.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.520+0000 7fda32a00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 0x7fda2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.520+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda2c08ebe0 con 0x7fda3c1004e0 2024-02-28T23:22:27.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.520+0000 7fda39400700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 0x7fda2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.520+0000 7fda39400700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 0x7fda2006fd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda28005fd0 tx=0x7fda28005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.524+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda2c0582f0 con 0x7fda3c1004e0 2024-02-28T23:22:27.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.762+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda3c19b230 con 0x7fda2006d8a0 2024-02-28T23:22:27.762 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:27 smithi057 ceph-mon[89491]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:27.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.778+0000 7fda32a00700 1 -- 172.21.15.57:0/3590895751 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fda3c19b230 con 0x7fda2006d8a0 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (12m) 3m ago 25m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (22m) 3m ago 26m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (24m) 3m ago 24m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (8m) 3m ago 26m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (8m) 3m ago 24m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (11m) 3m ago 25m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (4m) 3m ago 18m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (3m) 3m ago 18m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (18m) 3m ago 18m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:22:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (3m) 3m ago 18m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (14m) 3m ago 27m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (13m) 3m ago 24m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (9m) 3m ago 28m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (9m) 3m ago 24m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (13m) 3m ago 26m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (12m) 3m ago 24m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (7m) 3m ago 23m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (6m) 3m ago 22m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (6m) 3m ago 22m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (5m) 3m ago 21m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (5m) 3m ago 21m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (4m) 3m ago 20m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:22:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (12m) 3m ago 25m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.783+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 msgr2=0x7fda2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.783+0000 7fda40c00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 0x7fda2006fd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda28005fd0 tx=0x7fda28005dc0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.783+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 msgr2=0x7fda3c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.783+0000 7fda40c00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b5c0 tx=0x7fda2c00be00 comp rx=0 tx=0).stop 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.784+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 shutdown_connections 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.784+0000 7fda40c00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2006d8a0 0x7fda2006fd50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.784+0000 7fda40c00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda3c1004e0 0x7fda3c195ef0 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b5c0 tx=0x7fda2c00be00 comp rx=0 tx=0).stop 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.784+0000 7fda40c00700 1 --2- 172.21.15.57:0/3590895751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3c100e80 0x7fda3c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.784+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 >> 172.21.15.57:0/3590895751 conn(0x7fda3c0fbda0 msgr2=0x7fda3c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:27.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.787+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 shutdown_connections 2024-02-28T23:22:27.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.787+0000 7fda40c00700 1 -- 172.21.15.57:0/3590895751 wait complete. 2024-02-28T23:22:27.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:27 smithi084 ceph-mon[70440]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:27.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 -- 172.21.15.57:0/3365740746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068ee0 msgr2=0x7feee0110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 --2- 172.21.15.57:0/3365740746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068ee0 0x7feee0110a10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feecc009940 tx=0x7feecc009c50 comp rx=0 tx=0).stop 2024-02-28T23:22:27.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 -- 172.21.15.57:0/3365740746 shutdown_connections 2024-02-28T23:22:27.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 --2- 172.21.15.57:0/3365740746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068ee0 0x7feee0110a10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 --2- 172.21.15.57:0/3365740746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0068540 0x7feee0068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.937+0000 7feee5c00700 1 -- 172.21.15.57:0/3365740746 >> 172.21.15.57:0/3365740746 conn(0x7feee0075130 msgr2=0x7feee0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.938+0000 7feee5c00700 1 -- 172.21.15.57:0/3365740746 shutdown_connections 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.938+0000 7feee5c00700 1 -- 172.21.15.57:0/3365740746 wait complete. 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.939+0000 7feee5c00700 1 Processor -- start 2024-02-28T23:22:27.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.939+0000 7feee5c00700 1 -- start start 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feee5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feee5c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee01a2ad0 0x7feee019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feee5c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01a3070 con 0x7feee0068540 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feee5c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee019cac0 con 0x7feee01a2ad0 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46538/0 (socket says 172.21.15.57:46538) 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 -- 172.21.15.57:0/3161528995 learned_addr learned my addr 172.21.15.57:0/3161528995 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feede000700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee01a2ad0 0x7feee019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 -- 172.21.15.57:0/3161528995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee01a2ad0 msgr2=0x7feee019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee01a2ad0 0x7feee019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.940+0000 7feedea00700 1 -- 172.21.15.57:0/3161528995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0095f0 con 0x7feee0068540 2024-02-28T23:22:27.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feedea00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feed000d8d0 tx=0x7feed000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feed0009940 con 0x7feee0068540 2024-02-28T23:22:27.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feed0010460 con 0x7feee0068540 2024-02-28T23:22:27.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee019cd50 con 0x7feee0068540 2024-02-28T23:22:27.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feed000f5f0 con 0x7feee0068540 2024-02-28T23:22:27.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.941+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee019d2a0 con 0x7feee0068540 2024-02-28T23:22:27.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.942+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee0101410 con 0x7feee0068540 2024-02-28T23:22:27.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.943+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7feed00105d0 con 0x7feee0068540 2024-02-28T23:22:27.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.944+0000 7feed7400700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 0x7feec406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:27.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.944+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feed008d5d0 con 0x7feee0068540 2024-02-28T23:22:27.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.945+0000 7feede000700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 0x7feec406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:27.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.945+0000 7feede000700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 0x7feec406fd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b580 tx=0x7feecc005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:27.949+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feed0056ce0 con 0x7feee0068540 2024-02-28T23:22:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.271+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feee019dab0 con 0x7feee0068540 2024-02-28T23:22:28.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.272+0000 7feed7400700 1 -- 172.21.15.57:0/3161528995 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7feed005a3a0 con 0x7feee0068540 2024-02-28T23:22:28.271 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:22:28.271 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:22:28.272 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:22:28.273 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:22:28.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 msgr2=0x7feec406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 0x7feec406fd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b580 tx=0x7feecc005960 comp rx=0 tx=0).stop 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 msgr2=0x7feee01a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feed000d8d0 tx=0x7feed000dc90 comp rx=0 tx=0).stop 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 shutdown_connections 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feec406d8a0 0x7feec406fd50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feee0068540 0x7feee01a2590 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 --2- 172.21.15.57:0/3161528995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee01a2ad0 0x7feee019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 >> 172.21.15.57:0/3161528995 conn(0x7feee0075130 msgr2=0x7feee01106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 shutdown_connections 2024-02-28T23:22:28.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.275+0000 7feee5c00700 1 -- 172.21.15.57:0/3161528995 wait complete. 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 -- 172.21.15.57:0/2163116753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c102470 msgr2=0x7f700c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 --2- 172.21.15.57:0/2163116753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c102470 0x7f700c1028e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009910 tx=0x7f6ff8009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 -- 172.21.15.57:0/2163116753 shutdown_connections 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 --2- 172.21.15.57:0/2163116753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c102470 0x7f700c1028e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 --2- 172.21.15.57:0/2163116753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c108470 0x7f700c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.437+0000 7f7011400700 1 -- 172.21.15.57:0/2163116753 >> 172.21.15.57:0/2163116753 conn(0x7f700c0fdf90 msgr2=0x7f700c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:28.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.438+0000 7f7011400700 1 -- 172.21.15.57:0/2163116753 shutdown_connections 2024-02-28T23:22:28.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.438+0000 7f7011400700 1 -- 172.21.15.57:0/2163116753 wait complete. 2024-02-28T23:22:28.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.439+0000 7f7011400700 1 Processor -- start 2024-02-28T23:22:28.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.440+0000 7f7011400700 1 -- start start 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.440+0000 7f7011400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c102470 0x7f700c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.440+0000 7f7011400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.440+0000 7f7011400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c198e00 con 0x7f700c108470 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.440+0000 7f7011400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c198f40 con 0x7f700c102470 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46564/0 (socket says 172.21.15.57:46564) 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 -- 172.21.15.57:0/3880077438 learned_addr learned my addr 172.21.15.57:0/3880077438 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:28.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f700a800700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c102470 0x7f700c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 -- 172.21.15.57:0/3880077438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c102470 msgr2=0x7f700c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c102470 0x7f700c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.441+0000 7f7009e00700 1 -- 172.21.15.57:0/3880077438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80095f0 con 0x7f700c108470 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7009e00700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b5c0 tx=0x7f6ff800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff801d070 con 0x7f700c108470 2024-02-28T23:22:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c19cd30 con 0x7f700c108470 2024-02-28T23:22:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff800f460 con 0x7f700c108470 2024-02-28T23:22:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c19d220 con 0x7f700c108470 2024-02-28T23:22:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.442+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff80225f0 con 0x7f700c108470 2024-02-28T23:22:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.443+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c10a970 con 0x7f700c108470 2024-02-28T23:22:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.444+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6ff800f5d0 con 0x7f700c108470 2024-02-28T23:22:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.445+0000 7f7003400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 0x7f6ff006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.445+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6ff808edb0 con 0x7f700c108470 2024-02-28T23:22:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.445+0000 7f700a800700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 0x7f6ff006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.446+0000 7f700a800700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 0x7f6ff006fd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005ea0 tx=0x7f6ffc005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.451+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ff80584c0 con 0x7f700c108470 2024-02-28T23:22:28.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:28 smithi057 ceph-mon[89491]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:28 smithi057 ceph-mon[89491]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:28 smithi057 ceph-mon[89491]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:28 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3161528995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:22:28.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.727+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f700c04f2a0 con 0x7f700c108470 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.727+0000 7f7003400700 1 -- 172.21.15.57:0/3880077438 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f6ff805bb80 con 0x7f700c108470 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:22:28.729 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:22:28.730 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:22:28.731 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:22:28.732 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:22:28.733 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:22:28.733 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:22:28.733 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.730+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 msgr2=0x7f6ff006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.730+0000 7f7011400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 0x7f6ff006fd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005ea0 tx=0x7f6ffc005e30 comp rx=0 tx=0).stop 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.730+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 msgr2=0x7f700c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.730+0000 7f7011400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b5c0 tx=0x7f6ff800be00 comp rx=0 tx=0).stop 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 shutdown_connections 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ff006d8a0 0x7f6ff006fd50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f700c102470 0x7f700c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 --2- 172.21.15.57:0/3880077438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f700c108470 0x7f700c1987b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 >> 172.21.15.57:0/3880077438 conn(0x7f700c0fdf90 msgr2=0x7f700c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:28.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 shutdown_connections 2024-02-28T23:22:28.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.731+0000 7f7011400700 1 -- 172.21.15.57:0/3880077438 wait complete. 2024-02-28T23:22:28.735 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:22:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:28 smithi084 ceph-mon[70440]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:28 smithi084 ceph-mon[70440]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:28 smithi084 ceph-mon[70440]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:28 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3161528995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 -- 172.21.15.57:0/1365297900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 msgr2=0x7f7368104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 --2- 172.21.15.57:0/1365297900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368104c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7354009910 tx=0x7f7354009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 -- 172.21.15.57:0/1365297900 shutdown_connections 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 --2- 172.21.15.57:0/1365297900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368104c30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 --2- 172.21.15.57:0/1365297900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 -- 172.21.15.57:0/1365297900 >> 172.21.15.57:0/1365297900 conn(0x7f73680fbb60 msgr2=0x7f73680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 -- 172.21.15.57:0/1365297900 shutdown_connections 2024-02-28T23:22:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.891+0000 7f736e000700 1 -- 172.21.15.57:0/1365297900 wait complete. 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.892+0000 7f736e000700 1 Processor -- start 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.892+0000 7f736e000700 1 -- start start 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736e000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736e000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7368198a20 con 0x7f7368100c60 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736e000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736819c7b0 con 0x7f73681002c0 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46576/0 (socket says 172.21.15.57:46576) 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 -- 172.21.15.57:0/2384762581 learned_addr learned my addr 172.21.15.57:0/2384762581 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736cc00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 -- 172.21.15.57:0/2384762581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 msgr2=0x7f7368197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:28.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f7367e00700 1 -- 172.21.15.57:0/2384762581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540095f0 con 0x7f7368100c60 2024-02-28T23:22:28.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.893+0000 7f736cc00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:22:28.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f7367e00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f735400bda0 tx=0x7f735400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:28.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735401d070 con 0x7f7368100c60 2024-02-28T23:22:28.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7354004fb0 con 0x7f7368100c60 2024-02-28T23:22:28.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736819ca30 con 0x7f7368100c60 2024-02-28T23:22:28.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735400f460 con 0x7f7368100c60 2024-02-28T23:22:28.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.894+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736819cf20 con 0x7f7368100c60 2024-02-28T23:22:28.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.895+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73681043b0 con 0x7f7368100c60 2024-02-28T23:22:28.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.896+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7354022800 con 0x7f7368100c60 2024-02-28T23:22:28.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.897+0000 7f7365600700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 0x7f735006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:28.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.897+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f735408eb00 con 0x7f7368100c60 2024-02-28T23:22:28.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.898+0000 7f736cc00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 0x7f735006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:28.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.898+0000 7f736cc00700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 0x7f735006fd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f735c005fd0 tx=0x7f735c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:28.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:28.902+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73540583e0 con 0x7f7368100c60 2024-02-28T23:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.150+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f736819d200 con 0x7f735006d8a0 2024-02-28T23:22:29.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.152+0000 7f7365600700 1 -- 172.21.15.57:0/2384762581 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f736819d200 con 0x7f735006d8a0 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:22:29.152 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:22:29.153 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:22:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 msgr2=0x7f735006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 0x7f735006fd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f735c005fd0 tx=0x7f735c005dc0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 msgr2=0x7f7368198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f735400bda0 tx=0x7f735400be80 comp rx=0 tx=0).stop 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 shutdown_connections 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f735006d8a0 0x7f735006fd50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73681002c0 0x7f7368197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 --2- 172.21.15.57:0/2384762581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7368100c60 0x7f7368198340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.155+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 >> 172.21.15.57:0/2384762581 conn(0x7f73680fbb60 msgr2=0x7f73680fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.156+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 shutdown_connections 2024-02-28T23:22:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.156+0000 7f736e000700 1 -- 172.21.15.57:0/2384762581 wait complete. 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.315+0000 7f2aa8800700 1 -- 172.21.15.57:0/632918021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 msgr2=0x7f2aa0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.315+0000 7f2aa8800700 1 --2- 172.21.15.57:0/632918021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa0073440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009940 tx=0x7f2a94009c50 comp rx=0 tx=0).stop 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 -- 172.21.15.57:0/632918021 shutdown_connections 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 --2- 172.21.15.57:0/632918021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 0x7f2aa010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 --2- 172.21.15.57:0/632918021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa0073440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 -- 172.21.15.57:0/632918021 >> 172.21.15.57:0/632918021 conn(0x7f2aa00fbda0 msgr2=0x7f2aa00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 -- 172.21.15.57:0/632918021 shutdown_connections 2024-02-28T23:22:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.316+0000 7f2aa8800700 1 -- 172.21.15.57:0/632918021 wait complete. 2024-02-28T23:22:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 Processor -- start 2024-02-28T23:22:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 -- start start 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 0x7f2aa0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0198df0 con 0x7f2aa0073070 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.317+0000 7f2aa8800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa019cb80 con 0x7f2aa0073980 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46590/0 (socket says 172.21.15.57:46590) 2024-02-28T23:22:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 -- 172.21.15.57:0/3012320843 learned_addr learned my addr 172.21.15.57:0/3012320843 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5400700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 0x7f2aa0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 -- 172.21.15.57:0/3012320843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 msgr2=0x7f2aa0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 0x7f2aa0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.318+0000 7f2aa5e00700 1 -- 172.21.15.57:0/3012320843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940095f0 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2aa5e00700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a940055f0 tx=0x7f2a940056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9401d070 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a9400f460 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa019ce00 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a94021640 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.319+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa019d2f0 con 0x7f2aa0073070 2024-02-28T23:22:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.320+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa0109d00 con 0x7f2aa0073070 2024-02-28T23:22:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.321+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a9400fa60 con 0x7f2aa0073070 2024-02-28T23:22:29.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.322+0000 7f2a9aa00700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 0x7f2a8806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:29.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.322+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2a9408e7f0 con 0x7f2aa0073070 2024-02-28T23:22:29.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.322+0000 7f2aa5400700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 0x7f2a8806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:29.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.323+0000 7f2aa5400700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 0x7f2a8806fd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005950 tx=0x7f2a900058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:29.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.327+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a94057f00 con 0x7f2aa0073070 2024-02-28T23:22:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:29 smithi057 ceph-mon[89491]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3880077438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:22:29.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.639+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2aa004f2a0 con 0x7f2aa0073070 2024-02-28T23:22:29.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.640+0000 7f2a9aa00700 1 -- 172.21.15.57:0/3012320843 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2a9405b5c0 con 0x7f2aa0073070 2024-02-28T23:22:29.640 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:22:29.640 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:22:29.640 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:22:29.640 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:22:29.640 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 msgr2=0x7f2a8806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 0x7f2a8806fd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005950 tx=0x7f2a900058e0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 msgr2=0x7f2aa01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a940055f0 tx=0x7f2a940056d0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 shutdown_connections 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2a8806d8a0 0x7f2a8806fd50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2aa0073070 0x7f2aa01981d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 --2- 172.21.15.57:0/3012320843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0073980 0x7f2aa0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.643+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 >> 172.21.15.57:0/3012320843 conn(0x7f2aa00fbda0 msgr2=0x7f2aa0106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.644+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 shutdown_connections 2024-02-28T23:22:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:29.644+0000 7f2aa8800700 1 -- 172.21.15.57:0/3012320843 wait complete. 2024-02-28T23:22:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:29 smithi084 ceph-mon[70440]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3880077438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:22:30.681 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:30 smithi057 ceph-mon[89491]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:30.681 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3012320843' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:22:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:30 smithi084 ceph-mon[70440]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:22:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3012320843' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:22:31.780 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:31 smithi084 ceph-mon[70440]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:31 smithi057 ceph-mon[89491]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:33 smithi057 ceph-mon[89491]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:33 smithi084 ceph-mon[70440]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:35 smithi057 ceph-mon[89491]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:35.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:35 smithi084 ceph-mon[70440]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:37.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:37 smithi057 ceph-mon[89491]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:37 smithi084 ceph-mon[70440]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:39 smithi057 ceph-mon[89491]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:39.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:39 smithi084 ceph-mon[70440]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:41 smithi057 ceph-mon[89491]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:41.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:41 smithi084 ceph-mon[70440]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:43.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:43 smithi057 ceph-mon[89491]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:43.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:43 smithi084 ceph-mon[70440]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:45.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:45 smithi057 ceph-mon[89491]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:45.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:45 smithi084 ceph-mon[70440]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:47.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:47 smithi084 ceph-mon[70440]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:47.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:47 smithi057 ceph-mon[89491]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:48.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:22:49.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:49 smithi057 ceph-mon[89491]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:49.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:49 smithi084 ceph-mon[70440]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:51 smithi057 ceph-mon[89491]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:51 smithi084 ceph-mon[70440]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:53.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:53 smithi057 ceph-mon[89491]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:53.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:53 smithi084 ceph-mon[70440]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:55.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:55 smithi057 ceph-mon[89491]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:55.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:55 smithi084 ceph-mon[70440]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:57.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:57 smithi057 ceph-mon[89491]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:57.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:57 smithi084 ceph-mon[70440]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:59.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 -- 172.21.15.57:0/3648594009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb32410c8f0 msgr2=0x7fb32410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:59.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 --2- 172.21.15.57:0/3648594009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb32410c8f0 0x7fb32410ccc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb310009910 tx=0x7fb310009c20 comp rx=0 tx=0).stop 2024-02-28T23:22:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:22:59 smithi057 ceph-mon[89491]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 -- 172.21.15.57:0/3648594009 shutdown_connections 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 --2- 172.21.15.57:0/3648594009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb324106b80 0x7fb324106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 --2- 172.21.15.57:0/3648594009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb32410c8f0 0x7fb32410ccc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 -- 172.21.15.57:0/3648594009 >> 172.21.15.57:0/3648594009 conn(0x7fb324076060 msgr2=0x7fb324078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 -- 172.21.15.57:0/3648594009 shutdown_connections 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.819+0000 7fb32c800700 1 -- 172.21.15.57:0/3648594009 wait complete. 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.820+0000 7fb32c800700 1 Processor -- start 2024-02-28T23:22:59.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb32c800700 1 -- start start 2024-02-28T23:22:59.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb32c800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:59.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb32c800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 0x7fb32419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:59.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb32c800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32419d120 con 0x7fb324106b80 2024-02-28T23:22:59.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb32c800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241a0eb0 con 0x7fb32410c8f0 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb329e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb329e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50970/0 (socket says 172.21.15.57:50970) 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.821+0000 7fb329e00700 1 -- 172.21.15.57:0/993204742 learned_addr learned my addr 172.21.15.57:0/993204742 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329400700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 0x7fb32419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329e00700 1 -- 172.21.15.57:0/993204742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 msgr2=0x7fb32419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:22:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329e00700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 0x7fb32419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329e00700 1 -- 172.21.15.57:0/993204742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100095f0 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329400700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 0x7fb32419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb329e00700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3100092e0 tx=0x7fb31000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31001d070 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb310005e00 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.822+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3241a1130 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.823+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb310017400 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.823+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241a1620 con 0x7fb324106b80 2024-02-28T23:22:59.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.824+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324109b20 con 0x7fb324106b80 2024-02-28T23:22:59.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.825+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb31000f460 con 0x7fb324106b80 2024-02-28T23:22:59.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.826+0000 7fb31ea00700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 0x7fb30c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:22:59.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.826+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb31008ea30 con 0x7fb324106b80 2024-02-28T23:22:59.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.826+0000 7fb329400700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 0x7fb30c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:22:59.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.827+0000 7fb329400700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 0x7fb30c06fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb318005fd0 tx=0x7fb318005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:22:59.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:22:59.831+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3100581c0 con 0x7fb324106b80 2024-02-28T23:22:59.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:22:59 smithi084 ceph-mon[70440]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:00.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.082+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3241a1890 con 0x7fb30c06d8a0 2024-02-28T23:23:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.085+0000 7fb31ea00700 1 -- 172.21.15.57:0/993204742 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb3241a1890 con 0x7fb30c06d8a0 2024-02-28T23:23:00.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 msgr2=0x7fb30c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 0x7fb30c06fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb318005fd0 tx=0x7fb318005ee0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 msgr2=0x7fb32419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3100092e0 tx=0x7fb31000be20 comp rx=0 tx=0).stop 2024-02-28T23:23:00.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 shutdown_connections 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb30c06d8a0 0x7fb30c06fd50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb324106b80 0x7fb32419c500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 --2- 172.21.15.57:0/993204742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32410c8f0 0x7fb32419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 >> 172.21.15.57:0/993204742 conn(0x7fb324076060 msgr2=0x7fb324077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 shutdown_connections 2024-02-28T23:23:00.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.088+0000 7fb32c800700 1 -- 172.21.15.57:0/993204742 wait complete. 2024-02-28T23:23:00.102 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 -- 172.21.15.57:0/339638709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 msgr2=0x7f0544108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 --2- 172.21.15.57:0/339638709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544108840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0534009910 tx=0x7f0534009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 -- 172.21.15.57:0/339638709 shutdown_connections 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 --2- 172.21.15.57:0/339638709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 0x7f05441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 --2- 172.21.15.57:0/339638709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544108840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 -- 172.21.15.57:0/339638709 >> 172.21.15.57:0/339638709 conn(0x7f05440fdf90 msgr2=0x7f05441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 -- 172.21.15.57:0/339638709 shutdown_connections 2024-02-28T23:23:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.255+0000 7f054a000700 1 -- 172.21.15.57:0/339638709 wait complete. 2024-02-28T23:23:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.256+0000 7f054a000700 1 Processor -- start 2024-02-28T23:23:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.256+0000 7f054a000700 1 -- start start 2024-02-28T23:23:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f054a000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 0x7f05441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f054a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f054a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544198e10 con 0x7f0544108470 2024-02-28T23:23:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f054a000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054419cba0 con 0x7f0544102470 2024-02-28T23:23:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f0542a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f0542a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50982/0 (socket says 172.21.15.57:50982) 2024-02-28T23:23:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f0542a00700 1 -- 172.21.15.57:0/1214134910 learned_addr learned my addr 172.21.15.57:0/1214134910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.257+0000 7f0543400700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 0x7f05441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f0542a00700 1 -- 172.21.15.57:0/1214134910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 msgr2=0x7f05441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f0542a00700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 0x7f05441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f0542a00700 1 -- 172.21.15.57:0/1214134910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340095f0 con 0x7f0544108470 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f0542a00700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f053000d8d0 tx=0x7f053000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0530009940 con 0x7f0544108470 2024-02-28T23:23:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0530010460 con 0x7f0544108470 2024-02-28T23:23:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054419ce80 con 0x7f0544108470 2024-02-28T23:23:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.258+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053000f5f0 con 0x7f0544108470 2024-02-28T23:23:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.259+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054419d3d0 con 0x7f0544108470 2024-02-28T23:23:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.260+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054410a9e0 con 0x7f0544108470 2024-02-28T23:23:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.261+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f05300105d0 con 0x7f0544108470 2024-02-28T23:23:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.262+0000 7f053be00700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 0x7f052806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.262+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f053008d5d0 con 0x7f0544108470 2024-02-28T23:23:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.262+0000 7f0543400700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 0x7f052806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.263+0000 7f0543400700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 0x7f052806fd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0534006010 tx=0x7f053401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.266+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0530056ce0 con 0x7f0544108470 2024-02-28T23:23:00.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.520+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f054419d5f0 con 0x7f052806d8a0 2024-02-28T23:23:00.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.523+0000 7f053be00700 1 -- 172.21.15.57:0/1214134910 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f054419d5f0 con 0x7f052806d8a0 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.525+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 msgr2=0x7f052806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.525+0000 7f054a000700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 0x7f052806fd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0534006010 tx=0x7f053401a040 comp rx=0 tx=0).stop 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.525+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 msgr2=0x7f0544198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.525+0000 7f054a000700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f053000d8d0 tx=0x7f053000dc90 comp rx=0 tx=0).stop 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 shutdown_connections 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f052806d8a0 0x7f052806fd50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0544102470 0x7f05441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 --2- 172.21.15.57:0/1214134910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0544108470 0x7f0544198730 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 >> 172.21.15.57:0/1214134910 conn(0x7f05440fdf90 msgr2=0x7f05441002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:00.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 shutdown_connections 2024-02-28T23:23:00.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.526+0000 7f054a000700 1 -- 172.21.15.57:0/1214134910 wait complete. 2024-02-28T23:23:00.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 -- 172.21.15.57:0/3007353205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f98102e30 msgr2=0x7f7f98103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 --2- 172.21.15.57:0/3007353205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f98102e30 0x7f7f98103200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009910 tx=0x7f7f8c009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:00.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 -- 172.21.15.57:0/3007353205 shutdown_connections 2024-02-28T23:23:00.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 --2- 172.21.15.57:0/3007353205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f980ff9e0 0x7f7f980ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 --2- 172.21.15.57:0/3007353205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f98102e30 0x7f7f98103200 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.686+0000 7f7f9e000700 1 -- 172.21.15.57:0/3007353205 >> 172.21.15.57:0/3007353205 conn(0x7f7f98074fa0 msgr2=0x7f7f980753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:00.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.687+0000 7f7f9e000700 1 -- 172.21.15.57:0/3007353205 shutdown_connections 2024-02-28T23:23:00.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.687+0000 7f7f9e000700 1 -- 172.21.15.57:0/3007353205 wait complete. 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 Processor -- start 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 -- start start 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f98102e30 0x7f7f980fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980fd510 con 0x7f7f980ff9e0 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9e000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980fd650 con 0x7f7f98102e30 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50998/0 (socket says 172.21.15.57:50998) 2024-02-28T23:23:00.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.688+0000 7f7f9cc00700 1 -- 172.21.15.57:0/277140409 learned_addr learned my addr 172.21.15.57:0/277140409 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:00.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.689+0000 7f7f97e00700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f98102e30 0x7f7f980fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.689+0000 7f7f9cc00700 1 -- 172.21.15.57:0/277140409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f98102e30 msgr2=0x7f7f980fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.689+0000 7f7f9cc00700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f98102e30 0x7f7f980fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.689+0000 7f7f9cc00700 1 -- 172.21.15.57:0/277140409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0095f0 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.689+0000 7f7f9cc00700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.690+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c01d070 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.690+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f8c00f460 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.690+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f980fd8d0 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.690+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c017630 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.690+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f981a2120 con 0x7f7f980ff9e0 2024-02-28T23:23:00.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.691+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f98108560 con 0x7f7f980ff9e0 2024-02-28T23:23:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.692+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7f8c00f5d0 con 0x7f7f980ff9e0 2024-02-28T23:23:00.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.693+0000 7f7f95600700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 0x7f7f8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:00.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.693+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7f8c08ea80 con 0x7f7f980ff9e0 2024-02-28T23:23:00.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.693+0000 7f7f97e00700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 0x7f7f8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:00.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.694+0000 7f7f97e00700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 0x7f7f8006fd50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005950 tx=0x7f7f840058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.697+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f8c058190 con 0x7f7f980ff9e0 2024-02-28T23:23:00.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.930+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f98061190 con 0x7f7f8006d8a0 2024-02-28T23:23:00.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.963+0000 7f7f95600700 1 -- 172.21.15.57:0/277140409 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7f98061190 con 0x7f7f8006d8a0 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (12m) 4m ago 26m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (22m) 4m ago 26m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (25m) 3m ago 25m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (8m) 4m ago 26m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (8m) 3m ago 25m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (12m) 4m ago 26m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (4m) 4m ago 19m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (4m) 4m ago 19m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (19m) 3m ago 19m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:23:00.962 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (3m) 3m ago 19m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (15m) 4m ago 28m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (14m) 3m ago 25m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (10m) 4m ago 28m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (9m) 3m ago 24m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (13m) 4m ago 26m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (13m) 3m ago 25m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (8m) 4m ago 24m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (7m) 4m ago 23m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (7m) 4m ago 22m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (6m) 3m ago 22m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (5m) 3m ago 21m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (5m) 3m ago 21m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:23:00.963 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (13m) 4m ago 25m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 msgr2=0x7f7f8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 0x7f7f8006fd50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005950 tx=0x7f7f840058e0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 msgr2=0x7f7f980fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00be20 comp rx=0 tx=0).stop 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 shutdown_connections 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f8006d8a0 0x7f7f8006fd50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f980ff9e0 0x7f7f980fca90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 --2- 172.21.15.57:0/277140409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f98102e30 0x7f7f980fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:00.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.966+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 >> 172.21.15.57:0/277140409 conn(0x7f7f98074fa0 msgr2=0x7f7f98068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:00.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.967+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 shutdown_connections 2024-02-28T23:23:00.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:00.967+0000 7f7f9e000700 1 -- 172.21.15.57:0/277140409 wait complete. 2024-02-28T23:23:01.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 -- 172.21.15.57:0/1145029334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 msgr2=0x7fdcd0075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1145029334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd0075650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009910 tx=0x7fdcc0009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 -- 172.21.15.57:0/1145029334 shutdown_connections 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1145029334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 0x7fdcd0110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1145029334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd0075650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.127+0000 7fdcd7200700 1 -- 172.21.15.57:0/1145029334 >> 172.21.15.57:0/1145029334 conn(0x7fdcd00fdf40 msgr2=0x7fdcd0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.128+0000 7fdcd7200700 1 -- 172.21.15.57:0/1145029334 shutdown_connections 2024-02-28T23:23:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.128+0000 7fdcd7200700 1 -- 172.21.15.57:0/1145029334 wait complete. 2024-02-28T23:23:01.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 Processor -- start 2024-02-28T23:23:01.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 -- start start 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 0x7fdcd019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019d130 con 0x7fdcd0075280 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.129+0000 7fdcd7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01a0e50 con 0x7fdcd0075b90 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51018/0 (socket says 172.21.15.57:51018) 2024-02-28T23:23:01.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 -- 172.21.15.57:0/1792016225 learned_addr learned my addr 172.21.15.57:0/1792016225 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccf400700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 0x7fdcd019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 -- 172.21.15.57:0/1792016225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 msgr2=0x7fdcd019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 0x7fdcd019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 -- 172.21.15.57:0/1792016225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc00095f0 con 0x7fdcd0075280 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.130+0000 7fdccfe00700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006010 tx=0x7fdcc00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:01.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.131+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc001d070 con 0x7fdcd0075280 2024-02-28T23:23:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.131+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcc0005dc0 con 0x7fdcd0075280 2024-02-28T23:23:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.131+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd01a10d0 con 0x7fdcd0075280 2024-02-28T23:23:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.131+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc0021410 con 0x7fdcd0075280 2024-02-28T23:23:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.131+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd01a1620 con 0x7fdcd0075280 2024-02-28T23:23:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.132+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd010e310 con 0x7fdcd0075280 2024-02-28T23:23:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.133+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdcc000f6f0 con 0x7fdcd0075280 2024-02-28T23:23:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.134+0000 7fdcccc00700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.134+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdcc008e740 con 0x7fdcd0075280 2024-02-28T23:23:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.135+0000 7fdccf400700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.135+0000 7fdccf400700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005950 tx=0x7fdcbc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:01.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.139+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdcc0057e50 con 0x7fdcd0075280 2024-02-28T23:23:01.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.450+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdcd019d870 con 0x7fdcd0075280 2024-02-28T23:23:01.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.451+0000 7fdcccc00700 1 -- 172.21.15.57:0/1792016225 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fdcc005b510 con 0x7fdcd0075280 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:23:01.450 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:23:01.451 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 msgr2=0x7fdcb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005950 tx=0x7fdcbc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 msgr2=0x7fdcd019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006010 tx=0x7fdcc00056a0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 shutdown_connections 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0075280 0x7fdcd019c510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 --2- 172.21.15.57:0/1792016225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0075b90 0x7fdcd019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.454+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 >> 172.21.15.57:0/1792016225 conn(0x7fdcd00fdf40 msgr2=0x7fdcd00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.455+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 shutdown_connections 2024-02-28T23:23:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.455+0000 7fdcd7200700 1 -- 172.21.15.57:0/1792016225 wait complete. 2024-02-28T23:23:01.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 -- 172.21.15.57:0/535813051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec0730f0 msgr2=0x7f1bec0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 --2- 172.21.15.57:0/535813051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec0730f0 0x7f1bec0734c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009910 tx=0x7f1be0009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 -- 172.21.15.57:0/535813051 shutdown_connections 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 --2- 172.21.15.57:0/535813051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec073a00 0x7f1bec110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 --2- 172.21.15.57:0/535813051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec0730f0 0x7f1bec0734c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 -- 172.21.15.57:0/535813051 >> 172.21.15.57:0/535813051 conn(0x7f1bec0fbda0 msgr2=0x7f1bec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 -- 172.21.15.57:0/535813051 shutdown_connections 2024-02-28T23:23:01.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.615+0000 7f1bf4600700 1 -- 172.21.15.57:0/535813051 wait complete. 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.616+0000 7f1bf4600700 1 Processor -- start 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.616+0000 7f1bf4600700 1 -- start start 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf4600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec0730f0 0x7f1bec19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf4600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf4600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec19d000 con 0x7f1bec073a00 2024-02-28T23:23:01.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf4600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a0d90 con 0x7f1bec0730f0 2024-02-28T23:23:01.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51034/0 (socket says 172.21.15.57:51034) 2024-02-28T23:23:01.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf1200700 1 -- 172.21.15.57:0/3276948655 learned_addr learned my addr 172.21.15.57:0/3276948655 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:01.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.617+0000 7f1bf1c00700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec0730f0 0x7f1bec19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bf1200700 1 -- 172.21.15.57:0/3276948655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec0730f0 msgr2=0x7f1bec19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bf1200700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec0730f0 0x7f1bec19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bf1200700 1 -- 172.21.15.57:0/3276948655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00095f0 con 0x7f1bec073a00 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bf1200700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d8d0 tx=0x7f1bd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd8009940 con 0x7f1bec073a00 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bd8010460 con 0x7f1bec073a00 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd800f5f0 con 0x7f1bec073a00 2024-02-28T23:23:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.618+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1a1070 con 0x7f1bec073a00 2024-02-28T23:23:01.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.619+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1a15c0 con 0x7f1bec073a00 2024-02-28T23:23:01.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.620+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec10e4d0 con 0x7f1bec073a00 2024-02-28T23:23:01.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.621+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1bd80105d0 con 0x7f1bec073a00 2024-02-28T23:23:01.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.622+0000 7f1bea000700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 0x7f1bd406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.622+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1bd808d510 con 0x7f1bec073a00 2024-02-28T23:23:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.623+0000 7f1bf1c00700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 0x7f1bd406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.624+0000 7f1bf1c00700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 0x7f1bd406fd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006010 tx=0x7f1be001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:01.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.627+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1bd8056ca0 con 0x7f1bec073a00 2024-02-28T23:23:01.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:01 smithi057 ceph-mon[89491]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:01.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:01 smithi057 ceph-mon[89491]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:01.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:01 smithi057 ceph-mon[89491]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:01.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:01 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1792016225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:23:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:01 smithi084 ceph-mon[70440]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:01 smithi084 ceph-mon[70440]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:01 smithi084 ceph-mon[70440]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:01.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:01 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1792016225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:23:01.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.908+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1bec04f2a0 con 0x7f1bec073a00 2024-02-28T23:23:01.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.909+0000 7f1bea000700 1 -- 172.21.15.57:0/3276948655 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f1bd8016740 con 0x7f1bec073a00 2024-02-28T23:23:01.908 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:23:01.908 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:23:01.908 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:23:01.909 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:23:01.910 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:23:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 msgr2=0x7f1bd406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 0x7f1bd406fd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006010 tx=0x7f1be001a040 comp rx=0 tx=0).stop 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 msgr2=0x7f1bec19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d8d0 tx=0x7f1bd800dc90 comp rx=0 tx=0).stop 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 shutdown_connections 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1bd406d8a0 0x7f1bd406fd50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec0730f0 0x7f1bec19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 --2- 172.21.15.57:0/3276948655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bec073a00 0x7f1bec19c920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:01.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 >> 172.21.15.57:0/3276948655 conn(0x7f1bec0fbda0 msgr2=0x7f1bec102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:01.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 shutdown_connections 2024-02-28T23:23:01.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:01.912+0000 7f1bf4600700 1 -- 172.21.15.57:0/3276948655 wait complete. 2024-02-28T23:23:01.913 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:23:02.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 -- 172.21.15.57:0/2679980802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4200739a0 msgr2=0x7ff42010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 --2- 172.21.15.57:0/2679980802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4200739a0 0x7ff42010c5e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009940 tx=0x7ff40c009c50 comp rx=0 tx=0).stop 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 -- 172.21.15.57:0/2679980802 shutdown_connections 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 --2- 172.21.15.57:0/2679980802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4200739a0 0x7ff42010c5e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 --2- 172.21.15.57:0/2679980802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff420073090 0x7ff420073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 -- 172.21.15.57:0/2679980802 >> 172.21.15.57:0/2679980802 conn(0x7ff4200fbda0 msgr2=0x7ff4200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 -- 172.21.15.57:0/2679980802 shutdown_connections 2024-02-28T23:23:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.062+0000 7ff427a00700 1 -- 172.21.15.57:0/2679980802 wait complete. 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.063+0000 7ff427a00700 1 Processor -- start 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.063+0000 7ff427a00700 1 -- start start 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff427a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff427a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4200739a0 0x7ff420198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff427a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420198e40 con 0x7ff420073090 2024-02-28T23:23:02.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff427a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42019cbd0 con 0x7ff4200739a0 2024-02-28T23:23:02.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff425000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:02.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff425000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51048/0 (socket says 172.21.15.57:51048) 2024-02-28T23:23:02.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff425000700 1 -- 172.21.15.57:0/3995399569 learned_addr learned my addr 172.21.15.57:0/3995399569 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff425000700 1 -- 172.21.15.57:0/3995399569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4200739a0 msgr2=0x7ff420198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.064+0000 7ff425000700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4200739a0 0x7ff420198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff425000700 1 -- 172.21.15.57:0/3995399569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0095f0 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff425000700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c8a0 tx=0x7ff41400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41400cea0 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff414004830 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4140055b0 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42019ceb0 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.065+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42019d400 con 0x7ff420073090 2024-02-28T23:23:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.066+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff420109ce0 con 0x7ff420073090 2024-02-28T23:23:02.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.068+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff4140049a0 con 0x7ff420073090 2024-02-28T23:23:02.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.068+0000 7ff41d600700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 0x7ff40806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.069+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff41408d190 con 0x7ff420073090 2024-02-28T23:23:02.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.069+0000 7ff41fe00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 0x7ff40806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:02.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.069+0000 7ff41fe00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 0x7ff40806fd50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff40c006010 tx=0x7ff40c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:02.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.073+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff414056920 con 0x7ff420073090 2024-02-28T23:23:02.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.314+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff42019d6d0 con 0x7ff40806d8a0 2024-02-28T23:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.317+0000 7ff41d600700 1 -- 172.21.15.57:0/3995399569 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff42019d6d0 con 0x7ff40806d8a0 2024-02-28T23:23:02.316 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:23:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:23:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:23:02.317 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.321+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 msgr2=0x7ff40806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.321+0000 7ff427a00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 0x7ff40806fd50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff40c006010 tx=0x7ff40c000bc0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.321+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 msgr2=0x7ff420198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.321+0000 7ff427a00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c8a0 tx=0x7ff41400cc60 comp rx=0 tx=0).stop 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.321+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 shutdown_connections 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff40806d8a0 0x7ff40806fd50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff420073090 0x7ff420198220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 --2- 172.21.15.57:0/3995399569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4200739a0 0x7ff420198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 >> 172.21.15.57:0/3995399569 conn(0x7ff4200fbda0 msgr2=0x7ff420106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:02.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 shutdown_connections 2024-02-28T23:23:02.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.322+0000 7ff427a00700 1 -- 172.21.15.57:0/3995399569 wait complete. 2024-02-28T23:23:02.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 -- 172.21.15.57:0/2064068389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c068540 msgr2=0x7f801c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 --2- 172.21.15.57:0/2064068389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c068540 0x7f801c0689b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8008009910 tx=0x7f8008009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 -- 172.21.15.57:0/2064068389 shutdown_connections 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 --2- 172.21.15.57:0/2064068389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c068540 0x7f801c0689b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 --2- 172.21.15.57:0/2064068389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c101000 0x7f801c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 -- 172.21.15.57:0/2064068389 >> 172.21.15.57:0/2064068389 conn(0x7f801c075130 msgr2=0x7f801c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 -- 172.21.15.57:0/2064068389 shutdown_connections 2024-02-28T23:23:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.479+0000 7f8021000700 1 -- 172.21.15.57:0/2064068389 wait complete. 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.480+0000 7f8021000700 1 Processor -- start 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.480+0000 7f8021000700 1 -- start start 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8021000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c068540 0x7f801c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8021000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8021000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c198c90 con 0x7f801c101000 2024-02-28T23:23:02.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8021000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c19ca20 con 0x7f801c068540 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8019400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8019400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51060/0 (socket says 172.21.15.57:51060) 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8019400700 1 -- 172.21.15.57:0/798396152 learned_addr learned my addr 172.21.15.57:0/798396152 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.481+0000 7f8019e00700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c068540 0x7f801c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.482+0000 7f8019400700 1 -- 172.21.15.57:0/798396152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c068540 msgr2=0x7f801c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.482+0000 7f8019400700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c068540 0x7f801c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.482+0000 7f8019400700 1 -- 172.21.15.57:0/798396152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080095f0 con 0x7f801c101000 2024-02-28T23:23:02.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.482+0000 7f8019400700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f800800b5c0 tx=0x7f800800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:02.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.484+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800801d070 con 0x7f801c101000 2024-02-28T23:23:02.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.484+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800800f460 con 0x7f801c101000 2024-02-28T23:23:02.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.484+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8008017610 con 0x7f801c101000 2024-02-28T23:23:02.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.484+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c19cca0 con 0x7f801c101000 2024-02-28T23:23:02.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.484+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c19d190 con 0x7f801c101000 2024-02-28T23:23:02.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.485+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c04f2a0 con 0x7f801c101000 2024-02-28T23:23:02.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.486+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f800800f5d0 con 0x7f801c101000 2024-02-28T23:23:02.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.487+0000 7f8012a00700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 0x7f800006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:02.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.487+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f800808ec00 con 0x7f801c101000 2024-02-28T23:23:02.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.487+0000 7f8019e00700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 0x7f800006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:02.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.488+0000 7f8019e00700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 0x7f800006fd50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:02.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.491+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8008058310 con 0x7f801c101000 2024-02-28T23:23:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:02 smithi057 ceph-mon[89491]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:02.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:02 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3276948655' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.807+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f801c19d450 con 0x7f801c101000 2024-02-28T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.808+0000 7f8012a00700 1 -- 172.21.15.57:0/798396152 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f800805b9d0 con 0x7f801c101000 2024-02-28T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.810+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 msgr2=0x7f800006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.810+0000 7f8021000700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 0x7f800006fd50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c00a560 comp rx=0 tx=0).stop 2024-02-28T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.810+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 msgr2=0x7f801c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.810+0000 7f8021000700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f800800b5c0 tx=0x7f800800be00 comp rx=0 tx=0).stop 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 shutdown_connections 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f800006d8a0 0x7f800006fd50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f801c068540 0x7f801c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 --2- 172.21.15.57:0/798396152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c101000 0x7f801c1985b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 >> 172.21.15.57:0/798396152 conn(0x7f801c075130 msgr2=0x7f801c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 shutdown_connections 2024-02-28T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:02.811+0000 7f8021000700 1 -- 172.21.15.57:0/798396152 wait complete. 2024-02-28T23:23:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:02 smithi084 ceph-mon[70440]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:02 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3276948655' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:23:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:03 smithi057 ceph-mon[89491]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:03 smithi057 ceph-mon[89491]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:03 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/798396152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:23:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:03 smithi084 ceph-mon[70440]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:03 smithi084 ceph-mon[70440]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:03 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/798396152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:23:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:05 smithi057 ceph-mon[89491]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:05.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:05 smithi084 ceph-mon[70440]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:07 smithi057 ceph-mon[89491]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:07.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:07 smithi084 ceph-mon[70440]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:09 smithi057 ceph-mon[89491]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:09.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:09 smithi084 ceph-mon[70440]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:11 smithi057 ceph-mon[89491]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:11.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:11 smithi084 ceph-mon[70440]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:13.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:13 smithi057 ceph-mon[89491]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:13.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:13 smithi084 ceph-mon[70440]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:15 smithi057 ceph-mon[89491]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:15.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:15 smithi084 ceph-mon[70440]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:17.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:17 smithi057 ceph-mon[89491]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:17.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:17 smithi084 ceph-mon[70440]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:18.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:19.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:19 smithi084 ceph-mon[70440]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:19 smithi057 ceph-mon[89491]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:21 smithi057 ceph-mon[89491]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:21 smithi084 ceph-mon[70440]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:23.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:23 smithi084 ceph-mon[70440]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:23 smithi057 ceph-mon[89491]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:25 smithi057 ceph-mon[89491]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:25 smithi084 ceph-mon[70440]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:27 smithi057 ceph-mon[89491]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:27 smithi084 ceph-mon[70440]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:30.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:29 smithi057 ceph-mon[89491]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:29 smithi084 ceph-mon[70440]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:31 smithi057 ceph-mon[89491]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:32.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:31 smithi084 ceph-mon[70440]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 -- 172.21.15.57:0/4043278863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068540 msgr2=0x7f0ac8068910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 --2- 172.21.15.57:0/4043278863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068540 0x7f0ac8068910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009940 tx=0x7f0ab8009c50 comp rx=0 tx=0).stop 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 -- 172.21.15.57:0/4043278863 shutdown_connections 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 --2- 172.21.15.57:0/4043278863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068ee0 0x7f0ac8110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 --2- 172.21.15.57:0/4043278863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068540 0x7f0ac8068910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 -- 172.21.15.57:0/4043278863 >> 172.21.15.57:0/4043278863 conn(0x7f0ac8075130 msgr2=0x7f0ac8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 -- 172.21.15.57:0/4043278863 shutdown_connections 2024-02-28T23:23:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.984+0000 7f0accc00700 1 -- 172.21.15.57:0/4043278863 wait complete. 2024-02-28T23:23:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.985+0000 7f0accc00700 1 Processor -- start 2024-02-28T23:23:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0accc00700 1 -- start start 2024-02-28T23:23:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0accc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068540 0x7f0ac819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0accc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0accc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819d070 con 0x7f0ac8068ee0 2024-02-28T23:23:32.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0accc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81a0e00 con 0x7f0ac8068540 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0abfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0abfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52392/0 (socket says 172.21.15.57:52392) 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.986+0000 7f0abfe00700 1 -- 172.21.15.57:0/1481385914 learned_addr learned my addr 172.21.15.57:0/1481385914 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0ac5e00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068540 0x7f0ac819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abfe00700 1 -- 172.21.15.57:0/1481385914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068540 msgr2=0x7f0ac819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abfe00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068540 0x7f0ac819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:32.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abfe00700 1 -- 172.21.15.57:0/1481385914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80095f0 con 0x7f0ac8068ee0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abfe00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d8d0 tx=0x7f0ab000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab0009940 con 0x7f0ac8068ee0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.987+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ab0010460 con 0x7f0ac8068ee0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.988+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81a10e0 con 0x7f0ac8068ee0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.988+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab000f5f0 con 0x7f0ac8068ee0 2024-02-28T23:23:32.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.988+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac81a1630 con 0x7f0ac8068ee0 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.989+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8101410 con 0x7f0ac8068ee0 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.990+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0ab00105d0 con 0x7f0ac8068ee0 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.991+0000 7f0abea00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 0x7f0aac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.992+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ab008d4c0 con 0x7f0ac8068ee0 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.992+0000 7f0ac5e00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 0x7f0aac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:32.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.992+0000 7f0ac5e00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 0x7f0aac06fd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b810 tx=0x7f0ab8005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:32.997+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ab0056c50 con 0x7f0ac8068ee0 2024-02-28T23:23:33.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.249+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac81a18a0 con 0x7f0aac06d8a0 2024-02-28T23:23:33.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.251+0000 7f0abea00700 1 -- 172.21.15.57:0/1481385914 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0ac81a18a0 con 0x7f0aac06d8a0 2024-02-28T23:23:33.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 msgr2=0x7f0aac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 0x7f0aac06fd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b810 tx=0x7f0ab8005fb0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 msgr2=0x7f0ac819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d8d0 tx=0x7f0ab000dc90 comp rx=0 tx=0).stop 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 shutdown_connections 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0aac06d8a0 0x7f0aac06fd50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac8068540 0x7f0ac819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 --2- 172.21.15.57:0/1481385914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ac8068ee0 0x7f0ac819c990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.254+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 >> 172.21.15.57:0/1481385914 conn(0x7f0ac8075130 msgr2=0x7f0ac810eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.255+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 shutdown_connections 2024-02-28T23:23:33.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.255+0000 7f0accc00700 1 -- 172.21.15.57:0/1481385914 wait complete. 2024-02-28T23:23:33.270 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 -- 172.21.15.57:0/4226824755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 msgr2=0x7f6fb41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 --2- 172.21.15.57:0/4226824755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb41028a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b0a0 tx=0x7f6fa800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 -- 172.21.15.57:0/4226824755 shutdown_connections 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 --2- 172.21.15.57:0/4226824755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 --2- 172.21.15.57:0/4226824755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb41028a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.413+0000 7f6fba200700 1 -- 172.21.15.57:0/4226824755 >> 172.21.15.57:0/4226824755 conn(0x7f6fb40fdd90 msgr2=0x7f6fb41001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.414+0000 7f6fba200700 1 -- 172.21.15.57:0/4226824755 shutdown_connections 2024-02-28T23:23:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.414+0000 7f6fba200700 1 -- 172.21.15.57:0/4226824755 wait complete. 2024-02-28T23:23:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 Processor -- start 2024-02-28T23:23:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 -- start start 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4198a80 con 0x7f6fb41024d0 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.415+0000 7f6fba200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb419c810 con 0x7f6fb4102e70 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52414/0 (socket says 172.21.15.57:52414) 2024-02-28T23:23:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 -- 172.21.15.57:0/3429319684 learned_addr learned my addr 172.21.15.57:0/3429319684 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 -- 172.21.15.57:0/3429319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 msgr2=0x7f6fb41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb3e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb8e00700 1 -- 172.21.15.57:0/3429319684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009d00 con 0x7f6fb41024d0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.416+0000 7f6fb3e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fb8e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b070 tx=0x7f6fa800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fa800e040 con 0x7f6fb41024d0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fa8003de0 con 0x7f6fb41024d0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb419ca90 con 0x7f6fb41024d0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fa801d8e0 con 0x7f6fb41024d0 2024-02-28T23:23:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.417+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb419cf80 con 0x7f6fb41024d0 2024-02-28T23:23:33.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.418+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb410a7e0 con 0x7f6fb41024d0 2024-02-28T23:23:33.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.419+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6fa8004040 con 0x7f6fb41024d0 2024-02-28T23:23:33.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.420+0000 7f6fb1600700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 0x7f6f9c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.420+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6fa808ece0 con 0x7f6fb41024d0 2024-02-28T23:23:33.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.420+0000 7f6fb3e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 0x7f6f9c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.421+0000 7f6fb3e00700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 0x7f6f9c06fd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005fd0 tx=0x7f6fa0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.425+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fa8058470 con 0x7f6fb41024d0 2024-02-28T23:23:33.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.678+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fb419d320 con 0x7f6f9c06d8a0 2024-02-28T23:23:33.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.681+0000 7f6fb1600700 1 -- 172.21.15.57:0/3429319684 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6fb419d320 con 0x7f6f9c06d8a0 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 msgr2=0x7f6f9c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 0x7f6f9c06fd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005fd0 tx=0x7f6fa0005ee0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 msgr2=0x7f6fb4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b070 tx=0x7f6fa800cdd0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 shutdown_connections 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6f9c06d8a0 0x7f6f9c06fd50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fb41024d0 0x7f6fb4197e60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 --2- 172.21.15.57:0/3429319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fb4102e70 0x7f6fb41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.684+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 >> 172.21.15.57:0/3429319684 conn(0x7f6fb40fdd90 msgr2=0x7f6fb40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.685+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 shutdown_connections 2024-02-28T23:23:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.685+0000 7f6fba200700 1 -- 172.21.15.57:0/3429319684 wait complete. 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 -- 172.21.15.57:0/3633503762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb8103e40 msgr2=0x7fcbb81042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 --2- 172.21.15.57:0/3633503762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb8103e40 0x7fcbb81042b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009910 tx=0x7fcba0009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 -- 172.21.15.57:0/3633503762 shutdown_connections 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 --2- 172.21.15.57:0/3633503762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb8103e40 0x7fcbb81042b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 --2- 172.21.15.57:0/3633503762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb81025e0 0x7fcbb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 -- 172.21.15.57:0/3633503762 >> 172.21.15.57:0/3633503762 conn(0x7fcbb80fde00 msgr2=0x7fcbb8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:33.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 -- 172.21.15.57:0/3633503762 shutdown_connections 2024-02-28T23:23:33.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.845+0000 7fcbb7200700 1 -- 172.21.15.57:0/3633503762 wait complete. 2024-02-28T23:23:33.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.846+0000 7fcbb7200700 1 Processor -- start 2024-02-28T23:23:33.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.846+0000 7fcbb7200700 1 -- start start 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.846+0000 7fcbb7200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 0x7fcbb8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb7200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8198b10 con 0x7fcbb81025e0 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819c8a0 con 0x7fcbb8103e40 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52428/0 (socket says 172.21.15.57:52428) 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 -- 172.21.15.57:0/893257530 learned_addr learned my addr 172.21.15.57:0/893257530 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbad400700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 0x7fcbb8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 -- 172.21.15.57:0/893257530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 msgr2=0x7fcbb8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:33.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 0x7fcbb8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:33.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbb5e00700 1 -- 172.21.15.57:0/893257530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00095f0 con 0x7fcbb81025e0 2024-02-28T23:23:33.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.847+0000 7fcbad400700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 0x7fcbb8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:23:33.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbb5e00700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcba80060b0 tx=0x7fcba800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba800dda0 con 0x7fcbb81025e0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcba800f040 con 0x7fcbb81025e0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba8014790 con 0x7fcbb81025e0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb819cb80 con 0x7fcbb81025e0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.848+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb819d0d0 con 0x7fcbb81025e0 2024-02-28T23:23:33.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.849+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb8062380 con 0x7fcbb81025e0 2024-02-28T23:23:33.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.851+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcba80148f0 con 0x7fcbb81025e0 2024-02-28T23:23:33.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.851+0000 7fcbaf400700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:33.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.852+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcba808cf90 con 0x7fcbb81025e0 2024-02-28T23:23:33.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.852+0000 7fcbad400700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:33.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.853+0000 7fcbad400700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:33.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:33.857+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcba80567d0 con 0x7fcbb81025e0 2024-02-28T23:23:33.882 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:33 smithi084 ceph-mon[70440]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:33.882 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:33 smithi057 ceph-mon[89491]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:34.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.102+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbb819d440 con 0x7fcb9c06d8a0 2024-02-28T23:23:34.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.132+0000 7fcbaf400700 1 -- 172.21.15.57:0/893257530 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcbb819d440 con 0x7fcb9c06d8a0 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (13m) 4m ago 27m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (23m) 4m ago 27m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (25m) 4m ago 25m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (9m) 4m ago 27m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (9m) 4m ago 25m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (12m) 4m ago 26m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (5m) 4m ago 19m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (4m) 4m ago 19m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (19m) 4m ago 19m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (4m) 4m ago 19m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:23:34.132 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (15m) 4m ago 28m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (14m) 4m ago 25m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (10m) 4m ago 29m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (10m) 4m ago 25m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (14m) 4m ago 27m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (14m) 4m ago 25m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (8m) 4m ago 24m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (8m) 4m ago 23m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (7m) 4m ago 23m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (7m) 4m ago 22m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (6m) 4m ago 22m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (5m) 4m ago 21m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:23:34.133 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (13m) 4m ago 26m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:23:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.135+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 msgr2=0x7fcb9c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.135+0000 7fcbb7200700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba000b540 comp rx=0 tx=0).stop 2024-02-28T23:23:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.135+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 msgr2=0x7fcbb8197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.135+0000 7fcbb7200700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcba80060b0 tx=0x7fcba800d3b0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 shutdown_connections 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbb81025e0 0x7fcbb8197ef0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 --2- 172.21.15.57:0/893257530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb8103e40 0x7fcbb8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 >> 172.21.15.57:0/893257530 conn(0x7fcbb80fde00 msgr2=0x7fcbb80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:34.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 shutdown_connections 2024-02-28T23:23:34.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.136+0000 7fcbb7200700 1 -- 172.21.15.57:0/893257530 wait complete. 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 -- 172.21.15.57:0/2862034022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae010c660 msgr2=0x7f8ae010ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 --2- 172.21.15.57:0/2862034022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae010c660 0x7f8ae010ca30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009910 tx=0x7f8ad4009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 -- 172.21.15.57:0/2862034022 shutdown_connections 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 --2- 172.21.15.57:0/2862034022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae0106b80 0x7f8ae0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 --2- 172.21.15.57:0/2862034022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae010c660 0x7f8ae010ca30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 -- 172.21.15.57:0/2862034022 >> 172.21.15.57:0/2862034022 conn(0x7f8ae0075e90 msgr2=0x7f8ae00782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 -- 172.21.15.57:0/2862034022 shutdown_connections 2024-02-28T23:23:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.302+0000 7f8ae6800700 1 -- 172.21.15.57:0/2862034022 wait complete. 2024-02-28T23:23:34.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.303+0000 7f8ae6800700 1 Processor -- start 2024-02-28T23:23:34.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae6800700 1 -- start start 2024-02-28T23:23:34.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae6800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae6800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae010c660 0x7f8ae019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae6800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae019ce90 con 0x7f8ae0106b80 2024-02-28T23:23:34.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae6800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01a0c20 con 0x7f8ae010c660 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52448/0 (socket says 172.21.15.57:52448) 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.304+0000 7f8ae5400700 1 -- 172.21.15.57:0/4053102014 learned_addr learned my addr 172.21.15.57:0/4053102014 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.305+0000 7f8ae4a00700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae010c660 0x7f8ae019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.305+0000 7f8ae5400700 1 -- 172.21.15.57:0/4053102014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae010c660 msgr2=0x7f8ae019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.305+0000 7f8ae5400700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae010c660 0x7f8ae019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.305+0000 7f8ae5400700 1 -- 172.21.15.57:0/4053102014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40095f0 con 0x7f8ae0106b80 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.305+0000 7f8ae5400700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400b5c0 tx=0x7f8ad400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.306+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad401d070 con 0x7f8ae0106b80 2024-02-28T23:23:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.306+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad4005e00 con 0x7f8ae0106b80 2024-02-28T23:23:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.306+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad4017400 con 0x7f8ae0106b80 2024-02-28T23:23:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.306+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae01a0ea0 con 0x7f8ae0106b80 2024-02-28T23:23:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.306+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae01a1390 con 0x7f8ae0106b80 2024-02-28T23:23:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.307+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae010eb60 con 0x7f8ae0106b80 2024-02-28T23:23:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.308+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ad400f460 con 0x7f8ae0106b80 2024-02-28T23:23:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.309+0000 7f8ade000700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 0x7f8ac806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.309+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ad408e970 con 0x7f8ae0106b80 2024-02-28T23:23:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.309+0000 7f8ae4a00700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 0x7f8ac806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.310+0000 7f8ae4a00700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 0x7f8ac806fd50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8acc005950 tx=0x7f8acc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:34.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.314+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad40581a0 con 0x7f8ae0106b80 2024-02-28T23:23:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.635+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ae019d6d0 con 0x7f8ae0106b80 2024-02-28T23:23:34.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.638+0000 7f8ade000700 1 -- 172.21.15.57:0/4053102014 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f8ad405b860 con 0x7f8ae0106b80 2024-02-28T23:23:34.637 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:23:34.638 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:23:34.639 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:23:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 msgr2=0x7f8ac806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 0x7f8ac806fd50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8acc005950 tx=0x7f8acc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 msgr2=0x7f8ae019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400b5c0 tx=0x7f8ad400be20 comp rx=0 tx=0).stop 2024-02-28T23:23:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 shutdown_connections 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac806d8a0 0x7f8ac806fd50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0106b80 0x7f8ae019c270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 --2- 172.21.15.57:0/4053102014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ae010c660 0x7f8ae019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.641+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 >> 172.21.15.57:0/4053102014 conn(0x7f8ae0075e90 msgr2=0x7f8ae00778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.642+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 shutdown_connections 2024-02-28T23:23:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.642+0000 7f8ae6800700 1 -- 172.21.15.57:0/4053102014 wait complete. 2024-02-28T23:23:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 -- 172.21.15.57:0/4246768185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 msgr2=0x7fa44c10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 --2- 172.21.15.57:0/4246768185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c10c3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa438009910 tx=0x7fa438009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 -- 172.21.15.57:0/4246768185 shutdown_connections 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 --2- 172.21.15.57:0/4246768185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c10c3f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 --2- 172.21.15.57:0/4246768185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 0x7fa44c0ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 -- 172.21.15.57:0/4246768185 >> 172.21.15.57:0/4246768185 conn(0x7fa44c0faa10 msgr2=0x7fa44c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 -- 172.21.15.57:0/4246768185 shutdown_connections 2024-02-28T23:23:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.805+0000 7fa44b200700 1 -- 172.21.15.57:0/4246768185 wait complete. 2024-02-28T23:23:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.807+0000 7fa44b200700 1 Processor -- start 2024-02-28T23:23:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.807+0000 7fa44b200700 1 -- start start 2024-02-28T23:23:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa44b200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 0x7fa44c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa44b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa44b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c1989c0 con 0x7fa44c0ff690 2024-02-28T23:23:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa44b200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c19c750 con 0x7fa44c0fed80 2024-02-28T23:23:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa449400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa449400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52472/0 (socket says 172.21.15.57:52472) 2024-02-28T23:23:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa449400700 1 -- 172.21.15.57:0/1486316782 learned_addr learned my addr 172.21.15.57:0/1486316782 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.808+0000 7fa449e00700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 0x7fa44c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa449400700 1 -- 172.21.15.57:0/1486316782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 msgr2=0x7fa44c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa449400700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 0x7fa44c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa449400700 1 -- 172.21.15.57:0/1486316782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380095f0 con 0x7fa44c0ff690 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa449400700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa4380092e0 tx=0x7fa43800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa43801d070 con 0x7fa44c0ff690 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa438005e00 con 0x7fa44c0ff690 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438017400 con 0x7fa44c0ff690 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c19ca30 con 0x7fa44c0ff690 2024-02-28T23:23:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.809+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c19cf80 con 0x7fa44c0ff690 2024-02-28T23:23:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.811+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c109b60 con 0x7fa44c0ff690 2024-02-28T23:23:34.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.812+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa43800f460 con 0x7fa44c0ff690 2024-02-28T23:23:34.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.813+0000 7fa442a00700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 0x7fa43006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:34.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.813+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa43808ea90 con 0x7fa44c0ff690 2024-02-28T23:23:34.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.813+0000 7fa449e00700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 0x7fa43006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:34.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.814+0000 7fa449e00700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 0x7fa43006fd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa43c005950 tx=0x7fa43c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:34.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:34.818+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4380581a0 con 0x7fa44c0ff690 2024-02-28T23:23:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:34 smithi057 ceph-mon[89491]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:34 smithi057 ceph-mon[89491]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:34 smithi084 ceph-mon[70440]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:34 smithi084 ceph-mon[70440]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:35.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.097+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa44c062380 con 0x7fa44c0ff690 2024-02-28T23:23:35.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.098+0000 7fa442a00700 1 -- 172.21.15.57:0/1486316782 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fa43805b860 con 0x7fa44c0ff690 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:23:35.098 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:23:35.099 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:35.100 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:23:35.101 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 msgr2=0x7fa43006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 0x7fa43006fd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa43c005950 tx=0x7fa43c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 msgr2=0x7fa44c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa4380092e0 tx=0x7fa43800be20 comp rx=0 tx=0).stop 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 shutdown_connections 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa43006d8a0 0x7fa43006fd50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa44c0fed80 0x7fa44c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 --2- 172.21.15.57:0/1486316782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa44c0ff690 0x7fa44c1982e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.101+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 >> 172.21.15.57:0/1486316782 conn(0x7fa44c0faa10 msgr2=0x7fa44c0fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:35.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.102+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 shutdown_connections 2024-02-28T23:23:35.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.102+0000 7fa44b200700 1 -- 172.21.15.57:0/1486316782 wait complete. 2024-02-28T23:23:35.103 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:23:35.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 -- 172.21.15.57:0/2109034909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 msgr2=0x7f4a701020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2109034909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a701020b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009910 tx=0x7f4a5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 -- 172.21.15.57:0/2109034909 shutdown_connections 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2109034909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a701020b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2109034909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 0x7f4a701007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 -- 172.21.15.57:0/2109034909 >> 172.21.15.57:0/2109034909 conn(0x7f4a700fbbc0 msgr2=0x7f4a700fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 -- 172.21.15.57:0/2109034909 shutdown_connections 2024-02-28T23:23:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.269+0000 7f4a76a00700 1 -- 172.21.15.57:0/2109034909 wait complete. 2024-02-28T23:23:35.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.270+0000 7f4a76a00700 1 Processor -- start 2024-02-28T23:23:35.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a76a00700 1 -- start start 2024-02-28T23:23:35.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a76a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 0x7f4a70197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a76a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a76a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a70198b00 con 0x7f4a70101c40 2024-02-28T23:23:35.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a76a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7019c890 con 0x7f4a701003e0 2024-02-28T23:23:35.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.271+0000 7f4a74c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59060/0 (socket says 172.21.15.57:59060) 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 -- 172.21.15.57:0/2914227813 learned_addr learned my addr 172.21.15.57:0/2914227813 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a75600700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 0x7f4a70197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 -- 172.21.15.57:0/2914227813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 msgr2=0x7f4a70197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 0x7f4a70197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 -- 172.21.15.57:0/2914227813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c0095f0 con 0x7f4a70101c40 2024-02-28T23:23:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.272+0000 7f4a74c00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0092e0 tx=0x7f4a5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.273+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5c01d070 con 0x7f4a70101c40 2024-02-28T23:23:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.273+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a5c005e00 con 0x7f4a70101c40 2024-02-28T23:23:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.273+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5c017400 con 0x7f4a70101c40 2024-02-28T23:23:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.273+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7019cb10 con 0x7f4a70101c40 2024-02-28T23:23:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.273+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7019d080 con 0x7f4a70101c40 2024-02-28T23:23:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.274+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a70062380 con 0x7f4a70101c40 2024-02-28T23:23:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.275+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a5c00f460 con 0x7f4a70101c40 2024-02-28T23:23:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.276+0000 7f4a6e000700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.276+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a5c08ea30 con 0x7f4a70101c40 2024-02-28T23:23:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.276+0000 7f4a75600700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.277+0000 7f4a75600700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005950 tx=0x7f4a640058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:35.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.281+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a5c0581c0 con 0x7f4a70101c40 2024-02-28T23:23:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.532+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a7019d420 con 0x7f4a5806d8a0 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.535+0000 7f4a6e000700 1 -- 172.21.15.57:0/2914227813 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a7019d420 con 0x7f4a5806d8a0 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:23:35.534 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:23:35.535 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:23:35.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.537+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 msgr2=0x7f4a5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.537+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005950 tx=0x7f4a640058e0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.537+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 msgr2=0x7f4a70198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.537+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0092e0 tx=0x7f4a5c00be20 comp rx=0 tx=0).stop 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 shutdown_connections 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a701003e0 0x7f4a70197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 --2- 172.21.15.57:0/2914227813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70101c40 0x7f4a70198420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 >> 172.21.15.57:0/2914227813 conn(0x7f4a700fbbc0 msgr2=0x7f4a70072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 shutdown_connections 2024-02-28T23:23:35.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.538+0000 7f4a76a00700 1 -- 172.21.15.57:0/2914227813 wait complete. 2024-02-28T23:23:35.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 -- 172.21.15.57:0/2079167459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 msgr2=0x7f192c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 --2- 172.21.15.57:0/2079167459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c1008b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f191c009910 tx=0x7f191c009c20 comp rx=0 tx=0).stop 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 -- 172.21.15.57:0/2079167459 shutdown_connections 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 --2- 172.21.15.57:0/2079167459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 --2- 172.21.15.57:0/2079167459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c1008b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 -- 172.21.15.57:0/2079167459 >> 172.21.15.57:0/2079167459 conn(0x7f192c0fbda0 msgr2=0x7f192c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 -- 172.21.15.57:0/2079167459 shutdown_connections 2024-02-28T23:23:35.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.694+0000 7f1933600700 1 -- 172.21.15.57:0/2079167459 wait complete. 2024-02-28T23:23:35.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.695+0000 7f1933600700 1 Processor -- start 2024-02-28T23:23:35.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1933600700 1 -- start start 2024-02-28T23:23:35.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1933600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1933600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1933600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c073ad0 con 0x7f192c1004e0 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1933600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c073c10 con 0x7f192c100e80 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1930c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1930c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59090/0 (socket says 172.21.15.57:59090) 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.696+0000 7f1930c00700 1 -- 172.21.15.57:0/4125396065 learned_addr learned my addr 172.21.15.57:0/4125396065 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1923e00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1930c00700 1 -- 172.21.15.57:0/4125396065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 msgr2=0x7f192c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1930c00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1930c00700 1 -- 172.21.15.57:0/4125396065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0095f0 con 0x7f192c1004e0 2024-02-28T23:23:35.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1923e00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:23:35.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.697+0000 7f1930c00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f191c00b5c0 tx=0x7f191c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:35.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.698+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191c01d070 con 0x7f192c1004e0 2024-02-28T23:23:35.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.698+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f191c00f460 con 0x7f192c1004e0 2024-02-28T23:23:35.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.698+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191c017610 con 0x7f192c1004e0 2024-02-28T23:23:35.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.698+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c073e90 con 0x7f192c1004e0 2024-02-28T23:23:35.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.698+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c1a23f0 con 0x7f192c1004e0 2024-02-28T23:23:35.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.699+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c1087f0 con 0x7f192c1004e0 2024-02-28T23:23:35.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.701+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f191c00f5d0 con 0x7f192c1004e0 2024-02-28T23:23:35.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.701+0000 7f192a000700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 0x7f191006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:23:35.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.702+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f191c08eb30 con 0x7f192c1004e0 2024-02-28T23:23:35.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.702+0000 7f1923e00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 0x7f191006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:23:35.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.703+0000 7f1923e00700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 0x7f191006fd50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1914005fd0 tx=0x7f1914005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:23:35.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:35.711+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f191c0582c0 con 0x7f192c1004e0 2024-02-28T23:23:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:35 smithi057 ceph-mon[89491]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:35 smithi057 ceph-mon[89491]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:35.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4053102014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:23:35.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1486316782' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:23:36.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.028+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f192c074690 con 0x7f192c1004e0 2024-02-28T23:23:36.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.029+0000 7f192a000700 1 -- 172.21.15.57:0/4125396065 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f191c05b980 con 0x7f192c1004e0 2024-02-28T23:23:36.028 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:23:36.029 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:23:36.029 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:23:36.029 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:23:36.029 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:23:36.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 msgr2=0x7f191006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 0x7f191006fd50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1914005fd0 tx=0x7f1914005dc0 comp rx=0 tx=0).stop 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 msgr2=0x7f192c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f191c00b5c0 tx=0x7f191c00be00 comp rx=0 tx=0).stop 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 shutdown_connections 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f191006d8a0 0x7f191006fd50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f192c1004e0 0x7f192c073050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 --2- 172.21.15.57:0/4125396065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192c100e80 0x7f192c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 >> 172.21.15.57:0/4125396065 conn(0x7f192c0fbda0 msgr2=0x7f192c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 shutdown_connections 2024-02-28T23:23:36.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:23:36.032+0000 7f1933600700 1 -- 172.21.15.57:0/4125396065 wait complete. 2024-02-28T23:23:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:35 smithi084 ceph-mon[70440]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:35 smithi084 ceph-mon[70440]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4053102014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:23:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1486316782' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:23:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:36 smithi057 ceph-mon[89491]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4125396065' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:23:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:36 smithi084 ceph-mon[70440]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:23:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4125396065' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:23:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:37 smithi057 ceph-mon[89491]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:37 smithi084 ceph-mon[70440]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:39 smithi057 ceph-mon[89491]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:39 smithi084 ceph-mon[70440]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:41 smithi057 ceph-mon[89491]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:41 smithi084 ceph-mon[70440]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:43.987 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:43 smithi084 ceph-mon[70440]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:43 smithi057 ceph-mon[89491]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:45 smithi057 ceph-mon[89491]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:45 smithi084 ceph-mon[70440]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:47 smithi057 ceph-mon[89491]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:47 smithi084 ceph-mon[70440]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:23:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:49 smithi057 ceph-mon[89491]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:49 smithi084 ceph-mon[70440]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:51 smithi057 ceph-mon[89491]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:51 smithi084 ceph-mon[70440]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:53 smithi057 ceph-mon[89491]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:53 smithi084 ceph-mon[70440]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:55 smithi057 ceph-mon[89491]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:55 smithi084 ceph-mon[70440]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:57 smithi057 ceph-mon[89491]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:23:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:57 smithi084 ceph-mon[70440]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:23:59 smithi057 ceph-mon[89491]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:23:59 smithi084 ceph-mon[70440]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:01 smithi057 ceph-mon[89491]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:02.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:01 smithi084 ceph-mon[70440]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:03 smithi057 ceph-mon[89491]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:03 smithi084 ceph-mon[70440]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:05 smithi057 ceph-mon[89491]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:05 smithi084 ceph-mon[70440]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 -- 172.21.15.57:0/2026456387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 msgr2=0x7f7028108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 --2- 172.21.15.57:0/2026456387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028108840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7018009910 tx=0x7f7018009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 -- 172.21.15.57:0/2026456387 shutdown_connections 2024-02-28T23:24:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 --2- 172.21.15.57:0/2026456387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 0x7f70281028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 --2- 172.21.15.57:0/2026456387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028108840 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 -- 172.21.15.57:0/2026456387 >> 172.21.15.57:0/2026456387 conn(0x7f70280fdf90 msgr2=0x7f70281003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 -- 172.21.15.57:0/2026456387 shutdown_connections 2024-02-28T23:24:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.209+0000 7f702e600700 1 -- 172.21.15.57:0/2026456387 wait complete. 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.210+0000 7f702e600700 1 Processor -- start 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.210+0000 7f702e600700 1 -- start start 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f702e600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 0x7f7028198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f702e600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f702e600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028198e30 con 0x7f7028108470 2024-02-28T23:24:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f702e600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702819cbc0 con 0x7f7028102470 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f701fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f701fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60286/0 (socket says 172.21.15.57:60286) 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f701fe00700 1 -- 172.21.15.57:0/3510799544 learned_addr learned my addr 172.21.15.57:0/3510799544 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.211+0000 7f7027400700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 0x7f7028198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f701fe00700 1 -- 172.21.15.57:0/3510799544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 msgr2=0x7f7028198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f701fe00700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 0x7f7028198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f701fe00700 1 -- 172.21.15.57:0/3510799544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180095f0 con 0x7f7028108470 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f701fe00700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f701000b4b0 tx=0x7f701000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7010011800 con 0x7f7028108470 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7010011e20 con 0x7f7028108470 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701000f2e0 con 0x7f7028108470 2024-02-28T23:24:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702819cea0 con 0x7f7028108470 2024-02-28T23:24:06.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.212+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702819d3f0 con 0x7f7028108470 2024-02-28T23:24:06.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.214+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702810a9e0 con 0x7f7028108470 2024-02-28T23:24:06.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.215+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7010010380 con 0x7f7028108470 2024-02-28T23:24:06.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.216+0000 7f7024c00700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 0x7f700c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.216+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f701008cff0 con 0x7f7028108470 2024-02-28T23:24:06.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.216+0000 7f7027400700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 0x7f700c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.217+0000 7f7027400700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 0x7f700c06fd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7018006010 tx=0x7f701801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.221+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70100c0d20 con 0x7f7028108470 2024-02-28T23:24:06.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.472+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f702819d6c0 con 0x7f700c06d8a0 2024-02-28T23:24:06.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.476+0000 7f7024c00700 1 -- 172.21.15.57:0/3510799544 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f702819d6c0 con 0x7f700c06d8a0 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 msgr2=0x7f700c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 0x7f700c06fd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7018006010 tx=0x7f701801a040 comp rx=0 tx=0).stop 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 msgr2=0x7f7028198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f701000b4b0 tx=0x7f701000b7c0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 shutdown_connections 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f700c06d8a0 0x7f700c06fd50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7028102470 0x7f7028198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 --2- 172.21.15.57:0/3510799544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7028108470 0x7f7028198750 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.478+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 >> 172.21.15.57:0/3510799544 conn(0x7f70280fdf90 msgr2=0x7f70280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:06.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.479+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 shutdown_connections 2024-02-28T23:24:06.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.479+0000 7f702e600700 1 -- 172.21.15.57:0/3510799544 wait complete. 2024-02-28T23:24:06.494 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:24:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 -- 172.21.15.57:0/2016114625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d4100c70 msgr2=0x7f84d4104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 --2- 172.21.15.57:0/2016114625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d4100c70 0x7f84d4104c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009910 tx=0x7f84c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 -- 172.21.15.57:0/2016114625 shutdown_connections 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 --2- 172.21.15.57:0/2016114625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d4100c70 0x7f84d4104c40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 --2- 172.21.15.57:0/2016114625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d41002d0 0x7f84d41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 -- 172.21.15.57:0/2016114625 >> 172.21.15.57:0/2016114625 conn(0x7f84d40fbbb0 msgr2=0x7f84d40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 -- 172.21.15.57:0/2016114625 shutdown_connections 2024-02-28T23:24:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.647+0000 7f84da400700 1 -- 172.21.15.57:0/2016114625 wait complete. 2024-02-28T23:24:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 Processor -- start 2024-02-28T23:24:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 -- start start 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d4100c70 0x7f84d419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d419c600 con 0x7f84d41002d0 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.649+0000 7f84da400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d4195ab0 con 0x7f84d4100c70 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60306/0 (socket says 172.21.15.57:60306) 2024-02-28T23:24:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 -- 172.21.15.57:0/2682615906 learned_addr learned my addr 172.21.15.57:0/2682615906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d3e00700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d4100c70 0x7f84d419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 -- 172.21.15.57:0/2682615906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d4100c70 msgr2=0x7f84d419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d4100c70 0x7f84d419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 -- 172.21.15.57:0/2682615906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00095f0 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.650+0000 7f84d9000700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e920 tx=0x7f84c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.651+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c8009790 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.651+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84c800cba0 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.651+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d4195d90 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.651+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c8018650 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.651+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d41962e0 con 0x7f84d41002d0 2024-02-28T23:24:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.652+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d4062380 con 0x7f84d41002d0 2024-02-28T23:24:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.653+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f84c800cd10 con 0x7f84d41002d0 2024-02-28T23:24:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.654+0000 7f84d1600700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 0x7f84bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.654+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f84c8014070 con 0x7f84d41002d0 2024-02-28T23:24:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.654+0000 7f84d3e00700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 0x7f84bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.655+0000 7f84d3e00700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 0x7f84bc06fd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84c000b760 tx=0x7f84c0009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:06.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.659+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84c8052d20 con 0x7f84d41002d0 2024-02-28T23:24:06.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.911+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84d41966e0 con 0x7f84bc06d8a0 2024-02-28T23:24:06.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.913+0000 7f84d1600700 1 -- 172.21.15.57:0/2682615906 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f84d41966e0 con 0x7f84bc06d8a0 2024-02-28T23:24:06.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 msgr2=0x7f84bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 0x7f84bc06fd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84c000b760 tx=0x7f84c0009f90 comp rx=0 tx=0).stop 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 msgr2=0x7f84d419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e920 tx=0x7f84c800ece0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 shutdown_connections 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f84bc06d8a0 0x7f84bc06fd50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d41002d0 0x7f84d419ba30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 --2- 172.21.15.57:0/2682615906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d4100c70 0x7f84d419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 >> 172.21.15.57:0/2682615906 conn(0x7f84d40fbbb0 msgr2=0x7f84d40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 shutdown_connections 2024-02-28T23:24:06.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:06.916+0000 7f84da400700 1 -- 172.21.15.57:0/2682615906 wait complete. 2024-02-28T23:24:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 -- 172.21.15.57:0/1595812010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 msgr2=0x7fdb40073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 --2- 172.21.15.57:0/1595812010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40073460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009940 tx=0x7fdb30009c50 comp rx=0 tx=0).stop 2024-02-28T23:24:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 -- 172.21.15.57:0/1595812010 shutdown_connections 2024-02-28T23:24:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 --2- 172.21.15.57:0/1595812010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 0x7fdb4010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 --2- 172.21.15.57:0/1595812010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40073460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 -- 172.21.15.57:0/1595812010 >> 172.21.15.57:0/1595812010 conn(0x7fdb400fbda0 msgr2=0x7fdb400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:07.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 -- 172.21.15.57:0/1595812010 shutdown_connections 2024-02-28T23:24:07.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.071+0000 7fdb47000700 1 -- 172.21.15.57:0/1595812010 wait complete. 2024-02-28T23:24:07.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.072+0000 7fdb47000700 1 Processor -- start 2024-02-28T23:24:07.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.072+0000 7fdb47000700 1 -- start start 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb47000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb47000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 0x7fdb401986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb47000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40198da0 con 0x7fdb40073090 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb47000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4019cb30 con 0x7fdb400739a0 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60332/0 (socket says 172.21.15.57:60332) 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 -- 172.21.15.57:0/859592778 learned_addr learned my addr 172.21.15.57:0/859592778 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb37e00700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 0x7fdb401986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 -- 172.21.15.57:0/859592778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 msgr2=0x7fdb401986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 0x7fdb401986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.073+0000 7fdb3fe00700 1 -- 172.21.15.57:0/859592778 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300095f0 con 0x7fdb40073090 2024-02-28T23:24:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb3fe00700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdb300055f0 tx=0x7fdb300056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3001d070 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4019cdb0 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb3000f460 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4019d2a0 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.074+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb300215e0 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.075+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb40109ce0 con 0x7fdb40073090 2024-02-28T23:24:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.076+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdb3000fa20 con 0x7fdb40073090 2024-02-28T23:24:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.077+0000 7fdb3d600700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 0x7fdb2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.077+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb3008e750 con 0x7fdb40073090 2024-02-28T23:24:07.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.078+0000 7fdb37e00700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 0x7fdb2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.078+0000 7fdb37e00700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 0x7fdb2406fd50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb28005950 tx=0x7fdb280058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.082+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb30057e60 con 0x7fdb40073090 2024-02-28T23:24:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.319+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb4019d510 con 0x7fdb2406d8a0 2024-02-28T23:24:07.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.352+0000 7fdb3d600700 1 -- 172.21.15.57:0/859592778 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdb4019d510 con 0x7fdb2406d8a0 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (13m) 5m ago 27m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (23m) 5m ago 27m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (26m) 4m ago 26m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (9m) 5m ago 27m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (9m) 4m ago 26m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (13m) 5m ago 27m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:24:07.351 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (5m) 5m ago 20m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (5m) 5m ago 20m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (20m) 4m ago 20m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (4m) 4m ago 20m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (16m) 5m ago 29m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (15m) 4m ago 26m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (11m) 5m ago 29m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (10m) 4m ago 26m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (14m) 5m ago 27m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (14m) 4m ago 26m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (9m) 5m ago 25m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (8m) 5m ago 24m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:24:07.352 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (8m) 5m ago 23m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:24:07.353 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (7m) 4m ago 23m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:24:07.353 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (6m) 4m ago 22m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:24:07.353 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (6m) 4m ago 22m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:24:07.353 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (14m) 5m ago 26m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:24:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.355+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 msgr2=0x7fdb2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.355+0000 7fdb47000700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 0x7fdb2406fd50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb28005950 tx=0x7fdb280058e0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.355+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 msgr2=0x7fdb40198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.355+0000 7fdb47000700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdb300055f0 tx=0x7fdb300056d0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 shutdown_connections 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdb2406d8a0 0x7fdb2406fd50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb40073090 0x7fdb40198180 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 --2- 172.21.15.57:0/859592778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb400739a0 0x7fdb401986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 >> 172.21.15.57:0/859592778 conn(0x7fdb400fbda0 msgr2=0x7fdb40106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 shutdown_connections 2024-02-28T23:24:07.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.356+0000 7fdb47000700 1 -- 172.21.15.57:0/859592778 wait complete. 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 -- 172.21.15.57:0/1270314222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7224068540 msgr2=0x7f72240689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 --2- 172.21.15.57:0/1270314222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7224068540 0x7f72240689b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7210009910 tx=0x7f7210009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 -- 172.21.15.57:0/1270314222 shutdown_connections 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 --2- 172.21.15.57:0/1270314222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7224068540 0x7f72240689b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 --2- 172.21.15.57:0/1270314222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72241061b0 0x7f7224106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 -- 172.21.15.57:0/1270314222 >> 172.21.15.57:0/1270314222 conn(0x7f7224074fa0 msgr2=0x7f72240753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 -- 172.21.15.57:0/1270314222 shutdown_connections 2024-02-28T23:24:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.517+0000 7f722aa00700 1 -- 172.21.15.57:0/1270314222 wait complete. 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.518+0000 7f722aa00700 1 Processor -- start 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f722aa00700 1 -- start start 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f722aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 0x7f7224197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f722aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f722aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224198960 con 0x7f72241061b0 2024-02-28T23:24:07.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f722aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722419c6f0 con 0x7f7224068540 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f7228c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f7228c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60348/0 (socket says 172.21.15.57:60348) 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.519+0000 7f7228c00700 1 -- 172.21.15.57:0/1056631376 learned_addr learned my addr 172.21.15.57:0/1056631376 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7229600700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 0x7f7224197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7228c00700 1 -- 172.21.15.57:0/1056631376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 msgr2=0x7f7224197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7228c00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 0x7f7224197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7228c00700 1 -- 172.21.15.57:0/1056631376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100095f0 con 0x7f72241061b0 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7229600700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 0x7f7224197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:24:07.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7228c00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f721000b5c0 tx=0x7f721000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721001d070 con 0x7f72241061b0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7210005e00 con 0x7f72241061b0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.520+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722419c970 con 0x7f72241061b0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.521+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7210017400 con 0x7f72241061b0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.521+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722419cee0 con 0x7f72241061b0 2024-02-28T23:24:07.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.522+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7224100a20 con 0x7f72241061b0 2024-02-28T23:24:07.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.523+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f721000f460 con 0x7f72241061b0 2024-02-28T23:24:07.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.524+0000 7f7222000700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 0x7f720c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:07.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.524+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f721008ea10 con 0x7f72241061b0 2024-02-28T23:24:07.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.524+0000 7f7229600700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 0x7f720c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:07.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.525+0000 7f7229600700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 0x7f720c06fd50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7218005fd0 tx=0x7f7218005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:07.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.528+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72100581a0 con 0x7f72241061b0 2024-02-28T23:24:07.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.846+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7224062380 con 0x7f72241061b0 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.846+0000 7f7222000700 1 -- 172.21.15.57:0/1056631376 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f721005b860 con 0x7f72241061b0 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:24:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:24:07.846 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 msgr2=0x7f720c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 0x7f720c06fd50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7218005fd0 tx=0x7f7218005ee0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 msgr2=0x7f7224198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f721000b5c0 tx=0x7f721000be20 comp rx=0 tx=0).stop 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 shutdown_connections 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f720c06d8a0 0x7f720c06fd50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7224068540 0x7f7224197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.849+0000 7f722aa00700 1 --2- 172.21.15.57:0/1056631376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72241061b0 0x7f7224198280 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:07.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.850+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 >> 172.21.15.57:0/1056631376 conn(0x7f7224074fa0 msgr2=0x7f722410a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:07.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.850+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 shutdown_connections 2024-02-28T23:24:07.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:07.850+0000 7f722aa00700 1 -- 172.21.15.57:0/1056631376 wait complete. 2024-02-28T23:24:08.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 -- 172.21.15.57:0/2399846651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 msgr2=0x7fb5bc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 --2- 172.21.15.57:0/2399846651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc106ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009910 tx=0x7fb5a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:08.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 -- 172.21.15.57:0/2399846651 shutdown_connections 2024-02-28T23:24:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 --2- 172.21.15.57:0/2399846651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc106ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 --2- 172.21.15.57:0/2399846651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 -- 172.21.15.57:0/2399846651 >> 172.21.15.57:0/2399846651 conn(0x7fb5bc075e90 msgr2=0x7fb5bc0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 -- 172.21.15.57:0/2399846651 shutdown_connections 2024-02-28T23:24:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.005+0000 7fb5bb200700 1 -- 172.21.15.57:0/2399846651 wait complete. 2024-02-28T23:24:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.006+0000 7fb5bb200700 1 Processor -- start 2024-02-28T23:24:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.006+0000 7fb5bb200700 1 -- start start 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5bb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5bb200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5bb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc19cf60 con 0x7fb5bc106b80 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5bb200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1a0cf0 con 0x7fb5bc10b640 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60370/0 (socket says 172.21.15.57:60370) 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9e00700 1 -- 172.21.15.57:0/792533034 learned_addr learned my addr 172.21.15.57:0/792533034 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9400700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9e00700 1 -- 172.21.15.57:0/792533034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 msgr2=0x7fb5bc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.007+0000 7fb5b9e00700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b9e00700 1 -- 172.21.15.57:0/792533034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80095f0 con 0x7fb5bc106b80 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b9400700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:24:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b9e00700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00b4b0 tx=0x7fb5ac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ac011800 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5ac011e20 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc1a0fd0 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ac00f2e0 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.008+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc1a1520 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.009+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc10eba0 con 0x7fb5bc106b80 2024-02-28T23:24:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.010+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb5ac010380 con 0x7fb5bc106b80 2024-02-28T23:24:08.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.011+0000 7fb5b2a00700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 0x7fb5a006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.011+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb5ac08d180 con 0x7fb5bc106b80 2024-02-28T23:24:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.012+0000 7fb5b9400700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 0x7fb5a006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.012+0000 7fb5b9400700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 0x7fb5a006fd50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.016+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5ac056890 con 0x7fb5bc106b80 2024-02-28T23:24:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:07 smithi057 ceph-mon[89491]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:07 smithi057 ceph-mon[89491]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:07 smithi084 ceph-mon[70440]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:07 smithi084 ceph-mon[70440]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:08.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.281+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb5bc1a17a0 con 0x7fb5bc106b80 2024-02-28T23:24:08.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.282+0000 7fb5b2a00700 1 -- 172.21.15.57:0/792533034 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fb5ac059f50 con 0x7fb5bc106b80 2024-02-28T23:24:08.281 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:24:08.281 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:24:08.281 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:24:08.282 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:24:08.283 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:08.284 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:24:08.285 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:08.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 msgr2=0x7fb5a006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 0x7fb5a006fd50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a800b540 comp rx=0 tx=0).stop 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 msgr2=0x7fb5bc19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00b4b0 tx=0x7fb5ac00b870 comp rx=0 tx=0).stop 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 shutdown_connections 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb5a006d8a0 0x7fb5a006fd50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5bc106b80 0x7fb5bc19c340 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 --2- 172.21.15.57:0/792533034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5bc10b640 0x7fb5bc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 >> 172.21.15.57:0/792533034 conn(0x7fb5bc075e90 msgr2=0x7fb5bc076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 shutdown_connections 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.285+0000 7fb5bb200700 1 -- 172.21.15.57:0/792533034 wait complete. 2024-02-28T23:24:08.286 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3432820694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 msgr2=0x7f2b3810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3432820694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3810ccc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009910 tx=0x7f2b2c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3432820694 shutdown_connections 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3432820694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 0x7f2b38106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3432820694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3810ccc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3432820694 >> 172.21.15.57:0/3432820694 conn(0x7f2b38076060 msgr2=0x7f2b38078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:08.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3432820694 shutdown_connections 2024-02-28T23:24:08.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.436+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3432820694 wait complete. 2024-02-28T23:24:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.437+0000 7f2b3fc00700 1 Processor -- start 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b3fc00700 1 -- start start 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b3fc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 0x7f2b3819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b3fc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b3fc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3819d120 con 0x7f2b3810c8f0 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b3fc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b381a0eb0 con 0x7f2b38106b80 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b37e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b37e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60384/0 (socket says 172.21.15.57:60384) 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.438+0000 7f2b37e00700 1 -- 172.21.15.57:0/3612490437 learned_addr learned my addr 172.21.15.57:0/3612490437 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b3d200700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 0x7f2b3819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b37e00700 1 -- 172.21.15.57:0/3612490437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 msgr2=0x7f2b3819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b37e00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 0x7f2b3819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b37e00700 1 -- 172.21.15.57:0/3612490437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0095f0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b37e00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d8d0 tx=0x7f2b2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b24009940 con 0x7f2b3810c8f0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b24010460 con 0x7f2b3810c8f0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2400f5f0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b381a1190 con 0x7f2b3810c8f0 2024-02-28T23:24:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.439+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b381a16e0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.441+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b38109b20 con 0x7f2b3810c8f0 2024-02-28T23:24:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.441+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2b240105d0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.442+0000 7f2b35600700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 0x7f2b2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.443+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b2408d5d0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.443+0000 7f2b3d200700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 0x7f2b2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.443+0000 7f2b3d200700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 0x7f2b2006fd50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006010 tx=0x7f2b2c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.448+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b24056ce0 con 0x7f2b3810c8f0 2024-02-28T23:24:08.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.688+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b381a1900 con 0x7f2b2006d8a0 2024-02-28T23:24:08.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.691+0000 7f2b35600700 1 -- 172.21.15.57:0/3612490437 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b381a1900 con 0x7f2b2006d8a0 2024-02-28T23:24:08.689 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:24:08.690 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:24:08.691 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:24:08.691 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:24:08.691 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 msgr2=0x7f2b2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 0x7f2b2006fd50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006010 tx=0x7f2b2c00b540 comp rx=0 tx=0).stop 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 msgr2=0x7f2b3819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d8d0 tx=0x7f2b2400dc90 comp rx=0 tx=0).stop 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 shutdown_connections 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b2006d8a0 0x7f2b2006fd50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b38106b80 0x7f2b3819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 --2- 172.21.15.57:0/3612490437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b3810c8f0 0x7f2b3819ca40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 >> 172.21.15.57:0/3612490437 conn(0x7f2b38076060 msgr2=0x7f2b38077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.694+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 shutdown_connections 2024-02-28T23:24:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.695+0000 7f2b3fc00700 1 -- 172.21.15.57:0/3612490437 wait complete. 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 -- 172.21.15.57:0/888398676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3640ff9e0 msgr2=0x7ff3640ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 --2- 172.21.15.57:0/888398676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3640ff9e0 0x7ff3640ffe50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009910 tx=0x7ff34c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 -- 172.21.15.57:0/888398676 shutdown_connections 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 --2- 172.21.15.57:0/888398676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3640ff9e0 0x7ff3640ffe50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 --2- 172.21.15.57:0/888398676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff364102e30 0x7ff364103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 -- 172.21.15.57:0/888398676 >> 172.21.15.57:0/888398676 conn(0x7ff364074fa0 msgr2=0x7ff3640753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.859+0000 7ff36a000700 1 -- 172.21.15.57:0/888398676 shutdown_connections 2024-02-28T23:24:08.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.860+0000 7ff36a000700 1 -- 172.21.15.57:0/888398676 wait complete. 2024-02-28T23:24:08.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 Processor -- start 2024-02-28T23:24:08.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 -- start start 2024-02-28T23:24:08.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 0x7ff364197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364198980 con 0x7ff364102e30 2024-02-28T23:24:08.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.861+0000 7ff36a000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419c710 con 0x7ff3640ff9e0 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60400/0 (socket says 172.21.15.57:60400) 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 -- 172.21.15.57:0/2803347258 learned_addr learned my addr 172.21.15.57:0/2803347258 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff368c00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 0x7ff364197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 -- 172.21.15.57:0/2803347258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 msgr2=0x7ff364197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 0x7ff364197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff35be00700 1 -- 172.21.15.57:0/2803347258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c0095f0 con 0x7ff364102e30 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.862+0000 7ff368c00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 0x7ff364197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff35be00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00f690 tx=0x7ff34c00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c01d070 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff34c0050d0 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36419c990 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c022470 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.863+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36419cf00 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.865+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364062380 con 0x7ff364102e30 2024-02-28T23:24:08.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.865+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff34c003680 con 0x7ff364102e30 2024-02-28T23:24:08.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.866+0000 7ff362000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 0x7ff34406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:08.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.866+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff34c08ea10 con 0x7ff364102e30 2024-02-28T23:24:08.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.867+0000 7ff368c00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 0x7ff34406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:08.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.867+0000 7ff368c00700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 0x7ff34406fd50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff354009be0 tx=0x7ff354009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:08.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:08.871+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff34c058120 con 0x7ff364102e30 2024-02-28T23:24:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:08 smithi057 ceph-mon[89491]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:08 smithi057 ceph-mon[89491]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1056631376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:24:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/792533034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:24:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:08 smithi084 ceph-mon[70440]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:08 smithi084 ceph-mon[70440]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1056631376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:24:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/792533034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:24:09.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.195+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff36419d220 con 0x7ff364102e30 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.195+0000 7ff362000700 1 -- 172.21.15.57:0/2803347258 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff34c05b7e0 con 0x7ff364102e30 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:24:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:24:09.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.198+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 msgr2=0x7ff34406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:09.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.198+0000 7ff36a000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 0x7ff34406fd50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff354009be0 tx=0x7ff354009260 comp rx=0 tx=0).stop 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.198+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 msgr2=0x7ff3641982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.198+0000 7ff36a000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00f690 tx=0x7ff34c00f770 comp rx=0 tx=0).stop 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 shutdown_connections 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff34406d8a0 0x7ff34406fd50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3640ff9e0 0x7ff364197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:09.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 --2- 172.21.15.57:0/2803347258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff364102e30 0x7ff3641982a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:09.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 >> 172.21.15.57:0/2803347258 conn(0x7ff364074fa0 msgr2=0x7ff3641095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:09.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 shutdown_connections 2024-02-28T23:24:09.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:09.199+0000 7ff36a000700 1 -- 172.21.15.57:0/2803347258 wait complete. 2024-02-28T23:24:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:09 smithi057 ceph-mon[89491]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:09 smithi057 ceph-mon[89491]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:10.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:09 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2803347258' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:24:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:09 smithi084 ceph-mon[70440]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:09 smithi084 ceph-mon[70440]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:09 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2803347258' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:24:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:11 smithi084 ceph-mon[70440]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:12.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:11 smithi057 ceph-mon[89491]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:14.121 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:13 smithi057 ceph-mon[89491]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:14.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:13 smithi084 ceph-mon[70440]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:16.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:15 smithi057 ceph-mon[89491]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:15 smithi084 ceph-mon[70440]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:18.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:17 smithi057 ceph-mon[89491]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:18.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:17 smithi084 ceph-mon[70440]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:19.037 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:19 smithi057 ceph-mon[89491]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:19 smithi084 ceph-mon[70440]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:22.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:21 smithi057 ceph-mon[89491]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:22.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:21 smithi084 ceph-mon[70440]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:24.203 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:23 smithi057 ceph-mon[89491]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:24.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:23 smithi084 ceph-mon[70440]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:25.246 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:24 smithi084 ceph-mon[70440]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:25.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:24 smithi057 ceph-mon[89491]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:27.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:27 smithi057 ceph-mon[89491]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:27.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:27 smithi084 ceph-mon[70440]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:29.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:29 smithi057 ceph-mon[89491]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:29 smithi084 ceph-mon[70440]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:31.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:31 smithi057 ceph-mon[89491]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:31.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:31 smithi084 ceph-mon[70440]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:33 smithi057 ceph-mon[89491]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:33.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:33 smithi084 ceph-mon[70440]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:33.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:35.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:35 smithi084 ceph-mon[70440]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:35 smithi057 ceph-mon[89491]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:37 smithi057 ceph-mon[89491]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:37 smithi084 ceph-mon[70440]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:39.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 -- 172.21.15.57:0/3672094195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 msgr2=0x7fed701028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 --2- 172.21.15.57:0/3672094195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701028e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009910 tx=0x7fed5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 -- 172.21.15.57:0/3672094195 shutdown_connections 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 --2- 172.21.15.57:0/3672094195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701028e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 --2- 172.21.15.57:0/3672094195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 0x7fed70108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 -- 172.21.15.57:0/3672094195 >> 172.21.15.57:0/3672094195 conn(0x7fed700fdf90 msgr2=0x7fed701003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 -- 172.21.15.57:0/3672094195 shutdown_connections 2024-02-28T23:24:39.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.375+0000 7fed77400700 1 -- 172.21.15.57:0/3672094195 wait complete. 2024-02-28T23:24:39.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.376+0000 7fed77400700 1 Processor -- start 2024-02-28T23:24:39.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.376+0000 7fed77400700 1 -- start start 2024-02-28T23:24:39.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed77400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed77400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 0x7fed70198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed77400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed70198e10 con 0x7fed70102470 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed77400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019cba0 con 0x7fed70108470 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed74a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed74a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55078/0 (socket says 172.21.15.57:55078) 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed74a00700 1 -- 172.21.15.57:0/3838483044 learned_addr learned my addr 172.21.15.57:0/3838483044 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:39.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed6fe00700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 0x7fed70198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed74a00700 1 -- 172.21.15.57:0/3838483044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 msgr2=0x7fed70198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.377+0000 7fed74a00700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 0x7fed70198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed74a00700 1 -- 172.21.15.57:0/3838483044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0095f0 con 0x7fed70102470 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed74a00700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d8d0 tx=0x7fed6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed64009940 con 0x7fed70102470 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed64010460 con 0x7fed70102470 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7019ce80 con 0x7fed70102470 2024-02-28T23:24:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6400f5f0 con 0x7fed70102470 2024-02-28T23:24:39.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.378+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7019d3d0 con 0x7fed70102470 2024-02-28T23:24:39.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.380+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7010a970 con 0x7fed70102470 2024-02-28T23:24:39.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.381+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fed640105d0 con 0x7fed70102470 2024-02-28T23:24:39.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.382+0000 7fed6d600700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 0x7fed5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.382+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed6408d4c0 con 0x7fed70102470 2024-02-28T23:24:39.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.382+0000 7fed6fe00700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 0x7fed5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.383+0000 7fed6fe00700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 0x7fed5806fd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed70199810 tx=0x7fed5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:39.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.387+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed64056c50 con 0x7fed70102470 2024-02-28T23:24:39.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.641+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed70199690 con 0x7fed5806d8a0 2024-02-28T23:24:39.641 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:39 smithi057 ceph-mon[89491]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:39.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.643+0000 7fed6d600700 1 -- 172.21.15.57:0/3838483044 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed5c0055e0 con 0x7fed5806d8a0 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 msgr2=0x7fed5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 0x7fed5806fd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed70199810 tx=0x7fed5c01a040 comp rx=0 tx=0).stop 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 msgr2=0x7fed701981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d8d0 tx=0x7fed6400dc90 comp rx=0 tx=0).stop 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 shutdown_connections 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5806d8a0 0x7fed5806fd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed70102470 0x7fed701981f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 --2- 172.21.15.57:0/3838483044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70108470 0x7fed70198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 >> 172.21.15.57:0/3838483044 conn(0x7fed700fdf90 msgr2=0x7fed700ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 shutdown_connections 2024-02-28T23:24:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.647+0000 7fed77400700 1 -- 172.21.15.57:0/3838483044 wait complete. 2024-02-28T23:24:39.664 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 -- 172.21.15.57:0/1164079673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 msgr2=0x7f23041008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 --2- 172.21.15.57:0/1164079673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f23041008b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009910 tx=0x7f22f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 -- 172.21.15.57:0/1164079673 shutdown_connections 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 --2- 172.21.15.57:0/1164079673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 0x7f2304104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 --2- 172.21.15.57:0/1164079673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f23041008b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.822+0000 7f2309200700 1 -- 172.21.15.57:0/1164079673 >> 172.21.15.57:0/1164079673 conn(0x7f23040fbda0 msgr2=0x7f23040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:39.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.823+0000 7f2309200700 1 -- 172.21.15.57:0/1164079673 shutdown_connections 2024-02-28T23:24:39.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.823+0000 7f2309200700 1 -- 172.21.15.57:0/1164079673 wait complete. 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 Processor -- start 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 -- start start 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 0x7f2304073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304073ad0 con 0x7f23041004e0 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.824+0000 7f2309200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304073c10 con 0x7f2304100e80 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55106/0 (socket says 172.21.15.57:55106) 2024-02-28T23:24:39.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 -- 172.21.15.57:0/3864137866 learned_addr learned my addr 172.21.15.57:0/3864137866 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:39.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301400700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 0x7f2304073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 -- 172.21.15.57:0/3864137866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 msgr2=0x7f2304073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:39.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 0x7f2304073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.825+0000 7f2301e00700 1 -- 172.21.15.57:0/3864137866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40095f0 con 0x7f23041004e0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f2301e00700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22f40092e0 tx=0x7f22f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f401d070 con 0x7f23041004e0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22f4005e00 con 0x7f23041004e0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f4017400 con 0x7f23041004e0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2304073e90 con 0x7f23041004e0 2024-02-28T23:24:39.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.826+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23041a23f0 con 0x7f23041004e0 2024-02-28T23:24:39.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.828+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23041087f0 con 0x7f23041004e0 2024-02-28T23:24:39.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.829+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f22f400f460 con 0x7f23041004e0 2024-02-28T23:24:39.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.829+0000 7f22faa00700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:39.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.830+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f22f408e990 con 0x7f23041004e0 2024-02-28T23:24:39.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.830+0000 7f2301400700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:39.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.831+0000 7f2301400700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005950 tx=0x7f22f00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:39.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:39.835+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22f40581c0 con 0x7f23041004e0 2024-02-28T23:24:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:39 smithi084 ceph-mon[70440]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:40.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.082+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2304061190 con 0x7f22e806d8a0 2024-02-28T23:24:40.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.084+0000 7f22faa00700 1 -- 172.21.15.57:0/3864137866 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2304061190 con 0x7f22e806d8a0 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 msgr2=0x7f22e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005950 tx=0x7f22f00058e0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 msgr2=0x7f2304073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22f40092e0 tx=0x7f22f400be20 comp rx=0 tx=0).stop 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 shutdown_connections 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23041004e0 0x7f2304073050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 --2- 172.21.15.57:0/3864137866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304100e80 0x7f2304073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.087+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 >> 172.21.15.57:0/3864137866 conn(0x7f23040fbda0 msgr2=0x7f23040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:40.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.088+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 shutdown_connections 2024-02-28T23:24:40.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.088+0000 7f2309200700 1 -- 172.21.15.57:0/3864137866 wait complete. 2024-02-28T23:24:40.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 -- 172.21.15.57:0/131508757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 msgr2=0x7f61c4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 --2- 172.21.15.57:0/131508757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c4106ff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009910 tx=0x7f61b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 -- 172.21.15.57:0/131508757 shutdown_connections 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 --2- 172.21.15.57:0/131508757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c4106ff0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 --2- 172.21.15.57:0/131508757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 0x7f61c410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 -- 172.21.15.57:0/131508757 >> 172.21.15.57:0/131508757 conn(0x7f61c4076060 msgr2=0x7f61c4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 -- 172.21.15.57:0/131508757 shutdown_connections 2024-02-28T23:24:40.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.255+0000 7f61cc200700 1 -- 172.21.15.57:0/131508757 wait complete. 2024-02-28T23:24:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.256+0000 7f61cc200700 1 Processor -- start 2024-02-28T23:24:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.256+0000 7f61cc200700 1 -- start start 2024-02-28T23:24:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61cc200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61cc200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 0x7f61c419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61cc200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c419d060 con 0x7f61c4106b80 2024-02-28T23:24:40.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61cc200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c41a0df0 con 0x7f61c410c8f0 2024-02-28T23:24:40.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61c9800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61c9800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55116/0 (socket says 172.21.15.57:55116) 2024-02-28T23:24:40.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.257+0000 7f61c9800700 1 -- 172.21.15.57:0/3218137580 learned_addr learned my addr 172.21.15.57:0/3218137580 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c8e00700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 0x7f61c419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c9800700 1 -- 172.21.15.57:0/3218137580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 msgr2=0x7f61c419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c9800700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 0x7f61c419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c9800700 1 -- 172.21.15.57:0/3218137580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00095f0 con 0x7f61c4106b80 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c9800700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f61b800d8d0 tx=0x7f61b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61b8009940 con 0x7f61c4106b80 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61b8010460 con 0x7f61c4106b80 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.258+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61b800f5f0 con 0x7f61c4106b80 2024-02-28T23:24:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.259+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c41a10d0 con 0x7f61c4106b80 2024-02-28T23:24:40.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.259+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c41a1620 con 0x7f61c4106b80 2024-02-28T23:24:40.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.260+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c410ecc0 con 0x7f61c4106b80 2024-02-28T23:24:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.261+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f61b80105d0 con 0x7f61c4106b80 2024-02-28T23:24:40.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.262+0000 7f61c2000700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 0x7f61ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.262+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f61b808d4e0 con 0x7f61c4106b80 2024-02-28T23:24:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.262+0000 7f61c8e00700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 0x7f61ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.263+0000 7f61c8e00700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 0x7f61ac06fd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61c419da60 tx=0x7f61b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.267+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61b8056c70 con 0x7f61c4106b80 2024-02-28T23:24:40.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.507+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c41a1900 con 0x7f61ac06d8a0 2024-02-28T23:24:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.542+0000 7f61c2000700 1 -- 172.21.15.57:0/3218137580 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f61c41a1900 con 0x7f61ac06d8a0 2024-02-28T23:24:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:24:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (14m) 5m ago 28m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:24:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (24m) 5m ago 28m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (27m) 5m ago 27m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (10m) 5m ago 28m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (10m) 5m ago 26m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (13m) 5m ago 27m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (6m) 5m ago 20m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (5m) 5m ago 20m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (20m) 5m ago 20m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (5m) 5m ago 20m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (16m) 5m ago 29m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (15m) 5m ago 26m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (12m) 5m ago 30m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (11m) 5m ago 26m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (15m) 5m ago 28m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (15m) 5m ago 26m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:24:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (9m) 5m ago 25m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (9m) 5m ago 25m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (8m) 5m ago 24m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (8m) 5m ago 23m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (7m) 5m ago 23m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (6m) 5m ago 22m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:24:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (14m) 5m ago 27m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 msgr2=0x7f61ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 0x7f61ac06fd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61c419da60 tx=0x7f61b001a040 comp rx=0 tx=0).stop 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 msgr2=0x7f61c419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f61b800d8d0 tx=0x7f61b800dc90 comp rx=0 tx=0).stop 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 shutdown_connections 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f61ac06d8a0 0x7f61ac06fd50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61c4106b80 0x7f61c419c440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 --2- 172.21.15.57:0/3218137580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c410c8f0 0x7f61c419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 >> 172.21.15.57:0/3218137580 conn(0x7f61c4076060 msgr2=0x7f61c4077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:40.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 shutdown_connections 2024-02-28T23:24:40.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.546+0000 7f61cc200700 1 -- 172.21.15.57:0/3218137580 wait complete. 2024-02-28T23:24:40.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.695+0000 7f4083e00700 1 -- 172.21.15.57:0/982441656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4084100c70 msgr2=0x7f4084104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.695+0000 7f4083e00700 1 --2- 172.21.15.57:0/982441656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4084100c70 0x7f4084104c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f406c009910 tx=0x7f406c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 -- 172.21.15.57:0/982441656 shutdown_connections 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 --2- 172.21.15.57:0/982441656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4084100c70 0x7f4084104c40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 --2- 172.21.15.57:0/982441656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40841002d0 0x7f40841006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 -- 172.21.15.57:0/982441656 >> 172.21.15.57:0/982441656 conn(0x7f40840fbbb0 msgr2=0x7f40840fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 -- 172.21.15.57:0/982441656 shutdown_connections 2024-02-28T23:24:40.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.696+0000 7f4083e00700 1 -- 172.21.15.57:0/982441656 wait complete. 2024-02-28T23:24:40.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 Processor -- start 2024-02-28T23:24:40.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 -- start start 2024-02-28T23:24:40.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 0x7f4084073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084073ad0 con 0x7f40841002d0 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4083e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084073c10 con 0x7f4084100c70 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.697+0000 7f4082a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55132/0 (socket says 172.21.15.57:55132) 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 -- 172.21.15.57:0/3191311166 learned_addr learned my addr 172.21.15.57:0/3191311166 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f407be00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 0x7f4084073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 -- 172.21.15.57:0/3191311166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 msgr2=0x7f4084073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 0x7f4084073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 -- 172.21.15.57:0/3191311166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0095f0 con 0x7f40841002d0 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f407be00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 0x7f4084073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:24:40.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.698+0000 7f4082a00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f40740060b0 tx=0x7f407400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.699+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407400dda0 con 0x7f40841002d0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.699+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f407400f040 con 0x7f40841002d0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.699+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4074014790 con 0x7f40841002d0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.699+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40841a1e30 con 0x7f40841002d0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.699+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40841a2240 con 0x7f40841002d0 2024-02-28T23:24:40.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.700+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4084062380 con 0x7f40841002d0 2024-02-28T23:24:40.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.701+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f40740148f0 con 0x7f40841002d0 2024-02-28T23:24:40.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.702+0000 7f407b400700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 0x7f406806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:40.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.702+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f407408cf90 con 0x7f40841002d0 2024-02-28T23:24:40.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.702+0000 7f407be00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 0x7f406806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:40.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.703+0000 7f407be00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 0x7f406806fd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f406c006010 tx=0x7f406c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:40.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:40.707+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40740567d0 con 0x7f40841002d0 2024-02-28T23:24:40.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:40 smithi057 ceph-mon[89491]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:41.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.011+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40841a2610 con 0x7f40841002d0 2024-02-28T23:24:41.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.012+0000 7f407b400700 1 -- 172.21.15.57:0/3191311166 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f407401a340 con 0x7f40841002d0 2024-02-28T23:24:41.011 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:24:41.012 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:24:41.013 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 msgr2=0x7f406806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 0x7f406806fd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f406c006010 tx=0x7f406c00b540 comp rx=0 tx=0).stop 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 msgr2=0x7f4084073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f40740060b0 tx=0x7f407400d3b0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 shutdown_connections 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f406806d8a0 0x7f406806fd50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40841002d0 0x7f4084073050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 --2- 172.21.15.57:0/3191311166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4084100c70 0x7f4084073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 >> 172.21.15.57:0/3191311166 conn(0x7f40840fbbb0 msgr2=0x7f40840fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.015+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 shutdown_connections 2024-02-28T23:24:41.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.016+0000 7f4083e00700 1 -- 172.21.15.57:0/3191311166 wait complete. 2024-02-28T23:24:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:40 smithi084 ceph-mon[70440]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.177+0000 7f2030a00700 1 -- 172.21.15.57:0/2322078683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075b90 msgr2=0x7f202c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.177+0000 7f2030a00700 1 --2- 172.21.15.57:0/2322078683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075b90 0x7f202c110ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2018009910 tx=0x7f2018009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 -- 172.21.15.57:0/2322078683 shutdown_connections 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 --2- 172.21.15.57:0/2322078683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075b90 0x7f202c110ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 --2- 172.21.15.57:0/2322078683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075280 0x7f202c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 -- 172.21.15.57:0/2322078683 >> 172.21.15.57:0/2322078683 conn(0x7f202c0fdf40 msgr2=0x7f202c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:41.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 -- 172.21.15.57:0/2322078683 shutdown_connections 2024-02-28T23:24:41.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.178+0000 7f2030a00700 1 -- 172.21.15.57:0/2322078683 wait complete. 2024-02-28T23:24:41.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 Processor -- start 2024-02-28T23:24:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 -- start start 2024-02-28T23:24:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075b90 0x7f202c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c19d090 con 0x7f202c075280 2024-02-28T23:24:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.179+0000 7f2030a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a0e20 con 0x7f202c075b90 2024-02-28T23:24:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55154/0 (socket says 172.21.15.57:55154) 2024-02-28T23:24:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 -- 172.21.15.57:0/4073350198 learned_addr learned my addr 172.21.15.57:0/4073350198 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029400700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075b90 0x7f202c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 -- 172.21.15.57:0/4073350198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075b90 msgr2=0x7f202c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075b90 0x7f202c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.180+0000 7f2029e00700 1 -- 172.21.15.57:0/4073350198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180095f0 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2029e00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f201c00d8d0 tx=0x7f201c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f201c009940 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f201c010460 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f201c00f5f0 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1a1100 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.181+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c1a1650 con 0x7f202c075280 2024-02-28T23:24:41.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.182+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c10e310 con 0x7f202c075280 2024-02-28T23:24:41.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.184+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f201c0105d0 con 0x7f202c075280 2024-02-28T23:24:41.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.185+0000 7f2022a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 0x7f201006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.185+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f201c08d4b0 con 0x7f202c075280 2024-02-28T23:24:41.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.185+0000 7f2029400700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 0x7f201006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.186+0000 7f2029400700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 0x7f201006fd50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2018005b40 tx=0x7f2018005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.188+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f201c056d70 con 0x7f202c075280 2024-02-28T23:24:41.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.471+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f202c04f2a0 con 0x7f202c075280 2024-02-28T23:24:41.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.472+0000 7f2022a00700 1 -- 172.21.15.57:0/4073350198 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f201c016740 con 0x7f202c075280 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:24:41.472 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:24:41.473 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:41.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:41.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:24:41.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 msgr2=0x7f201006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 0x7f201006fd50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2018005b40 tx=0x7f2018005a90 comp rx=0 tx=0).stop 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 msgr2=0x7f202c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f201c00d8d0 tx=0x7f201c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 shutdown_connections 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f201006d8a0 0x7f201006fd50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f202c075280 0x7f202c19c470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 --2- 172.21.15.57:0/4073350198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f202c075b90 0x7f202c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 >> 172.21.15.57:0/4073350198 conn(0x7f202c0fdf40 msgr2=0x7f202c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 shutdown_connections 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.475+0000 7f2030a00700 1 -- 172.21.15.57:0/4073350198 wait complete. 2024-02-28T23:24:41.476 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:24:41.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.636+0000 7f4f5f000700 1 -- 172.21.15.57:0/1118200386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f581002c0 msgr2=0x7f4f58100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.636+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1118200386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f581002c0 0x7f4f58100690 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009910 tx=0x7f4f4c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:41.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 -- 172.21.15.57:0/1118200386 shutdown_connections 2024-02-28T23:24:41.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1118200386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f58100c60 0x7f4f58104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1118200386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f581002c0 0x7f4f58100690 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 -- 172.21.15.57:0/1118200386 >> 172.21.15.57:0/1118200386 conn(0x7f4f580fbb60 msgr2=0x7f4f580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:41.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 -- 172.21.15.57:0/1118200386 shutdown_connections 2024-02-28T23:24:41.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.637+0000 7f4f5f000700 1 -- 172.21.15.57:0/1118200386 wait complete. 2024-02-28T23:24:41.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.638+0000 7f4f5f000700 1 Processor -- start 2024-02-28T23:24:41.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.638+0000 7f4f5f000700 1 -- start start 2024-02-28T23:24:41.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5f000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 0x7f4f5819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5819c590 con 0x7f4f58100c60 2024-02-28T23:24:41.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5f000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58195a40 con 0x7f4f581002c0 2024-02-28T23:24:41.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55170/0 (socket says 172.21.15.57:55170) 2024-02-28T23:24:41.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5d200700 1 -- 172.21.15.57:0/1058214380 learned_addr learned my addr 172.21.15.57:0/1058214380 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:41.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.639+0000 7f4f5dc00700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 0x7f4f5819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f5d200700 1 -- 172.21.15.57:0/1058214380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 msgr2=0x7f4f5819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f5d200700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 0x7f4f5819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f5d200700 1 -- 172.21.15.57:0/1058214380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0095f0 con 0x7f4f58100c60 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f5dc00700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 0x7f4f5819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f5d200700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d900 tx=0x7f4f4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.640+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f44009900 con 0x7f4f58100c60 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.641+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f44010460 con 0x7f4f58100c60 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.641+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f4400f5f0 con 0x7f4f58100c60 2024-02-28T23:24:41.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.641+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f58195d20 con 0x7f4f58100c60 2024-02-28T23:24:41.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.641+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f58196270 con 0x7f4f58100c60 2024-02-28T23:24:41.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.642+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f581085d0 con 0x7f4f58100c60 2024-02-28T23:24:41.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.643+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4f440105d0 con 0x7f4f58100c60 2024-02-28T23:24:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.645+0000 7f4f56000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 0x7f4f4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.645+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4f4408d100 con 0x7f4f58100c60 2024-02-28T23:24:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.645+0000 7f4f5dc00700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 0x7f4f4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:41.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.646+0000 7f4f5dc00700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 0x7f4f4006fd50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b5c0 tx=0x7f4f4c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:41.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.649+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f4405a3c0 con 0x7f4f58100c60 2024-02-28T23:24:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:41 smithi057 ceph-mon[89491]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:41 smithi057 ceph-mon[89491]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:41 smithi057 ceph-mon[89491]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:41.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:41 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3191311166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:24:41.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:41 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4073350198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:24:41.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.902+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f581965a0 con 0x7f4f4006d8a0 2024-02-28T23:24:41.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.904+0000 7f4f56000700 1 -- 172.21.15.57:0/1058214380 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4f581965a0 con 0x7f4f4006d8a0 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:24:41.904 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:24:41.905 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:24:41.905 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:24:41.905 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:24:41.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 msgr2=0x7f4f4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 0x7f4f4006fd50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b5c0 tx=0x7f4f4c01a040 comp rx=0 tx=0).stop 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 msgr2=0x7f4f5819bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d900 tx=0x7f4f4400dcc0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 shutdown_connections 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4f4006d8a0 0x7f4f4006fd50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f581002c0 0x7f4f5819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 --2- 172.21.15.57:0/1058214380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f58100c60 0x7f4f5819bf00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 >> 172.21.15.57:0/1058214380 conn(0x7f4f580fbb60 msgr2=0x7f4f580fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 shutdown_connections 2024-02-28T23:24:41.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:41.907+0000 7f4f5f000700 1 -- 172.21.15.57:0/1058214380 wait complete. 2024-02-28T23:24:42.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 -- 172.21.15.57:0/4178011324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa250100e80 msgr2=0x7fa250104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:42.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 --2- 172.21.15.57:0/4178011324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa250100e80 0x7fa250104e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009910 tx=0x7fa23c009c20 comp rx=0 tx=0).stop 2024-02-28T23:24:42.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 -- 172.21.15.57:0/4178011324 shutdown_connections 2024-02-28T23:24:42.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 --2- 172.21.15.57:0/4178011324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa250100e80 0x7fa250104e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 --2- 172.21.15.57:0/4178011324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2501004e0 0x7fa2501008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 -- 172.21.15.57:0/4178011324 >> 172.21.15.57:0/4178011324 conn(0x7fa2500fbda0 msgr2=0x7fa2500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:42.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 -- 172.21.15.57:0/4178011324 shutdown_connections 2024-02-28T23:24:42.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.068+0000 7fa257800700 1 -- 172.21.15.57:0/4178011324 wait complete. 2024-02-28T23:24:42.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.069+0000 7fa257800700 1 Processor -- start 2024-02-28T23:24:42.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.069+0000 7fa257800700 1 -- start start 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa257800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa257800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa250100e80 0x7fa2501963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa257800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa250196ac0 con 0x7fa2501004e0 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa257800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25019a850 con 0x7fa250100e80 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:42.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55184/0 (socket says 172.21.15.57:55184) 2024-02-28T23:24:42.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 -- 172.21.15.57:0/679206357 learned_addr learned my addr 172.21.15.57:0/679206357 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:24:42.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa24fe00700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa250100e80 0x7fa2501963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:42.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 -- 172.21.15.57:0/679206357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa250100e80 msgr2=0x7fa2501963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa250100e80 0x7fa2501963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.070+0000 7fa254e00700 1 -- 172.21.15.57:0/679206357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0095f0 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa254e00700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa24400d900 tx=0x7fa24400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa244004a00 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2440054e0 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25019ab30 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa244009d60 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.071+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25019b080 con 0x7fa2501004e0 2024-02-28T23:24:42.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.072+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa250108760 con 0x7fa2501004e0 2024-02-28T23:24:42.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.074+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa244005020 con 0x7fa2501004e0 2024-02-28T23:24:42.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.075+0000 7fa24d600700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 0x7fa23806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:24:42.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.075+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa24408d720 con 0x7fa2501004e0 2024-02-28T23:24:42.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.075+0000 7fa24fe00700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 0x7fa23806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:24:42.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.076+0000 7fa24fe00700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 0x7fa23806fd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2501974c0 tx=0x7fa23c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:24:42.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.080+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa244056eb0 con 0x7fa2501004e0 2024-02-28T23:24:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:41 smithi084 ceph-mon[70440]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:41 smithi084 ceph-mon[70440]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:41 smithi084 ceph-mon[70440]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:41 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3191311166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:24:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:41 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4073350198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:24:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.397+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa25004f2a0 con 0x7fa2501004e0 2024-02-28T23:24:42.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.398+0000 7fa24d600700 1 -- 172.21.15.57:0/679206357 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa24405a570 con 0x7fa2501004e0 2024-02-28T23:24:42.397 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:24:42.398 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:24:42.398 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:24:42.398 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:24:42.398 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:24:42.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 msgr2=0x7fa23806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:42.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 0x7fa23806fd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2501974c0 tx=0x7fa23c01a040 comp rx=0 tx=0).stop 2024-02-28T23:24:42.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 msgr2=0x7fa250195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:24:42.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa24400d900 tx=0x7fa24400dcc0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 shutdown_connections 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa23806d8a0 0x7fa23806fd50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2501004e0 0x7fa250195ea0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 --2- 172.21.15.57:0/679206357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa250100e80 0x7fa2501963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 >> 172.21.15.57:0/679206357 conn(0x7fa2500fbda0 msgr2=0x7fa2501046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.401+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 shutdown_connections 2024-02-28T23:24:42.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:24:42.402+0000 7fa257800700 1 -- 172.21.15.57:0/679206357 wait complete. 2024-02-28T23:24:42.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:42 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/679206357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:24:43.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:42 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/679206357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:24:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:43 smithi057 ceph-mon[89491]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:43 smithi057 ceph-mon[89491]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:43 smithi084 ceph-mon[70440]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:24:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:43 smithi084 ceph-mon[70440]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:45 smithi057 ceph-mon[89491]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:45 smithi084 ceph-mon[70440]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:47 smithi057 ceph-mon[89491]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:47 smithi084 ceph-mon[70440]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:24:49.992 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:49 smithi057 ceph-mon[89491]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:49 smithi084 ceph-mon[70440]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:51 smithi057 ceph-mon[89491]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:51 smithi084 ceph-mon[70440]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:53 smithi057 ceph-mon[89491]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:53 smithi084 ceph-mon[70440]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:55 smithi057 ceph-mon[89491]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:55 smithi084 ceph-mon[70440]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:57 smithi057 ceph-mon[89491]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:24:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:57 smithi084 ceph-mon[70440]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:24:59 smithi057 ceph-mon[89491]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:24:59 smithi084 ceph-mon[70440]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:01 smithi057 ceph-mon[89491]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:01 smithi084 ceph-mon[70440]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:03 smithi057 ceph-mon[89491]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:03 smithi084 ceph-mon[70440]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:05.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:05 smithi057 ceph-mon[89491]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:05 smithi084 ceph-mon[70440]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:07 smithi057 ceph-mon[89491]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:07 smithi084 ceph-mon[70440]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:09 smithi057 ceph-mon[89491]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:09 smithi084 ceph-mon[70440]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:11 smithi057 ceph-mon[89491]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:11 smithi084 ceph-mon[70440]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:12.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 -- 172.21.15.57:0/2263592859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 msgr2=0x7f60d0100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 --2- 172.21.15.57:0/2263592859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0100890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009910 tx=0x7f60c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 -- 172.21.15.57:0/2263592859 shutdown_connections 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 --2- 172.21.15.57:0/2263592859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 0x7f60d0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 --2- 172.21.15.57:0/2263592859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0100890 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 -- 172.21.15.57:0/2263592859 >> 172.21.15.57:0/2263592859 conn(0x7f60d00fbd80 msgr2=0x7f60d00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 -- 172.21.15.57:0/2263592859 shutdown_connections 2024-02-28T23:25:12.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.578+0000 7f60d7800700 1 -- 172.21.15.57:0/2263592859 wait complete. 2024-02-28T23:25:12.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.580+0000 7f60d7800700 1 Processor -- start 2024-02-28T23:25:12.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.580+0000 7f60d7800700 1 -- start start 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d7800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d7800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 0x7f60d0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d7800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0198d20 con 0x7f60d01004c0 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d7800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d019cab0 con 0x7f60d0100e60 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d4e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d4e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38540/0 (socket says 172.21.15.57:38540) 2024-02-28T23:25:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60d4e00700 1 -- 172.21.15.57:0/3557526925 learned_addr learned my addr 172.21.15.57:0/3557526925 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.581+0000 7f60cfe00700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 0x7f60d0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60d4e00700 1 -- 172.21.15.57:0/3557526925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 msgr2=0x7f60d0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60d4e00700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 0x7f60d0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60d4e00700 1 -- 172.21.15.57:0/3557526925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40095f0 con 0x7f60d01004c0 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60d4e00700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f60c40092e0 tx=0x7f60c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c401d070 con 0x7f60d01004c0 2024-02-28T23:25:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60c4005e00 con 0x7f60d01004c0 2024-02-28T23:25:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c4017400 con 0x7f60d01004c0 2024-02-28T23:25:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.582+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d019cd30 con 0x7f60d01004c0 2024-02-28T23:25:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.583+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d019d220 con 0x7f60d01004c0 2024-02-28T23:25:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.584+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d01045b0 con 0x7f60d01004c0 2024-02-28T23:25:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.585+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f60c400f460 con 0x7f60d01004c0 2024-02-28T23:25:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.586+0000 7f60cd600700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.586+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f60c408ea30 con 0x7f60d01004c0 2024-02-28T23:25:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.586+0000 7f60cfe00700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.587+0000 7f60cfe00700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005950 tx=0x7f60bc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.592+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60c40581c0 con 0x7f60d01004c0 2024-02-28T23:25:12.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.842+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d019d490 con 0x7f60b806d8a0 2024-02-28T23:25:12.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.846+0000 7f60cd600700 1 -- 172.21.15.57:0/3557526925 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f60d019d490 con 0x7f60b806d8a0 2024-02-28T23:25:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 msgr2=0x7f60b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005950 tx=0x7f60bc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 msgr2=0x7f60d0198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f60c40092e0 tx=0x7f60c400be20 comp rx=0 tx=0).stop 2024-02-28T23:25:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 shutdown_connections 2024-02-28T23:25:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d01004c0 0x7f60d0198100 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 --2- 172.21.15.57:0/3557526925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0100e60 0x7f60d0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:12.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.849+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 >> 172.21.15.57:0/3557526925 conn(0x7f60d00fbd80 msgr2=0x7f60d00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:12.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.850+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 shutdown_connections 2024-02-28T23:25:12.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:12.850+0000 7f60d7800700 1 -- 172.21.15.57:0/3557526925 wait complete. 2024-02-28T23:25:12.864 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 -- 172.21.15.57:0/2294689760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 msgr2=0x7f6de41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 --2- 172.21.15.57:0/2294689760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41028e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009910 tx=0x7f6dd0009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 -- 172.21.15.57:0/2294689760 shutdown_connections 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 --2- 172.21.15.57:0/2294689760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41028e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 --2- 172.21.15.57:0/2294689760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 0x7f6de4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 -- 172.21.15.57:0/2294689760 >> 172.21.15.57:0/2294689760 conn(0x7f6de40fdf90 msgr2=0x7f6de41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 -- 172.21.15.57:0/2294689760 shutdown_connections 2024-02-28T23:25:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.004+0000 7f6dec600700 1 -- 172.21.15.57:0/2294689760 wait complete. 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.005+0000 7f6dec600700 1 Processor -- start 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.005+0000 7f6dec600700 1 -- start start 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6dec600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6dec600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 0x7f6de4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6dec600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4198d80 con 0x7f6de4102470 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6dec600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4198ec0 con 0x7f6de4108470 2024-02-28T23:25:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6de9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6de9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38552/0 (socket says 172.21.15.57:38552) 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6de9c00700 1 -- 172.21.15.57:0/3920277005 learned_addr learned my addr 172.21.15.57:0/3920277005 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.006+0000 7f6de9200700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 0x7f6de4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de9c00700 1 -- 172.21.15.57:0/3920277005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 msgr2=0x7f6de4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de9c00700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 0x7f6de4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de9c00700 1 -- 172.21.15.57:0/3920277005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00095f0 con 0x7f6de4102470 2024-02-28T23:25:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de9c00700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c930 tx=0x7f6dd800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd8007ad0 con 0x7f6de4102470 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd8007c30 con 0x7f6de4102470 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd8018610 con 0x7f6de4102470 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de419cd10 con 0x7f6de4102470 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.007+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de419d210 con 0x7f6de4102470 2024-02-28T23:25:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.008+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de410a970 con 0x7f6de4102470 2024-02-28T23:25:13.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.009+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6dd800f450 con 0x7f6de4102470 2024-02-28T23:25:13.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.010+0000 7f6de2000700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.010+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6dd808d810 con 0x7f6de4102470 2024-02-28T23:25:13.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.011+0000 7f6de9200700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.011+0000 7f6de9200700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6de4199810 tx=0x7f6dd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.015+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dd8056f20 con 0x7f6de4102470 2024-02-28T23:25:13.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.265+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6de4199660 con 0x7f6dcc06d8a0 2024-02-28T23:25:13.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.268+0000 7f6de2000700 1 -- 172.21.15.57:0/3920277005 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6de4199660 con 0x7f6dcc06d8a0 2024-02-28T23:25:13.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 msgr2=0x7f6dcc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6de4199810 tx=0x7f6dd001a040 comp rx=0 tx=0).stop 2024-02-28T23:25:13.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 msgr2=0x7f6de41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c930 tx=0x7f6dd800ccf0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 shutdown_connections 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de4102470 0x7f6de41981f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 --2- 172.21.15.57:0/3920277005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6de4108470 0x7f6de4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 >> 172.21.15.57:0/3920277005 conn(0x7f6de40fdf90 msgr2=0x7f6de40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 shutdown_connections 2024-02-28T23:25:13.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.271+0000 7f6dec600700 1 -- 172.21.15.57:0/3920277005 wait complete. 2024-02-28T23:25:13.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 -- 172.21.15.57:0/1798763465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 msgr2=0x7f9f2c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 --2- 172.21.15.57:0/1798763465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c10ccc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009910 tx=0x7f9f1c009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 -- 172.21.15.57:0/1798763465 shutdown_connections 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 --2- 172.21.15.57:0/1798763465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 --2- 172.21.15.57:0/1798763465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c10ccc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 -- 172.21.15.57:0/1798763465 >> 172.21.15.57:0/1798763465 conn(0x7f9f2c076060 msgr2=0x7f9f2c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 -- 172.21.15.57:0/1798763465 shutdown_connections 2024-02-28T23:25:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.433+0000 7f9f33000700 1 -- 172.21.15.57:0/1798763465 wait complete. 2024-02-28T23:25:13.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.434+0000 7f9f33000700 1 Processor -- start 2024-02-28T23:25:13.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f33000700 1 -- start start 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f33000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f33000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f33000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19d120 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f33000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c1a0eb0 con 0x7f9f2c106b80 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.435+0000 7f9f23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38570/0 (socket says 172.21.15.57:38570) 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 -- 172.21.15.57:0/407806171 learned_addr learned my addr 172.21.15.57:0/407806171 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f2be00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 -- 172.21.15.57:0/407806171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 msgr2=0x7f9f2c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 -- 172.21.15.57:0/407806171 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0095f0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f2be00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.436+0000 7f9f23e00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d8d0 tx=0x7f9f1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.437+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14009940 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.437+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c1a1190 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.437+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f14010460 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.437+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1400f5f0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.437+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c1a16e0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.438+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c109b20 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.439+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9f140105d0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.440+0000 7f9f29600700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 0x7f9f1006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.440+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9f1408d5d0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.441+0000 7f9f2be00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 0x7f9f1006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.441+0000 7f9f2be00700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 0x7f9f1006fd50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005b40 tx=0x7f9f1c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.446+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f14056ce0 con 0x7f9f2c10c8f0 2024-02-28T23:25:13.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.675+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f2c1a1900 con 0x7f9f1006d8a0 2024-02-28T23:25:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.710+0000 7f9f29600700 1 -- 172.21.15.57:0/407806171 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9f2c1a1900 con 0x7f9f1006d8a0 2024-02-28T23:25:13.709 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:25:13.709 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (14m) 6m ago 28m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:25:13.709 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (25m) 6m ago 29m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:25:13.709 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (27m) 5m ago 27m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (10m) 6m ago 28m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (10m) 5m ago 27m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (14m) 6m ago 28m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (6m) 6m ago 21m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (6m) 6m ago 21m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (21m) 5m ago 21m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (5m) 5m ago 21m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (17m) 6m ago 30m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (16m) 5m ago 27m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (12m) 6m ago 30m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (11m) 5m ago 27m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (15m) 6m ago 28m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (15m) 5m ago 27m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:25:13.710 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (10m) 6m ago 26m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (9m) 6m ago 25m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (9m) 6m ago 24m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (8m) 5m ago 24m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (7m) 5m ago 23m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (7m) 5m ago 23m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (15m) 6m ago 28m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:25:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.712+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 msgr2=0x7f9f1006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.712+0000 7f9f33000700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 0x7f9f1006fd50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005b40 tx=0x7f9f1c01a040 comp rx=0 tx=0).stop 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.712+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 msgr2=0x7f9f2c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.712+0000 7f9f33000700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d8d0 tx=0x7f9f1400dc90 comp rx=0 tx=0).stop 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 shutdown_connections 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f1006d8a0 0x7f9f1006fd50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f2c106b80 0x7f9f2c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 --2- 172.21.15.57:0/407806171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f2c10c8f0 0x7f9f2c19ca40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 >> 172.21.15.57:0/407806171 conn(0x7f9f2c076060 msgr2=0x7f9f2c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 shutdown_connections 2024-02-28T23:25:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.713+0000 7f9f33000700 1 -- 172.21.15.57:0/407806171 wait complete. 2024-02-28T23:25:13.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 -- 172.21.15.57:0/3071762340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 msgr2=0x7fa180106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 --2- 172.21.15.57:0/3071762340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa180106ff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009910 tx=0x7fa16c009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 -- 172.21.15.57:0/3071762340 shutdown_connections 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 --2- 172.21.15.57:0/3071762340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa180106ff0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 --2- 172.21.15.57:0/3071762340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 -- 172.21.15.57:0/3071762340 >> 172.21.15.57:0/3071762340 conn(0x7fa180076060 msgr2=0x7fa180078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 -- 172.21.15.57:0/3071762340 shutdown_connections 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.871+0000 7fa186200700 1 -- 172.21.15.57:0/3071762340 wait complete. 2024-02-28T23:25:13.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.872+0000 7fa186200700 1 Processor -- start 2024-02-28T23:25:13.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.872+0000 7fa186200700 1 -- start start 2024-02-28T23:25:13.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa186200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa186200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa186200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019d100 con 0x7fa180106b80 2024-02-28T23:25:13.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa186200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a0e90 con 0x7fa18010c8f0 2024-02-28T23:25:13.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa17f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa17f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38590/0 (socket says 172.21.15.57:38590) 2024-02-28T23:25:13.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.873+0000 7fa17f400700 1 -- 172.21.15.57:0/667031319 learned_addr learned my addr 172.21.15.57:0/667031319 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17ea00700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17f400700 1 -- 172.21.15.57:0/667031319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 msgr2=0x7fa18019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17f400700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17f400700 1 -- 172.21.15.57:0/667031319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0095f0 con 0x7fa180106b80 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17ea00700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa17f400700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b4b0 tx=0x7fa17000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa170011800 con 0x7fa180106b80 2024-02-28T23:25:13.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa170011e20 con 0x7fa180106b80 2024-02-28T23:25:13.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17000f2e0 con 0x7fa180106b80 2024-02-28T23:25:13.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.874+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801a1170 con 0x7fa180106b80 2024-02-28T23:25:13.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.875+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801a16c0 con 0x7fa180106b80 2024-02-28T23:25:13.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.876+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180109b20 con 0x7fa180106b80 2024-02-28T23:25:13.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.877+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa170010380 con 0x7fa180106b80 2024-02-28T23:25:13.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.878+0000 7fa177e00700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 0x7fa16406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:13.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.878+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa17008d180 con 0x7fa180106b80 2024-02-28T23:25:13.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.878+0000 7fa17ea00700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 0x7fa16406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:13.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.879+0000 7fa17ea00700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 0x7fa16406fd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006010 tx=0x7fa16c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:13.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:13.883+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa170056890 con 0x7fa180106b80 2024-02-28T23:25:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:13 smithi057 ceph-mon[89491]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:13 smithi057 ceph-mon[89491]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:13 smithi084 ceph-mon[70440]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:14.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:13 smithi084 ceph-mon[70440]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.202+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1801a1940 con 0x7fa180106b80 2024-02-28T23:25:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.203+0000 7fa177e00700 1 -- 172.21.15.57:0/667031319 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa170059f50 con 0x7fa180106b80 2024-02-28T23:25:14.202 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:25:14.202 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:25:14.202 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:25:14.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:25:14.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:25:14.204 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:25:14.205 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:25:14.205 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:25:14.205 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:25:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 msgr2=0x7fa16406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 0x7fa16406fd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006010 tx=0x7fa16c00b540 comp rx=0 tx=0).stop 2024-02-28T23:25:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 msgr2=0x7fa18019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b4b0 tx=0x7fa17000b870 comp rx=0 tx=0).stop 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 shutdown_connections 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa16406d8a0 0x7fa16406fd50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa180106b80 0x7fa18019c4e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 --2- 172.21.15.57:0/667031319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18010c8f0 0x7fa18019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 >> 172.21.15.57:0/667031319 conn(0x7fa180076060 msgr2=0x7fa180077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.206+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 shutdown_connections 2024-02-28T23:25:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.207+0000 7fa186200700 1 -- 172.21.15.57:0/667031319 wait complete. 2024-02-28T23:25:14.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 -- 172.21.15.57:0/1555613140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff340687a0 msgr2=0x7eff34068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1555613140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff340687a0 0x7eff34068b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7eff28009910 tx=0x7eff28009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 -- 172.21.15.57:0/1555613140 shutdown_connections 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1555613140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34069140 0x7eff34104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1555613140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff340687a0 0x7eff34068b70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 -- 172.21.15.57:0/1555613140 >> 172.21.15.57:0/1555613140 conn(0x7eff34075130 msgr2=0x7eff34075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.371+0000 7eff3ba00700 1 -- 172.21.15.57:0/1555613140 shutdown_connections 2024-02-28T23:25:14.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.372+0000 7eff3ba00700 1 -- 172.21.15.57:0/1555613140 wait complete. 2024-02-28T23:25:14.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.373+0000 7eff3ba00700 1 Processor -- start 2024-02-28T23:25:14.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.373+0000 7eff3ba00700 1 -- start start 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff3ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff3ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34198770 0x7eff3419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff3ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34198cf0 con 0x7eff34069140 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff3ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34198e60 con 0x7eff34198770 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff39000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff39000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38596/0 (socket says 172.21.15.57:38596) 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.374+0000 7eff39000700 1 -- 172.21.15.57:0/1226836098 learned_addr learned my addr 172.21.15.57:0/1226836098 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff33e00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34198770 0x7eff3419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff39000700 1 -- 172.21.15.57:0/1226836098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34198770 msgr2=0x7eff3419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff39000700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34198770 0x7eff3419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff39000700 1 -- 172.21.15.57:0/1226836098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280095f0 con 0x7eff34069140 2024-02-28T23:25:14.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff39000700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7eff34077bc0 tx=0x7eff2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:14.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2801d070 con 0x7eff34069140 2024-02-28T23:25:14.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.375+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3419d180 con 0x7eff34069140 2024-02-28T23:25:14.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.376+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff2800f460 con 0x7eff34069140 2024-02-28T23:25:14.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.376+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff28017630 con 0x7eff34069140 2024-02-28T23:25:14.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.376+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3419d680 con 0x7eff34069140 2024-02-28T23:25:14.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.377+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3404f2a0 con 0x7eff34069140 2024-02-28T23:25:14.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.378+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7eff2800fab0 con 0x7eff34069140 2024-02-28T23:25:14.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.379+0000 7eff31600700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 0x7eff1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.379+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7eff2808e9c0 con 0x7eff34069140 2024-02-28T23:25:14.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.379+0000 7eff33e00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 0x7eff1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.380+0000 7eff33e00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 0x7eff1c06fd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ba60 tx=0x7eff20005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:14.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.384+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff280580d0 con 0x7eff34069140 2024-02-28T23:25:14.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.660+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7eff3419d8a0 con 0x7eff34069140 2024-02-28T23:25:14.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.661+0000 7eff31600700 1 -- 172.21.15.57:0/1226836098 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7eff2805b790 con 0x7eff34069140 2024-02-28T23:25:14.660 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:25:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:25:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:25:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:14.664 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:25:14.665 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:14.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 msgr2=0x7eff1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 0x7eff1c06fd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ba60 tx=0x7eff20005d50 comp rx=0 tx=0).stop 2024-02-28T23:25:14.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 msgr2=0x7eff34198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7eff34077bc0 tx=0x7eff2800be20 comp rx=0 tx=0).stop 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 shutdown_connections 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7eff1c06d8a0 0x7eff1c06fd50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff34069140 0x7eff34198230 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.664+0000 7eff3ba00700 1 --2- 172.21.15.57:0/1226836098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff34198770 0x7eff3419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.665+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 >> 172.21.15.57:0/1226836098 conn(0x7eff34075130 msgr2=0x7eff341015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.665+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 shutdown_connections 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.665+0000 7eff3ba00700 1 -- 172.21.15.57:0/1226836098 wait complete. 2024-02-28T23:25:14.666 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:25:14.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 -- 172.21.15.57:0/3807848692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004108470 msgr2=0x7f7004108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 --2- 172.21.15.57:0/3807848692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004108470 0x7f7004108840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009910 tx=0x7f6ff8009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:14.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 -- 172.21.15.57:0/3807848692 shutdown_connections 2024-02-28T23:25:14.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 --2- 172.21.15.57:0/3807848692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004102470 0x7f70041028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 --2- 172.21.15.57:0/3807848692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004108470 0x7f7004108840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 -- 172.21.15.57:0/3807848692 >> 172.21.15.57:0/3807848692 conn(0x7f70040fdf90 msgr2=0x7f70041003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:14.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 -- 172.21.15.57:0/3807848692 shutdown_connections 2024-02-28T23:25:14.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.830+0000 7f700b800700 1 -- 172.21.15.57:0/3807848692 wait complete. 2024-02-28T23:25:14.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.831+0000 7f700b800700 1 Processor -- start 2024-02-28T23:25:14.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.831+0000 7f700b800700 1 -- start start 2024-02-28T23:25:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f700b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f700b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 0x7f7004198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f700b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004198e10 con 0x7f7004102470 2024-02-28T23:25:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f700b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700419cba0 con 0x7f7004108470 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f7008e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f7008e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38604/0 (socket says 172.21.15.57:38604) 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f7008e00700 1 -- 172.21.15.57:0/2987174185 learned_addr learned my addr 172.21.15.57:0/2987174185 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.832+0000 7f7003e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 0x7f7004198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7008e00700 1 -- 172.21.15.57:0/2987174185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 msgr2=0x7f7004198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7008e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 0x7f7004198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7008e00700 1 -- 172.21.15.57:0/2987174185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80095f0 con 0x7f7004102470 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7003e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 0x7f7004198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7008e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80092e0 tx=0x7f6ff800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff801d070 con 0x7f7004102470 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff8005e00 con 0x7f7004102470 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff8017400 con 0x7f7004102470 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700419ce20 con 0x7f7004102470 2024-02-28T23:25:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.833+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700419d310 con 0x7f7004102470 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.834+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700410a9e0 con 0x7f7004102470 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.835+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6ff800f460 con 0x7f7004102470 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.836+0000 7f7001600700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 0x7f6fec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.836+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6ff808ea30 con 0x7f7004102470 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.837+0000 7f7003e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 0x7f6fec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.837+0000 7f7003e00700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 0x7f6fec06fd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005fd0 tx=0x7f6ff0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:14.841+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ff80581c0 con 0x7f7004102470 2024-02-28T23:25:15.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:14 smithi057 ceph-mon[89491]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:14 smithi057 ceph-mon[89491]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:14 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/667031319' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:25:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:14 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1226836098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:25:15.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.081+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7004199660 con 0x7f6fec06d8a0 2024-02-28T23:25:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:14 smithi084 ceph-mon[70440]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:14 smithi084 ceph-mon[70440]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:14 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/667031319' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:25:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:14 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1226836098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:25:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.083+0000 7f7001600700 1 -- 172.21.15.57:0/2987174185 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7004199660 con 0x7f6fec06d8a0 2024-02-28T23:25:15.082 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:25:15.082 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:25:15.083 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:25:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:25:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:25:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:25:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:25:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:25:15.085 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:25:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.086+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 msgr2=0x7f6fec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.086+0000 7f700b800700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 0x7f6fec06fd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005fd0 tx=0x7f6ff0005ee0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.086+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 msgr2=0x7f70041981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.086+0000 7f700b800700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80092e0 tx=0x7f6ff800be20 comp rx=0 tx=0).stop 2024-02-28T23:25:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.086+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 shutdown_connections 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6fec06d8a0 0x7f6fec06fd50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7004102470 0x7f70041981f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 --2- 172.21.15.57:0/2987174185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7004108470 0x7f7004198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 >> 172.21.15.57:0/2987174185 conn(0x7f70040fdf90 msgr2=0x7f70041002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 shutdown_connections 2024-02-28T23:25:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.087+0000 7f700b800700 1 -- 172.21.15.57:0/2987174185 wait complete. 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.236+0000 7f3a5aa00700 1 -- 172.21.15.57:0/410175730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 msgr2=0x7f3a54106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.236+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/410175730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54106e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009910 tx=0x7f3a40009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 -- 172.21.15.57:0/410175730 shutdown_connections 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/410175730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54106e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/410175730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 -- 172.21.15.57:0/410175730 >> 172.21.15.57:0/410175730 conn(0x7f3a540fdd90 msgr2=0x7f3a541001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 -- 172.21.15.57:0/410175730 shutdown_connections 2024-02-28T23:25:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.237+0000 7f3a5aa00700 1 -- 172.21.15.57:0/410175730 wait complete. 2024-02-28T23:25:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 Processor -- start 2024-02-28T23:25:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 -- start start 2024-02-28T23:25:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a54197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541989a0 con 0x7f3a54102e70 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.238+0000 7f3a5aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54198ae0 con 0x7f3a541024d0 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42334/0 (socket says 172.21.15.57:42334) 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 -- 172.21.15.57:0/4010699806 learned_addr learned my addr 172.21.15.57:0/4010699806 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a59600700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a54197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 -- 172.21.15.57:0/4010699806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 msgr2=0x7f3a54197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a54197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a58c00700 1 -- 172.21.15.57:0/4010699806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400095f0 con 0x7f3a54102e70 2024-02-28T23:25:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.239+0000 7f3a59600700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a54197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a58c00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000bda0 tx=0x7f3a4000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4001d070 con 0x7f3a54102e70 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a40004fb0 con 0x7f3a54102e70 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5419c8d0 con 0x7f3a54102e70 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4000f480 con 0x7f3a54102e70 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.240+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5419cdc0 con 0x7f3a54102e70 2024-02-28T23:25:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.241+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5410a840 con 0x7f3a54102e70 2024-02-28T23:25:15.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.243+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3a40022870 con 0x7f3a54102e70 2024-02-28T23:25:15.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.244+0000 7f3a52000700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 0x7f3a3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:15.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.244+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a4008ea30 con 0x7f3a54102e70 2024-02-28T23:25:15.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.244+0000 7f3a59600700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 0x7f3a3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:15.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.245+0000 7f3a59600700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 0x7f3a3c06fd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005fd0 tx=0x7f3a48005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:15.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.249+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a40058270 con 0x7f3a54102e70 2024-02-28T23:25:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.563+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3a54062380 con 0x7f3a54102e70 2024-02-28T23:25:15.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.564+0000 7f3a52000700 1 -- 172.21.15.57:0/4010699806 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3a4005b930 con 0x7f3a54102e70 2024-02-28T23:25:15.562 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:25:15.562 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:25:15.562 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:25:15.563 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:25:15.563 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:25:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 msgr2=0x7f3a3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 0x7f3a3c06fd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005fd0 tx=0x7f3a48005e30 comp rx=0 tx=0).stop 2024-02-28T23:25:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 msgr2=0x7f3a54198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000bda0 tx=0x7f3a4000be80 comp rx=0 tx=0).stop 2024-02-28T23:25:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 shutdown_connections 2024-02-28T23:25:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a3c06d8a0 0x7f3a3c06fd50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a541024d0 0x7f3a54197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 --2- 172.21.15.57:0/4010699806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a54102e70 0x7f3a54198350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:15.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 >> 172.21.15.57:0/4010699806 conn(0x7f3a540fdd90 msgr2=0x7f3a54100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:15.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 shutdown_connections 2024-02-28T23:25:15.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:15.567+0000 7f3a5aa00700 1 -- 172.21.15.57:0/4010699806 wait complete. 2024-02-28T23:25:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:15 smithi057 ceph-mon[89491]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:15.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:15 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4010699806' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:25:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:15 smithi084 ceph-mon[70440]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:15 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4010699806' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:25:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:16 smithi057 ceph-mon[89491]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:16 smithi084 ceph-mon[70440]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:17 smithi057 ceph-mon[89491]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:17 smithi084 ceph-mon[70440]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:19 smithi057 ceph-mon[89491]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:19 smithi084 ceph-mon[70440]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:21 smithi057 ceph-mon[89491]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:21 smithi084 ceph-mon[70440]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:23 smithi057 ceph-mon[89491]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:23 smithi084 ceph-mon[70440]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:25 smithi057 ceph-mon[89491]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:26.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:25 smithi084 ceph-mon[70440]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:27 smithi057 ceph-mon[89491]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:27 smithi084 ceph-mon[70440]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:29 smithi084 ceph-mon[70440]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:30.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:29 smithi057 ceph-mon[89491]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:32.301 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:31 smithi084 ceph-mon[70440]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:31 smithi057 ceph-mon[89491]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:33 smithi057 ceph-mon[89491]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:33 smithi084 ceph-mon[70440]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:35 smithi057 ceph-mon[89491]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:36.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:35 smithi084 ceph-mon[70440]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:37 smithi057 ceph-mon[89491]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:38.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:37 smithi084 ceph-mon[70440]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:39 smithi057 ceph-mon[89491]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:40.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:39 smithi084 ceph-mon[70440]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:42.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:41 smithi057 ceph-mon[89491]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:42.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:41 smithi084 ceph-mon[70440]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:44.121 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:43 smithi057 ceph-mon[89491]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:44.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:43 smithi084 ceph-mon[70440]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:45.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.741+0000 7ffab2400700 1 -- 172.21.15.57:0/2780989759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 msgr2=0x7ffaac0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:45.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.741+0000 7ffab2400700 1 --2- 172.21.15.57:0/2780989759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac0ff720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009910 tx=0x7ffa9c009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:45.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.741+0000 7ffab2400700 1 -- 172.21.15.57:0/2780989759 shutdown_connections 2024-02-28T23:25:45.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.741+0000 7ffab2400700 1 --2- 172.21.15.57:0/2780989759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 0x7ffaac10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:45.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.742+0000 7ffab2400700 1 --2- 172.21.15.57:0/2780989759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac0ff720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:45.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.742+0000 7ffab2400700 1 -- 172.21.15.57:0/2780989759 >> 172.21.15.57:0/2780989759 conn(0x7ffaac0748b0 msgr2=0x7ffaac074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:45.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.742+0000 7ffab2400700 1 -- 172.21.15.57:0/2780989759 shutdown_connections 2024-02-28T23:25:45.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.742+0000 7ffab2400700 1 -- 172.21.15.57:0/2780989759 wait complete. 2024-02-28T23:25:45.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 Processor -- start 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 -- start start 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 0x7ffaac198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac198c60 con 0x7ffaac0ff350 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.743+0000 7ffab2400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19c9f0 con 0x7ffaac0ffc60 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47732/0 (socket says 172.21.15.57:47732) 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 -- 172.21.15.57:0/1151487125 learned_addr learned my addr 172.21.15.57:0/1151487125 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaaaa00700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 0x7ffaac198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 -- 172.21.15.57:0/1151487125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 msgr2=0x7ffaac198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:45.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 0x7ffaac198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:45.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.744+0000 7ffaab400700 1 -- 172.21.15.57:0/1151487125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0095f0 con 0x7ffaac0ff350 2024-02-28T23:25:45.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffaab400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bd70 tx=0x7ffa9c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:45.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c01d070 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa9c005dc0 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac19cc70 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c017400 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.745+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac19d160 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.746+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac10a230 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.747+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ffa9c00f7f0 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.748+0000 7ffaa3e00700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 0x7ffa9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.748+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffa9c08ea30 con 0x7ffaac0ff350 2024-02-28T23:25:45.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.749+0000 7ffaaaa00700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 0x7ffa9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:45.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.749+0000 7ffaaaa00700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 0x7ffa9006fd50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005950 tx=0x7ffa980058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:45.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:45.753+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa9c0581c0 con 0x7ffaac0ff350 2024-02-28T23:25:46.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.002+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaac19d490 con 0x7ffa9006d8a0 2024-02-28T23:25:46.002 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:45 smithi057 ceph-mon[89491]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:46.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.004+0000 7ffaa3e00700 1 -- 172.21.15.57:0/1151487125 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffaac19d490 con 0x7ffa9006d8a0 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.006+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 msgr2=0x7ffa9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.006+0000 7ffab2400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 0x7ffa9006fd50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005950 tx=0x7ffa980058e0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.006+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 msgr2=0x7ffaac198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.006+0000 7ffab2400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bd70 tx=0x7ffa9c00be50 comp rx=0 tx=0).stop 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 shutdown_connections 2024-02-28T23:25:46.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffa9006d8a0 0x7ffa9006fd50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaac0ff350 0x7ffaac198040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 --2- 172.21.15.57:0/1151487125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaac0ffc60 0x7ffaac198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 >> 172.21.15.57:0/1151487125 conn(0x7ffaac0748b0 msgr2=0x7ffaac100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 shutdown_connections 2024-02-28T23:25:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.007+0000 7ffab2400700 1 -- 172.21.15.57:0/1151487125 wait complete. 2024-02-28T23:25:46.021 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.155+0000 7f4816600700 1 -- 172.21.15.57:0/1677972072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 msgr2=0x7f4810102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.155+0000 7f4816600700 1 --2- 172.21.15.57:0/1677972072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810102980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4800009910 tx=0x7f4800009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.155+0000 7f4816600700 1 -- 172.21.15.57:0/1677972072 shutdown_connections 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.155+0000 7f4816600700 1 --2- 172.21.15.57:0/1677972072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4810102f50 0x7f4810106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.155+0000 7f4816600700 1 --2- 172.21.15.57:0/1677972072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810102980 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.156+0000 7f4816600700 1 -- 172.21.15.57:0/1677972072 >> 172.21.15.57:0/1677972072 conn(0x7f48100fde90 msgr2=0x7f48101002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:46.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.156+0000 7f4816600700 1 -- 172.21.15.57:0/1677972072 shutdown_connections 2024-02-28T23:25:46.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.156+0000 7f4816600700 1 -- 172.21.15.57:0/1677972072 wait complete. 2024-02-28T23:25:46.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 Processor -- start 2024-02-28T23:25:46.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 -- start start 2024-02-28T23:25:46.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4810102f50 0x7f4810198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4810198b20 con 0x7f48101025b0 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f4816600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481019c8b0 con 0x7f4810102f50 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f480f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f480f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47760/0 (socket says 172.21.15.57:47760) 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.157+0000 7f480f400700 1 -- 172.21.15.57:0/3407233739 learned_addr learned my addr 172.21.15.57:0/3407233739 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f480f400700 1 -- 172.21.15.57:0/3407233739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4810102f50 msgr2=0x7f4810198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f480f400700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4810102f50 0x7f4810198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f480f400700 1 -- 172.21.15.57:0/3407233739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000095f0 con 0x7f48101025b0 2024-02-28T23:25:46.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f480f400700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f48000092e0 tx=0x7f480000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480001d070 con 0x7f48101025b0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4800005e00 con 0x7f48101025b0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481019cb30 con 0x7f48101025b0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.158+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4800017400 con 0x7f48101025b0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.159+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481019d020 con 0x7f48101025b0 2024-02-28T23:25:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.160+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481010a8c0 con 0x7f48101025b0 2024-02-28T23:25:46.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.161+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f480000f460 con 0x7f48101025b0 2024-02-28T23:25:46.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.161+0000 7f4807e00700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 0x7f47f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.162+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f480008ea30 con 0x7f48101025b0 2024-02-28T23:25:46.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.162+0000 7f480ea00700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 0x7f47f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:46.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.163+0000 7f480ea00700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 0x7f47f406fd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47fc005950 tx=0x7f47fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:46.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.166+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48000581c0 con 0x7f48101025b0 2024-02-28T23:25:46.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:45 smithi084 ceph-mon[70440]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:46.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.417+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f481002cd10 con 0x7f47f406d8a0 2024-02-28T23:25:46.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.420+0000 7f4807e00700 1 -- 172.21.15.57:0/3407233739 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f481002cd10 con 0x7f47f406d8a0 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 msgr2=0x7f47f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 0x7f47f406fd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47fc005950 tx=0x7f47fc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 msgr2=0x7f4810197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f48000092e0 tx=0x7f480000be20 comp rx=0 tx=0).stop 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 shutdown_connections 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f47f406d8a0 0x7f47f406fd50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48101025b0 0x7f4810197f00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 --2- 172.21.15.57:0/3407233739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4810102f50 0x7f4810198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 >> 172.21.15.57:0/3407233739 conn(0x7f48100fde90 msgr2=0x7f4810100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:46.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.423+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 shutdown_connections 2024-02-28T23:25:46.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.424+0000 7f4816600700 1 -- 172.21.15.57:0/3407233739 wait complete. 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 -- 172.21.15.57:0/1712529916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc0687a0 msgr2=0x7f2afc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 --2- 172.21.15.57:0/1712529916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc0687a0 0x7f2afc068b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009910 tx=0x7f2af0009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 -- 172.21.15.57:0/1712529916 shutdown_connections 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 --2- 172.21.15.57:0/1712529916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 0x7f2afc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 --2- 172.21.15.57:0/1712529916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc0687a0 0x7f2afc068b70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 -- 172.21.15.57:0/1712529916 >> 172.21.15.57:0/1712529916 conn(0x7f2afc075130 msgr2=0x7f2afc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:46.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 -- 172.21.15.57:0/1712529916 shutdown_connections 2024-02-28T23:25:46.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.590+0000 7f2b03400700 1 -- 172.21.15.57:0/1712529916 wait complete. 2024-02-28T23:25:46.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.591+0000 7f2b03400700 1 Processor -- start 2024-02-28T23:25:46.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.591+0000 7f2b03400700 1 -- start start 2024-02-28T23:25:46.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2b03400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 0x7f2afc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2b03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2b03400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc198e40 con 0x7f2afc198830 2024-02-28T23:25:46.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2b03400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc198fb0 con 0x7f2afc069140 2024-02-28T23:25:46.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2afbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:46.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2afbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47774/0 (socket says 172.21.15.57:47774) 2024-02-28T23:25:46.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2afbe00700 1 -- 172.21.15.57:0/3443798315 learned_addr learned my addr 172.21.15.57:0/3443798315 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:46.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.592+0000 7f2b00a00700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 0x7f2afc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2afbe00700 1 -- 172.21.15.57:0/3443798315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 msgr2=0x7f2afc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2afbe00700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 0x7f2afc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2afbe00700 1 -- 172.21.15.57:0/3443798315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00095f0 con 0x7f2afc198830 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2afbe00700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d930 tx=0x7f2ae800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ae8009900 con 0x7f2afc198830 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ae8010460 con 0x7f2afc198830 2024-02-28T23:25:46.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc19d2a0 con 0x7f2afc198830 2024-02-28T23:25:46.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ae800f5f0 con 0x7f2afc198830 2024-02-28T23:25:46.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.593+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc19d7f0 con 0x7f2afc198830 2024-02-28T23:25:46.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.595+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04f2a0 con 0x7f2afc198830 2024-02-28T23:25:46.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.596+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2ae80105d0 con 0x7f2afc198830 2024-02-28T23:25:46.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.596+0000 7f2af9600700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 0x7f2ae406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:46.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.597+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2ae808d170 con 0x7f2afc198830 2024-02-28T23:25:46.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.597+0000 7f2b00a00700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 0x7f2ae406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:46.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.598+0000 7f2b00a00700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 0x7f2ae406fd50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005b40 tx=0x7f2af001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:46.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.601+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ae8056880 con 0x7f2afc198830 2024-02-28T23:25:46.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.845+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2afc102630 con 0x7f2ae406d8a0 2024-02-28T23:25:46.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.886+0000 7f2af9600700 1 -- 172.21.15.57:0/3443798315 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2afc102630 con 0x7f2ae406d8a0 2024-02-28T23:25:46.886 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:25:46.886 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (15m) 6m ago 29m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:25:46.886 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (25m) 6m ago 29m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:25:46.886 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (28m) 6m ago 28m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:25:46.886 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (11m) 6m ago 29m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (11m) 6m ago 28m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (14m) 6m ago 28m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (7m) 6m ago 22m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (6m) 6m ago 21m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (21m) 6m ago 21m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (6m) 6m ago 21m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (17m) 6m ago 31m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (16m) 6m ago 27m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (13m) 6m ago 31m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (12m) 6m ago 27m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (16m) 6m ago 29m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (16m) 6m ago 27m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (10m) 6m ago 26m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:25:46.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (10m) 6m ago 26m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (9m) 6m ago 25m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (9m) 6m ago 24m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (8m) 6m ago 24m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (7m) 6m ago 23m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (15m) 6m ago 28m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:25:46.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 msgr2=0x7f2ae406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 0x7f2ae406fd50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005b40 tx=0x7f2af001a040 comp rx=0 tx=0).stop 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 msgr2=0x7f2afc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d930 tx=0x7f2ae800dc40 comp rx=0 tx=0).stop 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 shutdown_connections 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2ae406d8a0 0x7f2ae406fd50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc069140 0x7f2afc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 --2- 172.21.15.57:0/3443798315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2afc198830 0x7f2afc19cca0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 >> 172.21.15.57:0/3443798315 conn(0x7f2afc075130 msgr2=0x7f2afc101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.889+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 shutdown_connections 2024-02-28T23:25:46.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:46.890+0000 7f2b03400700 1 -- 172.21.15.57:0/3443798315 wait complete. 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.037+0000 7fdef4600700 1 -- 172.21.15.57:0/3498834071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec0739a0 msgr2=0x7fdeec10c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.037+0000 7fdef4600700 1 --2- 172.21.15.57:0/3498834071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec0739a0 0x7fdeec10c5e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fded8009940 tx=0x7fded8009c50 comp rx=0 tx=0).stop 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 -- 172.21.15.57:0/3498834071 shutdown_connections 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 --2- 172.21.15.57:0/3498834071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec0739a0 0x7fdeec10c5e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 --2- 172.21.15.57:0/3498834071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec073090 0x7fdeec073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 -- 172.21.15.57:0/3498834071 >> 172.21.15.57:0/3498834071 conn(0x7fdeec0fbda0 msgr2=0x7fdeec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:47.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 -- 172.21.15.57:0/3498834071 shutdown_connections 2024-02-28T23:25:47.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.038+0000 7fdef4600700 1 -- 172.21.15.57:0/3498834071 wait complete. 2024-02-28T23:25:47.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.039+0000 7fdef4600700 1 Processor -- start 2024-02-28T23:25:47.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.039+0000 7fdef4600700 1 -- start start 2024-02-28T23:25:47.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef4600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef4600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec0739a0 0x7fdeec198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef4600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec198e40 con 0x7fdeec073090 2024-02-28T23:25:47.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef4600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec19cbd0 con 0x7fdeec0739a0 2024-02-28T23:25:47.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef1c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef1c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47800/0 (socket says 172.21.15.57:47800) 2024-02-28T23:25:47.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef1c00700 1 -- 172.21.15.57:0/457964248 learned_addr learned my addr 172.21.15.57:0/457964248 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:47.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.040+0000 7fdef1200700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec0739a0 0x7fdeec198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef1c00700 1 -- 172.21.15.57:0/457964248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec0739a0 msgr2=0x7fdeec198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef1c00700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec0739a0 0x7fdeec198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef1c00700 1 -- 172.21.15.57:0/457964248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80095f0 con 0x7fdeec073090 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef1c00700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdee000d8d0 tx=0x7fdee000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee0009940 con 0x7fdeec073090 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdee0010460 con 0x7fdeec073090 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee000f5f0 con 0x7fdeec073090 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec19ceb0 con 0x7fdeec073090 2024-02-28T23:25:47.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.041+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec19d400 con 0x7fdeec073090 2024-02-28T23:25:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.043+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec109ce0 con 0x7fdeec073090 2024-02-28T23:25:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.044+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdee00105d0 con 0x7fdeec073090 2024-02-28T23:25:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.045+0000 7fdeea000700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.045+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdee008d4c0 con 0x7fdeec073090 2024-02-28T23:25:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.045+0000 7fdef1200700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.046+0000 7fdef1200700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fded8005b40 tx=0x7fded8005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.049+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdee0056c50 con 0x7fdeec073090 2024-02-28T23:25:47.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.357+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdeec199650 con 0x7fdeec073090 2024-02-28T23:25:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.358+0000 7fdeea000700 1 -- 172.21.15.57:0/457964248 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fdee005a310 con 0x7fdeec073090 2024-02-28T23:25:47.357 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:25:47.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:25:47.359 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:25:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.361+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 msgr2=0x7fded406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.361+0000 7fdef4600700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fded8005b40 tx=0x7fded8005a90 comp rx=0 tx=0).stop 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.361+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 msgr2=0x7fdeec198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.361+0000 7fdef4600700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdee000d8d0 tx=0x7fdee000dc90 comp rx=0 tx=0).stop 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 shutdown_connections 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeec073090 0x7fdeec198220 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 --2- 172.21.15.57:0/457964248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdeec0739a0 0x7fdeec198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 >> 172.21.15.57:0/457964248 conn(0x7fdeec0fbda0 msgr2=0x7fdeec106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 shutdown_connections 2024-02-28T23:25:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.362+0000 7fdef4600700 1 -- 172.21.15.57:0/457964248 wait complete. 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 -- 172.21.15.57:0/2978332027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec1024d0 msgr2=0x7f49ec1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 --2- 172.21.15.57:0/2978332027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec1024d0 0x7f49ec1028a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009910 tx=0x7f49dc009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 -- 172.21.15.57:0/2978332027 shutdown_connections 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 --2- 172.21.15.57:0/2978332027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec102e70 0x7f49ec106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 --2- 172.21.15.57:0/2978332027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec1024d0 0x7f49ec1028a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.522+0000 7f49ebe00700 1 -- 172.21.15.57:0/2978332027 >> 172.21.15.57:0/2978332027 conn(0x7f49ec0fdd90 msgr2=0x7f49ec1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.523+0000 7f49ebe00700 1 -- 172.21.15.57:0/2978332027 shutdown_connections 2024-02-28T23:25:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.523+0000 7f49ebe00700 1 -- 172.21.15.57:0/2978332027 wait complete. 2024-02-28T23:25:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.524+0000 7f49ebe00700 1 Processor -- start 2024-02-28T23:25:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.524+0000 7f49ebe00700 1 -- start start 2024-02-28T23:25:47.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec1024d0 0x7f49ec197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ebe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec198a80 con 0x7f49ec102e70 2024-02-28T23:25:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ebe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec19c810 con 0x7f49ec1024d0 2024-02-28T23:25:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ea000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ea000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47812/0 (socket says 172.21.15.57:47812) 2024-02-28T23:25:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.525+0000 7f49ea000700 1 -- 172.21.15.57:0/3446694376 learned_addr learned my addr 172.21.15.57:0/3446694376 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.526+0000 7f49eaa00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec1024d0 0x7f49ec197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.526+0000 7f49ea000700 1 -- 172.21.15.57:0/3446694376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec1024d0 msgr2=0x7f49ec197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.526+0000 7f49ea000700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec1024d0 0x7f49ec197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.526+0000 7f49ea000700 1 -- 172.21.15.57:0/3446694376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0095f0 con 0x7f49ec102e70 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.526+0000 7f49ea000700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f49d800d8d0 tx=0x7f49d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.527+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d80041f0 con 0x7f49ec102e70 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.527+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49d8007620 con 0x7f49ec102e70 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.527+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec19caf0 con 0x7f49ec102e70 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.527+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d8003c50 con 0x7f49ec102e70 2024-02-28T23:25:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.527+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec19d040 con 0x7f49ec102e70 2024-02-28T23:25:47.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.528+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec10a7e0 con 0x7f49ec102e70 2024-02-28T23:25:47.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.529+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f49d8004350 con 0x7f49ec102e70 2024-02-28T23:25:47.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.530+0000 7f49f0a00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 0x7f49d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.530+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f49d808d150 con 0x7f49ec102e70 2024-02-28T23:25:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.530+0000 7f49eaa00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 0x7f49d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.531+0000 7f49eaa00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 0x7f49d006fd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006010 tx=0x7f49dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.535+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49d8056860 con 0x7f49ec102e70 2024-02-28T23:25:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.812+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49ec062380 con 0x7f49ec102e70 2024-02-28T23:25:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.813+0000 7f49f0a00700 1 -- 172.21.15.57:0/3446694376 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f49d8003e80 con 0x7f49ec102e70 2024-02-28T23:25:47.813 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:25:47.813 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:25:47.814 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:25:47.815 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:25:47.816 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 msgr2=0x7f49d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 0x7f49d006fd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006010 tx=0x7f49dc01a040 comp rx=0 tx=0).stop 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 msgr2=0x7f49ec1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f49d800d8d0 tx=0x7f49d800dc90 comp rx=0 tx=0).stop 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 shutdown_connections 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49d006d8a0 0x7f49d006fd50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49ec1024d0 0x7f49ec197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 --2- 172.21.15.57:0/3446694376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49ec102e70 0x7f49ec1983a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 >> 172.21.15.57:0/3446694376 conn(0x7f49ec0fdd90 msgr2=0x7f49ec100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 shutdown_connections 2024-02-28T23:25:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.817+0000 7f49ebe00700 1 -- 172.21.15.57:0/3446694376 wait complete. 2024-02-28T23:25:47.818 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 -- 172.21.15.57:0/1934123088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 msgr2=0x7f2da0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 --2- 172.21.15.57:0/1934123088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0073480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009940 tx=0x7f2d88009c50 comp rx=0 tx=0).stop 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 -- 172.21.15.57:0/1934123088 shutdown_connections 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 --2- 172.21.15.57:0/1934123088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 --2- 172.21.15.57:0/1934123088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0073480 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.967+0000 7f2da7c00700 1 -- 172.21.15.57:0/1934123088 >> 172.21.15.57:0/1934123088 conn(0x7f2da00fbda0 msgr2=0x7f2da00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.968+0000 7f2da7c00700 1 -- 172.21.15.57:0/1934123088 shutdown_connections 2024-02-28T23:25:47.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.968+0000 7f2da7c00700 1 -- 172.21.15.57:0/1934123088 wait complete. 2024-02-28T23:25:47.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 Processor -- start 2024-02-28T23:25:47.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 -- start start 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0198e30 con 0x7f2da00730b0 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.969+0000 7f2da7c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da019cbc0 con 0x7f2da00739c0 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47840/0 (socket says 172.21.15.57:47840) 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 -- 172.21.15.57:0/3830800205 learned_addr learned my addr 172.21.15.57:0/3830800205 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 -- 172.21.15.57:0/3830800205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 msgr2=0x7f2da0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2d97e00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 -- 172.21.15.57:0/3830800205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880095f0 con 0x7f2da00730b0 2024-02-28T23:25:47.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2d97e00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.970+0000 7f2da5200700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d880055f0 tx=0x7f2d880056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.971+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8801d070 con 0x7f2da00730b0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.971+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d8800f460 con 0x7f2da00730b0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.971+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d88021640 con 0x7f2da00730b0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.971+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da019ce40 con 0x7f2da00730b0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.971+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da019d330 con 0x7f2da00730b0 2024-02-28T23:25:47.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.972+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da0109cc0 con 0x7f2da00730b0 2024-02-28T23:25:47.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.973+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2d8800fa60 con 0x7f2da00730b0 2024-02-28T23:25:47.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.974+0000 7f2d9e000700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 0x7f2d8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:47.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.974+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d8808e730 con 0x7f2da00730b0 2024-02-28T23:25:47.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.974+0000 7f2d97e00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 0x7f2d8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:47.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.975+0000 7f2d97e00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 0x7f2d8006fd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005fd0 tx=0x7f2d90005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:47.979+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d88057ec0 con 0x7f2da00730b0 2024-02-28T23:25:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/457964248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:25:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:47 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3446694376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/457964248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:25:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:47 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3446694376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:25:48.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.221+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2da01996b0 con 0x7f2d8006d8a0 2024-02-28T23:25:48.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.224+0000 7f2d9e000700 1 -- 172.21.15.57:0/3830800205 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d900098a0 con 0x7f2d8006d8a0 2024-02-28T23:25:48.223 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:25:48.223 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:25:48.223 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:25:48.224 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:25:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.226+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 msgr2=0x7f2d8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.226+0000 7f2da7c00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 0x7f2d8006fd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005fd0 tx=0x7f2d90005ee0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 msgr2=0x7f2da0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d880055f0 tx=0x7f2d880056d0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 shutdown_connections 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d8a0 0x7f2d8006fd50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2da00730b0 0x7f2da0198210 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 --2- 172.21.15.57:0/3830800205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00739c0 0x7f2da0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 >> 172.21.15.57:0/3830800205 conn(0x7f2da00fbda0 msgr2=0x7f2da0106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 shutdown_connections 2024-02-28T23:25:48.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.227+0000 7f2da7c00700 1 -- 172.21.15.57:0/3830800205 wait complete. 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 -- 172.21.15.57:0/3971134785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e041026a0 msgr2=0x7f0e04102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 --2- 172.21.15.57:0/3971134785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e041026a0 0x7f0e04102a70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009910 tx=0x7f0df4009c20 comp rx=0 tx=0).stop 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 -- 172.21.15.57:0/3971134785 shutdown_connections 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 --2- 172.21.15.57:0/3971134785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 0x7f0e04107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 --2- 172.21.15.57:0/3971134785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e041026a0 0x7f0e04102a70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.377+0000 7f0e09200700 1 -- 172.21.15.57:0/3971134785 >> 172.21.15.57:0/3971134785 conn(0x7f0e040fdf40 msgr2=0x7f0e04100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.378+0000 7f0e09200700 1 -- 172.21.15.57:0/3971134785 shutdown_connections 2024-02-28T23:25:48.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.378+0000 7f0e09200700 1 -- 172.21.15.57:0/3971134785 wait complete. 2024-02-28T23:25:48.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 Processor -- start 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 -- start start 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 0x7f0e041982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04198da0 con 0x7f0e04198820 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e09200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04198f10 con 0x7f0e04103040 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e01400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e01400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47868/0 (socket says 172.21.15.57:47868) 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.379+0000 7f0e01400700 1 -- 172.21.15.57:0/929888427 learned_addr learned my addr 172.21.15.57:0/929888427 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:25:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0e01e00700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 0x7f0e041982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0e01400700 1 -- 172.21.15.57:0/929888427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 msgr2=0x7f0e041982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0e01400700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 0x7f0e041982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0e01400700 1 -- 172.21.15.57:0/929888427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40095f0 con 0x7f0e04198820 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0e01400700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c960 tx=0x7f0df000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.380+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df0007a70 con 0x7f0e04198820 2024-02-28T23:25:48.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.381+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0df0007bd0 con 0x7f0e04198820 2024-02-28T23:25:48.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.381+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0419d230 con 0x7f0e04198820 2024-02-28T23:25:48.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.381+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df00186c0 con 0x7f0e04198820 2024-02-28T23:25:48.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.381+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0419d730 con 0x7f0e04198820 2024-02-28T23:25:48.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.382+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0410a9b0 con 0x7f0e04198820 2024-02-28T23:25:48.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.383+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0df000faa0 con 0x7f0e04198820 2024-02-28T23:25:48.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.384+0000 7f0dfaa00700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 0x7f0de806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:25:48.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.384+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0df008d640 con 0x7f0e04198820 2024-02-28T23:25:48.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.384+0000 7f0e01e00700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 0x7f0de806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:25:48.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.385+0000 7f0e01e00700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 0x7f0de806fd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009fd0 tx=0x7f0df401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:25:48.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.389+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0df005a880 con 0x7f0e04198820 2024-02-28T23:25:48.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.691+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0e0404f2a0 con 0x7f0e04198820 2024-02-28T23:25:48.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.692+0000 7f0dfaa00700 1 -- 172.21.15.57:0/929888427 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f0df005a410 con 0x7f0e04198820 2024-02-28T23:25:48.691 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:25:48.691 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:25:48.691 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:25:48.691 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:25:48.692 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:25:48.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.694+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 msgr2=0x7f0de806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.694+0000 7f0e09200700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 0x7f0de806fd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009fd0 tx=0x7f0df401a040 comp rx=0 tx=0).stop 2024-02-28T23:25:48.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.694+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 msgr2=0x7f0e0419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:25:48.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c960 tx=0x7f0df000cd20 comp rx=0 tx=0).stop 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 shutdown_connections 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0de806d8a0 0x7f0de806fd50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e04103040 0x7f0e041982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 --2- 172.21.15.57:0/929888427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e04198820 0x7f0e0419cc90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 >> 172.21.15.57:0/929888427 conn(0x7f0e040fdf40 msgr2=0x7f0e040ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 shutdown_connections 2024-02-28T23:25:48.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:25:48.695+0000 7f0e09200700 1 -- 172.21.15.57:0/929888427 wait complete. 2024-02-28T23:25:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:48 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/929888427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:25:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:25:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:48 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/929888427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:25:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:49 smithi057 ceph-mon[89491]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:49 smithi057 ceph-mon[89491]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:49 smithi084 ceph-mon[70440]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:25:50.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:49 smithi084 ceph-mon[70440]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:50 smithi057 ceph-mon[89491]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:50 smithi084 ceph-mon[70440]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:53.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:53 smithi084 ceph-mon[70440]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:53 smithi057 ceph-mon[89491]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:55.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:55 smithi084 ceph-mon[70440]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:55 smithi057 ceph-mon[89491]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:57 smithi057 ceph-mon[89491]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:25:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:57 smithi084 ceph-mon[70440]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:25:59 smithi057 ceph-mon[89491]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:25:59 smithi084 ceph-mon[70440]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:01.897 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:01 smithi084 ceph-mon[70440]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:01 smithi057 ceph-mon[89491]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:03 smithi057 ceph-mon[89491]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:03 smithi084 ceph-mon[70440]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:05 smithi057 ceph-mon[89491]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:06.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:05 smithi084 ceph-mon[70440]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:07.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:07 smithi057 ceph-mon[89491]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:07 smithi084 ceph-mon[70440]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:09 smithi057 ceph-mon[89491]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:09 smithi084 ceph-mon[70440]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:12.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:11 smithi057 ceph-mon[89491]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:11 smithi084 ceph-mon[70440]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:13.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:13 smithi084 ceph-mon[70440]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:13 smithi057 ceph-mon[89491]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:15 smithi057 ceph-mon[89491]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:15 smithi084 ceph-mon[70440]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:17 smithi057 ceph-mon[89491]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:17 smithi084 ceph-mon[70440]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 -- 172.21.15.57:0/2296105042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8106b80 msgr2=0x7f8ad8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 --2- 172.21.15.57:0/2296105042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8106b80 0x7f8ad8106ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009910 tx=0x7f8ac4009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 -- 172.21.15.57:0/2296105042 shutdown_connections 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 --2- 172.21.15.57:0/2296105042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8106b80 0x7f8ad8106ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 --2- 172.21.15.57:0/2296105042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad810c660 0x7f8ad810ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 -- 172.21.15.57:0/2296105042 >> 172.21.15.57:0/2296105042 conn(0x7f8ad8075e90 msgr2=0x7f8ad80782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:18.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 -- 172.21.15.57:0/2296105042 shutdown_connections 2024-02-28T23:26:18.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.873+0000 7f8adf000700 1 -- 172.21.15.57:0/2296105042 wait complete. 2024-02-28T23:26:18.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.875+0000 7f8adf000700 1 Processor -- start 2024-02-28T23:26:18.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.875+0000 7f8adf000700 1 -- start start 2024-02-28T23:26:18.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8adf000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 0x7f8ad819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:18.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8adf000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:18.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8adf000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819cd70 con 0x7f8ad810c660 2024-02-28T23:26:18.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8adf000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819ceb0 con 0x7f8ad8106b80 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8add200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8add200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33614/0 (socket says 172.21.15.57:33614) 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.876+0000 7f8add200700 1 -- 172.21.15.57:0/2521259986 learned_addr learned my addr 172.21.15.57:0/2521259986 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8addc00700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 0x7f8ad819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8add200700 1 -- 172.21.15.57:0/2521259986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 msgr2=0x7f8ad819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8add200700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 0x7f8ad819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8add200700 1 -- 172.21.15.57:0/2521259986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40095f0 con 0x7f8ad810c660 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8addc00700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 0x7f8ad819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:26:18.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8add200700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400b5c0 tx=0x7f8ac400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:18.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac401d070 con 0x7f8ad810c660 2024-02-28T23:26:18.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.877+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac400f460 con 0x7f8ad810c660 2024-02-28T23:26:18.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.878+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad81a0ca0 con 0x7f8ad810c660 2024-02-28T23:26:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.878+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac4017610 con 0x7f8ad810c660 2024-02-28T23:26:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.878+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a1190 con 0x7f8ad810c660 2024-02-28T23:26:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.879+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad810eb40 con 0x7f8ad810c660 2024-02-28T23:26:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.880+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ac400f5d0 con 0x7f8ad810c660 2024-02-28T23:26:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.881+0000 7f8ad6000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 0x7f8ac006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.881+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ac408eb50 con 0x7f8ad810c660 2024-02-28T23:26:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.881+0000 7f8addc00700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 0x7f8ac006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.882+0000 7f8addc00700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 0x7f8ac006fd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8acc005fd0 tx=0x7f8acc005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:18.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:18.886+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ac40582e0 con 0x7f8ad810c660 2024-02-28T23:26:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:19.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.141+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ad802cd10 con 0x7f8ac006d8a0 2024-02-28T23:26:19.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.143+0000 7f8ad6000700 1 -- 172.21.15.57:0/2521259986 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8ad802cd10 con 0x7f8ac006d8a0 2024-02-28T23:26:19.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 msgr2=0x7f8ac006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 0x7f8ac006fd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8acc005fd0 tx=0x7f8acc005e20 comp rx=0 tx=0).stop 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 msgr2=0x7f8ad819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400b5c0 tx=0x7f8ac400be00 comp rx=0 tx=0).stop 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 shutdown_connections 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ac006d8a0 0x7f8ac006fd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8106b80 0x7f8ad819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 --2- 172.21.15.57:0/2521259986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad810c660 0x7f8ad819c720 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 >> 172.21.15.57:0/2521259986 conn(0x7f8ad8075e90 msgr2=0x7f8ad80778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 shutdown_connections 2024-02-28T23:26:19.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.146+0000 7f8adf000700 1 -- 172.21.15.57:0/2521259986 wait complete. 2024-02-28T23:26:19.162 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:26:19.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 -- 172.21.15.57:0/1590994092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c068540 msgr2=0x7f159c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 --2- 172.21.15.57:0/1590994092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c068540 0x7f159c0689b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1584009910 tx=0x7f1584009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 -- 172.21.15.57:0/1590994092 shutdown_connections 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 --2- 172.21.15.57:0/1590994092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c068540 0x7f159c0689b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 --2- 172.21.15.57:0/1590994092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c1061b0 0x7f159c106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 -- 172.21.15.57:0/1590994092 >> 172.21.15.57:0/1590994092 conn(0x7f159c074fa0 msgr2=0x7f159c0753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 -- 172.21.15.57:0/1590994092 shutdown_connections 2024-02-28T23:26:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.312+0000 7f159b200700 1 -- 172.21.15.57:0/1590994092 wait complete. 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 Processor -- start 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 -- start start 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c068540 0x7f159c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c19c490 con 0x7f159c1061b0 2024-02-28T23:26:19.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.313+0000 7f159b200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1959d0 con 0x7f159c068540 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33630/0 (socket says 172.21.15.57:33630) 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 -- 172.21.15.57:0/4166120679 learned_addr learned my addr 172.21.15.57:0/4166120679 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1599e00700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c068540 0x7f159c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 -- 172.21.15.57:0/4166120679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c068540 msgr2=0x7f159c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c068540 0x7f159c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.314+0000 7f1591400700 1 -- 172.21.15.57:0/4166120679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840095f0 con 0x7f159c1061b0 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f1591400700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f158400b5c0 tx=0x7f158400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158401d070 con 0x7f159c1061b0 2024-02-28T23:26:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f158400f460 con 0x7f159c1061b0 2024-02-28T23:26:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c195c50 con 0x7f159c1061b0 2024-02-28T23:26:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1584017610 con 0x7f159c1061b0 2024-02-28T23:26:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.315+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c1961c0 con 0x7f159c1061b0 2024-02-28T23:26:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.316+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c062380 con 0x7f159c1061b0 2024-02-28T23:26:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.317+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f158400f5d0 con 0x7f159c1061b0 2024-02-28T23:26:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.318+0000 7f1593400700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 0x7f158006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.318+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f158408ebe0 con 0x7f159c1061b0 2024-02-28T23:26:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.319+0000 7f1599e00700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 0x7f158006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.319+0000 7f1599e00700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 0x7f158006fd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f158c005ea0 tx=0x7f158c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:19.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.323+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f158405be20 con 0x7f159c1061b0 2024-02-28T23:26:19.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.575+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f159c196450 con 0x7f158006d8a0 2024-02-28T23:26:19.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.577+0000 7f1593400700 1 -- 172.21.15.57:0/4166120679 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f159c196450 con 0x7f158006d8a0 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 msgr2=0x7f158006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 0x7f158006fd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f158c005ea0 tx=0x7f158c005e30 comp rx=0 tx=0).stop 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 msgr2=0x7f159c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f158400b5c0 tx=0x7f158400be00 comp rx=0 tx=0).stop 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 shutdown_connections 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f158006d8a0 0x7f158006fd50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f159c068540 0x7f159c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 --2- 172.21.15.57:0/4166120679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f159c1061b0 0x7f159c19be90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 >> 172.21.15.57:0/4166120679 conn(0x7f159c074fa0 msgr2=0x7f159c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:19.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.581+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 shutdown_connections 2024-02-28T23:26:19.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.582+0000 7f159b200700 1 -- 172.21.15.57:0/4166120679 wait complete. 2024-02-28T23:26:19.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 -- 172.21.15.57:0/3802495258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 msgr2=0x7f8ebc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 --2- 172.21.15.57:0/3802495258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc104e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009910 tx=0x7f8ea8009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 -- 172.21.15.57:0/3802495258 shutdown_connections 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 --2- 172.21.15.57:0/3802495258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc104e50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 --2- 172.21.15.57:0/3802495258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 -- 172.21.15.57:0/3802495258 >> 172.21.15.57:0/3802495258 conn(0x7f8ebc0fbda0 msgr2=0x7f8ebc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 -- 172.21.15.57:0/3802495258 shutdown_connections 2024-02-28T23:26:19.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.748+0000 7f8ec1200700 1 -- 172.21.15.57:0/3802495258 wait complete. 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.749+0000 7f8ec1200700 1 Processor -- start 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.749+0000 7f8ec1200700 1 -- start start 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8ec1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc196b50 con 0x7f8ebc100e80 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8ec1200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19a8e0 con 0x7f8ebc1004e0 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8eb9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8eb9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33654/0 (socket says 172.21.15.57:33654) 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.750+0000 7f8eb9400700 1 -- 172.21.15.57:0/2694055719 learned_addr learned my addr 172.21.15.57:0/2694055719 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9e00700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9400700 1 -- 172.21.15.57:0/2694055719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 msgr2=0x7f8ebc195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9400700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9400700 1 -- 172.21.15.57:0/2694055719 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80095f0 con 0x7f8ebc100e80 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9e00700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb9400700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800bda0 tx=0x7f8ea800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea801d070 con 0x7f8ebc100e80 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ea800f460 con 0x7f8ebc100e80 2024-02-28T23:26:19.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc19ab60 con 0x7f8ebc100e80 2024-02-28T23:26:19.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea8017630 con 0x7f8ebc100e80 2024-02-28T23:26:19.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.751+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc19b050 con 0x7f8ebc100e80 2024-02-28T23:26:19.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.753+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc108850 con 0x7f8ebc100e80 2024-02-28T23:26:19.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.754+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ea800fa40 con 0x7f8ebc100e80 2024-02-28T23:26:19.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.754+0000 7f8eb2a00700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 0x7f8ea006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:19.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.755+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ea808ea70 con 0x7f8ebc100e80 2024-02-28T23:26:19.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.755+0000 7f8eb9e00700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 0x7f8ea006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:19.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.756+0000 7f8eb9e00700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 0x7f8ea006fd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005fd0 tx=0x7f8eac005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:19.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:19.759+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ea8058180 con 0x7f8ebc100e80 2024-02-28T23:26:20.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.002+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ebc1973a0 con 0x7f8ea006d8a0 2024-02-28T23:26:20.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:19 smithi057 ceph-mon[89491]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:20.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.015+0000 7f8eb2a00700 1 -- 172.21.15.57:0/2694055719 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8ebc1973a0 con 0x7f8ea006d8a0 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (15m) 7m ago 29m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (26m) 7m ago 30m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (28m) 7m ago 28m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (12m) 7m ago 30m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (11m) 7m ago 28m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (15m) 7m ago 29m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:26:20.014 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (7m) 7m ago 22m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (7m) 7m ago 22m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (22m) 7m ago 22m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (7m) 7m ago 22m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (18m) 7m ago 31m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (17m) 7m ago 28m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (13m) 7m ago 31m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (13m) 7m ago 28m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (17m) 7m ago 29m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (16m) 7m ago 28m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:26:20.015 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (11m) 7m ago 27m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (10m) 7m ago 26m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (10m) 7m ago 26m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (9m) 7m ago 25m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (9m) 7m ago 24m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (8m) 7m ago 24m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:26:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (16m) 7m ago 29m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:26:20.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 msgr2=0x7f8ea006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 0x7f8ea006fd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005fd0 tx=0x7f8eac005ee0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 msgr2=0x7f8ebc196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800bda0 tx=0x7f8ea800be80 comp rx=0 tx=0).stop 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 shutdown_connections 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8ea006d8a0 0x7f8ea006fd50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ebc1004e0 0x7f8ebc195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 --2- 172.21.15.57:0/2694055719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ebc100e80 0x7f8ebc196470 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.018+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 >> 172.21.15.57:0/2694055719 conn(0x7f8ebc0fbda0 msgr2=0x7f8ebc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.019+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 shutdown_connections 2024-02-28T23:26:20.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.019+0000 7f8ec1200700 1 -- 172.21.15.57:0/2694055719 wait complete. 2024-02-28T23:26:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:19 smithi084 ceph-mon[70440]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 -- 172.21.15.57:0/2962169710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554102470 msgr2=0x7f55541028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 --2- 172.21.15.57:0/2962169710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554102470 0x7f55541028e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5540009910 tx=0x7f5540009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 -- 172.21.15.57:0/2962169710 shutdown_connections 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 --2- 172.21.15.57:0/2962169710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554102470 0x7f55541028e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 --2- 172.21.15.57:0/2962169710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554108470 0x7f5554108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 -- 172.21.15.57:0/2962169710 >> 172.21.15.57:0/2962169710 conn(0x7f55540fdf90 msgr2=0x7f55541003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 -- 172.21.15.57:0/2962169710 shutdown_connections 2024-02-28T23:26:20.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.182+0000 7f5559600700 1 -- 172.21.15.57:0/2962169710 wait complete. 2024-02-28T23:26:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.183+0000 7f5559600700 1 Processor -- start 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.183+0000 7f5559600700 1 -- start start 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5559600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 0x7f55541981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5559600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5559600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554198d80 con 0x7f5554108470 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5559600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554198ec0 con 0x7f5554102470 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33682/0 (socket says 172.21.15.57:33682) 2024-02-28T23:26:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 -- 172.21.15.57:0/606163435 learned_addr learned my addr 172.21.15.57:0/606163435 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552a00700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 0x7f55541981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 -- 172.21.15.57:0/606163435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 msgr2=0x7f55541981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 0x7f55541981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.184+0000 7f5552000700 1 -- 172.21.15.57:0/606163435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400095f0 con 0x7f5554108470 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f5552a00700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 0x7f55541981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f5552000700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f554000b5c0 tx=0x7f554000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554001d070 con 0x7f5554108470 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f554000f460 con 0x7f5554108470 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555419ccb0 con 0x7f5554108470 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5540017610 con 0x7f5554108470 2024-02-28T23:26:20.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.185+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555419d1a0 con 0x7f5554108470 2024-02-28T23:26:20.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.186+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555404f2a0 con 0x7f5554108470 2024-02-28T23:26:20.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.188+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f554000f5d0 con 0x7f5554108470 2024-02-28T23:26:20.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.188+0000 7f554b400700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 0x7f553806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.189+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f554008eb40 con 0x7f5554108470 2024-02-28T23:26:20.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.189+0000 7f5552a00700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 0x7f553806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.189+0000 7f5552a00700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 0x7f553806fd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f5544005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:20.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.194+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55400582f0 con 0x7f5554108470 2024-02-28T23:26:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.512+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f555410a350 con 0x7f5554108470 2024-02-28T23:26:20.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.514+0000 7f554b400700 1 -- 172.21.15.57:0/606163435 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f554005b9b0 con 0x7f5554108470 2024-02-28T23:26:20.512 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:26:20.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:26:20.514 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:26:20.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 msgr2=0x7f553806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 0x7f553806fd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f5544005e20 comp rx=0 tx=0).stop 2024-02-28T23:26:20.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 msgr2=0x7f5554198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f554000b5c0 tx=0x7f554000be00 comp rx=0 tx=0).stop 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 shutdown_connections 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f553806d8a0 0x7f553806fd50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5554102470 0x7f55541981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 --2- 172.21.15.57:0/606163435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5554108470 0x7f5554198730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.517+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 >> 172.21.15.57:0/606163435 conn(0x7f55540fdf90 msgr2=0x7f55540ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.518+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 shutdown_connections 2024-02-28T23:26:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.518+0000 7f5559600700 1 -- 172.21.15.57:0/606163435 wait complete. 2024-02-28T23:26:20.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 -- 172.21.15.57:0/2730228893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 msgr2=0x7fe8381008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 --2- 172.21.15.57:0/2730228893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe8381008b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009910 tx=0x7fe82c009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:20.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 -- 172.21.15.57:0/2730228893 shutdown_connections 2024-02-28T23:26:20.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 --2- 172.21.15.57:0/2730228893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 0x7fe838104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 --2- 172.21.15.57:0/2730228893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe8381008b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 -- 172.21.15.57:0/2730228893 >> 172.21.15.57:0/2730228893 conn(0x7fe8380fbda0 msgr2=0x7fe8380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:20.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.680+0000 7fe83f400700 1 -- 172.21.15.57:0/2730228893 shutdown_connections 2024-02-28T23:26:20.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.681+0000 7fe83f400700 1 -- 172.21.15.57:0/2730228893 wait complete. 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 Processor -- start 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 -- start start 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 0x7fe838073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838073ad0 con 0x7fe8381004e0 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.682+0000 7fe83f400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838073c10 con 0x7fe838100e80 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33696/0 (socket says 172.21.15.57:33696) 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 -- 172.21.15.57:0/1825883498 learned_addr learned my addr 172.21.15.57:0/1825883498 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe837e00700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 0x7fe838073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 -- 172.21.15.57:0/1825883498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 msgr2=0x7fe838073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 0x7fe838073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.683+0000 7fe83ca00700 1 -- 172.21.15.57:0/1825883498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0095f0 con 0x7fe8381004e0 2024-02-28T23:26:20.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe83ca00700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006010 tx=0x7fe82c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:20.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82c01d070 con 0x7fe8381004e0 2024-02-28T23:26:20.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82c004fb0 con 0x7fe8381004e0 2024-02-28T23:26:20.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe838073e90 con 0x7fe8381004e0 2024-02-28T23:26:20.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82c00f690 con 0x7fe8381004e0 2024-02-28T23:26:20.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.684+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8381a23f0 con 0x7fe8381004e0 2024-02-28T23:26:20.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.685+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8381087f0 con 0x7fe8381004e0 2024-02-28T23:26:20.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.686+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe82c0228a0 con 0x7fe8381004e0 2024-02-28T23:26:20.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.687+0000 7fe835600700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 0x7fe82006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:20.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.687+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe82c08ee60 con 0x7fe8381004e0 2024-02-28T23:26:20.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.688+0000 7fe837e00700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 0x7fe82006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:20.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.688+0000 7fe837e00700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 0x7fe82006fd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe824005fd0 tx=0x7fe82400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:20.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.692+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe82c058570 con 0x7fe8381004e0 2024-02-28T23:26:20.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:20 smithi057 ceph-mon[89491]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:20.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:20 smithi057 ceph-mon[89491]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:20.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:20 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/606163435' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:26:20.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.975+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe83804f2a0 con 0x7fe8381004e0 2024-02-28T23:26:20.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.976+0000 7fe835600700 1 -- 172.21.15.57:0/1825883498 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fe82c027070 con 0x7fe8381004e0 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:26:20.975 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:26:20.976 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:26:20.977 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:26:20.978 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:20.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 msgr2=0x7fe82006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 0x7fe82006fd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe824005fd0 tx=0x7fe82400a560 comp rx=0 tx=0).stop 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 msgr2=0x7fe838073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006010 tx=0x7fe82c00be20 comp rx=0 tx=0).stop 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 shutdown_connections 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe82006d8a0 0x7fe82006fd50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8381004e0 0x7fe838073050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 --2- 172.21.15.57:0/1825883498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe838100e80 0x7fe838073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.979+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 >> 172.21.15.57:0/1825883498 conn(0x7fe8380fbda0 msgr2=0x7fe8380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.980+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 shutdown_connections 2024-02-28T23:26:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:20.980+0000 7fe83f400700 1 -- 172.21.15.57:0/1825883498 wait complete. 2024-02-28T23:26:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:26:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:20 smithi084 ceph-mon[70440]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:20 smithi084 ceph-mon[70440]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:20 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/606163435' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.132+0000 7f274ee00700 1 -- 172.21.15.57:0/2601642239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 msgr2=0x7f274810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.132+0000 7f274ee00700 1 --2- 172.21.15.57:0/2601642239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f274810a230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2734009910 tx=0x7f2734009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 -- 172.21.15.57:0/2601642239 shutdown_connections 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 --2- 172.21.15.57:0/2601642239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f274810a230 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 --2- 172.21.15.57:0/2601642239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 0x7f27481017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 -- 172.21.15.57:0/2601642239 >> 172.21.15.57:0/2601642239 conn(0x7f27480fac80 msgr2=0x7f27480fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:21.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 -- 172.21.15.57:0/2601642239 shutdown_connections 2024-02-28T23:26:21.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.133+0000 7f274ee00700 1 -- 172.21.15.57:0/2601642239 wait complete. 2024-02-28T23:26:21.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 Processor -- start 2024-02-28T23:26:21.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 -- start start 2024-02-28T23:26:21.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 0x7f27480ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480ffb00 con 0x7f2748101d30 2024-02-28T23:26:21.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.134+0000 7f274ee00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480ffc40 con 0x7f2748101420 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33716/0 (socket says 172.21.15.57:33716) 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 -- 172.21.15.57:0/170012262 learned_addr learned my addr 172.21.15.57:0/170012262 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747e00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 0x7f27480ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 -- 172.21.15.57:0/170012262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 msgr2=0x7f27480ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 0x7f27480ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.135+0000 7f2747400700 1 -- 172.21.15.57:0/170012262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340095f0 con 0x7f2748101d30 2024-02-28T23:26:21.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f2747400700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2734006010 tx=0x7f273400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:21.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273401d070 con 0x7f2748101d30 2024-02-28T23:26:21.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2734004fb0 con 0x7f2748101d30 2024-02-28T23:26:21.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273400f690 con 0x7f2748101d30 2024-02-28T23:26:21.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27480ffec0 con 0x7f2748101d30 2024-02-28T23:26:21.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.136+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27481a2410 con 0x7f2748101d30 2024-02-28T23:26:21.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.137+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27481078a0 con 0x7f2748101d30 2024-02-28T23:26:21.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.139+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f27340228a0 con 0x7f2748101d30 2024-02-28T23:26:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.140+0000 7f2744c00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 0x7f272c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.140+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f273408ee60 con 0x7f2748101d30 2024-02-28T23:26:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.140+0000 7f2747e00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 0x7f272c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.141+0000 7f2747e00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 0x7f272c06fd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2738006fd0 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:21.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.145+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2734058570 con 0x7f2748101d30 2024-02-28T23:26:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.396+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2748061190 con 0x7f272c06d8a0 2024-02-28T23:26:21.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.397+0000 7f2744c00700 1 -- 172.21.15.57:0/170012262 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2748061190 con 0x7f272c06d8a0 2024-02-28T23:26:21.396 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:26:21.396 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:26:21.396 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:26:21.396 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:26:21.397 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:26:21.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.400+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 msgr2=0x7f272c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.400+0000 7f274ee00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 0x7f272c06fd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2738006fd0 tx=0x7f2738008040 comp rx=0 tx=0).stop 2024-02-28T23:26:21.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.400+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 msgr2=0x7f27480ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.400+0000 7f274ee00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2734006010 tx=0x7f273400be20 comp rx=0 tx=0).stop 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 shutdown_connections 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f272c06d8a0 0x7f272c06fd50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2748101420 0x7f27480ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 --2- 172.21.15.57:0/170012262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2748101d30 0x7f27480ff5c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 >> 172.21.15.57:0/170012262 conn(0x7f27480fac80 msgr2=0x7f27481049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 shutdown_connections 2024-02-28T23:26:21.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.401+0000 7f274ee00700 1 -- 172.21.15.57:0/170012262 wait complete. 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 -- 172.21.15.57:0/85270538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 msgr2=0x7fbc80073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 --2- 172.21.15.57:0/85270538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80073460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000b0d0 tx=0x7fbc7000b3e0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 -- 172.21.15.57:0/85270538 shutdown_connections 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 --2- 172.21.15.57:0/85270538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc8010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 --2- 172.21.15.57:0/85270538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80073460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 -- 172.21.15.57:0/85270538 >> 172.21.15.57:0/85270538 conn(0x7fbc800fbda0 msgr2=0x7fbc800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 -- 172.21.15.57:0/85270538 shutdown_connections 2024-02-28T23:26:21.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.559+0000 7fbc85600700 1 -- 172.21.15.57:0/85270538 wait complete. 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.560+0000 7fbc85600700 1 Processor -- start 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.560+0000 7fbc85600700 1 -- start start 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc85600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc85600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc80198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc85600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc80198e60 con 0x7fbc80073090 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc85600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8019cbf0 con 0x7fbc800739a0 2024-02-28T23:26:21.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33740/0 (socket says 172.21.15.57:33740) 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.561+0000 7fbc7ea00700 1 -- 172.21.15.57:0/1253332665 learned_addr learned my addr 172.21.15.57:0/1253332665 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7e000700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc80198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7ea00700 1 -- 172.21.15.57:0/1253332665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 msgr2=0x7fbc80198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7ea00700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc80198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7ea00700 1 -- 172.21.15.57:0/1253332665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70009d00 con 0x7fbc80073090 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7e000700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc80198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc7ea00700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007be0 tx=0x7fbc70007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc7000e040 con 0x7fbc80073090 2024-02-28T23:26:21.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8019ce70 con 0x7fbc80073090 2024-02-28T23:26:21.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.562+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc70003c10 con 0x7fbc80073090 2024-02-28T23:26:21.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.563+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8019d360 con 0x7fbc80073090 2024-02-28T23:26:21.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.563+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc700127c0 con 0x7fbc80073090 2024-02-28T23:26:21.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.564+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc80109ce0 con 0x7fbc80073090 2024-02-28T23:26:21.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.564+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbc70004710 con 0x7fbc80073090 2024-02-28T23:26:21.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.565+0000 7fbc77400700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 0x7fbc6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:21.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.565+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbc7008e9e0 con 0x7fbc80073090 2024-02-28T23:26:21.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.566+0000 7fbc7e000700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 0x7fbc6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:21.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.566+0000 7fbc7e000700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 0x7fbc6406fd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc80199860 tx=0x7fbc6c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:21.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.570+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc700580f0 con 0x7fbc80073090 2024-02-28T23:26:21.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.878+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbc80199690 con 0x7fbc80073090 2024-02-28T23:26:21.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.879+0000 7fbc77400700 1 -- 172.21.15.57:0/1253332665 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fbc7005b7b0 con 0x7fbc80073090 2024-02-28T23:26:21.877 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:26:21.877 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:26:21.877 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:26:21.878 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:26:21.878 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:26:21.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 msgr2=0x7fbc6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 0x7fbc6406fd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc80199860 tx=0x7fbc6c008040 comp rx=0 tx=0).stop 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 msgr2=0x7fbc80198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007be0 tx=0x7fbc70007cc0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 shutdown_connections 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbc6406d8a0 0x7fbc6406fd50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc80073090 0x7fbc80198240 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 --2- 172.21.15.57:0/1253332665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc800739a0 0x7fbc80198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 >> 172.21.15.57:0/1253332665 conn(0x7fbc800fbda0 msgr2=0x7fbc80106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 shutdown_connections 2024-02-28T23:26:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:21.882+0000 7fbc85600700 1 -- 172.21.15.57:0/1253332665 wait complete. 2024-02-28T23:26:22.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:21 smithi057 ceph-mon[89491]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:22.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:21 smithi057 ceph-mon[89491]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:22.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:21 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1825883498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:26:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:21 smithi084 ceph-mon[70440]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:21 smithi084 ceph-mon[70440]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:21 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1825883498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:26:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:22 smithi057 ceph-mon[89491]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:23.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1253332665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:26:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:22 smithi084 ceph-mon[70440]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1253332665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:26:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:23 smithi057 ceph-mon[89491]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:23 smithi084 ceph-mon[70440]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:25 smithi057 ceph-mon[89491]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:26.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:25 smithi084 ceph-mon[70440]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:27 smithi057 ceph-mon[89491]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:28.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:27 smithi084 ceph-mon[70440]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:29 smithi084 ceph-mon[70440]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:29 smithi057 ceph-mon[89491]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:31.840 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:31 smithi084 ceph-mon[70440]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:31 smithi057 ceph-mon[89491]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:33 smithi057 ceph-mon[89491]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:33 smithi084 ceph-mon[70440]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:35 smithi057 ceph-mon[89491]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:35 smithi084 ceph-mon[70440]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:37 smithi057 ceph-mon[89491]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:37 smithi084 ceph-mon[70440]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:39 smithi057 ceph-mon[89491]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:39 smithi084 ceph-mon[70440]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:41 smithi057 ceph-mon[89491]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:41 smithi084 ceph-mon[70440]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:43 smithi057 ceph-mon[89491]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:43 smithi084 ceph-mon[70440]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:45 smithi057 ceph-mon[89491]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:45 smithi084 ceph-mon[70440]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:47 smithi057 ceph-mon[89491]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:47 smithi084 ceph-mon[70440]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:26:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:49 smithi057 ceph-mon[89491]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:49 smithi084 ceph-mon[70440]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 -- 172.21.15.57:0/1103300157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b241024d0 msgr2=0x7f8b241028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 --2- 172.21.15.57:0/1103300157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b241024d0 0x7f8b241028a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009910 tx=0x7f8b18009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:52.057 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:51 smithi057 ceph-mon[89491]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 -- 172.21.15.57:0/1103300157 shutdown_connections 2024-02-28T23:26:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 --2- 172.21.15.57:0/1103300157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b24102e70 0x7f8b24106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 --2- 172.21.15.57:0/1103300157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b241024d0 0x7f8b241028a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 -- 172.21.15.57:0/1103300157 >> 172.21.15.57:0/1103300157 conn(0x7f8b240fdd90 msgr2=0x7f8b241001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 -- 172.21.15.57:0/1103300157 shutdown_connections 2024-02-28T23:26:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.057+0000 7f8b2a800700 1 -- 172.21.15.57:0/1103300157 wait complete. 2024-02-28T23:26:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.058+0000 7f8b2a800700 1 Processor -- start 2024-02-28T23:26:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.058+0000 7f8b2a800700 1 -- start start 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b2a800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 0x7f8b24197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b2a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b2a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24198a80 con 0x7f8b24102e70 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b2a800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2419c810 con 0x7f8b241024d0 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b28a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b28a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53260/0 (socket says 172.21.15.57:53260) 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b28a00700 1 -- 172.21.15.57:0/111159176 learned_addr learned my addr 172.21.15.57:0/111159176 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.059+0000 7f8b29400700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 0x7f8b24197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b28a00700 1 -- 172.21.15.57:0/111159176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 msgr2=0x7f8b24197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b28a00700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 0x7f8b24197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b28a00700 1 -- 172.21.15.57:0/111159176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180095f0 con 0x7f8b24102e70 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b29400700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 0x7f8b24197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b28a00700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d900 tx=0x7f8b1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b10009900 con 0x7f8b24102e70 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b10010460 con 0x7f8b24102e70 2024-02-28T23:26:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1000f5f0 con 0x7f8b24102e70 2024-02-28T23:26:52.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2419caf0 con 0x7f8b24102e70 2024-02-28T23:26:52.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.060+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2419cff0 con 0x7f8b24102e70 2024-02-28T23:26:52.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.062+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2410a7e0 con 0x7f8b24102e70 2024-02-28T23:26:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.063+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8b100105d0 con 0x7f8b24102e70 2024-02-28T23:26:52.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.064+0000 7f8b22000700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 0x7f8b0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.064+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8b1008d430 con 0x7f8b24102e70 2024-02-28T23:26:52.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.064+0000 7f8b29400700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 0x7f8b0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.065+0000 7f8b29400700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 0x7f8b0c06fd50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b18005b40 tx=0x7f8b1801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.069+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b10056b40 con 0x7f8b24102e70 2024-02-28T23:26:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:51 smithi084 ceph-mon[70440]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:52.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.320+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b2419d2d0 con 0x7f8b0c06d8a0 2024-02-28T23:26:52.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.322+0000 7f8b22000700 1 -- 172.21.15.57:0/111159176 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8b2419d2d0 con 0x7f8b0c06d8a0 2024-02-28T23:26:52.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 msgr2=0x7f8b0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 0x7f8b0c06fd50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b18005b40 tx=0x7f8b1801a040 comp rx=0 tx=0).stop 2024-02-28T23:26:52.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 msgr2=0x7f8b241983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d900 tx=0x7f8b1000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 shutdown_connections 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b0c06d8a0 0x7f8b0c06fd50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b241024d0 0x7f8b24197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 --2- 172.21.15.57:0/111159176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24102e70 0x7f8b241983a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 >> 172.21.15.57:0/111159176 conn(0x7f8b240fdd90 msgr2=0x7f8b24100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 shutdown_connections 2024-02-28T23:26:52.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.326+0000 7f8b2a800700 1 -- 172.21.15.57:0/111159176 wait complete. 2024-02-28T23:26:52.342 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.492+0000 7f55bc800700 1 -- 172.21.15.57:0/3601660292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 msgr2=0x7f55b41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.492+0000 7f55bc800700 1 --2- 172.21.15.57:0/3601660292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b41008b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009910 tx=0x7f55a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 -- 172.21.15.57:0/3601660292 shutdown_connections 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 --2- 172.21.15.57:0/3601660292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 0x7f55b4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 --2- 172.21.15.57:0/3601660292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b41008b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 -- 172.21.15.57:0/3601660292 >> 172.21.15.57:0/3601660292 conn(0x7f55b40fbda0 msgr2=0x7f55b40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 -- 172.21.15.57:0/3601660292 shutdown_connections 2024-02-28T23:26:52.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.493+0000 7f55bc800700 1 -- 172.21.15.57:0/3601660292 wait complete. 2024-02-28T23:26:52.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 Processor -- start 2024-02-28T23:26:52.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 -- start start 2024-02-28T23:26:52.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 0x7f55b4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4073ad0 con 0x7f55b41004e0 2024-02-28T23:26:52.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.494+0000 7f55bc800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4073c10 con 0x7f55b4100e80 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53284/0 (socket says 172.21.15.57:53284) 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 -- 172.21.15.57:0/133270507 learned_addr learned my addr 172.21.15.57:0/133270507 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9400700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 0x7f55b4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 -- 172.21.15.57:0/133270507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 msgr2=0x7f55b4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 0x7f55b4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.495+0000 7f55b9e00700 1 -- 172.21.15.57:0/133270507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80095f0 con 0x7f55b41004e0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55b9e00700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55a80092e0 tx=0x7f55a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a801d070 con 0x7f55b41004e0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55a8005e00 con 0x7f55b41004e0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4073e90 con 0x7f55b41004e0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a8017400 con 0x7f55b41004e0 2024-02-28T23:26:52.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.496+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b41a23f0 con 0x7f55b41004e0 2024-02-28T23:26:52.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.497+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b41087f0 con 0x7f55b41004e0 2024-02-28T23:26:52.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.498+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f55a800f460 con 0x7f55b41004e0 2024-02-28T23:26:52.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.499+0000 7f55aea00700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 0x7f559c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.499+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f55a808ead0 con 0x7f55b41004e0 2024-02-28T23:26:52.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.500+0000 7f55b9400700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 0x7f559c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.500+0000 7f55b9400700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 0x7f559c06fd50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005950 tx=0x7f55a40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.504+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55a80581e0 con 0x7f55b41004e0 2024-02-28T23:26:52.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.757+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55b4074670 con 0x7f559c06d8a0 2024-02-28T23:26:52.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.760+0000 7f55aea00700 1 -- 172.21.15.57:0/133270507 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f55b4074670 con 0x7f559c06d8a0 2024-02-28T23:26:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 msgr2=0x7f559c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 0x7f559c06fd50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005950 tx=0x7f55a40058e0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 msgr2=0x7f55b4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55a80092e0 tx=0x7f55a800be20 comp rx=0 tx=0).stop 2024-02-28T23:26:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 shutdown_connections 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f559c06d8a0 0x7f559c06fd50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55b41004e0 0x7f55b4073050 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 --2- 172.21.15.57:0/133270507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b4100e80 0x7f55b4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.763+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 >> 172.21.15.57:0/133270507 conn(0x7f55b40fbda0 msgr2=0x7f55b40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.764+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 shutdown_connections 2024-02-28T23:26:52.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.764+0000 7f55bc800700 1 -- 172.21.15.57:0/133270507 wait complete. 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 -- 172.21.15.57:0/166816467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6954106b80 msgr2=0x7f6954106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 --2- 172.21.15.57:0/166816467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6954106b80 0x7f6954106ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6940009910 tx=0x7f6940009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 -- 172.21.15.57:0/166816467 shutdown_connections 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 --2- 172.21.15.57:0/166816467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6954106b80 0x7f6954106ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 --2- 172.21.15.57:0/166816467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695410c8f0 0x7f695410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 -- 172.21.15.57:0/166816467 >> 172.21.15.57:0/166816467 conn(0x7f6954076060 msgr2=0x7f6954078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 -- 172.21.15.57:0/166816467 shutdown_connections 2024-02-28T23:26:52.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.925+0000 7f695c200700 1 -- 172.21.15.57:0/166816467 wait complete. 2024-02-28T23:26:52.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.926+0000 7f695c200700 1 Processor -- start 2024-02-28T23:26:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.927+0000 7f695c200700 1 -- start start 2024-02-28T23:26:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.927+0000 7f695c200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954106b80 0x7f695419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.927+0000 7f695c200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.927+0000 7f695c200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695419d060 con 0x7f695410c8f0 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.927+0000 7f695c200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69541a0df0 con 0x7f6954106b80 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53304/0 (socket says 172.21.15.57:53304) 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 -- 172.21.15.57:0/1760307325 learned_addr learned my addr 172.21.15.57:0/1760307325 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6959800700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954106b80 0x7f695419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 -- 172.21.15.57:0/1760307325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954106b80 msgr2=0x7f695419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954106b80 0x7f695419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.928+0000 7f6958e00700 1 -- 172.21.15.57:0/1760307325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400095f0 con 0x7f695410c8f0 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f6958e00700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f694000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694001d070 con 0x7f695410c8f0 2024-02-28T23:26:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69541a1070 con 0x7f695410c8f0 2024-02-28T23:26:52.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6940005dc0 con 0x7f695410c8f0 2024-02-28T23:26:52.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6940017400 con 0x7f695410c8f0 2024-02-28T23:26:52.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.929+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69541a1560 con 0x7f695410c8f0 2024-02-28T23:26:52.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.930+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695410ecc0 con 0x7f695410c8f0 2024-02-28T23:26:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.931+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6940003a60 con 0x7f695410c8f0 2024-02-28T23:26:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.932+0000 7f6952000700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 0x7f693c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.933+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f694008e990 con 0x7f695410c8f0 2024-02-28T23:26:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.933+0000 7f6959800700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 0x7f693c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.933+0000 7f6959800700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 0x7f693c06fd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6948005950 tx=0x7f694800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:52.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:52.938+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6940058120 con 0x7f695410c8f0 2024-02-28T23:26:53.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.174+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69541a1890 con 0x7f693c06d8a0 2024-02-28T23:26:53.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.190+0000 7f6952000700 1 -- 172.21.15.57:0/1760307325 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f69541a1890 con 0x7f693c06d8a0 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (16m) 7m ago 30m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (26m) 7m ago 30m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (29m) 7m ago 29m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (12m) 7m ago 30m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (12m) 7m ago 29m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (15m) 7m ago 29m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (8m) 7m ago 23m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:26:53.190 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (8m) 7m ago 23m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (22m) 7m ago 22m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (7m) 7m ago 23m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (18m) 7m ago 32m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (17m) 7m ago 28m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (14m) 7m ago 32m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (13m) 7m ago 28m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (17m) 7m ago 30m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (17m) 7m ago 29m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (11m) 7m ago 27m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (11m) 7m ago 27m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (10m) 7m ago 26m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (10m) 7m ago 26m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (9m) 7m ago 25m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (8m) 7m ago 24m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:26:53.191 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (16m) 7m ago 29m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:26:53.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 msgr2=0x7f693c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 0x7f693c06fd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6948005950 tx=0x7f694800b410 comp rx=0 tx=0).stop 2024-02-28T23:26:53.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 msgr2=0x7f695419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f694000bd70 comp rx=0 tx=0).stop 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 shutdown_connections 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f693c06d8a0 0x7f693c06fd50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954106b80 0x7f695419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 --2- 172.21.15.57:0/1760307325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695410c8f0 0x7f695419c980 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 >> 172.21.15.57:0/1760307325 conn(0x7f6954076060 msgr2=0x7f6954077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 shutdown_connections 2024-02-28T23:26:53.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.193+0000 7f695c200700 1 -- 172.21.15.57:0/1760307325 wait complete. 2024-02-28T23:26:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 -- 172.21.15.57:0/2992630014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 msgr2=0x7fc0e4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2992630014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4104e50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009910 tx=0x7fc0d0009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 -- 172.21.15.57:0/2992630014 shutdown_connections 2024-02-28T23:26:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2992630014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4104e50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2992630014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 0x7fc0e41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 -- 172.21.15.57:0/2992630014 >> 172.21.15.57:0/2992630014 conn(0x7fc0e40fbda0 msgr2=0x7fc0e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 -- 172.21.15.57:0/2992630014 shutdown_connections 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.342+0000 7fc0eac00700 1 -- 172.21.15.57:0/2992630014 wait complete. 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.343+0000 7fc0eac00700 1 Processor -- start 2024-02-28T23:26:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.343+0000 7fc0eac00700 1 -- start start 2024-02-28T23:26:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0eac00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 0x7fc0e4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0eac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0eac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4198d30 con 0x7fc0e4100e80 2024-02-28T23:26:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0eac00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e419cac0 con 0x7fc0e41004e0 2024-02-28T23:26:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0e3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53312/0 (socket says 172.21.15.57:53312) 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0e3400700 1 -- 172.21.15.57:0/2781166746 learned_addr learned my addr 172.21.15.57:0/2781166746 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.344+0000 7fc0e3e00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 0x7fc0e4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.345+0000 7fc0e3400700 1 -- 172.21.15.57:0/2781166746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 msgr2=0x7fc0e4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.345+0000 7fc0e3400700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 0x7fc0e4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.345+0000 7fc0e3400700 1 -- 172.21.15.57:0/2781166746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0e4100e80 2024-02-28T23:26:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.345+0000 7fc0e3400700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000bd70 tx=0x7fc0d000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.345+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0d001d070 con 0x7fc0e4100e80 2024-02-28T23:26:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.346+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e419cd40 con 0x7fc0e4100e80 2024-02-28T23:26:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.346+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0d0005dc0 con 0x7fc0e4100e80 2024-02-28T23:26:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.346+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0d0017400 con 0x7fc0e4100e80 2024-02-28T23:26:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.346+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e419d230 con 0x7fc0e4100e80 2024-02-28T23:26:53.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.347+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e41045d0 con 0x7fc0e4100e80 2024-02-28T23:26:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.348+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc0d000f460 con 0x7fc0e4100e80 2024-02-28T23:26:53.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.349+0000 7fc0e0c00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 0x7fc0c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.349+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc0d008eab0 con 0x7fc0e4100e80 2024-02-28T23:26:53.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.349+0000 7fc0e3e00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 0x7fc0c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.350+0000 7fc0e3e00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 0x7fc0c806fd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4005950 tx=0x7fc0d400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:53.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.354+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc0d0090e60 con 0x7fc0e4100e80 2024-02-28T23:26:53.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.674+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0e404f2a0 con 0x7fc0e4100e80 2024-02-28T23:26:53.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.675+0000 7fc0e0c00700 1 -- 172.21.15.57:0/2781166746 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fc0d00581c0 con 0x7fc0e4100e80 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:26:53.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:26:53.676 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 msgr2=0x7fc0c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 0x7fc0c806fd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4005950 tx=0x7fc0d400b410 comp rx=0 tx=0).stop 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 msgr2=0x7fc0e4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000bd70 tx=0x7fc0d000bda0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 shutdown_connections 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc0c806d8a0 0x7fc0c806fd50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e41004e0 0x7fc0e4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 --2- 172.21.15.57:0/2781166746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0e4100e80 0x7fc0e4198650 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.678+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 >> 172.21.15.57:0/2781166746 conn(0x7fc0e40fbda0 msgr2=0x7fc0e40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:53.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.679+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 shutdown_connections 2024-02-28T23:26:53.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.679+0000 7fc0eac00700 1 -- 172.21.15.57:0/2781166746 wait complete. 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 -- 172.21.15.57:0/1985417164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f841025f0 msgr2=0x7f5f841029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 --2- 172.21.15.57:0/1985417164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f841025f0 0x7f5f841029c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009910 tx=0x7f5f74009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 -- 172.21.15.57:0/1985417164 shutdown_connections 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 --2- 172.21.15.57:0/1985417164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84103e50 0x7f5f841042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 --2- 172.21.15.57:0/1985417164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f841025f0 0x7f5f841029c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 -- 172.21.15.57:0/1985417164 >> 172.21.15.57:0/1985417164 conn(0x7f5f840fddb0 msgr2=0x7f5f841001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 -- 172.21.15.57:0/1985417164 shutdown_connections 2024-02-28T23:26:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.840+0000 7f5f89c00700 1 -- 172.21.15.57:0/1985417164 wait complete. 2024-02-28T23:26:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.842+0000 7f5f89c00700 1 Processor -- start 2024-02-28T23:26:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.842+0000 7f5f89c00700 1 -- start start 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f89c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f89c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84198650 0x7f5f8419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f89c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f84198c60 con 0x7f5f84103e50 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f89c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f84198dd0 con 0x7f5f84198650 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f83e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f83e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53338/0 (socket says 172.21.15.57:53338) 2024-02-28T23:26:53.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f83e00700 1 -- 172.21.15.57:0/777683707 learned_addr learned my addr 172.21.15.57:0/777683707 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:53.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.843+0000 7f5f83400700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84198650 0x7f5f8419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f83e00700 1 -- 172.21.15.57:0/777683707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84198650 msgr2=0x7f5f8419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f83e00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84198650 0x7f5f8419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f83e00700 1 -- 172.21.15.57:0/777683707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740095f0 con 0x7f5f84103e50 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f83e00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f74015000 tx=0x7f5f7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7401d070 con 0x7f5f84103e50 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f74005dc0 con 0x7f5f84103e50 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8419d060 con 0x7f5f84103e50 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.844+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f74017400 con 0x7f5f84103e50 2024-02-28T23:26:53.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.845+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8419d5b0 con 0x7f5f84103e50 2024-02-28T23:26:53.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.846+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8410a8d0 con 0x7f5f84103e50 2024-02-28T23:26:53.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.847+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5f7400f460 con 0x7f5f84103e50 2024-02-28T23:26:53.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.848+0000 7f5f80c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 0x7f5f6806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:53.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.848+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f7408eab0 con 0x7f5f84103e50 2024-02-28T23:26:53.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.849+0000 7f5f83400700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 0x7f5f6806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:53.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.849+0000 7f5f83400700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 0x7f5f6806fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000bbd0 tx=0x7f5f7000b5b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:53.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:53.853+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f74090e60 con 0x7f5f84103e50 2024-02-28T23:26:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:53 smithi057 ceph-mon[89491]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:53 smithi057 ceph-mon[89491]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:53 smithi057 ceph-mon[89491]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:53 smithi084 ceph-mon[70440]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:53 smithi084 ceph-mon[70440]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:53 smithi084 ceph-mon[70440]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:54.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.130+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5f841994a0 con 0x7f5f84103e50 2024-02-28T23:26:54.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.131+0000 7f5f80c00700 1 -- 172.21.15.57:0/777683707 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f5f740581c0 con 0x7f5f84103e50 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:26:54.132 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:26:54.133 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:54.134 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:26:54.135 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:26:54.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 msgr2=0x7f5f6806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 0x7f5f6806fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000bbd0 tx=0x7f5f7000b5b0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 msgr2=0x7f5f84198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f74015000 tx=0x7f5f7400be20 comp rx=0 tx=0).stop 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 shutdown_connections 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5f6806d8a0 0x7f5f6806fd50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f84103e50 0x7f5f84198110 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 --2- 172.21.15.57:0/777683707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f84198650 0x7f5f8419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 >> 172.21.15.57:0/777683707 conn(0x7f5f840fddb0 msgr2=0x7f5f840fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.134+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 shutdown_connections 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.135+0000 7f5f89c00700 1 -- 172.21.15.57:0/777683707 wait complete. 2024-02-28T23:26:54.136 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.291+0000 7f92f5200700 1 -- 172.21.15.57:0/2417204253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 msgr2=0x7f92f010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.291+0000 7f92f5200700 1 --2- 172.21.15.57:0/2417204253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f010ccc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009910 tx=0x7f92e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 -- 172.21.15.57:0/2417204253 shutdown_connections 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 --2- 172.21.15.57:0/2417204253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 0x7f92f0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 --2- 172.21.15.57:0/2417204253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f010ccc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 -- 172.21.15.57:0/2417204253 >> 172.21.15.57:0/2417204253 conn(0x7f92f0076020 msgr2=0x7f92f0078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 -- 172.21.15.57:0/2417204253 shutdown_connections 2024-02-28T23:26:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.292+0000 7f92f5200700 1 -- 172.21.15.57:0/2417204253 wait complete. 2024-02-28T23:26:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 Processor -- start 2024-02-28T23:26:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 -- start start 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 0x7f92f019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f019cf60 con 0x7f92f010c8f0 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.293+0000 7f92f5200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f019d0a0 con 0x7f92f0106b80 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53368/0 (socket says 172.21.15.57:53368) 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 -- 172.21.15.57:0/1810793420 learned_addr learned my addr 172.21.15.57:0/1810793420 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ede00700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 0x7f92f019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 -- 172.21.15.57:0/1810793420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 msgr2=0x7f92f019c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 0x7f92f019c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.294+0000 7f92ed400700 1 -- 172.21.15.57:0/1810793420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00095f0 con 0x7f92f010c8f0 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92ed400700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c930 tx=0x7f92dc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc007ad0 con 0x7f92f010c8f0 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f01a0ef0 con 0x7f92f010c8f0 2024-02-28T23:26:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92dc007c30 con 0x7f92f010c8f0 2024-02-28T23:26:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc018610 con 0x7f92f010c8f0 2024-02-28T23:26:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.295+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f01a1440 con 0x7f92f010c8f0 2024-02-28T23:26:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.296+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f004f2a0 con 0x7f92f010c8f0 2024-02-28T23:26:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.297+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f92dc00f450 con 0x7f92f010c8f0 2024-02-28T23:26:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.299+0000 7f92e6a00700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 0x7f92d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.299+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92dc08d770 con 0x7f92f010c8f0 2024-02-28T23:26:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.299+0000 7f92ede00700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 0x7f92d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.300+0000 7f92ede00700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 0x7f92d406fd50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006010 tx=0x7f92e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:54.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.303+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92dc056f20 con 0x7f92f010c8f0 2024-02-28T23:26:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.547+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f01a1870 con 0x7f92d406d8a0 2024-02-28T23:26:54.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.551+0000 7f92e6a00700 1 -- 172.21.15.57:0/1810793420 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f92f01a1870 con 0x7f92d406d8a0 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:26:54.552 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:26:54.553 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:26:54.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.554+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 msgr2=0x7f92d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.554+0000 7f92f5200700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 0x7f92d406fd50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006010 tx=0x7f92e001a040 comp rx=0 tx=0).stop 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.554+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 msgr2=0x7f92f019c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.554+0000 7f92f5200700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c930 tx=0x7f92dc00ccf0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 shutdown_connections 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92d406d8a0 0x7f92d406fd50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f0106b80 0x7f92f019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 --2- 172.21.15.57:0/1810793420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92f010c8f0 0x7f92f019c910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 >> 172.21.15.57:0/1810793420 conn(0x7f92f0076020 msgr2=0x7f92f0077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:54.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 shutdown_connections 2024-02-28T23:26:54.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.555+0000 7f92f5200700 1 -- 172.21.15.57:0/1810793420 wait complete. 2024-02-28T23:26:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 -- 172.21.15.57:0/1525933422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 msgr2=0x7f05fc1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 --2- 172.21.15.57:0/1525933422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1028e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009910 tx=0x7f05e8009c20 comp rx=0 tx=0).stop 2024-02-28T23:26:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 -- 172.21.15.57:0/1525933422 shutdown_connections 2024-02-28T23:26:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 --2- 172.21.15.57:0/1525933422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1028e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 --2- 172.21.15.57:0/1525933422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 0x7f05fc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 -- 172.21.15.57:0/1525933422 >> 172.21.15.57:0/1525933422 conn(0x7f05fc0fdf90 msgr2=0x7f05fc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 -- 172.21.15.57:0/1525933422 shutdown_connections 2024-02-28T23:26:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.702+0000 7f0601600700 1 -- 172.21.15.57:0/1525933422 wait complete. 2024-02-28T23:26:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 Processor -- start 2024-02-28T23:26:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 -- start start 2024-02-28T23:26:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 0x7f05fc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc198d80 con 0x7f05fc102470 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.704+0000 7f0601600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc198ec0 con 0x7f05fc108470 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53388/0 (socket says 172.21.15.57:53388) 2024-02-28T23:26:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 -- 172.21.15.57:0/3531321628 learned_addr learned my addr 172.21.15.57:0/3531321628 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05fa000700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 0x7f05fc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 -- 172.21.15.57:0/3531321628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 msgr2=0x7f05fc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 0x7f05fc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 -- 172.21.15.57:0/3531321628 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80095f0 con 0x7f05fc102470 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.705+0000 7f05faa00700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c930 tx=0x7f05ec00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.706+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ec007ad0 con 0x7f05fc102470 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.706+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05ec007c30 con 0x7f05fc102470 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.706+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ec018610 con 0x7f05fc102470 2024-02-28T23:26:54.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.706+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc19cd10 con 0x7f05fc102470 2024-02-28T23:26:54.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.706+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc19d210 con 0x7f05fc102470 2024-02-28T23:26:54.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.707+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc10a970 con 0x7f05fc102470 2024-02-28T23:26:54.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.708+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f05ec00f450 con 0x7f05fc102470 2024-02-28T23:26:54.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.709+0000 7f05f3400700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 0x7f05e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:26:54.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.709+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f05ec08d810 con 0x7f05fc102470 2024-02-28T23:26:54.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.709+0000 7f05fa000700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 0x7f05e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:26:54.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.710+0000 7f05fa000700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 0x7f05e006fd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000c00 tx=0x7f05e8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:26:54.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:54.714+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05ec056f20 con 0x7f05fc102470 2024-02-28T23:26:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:54 smithi057 ceph-mon[89491]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:54 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2781166746' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:26:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:54 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/777683707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:26:55.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.024+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f05fc04f2a0 con 0x7f05fc102470 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.025+0000 7f05f3400700 1 -- 172.21.15.57:0/3531321628 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f05fc04f2a0 con 0x7f05fc102470 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:26:55.024 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.027+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 msgr2=0x7f05e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.027+0000 7f0601600700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 0x7f05e006fd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000c00 tx=0x7f05e8005fd0 comp rx=0 tx=0).stop 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.027+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 msgr2=0x7f05fc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.027+0000 7f0601600700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c930 tx=0x7f05ec00ccf0 comp rx=0 tx=0).stop 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 shutdown_connections 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05e006d8a0 0x7f05e006fd50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:55.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05fc102470 0x7f05fc1981f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:55.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 --2- 172.21.15.57:0/3531321628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc108470 0x7f05fc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:26:55.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 >> 172.21.15.57:0/3531321628 conn(0x7f05fc0fdf90 msgr2=0x7f05fc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:26:55.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 shutdown_connections 2024-02-28T23:26:55.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:26:55.028+0000 7f0601600700 1 -- 172.21.15.57:0/3531321628 wait complete. 2024-02-28T23:26:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:54 smithi084 ceph-mon[70440]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:54 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2781166746' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:26:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:54 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/777683707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:26:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:55 smithi057 ceph-mon[89491]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:55 smithi057 ceph-mon[89491]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:55 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3531321628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:26:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:55 smithi084 ceph-mon[70440]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:55 smithi084 ceph-mon[70440]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:26:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:55 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3531321628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:26:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:57 smithi057 ceph-mon[89491]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:57 smithi084 ceph-mon[70440]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:26:59.987 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:26:59 smithi084 ceph-mon[70440]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:26:59 smithi057 ceph-mon[89491]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:01 smithi057 ceph-mon[89491]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:01 smithi084 ceph-mon[70440]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:03 smithi057 ceph-mon[89491]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:03 smithi084 ceph-mon[70440]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:05 smithi057 ceph-mon[89491]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:05 smithi084 ceph-mon[70440]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:07 smithi057 ceph-mon[89491]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:07 smithi084 ceph-mon[70440]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:09 smithi057 ceph-mon[89491]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:10.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:09 smithi084 ceph-mon[70440]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:11 smithi057 ceph-mon[89491]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:11 smithi084 ceph-mon[70440]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:13 smithi057 ceph-mon[89491]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:13 smithi084 ceph-mon[70440]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:15 smithi057 ceph-mon[89491]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:16.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:15 smithi084 ceph-mon[70440]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:17 smithi057 ceph-mon[89491]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:17 smithi084 ceph-mon[70440]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:19 smithi057 ceph-mon[89491]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:19 smithi084 ceph-mon[70440]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:22.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:21 smithi084 ceph-mon[70440]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:22.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:21 smithi057 ceph-mon[89491]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:23 smithi057 ceph-mon[89491]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:24.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:23 smithi084 ceph-mon[70440]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.203+0000 7f9a4b400700 1 -- 172.21.15.57:0/851405140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44100e60 msgr2=0x7f9a44104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.203+0000 7f9a4b400700 1 --2- 172.21.15.57:0/851405140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44100e60 0x7f9a44104e30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009910 tx=0x7f9a30009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 -- 172.21.15.57:0/851405140 shutdown_connections 2024-02-28T23:27:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 --2- 172.21.15.57:0/851405140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44100e60 0x7f9a44104e30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 --2- 172.21.15.57:0/851405140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a441004c0 0x7f9a44100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 -- 172.21.15.57:0/851405140 >> 172.21.15.57:0/851405140 conn(0x7f9a440fbd80 msgr2=0x7f9a440fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 -- 172.21.15.57:0/851405140 shutdown_connections 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.204+0000 7f9a4b400700 1 -- 172.21.15.57:0/851405140 wait complete. 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.205+0000 7f9a4b400700 1 Processor -- start 2024-02-28T23:27:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a4b400700 1 -- start start 2024-02-28T23:27:25.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a4b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a4b400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a44100e60 0x7f9a44073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a4b400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44073ad0 con 0x7f9a441004c0 2024-02-28T23:27:25.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a4b400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44073c10 con 0x7f9a44100e60 2024-02-28T23:27:25.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.206+0000 7f9a48a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a48a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48752/0 (socket says 172.21.15.57:48752) 2024-02-28T23:27:25.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a48a00700 1 -- 172.21.15.57:0/1276841041 learned_addr learned my addr 172.21.15.57:0/1276841041 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:25.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a43e00700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a44100e60 0x7f9a44073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a48a00700 1 -- 172.21.15.57:0/1276841041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a44100e60 msgr2=0x7f9a44073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a48a00700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a44100e60 0x7f9a44073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.207+0000 7f9a48a00700 1 -- 172.21.15.57:0/1276841041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300095f0 con 0x7f9a441004c0 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a48a00700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800b4b0 tx=0x7f9a3800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a38011800 con 0x7f9a441004c0 2024-02-28T23:27:25.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a38011e20 con 0x7f9a441004c0 2024-02-28T23:27:25.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a3800f2e0 con 0x7f9a441004c0 2024-02-28T23:27:25.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a441a2190 con 0x7f9a441004c0 2024-02-28T23:27:25.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.208+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a441a25a0 con 0x7f9a441004c0 2024-02-28T23:27:25.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.209+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a44108760 con 0x7f9a441004c0 2024-02-28T23:27:25.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.211+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9a38010380 con 0x7f9a441004c0 2024-02-28T23:27:25.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.212+0000 7f9a41600700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 0x7f9a2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.212+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a3808cff0 con 0x7f9a441004c0 2024-02-28T23:27:25.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.212+0000 7f9a43e00700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 0x7f9a2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.213+0000 7f9a43e00700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 0x7f9a2c06fd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a44074840 tx=0x7f9a3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:25.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.218+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a380c0d20 con 0x7f9a441004c0 2024-02-28T23:27:25.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.469+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a44061190 con 0x7f9a2c06d8a0 2024-02-28T23:27:25.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.472+0000 7f9a41600700 1 -- 172.21.15.57:0/1276841041 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9a44061190 con 0x7f9a2c06d8a0 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 msgr2=0x7f9a2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 0x7f9a2c06fd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a44074840 tx=0x7f9a3001a040 comp rx=0 tx=0).stop 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 msgr2=0x7f9a44073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800b4b0 tx=0x7f9a3800b7c0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 shutdown_connections 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a2c06d8a0 0x7f9a2c06fd50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a441004c0 0x7f9a44073050 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 --2- 172.21.15.57:0/1276841041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a44100e60 0x7f9a44073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.474+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 >> 172.21.15.57:0/1276841041 conn(0x7f9a440fbd80 msgr2=0x7f9a440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:25.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.475+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 shutdown_connections 2024-02-28T23:27:25.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.475+0000 7f9a4b400700 1 -- 172.21.15.57:0/1276841041 wait complete. 2024-02-28T23:27:25.490 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.640+0000 7fddcbc00700 1 -- 172.21.15.57:0/2159041318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 msgr2=0x7fddc410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.640+0000 7fddcbc00700 1 --2- 172.21.15.57:0/2159041318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc410ccc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009910 tx=0x7fddb8009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.640+0000 7fddcbc00700 1 -- 172.21.15.57:0/2159041318 shutdown_connections 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.640+0000 7fddcbc00700 1 --2- 172.21.15.57:0/2159041318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 0x7fddc4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.641+0000 7fddcbc00700 1 --2- 172.21.15.57:0/2159041318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc410ccc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.641+0000 7fddcbc00700 1 -- 172.21.15.57:0/2159041318 >> 172.21.15.57:0/2159041318 conn(0x7fddc4076060 msgr2=0x7fddc4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:25.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.641+0000 7fddcbc00700 1 -- 172.21.15.57:0/2159041318 shutdown_connections 2024-02-28T23:27:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.641+0000 7fddcbc00700 1 -- 172.21.15.57:0/2159041318 wait complete. 2024-02-28T23:27:25.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 Processor -- start 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 -- start start 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 0x7fddc419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc419d100 con 0x7fddc410c8f0 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.642+0000 7fddcbc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41a0e90 con 0x7fddc4106b80 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48772/0 (socket says 172.21.15.57:48772) 2024-02-28T23:27:25.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 -- 172.21.15.57:0/447897552 learned_addr learned my addr 172.21.15.57:0/447897552 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc9200700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 0x7fddc419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 -- 172.21.15.57:0/447897552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 msgr2=0x7fddc419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 0x7fddc419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.643+0000 7fddc3e00700 1 -- 172.21.15.57:0/447897552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80095f0 con 0x7fddc410c8f0 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddc3e00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b4b0 tx=0x7fddb000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:25.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb0011800 con 0x7fddc410c8f0 2024-02-28T23:27:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc41a1170 con 0x7fddc410c8f0 2024-02-28T23:27:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddb0011e20 con 0x7fddc410c8f0 2024-02-28T23:27:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb000f2e0 con 0x7fddc410c8f0 2024-02-28T23:27:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.644+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc41a16c0 con 0x7fddc410c8f0 2024-02-28T23:27:25.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.645+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4109b20 con 0x7fddc410c8f0 2024-02-28T23:27:25.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.646+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fddb0010380 con 0x7fddc410c8f0 2024-02-28T23:27:25.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.648+0000 7fddc1600700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 0x7fddac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:25.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.648+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fddb008cff0 con 0x7fddc410c8f0 2024-02-28T23:27:25.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.648+0000 7fddc9200700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 0x7fddac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:25.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.649+0000 7fddc9200700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 0x7fddac06fd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fddb8006010 tx=0x7fddb801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:25.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.653+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddb00567c0 con 0x7fddc410c8f0 2024-02-28T23:27:25.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.904+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddc41a1990 con 0x7fddac06d8a0 2024-02-28T23:27:25.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.907+0000 7fddc1600700 1 -- 172.21.15.57:0/447897552 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fddc41a1990 con 0x7fddac06d8a0 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 msgr2=0x7fddac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 0x7fddac06fd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fddb8006010 tx=0x7fddb801a040 comp rx=0 tx=0).stop 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 msgr2=0x7fddc419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b4b0 tx=0x7fddb000b7c0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 shutdown_connections 2024-02-28T23:27:25.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddac06d8a0 0x7fddac06fd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4106b80 0x7fddc419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 --2- 172.21.15.57:0/447897552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fddc410c8f0 0x7fddc419ca20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:25.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 >> 172.21.15.57:0/447897552 conn(0x7fddc4076060 msgr2=0x7fddc4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:25.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 shutdown_connections 2024-02-28T23:27:25.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:25.910+0000 7fddcbc00700 1 -- 172.21.15.57:0/447897552 wait complete. 2024-02-28T23:27:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 -- 172.21.15.57:0/598331278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d0073090 msgr2=0x7f48d0073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 --2- 172.21.15.57:0/598331278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d0073090 0x7f48d0073460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009940 tx=0x7f48c4009c50 comp rx=0 tx=0).stop 2024-02-28T23:27:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 -- 172.21.15.57:0/598331278 shutdown_connections 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 --2- 172.21.15.57:0/598331278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d00739a0 0x7f48d010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 --2- 172.21.15.57:0/598331278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d0073090 0x7f48d0073460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 -- 172.21.15.57:0/598331278 >> 172.21.15.57:0/598331278 conn(0x7f48d00fbd80 msgr2=0x7f48d00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 -- 172.21.15.57:0/598331278 shutdown_connections 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.078+0000 7f48d8600700 1 -- 172.21.15.57:0/598331278 wait complete. 2024-02-28T23:27:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.079+0000 7f48d8600700 1 Processor -- start 2024-02-28T23:27:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d8600700 1 -- start start 2024-02-28T23:27:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d8600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d0073090 0x7f48d01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d8600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d8600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0198d00 con 0x7f48d00739a0 2024-02-28T23:27:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d8600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019ca40 con 0x7f48d0073090 2024-02-28T23:27:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.080+0000 7f48d5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48790/0 (socket says 172.21.15.57:48790) 2024-02-28T23:27:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5200700 1 -- 172.21.15.57:0/2774781018 learned_addr learned my addr 172.21.15.57:0/2774781018 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5c00700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d0073090 0x7f48d01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5200700 1 -- 172.21.15.57:0/2774781018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d0073090 msgr2=0x7f48d01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5200700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d0073090 0x7f48d01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5200700 1 -- 172.21.15.57:0/2774781018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c40095f0 con 0x7f48d00739a0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.081+0000 7f48d5200700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d8d0 tx=0x7f48bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.082+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48bc009940 con 0x7f48d00739a0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.082+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48bc010460 con 0x7f48d00739a0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.082+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48bc00f5f0 con 0x7f48d00739a0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.082+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d019cd20 con 0x7f48d00739a0 2024-02-28T23:27:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.082+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d019d220 con 0x7f48d00739a0 2024-02-28T23:27:26.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:25 smithi084 ceph-mon[70440]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.083+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0109cc0 con 0x7f48d00739a0 2024-02-28T23:27:26.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.084+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f48bc0105d0 con 0x7f48d00739a0 2024-02-28T23:27:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.085+0000 7f48ce000700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 0x7f48b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.085+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f48bc08d5d0 con 0x7f48d00739a0 2024-02-28T23:27:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.085+0000 7f48d5c00700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 0x7f48b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.086+0000 7f48d5c00700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 0x7f48b806fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005b40 tx=0x7f48c4005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.090+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48bc056ce0 con 0x7f48d00739a0 2024-02-28T23:27:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:25 smithi057 ceph-mon[89491]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.342+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48d019d500 con 0x7f48b806d8a0 2024-02-28T23:27:26.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.374+0000 7f48ce000700 1 -- 172.21.15.57:0/2774781018 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f48d019d500 con 0x7f48b806d8a0 2024-02-28T23:27:26.373 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:27:26.373 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (17m) 8m ago 30m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:27:26.373 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (27m) 8m ago 31m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (29m) 8m ago 29m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (13m) 8m ago 31m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (13m) 8m ago 29m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (16m) 8m ago 30m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (9m) 8m ago 23m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (8m) 8m ago 23m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (23m) 8m ago 23m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (8m) 8m ago 23m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (19m) 8m ago 32m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (18m) 8m ago 29m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (14m) 8m ago 33m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (14m) 8m ago 29m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (18m) 8m ago 31m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:27:26.374 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (17m) 8m ago 29m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (12m) 8m ago 28m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (11m) 8m ago 27m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (11m) 8m ago 27m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (10m) 8m ago 26m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (10m) 8m ago 26m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (9m) 8m ago 25m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:27:26.375 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (17m) 8m ago 30m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:27:26.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 msgr2=0x7f48b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 0x7f48b806fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005b40 tx=0x7f48c4005ab0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 msgr2=0x7f48d0198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d8d0 tx=0x7f48bc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:27:26.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 shutdown_connections 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48b806d8a0 0x7f48b806fd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48d0073090 0x7f48d01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 --2- 172.21.15.57:0/2774781018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48d00739a0 0x7f48d0198620 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.378+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 >> 172.21.15.57:0/2774781018 conn(0x7f48d00fbd80 msgr2=0x7f48d0106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.379+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 shutdown_connections 2024-02-28T23:27:26.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.379+0000 7f48d8600700 1 -- 172.21.15.57:0/2774781018 wait complete. 2024-02-28T23:27:26.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 -- 172.21.15.57:0/2273065584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd308106b80 msgr2=0x7fd308106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 --2- 172.21.15.57:0/2273065584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd308106b80 0x7fd308106ff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 -- 172.21.15.57:0/2273065584 shutdown_connections 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 --2- 172.21.15.57:0/2273065584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd308106b80 0x7fd308106ff0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 --2- 172.21.15.57:0/2273065584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 0x7fd30810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 -- 172.21.15.57:0/2273065584 >> 172.21.15.57:0/2273065584 conn(0x7fd308076060 msgr2=0x7fd308078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 -- 172.21.15.57:0/2273065584 shutdown_connections 2024-02-28T23:27:26.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.545+0000 7fd30ec00700 1 -- 172.21.15.57:0/2273065584 wait complete. 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.546+0000 7fd30ec00700 1 Processor -- start 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd30ec00700 1 -- start start 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd30ec00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 0x7fd30819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd30ec00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd30ec00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30819d1a0 con 0x7fd30819cb90 2024-02-28T23:27:26.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd30ec00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30819d310 con 0x7fd30810c8f0 2024-02-28T23:27:26.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.547+0000 7fd307400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48800/0 (socket says 172.21.15.57:48800) 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 -- 172.21.15.57:0/3654899922 learned_addr learned my addr 172.21.15.57:0/3654899922 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307e00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 0x7fd30819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 -- 172.21.15.57:0/3654899922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 msgr2=0x7fd30819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 0x7fd30819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 -- 172.21.15.57:0/3654899922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40095f0 con 0x7fd30819cb90 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.548+0000 7fd307400700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400bd70 tx=0x7fd2f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.549+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f401d070 con 0x7fd30819cb90 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.549+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f400f460 con 0x7fd30819cb90 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.549+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f4017630 con 0x7fd30819cb90 2024-02-28T23:27:26.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.549+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3081a15a0 con 0x7fd30819cb90 2024-02-28T23:27:26.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.549+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3081a1af0 con 0x7fd30819cb90 2024-02-28T23:27:26.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.550+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308109b20 con 0x7fd30819cb90 2024-02-28T23:27:26.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.551+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd2f400fab0 con 0x7fd30819cb90 2024-02-28T23:27:26.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.552+0000 7fd304c00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 0x7fd2ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:26.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.552+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2f408e940 con 0x7fd30819cb90 2024-02-28T23:27:26.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.552+0000 7fd307e00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 0x7fd2ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:26.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.553+0000 7fd307e00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 0x7fd2ec06fd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005950 tx=0x7fd2f8005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:26.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.557+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2f40580d0 con 0x7fd30819cb90 2024-02-28T23:27:26.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.877+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd3080008d0 con 0x7fd30819cb90 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.878+0000 7fd304c00700 1 -- 172.21.15.57:0/3654899922 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fd2f405b790 con 0x7fd30819cb90 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:27:26.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:27:26.878 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:27:26.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 msgr2=0x7fd2ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 0x7fd2ec06fd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005950 tx=0x7fd2f8005cf0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 msgr2=0x7fd3081a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400bd70 tx=0x7fd2f400be50 comp rx=0 tx=0).stop 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 shutdown_connections 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd2ec06d8a0 0x7fd2ec06fd50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd30810c8f0 0x7fd30819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 --2- 172.21.15.57:0/3654899922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd30819cb90 0x7fd3081a1000 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 >> 172.21.15.57:0/3654899922 conn(0x7fd308076060 msgr2=0x7fd308077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 shutdown_connections 2024-02-28T23:27:26.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:26.881+0000 7fd30ec00700 1 -- 172.21.15.57:0/3654899922 wait complete. 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 -- 172.21.15.57:0/429219891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8102470 msgr2=0x7fe8e81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/429219891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8102470 0x7fe8e81028e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 -- 172.21.15.57:0/429219891 shutdown_connections 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/429219891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8102470 0x7fe8e81028e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/429219891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8e8108470 0x7fe8e8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 -- 172.21.15.57:0/429219891 >> 172.21.15.57:0/429219891 conn(0x7fe8e80fdf90 msgr2=0x7fe8e81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 -- 172.21.15.57:0/429219891 shutdown_connections 2024-02-28T23:27:27.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.043+0000 7fe8ecc00700 1 -- 172.21.15.57:0/429219891 wait complete. 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.044+0000 7fe8ecc00700 1 Processor -- start 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8ecc00700 1 -- start start 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8ecc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8e8102470 0x7fe8e8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8ecc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8ecc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8198db0 con 0x7fe8e8108470 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8ecc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e819cb40 con 0x7fe8e8102470 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8e5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8e5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48820/0 (socket says 172.21.15.57:48820) 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.045+0000 7fe8e5400700 1 -- 172.21.15.57:0/2595669616 learned_addr learned my addr 172.21.15.57:0/2595669616 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:27.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8e5400700 1 -- 172.21.15.57:0/2595669616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8e8102470 msgr2=0x7fe8e8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8e5400700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8e8102470 0x7fe8e8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8e5400700 1 -- 172.21.15.57:0/2595669616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8e8108470 2024-02-28T23:27:27.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8e5400700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b5c0 tx=0x7fe8d400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d401d070 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8d400f460 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d4017610 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.046+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e819cdc0 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.047+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e819d260 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.048+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e810aa00 con 0x7fe8e8108470 2024-02-28T23:27:27.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.049+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe8d400f5d0 con 0x7fe8e8108470 2024-02-28T23:27:27.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.049+0000 7fe8dea00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 0x7fe8cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.050+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe8d408ec00 con 0x7fe8e8108470 2024-02-28T23:27:27.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.050+0000 7fe8e5e00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 0x7fe8cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.050+0000 7fe8e5e00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 0x7fe8cc06fd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097b0 tx=0x7fe8d8006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.054+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8d4058310 con 0x7fe8e8108470 2024-02-28T23:27:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:26 smithi057 ceph-mon[89491]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:26 smithi057 ceph-mon[89491]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.325+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8e804f2a0 con 0x7fe8e8108470 2024-02-28T23:27:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.326+0000 7fe8dea00700 1 -- 172.21.15.57:0/2595669616 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fe8d405b9d0 con 0x7fe8e8108470 2024-02-28T23:27:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:27:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:27:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:27:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:27:27.325 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:27:27.326 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:27:27.327 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.328+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 msgr2=0x7fe8cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.328+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 0x7fe8cc06fd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097b0 tx=0x7fe8d8006d20 comp rx=0 tx=0).stop 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.328+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 msgr2=0x7fe8e81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.328+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b5c0 tx=0x7fe8d400be00 comp rx=0 tx=0).stop 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 shutdown_connections 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8cc06d8a0 0x7fe8cc06fd50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8e8102470 0x7fe8e8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 --2- 172.21.15.57:0/2595669616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8e8108470 0x7fe8e81986d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 >> 172.21.15.57:0/2595669616 conn(0x7fe8e80fdf90 msgr2=0x7fe8e80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 shutdown_connections 2024-02-28T23:27:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.329+0000 7fe8ecc00700 1 -- 172.21.15.57:0/2595669616 wait complete. 2024-02-28T23:27:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:26 smithi084 ceph-mon[70440]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:26 smithi084 ceph-mon[70440]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:27.330 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:27:27.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 -- 172.21.15.57:0/1986248505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ffd60 msgr2=0x7ff214103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 --2- 172.21.15.57:0/1986248505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ffd60 0x7ff214103d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff200009910 tx=0x7ff200009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 -- 172.21.15.57:0/1986248505 shutdown_connections 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 --2- 172.21.15.57:0/1986248505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ffd60 0x7ff214103d30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 --2- 172.21.15.57:0/1986248505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2140ff3c0 0x7ff2140ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 -- 172.21.15.57:0/1986248505 >> 172.21.15.57:0/1986248505 conn(0x7ff2140fac80 msgr2=0x7ff2140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 -- 172.21.15.57:0/1986248505 shutdown_connections 2024-02-28T23:27:27.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.488+0000 7ff21ba00700 1 -- 172.21.15.57:0/1986248505 wait complete. 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.489+0000 7ff21ba00700 1 Processor -- start 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.489+0000 7ff21ba00700 1 -- start start 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff21ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff21ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2140ffd60 0x7ff214198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff21ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214198e10 con 0x7ff2140ff3c0 2024-02-28T23:27:27.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff21ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21419cba0 con 0x7ff2140ffd60 2024-02-28T23:27:27.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff219000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff219000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48836/0 (socket says 172.21.15.57:48836) 2024-02-28T23:27:27.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.490+0000 7ff219000700 1 -- 172.21.15.57:0/2198754495 learned_addr learned my addr 172.21.15.57:0/2198754495 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:27.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff219000700 1 -- 172.21.15.57:0/2198754495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2140ffd60 msgr2=0x7ff214198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff219000700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2140ffd60 0x7ff214198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff219000700 1 -- 172.21.15.57:0/2198754495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000095f0 con 0x7ff2140ff3c0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff219000700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff20800b4b0 tx=0x7ff20800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff208011800 con 0x7ff2140ff3c0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff208011e20 con 0x7ff2140ff3c0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20800f2e0 con 0x7ff2140ff3c0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21419ce80 con 0x7ff2140ff3c0 2024-02-28T23:27:27.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.491+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21419d3d0 con 0x7ff2140ff3c0 2024-02-28T23:27:27.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.492+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21404f2a0 con 0x7ff2140ff3c0 2024-02-28T23:27:27.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.494+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff208010380 con 0x7ff2140ff3c0 2024-02-28T23:27:27.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.495+0000 7ff211600700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 0x7ff1fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.495+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff20808d0f0 con 0x7ff2140ff3c0 2024-02-28T23:27:27.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.495+0000 7ff213e00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 0x7ff1fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.496+0000 7ff213e00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 0x7ff1fc06fd50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b5c0 tx=0x7ff200005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.500+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff208056930 con 0x7ff2140ff3c0 2024-02-28T23:27:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.741+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff21419d850 con 0x7ff1fc06d8a0 2024-02-28T23:27:27.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.744+0000 7ff211600700 1 -- 172.21.15.57:0/2198754495 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff21419d850 con 0x7ff1fc06d8a0 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:27:27.743 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:27:27.744 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:27:27.744 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:27:27.744 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:27:27.744 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:27:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:27:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 msgr2=0x7ff1fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 0x7ff1fc06fd50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b5c0 tx=0x7ff200005fd0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 msgr2=0x7ff2141981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff20800b4b0 tx=0x7ff20800b870 comp rx=0 tx=0).stop 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 shutdown_connections 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff1fc06d8a0 0x7ff1fc06fd50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2140ff3c0 0x7ff2141981f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 --2- 172.21.15.57:0/2198754495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2140ffd60 0x7ff214198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 >> 172.21.15.57:0/2198754495 conn(0x7ff2140fac80 msgr2=0x7ff214069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 shutdown_connections 2024-02-28T23:27:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.747+0000 7ff21ba00700 1 -- 172.21.15.57:0/2198754495 wait complete. 2024-02-28T23:27:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 -- 172.21.15.57:0/2422926200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 msgr2=0x7fc9b00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/2422926200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b00734c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009910 tx=0x7fc9a0009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 -- 172.21.15.57:0/2422926200 shutdown_connections 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/2422926200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 0x7fc9b010c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/2422926200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b00734c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 -- 172.21.15.57:0/2422926200 >> 172.21.15.57:0/2422926200 conn(0x7fc9b00fbe10 msgr2=0x7fc9b00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 -- 172.21.15.57:0/2422926200 shutdown_connections 2024-02-28T23:27:27.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.904+0000 7fc9b6a00700 1 -- 172.21.15.57:0/2422926200 wait complete. 2024-02-28T23:27:27.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 Processor -- start 2024-02-28T23:27:27.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 -- start start 2024-02-28T23:27:27.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 0x7fc9b01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b019c910 con 0x7fc9b00730f0 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.905+0000 7fc9b6a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b019ca80 con 0x7fc9b0073a00 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48850/0 (socket says 172.21.15.57:48850) 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 -- 172.21.15.57:0/3065131348 learned_addr learned my addr 172.21.15.57:0/3065131348 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9a7400700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 0x7fc9b01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 -- 172.21.15.57:0/3065131348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 msgr2=0x7fc9b01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 0x7fc9b01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 -- 172.21.15.57:0/3065131348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00095f0 con 0x7fc9b00730f0 2024-02-28T23:27:27.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.906+0000 7fc9afe00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.907+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a001d070 con 0x7fc9b00730f0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.907+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9a000f460 con 0x7fc9b00730f0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.907+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b019cd60 con 0x7fc9b00730f0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.907+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a0017630 con 0x7fc9b00730f0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.907+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b019d2b0 con 0x7fc9b00730f0 2024-02-28T23:27:27.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.908+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b0109cd0 con 0x7fc9b00730f0 2024-02-28T23:27:27.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.909+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc9a000f5d0 con 0x7fc9b00730f0 2024-02-28T23:27:27.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.910+0000 7fc9ad600700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 0x7fc99406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:27.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.910+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc9a008ea20 con 0x7fc9b00730f0 2024-02-28T23:27:27.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.910+0000 7fc9a7400700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 0x7fc99406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:27.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.911+0000 7fc9a7400700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 0x7fc99406fd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc998009c10 tx=0x7fc998009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:27.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:27.915+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9a0058130 con 0x7fc9b00730f0 2024-02-28T23:27:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:27 smithi057 ceph-mon[89491]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:27 smithi057 ceph-mon[89491]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:27 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3654899922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:27:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:27 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2595669616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:27:28.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.223+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc9b004f2a0 con 0x7fc9b00730f0 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.224+0000 7fc9ad600700 1 -- 172.21.15.57:0/3065131348 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc9a005b7f0 con 0x7fc9b00730f0 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:27:28.222 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:27:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 msgr2=0x7fc99406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 0x7fc99406fd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc998009c10 tx=0x7fc998009040 comp rx=0 tx=0).stop 2024-02-28T23:27:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 msgr2=0x7fc9b0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a000be20 comp rx=0 tx=0).stop 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 shutdown_connections 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc99406d8a0 0x7fc99406fd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b00730f0 0x7fc9b0197f60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 --2- 172.21.15.57:0/3065131348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b0073a00 0x7fc9b01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 >> 172.21.15.57:0/3065131348 conn(0x7fc9b00fbe10 msgr2=0x7fc9b0106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 shutdown_connections 2024-02-28T23:27:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:28.227+0000 7fc9b6a00700 1 -- 172.21.15.57:0/3065131348 wait complete. 2024-02-28T23:27:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:27 smithi084 ceph-mon[70440]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:27 smithi084 ceph-mon[70440]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:27 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3654899922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:27:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:27 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2595669616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:27:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:28 smithi057 ceph-mon[89491]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:28 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3065131348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:27:29.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:28 smithi084 ceph-mon[70440]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:27:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:28 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3065131348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:27:30.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:29 smithi057 ceph-mon[89491]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:30.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:29 smithi084 ceph-mon[70440]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:31 smithi057 ceph-mon[89491]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:32.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:31 smithi084 ceph-mon[70440]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:33 smithi057 ceph-mon[89491]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:33 smithi084 ceph-mon[70440]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:35 smithi057 ceph-mon[89491]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:36.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:35 smithi084 ceph-mon[70440]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:37 smithi057 ceph-mon[89491]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:38.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:37 smithi084 ceph-mon[70440]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:39 smithi057 ceph-mon[89491]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:40.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:39 smithi084 ceph-mon[70440]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:42.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:41 smithi057 ceph-mon[89491]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:42.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:41 smithi084 ceph-mon[70440]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:44.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:43 smithi057 ceph-mon[89491]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:44.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:43 smithi084 ceph-mon[70440]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:46.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:45 smithi057 ceph-mon[89491]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:46.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:45 smithi084 ceph-mon[70440]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:46 smithi057 ceph-mon[89491]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:46 smithi084 ceph-mon[70440]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:48.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:27:49.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:49 smithi057 ceph-mon[89491]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:49.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:49 smithi084 ceph-mon[70440]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:51 smithi084 ceph-mon[70440]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:51 smithi057 ceph-mon[89491]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:53 smithi057 ceph-mon[89491]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:53 smithi084 ceph-mon[70440]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:55 smithi057 ceph-mon[89491]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:55 smithi084 ceph-mon[70440]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:57 smithi057 ceph-mon[89491]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:57 smithi084 ceph-mon[70440]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:58.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 -- 172.21.15.57:0/239116547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc40687a0 msgr2=0x7fccc4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 --2- 172.21.15.57:0/239116547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc40687a0 0x7fccc4068b70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fccb800b0a0 tx=0x7fccb800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 -- 172.21.15.57:0/239116547 shutdown_connections 2024-02-28T23:27:58.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 --2- 172.21.15.57:0/239116547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4069140 0x7fccc4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 --2- 172.21.15.57:0/239116547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc40687a0 0x7fccc4068b70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 -- 172.21.15.57:0/239116547 >> 172.21.15.57:0/239116547 conn(0x7fccc4075130 msgr2=0x7fccc4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:58.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 -- 172.21.15.57:0/239116547 shutdown_connections 2024-02-28T23:27:58.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.402+0000 7fcccc400700 1 -- 172.21.15.57:0/239116547 wait complete. 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.403+0000 7fcccc400700 1 Processor -- start 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.403+0000 7fcccc400700 1 -- start start 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fcccc400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc40687a0 0x7fccc4195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fcccc400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fcccc400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4196b40 con 0x7fccc4069140 2024-02-28T23:27:58.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fcccc400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419a8d0 con 0x7fccc40687a0 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fccc9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fccc9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56714/0 (socket says 172.21.15.57:56714) 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fccc9000700 1 -- 172.21.15.57:0/3683427556 learned_addr learned my addr 172.21.15.57:0/3683427556 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fccc9a00700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc40687a0 0x7fccc4195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.404+0000 7fccc9000700 1 -- 172.21.15.57:0/3683427556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc40687a0 msgr2=0x7fccc4195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc9000700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc40687a0 0x7fccc4195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc9000700 1 -- 172.21.15.57:0/3683427556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8009d00 con 0x7fccc4069140 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc9000700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccb000d900 tx=0x7fccb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb0009900 con 0x7fccc4069140 2024-02-28T23:27:58.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccb0010460 con 0x7fccc4069140 2024-02-28T23:27:58.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc419abb0 con 0x7fccc4069140 2024-02-28T23:27:58.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb000f5f0 con 0x7fccc4069140 2024-02-28T23:27:58.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.405+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc419b100 con 0x7fccc4069140 2024-02-28T23:27:58.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.407+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4108770 con 0x7fccc4069140 2024-02-28T23:27:58.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.408+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fccb00105d0 con 0x7fccc4069140 2024-02-28T23:27:58.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.409+0000 7fccc2000700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 0x7fccac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.409+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fccb008d430 con 0x7fccc4069140 2024-02-28T23:27:58.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.409+0000 7fccc9a00700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 0x7fccac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.410+0000 7fccc9a00700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 0x7fccac06fd50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009480 tx=0x7fccb800b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:58.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.414+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccb0056b40 con 0x7fccc4069140 2024-02-28T23:27:58.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.667+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc419b320 con 0x7fccac06d8a0 2024-02-28T23:27:58.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.669+0000 7fccc2000700 1 -- 172.21.15.57:0/3683427556 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fccc419b320 con 0x7fccac06d8a0 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 msgr2=0x7fccac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 0x7fccac06fd50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009480 tx=0x7fccb800b7f0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 msgr2=0x7fccc4196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccb000d900 tx=0x7fccb000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 shutdown_connections 2024-02-28T23:27:58.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fccac06d8a0 0x7fccac06fd50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc40687a0 0x7fccc4195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 --2- 172.21.15.57:0/3683427556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccc4069140 0x7fccc4196460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 >> 172.21.15.57:0/3683427556 conn(0x7fccc4075130 msgr2=0x7fccc40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 shutdown_connections 2024-02-28T23:27:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.673+0000 7fcccc400700 1 -- 172.21.15.57:0/3683427556 wait complete. 2024-02-28T23:27:58.689 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:27:58.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 -- 172.21.15.57:0/2872148974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644108470 msgr2=0x7f7644108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 --2- 172.21.15.57:0/2872148974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644108470 0x7f7644108840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7638009910 tx=0x7f7638009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:58.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 -- 172.21.15.57:0/2872148974 shutdown_connections 2024-02-28T23:27:58.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 --2- 172.21.15.57:0/2872148974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644102470 0x7f76441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 --2- 172.21.15.57:0/2872148974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644108470 0x7f7644108840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 -- 172.21.15.57:0/2872148974 >> 172.21.15.57:0/2872148974 conn(0x7f76440fdf90 msgr2=0x7f76441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:58.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 -- 172.21.15.57:0/2872148974 shutdown_connections 2024-02-28T23:27:58.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.839+0000 7f764ba00700 1 -- 172.21.15.57:0/2872148974 wait complete. 2024-02-28T23:27:58.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.840+0000 7f764ba00700 1 Processor -- start 2024-02-28T23:27:58.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f764ba00700 1 -- start start 2024-02-28T23:27:58.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f764ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f764ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 0x7f7644198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f764ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644198e10 con 0x7f7644102470 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f764ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764419cba0 con 0x7f7644108470 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f7649000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f7649000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56726/0 (socket says 172.21.15.57:56726) 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.841+0000 7f7649000700 1 -- 172.21.15.57:0/2915793557 learned_addr learned my addr 172.21.15.57:0/2915793557 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7649000700 1 -- 172.21.15.57:0/2915793557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 msgr2=0x7f7644198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:58.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7643e00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 0x7f7644198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7649000700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 0x7f7644198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7649000700 1 -- 172.21.15.57:0/2915793557 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380095f0 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7643e00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 0x7f7644198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7649000700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f763800b5c0 tx=0x7f763800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.842+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763801d070 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.843+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763800f460 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.843+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764419ce20 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.843+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7638017610 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.843+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764419d310 con 0x7f7644102470 2024-02-28T23:27:58.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.844+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764410a9e0 con 0x7f7644102470 2024-02-28T23:27:58.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.845+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f763800f5d0 con 0x7f7644102470 2024-02-28T23:27:58.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.846+0000 7f7641600700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 0x7f762c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:58.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.846+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f763808ec00 con 0x7f7644102470 2024-02-28T23:27:58.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.846+0000 7f7643e00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 0x7f762c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:58.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.847+0000 7f7643e00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 0x7f762c06fd50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7630005fd0 tx=0x7f7630005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:58.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:58.851+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7638058310 con 0x7f7644102470 2024-02-28T23:27:59.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.103+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76441995f0 con 0x7f762c06d8a0 2024-02-28T23:27:59.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.105+0000 7f7641600700 1 -- 172.21.15.57:0/2915793557 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f76441995f0 con 0x7f762c06d8a0 2024-02-28T23:27:59.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.108+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 msgr2=0x7f762c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.108+0000 7f764ba00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 0x7f762c06fd50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7630005fd0 tx=0x7f7630005dc0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.108+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 msgr2=0x7f76441981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.108+0000 7f764ba00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f763800b5c0 tx=0x7f763800be00 comp rx=0 tx=0).stop 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 shutdown_connections 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f762c06d8a0 0x7f762c06fd50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7644102470 0x7f76441981f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 --2- 172.21.15.57:0/2915793557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7644108470 0x7f7644198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 >> 172.21.15.57:0/2915793557 conn(0x7f76440fdf90 msgr2=0x7f76441002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 shutdown_connections 2024-02-28T23:27:59.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.109+0000 7f764ba00700 1 -- 172.21.15.57:0/2915793557 wait complete. 2024-02-28T23:27:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.275+0000 7fb915a00700 1 -- 172.21.15.57:0/3899507840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 msgr2=0x7fb9101008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.275+0000 7fb915a00700 1 --2- 172.21.15.57:0/3899507840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb9101008b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b0a0 tx=0x7fb90000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 -- 172.21.15.57:0/3899507840 shutdown_connections 2024-02-28T23:27:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 --2- 172.21.15.57:0/3899507840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 0x7fb910104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 --2- 172.21.15.57:0/3899507840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb9101008b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 -- 172.21.15.57:0/3899507840 >> 172.21.15.57:0/3899507840 conn(0x7fb9100fbda0 msgr2=0x7fb9100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 -- 172.21.15.57:0/3899507840 shutdown_connections 2024-02-28T23:27:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.276+0000 7fb915a00700 1 -- 172.21.15.57:0/3899507840 wait complete. 2024-02-28T23:27:59.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 Processor -- start 2024-02-28T23:27:59.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 -- start start 2024-02-28T23:27:59.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 0x7fb9101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb910198d90 con 0x7fb9101004e0 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.277+0000 7fb915a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91019cb20 con 0x7fb910100e80 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56746/0 (socket says 172.21.15.57:56746) 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 -- 172.21.15.57:0/4126942365 learned_addr learned my addr 172.21.15.57:0/4126942365 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90e000700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 0x7fb9101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 -- 172.21.15.57:0/4126942365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 msgr2=0x7fb9101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 0x7fb9101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.278+0000 7fb90ea00700 1 -- 172.21.15.57:0/4126942365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900009d00 con 0x7fb9101004e0 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb90ea00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b070 tx=0x7fb90000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90000e040 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb900003de0 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90001d8e0 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91019cda0 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.279+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91019d290 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.280+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9101045d0 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.281+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb900004040 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.282+0000 7fb907400700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 0x7fb8f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.282+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb90008ece0 con 0x7fb9101004e0 2024-02-28T23:27:59.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.283+0000 7fb90e000700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 0x7fb8f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.283+0000 7fb90e000700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 0x7fb8f406fd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005950 tx=0x7fb8fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:59.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.287+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb900058470 con 0x7fb9101004e0 2024-02-28T23:27:59.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.528+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb91019d5c0 con 0x7fb8f406d8a0 2024-02-28T23:27:59.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.560+0000 7fb907400700 1 -- 172.21.15.57:0/4126942365 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb91019d5c0 con 0x7fb8f406d8a0 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (17m) 9m ago 31m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (27m) 9m ago 31m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (30m) 8m ago 30m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (13m) 9m ago 31m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (13m) 8m ago 30m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (17m) 9m ago 31m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:27:59.561 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (9m) 9m ago 24m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (9m) 9m ago 24m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (24m) 8m ago 24m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (8m) 8m ago 24m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (20m) 9m ago 33m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (19m) 8m ago 30m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (15m) 9m ago 33m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (14m) 8m ago 29m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (18m) 9m ago 31m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (18m) 8m ago 30m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (13m) 9m ago 28m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (12m) 9m ago 28m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:27:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (11m) 9m ago 27m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:27:59.563 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (11m) 8m ago 27m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:27:59.563 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (10m) 8m ago 26m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:27:59.563 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (10m) 8m ago 26m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:27:59.563 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (18m) 9m ago 30m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 msgr2=0x7fb8f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 0x7fb8f406fd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005950 tx=0x7fb8fc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 msgr2=0x7fb910198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b070 tx=0x7fb90000cdd0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 shutdown_connections 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb8f406d8a0 0x7fb8f406fd50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9101004e0 0x7fb910198170 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 --2- 172.21.15.57:0/4126942365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb910100e80 0x7fb9101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 >> 172.21.15.57:0/4126942365 conn(0x7fb9100fbda0 msgr2=0x7fb9100fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 shutdown_connections 2024-02-28T23:27:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.564+0000 7fb915a00700 1 -- 172.21.15.57:0/4126942365 wait complete. 2024-02-28T23:27:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 -- 172.21.15.57:0/820548611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075280 msgr2=0x7ff0a0075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 --2- 172.21.15.57:0/820548611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075280 0x7ff0a0075650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff094009910 tx=0x7ff094009c20 comp rx=0 tx=0).stop 2024-02-28T23:27:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 -- 172.21.15.57:0/820548611 shutdown_connections 2024-02-28T23:27:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 --2- 172.21.15.57:0/820548611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075b90 0x7ff0a0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 --2- 172.21.15.57:0/820548611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075280 0x7ff0a0075650 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 -- 172.21.15.57:0/820548611 >> 172.21.15.57:0/820548611 conn(0x7ff0a00fdf90 msgr2=0x7ff0a01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:27:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 -- 172.21.15.57:0/820548611 shutdown_connections 2024-02-28T23:27:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.730+0000 7ff0a7800700 1 -- 172.21.15.57:0/820548611 wait complete. 2024-02-28T23:27:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 Processor -- start 2024-02-28T23:27:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 -- start start 2024-02-28T23:27:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075280 0x7ff0a019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a019d070 con 0x7ff0a0075b90 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.731+0000 7ff0a7800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01a0e00 con 0x7ff0a0075280 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56758/0 (socket says 172.21.15.57:56758) 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 -- 172.21.15.57:0/1746581482 learned_addr learned my addr 172.21.15.57:0/1746581482 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff0a4e00700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075280 0x7ff0a019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 -- 172.21.15.57:0/1746581482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075280 msgr2=0x7ff0a019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075280 0x7ff0a019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:27:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.732+0000 7ff09fe00700 1 -- 172.21.15.57:0/1746581482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940095f0 con 0x7ff0a0075b90 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff09fe00700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d900 tx=0x7ff08c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08c004a00 con 0x7ff0a0075b90 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff08c0054e0 con 0x7ff0a0075b90 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08c009d60 con 0x7ff0a0075b90 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a01a10e0 con 0x7ff0a0075b90 2024-02-28T23:27:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.733+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a01a1630 con 0x7ff0a0075b90 2024-02-28T23:27:59.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.734+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a010e340 con 0x7ff0a0075b90 2024-02-28T23:27:59.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.735+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff08c005020 con 0x7ff0a0075b90 2024-02-28T23:27:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.736+0000 7ff09d600700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 0x7ff08806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:27:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.737+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff08c08d720 con 0x7ff0a0075b90 2024-02-28T23:27:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.737+0000 7ff0a4e00700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 0x7ff08806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:27:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.737+0000 7ff0a4e00700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 0x7ff08806fd50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b5c0 tx=0x7ff094005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:27:59.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:27:59.741+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff08c056eb0 con 0x7ff0a0075b90 2024-02-28T23:27:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:59 smithi057 ceph-mon[89491]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:27:59.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:27:59 smithi057 ceph-mon[89491]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:00.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.063+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0a004f2a0 con 0x7ff0a0075b90 2024-02-28T23:28:00.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.064+0000 7ff09d600700 1 -- 172.21.15.57:0/1746581482 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff08c05a570 con 0x7ff0a0075b90 2024-02-28T23:28:00.063 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:28:00.064 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:28:00.065 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:28:00.065 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:28:00.065 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:28:00.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 msgr2=0x7ff08806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 0x7ff08806fd50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b5c0 tx=0x7ff094005a90 comp rx=0 tx=0).stop 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 msgr2=0x7ff0a019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d900 tx=0x7ff08c00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 shutdown_connections 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff08806d8a0 0x7ff08806fd50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0075280 0x7ff0a019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 --2- 172.21.15.57:0/1746581482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0a0075b90 0x7ff0a019c990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 >> 172.21.15.57:0/1746581482 conn(0x7ff0a00fdf90 msgr2=0x7ff0a00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 shutdown_connections 2024-02-28T23:28:00.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.067+0000 7ff0a7800700 1 -- 172.21.15.57:0/1746581482 wait complete. 2024-02-28T23:28:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:59 smithi084 ceph-mon[70440]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:27:59 smithi084 ceph-mon[70440]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 -- 172.21.15.57:0/2607437049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb530104130 msgr2=0x7fb530104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 --2- 172.21.15.57:0/2607437049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb530104130 0x7fb530104500 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb520009910 tx=0x7fb520009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 -- 172.21.15.57:0/2607437049 shutdown_connections 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 --2- 172.21.15.57:0/2607437049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5300ffbd0 0x7fb530100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 --2- 172.21.15.57:0/2607437049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb530104130 0x7fb530104500 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 -- 172.21.15.57:0/2607437049 >> 172.21.15.57:0/2607437049 conn(0x7fb530075130 msgr2=0x7fb530075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 -- 172.21.15.57:0/2607437049 shutdown_connections 2024-02-28T23:28:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.218+0000 7fb536c00700 1 -- 172.21.15.57:0/2607437049 wait complete. 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.219+0000 7fb536c00700 1 Processor -- start 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.219+0000 7fb536c00700 1 -- start start 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.219+0000 7fb536c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.219+0000 7fb536c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 0x7fb530196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb536c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb530196a70 con 0x7fb5300ffbd0 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb536c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019a800 con 0x7fb530104130 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56774/0 (socket says 172.21.15.57:56774) 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 -- 172.21.15.57:0/2579671384 learned_addr learned my addr 172.21.15.57:0/2579671384 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52f400700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 0x7fb530196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 -- 172.21.15.57:0/2579671384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 msgr2=0x7fb530196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 0x7fb530196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52fe00700 1 -- 172.21.15.57:0/2579671384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200095f0 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.220+0000 7fb52f400700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 0x7fb530196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb52fe00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5200092e0 tx=0x7fb52000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb52001d070 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb52000f460 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb520005e20 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53019aa80 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.221+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53019af70 con 0x7fb5300ffbd0 2024-02-28T23:28:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.222+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5301086a0 con 0x7fb5300ffbd0 2024-02-28T23:28:00.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.223+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb52000f5d0 con 0x7fb5300ffbd0 2024-02-28T23:28:00.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.224+0000 7fb52cc00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 0x7fb51406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.224+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb52008ea10 con 0x7fb5300ffbd0 2024-02-28T23:28:00.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.224+0000 7fb52f400700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 0x7fb51406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.225+0000 7fb52f400700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 0x7fb51406fd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.228+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb520058260 con 0x7fb5300ffbd0 2024-02-28T23:28:00.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.506+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb530108080 con 0x7fb5300ffbd0 2024-02-28T23:28:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.507+0000 7fb52cc00700 1 -- 172.21.15.57:0/2579671384 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fb52005b920 con 0x7fb5300ffbd0 2024-02-28T23:28:00.506 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:28:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:28:00.508 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:28:00.509 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 msgr2=0x7fb51406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 0x7fb51406fd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 msgr2=0x7fb530195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5200092e0 tx=0x7fb52000be20 comp rx=0 tx=0).stop 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 shutdown_connections 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb51406d8a0 0x7fb51406fd50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5300ffbd0 0x7fb530195e50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 --2- 172.21.15.57:0/2579671384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb530104130 0x7fb530196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 >> 172.21.15.57:0/2579671384 conn(0x7fb530075130 msgr2=0x7fb5300fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 shutdown_connections 2024-02-28T23:28:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.510+0000 7fb536c00700 1 -- 172.21.15.57:0/2579671384 wait complete. 2024-02-28T23:28:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.672+0000 7f2d53200700 1 -- 172.21.15.57:0/4012553290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 msgr2=0x7f2d541015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.672+0000 7f2d53200700 1 --2- 172.21.15.57:0/4012553290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d541015f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009910 tx=0x7f2d44009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.672+0000 7f2d53200700 1 -- 172.21.15.57:0/4012553290 shutdown_connections 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.672+0000 7f2d53200700 1 --2- 172.21.15.57:0/4012553290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 0x7f2d5410a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.673+0000 7f2d53200700 1 --2- 172.21.15.57:0/4012553290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d541015f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.673+0000 7f2d53200700 1 -- 172.21.15.57:0/4012553290 >> 172.21.15.57:0/4012553290 conn(0x7f2d540faa60 msgr2=0x7f2d540fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.673+0000 7f2d53200700 1 -- 172.21.15.57:0/4012553290 shutdown_connections 2024-02-28T23:28:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.673+0000 7f2d53200700 1 -- 172.21.15.57:0/4012553290 wait complete. 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 Processor -- start 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 -- start start 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 0x7f2d540ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540ff900 con 0x7f2d54101220 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d53200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540ffa40 con 0x7f2d54101b30 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.674+0000 7f2d51e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56790/0 (socket says 172.21.15.57:56790) 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 -- 172.21.15.57:0/533380472 learned_addr learned my addr 172.21.15.57:0/533380472 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:00.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51400700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 0x7f2d540ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 -- 172.21.15.57:0/533380472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 msgr2=0x7f2d540ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 0x7f2d540ff3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 -- 172.21.15.57:0/533380472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d440095f0 con 0x7f2d54101220 2024-02-28T23:28:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.675+0000 7f2d51e00700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d44006010 tx=0x7f2d4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.676+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4401d070 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.676+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d44004fb0 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.676+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d540ffcc0 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.676+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d44022ea0 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.676+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d541a2090 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.677+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d54107700 con 0x7f2d54101220 2024-02-28T23:28:00.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.678+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2d44022470 con 0x7f2d54101220 2024-02-28T23:28:00.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.679+0000 7f2d4aa00700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 0x7f2d3806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:00.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.679+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d4408eca0 con 0x7f2d54101220 2024-02-28T23:28:00.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.679+0000 7f2d51400700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 0x7f2d3806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:00.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.680+0000 7f2d51400700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 0x7f2d3806fd50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d54100670 tx=0x7f2d40005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:00.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.683+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d44058580 con 0x7f2d54101220 2024-02-28T23:28:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:00 smithi057 ceph-mon[89491]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:00 smithi057 ceph-mon[89491]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:00 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1746581482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:28:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:00 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2579671384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:28:00.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.937+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d54061190 con 0x7f2d3806d8a0 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.939+0000 7f2d4aa00700 1 -- 172.21.15.57:0/533380472 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d54061190 con 0x7f2d3806d8a0 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:28:00.938 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:28:00.939 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.941+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 msgr2=0x7f2d3806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.941+0000 7f2d53200700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 0x7f2d3806fd50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d54100670 tx=0x7f2d40005d50 comp rx=0 tx=0).stop 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.941+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 msgr2=0x7f2d540fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.941+0000 7f2d53200700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d44006010 tx=0x7f2d4400be20 comp rx=0 tx=0).stop 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.941+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 shutdown_connections 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d3806d8a0 0x7f2d3806fd50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54101220 0x7f2d540fee80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 --2- 172.21.15.57:0/533380472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54101b30 0x7f2d540ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:00.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 >> 172.21.15.57:0/533380472 conn(0x7f2d540faa60 msgr2=0x7f2d54104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:00.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 shutdown_connections 2024-02-28T23:28:00.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:00.942+0000 7f2d53200700 1 -- 172.21.15.57:0/533380472 wait complete. 2024-02-28T23:28:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:00 smithi084 ceph-mon[70440]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:00 smithi084 ceph-mon[70440]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:00 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1746581482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:28:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:00 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2579671384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 -- 172.21.15.57:0/1028021402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 msgr2=0x7f8768104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 --2- 172.21.15.57:0/1028021402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768104e50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8754009910 tx=0x7f8754009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 -- 172.21.15.57:0/1028021402 shutdown_connections 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 --2- 172.21.15.57:0/1028021402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768104e50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 --2- 172.21.15.57:0/1028021402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 0x7f87681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 -- 172.21.15.57:0/1028021402 >> 172.21.15.57:0/1028021402 conn(0x7f87680fbda0 msgr2=0x7f87680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:01.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 -- 172.21.15.57:0/1028021402 shutdown_connections 2024-02-28T23:28:01.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.103+0000 7f876e400700 1 -- 172.21.15.57:0/1028021402 wait complete. 2024-02-28T23:28:01.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.104+0000 7f876e400700 1 Processor -- start 2024-02-28T23:28:01.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.105+0000 7f876e400700 1 -- start start 2024-02-28T23:28:01.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.105+0000 7f876e400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 0x7f8768198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:01.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.105+0000 7f876e400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:01.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.105+0000 7f876e400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8768198d30 con 0x7f8768100e80 2024-02-28T23:28:01.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.105+0000 7f876e400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876819cac0 con 0x7f87681004e0 2024-02-28T23:28:01.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:01.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56800/0 (socket says 172.21.15.57:56800) 2024-02-28T23:28:01.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 -- 172.21.15.57:0/2368943473 learned_addr learned my addr 172.21.15.57:0/2368943473 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:01.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8767400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 0x7f8768198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 -- 172.21.15.57:0/2368943473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 msgr2=0x7f8768198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 0x7f8768198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.106+0000 7f8766a00700 1 -- 172.21.15.57:0/2368943473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540095f0 con 0x7f8768100e80 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f8766a00700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f875400bda0 tx=0x7f875400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f875401d070 con 0x7f8768100e80 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876819cd40 con 0x7f8768100e80 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8754005dc0 con 0x7f8768100e80 2024-02-28T23:28:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8754017400 con 0x7f8768100e80 2024-02-28T23:28:01.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.107+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876819d230 con 0x7f8768100e80 2024-02-28T23:28:01.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.109+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87681045d0 con 0x7f8768100e80 2024-02-28T23:28:01.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.110+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f875400f6f0 con 0x7f8768100e80 2024-02-28T23:28:01.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.111+0000 7f875fe00700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 0x7f874c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:01.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.111+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f875408e8d0 con 0x7f8768100e80 2024-02-28T23:28:01.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.111+0000 7f8767400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 0x7f874c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:01.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.112+0000 7f8767400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 0x7f874c06fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8758005950 tx=0x7f875800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:01.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.116+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8754058060 con 0x7f8768100e80 2024-02-28T23:28:01.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.432+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f876804f2a0 con 0x7f8768100e80 2024-02-28T23:28:01.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.433+0000 7f875fe00700 1 -- 172.21.15.57:0/2368943473 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f875405b720 con 0x7f8768100e80 2024-02-28T23:28:01.432 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:28:01.432 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:28:01.433 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:28:01.433 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:28:01.433 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:28:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 msgr2=0x7f874c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 0x7f874c06fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8758005950 tx=0x7f875800b410 comp rx=0 tx=0).stop 2024-02-28T23:28:01.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 msgr2=0x7f8768198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f875400bda0 tx=0x7f875400bdd0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 shutdown_connections 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f874c06d8a0 0x7f874c06fd50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87681004e0 0x7f8768198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 --2- 172.21.15.57:0/2368943473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8768100e80 0x7f8768198650 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 >> 172.21.15.57:0/2368943473 conn(0x7f87680fbda0 msgr2=0x7f87680fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 shutdown_connections 2024-02-28T23:28:01.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:01.436+0000 7f876e400700 1 -- 172.21.15.57:0/2368943473 wait complete. 2024-02-28T23:28:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:01 smithi057 ceph-mon[89491]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:01.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:01 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2368943473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:28:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:01 smithi084 ceph-mon[70440]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:02.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:01 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2368943473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:28:03.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:02 smithi057 ceph-mon[89491]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:02 smithi084 ceph-mon[70440]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:03 smithi057 ceph-mon[89491]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:03 smithi084 ceph-mon[70440]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:06.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:05 smithi057 ceph-mon[89491]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:05 smithi084 ceph-mon[70440]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:08.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:07 smithi057 ceph-mon[89491]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:08.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:07 smithi084 ceph-mon[70440]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:10.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:09 smithi057 ceph-mon[89491]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:10.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:09 smithi084 ceph-mon[70440]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:11 smithi057 ceph-mon[89491]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:11 smithi084 ceph-mon[70440]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:13 smithi057 ceph-mon[89491]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:13 smithi084 ceph-mon[70440]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:15 smithi057 ceph-mon[89491]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:15 smithi084 ceph-mon[70440]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:17 smithi057 ceph-mon[89491]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:18.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:17 smithi084 ceph-mon[70440]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:19 smithi057 ceph-mon[89491]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:20.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:19 smithi084 ceph-mon[70440]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:21 smithi057 ceph-mon[89491]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:22.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:21 smithi084 ceph-mon[70440]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:24.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:23 smithi057 ceph-mon[89491]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:23 smithi084 ceph-mon[70440]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:25 smithi057 ceph-mon[89491]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:25 smithi084 ceph-mon[70440]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:27 smithi057 ceph-mon[89491]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:27 smithi084 ceph-mon[70440]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:30.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:29 smithi057 ceph-mon[89491]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:30.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:29 smithi084 ceph-mon[70440]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:31.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 -- 172.21.15.57:0/409182548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a88106b80 msgr2=0x7f3a88106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 --2- 172.21.15.57:0/409182548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a88106b80 0x7f3a88106ff0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009910 tx=0x7f3a74009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 -- 172.21.15.57:0/409182548 shutdown_connections 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 --2- 172.21.15.57:0/409182548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a88106b80 0x7f3a88106ff0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 --2- 172.21.15.57:0/409182548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8810c660 0x7f3a8810ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 -- 172.21.15.57:0/409182548 >> 172.21.15.57:0/409182548 conn(0x7f3a88075e90 msgr2=0x7f3a880782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 -- 172.21.15.57:0/409182548 shutdown_connections 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.613+0000 7f3a8e000700 1 -- 172.21.15.57:0/409182548 wait complete. 2024-02-28T23:28:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.614+0000 7f3a8e000700 1 Processor -- start 2024-02-28T23:28:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.615+0000 7f3a8e000700 1 -- start start 2024-02-28T23:28:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.615+0000 7f3a8e000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a88106b80 0x7f3a8819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.615+0000 7f3a8e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.615+0000 7f3a8e000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819ce00 con 0x7f3a8810c660 2024-02-28T23:28:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.615+0000 7f3a8e000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881a0b90 con 0x7f3a88106b80 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46908/0 (socket says 172.21.15.57:46908) 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 -- 172.21.15.57:0/419307910 learned_addr learned my addr 172.21.15.57:0/419307910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a8cc00700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a88106b80 0x7f3a8819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 -- 172.21.15.57:0/419307910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a88106b80 msgr2=0x7f3a8819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a88106b80 0x7f3a8819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.616+0000 7f3a87e00700 1 -- 172.21.15.57:0/419307910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740095f0 con 0x7f3a8810c660 2024-02-28T23:28:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a87e00700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006010 tx=0x7f3a7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a7401d070 con 0x7f3a8810c660 2024-02-28T23:28:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a74005dc0 con 0x7f3a8810c660 2024-02-28T23:28:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a881a0e10 con 0x7f3a8810c660 2024-02-28T23:28:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a74017400 con 0x7f3a8810c660 2024-02-28T23:28:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.617+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a881a1300 con 0x7f3a8810c660 2024-02-28T23:28:31.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.619+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8810eb40 con 0x7f3a8810c660 2024-02-28T23:28:31.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.620+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3a74003a60 con 0x7f3a8810c660 2024-02-28T23:28:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.621+0000 7f3a85600700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 0x7f3a7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.621+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a7408e9c0 con 0x7f3a8810c660 2024-02-28T23:28:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.621+0000 7f3a8cc00700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 0x7f3a7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:31.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.622+0000 7f3a8cc00700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 0x7f3a7006fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005950 tx=0x7f3a7c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.626+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a74058170 con 0x7f3a8810c660 2024-02-28T23:28:31.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.875+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a881a15e0 con 0x7f3a7006d8a0 2024-02-28T23:28:31.874 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:31 smithi057 ceph-mon[89491]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.879+0000 7f3a85600700 1 -- 172.21.15.57:0/419307910 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3a881a15e0 con 0x7f3a7006d8a0 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 msgr2=0x7f3a7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 0x7f3a7006fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005950 tx=0x7f3a7c00b410 comp rx=0 tx=0).stop 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 msgr2=0x7f3a8819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006010 tx=0x7f3a7400bd70 comp rx=0 tx=0).stop 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 shutdown_connections 2024-02-28T23:28:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3a7006d8a0 0x7f3a7006fd50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a88106b80 0x7f3a8819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 --2- 172.21.15.57:0/419307910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8810c660 0x7f3a8819c720 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.881+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 >> 172.21.15.57:0/419307910 conn(0x7f3a88075e90 msgr2=0x7f3a880778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.882+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 shutdown_connections 2024-02-28T23:28:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:31.882+0000 7f3a8e000700 1 -- 172.21.15.57:0/419307910 wait complete. 2024-02-28T23:28:31.895 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.037+0000 7fd706600700 1 -- 172.21.15.57:0/2579509814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 msgr2=0x7fd7000689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.037+0000 7fd706600700 1 --2- 172.21.15.57:0/2579509814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd7000689b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009910 tx=0x7fd6ec009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 -- 172.21.15.57:0/2579509814 shutdown_connections 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 --2- 172.21.15.57:0/2579509814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd7000689b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 --2- 172.21.15.57:0/2579509814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 0x7fd7001065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 -- 172.21.15.57:0/2579509814 >> 172.21.15.57:0/2579509814 conn(0x7fd700074fa0 msgr2=0x7fd7000753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 -- 172.21.15.57:0/2579509814 shutdown_connections 2024-02-28T23:28:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.038+0000 7fd706600700 1 -- 172.21.15.57:0/2579509814 wait complete. 2024-02-28T23:28:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.042+0000 7fd706600700 1 Processor -- start 2024-02-28T23:28:32.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.042+0000 7fd706600700 1 -- start start 2024-02-28T23:28:32.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd706600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd706600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 0x7fd70019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd705200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd705200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46932/0 (socket says 172.21.15.57:46932) 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd705200700 1 -- 172.21.15.57:0/1054092150 learned_addr learned my addr 172.21.15.57:0/1054092150 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd706600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70019c500 con 0x7fd700068540 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001959b0 con 0x7fd700106200 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.044+0000 7fd6ffe00700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 0x7fd70019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd705200700 1 -- 172.21.15.57:0/1054092150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 msgr2=0x7fd70019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd705200700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 0x7fd70019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd705200700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0095f0 con 0x7fd700068540 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd705200700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400d8d0 tx=0x7fd6f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f4009940 con 0x7fd700068540 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6f4010460 con 0x7fd700068540 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700195c90 con 0x7fd700068540 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.045+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f400f5f0 con 0x7fd700068540 2024-02-28T23:28:32.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.046+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd700196260 con 0x7fd700068540 2024-02-28T23:28:32.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.047+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd700062380 con 0x7fd700068540 2024-02-28T23:28:32.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.048+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd6f40105d0 con 0x7fd700068540 2024-02-28T23:28:32.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.049+0000 7fd6fd600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 0x7fd6e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.049+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd6f408d5d0 con 0x7fd700068540 2024-02-28T23:28:32.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.049+0000 7fd6ffe00700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 0x7fd6e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.050+0000 7fd6ffe00700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 0x7fd6e806fd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7000fe110 tx=0x7fd6ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.054+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6f405a810 con 0x7fd700068540 2024-02-28T23:28:32.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:31 smithi084 ceph-mon[70440]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.301+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd700196610 con 0x7fd6e806d8a0 2024-02-28T23:28:32.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.304+0000 7fd6fd600700 1 -- 172.21.15.57:0/1054092150 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd700196610 con 0x7fd6e806d8a0 2024-02-28T23:28:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 msgr2=0x7fd6e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 0x7fd6e806fd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7000fe110 tx=0x7fd6ec01a040 comp rx=0 tx=0).stop 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 msgr2=0x7fd70019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400d8d0 tx=0x7fd6f400dc90 comp rx=0 tx=0).stop 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 shutdown_connections 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6e806d8a0 0x7fd6e806fd50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd700068540 0x7fd70019b930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 --2- 172.21.15.57:0/1054092150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd700106200 0x7fd70019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 >> 172.21.15.57:0/1054092150 conn(0x7fd700074fa0 msgr2=0x7fd7000fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.307+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 shutdown_connections 2024-02-28T23:28:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.308+0000 7fd706600700 1 -- 172.21.15.57:0/1054092150 wait complete. 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 -- 172.21.15.57:0/2588464456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6058100c60 msgr2=0x7f6058104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 --2- 172.21.15.57:0/2588464456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6058100c60 0x7f6058104c30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6040009910 tx=0x7f6040009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 -- 172.21.15.57:0/2588464456 shutdown_connections 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 --2- 172.21.15.57:0/2588464456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6058100c60 0x7f6058104c30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 --2- 172.21.15.57:0/2588464456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60581002c0 0x7f6058100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 -- 172.21.15.57:0/2588464456 >> 172.21.15.57:0/2588464456 conn(0x7f60580fbb80 msgr2=0x7f60580fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.467+0000 7f605d400700 1 -- 172.21.15.57:0/2588464456 shutdown_connections 2024-02-28T23:28:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.468+0000 7f605d400700 1 -- 172.21.15.57:0/2588464456 wait complete. 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.468+0000 7f605d400700 1 Processor -- start 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f605d400700 1 -- start start 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f605d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f605d400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6058100c60 0x7f6058073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f605d400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058073ad0 con 0x7f60581002c0 2024-02-28T23:28:32.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f605d400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058073c10 con 0x7f6058100c60 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f6057e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f6057e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46950/0 (socket says 172.21.15.57:46950) 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.469+0000 7f6057e00700 1 -- 172.21.15.57:0/4197499166 learned_addr learned my addr 172.21.15.57:0/4197499166 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f604fe00700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6058100c60 0x7f6058073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f6057e00700 1 -- 172.21.15.57:0/4197499166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6058100c60 msgr2=0x7f6058073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f6057e00700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6058100c60 0x7f6058073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f6057e00700 1 -- 172.21.15.57:0/4197499166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400095f0 con 0x7f60581002c0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f6057e00700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f604800c8a0 tx=0x7f604800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.470+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604800cea0 con 0x7f60581002c0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.471+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6048004830 con 0x7f60581002c0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.471+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60480055b0 con 0x7f60581002c0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.471+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60581a1de0 con 0x7f60581002c0 2024-02-28T23:28:32.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.471+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60581a2190 con 0x7f60581002c0 2024-02-28T23:28:32.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.472+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6058108630 con 0x7f60581002c0 2024-02-28T23:28:32.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.473+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f60480049a0 con 0x7f60581002c0 2024-02-28T23:28:32.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.474+0000 7f6055600700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 0x7f603c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.474+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f604808d260 con 0x7f60581002c0 2024-02-28T23:28:32.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.474+0000 7f604fe00700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 0x7f603c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.475+0000 7f604fe00700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 0x7f603c06fd50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6040006010 tx=0x7f6040005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.478+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6048056970 con 0x7f60581002c0 2024-02-28T23:28:32.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.708+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f605802cda0 con 0x7f603c06d8a0 2024-02-28T23:28:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.725+0000 7f6055600700 1 -- 172.21.15.57:0/4197499166 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f605802cda0 con 0x7f603c06d8a0 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (18m) 9m ago 31m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (28m) 9m ago 32m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (30m) 9m ago 30m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (14m) 9m ago 32m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (14m) 9m ago 30m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:28:32.724 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (17m) 9m ago 31m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (10m) 9m ago 24m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (9m) 9m ago 24m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (24m) 9m ago 24m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (9m) 9m ago 24m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (20m) 9m ago 33m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (19m) 9m ago 30m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (15m) 9m ago 34m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (15m) 9m ago 30m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (19m) 9m ago 32m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (19m) 9m ago 30m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (13m) 9m ago 29m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (13m) 9m ago 28m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (12m) 9m ago 28m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (11m) 9m ago 27m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:28:32.725 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (11m) 9m ago 27m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:28:32.726 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (10m) 9m ago 26m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:28:32.726 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (18m) 9m ago 31m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:28:32.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 msgr2=0x7f603c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 0x7f603c06fd50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6040006010 tx=0x7f6040005960 comp rx=0 tx=0).stop 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 msgr2=0x7f6058073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f604800c8a0 tx=0x7f604800cc60 comp rx=0 tx=0).stop 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 shutdown_connections 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f603c06d8a0 0x7f603c06fd50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60581002c0 0x7f6058073050 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 --2- 172.21.15.57:0/4197499166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6058100c60 0x7f6058073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 >> 172.21.15.57:0/4197499166 conn(0x7f60580fbb80 msgr2=0x7f60580fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 shutdown_connections 2024-02-28T23:28:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.728+0000 7f605d400700 1 -- 172.21.15.57:0/4197499166 wait complete. 2024-02-28T23:28:32.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 -- 172.21.15.57:0/3071982742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c100e80 msgr2=0x7fb59c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 --2- 172.21.15.57:0/3071982742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c100e80 0x7fb59c104e50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb584009910 tx=0x7fb584009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 -- 172.21.15.57:0/3071982742 shutdown_connections 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 --2- 172.21.15.57:0/3071982742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c100e80 0x7fb59c104e50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 --2- 172.21.15.57:0/3071982742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c1004e0 0x7fb59c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 -- 172.21.15.57:0/3071982742 >> 172.21.15.57:0/3071982742 conn(0x7fb59c0fbda0 msgr2=0x7fb59c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 -- 172.21.15.57:0/3071982742 shutdown_connections 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.888+0000 7fb5a1800700 1 -- 172.21.15.57:0/3071982742 wait complete. 2024-02-28T23:28:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.889+0000 7fb5a1800700 1 Processor -- start 2024-02-28T23:28:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.889+0000 7fb5a1800700 1 -- start start 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb5a1800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb5a1800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c100e80 0x7fb59c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb5a1800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c196b50 con 0x7fb59c1004e0 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb5a1800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19a8e0 con 0x7fb59c100e80 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46970/0 (socket says 172.21.15.57:46970) 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.890+0000 7fb59aa00700 1 -- 172.21.15.57:0/600598928 learned_addr learned my addr 172.21.15.57:0/600598928 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb593e00700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c100e80 0x7fb59c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb59aa00700 1 -- 172.21.15.57:0/600598928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c100e80 msgr2=0x7fb59c196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb59aa00700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c100e80 0x7fb59c196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb59aa00700 1 -- 172.21.15.57:0/600598928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840095f0 con 0x7fb59c1004e0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb59aa00700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c8a0 tx=0x7fb58c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c00cea0 con 0x7fb59c1004e0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58c004830 con 0x7fb59c1004e0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c0055b0 con 0x7fb59c1004e0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.891+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c19abc0 con 0x7fb59c1004e0 2024-02-28T23:28:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.892+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c19b110 con 0x7fb59c1004e0 2024-02-28T23:28:32.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.893+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c108850 con 0x7fb59c1004e0 2024-02-28T23:28:32.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.894+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb58c0049a0 con 0x7fb59c1004e0 2024-02-28T23:28:32.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.895+0000 7fb593400700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 0x7fb58006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:32.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.895+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb58c08d190 con 0x7fb59c1004e0 2024-02-28T23:28:32.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.895+0000 7fb593e00700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 0x7fb58006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:32.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.896+0000 7fb593e00700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 0x7fb58006fd50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb59c197550 tx=0x7fb58401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:32.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:32.899+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb58c056920 con 0x7fb59c1004e0 2024-02-28T23:28:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:32 smithi057 ceph-mon[89491]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:33.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:32 smithi084 ceph-mon[70440]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:33.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.223+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb59c197390 con 0x7fb59c1004e0 2024-02-28T23:28:33.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.224+0000 7fb593400700 1 -- 172.21.15.57:0/600598928 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb58c01d340 con 0x7fb59c1004e0 2024-02-28T23:28:33.222 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:28:33.222 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:28:33.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:28:33.224 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:28:33.224 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:28:33.224 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:28:33.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 msgr2=0x7fb58006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 0x7fb58006fd50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb59c197550 tx=0x7fb58401a040 comp rx=0 tx=0).stop 2024-02-28T23:28:33.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 msgr2=0x7fb59c195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c8a0 tx=0x7fb58c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 shutdown_connections 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb58006d8a0 0x7fb58006fd50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb59c1004e0 0x7fb59c195f30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 --2- 172.21.15.57:0/600598928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb59c100e80 0x7fb59c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.227+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 >> 172.21.15.57:0/600598928 conn(0x7fb59c0fbda0 msgr2=0x7fb59c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.228+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 shutdown_connections 2024-02-28T23:28:33.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.228+0000 7fb5a1800700 1 -- 172.21.15.57:0/600598928 wait complete. 2024-02-28T23:28:33.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 -- 172.21.15.57:0/3697531687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 msgr2=0x7f86f0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 --2- 172.21.15.57:0/3697531687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f0106ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009910 tx=0x7f86d8009c20 comp rx=0 tx=0).stop 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 -- 172.21.15.57:0/3697531687 shutdown_connections 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 --2- 172.21.15.57:0/3697531687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f0106ff0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 --2- 172.21.15.57:0/3697531687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 0x7f86f010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 -- 172.21.15.57:0/3697531687 >> 172.21.15.57:0/3697531687 conn(0x7f86f0076060 msgr2=0x7f86f0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 -- 172.21.15.57:0/3697531687 shutdown_connections 2024-02-28T23:28:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.377+0000 7f86f6400700 1 -- 172.21.15.57:0/3697531687 wait complete. 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.378+0000 7f86f6400700 1 Processor -- start 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.378+0000 7f86f6400700 1 -- start start 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86f6400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86f6400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 0x7f86f019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86f6400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f019d100 con 0x7f86f0106b80 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86f6400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01a0e90 con 0x7f86f010c8f0 2024-02-28T23:28:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86ef400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86ef400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46996/0 (socket says 172.21.15.57:46996) 2024-02-28T23:28:33.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.379+0000 7f86ef400700 1 -- 172.21.15.57:0/3039041096 learned_addr learned my addr 172.21.15.57:0/3039041096 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86e7e00700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 0x7f86f019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ef400700 1 -- 172.21.15.57:0/3039041096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 msgr2=0x7f86f019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ef400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 0x7f86f019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ef400700 1 -- 172.21.15.57:0/3039041096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80095f0 con 0x7f86f0106b80 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ef400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4b0 tx=0x7f86e000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e0011800 con 0x7f86f0106b80 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86e0011e20 con 0x7f86f0106b80 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e000f2e0 con 0x7f86f0106b80 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01a1170 con 0x7f86f0106b80 2024-02-28T23:28:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.380+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01a16c0 con 0x7f86f0106b80 2024-02-28T23:28:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.381+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f0109b20 con 0x7f86f0106b80 2024-02-28T23:28:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.382+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f86e0010380 con 0x7f86f0106b80 2024-02-28T23:28:33.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.383+0000 7f86ecc00700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 0x7f86d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.383+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f86e008d0f0 con 0x7f86f0106b80 2024-02-28T23:28:33.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.383+0000 7f86e7e00700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 0x7f86d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.384+0000 7f86e7e00700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 0x7f86d406fd50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000c00 tx=0x7f86d8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:33.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.388+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86e0056930 con 0x7f86f0106b80 2024-02-28T23:28:33.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.660+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f86f004f2a0 con 0x7f86f0106b80 2024-02-28T23:28:33.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.661+0000 7f86ecc00700 1 -- 172.21.15.57:0/3039041096 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f86e0059ff0 con 0x7f86f0106b80 2024-02-28T23:28:33.660 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:28:33.660 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:28:33.660 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:28:33.660 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:28:33.661 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:28:33.662 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:28:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 msgr2=0x7f86d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 0x7f86d406fd50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000c00 tx=0x7f86d8005fd0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 msgr2=0x7f86f019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4b0 tx=0x7f86e000b870 comp rx=0 tx=0).stop 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 shutdown_connections 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f86d406d8a0 0x7f86d406fd50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86f0106b80 0x7f86f019c4e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 --2- 172.21.15.57:0/3039041096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f010c8f0 0x7f86f019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 >> 172.21.15.57:0/3039041096 conn(0x7f86f0076060 msgr2=0x7f86f0077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 shutdown_connections 2024-02-28T23:28:33.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.664+0000 7f86f6400700 1 -- 172.21.15.57:0/3039041096 wait complete. 2024-02-28T23:28:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 -- 172.21.15.57:0/2558251718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 msgr2=0x7fa544108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 --2- 172.21.15.57:0/2558251718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544108840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa53400b0a0 tx=0x7fa53400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 -- 172.21.15.57:0/2558251718 shutdown_connections 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 --2- 172.21.15.57:0/2558251718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa5441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 --2- 172.21.15.57:0/2558251718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544108840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 -- 172.21.15.57:0/2558251718 >> 172.21.15.57:0/2558251718 conn(0x7fa5440fdf90 msgr2=0x7fa5441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 -- 172.21.15.57:0/2558251718 shutdown_connections 2024-02-28T23:28:33.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.827+0000 7fa54a800700 1 -- 172.21.15.57:0/2558251718 wait complete. 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.828+0000 7fa54a800700 1 Processor -- start 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa54a800700 1 -- start start 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa54a800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa544198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa54a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa54a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa544198f30 con 0x7fa544108470 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa54a800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419ccc0 con 0x7fa544102470 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa53aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa53aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47014/0 (socket says 172.21.15.57:47014) 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.829+0000 7fa53aa00700 1 -- 172.21.15.57:0/218517963 learned_addr learned my addr 172.21.15.57:0/218517963 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa543400700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa544198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa53aa00700 1 -- 172.21.15.57:0/218517963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 msgr2=0x7fa544198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:33.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa53aa00700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa544198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa53aa00700 1 -- 172.21.15.57:0/218517963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534009d00 con 0x7fa544108470 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa543400700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa544198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa53aa00700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00d8d0 tx=0x7fa52c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52c009940 con 0x7fa544108470 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54419cfa0 con 0x7fa544108470 2024-02-28T23:28:33.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.830+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52c010460 con 0x7fa544108470 2024-02-28T23:28:33.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.831+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52c01b620 con 0x7fa544108470 2024-02-28T23:28:33.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.831+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54419d4f0 con 0x7fa544108470 2024-02-28T23:28:33.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.832+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54410a9e0 con 0x7fa544108470 2024-02-28T23:28:33.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.833+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa52c0105d0 con 0x7fa544108470 2024-02-28T23:28:33.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.834+0000 7fa540c00700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 0x7fa52806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:33.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.834+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa52c08da10 con 0x7fa544108470 2024-02-28T23:28:33.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.835+0000 7fa543400700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 0x7fa52806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:33.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.835+0000 7fa543400700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 0x7fa52806fd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa534009480 tx=0x7fa534000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:33.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:33.838+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa52c0571a0 con 0x7fa544108470 2024-02-28T23:28:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/600598928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:28:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:33 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3039041096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/600598928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:28:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:33 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3039041096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:28:34.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.087+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa54419d710 con 0x7fa52806d8a0 2024-02-28T23:28:34.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.090+0000 7fa540c00700 1 -- 172.21.15.57:0/218517963 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa54419d710 con 0x7fa52806d8a0 2024-02-28T23:28:34.088 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:28:34.088 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:28:34.088 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:28:34.088 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:28:34.089 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 msgr2=0x7fa52806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 0x7fa52806fd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa534009480 tx=0x7fa534000f40 comp rx=0 tx=0).stop 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 msgr2=0x7fa544198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00d8d0 tx=0x7fa52c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 shutdown_connections 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52806d8a0 0x7fa52806fd50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa544102470 0x7fa544198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 --2- 172.21.15.57:0/218517963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa544108470 0x7fa544198850 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 >> 172.21.15.57:0/218517963 conn(0x7fa5440fdf90 msgr2=0x7fa5440ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:34.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 shutdown_connections 2024-02-28T23:28:34.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.093+0000 7fa54a800700 1 -- 172.21.15.57:0/218517963 wait complete. 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.240+0000 7fd7c4200700 1 -- 172.21.15.57:0/185493109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101420 msgr2=0x7fd7bc1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.240+0000 7fd7c4200700 1 --2- 172.21.15.57:0/185493109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101420 0x7fd7bc1017f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000b0a0 tx=0x7fd7b000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.240+0000 7fd7c4200700 1 -- 172.21.15.57:0/185493109 shutdown_connections 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.240+0000 7fd7c4200700 1 --2- 172.21.15.57:0/185493109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101d30 0x7fd7bc10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.241+0000 7fd7c4200700 1 --2- 172.21.15.57:0/185493109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101420 0x7fd7bc1017f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.241+0000 7fd7c4200700 1 -- 172.21.15.57:0/185493109 >> 172.21.15.57:0/185493109 conn(0x7fd7bc0fac80 msgr2=0x7fd7bc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.241+0000 7fd7c4200700 1 -- 172.21.15.57:0/185493109 shutdown_connections 2024-02-28T23:28:34.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.241+0000 7fd7c4200700 1 -- 172.21.15.57:0/185493109 wait complete. 2024-02-28T23:28:34.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 Processor -- start 2024-02-28T23:28:34.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 -- start start 2024-02-28T23:28:34.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 0x7fd7bc195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:34.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:34.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc196ae0 con 0x7fd7bc101d30 2024-02-28T23:28:34.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.242+0000 7fd7c4200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc19a870 con 0x7fd7bc101420 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47024/0 (socket says 172.21.15.57:47024) 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 -- 172.21.15.57:0/3415477228 learned_addr learned my addr 172.21.15.57:0/3415477228 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c1800700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 0x7fd7bc195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 -- 172.21.15.57:0/3415477228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 msgr2=0x7fd7bc195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 0x7fd7bc195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 -- 172.21.15.57:0/3415477228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0009d00 con 0x7fd7bc101d30 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c1800700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 0x7fd7bc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:28:34.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.243+0000 7fd7c0e00700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d900 tx=0x7fd7a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.244+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a8004a00 con 0x7fd7bc101d30 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.244+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7a80054e0 con 0x7fd7bc101d30 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.244+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc19ab50 con 0x7fd7bc101d30 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.244+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a8009d60 con 0x7fd7bc101d30 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.244+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc19b0a0 con 0x7fd7bc101d30 2024-02-28T23:28:34.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.245+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc1078d0 con 0x7fd7bc101d30 2024-02-28T23:28:34.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.246+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd7a8005020 con 0x7fd7bc101d30 2024-02-28T23:28:34.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.247+0000 7fd7ba000700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 0x7fd7a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:28:34.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.247+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd7a808d720 con 0x7fd7bc101d30 2024-02-28T23:28:34.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.247+0000 7fd7c1800700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 0x7fd7a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:28:34.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.248+0000 7fd7c1800700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 0x7fd7a406fd50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009480 tx=0x7fd7b0000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:28:34.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.251+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7a8056eb0 con 0x7fd7bc101d30 2024-02-28T23:28:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.560+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd7bc04f2a0 con 0x7fd7bc101d30 2024-02-28T23:28:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.561+0000 7fd7ba000700 1 -- 172.21.15.57:0/3415477228 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd7a805a570 con 0x7fd7bc101d30 2024-02-28T23:28:34.559 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:28:34.559 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:28:34.560 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:28:34.560 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:28:34.560 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:28:34.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 msgr2=0x7fd7a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 0x7fd7a406fd50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009480 tx=0x7fd7b0000f40 comp rx=0 tx=0).stop 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 msgr2=0x7fd7bc196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d900 tx=0x7fd7a800dcc0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 shutdown_connections 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd7a406d8a0 0x7fd7a406fd50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7bc101420 0x7fd7bc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 --2- 172.21.15.57:0/3415477228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7bc101d30 0x7fd7bc196400 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 >> 172.21.15.57:0/3415477228 conn(0x7fd7bc0fac80 msgr2=0x7fd7bc104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 shutdown_connections 2024-02-28T23:28:34.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:28:34.564+0000 7fd7c4200700 1 -- 172.21.15.57:0/3415477228 wait complete. 2024-02-28T23:28:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:34 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3415477228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:28:35.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:34 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3415477228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:28:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:35 smithi057 ceph-mon[89491]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:35 smithi057 ceph-mon[89491]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:36.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:35 smithi084 ceph-mon[70440]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:28:36.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:35 smithi084 ceph-mon[70440]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:38.131 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:37 smithi084 ceph-mon[70440]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:37 smithi057 ceph-mon[89491]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:40.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:39 smithi057 ceph-mon[89491]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:40.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:39 smithi084 ceph-mon[70440]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:42.238 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:41 smithi057 ceph-mon[89491]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:42.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:41 smithi084 ceph-mon[70440]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:44.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:43 smithi057 ceph-mon[89491]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:44.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:43 smithi084 ceph-mon[70440]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:46.120 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:45 smithi057 ceph-mon[89491]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:46.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:45 smithi084 ceph-mon[70440]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:48.298 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:47 smithi084 ceph-mon[70440]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:48.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:47 smithi057 ceph-mon[89491]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:28:50.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:49 smithi057 ceph-mon[89491]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:50.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:49 smithi084 ceph-mon[70440]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:52.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:51 smithi057 ceph-mon[89491]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:52.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:51 smithi084 ceph-mon[70440]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:54.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:53 smithi057 ceph-mon[89491]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:54.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:53 smithi084 ceph-mon[70440]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:56.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:55 smithi057 ceph-mon[89491]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:56.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:55 smithi084 ceph-mon[70440]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:57 smithi057 ceph-mon[89491]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:28:58.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:57 smithi084 ceph-mon[70440]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:00.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:28:59 smithi057 ceph-mon[89491]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:00.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:28:59 smithi084 ceph-mon[70440]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:02.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:01 smithi057 ceph-mon[89491]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:01 smithi084 ceph-mon[70440]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:03.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:02 smithi057 ceph-mon[89491]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:03.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:02 smithi084 ceph-mon[70440]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.740+0000 7f6a9b800700 1 -- 172.21.15.57:0/1093587951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 msgr2=0x7f6a940734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.740+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1093587951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a940734c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009910 tx=0x7f6a88009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.740+0000 7f6a9b800700 1 -- 172.21.15.57:0/1093587951 shutdown_connections 2024-02-28T23:29:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.740+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1093587951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 0x7f6a94110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.741+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1093587951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a940734c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.741+0000 7f6a9b800700 1 -- 172.21.15.57:0/1093587951 >> 172.21.15.57:0/1093587951 conn(0x7f6a940fbda0 msgr2=0x7f6a940fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.741+0000 7f6a9b800700 1 -- 172.21.15.57:0/1093587951 shutdown_connections 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.741+0000 7f6a9b800700 1 -- 172.21.15.57:0/1093587951 wait complete. 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.742+0000 7f6a9b800700 1 Processor -- start 2024-02-28T23:29:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.742+0000 7f6a9b800700 1 -- start start 2024-02-28T23:29:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a9b800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a9b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 0x7f6a9419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a9b800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9419d150 con 0x7f6a940730f0 2024-02-28T23:29:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a9b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941a0ee0 con 0x7f6a94073a00 2024-02-28T23:29:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46026/0 (socket says 172.21.15.57:46026) 2024-02-28T23:29:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.743+0000 7f6a98e00700 1 -- 172.21.15.57:0/1090724106 learned_addr learned my addr 172.21.15.57:0/1090724106 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.744+0000 7f6a93e00700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 0x7f6a9419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.744+0000 7f6a98e00700 1 -- 172.21.15.57:0/1090724106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 msgr2=0x7f6a9419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.744+0000 7f6a98e00700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 0x7f6a9419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.744+0000 7f6a98e00700 1 -- 172.21.15.57:0/1090724106 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880095f0 con 0x7f6a940730f0 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.744+0000 7f6a98e00700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006010 tx=0x7f6a8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.745+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8801d070 con 0x7f6a940730f0 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.745+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a88004fb0 con 0x7f6a940730f0 2024-02-28T23:29:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.745+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a941a11c0 con 0x7f6a940730f0 2024-02-28T23:29:04.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.745+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88022ea0 con 0x7f6a940730f0 2024-02-28T23:29:04.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.745+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a941a1710 con 0x7f6a940730f0 2024-02-28T23:29:04.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.746+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9410e4d0 con 0x7f6a940730f0 2024-02-28T23:29:04.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.747+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6a88022470 con 0x7f6a940730f0 2024-02-28T23:29:04.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.749+0000 7f6a91600700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 0x7f6a7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:04.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.749+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6a8808eb50 con 0x7f6a940730f0 2024-02-28T23:29:04.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.749+0000 7f6a93e00700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 0x7f6a7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:04.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.750+0000 7f6a93e00700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 0x7f6a7c06fd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:04.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:04.754+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a880584b0 con 0x7f6a940730f0 2024-02-28T23:29:05.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.007+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a941a1a40 con 0x7f6a7c06d8a0 2024-02-28T23:29:05.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.011+0000 7f6a91600700 1 -- 172.21.15.57:0/1090724106 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6a941a1a40 con 0x7f6a7c06d8a0 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 msgr2=0x7f6a7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 0x7f6a7c06fd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80008040 comp rx=0 tx=0).stop 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 msgr2=0x7f6a9419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006010 tx=0x7f6a8800be20 comp rx=0 tx=0).stop 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 shutdown_connections 2024-02-28T23:29:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7c06d8a0 0x7f6a7c06fd50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a940730f0 0x7f6a9419c530 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 --2- 172.21.15.57:0/1090724106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a94073a00 0x7f6a9419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 >> 172.21.15.57:0/1090724106 conn(0x7f6a940fbda0 msgr2=0x7f6a94102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 shutdown_connections 2024-02-28T23:29:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.014+0000 7f6a9b800700 1 -- 172.21.15.57:0/1090724106 wait complete. 2024-02-28T23:29:05.030 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:29:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 -- 172.21.15.57:0/3733240029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 msgr2=0x7fb258104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 --2- 172.21.15.57:0/3733240029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb258104b40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb244009910 tx=0x7fb244009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 -- 172.21.15.57:0/3733240029 shutdown_connections 2024-02-28T23:29:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 --2- 172.21.15.57:0/3733240029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb258104b40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 --2- 172.21.15.57:0/3733240029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 0x7fb2581005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 -- 172.21.15.57:0/3733240029 >> 172.21.15.57:0/3733240029 conn(0x7fb2580fba70 msgr2=0x7fb2580fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 -- 172.21.15.57:0/3733240029 shutdown_connections 2024-02-28T23:29:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.181+0000 7fb257e00700 1 -- 172.21.15.57:0/3733240029 wait complete. 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.182+0000 7fb257e00700 1 Processor -- start 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.182+0000 7fb257e00700 1 -- start start 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb257e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 0x7fb258197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb257e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb257e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258198980 con 0x7fb258100b70 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb257e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819c710 con 0x7fb2581001d0 2024-02-28T23:29:05.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43596/0 (socket says 172.21.15.57:43596) 2024-02-28T23:29:05.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 -- 172.21.15.57:0/3684625734 learned_addr learned my addr 172.21.15.57:0/3684625734 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:05.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256a00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 0x7fb258197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 -- 172.21.15.57:0/3684625734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 msgr2=0x7fb258197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 0x7fb258197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.183+0000 7fb256000700 1 -- 172.21.15.57:0/3684625734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440095f0 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb256000700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bda0 tx=0x7fb24400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb24401d070 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb24400f460 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25819c990 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb244017610 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.184+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25819ce80 con 0x7fb258100b70 2024-02-28T23:29:05.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.186+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2581042c0 con 0x7fb258100b70 2024-02-28T23:29:05.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.186+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb24400fa40 con 0x7fb258100b70 2024-02-28T23:29:05.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.187+0000 7fb25ca00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 0x7fb23c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.187+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb24408ea70 con 0x7fb258100b70 2024-02-28T23:29:05.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.188+0000 7fb256a00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 0x7fb23c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.188+0000 7fb256a00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 0x7fb23c06fd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb248005950 tx=0x7fb2480058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.192+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb244058180 con 0x7fb258100b70 2024-02-28T23:29:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:05 smithi057 ceph-mon[89491]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:05.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:05 smithi084 ceph-mon[70440]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:05.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.441+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb25802cd20 con 0x7fb23c06d8a0 2024-02-28T23:29:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.444+0000 7fb25ca00700 1 -- 172.21.15.57:0/3684625734 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb25802cd20 con 0x7fb23c06d8a0 2024-02-28T23:29:05.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 msgr2=0x7fb23c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 0x7fb23c06fd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb248005950 tx=0x7fb2480058e0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 msgr2=0x7fb2581982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bda0 tx=0x7fb24400be80 comp rx=0 tx=0).stop 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 shutdown_connections 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb23c06d8a0 0x7fb23c06fd50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2581001d0 0x7fb258197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 --2- 172.21.15.57:0/3684625734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb258100b70 0x7fb2581982a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 >> 172.21.15.57:0/3684625734 conn(0x7fb2580fba70 msgr2=0x7fb2580fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 shutdown_connections 2024-02-28T23:29:05.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.447+0000 7fb257e00700 1 -- 172.21.15.57:0/3684625734 wait complete. 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 -- 172.21.15.57:0/534584492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf81061b0 msgr2=0x7facf8106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 --2- 172.21.15.57:0/534584492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf81061b0 0x7facf8106580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7face8009910 tx=0x7face8009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 -- 172.21.15.57:0/534584492 shutdown_connections 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 --2- 172.21.15.57:0/534584492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf8068540 0x7facf80689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 --2- 172.21.15.57:0/534584492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf81061b0 0x7facf8106580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 -- 172.21.15.57:0/534584492 >> 172.21.15.57:0/534584492 conn(0x7facf8074fa0 msgr2=0x7facf80753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 -- 172.21.15.57:0/534584492 shutdown_connections 2024-02-28T23:29:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.601+0000 7facfd600700 1 -- 172.21.15.57:0/534584492 wait complete. 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.602+0000 7facfd600700 1 Processor -- start 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.602+0000 7facfd600700 1 -- start start 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facfd600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facfd600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf81061b0 0x7facf81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facfd600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf8198980 con 0x7facf8068540 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facfd600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf819c710 con 0x7facf81061b0 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43624/0 (socket says 172.21.15.57:43624) 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7e00700 1 -- 172.21.15.57:0/4260711347 learned_addr learned my addr 172.21.15.57:0/4260711347 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7e00700 1 -- 172.21.15.57:0/4260711347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf81061b0 msgr2=0x7facf81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7400700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf81061b0 0x7facf81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.603+0000 7facf7e00700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf81061b0 0x7facf81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facf7e00700 1 -- 172.21.15.57:0/4260711347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80095f0 con 0x7facf8068540 2024-02-28T23:29:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facf7e00700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7face801d070 con 0x7facf8068540 2024-02-28T23:29:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7face800f460 con 0x7facf8068540 2024-02-28T23:29:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7face8017630 con 0x7facf8068540 2024-02-28T23:29:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf819c990 con 0x7facf8068540 2024-02-28T23:29:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.604+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf819ceb0 con 0x7facf8068540 2024-02-28T23:29:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.605+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf8100eb0 con 0x7facf8068540 2024-02-28T23:29:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.606+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7face800f5d0 con 0x7facf8068540 2024-02-28T23:29:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.607+0000 7facf4c00700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 0x7facdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.607+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7face808ea90 con 0x7facf8068540 2024-02-28T23:29:05.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.608+0000 7facf7400700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 0x7facdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:05.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.608+0000 7facf7400700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 0x7facdc06fd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7face4005950 tx=0x7face40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:05.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.612+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7face80581a0 con 0x7facf8068540 2024-02-28T23:29:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.842+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7facf819d250 con 0x7facdc06d8a0 2024-02-28T23:29:05.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.874+0000 7facf4c00700 1 -- 172.21.15.57:0/4260711347 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7facf819d250 con 0x7facdc06d8a0 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (18m) 10m ago 32m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (28m) 10m ago 32m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (31m) 9m ago 31m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (14m) 10m ago 32m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (14m) 9m ago 31m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (18m) 10m ago 32m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:29:05.874 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (10m) 10m ago 25m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (10m) 10m ago 25m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (25m) 9m ago 25m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (9m) 9m ago 25m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (21m) 10m ago 34m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (20m) 9m ago 31m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (16m) 10m ago 34m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (15m) 9m ago 31m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (19m) 10m ago 32m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (19m) 9m ago 31m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (14m) 10m ago 30m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (13m) 10m ago 29m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (13m) 10m ago 28m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (12m) 9m ago 28m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (11m) 9m ago 27m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:29:05.875 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (11m) 9m ago 27m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:29:05.876 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (19m) 10m ago 31m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:29:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 msgr2=0x7facdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 0x7facdc06fd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7face4005950 tx=0x7face40058e0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 msgr2=0x7facf8197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face800be20 comp rx=0 tx=0).stop 2024-02-28T23:29:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 shutdown_connections 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7facdc06d8a0 0x7facdc06fd50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facf8068540 0x7facf8197d60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 --2- 172.21.15.57:0/4260711347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf81061b0 0x7facf81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.878+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 >> 172.21.15.57:0/4260711347 conn(0x7facf8074fa0 msgr2=0x7facf8107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.879+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 shutdown_connections 2024-02-28T23:29:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:05.879+0000 7facfd600700 1 -- 172.21.15.57:0/4260711347 wait complete. 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 -- 172.21.15.57:0/2312358584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab84103e40 msgr2=0x7fab841042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 --2- 172.21.15.57:0/2312358584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab84103e40 0x7fab841042b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fab70009910 tx=0x7fab70009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 -- 172.21.15.57:0/2312358584 shutdown_connections 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 --2- 172.21.15.57:0/2312358584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab84103e40 0x7fab841042b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 --2- 172.21.15.57:0/2312358584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab841025e0 0x7fab841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 -- 172.21.15.57:0/2312358584 >> 172.21.15.57:0/2312358584 conn(0x7fab840fde00 msgr2=0x7fab84100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 -- 172.21.15.57:0/2312358584 shutdown_connections 2024-02-28T23:29:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.025+0000 7fab83200700 1 -- 172.21.15.57:0/2312358584 wait complete. 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.026+0000 7fab83200700 1 Processor -- start 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.026+0000 7fab83200700 1 -- start start 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab83200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab83200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab84103e40 0x7fab84198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab83200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab84198b10 con 0x7fab841025e0 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab83200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8419c8a0 con 0x7fab84103e40 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab81e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab81e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43638/0 (socket says 172.21.15.57:43638) 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab81e00700 1 -- 172.21.15.57:0/2674162868 learned_addr learned my addr 172.21.15.57:0/2674162868 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.027+0000 7fab81400700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab84103e40 0x7fab84198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab81e00700 1 -- 172.21.15.57:0/2674162868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab84103e40 msgr2=0x7fab84198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab81e00700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab84103e40 0x7fab84198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab81e00700 1 -- 172.21.15.57:0/2674162868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700095f0 con 0x7fab841025e0 2024-02-28T23:29:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab81e00700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c8a0 tx=0x7fab7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7400cea0 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab74004830 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab740055b0 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8419cb80 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.028+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8419d0d0 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.029+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab84062380 con 0x7fab841025e0 2024-02-28T23:29:06.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.030+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fab740049a0 con 0x7fab841025e0 2024-02-28T23:29:06.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.031+0000 7fab7aa00700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 0x7fab6806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.031+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fab7408d190 con 0x7fab841025e0 2024-02-28T23:29:06.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.032+0000 7fab81400700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 0x7fab6806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.032+0000 7fab81400700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 0x7fab6806fd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab84199510 tx=0x7fab7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.036+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab74056920 con 0x7fab841025e0 2024-02-28T23:29:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:06 smithi057 ceph-mon[89491]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:06 smithi057 ceph-mon[89491]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:06 smithi057 ceph-mon[89491]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:06 smithi084 ceph-mon[70440]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:06 smithi084 ceph-mon[70440]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:06 smithi084 ceph-mon[70440]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:06.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.345+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab8419d4a0 con 0x7fab841025e0 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.346+0000 7fab7aa00700 1 -- 172.21.15.57:0/2674162868 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fab7401d340 con 0x7fab841025e0 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:06.345 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:29:06.346 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:29:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.348+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 msgr2=0x7fab6806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.348+0000 7fab83200700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 0x7fab6806fd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab84199510 tx=0x7fab7001a040 comp rx=0 tx=0).stop 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 msgr2=0x7fab84197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c8a0 tx=0x7fab7400cc60 comp rx=0 tx=0).stop 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 shutdown_connections 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab6806d8a0 0x7fab6806fd50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab841025e0 0x7fab84197ef0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 --2- 172.21.15.57:0/2674162868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab84103e40 0x7fab84198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 >> 172.21.15.57:0/2674162868 conn(0x7fab840fde00 msgr2=0x7fab840fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:06.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 shutdown_connections 2024-02-28T23:29:06.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.349+0000 7fab83200700 1 -- 172.21.15.57:0/2674162868 wait complete. 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 -- 172.21.15.57:0/100219441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 msgr2=0x7fd77010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 --2- 172.21.15.57:0/100219441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd77010c580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009940 tx=0x7fd75c009c50 comp rx=0 tx=0).stop 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 -- 172.21.15.57:0/100219441 shutdown_connections 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 --2- 172.21.15.57:0/100219441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd77010c580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 --2- 172.21.15.57:0/100219441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 0x7fd770073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 -- 172.21.15.57:0/100219441 >> 172.21.15.57:0/100219441 conn(0x7fd7700fbd40 msgr2=0x7fd7700fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 -- 172.21.15.57:0/100219441 shutdown_connections 2024-02-28T23:29:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.509+0000 7fd775c00700 1 -- 172.21.15.57:0/100219441 wait complete. 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 Processor -- start 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 -- start start 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 0x7fd770198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd770198c70 con 0x7fd7700739c0 2024-02-28T23:29:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd775c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77019ca00 con 0x7fd7700730b0 2024-02-28T23:29:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.511+0000 7fd76e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43644/0 (socket says 172.21.15.57:43644) 2024-02-28T23:29:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 -- 172.21.15.57:0/968424728 learned_addr learned my addr 172.21.15.57:0/968424728 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76ea00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 0x7fd770198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 -- 172.21.15.57:0/968424728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 msgr2=0x7fd770198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 0x7fd770198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 -- 172.21.15.57:0/968424728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0095f0 con 0x7fd7700739c0 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.512+0000 7fd76e000700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0055f0 tx=0x7fd75c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.513+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75c01d070 con 0x7fd7700739c0 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.513+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd75c00f460 con 0x7fd7700739c0 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.513+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75c021640 con 0x7fd7700739c0 2024-02-28T23:29:06.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.513+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77019cc80 con 0x7fd7700739c0 2024-02-28T23:29:06.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.513+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77019d170 con 0x7fd7700739c0 2024-02-28T23:29:06.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.514+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd770109c80 con 0x7fd7700739c0 2024-02-28T23:29:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.515+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd75c00fa20 con 0x7fd7700739c0 2024-02-28T23:29:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.516+0000 7fd767400700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.516+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd75c08e730 con 0x7fd7700739c0 2024-02-28T23:29:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.517+0000 7fd76ea00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.517+0000 7fd76ea00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd760005950 tx=0x7fd7600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.523+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd75c057ec0 con 0x7fd7700739c0 2024-02-28T23:29:06.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.797+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd77004f2a0 con 0x7fd7700739c0 2024-02-28T23:29:06.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.798+0000 7fd767400700 1 -- 172.21.15.57:0/968424728 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fd75c05b580 con 0x7fd7700739c0 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:29:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:29:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:06.801 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:06.801 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:29:06.801 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 msgr2=0x7fd75406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd760005950 tx=0x7fd7600058e0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 msgr2=0x7fd770198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0055f0 tx=0x7fd75c0056d0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 shutdown_connections 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7700730b0 0x7fd770198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 --2- 172.21.15.57:0/968424728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7700739c0 0x7fd770198590 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 >> 172.21.15.57:0/968424728 conn(0x7fd7700fbd40 msgr2=0x7fd770106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 shutdown_connections 2024-02-28T23:29:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.801+0000 7fd775c00700 1 -- 172.21.15.57:0/968424728 wait complete. 2024-02-28T23:29:06.803 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:29:06.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 -- 172.21.15.57:0/1929296180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c100e80 msgr2=0x7f7f8c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 --2- 172.21.15.57:0/1929296180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c100e80 0x7f7f8c104e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009910 tx=0x7f7f7c009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 -- 172.21.15.57:0/1929296180 shutdown_connections 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 --2- 172.21.15.57:0/1929296180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c100e80 0x7f7f8c104e50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 --2- 172.21.15.57:0/1929296180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 -- 172.21.15.57:0/1929296180 >> 172.21.15.57:0/1929296180 conn(0x7f7f8c0fbda0 msgr2=0x7f7f8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 -- 172.21.15.57:0/1929296180 shutdown_connections 2024-02-28T23:29:06.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.961+0000 7f7f90e00700 1 -- 172.21.15.57:0/1929296180 wait complete. 2024-02-28T23:29:06.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 Processor -- start 2024-02-28T23:29:06.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 -- start start 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c100e80 0x7f7f8c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c198d30 con 0x7f7f8c1004e0 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.963+0000 7f7f90e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c19cac0 con 0x7f7f8c100e80 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43666/0 (socket says 172.21.15.57:43666) 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 -- 172.21.15.57:0/782686909 learned_addr learned my addr 172.21.15.57:0/782686909 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:06.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f81400700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c100e80 0x7f7f8c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 -- 172.21.15.57:0/782686909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c100e80 msgr2=0x7f7f8c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:06.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c100e80 0x7f7f8c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:06.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.964+0000 7f7f89e00700 1 -- 172.21.15.57:0/782686909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0095f0 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f89e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d8d0 tx=0x7f7f7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f74009940 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c19cda0 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f74010460 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c19d2f0 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.965+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7401b620 con 0x7f7f8c1004e0 2024-02-28T23:29:06.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.966+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c1045d0 con 0x7f7f8c1004e0 2024-02-28T23:29:06.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.967+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7f740105d0 con 0x7f7f8c1004e0 2024-02-28T23:29:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.968+0000 7f7f83400700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 0x7f7f7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.968+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7f7408d920 con 0x7f7f8c1004e0 2024-02-28T23:29:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.969+0000 7f7f81400700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 0x7f7f7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:06.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.969+0000 7f7f81400700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 0x7f7f7006fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c199730 tx=0x7f7f7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:06.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:06.973+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f74057030 con 0x7f7f8c1004e0 2024-02-28T23:29:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:07 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2674162868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:29:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:07 smithi057 ceph-mon[89491]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:07 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/968424728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:29:07.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.224+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f8c19d5d0 con 0x7f7f7006d8a0 2024-02-28T23:29:07.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.226+0000 7f7f83400700 1 -- 172.21.15.57:0/782686909 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7f8c19d5d0 con 0x7f7f7006d8a0 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:29:07.226 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:29:07.227 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:29:07.227 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:29:07.227 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:29:07.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 msgr2=0x7f7f7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 0x7f7f7006fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c199730 tx=0x7f7f7c01a040 comp rx=0 tx=0).stop 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 msgr2=0x7f7f8c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d8d0 tx=0x7f7f7400dc90 comp rx=0 tx=0).stop 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 shutdown_connections 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7f7006d8a0 0x7f7f7006fd50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1004e0 0x7f7f8c198110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 --2- 172.21.15.57:0/782686909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f8c100e80 0x7f7f8c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.229+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 >> 172.21.15.57:0/782686909 conn(0x7f7f8c0fbda0 msgr2=0x7f7f8c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.230+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 shutdown_connections 2024-02-28T23:29:07.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.230+0000 7f7f90e00700 1 -- 172.21.15.57:0/782686909 wait complete. 2024-02-28T23:29:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:07 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2674162868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:29:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:07 smithi084 ceph-mon[70440]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:07 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/968424728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 -- 172.21.15.57:0/3065523387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 msgr2=0x7f65b01007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 --2- 172.21.15.57:0/3065523387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b01007c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009910 tx=0x7f65a0009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 -- 172.21.15.57:0/3065523387 shutdown_connections 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 --2- 172.21.15.57:0/3065523387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 --2- 172.21.15.57:0/3065523387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b01007c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.385+0000 7f65b5c00700 1 -- 172.21.15.57:0/3065523387 >> 172.21.15.57:0/3065523387 conn(0x7f65b00fbbf0 msgr2=0x7f65b00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.386+0000 7f65b5c00700 1 -- 172.21.15.57:0/3065523387 shutdown_connections 2024-02-28T23:29:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.386+0000 7f65b5c00700 1 -- 172.21.15.57:0/3065523387 wait complete. 2024-02-28T23:29:07.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 Processor -- start 2024-02-28T23:29:07.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 -- start start 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0198a30 con 0x7f65b01003f0 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65b5c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0198b70 con 0x7f65b0101c50 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.387+0000 7f65afe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43678/0 (socket says 172.21.15.57:43678) 2024-02-28T23:29:07.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 -- 172.21.15.57:0/2140853521 learned_addr learned my addr 172.21.15.57:0/2140853521 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 -- 172.21.15.57:0/2140853521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 msgr2=0x7f65b01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65af400700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 -- 172.21.15.57:0/2140853521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00095f0 con 0x7f65b01003f0 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65af400700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:29:07.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.388+0000 7f65afe00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b5c0 tx=0x7f65a000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.389+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a001d070 con 0x7f65b01003f0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.389+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65a000f460 con 0x7f65b01003f0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.389+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a0017610 con 0x7f65b01003f0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.389+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b019c960 con 0x7f65b01003f0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.389+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b019ced0 con 0x7f65b01003f0 2024-02-28T23:29:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.390+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b0062380 con 0x7f65b01003f0 2024-02-28T23:29:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.391+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f65a000f5d0 con 0x7f65b01003f0 2024-02-28T23:29:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.392+0000 7f65acc00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 0x7f659406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.392+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f65a008eb50 con 0x7f65b01003f0 2024-02-28T23:29:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.392+0000 7f65af400700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 0x7f659406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.393+0000 7f65af400700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 0x7f659406fd50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f659c005fd0 tx=0x7f659c005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.397+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65a00582e0 con 0x7f65b01003f0 2024-02-28T23:29:07.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.706+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f65b0199200 con 0x7f65b01003f0 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.706+0000 7f65acc00700 1 -- 172.21.15.57:0/2140853521 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f65a005b9a0 con 0x7f65b01003f0 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:29:07.706 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 msgr2=0x7f659406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 0x7f659406fd50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f659c005fd0 tx=0x7f659c005e20 comp rx=0 tx=0).stop 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 msgr2=0x7f65b0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b5c0 tx=0x7f65a000be00 comp rx=0 tx=0).stop 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 shutdown_connections 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659406d8a0 0x7f659406fd50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65b01003f0 0x7f65b0197ea0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 --2- 172.21.15.57:0/2140853521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65b0101c50 0x7f65b01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:07.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.709+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 >> 172.21.15.57:0/2140853521 conn(0x7f65b00fbbf0 msgr2=0x7f65b00fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:07.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.710+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 shutdown_connections 2024-02-28T23:29:07.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:07.710+0000 7f65b5c00700 1 -- 172.21.15.57:0/2140853521 wait complete. 2024-02-28T23:29:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:08 smithi057 ceph-mon[89491]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2140853521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:29:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:08 smithi084 ceph-mon[70440]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2140853521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:29:09.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:09 smithi057 ceph-mon[89491]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:09 smithi084 ceph-mon[70440]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:11 smithi057 ceph-mon[89491]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:11 smithi084 ceph-mon[70440]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:13 smithi057 ceph-mon[89491]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:13 smithi084 ceph-mon[70440]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:16.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:15 smithi057 ceph-mon[89491]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:16.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:15 smithi084 ceph-mon[70440]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:17 smithi057 ceph-mon[89491]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:18.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:17 smithi084 ceph-mon[70440]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:20.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:19 smithi057 ceph-mon[89491]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:19 smithi084 ceph-mon[70440]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:22.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:21 smithi057 ceph-mon[89491]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:22.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:21 smithi084 ceph-mon[70440]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:23 smithi057 ceph-mon[89491]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:23 smithi084 ceph-mon[70440]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:26.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:25 smithi057 ceph-mon[89491]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:26.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:25 smithi084 ceph-mon[70440]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:27 smithi057 ceph-mon[89491]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:27 smithi084 ceph-mon[70440]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:29.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:29 smithi084 ceph-mon[70440]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:30.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:29 smithi057 ceph-mon[89491]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:31 smithi057 ceph-mon[89491]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:32.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:31 smithi084 ceph-mon[70440]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:33 smithi057 ceph-mon[89491]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:33 smithi084 ceph-mon[70440]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:35 smithi057 ceph-mon[89491]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:35 smithi084 ceph-mon[70440]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 -- 172.21.15.57:0/933518766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2401024d0 msgr2=0x7fc2401028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 --2- 172.21.15.57:0/933518766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2401024d0 0x7fc2401028a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc230009910 tx=0x7fc230009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 -- 172.21.15.57:0/933518766 shutdown_connections 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 --2- 172.21.15.57:0/933518766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc240102e70 0x7fc240106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 --2- 172.21.15.57:0/933518766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2401024d0 0x7fc2401028a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 -- 172.21.15.57:0/933518766 >> 172.21.15.57:0/933518766 conn(0x7fc2400fdd90 msgr2=0x7fc2401001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 -- 172.21.15.57:0/933518766 shutdown_connections 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.888+0000 7fc23f200700 1 -- 172.21.15.57:0/933518766 wait complete. 2024-02-28T23:29:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.889+0000 7fc23f200700 1 Processor -- start 2024-02-28T23:29:37.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23f200700 1 -- start start 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2401024d0 0x7fc240197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401989f0 con 0x7fc240102e70 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23f200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc240198b30 con 0x7fc2401024d0 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44684/0 (socket says 172.21.15.57:44684) 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.890+0000 7fc23d400700 1 -- 172.21.15.57:0/2535132844 learned_addr learned my addr 172.21.15.57:0/2535132844 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc23de00700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2401024d0 0x7fc240197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc23d400700 1 -- 172.21.15.57:0/2535132844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2401024d0 msgr2=0x7fc240197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc23d400700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2401024d0 0x7fc240197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:37.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc23d400700 1 -- 172.21.15.57:0/2535132844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2300095f0 con 0x7fc240102e70 2024-02-28T23:29:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc23d400700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c960 tx=0x7fc22c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.891+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22c007a70 con 0x7fc240102e70 2024-02-28T23:29:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.892+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc22c007bd0 con 0x7fc240102e70 2024-02-28T23:29:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.892+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22c0186c0 con 0x7fc240102e70 2024-02-28T23:29:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.892+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24019c980 con 0x7fc240102e70 2024-02-28T23:29:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.892+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24019ce80 con 0x7fc240102e70 2024-02-28T23:29:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.893+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24010a7e0 con 0x7fc240102e70 2024-02-28T23:29:37.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.894+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc22c00faa0 con 0x7fc240102e70 2024-02-28T23:29:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.895+0000 7fc236a00700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 0x7fc22406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.895+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc22c08d5f0 con 0x7fc240102e70 2024-02-28T23:29:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.895+0000 7fc23de00700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 0x7fc22406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.896+0000 7fc23de00700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 0x7fc22406fd50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc230006010 tx=0x7fc23001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:37.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:37.900+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc22c05a830 con 0x7fc240102e70 2024-02-28T23:29:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:37 smithi057 ceph-mon[89491]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:37 smithi084 ceph-mon[70440]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:38.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.150+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc24002cca0 con 0x7fc22406d8a0 2024-02-28T23:29:38.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.153+0000 7fc236a00700 1 -- 172.21.15.57:0/2535132844 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc24002cca0 con 0x7fc22406d8a0 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 msgr2=0x7fc22406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 0x7fc22406fd50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc230006010 tx=0x7fc23001a040 comp rx=0 tx=0).stop 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 msgr2=0x7fc2401983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c960 tx=0x7fc22c00cd20 comp rx=0 tx=0).stop 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 shutdown_connections 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc22406d8a0 0x7fc22406fd50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2401024d0 0x7fc240197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 --2- 172.21.15.57:0/2535132844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc240102e70 0x7fc2401983a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.155+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 >> 172.21.15.57:0/2535132844 conn(0x7fc2400fdd90 msgr2=0x7fc240100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.156+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 shutdown_connections 2024-02-28T23:29:38.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.156+0000 7fc23f200700 1 -- 172.21.15.57:0/2535132844 wait complete. 2024-02-28T23:29:38.170 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 -- 172.21.15.57:0/3259962000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 msgr2=0x7f79c810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 --2- 172.21.15.57:0/3259962000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c810c580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009910 tx=0x7f79b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 -- 172.21.15.57:0/3259962000 shutdown_connections 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 --2- 172.21.15.57:0/3259962000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c810c580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 --2- 172.21.15.57:0/3259962000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 0x7f79c80734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.318+0000 7f79cca00700 1 -- 172.21.15.57:0/3259962000 >> 172.21.15.57:0/3259962000 conn(0x7f79c80fbda0 msgr2=0x7f79c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.319+0000 7f79cca00700 1 -- 172.21.15.57:0/3259962000 shutdown_connections 2024-02-28T23:29:38.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.319+0000 7f79cca00700 1 -- 172.21.15.57:0/3259962000 wait complete. 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 Processor -- start 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 -- start start 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 0x7f79c8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819c930 con 0x7f79c8073a00 2024-02-28T23:29:38.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79cca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819caa0 con 0x7f79c80730f0 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79bd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79bd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44698/0 (socket says 172.21.15.57:44698) 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.320+0000 7f79bd400700 1 -- 172.21.15.57:0/614826822 learned_addr learned my addr 172.21.15.57:0/614826822 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.321+0000 7f79c5e00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 0x7f79c8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.321+0000 7f79bd400700 1 -- 172.21.15.57:0/614826822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 msgr2=0x7f79c8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.321+0000 7f79bd400700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 0x7f79c8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.321+0000 7f79bd400700 1 -- 172.21.15.57:0/614826822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00095f0 con 0x7f79c8073a00 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.321+0000 7f79bd400700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f79b000f690 tx=0x7f79b000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.322+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b001d070 con 0x7f79c8073a00 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.322+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79b00050d0 con 0x7f79c8073a00 2024-02-28T23:29:38.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.322+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b0022470 con 0x7f79c8073a00 2024-02-28T23:29:38.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.322+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c819cd20 con 0x7f79c8073a00 2024-02-28T23:29:38.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.322+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c819d270 con 0x7f79c8073a00 2024-02-28T23:29:38.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.323+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c8109c80 con 0x7f79c8073a00 2024-02-28T23:29:38.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.324+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f79b0003680 con 0x7f79c8073a00 2024-02-28T23:29:38.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.325+0000 7f79bf400700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 0x7f79ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.325+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f79b008ea60 con 0x7f79c8073a00 2024-02-28T23:29:38.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.325+0000 7f79c5e00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 0x7f79ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.326+0000 7f79c5e00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 0x7f79ac06fd50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009380 tx=0x7f79b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:38.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.330+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79b0058170 con 0x7f79c8073a00 2024-02-28T23:29:38.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.579+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c8199350 con 0x7f79ac06d8a0 2024-02-28T23:29:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.580+0000 7f79bf400700 1 -- 172.21.15.57:0/614826822 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f79c8199350 con 0x7f79ac06d8a0 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.583+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 msgr2=0x7f79ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.583+0000 7f79cca00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 0x7f79ac06fd50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009380 tx=0x7f79b800b040 comp rx=0 tx=0).stop 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.583+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 msgr2=0x7f79c81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.583+0000 7f79cca00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f79b000f690 tx=0x7f79b000f770 comp rx=0 tx=0).stop 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 shutdown_connections 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f79ac06d8a0 0x7f79ac06fd50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c80730f0 0x7f79c8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 --2- 172.21.15.57:0/614826822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79c8073a00 0x7f79c81985f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 >> 172.21.15.57:0/614826822 conn(0x7f79c80fbda0 msgr2=0x7f79c8106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 shutdown_connections 2024-02-28T23:29:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.584+0000 7f79cca00700 1 -- 172.21.15.57:0/614826822 wait complete. 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 -- 172.21.15.57:0/3687834245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d941024d0 msgr2=0x7f9d941028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3687834245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d941024d0 0x7f9d941028a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009910 tx=0x7f9d84009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 -- 172.21.15.57:0/3687834245 shutdown_connections 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3687834245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d94102e70 0x7f9d94106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3687834245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d941024d0 0x7f9d941028a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 -- 172.21.15.57:0/3687834245 >> 172.21.15.57:0/3687834245 conn(0x7f9d940fdd90 msgr2=0x7f9d941001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:38.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 -- 172.21.15.57:0/3687834245 shutdown_connections 2024-02-28T23:29:38.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.746+0000 7f9d98e00700 1 -- 172.21.15.57:0/3687834245 wait complete. 2024-02-28T23:29:38.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.747+0000 7f9d98e00700 1 Processor -- start 2024-02-28T23:29:38.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.747+0000 7f9d98e00700 1 -- start start 2024-02-28T23:29:38.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d98e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d941024d0 0x7f9d94197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d98e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d98e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94198a80 con 0x7f9d94102e70 2024-02-28T23:29:38.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d98e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9419c810 con 0x7f9d941024d0 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d92a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d92a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44722/0 (socket says 172.21.15.57:44722) 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.748+0000 7f9d92a00700 1 -- 172.21.15.57:0/3496717856 learned_addr learned my addr 172.21.15.57:0/3496717856 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d93400700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d941024d0 0x7f9d94197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d92a00700 1 -- 172.21.15.57:0/3496717856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d941024d0 msgr2=0x7f9d94197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d92a00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d941024d0 0x7f9d94197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d92a00700 1 -- 172.21.15.57:0/3496717856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840095f0 con 0x7f9d94102e70 2024-02-28T23:29:38.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d92a00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d900 tx=0x7f9d8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.749+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d80009900 con 0x7f9d94102e70 2024-02-28T23:29:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.750+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d80010460 con 0x7f9d94102e70 2024-02-28T23:29:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.750+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9419caf0 con 0x7f9d94102e70 2024-02-28T23:29:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.750+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d8000f5f0 con 0x7f9d94102e70 2024-02-28T23:29:38.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.750+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9419d040 con 0x7f9d94102e70 2024-02-28T23:29:38.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.751+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9410a7e0 con 0x7f9d94102e70 2024-02-28T23:29:38.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.752+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9d800105d0 con 0x7f9d94102e70 2024-02-28T23:29:38.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.753+0000 7f9d8be00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 0x7f9d7806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:38.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.753+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9d8008d390 con 0x7f9d94102e70 2024-02-28T23:29:38.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.753+0000 7f9d93400700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 0x7f9d7806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:38.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.754+0000 7f9d93400700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 0x7f9d7806fd50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:38.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.758+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d80056b20 con 0x7f9d94102e70 2024-02-28T23:29:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:38.998+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d9419d320 con 0x7f9d7806d8a0 2024-02-28T23:29:39.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.037+0000 7f9d8be00700 1 -- 172.21.15.57:0/3496717856 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9d9419d320 con 0x7f9d7806d8a0 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (19m) 10m ago 33m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (29m) 10m ago 33m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (32m) 10m ago 32m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (15m) 10m ago 33m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (15m) 10m ago 31m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:29:39.036 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (18m) 10m ago 32m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (11m) 10m ago 25m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (10m) 10m ago 25m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (25m) 10m ago 25m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (10m) 10m ago 25m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (21m) 10m ago 34m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (20m) 10m ago 31m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (17m) 10m ago 35m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (16m) 10m ago 31m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (20m) 10m ago 33m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (20m) 10m ago 31m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (14m) 10m ago 30m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (14m) 10m ago 30m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (13m) 10m ago 29m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:29:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (13m) 10m ago 28m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:29:39.038 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (12m) 10m ago 28m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:29:39.038 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (11m) 10m ago 27m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:29:39.038 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (19m) 10m ago 32m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:29:39.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.041+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 msgr2=0x7f9d7806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.041+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 0x7f9d7806fd50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8401a040 comp rx=0 tx=0).stop 2024-02-28T23:29:39.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.041+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 msgr2=0x7f9d941983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.041+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d900 tx=0x7f9d8000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 shutdown_connections 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d7806d8a0 0x7f9d7806fd50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d941024d0 0x7f9d94197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 --2- 172.21.15.57:0/3496717856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d94102e70 0x7f9d941983a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 >> 172.21.15.57:0/3496717856 conn(0x7f9d940fdd90 msgr2=0x7f9d94100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 shutdown_connections 2024-02-28T23:29:39.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.042+0000 7f9d98e00700 1 -- 172.21.15.57:0/3496717856 wait complete. 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 -- 172.21.15.57:0/1398144746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05010c8f0 msgr2=0x7ff05010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 --2- 172.21.15.57:0/1398144746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05010c8f0 0x7ff05010ccc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff040009910 tx=0x7ff040009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 -- 172.21.15.57:0/1398144746 shutdown_connections 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 --2- 172.21.15.57:0/1398144746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff050106b80 0x7ff050106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 --2- 172.21.15.57:0/1398144746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05010c8f0 0x7ff05010ccc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 -- 172.21.15.57:0/1398144746 >> 172.21.15.57:0/1398144746 conn(0x7ff050076060 msgr2=0x7ff050078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:39.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 -- 172.21.15.57:0/1398144746 shutdown_connections 2024-02-28T23:29:39.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.205+0000 7ff057000700 1 -- 172.21.15.57:0/1398144746 wait complete. 2024-02-28T23:29:39.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.207+0000 7ff057000700 1 Processor -- start 2024-02-28T23:29:39.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.207+0000 7ff057000700 1 -- start start 2024-02-28T23:29:39.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.207+0000 7ff057000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.207+0000 7ff057000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff05010c8f0 0x7ff05019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff057000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019d120 con 0x7ff050106b80 2024-02-28T23:29:39.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff057000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501a0eb0 con 0x7ff05010c8f0 2024-02-28T23:29:39.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:39.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44734/0 (socket says 172.21.15.57:44734) 2024-02-28T23:29:39.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 -- 172.21.15.57:0/1643710303 learned_addr learned my addr 172.21.15.57:0/1643710303 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04f400700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff05010c8f0 0x7ff05019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 -- 172.21.15.57:0/1643710303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff05010c8f0 msgr2=0x7ff05019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff05010c8f0 0x7ff05019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.208+0000 7ff04fe00700 1 -- 172.21.15.57:0/1643710303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0400095f0 con 0x7ff050106b80 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff04fe00700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b5c0 tx=0x7ff04000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04001d070 con 0x7ff050106b80 2024-02-28T23:29:39.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff040005e00 con 0x7ff050106b80 2024-02-28T23:29:39.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0501a1130 con 0x7ff050106b80 2024-02-28T23:29:39.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff040017400 con 0x7ff050106b80 2024-02-28T23:29:39.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.209+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501a1620 con 0x7ff050106b80 2024-02-28T23:29:39.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.210+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff050109b20 con 0x7ff050106b80 2024-02-28T23:29:39.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.211+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff04000f460 con 0x7ff050106b80 2024-02-28T23:29:39.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.212+0000 7ff04cc00700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 0x7ff03406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.213+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff04008ea10 con 0x7ff050106b80 2024-02-28T23:29:39.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.213+0000 7ff04f400700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 0x7ff03406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:39.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.214+0000 7ff04f400700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 0x7ff03406fd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005950 tx=0x7ff03c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:39.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.217+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0400581a0 con 0x7ff050106b80 2024-02-28T23:29:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.549+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff05019d890 con 0x7ff050106b80 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.550+0000 7ff04cc00700 1 -- 172.21.15.57:0/1643710303 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff04005b860 con 0x7ff050106b80 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:29:39.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:29:39.549 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:29:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.553+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 msgr2=0x7ff03406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.553+0000 7ff057000700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 0x7ff03406fd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005950 tx=0x7ff03c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.553+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 msgr2=0x7ff05019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.553+0000 7ff057000700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b5c0 tx=0x7ff04000be20 comp rx=0 tx=0).stop 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 shutdown_connections 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff03406d8a0 0x7ff03406fd50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff050106b80 0x7ff05019c500 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 --2- 172.21.15.57:0/1643710303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff05010c8f0 0x7ff05019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 >> 172.21.15.57:0/1643710303 conn(0x7ff050076060 msgr2=0x7ff050077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 shutdown_connections 2024-02-28T23:29:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.554+0000 7ff057000700 1 -- 172.21.15.57:0/1643710303 wait complete. 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.708+0000 7fc955200700 1 -- 172.21.15.57:0/166745617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc950103040 msgr2=0x7fc950107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.708+0000 7fc955200700 1 --2- 172.21.15.57:0/166745617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc950103040 0x7fc950107010 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009910 tx=0x7fc93c009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 -- 172.21.15.57:0/166745617 shutdown_connections 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 --2- 172.21.15.57:0/166745617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc950103040 0x7fc950107010 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 --2- 172.21.15.57:0/166745617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9501026a0 0x7fc950102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 -- 172.21.15.57:0/166745617 >> 172.21.15.57:0/166745617 conn(0x7fc9500fdf40 msgr2=0x7fc950100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:39.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 -- 172.21.15.57:0/166745617 shutdown_connections 2024-02-28T23:29:39.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.709+0000 7fc955200700 1 -- 172.21.15.57:0/166745617 wait complete. 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 Processor -- start 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 -- start start 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc950198800 0x7fc95019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950198e10 con 0x7fc9501026a0 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.710+0000 7fc955200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950198f80 con 0x7fc950198800 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44746/0 (socket says 172.21.15.57:44746) 2024-02-28T23:29:39.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 -- 172.21.15.57:0/2414335715 learned_addr learned my addr 172.21.15.57:0/2414335715 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 -- 172.21.15.57:0/2414335715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc950198800 msgr2=0x7fc95019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc950198800 0x7fc95019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 -- 172.21.15.57:0/2414335715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0095f0 con 0x7fc9501026a0 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.711+0000 7fc94de00700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc94000d900 tx=0x7fc94000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.712+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc940009b10 con 0x7fc9501026a0 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.712+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc94000dea0 con 0x7fc9501026a0 2024-02-28T23:29:39.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.712+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94000f5f0 con 0x7fc9501026a0 2024-02-28T23:29:39.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.712+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95019d270 con 0x7fc9501026a0 2024-02-28T23:29:39.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.712+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95019d7c0 con 0x7fc9501026a0 2024-02-28T23:29:39.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.713+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95010aa10 con 0x7fc9501026a0 2024-02-28T23:29:39.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.714+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc940009c70 con 0x7fc9501026a0 2024-02-28T23:29:39.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.715+0000 7fc946a00700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 0x7fc93406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:39.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.715+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc94008d4f0 con 0x7fc9501026a0 2024-02-28T23:29:39.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.716+0000 7fc94d400700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 0x7fc93406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:39.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.716+0000 7fc94d400700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 0x7fc93406fd50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0059d0 tx=0x7fc93c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:39.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.720+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc940056c00 con 0x7fc9501026a0 2024-02-28T23:29:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:39 smithi057 ceph-mon[89491]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:39 smithi057 ceph-mon[89491]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:39 smithi057 ceph-mon[89491]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:39 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1643710303' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:29:39.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.993+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc95004f2a0 con 0x7fc9501026a0 2024-02-28T23:29:39.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.994+0000 7fc946a00700 1 -- 172.21.15.57:0/2414335715 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fc94005a2c0 con 0x7fc9501026a0 2024-02-28T23:29:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:29:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:29:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:29:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:29:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:29:39.994 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:29:39.995 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 msgr2=0x7fc93406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 0x7fc93406fd50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0059d0 tx=0x7fc93c005960 comp rx=0 tx=0).stop 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 msgr2=0x7fc9501982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc94000d900 tx=0x7fc94000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 shutdown_connections 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc93406d8a0 0x7fc93406fd50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9501026a0 0x7fc9501982c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 --2- 172.21.15.57:0/2414335715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc950198800 0x7fc95019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:39.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.996+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 >> 172.21.15.57:0/2414335715 conn(0x7fc9500fdf40 msgr2=0x7fc9500ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:39.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.997+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 shutdown_connections 2024-02-28T23:29:39.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:39.997+0000 7fc955200700 1 -- 172.21.15.57:0/2414335715 wait complete. 2024-02-28T23:29:39.997 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:29:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:39 smithi084 ceph-mon[70440]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:39 smithi084 ceph-mon[70440]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:39 smithi084 ceph-mon[70440]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:40.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:39 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1643710303' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 -- 172.21.15.57:0/2972258282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd8100e80 msgr2=0x7fdbd8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 --2- 172.21.15.57:0/2972258282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd8100e80 0x7fdbd8104e50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009910 tx=0x7fdbc0009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 -- 172.21.15.57:0/2972258282 shutdown_connections 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 --2- 172.21.15.57:0/2972258282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd8100e80 0x7fdbd8104e50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 --2- 172.21.15.57:0/2972258282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd81004e0 0x7fdbd81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.152+0000 7fdbde400700 1 -- 172.21.15.57:0/2972258282 >> 172.21.15.57:0/2972258282 conn(0x7fdbd80fbda0 msgr2=0x7fdbd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.153+0000 7fdbde400700 1 -- 172.21.15.57:0/2972258282 shutdown_connections 2024-02-28T23:29:40.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.153+0000 7fdbde400700 1 -- 172.21.15.57:0/2972258282 wait complete. 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 Processor -- start 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 -- start start 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 0x7fdbd8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8198d50 con 0x7fdbd81004e0 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbde400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd819cae0 con 0x7fdbd8100e80 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44762/0 (socket says 172.21.15.57:44762) 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.154+0000 7fdbd7400700 1 -- 172.21.15.57:0/1659196019 learned_addr learned my addr 172.21.15.57:0/1659196019 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbcfe00700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 0x7fdbd8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbd7400700 1 -- 172.21.15.57:0/1659196019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 msgr2=0x7fdbd8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbd7400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 0x7fdbd8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbd7400700 1 -- 172.21.15.57:0/1659196019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc00095f0 con 0x7fdbd81004e0 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbcfe00700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 0x7fdbd8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:29:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbd7400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c8a0 tx=0x7fdbc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.155+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbc800cea0 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.156+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbc8004830 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.156+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd819cdc0 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.156+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbc80055b0 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.156+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd819d310 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.157+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd81045d0 con 0x7fdbd81004e0 2024-02-28T23:29:40.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.158+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdbc80049a0 con 0x7fdbd81004e0 2024-02-28T23:29:40.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.158+0000 7fdbd4c00700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 0x7fdbbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.159+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdbc808d180 con 0x7fdbd81004e0 2024-02-28T23:29:40.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.159+0000 7fdbcfe00700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 0x7fdbbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.159+0000 7fdbcfe00700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 0x7fdbbc06fd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006010 tx=0x7fdbc000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:40.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.163+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbc8056910 con 0x7fdbd81004e0 2024-02-28T23:29:40.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.401+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbd819d5e0 con 0x7fdbbc06d8a0 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.403+0000 7fdbd4c00700 1 -- 172.21.15.57:0/1659196019 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdbd819d5e0 con 0x7fdbbc06d8a0 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:29:40.402 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:29:40.403 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:29:40.403 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:29:40.403 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:29:40.403 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:29:40.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 msgr2=0x7fdbbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 0x7fdbbc06fd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006010 tx=0x7fdbc000b540 comp rx=0 tx=0).stop 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 msgr2=0x7fdbd8198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c8a0 tx=0x7fdbc800cc60 comp rx=0 tx=0).stop 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 shutdown_connections 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdbbc06d8a0 0x7fdbbc06fd50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbd81004e0 0x7fdbd8198130 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 --2- 172.21.15.57:0/1659196019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd8100e80 0x7fdbd8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.406+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 >> 172.21.15.57:0/1659196019 conn(0x7fdbd80fbda0 msgr2=0x7fdbd80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.407+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 shutdown_connections 2024-02-28T23:29:40.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.407+0000 7fdbde400700 1 -- 172.21.15.57:0/1659196019 wait complete. 2024-02-28T23:29:40.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 -- 172.21.15.57:0/959587291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c106b80 msgr2=0x7f3b3c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 --2- 172.21.15.57:0/959587291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c106b80 0x7f3b3c106ff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009910 tx=0x7f3b28009c20 comp rx=0 tx=0).stop 2024-02-28T23:29:40.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 -- 172.21.15.57:0/959587291 shutdown_connections 2024-02-28T23:29:40.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 --2- 172.21.15.57:0/959587291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c106b80 0x7f3b3c106ff0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 --2- 172.21.15.57:0/959587291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 -- 172.21.15.57:0/959587291 >> 172.21.15.57:0/959587291 conn(0x7f3b3c076060 msgr2=0x7f3b3c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:40.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 -- 172.21.15.57:0/959587291 shutdown_connections 2024-02-28T23:29:40.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.553+0000 7f3b43c00700 1 -- 172.21.15.57:0/959587291 wait complete. 2024-02-28T23:29:40.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.554+0000 7f3b43c00700 1 Processor -- start 2024-02-28T23:29:40.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.554+0000 7f3b43c00700 1 -- start start 2024-02-28T23:29:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b43c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c106b80 0x7f3b3c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b43c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b43c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c19d060 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b43c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1a0df0 con 0x7f3b3c106b80 2024-02-28T23:29:40.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b3be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b3be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44778/0 (socket says 172.21.15.57:44778) 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.555+0000 7f3b3be00700 1 -- 172.21.15.57:0/885380672 learned_addr learned my addr 172.21.15.57:0/885380672 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b41200700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c106b80 0x7f3b3c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b3be00700 1 -- 172.21.15.57:0/885380672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c106b80 msgr2=0x7f3b3c19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b3be00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c106b80 0x7f3b3c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b3be00700 1 -- 172.21.15.57:0/885380672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280095f0 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b3be00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b5c0 tx=0x7f3b2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b2801d070 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b28005e00 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c1a1070 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.556+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b28017400 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.557+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c1a1560 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.558+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c10ecc0 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.559+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3b2800f460 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.559+0000 7f3b39600700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 0x7f3b2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:29:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.560+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3b2808eb00 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.560+0000 7f3b41200700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 0x7f3b2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:29:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.560+0000 7f3b41200700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 0x7f3b2406fd50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005950 tx=0x7f3b300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:29:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.564+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b28058210 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.880+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3b3c04f2a0 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.881+0000 7f3b39600700 1 -- 172.21.15.57:0/885380672 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3b2805b8d0 con 0x7f3b3c10c8f0 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:29:40.879 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.883+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 msgr2=0x7f3b2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.883+0000 7f3b43c00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 0x7f3b2406fd50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005950 tx=0x7f3b300058e0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.883+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 msgr2=0x7f3b3c19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.883+0000 7f3b43c00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b5c0 tx=0x7f3b2800be20 comp rx=0 tx=0).stop 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 shutdown_connections 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b2406d8a0 0x7f3b2406fd50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b3c106b80 0x7f3b3c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 --2- 172.21.15.57:0/885380672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b3c10c8f0 0x7f3b3c19c980 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 >> 172.21.15.57:0/885380672 conn(0x7f3b3c076060 msgr2=0x7f3b3c077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:29:40.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 shutdown_connections 2024-02-28T23:29:40.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:29:40.884+0000 7f3b43c00700 1 -- 172.21.15.57:0/885380672 wait complete. 2024-02-28T23:29:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:40 smithi057 ceph-mon[89491]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:40 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2414335715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:29:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:40 smithi084 ceph-mon[70440]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:40 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2414335715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:29:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:41 smithi057 ceph-mon[89491]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:41 smithi057 ceph-mon[89491]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:41 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/885380672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:29:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:41 smithi084 ceph-mon[70440]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:29:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:41 smithi084 ceph-mon[70440]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:42.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:41 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/885380672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:29:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:43 smithi057 ceph-mon[89491]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:43 smithi084 ceph-mon[70440]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:45 smithi057 ceph-mon[89491]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:45 smithi084 ceph-mon[70440]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:47 smithi057 ceph-mon[89491]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:47 smithi084 ceph-mon[70440]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:29:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:49 smithi057 ceph-mon[89491]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:49 smithi084 ceph-mon[70440]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:51 smithi057 ceph-mon[89491]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:52.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:51 smithi084 ceph-mon[70440]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:53 smithi084 ceph-mon[70440]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:54.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:53 smithi057 ceph-mon[89491]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:56.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:55 smithi057 ceph-mon[89491]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:56.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:55 smithi084 ceph-mon[70440]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:57 smithi057 ceph-mon[89491]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:58.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:57 smithi084 ceph-mon[70440]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:29:59.998 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:29:59 smithi057 ceph-mon[89491]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:00.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:29:59.998+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:30:00.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:29:59 smithi084 ceph-mon[70440]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:01.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:00 smithi057 ceph-mon[89491]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:30:01.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:00 smithi084 ceph-mon[70440]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:30:02.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:01 smithi057 ceph-mon[89491]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:01 smithi084 ceph-mon[70440]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:03 smithi057 ceph-mon[89491]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:03 smithi084 ceph-mon[70440]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:06.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:05 smithi084 ceph-mon[70440]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:05 smithi057 ceph-mon[89491]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:07 smithi057 ceph-mon[89491]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:08.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:07 smithi084 ceph-mon[70440]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:09 smithi057 ceph-mon[89491]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:10.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:09 smithi084 ceph-mon[70440]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 -- 172.21.15.57:0/1400787191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 msgr2=0x7feefc10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 --2- 172.21.15.57:0/1400787191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc10ca30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feef0009910 tx=0x7feef0009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 -- 172.21.15.57:0/1400787191 shutdown_connections 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 --2- 172.21.15.57:0/1400787191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 0x7feefc106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 --2- 172.21.15.57:0/1400787191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc10ca30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 -- 172.21.15.57:0/1400787191 >> 172.21.15.57:0/1400787191 conn(0x7feefc075e90 msgr2=0x7feefc0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 -- 172.21.15.57:0/1400787191 shutdown_connections 2024-02-28T23:30:11.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.060+0000 7fef02a00700 1 -- 172.21.15.57:0/1400787191 wait complete. 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.061+0000 7fef02a00700 1 Processor -- start 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.061+0000 7fef02a00700 1 -- start start 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef02a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 0x7feefc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef02a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef02a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19cea0 con 0x7feefc10c660 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef02a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1a0c30 con 0x7feefc106b80 2024-02-28T23:30:11.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef00c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef00c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34264/0 (socket says 172.21.15.57:34264) 2024-02-28T23:30:11.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef00c00700 1 -- 172.21.15.57:0/1010545691 learned_addr learned my addr 172.21.15.57:0/1010545691 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.062+0000 7fef01600700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 0x7feefc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7fef00c00700 1 -- 172.21.15.57:0/1010545691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 msgr2=0x7feefc19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7fef00c00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 0x7feefc19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7fef00c00700 1 -- 172.21.15.57:0/1010545691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00095f0 con 0x7feefc10c660 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7fef00c00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feee800d8d0 tx=0x7feee800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee8009940 con 0x7feefc10c660 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feee8010460 con 0x7feefc10c660 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc1a0f10 con 0x7feefc10c660 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.063+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee800f5f0 con 0x7feefc10c660 2024-02-28T23:30:11.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.064+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1a1460 con 0x7feefc10c660 2024-02-28T23:30:11.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.065+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc10eb40 con 0x7feefc10c660 2024-02-28T23:30:11.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.066+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7feee80105d0 con 0x7feefc10c660 2024-02-28T23:30:11.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.067+0000 7feefa000700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 0x7feee406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.067+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feee808d4c0 con 0x7feefc10c660 2024-02-28T23:30:11.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.067+0000 7fef01600700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 0x7feee406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.068+0000 7fef01600700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 0x7feee406fd50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feef0006010 tx=0x7feef001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.072+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feee8056c50 con 0x7feefc10c660 2024-02-28T23:30:11.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.323+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feefc1a16d0 con 0x7feee406d8a0 2024-02-28T23:30:11.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.326+0000 7feefa000700 1 -- 172.21.15.57:0/1010545691 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feefc1a16d0 con 0x7feee406d8a0 2024-02-28T23:30:11.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 msgr2=0x7feee406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 0x7feee406fd50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feef0006010 tx=0x7feef001a040 comp rx=0 tx=0).stop 2024-02-28T23:30:11.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 msgr2=0x7feefc19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feee800d8d0 tx=0x7feee800dc90 comp rx=0 tx=0).stop 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 shutdown_connections 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feee406d8a0 0x7feee406fd50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc106b80 0x7feefc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 --2- 172.21.15.57:0/1010545691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feefc10c660 0x7feefc19c7c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 >> 172.21.15.57:0/1010545691 conn(0x7feefc075e90 msgr2=0x7feefc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 shutdown_connections 2024-02-28T23:30:11.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.329+0000 7fef02a00700 1 -- 172.21.15.57:0/1010545691 wait complete. 2024-02-28T23:30:11.344 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 -- 172.21.15.57:0/2452841167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e80739a0 msgr2=0x7f13e810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 --2- 172.21.15.57:0/2452841167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e80739a0 0x7f13e810c5e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009940 tx=0x7f13d4009c50 comp rx=0 tx=0).stop 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 -- 172.21.15.57:0/2452841167 shutdown_connections 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 --2- 172.21.15.57:0/2452841167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e80739a0 0x7f13e810c5e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 --2- 172.21.15.57:0/2452841167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8073090 0x7f13e8073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 -- 172.21.15.57:0/2452841167 >> 172.21.15.57:0/2452841167 conn(0x7f13e80fbda0 msgr2=0x7f13e80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:11.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 -- 172.21.15.57:0/2452841167 shutdown_connections 2024-02-28T23:30:11.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.496+0000 7f13ed600700 1 -- 172.21.15.57:0/2452841167 wait complete. 2024-02-28T23:30:11.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.497+0000 7f13ed600700 1 Processor -- start 2024-02-28T23:30:11.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.497+0000 7f13ed600700 1 -- start start 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13ed600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13ed600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80739a0 0x7f13e81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13ed600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8198dd0 con 0x7f13e8073090 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13ed600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e819cb60 con 0x7f13e80739a0 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13e6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34280/0 (socket says 172.21.15.57:34280) 2024-02-28T23:30:11.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.498+0000 7f13e6a00700 1 -- 172.21.15.57:0/3759480170 learned_addr learned my addr 172.21.15.57:0/3759480170 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13e6000700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80739a0 0x7f13e81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13e6a00700 1 -- 172.21.15.57:0/3759480170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80739a0 msgr2=0x7f13e81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13e6a00700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80739a0 0x7f13e81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13e6a00700 1 -- 172.21.15.57:0/3759480170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40095f0 con 0x7f13e8073090 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13e6a00700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d8d0 tx=0x7f13d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d8009940 con 0x7f13e8073090 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13d8010460 con 0x7f13e8073090 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e819ce40 con 0x7f13e8073090 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.499+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d800f5f0 con 0x7f13e8073090 2024-02-28T23:30:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.500+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e819d390 con 0x7f13e8073090 2024-02-28T23:30:11.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.501+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e8109ce0 con 0x7f13e8073090 2024-02-28T23:30:11.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.502+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f13d80105d0 con 0x7f13e8073090 2024-02-28T23:30:11.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.502+0000 7f13df400700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 0x7f13cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.503+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f13d808d4c0 con 0x7f13e8073090 2024-02-28T23:30:11.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.503+0000 7f13e6000700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 0x7f13cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.504+0000 7f13e6000700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 0x7f13cc06fd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13d400b870 tx=0x7f13d4005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.507+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13d8056c50 con 0x7f13e8073090 2024-02-28T23:30:11.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.758+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e8199650 con 0x7f13cc06d8a0 2024-02-28T23:30:11.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.761+0000 7f13df400700 1 -- 172.21.15.57:0/3759480170 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f13d400bbb0 con 0x7f13cc06d8a0 2024-02-28T23:30:11.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 msgr2=0x7f13cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 0x7f13cc06fd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13d400b870 tx=0x7f13d4005fb0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 msgr2=0x7f13e81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d8d0 tx=0x7f13d800dc90 comp rx=0 tx=0).stop 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 shutdown_connections 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13cc06d8a0 0x7f13cc06fd50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e8073090 0x7f13e81981b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 --2- 172.21.15.57:0/3759480170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80739a0 0x7f13e81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.765+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 >> 172.21.15.57:0/3759480170 conn(0x7f13e80fbda0 msgr2=0x7f13e8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.766+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 shutdown_connections 2024-02-28T23:30:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.766+0000 7f13ed600700 1 -- 172.21.15.57:0/3759480170 wait complete. 2024-02-28T23:30:11.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 -- 172.21.15.57:0/567793104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd41004e0 msgr2=0x7fcfd41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 --2- 172.21.15.57:0/567793104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd41004e0 0x7fcfd41008b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 -- 172.21.15.57:0/567793104 shutdown_connections 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 --2- 172.21.15.57:0/567793104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd4100e80 0x7fcfd4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 --2- 172.21.15.57:0/567793104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd41004e0 0x7fcfd41008b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 -- 172.21.15.57:0/567793104 >> 172.21.15.57:0/567793104 conn(0x7fcfd40fbda0 msgr2=0x7fcfd40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 -- 172.21.15.57:0/567793104 shutdown_connections 2024-02-28T23:30:11.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.928+0000 7fcfdb600700 1 -- 172.21.15.57:0/567793104 wait complete. 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.929+0000 7fcfdb600700 1 Processor -- start 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.929+0000 7fcfdb600700 1 -- start start 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfdb600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd41004e0 0x7fcfd4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfdb600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfdb600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4196bd0 con 0x7fcfd4100e80 2024-02-28T23:30:11.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfdb600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419a960 con 0x7fcfd41004e0 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.930+0000 7fcfd8c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd41004e0 0x7fcfd4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34302/0 (socket says 172.21.15.57:34302) 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 -- 172.21.15.57:0/1095241148 learned_addr learned my addr 172.21.15.57:0/1095241148 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 -- 172.21.15.57:0/1095241148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd41004e0 msgr2=0x7fcfd4195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd41004e0 0x7fcfd4195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:11.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 -- 172.21.15.57:0/1095241148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80095f0 con 0x7fcfd4100e80 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.931+0000 7fcfd3e00700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000b480 tx=0x7fcfc000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.932+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc0011800 con 0x7fcfd4100e80 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.932+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc0011e20 con 0x7fcfd4100e80 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.932+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd419ac40 con 0x7fcfd4100e80 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.932+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc0010ea0 con 0x7fcfd4100e80 2024-02-28T23:30:11.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.932+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419b190 con 0x7fcfd4100e80 2024-02-28T23:30:11.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.933+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd41087f0 con 0x7fcfd4100e80 2024-02-28T23:30:11.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.934+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcfc0010380 con 0x7fcfd4100e80 2024-02-28T23:30:11.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.935+0000 7fcfd1600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:11.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.935+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcfc008cff0 con 0x7fcfd4100e80 2024-02-28T23:30:11.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.936+0000 7fcfd8c00700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:11.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.936+0000 7fcfd8c00700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006010 tx=0x7fcfc801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:11.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:11.940+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcfc00567c0 con 0x7fcfd4100e80 2024-02-28T23:30:12.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:11 smithi057 ceph-mon[89491]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:12.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.177+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd419b480 con 0x7fcfbc06d8a0 2024-02-28T23:30:12.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.211+0000 7fcfd1600700 1 -- 172.21.15.57:0/1095241148 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcfd419b480 con 0x7fcfbc06d8a0 2024-02-28T23:30:12.210 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:30:12.210 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (19m) 11m ago 33m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (29m) 11m ago 34m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (32m) 10m ago 32m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (15m) 11m ago 33m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (15m) 10m ago 32m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (19m) 11m ago 33m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (11m) 11m ago 26m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (11m) 11m ago 26m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (26m) 10m ago 26m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (10m) 10m ago 26m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (22m) 11m ago 35m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (21m) 10m ago 32m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (17m) 11m ago 35m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (16m) 10m ago 32m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (20m) 11m ago 33m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:30:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (20m) 10m ago 32m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (15m) 11m ago 31m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (14m) 11m ago 30m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (14m) 11m ago 29m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (13m) 10m ago 29m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (12m) 10m ago 28m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (12m) 10m ago 28m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:30:12.212 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (20m) 11m ago 32m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.213+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 msgr2=0x7fcfbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006010 tx=0x7fcfc801a040 comp rx=0 tx=0).stop 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 msgr2=0x7fcfd41964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000b480 tx=0x7fcfc000b790 comp rx=0 tx=0).stop 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 shutdown_connections 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd41004e0 0x7fcfd4195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 --2- 172.21.15.57:0/1095241148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd4100e80 0x7fcfd41964f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 >> 172.21.15.57:0/1095241148 conn(0x7fcfd40fbda0 msgr2=0x7fcfd40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 shutdown_connections 2024-02-28T23:30:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.214+0000 7fcfdb600700 1 -- 172.21.15.57:0/1095241148 wait complete. 2024-02-28T23:30:12.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:11 smithi084 ceph-mon[70440]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 -- 172.21.15.57:0/1814826541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101d30 msgr2=0x7fa81410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 --2- 172.21.15.57:0/1814826541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101d30 0x7fa81410a230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009910 tx=0x7fa7fc009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 -- 172.21.15.57:0/1814826541 shutdown_connections 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 --2- 172.21.15.57:0/1814826541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101d30 0x7fa81410a230 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 --2- 172.21.15.57:0/1814826541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101420 0x7fa8141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 -- 172.21.15.57:0/1814826541 >> 172.21.15.57:0/1814826541 conn(0x7fa8140fac80 msgr2=0x7fa8140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:12.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 -- 172.21.15.57:0/1814826541 shutdown_connections 2024-02-28T23:30:12.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.379+0000 7fa81b000700 1 -- 172.21.15.57:0/1814826541 wait complete. 2024-02-28T23:30:12.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.380+0000 7fa81b000700 1 Processor -- start 2024-02-28T23:30:12.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.380+0000 7fa81b000700 1 -- start start 2024-02-28T23:30:12.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.380+0000 7fa81b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.380+0000 7fa81b000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101d30 0x7fa814198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa81b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814198d10 con 0x7fa814101420 2024-02-28T23:30:12.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa81b000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81419caa0 con 0x7fa814101d30 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34318/0 (socket says 172.21.15.57:34318) 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 -- 172.21.15.57:0/3188578050 learned_addr learned my addr 172.21.15.57:0/3188578050 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa80be00700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101d30 0x7fa814198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 -- 172.21.15.57:0/3188578050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101d30 msgr2=0x7fa814198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101d30 0x7fa814198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.381+0000 7fa813e00700 1 -- 172.21.15.57:0/3188578050 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0095f0 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa813e00700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa80400d8d0 tx=0x7fa80400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa804009940 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa804010460 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80400f5f0 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81419cd80 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.382+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81419d2d0 con 0x7fa814101420 2024-02-28T23:30:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.383+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814109d00 con 0x7fa814101420 2024-02-28T23:30:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.384+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa8040105d0 con 0x7fa814101420 2024-02-28T23:30:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.386+0000 7fa811600700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 0x7fa7f806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.386+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa80408d4c0 con 0x7fa814101420 2024-02-28T23:30:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.386+0000 7fa80be00700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 0x7fa7f806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.387+0000 7fa80be00700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 0x7fa7f806fda0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa814199710 tx=0x7fa7fc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:12.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.390+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa804056c50 con 0x7fa814101420 2024-02-28T23:30:12.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.712+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa81404f2a0 con 0x7fa814101420 2024-02-28T23:30:12.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.713+0000 7fa811600700 1 -- 172.21.15.57:0/3188578050 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa80405a310 con 0x7fa814101420 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:12.713 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:30:12.714 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:30:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 msgr2=0x7fa7f806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 0x7fa7f806fda0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa814199710 tx=0x7fa7fc01a040 comp rx=0 tx=0).stop 2024-02-28T23:30:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 msgr2=0x7fa8141980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa80400d8d0 tx=0x7fa80400dc90 comp rx=0 tx=0).stop 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 shutdown_connections 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f806d8f0 0x7fa7f806fda0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814101420 0x7fa8141980f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 --2- 172.21.15.57:0/3188578050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa814101d30 0x7fa814198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.717+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 >> 172.21.15.57:0/3188578050 conn(0x7fa8140fac80 msgr2=0x7fa8140ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.718+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 shutdown_connections 2024-02-28T23:30:12.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.718+0000 7fa81b000700 1 -- 172.21.15.57:0/3188578050 wait complete. 2024-02-28T23:30:12.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 -- 172.21.15.57:0/161391758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 msgr2=0x7fefbc108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 --2- 172.21.15.57:0/161391758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc108890 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fefac009910 tx=0x7fefac009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 -- 172.21.15.57:0/161391758 shutdown_connections 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 --2- 172.21.15.57:0/161391758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 0x7fefbc102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 --2- 172.21.15.57:0/161391758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc108890 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 -- 172.21.15.57:0/161391758 >> 172.21.15.57:0/161391758 conn(0x7fefbc0fdfa0 msgr2=0x7fefbc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 -- 172.21.15.57:0/161391758 shutdown_connections 2024-02-28T23:30:12.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.884+0000 7fefc0c00700 1 -- 172.21.15.57:0/161391758 wait complete. 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.885+0000 7fefc0c00700 1 Processor -- start 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefc0c00700 1 -- start start 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefc0c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 0x7fefbc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefc0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefc0c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc198de0 con 0x7fefbc1084c0 2024-02-28T23:30:12.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefc0c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc19cb70 con 0x7fefbc1024c0 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.886+0000 7fefb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34338/0 (socket says 172.21.15.57:34338) 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 -- 172.21.15.57:0/830898309 learned_addr learned my addr 172.21.15.57:0/830898309 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb9e00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 0x7fefbc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 -- 172.21.15.57:0/830898309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 msgr2=0x7fefbc1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:12.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 0x7fefbc1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 -- 172.21.15.57:0/830898309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0095f0 con 0x7fefbc1084c0 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.887+0000 7fefb3e00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b4b0 tx=0x7fefa400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.888+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa4011800 con 0x7fefbc1084c0 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.888+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefa4011e20 con 0x7fefbc1084c0 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.888+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa400f2e0 con 0x7fefbc1084c0 2024-02-28T23:30:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.888+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc19ce50 con 0x7fefbc1084c0 2024-02-28T23:30:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.888+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc19d3a0 con 0x7fefbc1084c0 2024-02-28T23:30:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.889+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc10aa30 con 0x7fefbc1084c0 2024-02-28T23:30:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.890+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fefa4011960 con 0x7fefbc1084c0 2024-02-28T23:30:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.891+0000 7fefb2a00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 0x7fefa006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.891+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fefa408cca0 con 0x7fefbc1084c0 2024-02-28T23:30:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.891+0000 7fefb9e00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 0x7fefa006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.892+0000 7fefb9e00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 0x7fefa006fd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefac00b5c0 tx=0x7fefac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:12.896+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fefa4091070 con 0x7fefbc1084c0 2024-02-28T23:30:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:12 smithi057 ceph-mon[89491]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:12 smithi057 ceph-mon[89491]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:12 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3188578050' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:30:13.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.179+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fefbc04f2a0 con 0x7fefbc1084c0 2024-02-28T23:30:13.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.180+0000 7fefb2a00700 1 -- 172.21.15.57:0/830898309 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fefa4015740 con 0x7fefbc1084c0 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:30:13.180 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:30:13.181 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:30:13.182 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 msgr2=0x7fefa006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 0x7fefa006fd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefac00b5c0 tx=0x7fefac01a040 comp rx=0 tx=0).stop 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 msgr2=0x7fefbc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b4b0 tx=0x7fefa400b870 comp rx=0 tx=0).stop 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 shutdown_connections 2024-02-28T23:30:13.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fefa006d8a0 0x7fefa006fd50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefbc1024c0 0x7fefbc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 --2- 172.21.15.57:0/830898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefbc1084c0 0x7fefbc198700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.183+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 >> 172.21.15.57:0/830898309 conn(0x7fefbc0fdfa0 msgr2=0x7fefbc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.184+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 shutdown_connections 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.184+0000 7fefc0c00700 1 -- 172.21.15.57:0/830898309 wait complete. 2024-02-28T23:30:13.184 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:30:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:12 smithi084 ceph-mon[70440]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:12 smithi084 ceph-mon[70440]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:12 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3188578050' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:30:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 -- 172.21.15.57:0/483597069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 msgr2=0x7fc7a00ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 --2- 172.21.15.57:0/483597069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a00ff580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc790009910 tx=0x7fc790009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 -- 172.21.15.57:0/483597069 shutdown_connections 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 --2- 172.21.15.57:0/483597069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 0x7fc7a0103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 --2- 172.21.15.57:0/483597069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a00ff580 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 -- 172.21.15.57:0/483597069 >> 172.21.15.57:0/483597069 conn(0x7fc7a00faa30 msgr2=0x7fc7a00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 -- 172.21.15.57:0/483597069 shutdown_connections 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.352+0000 7fc7a5200700 1 -- 172.21.15.57:0/483597069 wait complete. 2024-02-28T23:30:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 Processor -- start 2024-02-28T23:30:13.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 -- start start 2024-02-28T23:30:13.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 0x7fc7a019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a019c5c0 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc7a5200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0195a70 con 0x7fc7a00ffb50 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.354+0000 7fc79f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34360/0 (socket says 172.21.15.57:34360) 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79f400700 1 -- 172.21.15.57:0/4010431617 learned_addr learned my addr 172.21.15.57:0/4010431617 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:13.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79ea00700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 0x7fc7a019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79f400700 1 -- 172.21.15.57:0/4010431617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 msgr2=0x7fc7a019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79f400700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 0x7fc7a019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.355+0000 7fc79f400700 1 -- 172.21.15.57:0/4010431617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900095f0 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc79f400700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b5c0 tx=0x7fc79000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79001d070 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc790005e00 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a0195cf0 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc790017400 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.356+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a01961e0 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.358+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a0062380 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.359+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc79000f6f0 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.359+0000 7fc797e00700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 0x7fc78406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.360+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc79008ea50 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.360+0000 7fc79ea00700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 0x7fc78406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.361+0000 7fc79ea00700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 0x7fc78406fd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0103e60 tx=0x7fc78c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:13.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.365+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc79005bc90 con 0x7fc7a00ff1b0 2024-02-28T23:30:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.606+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a0196580 con 0x7fc78406d8a0 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.608+0000 7fc797e00700 1 -- 172.21.15.57:0/4010431617 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc7a0196580 con 0x7fc78406d8a0 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:30:13.607 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:30:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 msgr2=0x7fc78406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 0x7fc78406fd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0103e60 tx=0x7fc78c00b410 comp rx=0 tx=0).stop 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 msgr2=0x7fc7a019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b5c0 tx=0x7fc79000bd70 comp rx=0 tx=0).stop 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 shutdown_connections 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc78406d8a0 0x7fc78406fd50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7a00ff1b0 0x7fc7a019b9f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 --2- 172.21.15.57:0/4010431617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a00ffb50 0x7fc7a019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 >> 172.21.15.57:0/4010431617 conn(0x7fc7a00faa30 msgr2=0x7fc7a0068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.611+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 shutdown_connections 2024-02-28T23:30:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.612+0000 7fc7a5200700 1 -- 172.21.15.57:0/4010431617 wait complete. 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 -- 172.21.15.57:0/2591607809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 msgr2=0x7f13d8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 --2- 172.21.15.57:0/2591607809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8104e50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009910 tx=0x7f13c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 -- 172.21.15.57:0/2591607809 shutdown_connections 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 --2- 172.21.15.57:0/2591607809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8104e50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 --2- 172.21.15.57:0/2591607809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.767+0000 7f13dd000700 1 -- 172.21.15.57:0/2591607809 >> 172.21.15.57:0/2591607809 conn(0x7f13d80fbda0 msgr2=0x7f13d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.768+0000 7f13dd000700 1 -- 172.21.15.57:0/2591607809 shutdown_connections 2024-02-28T23:30:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.768+0000 7f13dd000700 1 -- 172.21.15.57:0/2591607809 wait complete. 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 Processor -- start 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 -- start start 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8073ad0 con 0x7f13d8100e80 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.769+0000 7f13dd000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8073c10 con 0x7f13d81004e0 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34368/0 (socket says 172.21.15.57:34368) 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 -- 172.21.15.57:0/2600496589 learned_addr learned my addr 172.21.15.57:0/2600496589 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13d5e00700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 -- 172.21.15.57:0/2600496589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 msgr2=0x7f13d8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 -- 172.21.15.57:0/2600496589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c00095f0 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13d5e00700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.770+0000 7f13cd400700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bda0 tx=0x7f13c000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.771+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c001d070 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.771+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13c000f460 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.771+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8073e90 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.771+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0017610 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.771+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81a2430 con 0x7f13d8100e80 2024-02-28T23:30:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.772+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8108850 con 0x7f13d8100e80 2024-02-28T23:30:13.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.773+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f13c000fa40 con 0x7f13d8100e80 2024-02-28T23:30:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.774+0000 7f13cf400700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 0x7f13bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.774+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f13c008e9a0 con 0x7f13d8100e80 2024-02-28T23:30:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.774+0000 7f13d5e00700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 0x7f13bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:13.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.775+0000 7f13d5e00700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 0x7f13bc06fd50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:13.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:13.779+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13c0058130 con 0x7f13d8100e80 2024-02-28T23:30:14.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.103+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f13d804f2a0 con 0x7f13d8100e80 2024-02-28T23:30:14.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.104+0000 7f13cf400700 1 -- 172.21.15.57:0/2600496589 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f13c005b7f0 con 0x7f13d8100e80 2024-02-28T23:30:14.103 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:30:14.103 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:30:14.104 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:30:14.104 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:30:14.104 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 msgr2=0x7f13bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 0x7f13bc06fd50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8005ee0 comp rx=0 tx=0).stop 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 msgr2=0x7f13d8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bda0 tx=0x7f13c000be80 comp rx=0 tx=0).stop 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 shutdown_connections 2024-02-28T23:30:14.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13bc06d8a0 0x7f13bc06fd50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:14.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d81004e0 0x7f13d8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:14.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 --2- 172.21.15.57:0/2600496589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13d8100e80 0x7f13d8073590 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:14.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.107+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 >> 172.21.15.57:0/2600496589 conn(0x7f13d80fbda0 msgr2=0x7f13d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:14.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.108+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 shutdown_connections 2024-02-28T23:30:14.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:14.108+0000 7f13dd000700 1 -- 172.21.15.57:0/2600496589 wait complete. 2024-02-28T23:30:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:13 smithi057 ceph-mon[89491]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:13 smithi057 ceph-mon[89491]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:13 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/830898309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:30:14.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:13 smithi084 ceph-mon[70440]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:14.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:13 smithi084 ceph-mon[70440]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:14.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:13 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/830898309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:30:15.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:14 smithi057 ceph-mon[89491]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:15.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:14 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2600496589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:30:15.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:14 smithi084 ceph-mon[70440]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:15.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:14 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2600496589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:30:16.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:15 smithi057 ceph-mon[89491]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:16.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:15 smithi084 ceph-mon[70440]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:18.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:17 smithi057 ceph-mon[89491]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:17 smithi084 ceph-mon[70440]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:20.310 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:19 smithi057 ceph-mon[89491]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:19 smithi084 ceph-mon[70440]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:21 smithi057 ceph-mon[89491]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:21.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:21 smithi084 ceph-mon[70440]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:24.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:23 smithi057 ceph-mon[89491]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:24.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:23 smithi084 ceph-mon[70440]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:25 smithi057 ceph-mon[89491]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:26.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:25 smithi084 ceph-mon[70440]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:28.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:27 smithi057 ceph-mon[89491]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:28.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:27 smithi084 ceph-mon[70440]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:29.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:29 smithi057 ceph-mon[89491]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:29.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:29 smithi084 ceph-mon[70440]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:31.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:31 smithi084 ceph-mon[70440]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:32.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:31 smithi057 ceph-mon[89491]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:33 smithi057 ceph-mon[89491]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:33 smithi084 ceph-mon[70440]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:36.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:35 smithi057 ceph-mon[89491]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:36.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:35 smithi084 ceph-mon[70440]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:37 smithi057 ceph-mon[89491]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:37 smithi084 ceph-mon[70440]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:39 smithi057 ceph-mon[89491]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:39 smithi084 ceph-mon[70440]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:41 smithi057 ceph-mon[89491]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:41 smithi084 ceph-mon[70440]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:43 smithi057 ceph-mon[89491]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:43 smithi084 ceph-mon[70440]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 -- 172.21.15.57:0/769918300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 msgr2=0x7f37f410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 --2- 172.21.15.57:0/769918300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f410ccc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009910 tx=0x7f37e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 -- 172.21.15.57:0/769918300 shutdown_connections 2024-02-28T23:30:44.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 --2- 172.21.15.57:0/769918300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 0x7f37f4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 --2- 172.21.15.57:0/769918300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f410ccc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 -- 172.21.15.57:0/769918300 >> 172.21.15.57:0/769918300 conn(0x7f37f4076060 msgr2=0x7f37f4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:44.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 -- 172.21.15.57:0/769918300 shutdown_connections 2024-02-28T23:30:44.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.286+0000 7f37fa000700 1 -- 172.21.15.57:0/769918300 wait complete. 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.287+0000 7f37fa000700 1 Processor -- start 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.287+0000 7f37fa000700 1 -- start start 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.287+0000 7f37fa000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 0x7f37f419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37fa000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37fa000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419d040 con 0x7f37f410c8f0 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37fa000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41a0dd0 con 0x7f37f4106b80 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54280/0 (socket says 172.21.15.57:54280) 2024-02-28T23:30:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 -- 172.21.15.57:0/3391518845 learned_addr learned my addr 172.21.15.57:0/3391518845 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f3400700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 0x7f37f419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 -- 172.21.15.57:0/3391518845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 msgr2=0x7f37f419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 0x7f37f419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.288+0000 7f37f2a00700 1 -- 172.21.15.57:0/3391518845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40095f0 con 0x7f37f410c8f0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37f2a00700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d8d0 tx=0x7f37e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e0009940 con 0x7f37f410c8f0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37e0010460 con 0x7f37f410c8f0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f41a10b0 con 0x7f37f410c8f0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e000f5f0 con 0x7f37f410c8f0 2024-02-28T23:30:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.289+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f41a1600 con 0x7f37f410c8f0 2024-02-28T23:30:44.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.290+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f4109b20 con 0x7f37f410c8f0 2024-02-28T23:30:44.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.292+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f37e00105d0 con 0x7f37f410c8f0 2024-02-28T23:30:44.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.292+0000 7f37ebe00700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 0x7f37d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.293+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f37e008d5d0 con 0x7f37f410c8f0 2024-02-28T23:30:44.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.293+0000 7f37f3400700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 0x7f37d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.293+0000 7f37f3400700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 0x7f37d806fd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006010 tx=0x7f37e401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:44.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.302+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37e0056ce0 con 0x7f37f410c8f0 2024-02-28T23:30:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.548+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37f41a1900 con 0x7f37d806d8a0 2024-02-28T23:30:44.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.551+0000 7f37ebe00700 1 -- 172.21.15.57:0/3391518845 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f37f41a1900 con 0x7f37d806d8a0 2024-02-28T23:30:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 msgr2=0x7f37d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 0x7f37d806fd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006010 tx=0x7f37e401a040 comp rx=0 tx=0).stop 2024-02-28T23:30:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 msgr2=0x7f37f419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d8d0 tx=0x7f37e000dc90 comp rx=0 tx=0).stop 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 shutdown_connections 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f37d806d8a0 0x7f37d806fd50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37f4106b80 0x7f37f419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 --2- 172.21.15.57:0/3391518845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37f410c8f0 0x7f37f419c960 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 >> 172.21.15.57:0/3391518845 conn(0x7f37f4076060 msgr2=0x7f37f4077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 shutdown_connections 2024-02-28T23:30:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.554+0000 7f37fa000700 1 -- 172.21.15.57:0/3391518845 wait complete. 2024-02-28T23:30:44.569 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:30:44.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 -- 172.21.15.57:0/2494051120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 msgr2=0x7f3cec100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/2494051120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec100040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009910 tx=0x7f3cd8009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:44.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 -- 172.21.15.57:0/2494051120 shutdown_connections 2024-02-28T23:30:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/2494051120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec100040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/2494051120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 0x7f3cec104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 -- 172.21.15.57:0/2494051120 >> 172.21.15.57:0/2494051120 conn(0x7f3cec075130 msgr2=0x7f3cec075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 -- 172.21.15.57:0/2494051120 shutdown_connections 2024-02-28T23:30:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.723+0000 7f3cf1a00700 1 -- 172.21.15.57:0/2494051120 wait complete. 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.724+0000 7f3cf1a00700 1 Processor -- start 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.724+0000 7f3cf1a00700 1 -- start start 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3cf1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3cf1a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 0x7f3cec196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3cf1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec196a20 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3cf1a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec19a7b0 con 0x7f3cec104130 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54312/0 (socket says 172.21.15.57:54312) 2024-02-28T23:30:44.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 -- 172.21.15.57:0/3408642461 learned_addr learned my addr 172.21.15.57:0/3408642461 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3cea000700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 0x7f3cec196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 -- 172.21.15.57:0/3408642461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 msgr2=0x7f3cec196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 0x7f3cec196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.725+0000 7f3ceaa00700 1 -- 172.21.15.57:0/3408642461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80095f0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3ceaa00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00d8d0 tx=0x7f3cdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc009940 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cdc010460 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec19aa90 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc00f5f0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.726+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec19afe0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.727+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec04f2a0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.728+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3cdc0105d0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.729+0000 7f3ce3400700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 0x7f3cd006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:44.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.729+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3cdc08d4c0 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.730+0000 7f3cea000700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 0x7f3cd006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:44.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.730+0000 7f3cea000700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 0x7f3cd006fd50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005b40 tx=0x7f3cd801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.735+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cdc056c50 con 0x7f3cec0ffbd0 2024-02-28T23:30:44.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.986+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cec19b360 con 0x7f3cd006d8a0 2024-02-28T23:30:44.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.988+0000 7f3ce3400700 1 -- 172.21.15.57:0/3408642461 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3cec19b360 con 0x7f3cd006d8a0 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 msgr2=0x7f3cd006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 0x7f3cd006fd50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005b40 tx=0x7f3cd801a040 comp rx=0 tx=0).stop 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 msgr2=0x7f3cec195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00d8d0 tx=0x7f3cdc00dc90 comp rx=0 tx=0).stop 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 shutdown_connections 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.991+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3cd006d8a0 0x7f3cd006fd50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.992+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cec0ffbd0 0x7f3cec195e00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.992+0000 7f3cf1a00700 1 --2- 172.21.15.57:0/3408642461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cec104130 0x7f3cec196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:44.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.992+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 >> 172.21.15.57:0/3408642461 conn(0x7f3cec075130 msgr2=0x7f3cec0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:44.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.992+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 shutdown_connections 2024-02-28T23:30:44.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:44.992+0000 7f3cf1a00700 1 -- 172.21.15.57:0/3408642461 wait complete. 2024-02-28T23:30:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 -- 172.21.15.57:0/2930834847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 msgr2=0x7fc60c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 --2- 172.21.15.57:0/2930834847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c1008b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009910 tx=0x7fc5fc009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 -- 172.21.15.57:0/2930834847 shutdown_connections 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 --2- 172.21.15.57:0/2930834847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 0x7fc60c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 --2- 172.21.15.57:0/2930834847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c1008b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 -- 172.21.15.57:0/2930834847 >> 172.21.15.57:0/2930834847 conn(0x7fc60c0fbda0 msgr2=0x7fc60c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 -- 172.21.15.57:0/2930834847 shutdown_connections 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.151+0000 7fc612200700 1 -- 172.21.15.57:0/2930834847 wait complete. 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.152+0000 7fc612200700 1 Processor -- start 2024-02-28T23:30:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc612200700 1 -- start start 2024-02-28T23:30:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc612200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc612200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 0x7fc60c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc612200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c074e00 con 0x7fc60c1004e0 2024-02-28T23:30:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc612200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c074f70 con 0x7fc60c100e80 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc60b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc60b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53342/0 (socket says 172.21.15.57:53342) 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.153+0000 7fc60b400700 1 -- 172.21.15.57:0/1194446234 learned_addr learned my addr 172.21.15.57:0/1194446234 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc603e00700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 0x7fc60c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc60b400700 1 -- 172.21.15.57:0/1194446234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 msgr2=0x7fc60c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc60b400700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 0x7fc60c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc60b400700 1 -- 172.21.15.57:0/1194446234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0095f0 con 0x7fc60c1004e0 2024-02-28T23:30:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc60b400700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00f690 tx=0x7fc5fc00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc01d070 con 0x7fc60c1004e0 2024-02-28T23:30:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5fc0050d0 con 0x7fc60c1004e0 2024-02-28T23:30:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc022470 con 0x7fc60c1004e0 2024-02-28T23:30:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.154+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c073b30 con 0x7fc60c1004e0 2024-02-28T23:30:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.155+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c1a6890 con 0x7fc60c1004e0 2024-02-28T23:30:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.156+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c1087f0 con 0x7fc60c1004e0 2024-02-28T23:30:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.157+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc5fc003680 con 0x7fc60c1004e0 2024-02-28T23:30:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.158+0000 7fc608c00700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 0x7fc5f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.158+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc5fc08e940 con 0x7fc60c1004e0 2024-02-28T23:30:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.158+0000 7fc603e00700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 0x7fc5f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.159+0000 7fc603e00700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 0x7fc5f006fd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc60c074840 tx=0x7fc5f4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.163+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc5fc0580d0 con 0x7fc60c1004e0 2024-02-28T23:30:45.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.396+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc60c061190 con 0x7fc5f006d8a0 2024-02-28T23:30:45.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.429+0000 7fc608c00700 1 -- 172.21.15.57:0/1194446234 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc60c061190 con 0x7fc5f006d8a0 2024-02-28T23:30:45.427 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (20m) 11m ago 34m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (30m) 11m ago 34m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (33m) 11m ago 33m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (16m) 11m ago 34m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (16m) 11m ago 33m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (19m) 11m ago 33m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (12m) 11m ago 27m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (11m) 11m ago 26m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (26m) 11m ago 26m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (11m) 11m ago 26m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:30:45.428 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (22m) 11m ago 36m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (21m) 11m ago 32m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (18m) 11m ago 36m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (17m) 11m ago 32m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (21m) 11m ago 34m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (21m) 11m ago 32m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (15m) 11m ago 31m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (15m) 11m ago 31m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (14m) 11m ago 30m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (14m) 11m ago 29m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (13m) 11m ago 29m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (12m) 11m ago 28m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:30:45.429 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (20m) 11m ago 33m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 msgr2=0x7fc5f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 0x7fc5f006fd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc60c074840 tx=0x7fc5f4009260 comp rx=0 tx=0).stop 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 msgr2=0x7fc60c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00f690 tx=0x7fc5fc00f770 comp rx=0 tx=0).stop 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 shutdown_connections 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc5f006d8a0 0x7fc5f006fd50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc60c1004e0 0x7fc60c073050 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 --2- 172.21.15.57:0/1194446234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c100e80 0x7fc60c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 >> 172.21.15.57:0/1194446234 conn(0x7fc60c0fbda0 msgr2=0x7fc60c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 shutdown_connections 2024-02-28T23:30:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.432+0000 7fc612200700 1 -- 172.21.15.57:0/1194446234 wait complete. 2024-02-28T23:30:45.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 -- 172.21.15.57:0/26154503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 msgr2=0x7f235c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 --2- 172.21.15.57:0/26154503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1028e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2344009910 tx=0x7f2344009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:45.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 -- 172.21.15.57:0/26154503 shutdown_connections 2024-02-28T23:30:45.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 --2- 172.21.15.57:0/26154503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1028e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 --2- 172.21.15.57:0/26154503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 0x7f235c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 -- 172.21.15.57:0/26154503 >> 172.21.15.57:0/26154503 conn(0x7f235c0fdf90 msgr2=0x7f235c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:45.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 -- 172.21.15.57:0/26154503 shutdown_connections 2024-02-28T23:30:45.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.578+0000 7f2361800700 1 -- 172.21.15.57:0/26154503 wait complete. 2024-02-28T23:30:45.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 Processor -- start 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 -- start start 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 0x7f235c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c198ef0 con 0x7f235c102470 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.579+0000 7f2361800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c19cc80 con 0x7f235c108470 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53354/0 (socket says 172.21.15.57:53354) 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 -- 172.21.15.57:0/188315975 learned_addr learned my addr 172.21.15.57:0/188315975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235a000700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 0x7f235c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 -- 172.21.15.57:0/188315975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 msgr2=0x7f235c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 0x7f235c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 -- 172.21.15.57:0/188315975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440095f0 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.580+0000 7f235aa00700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f234c00d900 tx=0x7f234c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.581+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234c004a00 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.581+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f234c0054e0 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.581+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c19cf60 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.581+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234c009d60 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.581+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c19d4b0 con 0x7f235c102470 2024-02-28T23:30:45.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.582+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c10a970 con 0x7f235c102470 2024-02-28T23:30:45.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.583+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f234c005020 con 0x7f235c102470 2024-02-28T23:30:45.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.584+0000 7f2353400700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 0x7f234006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:45.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.584+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f234c08d800 con 0x7f235c102470 2024-02-28T23:30:45.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.584+0000 7f235a000700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 0x7f234006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:45.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.585+0000 7f235a000700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 0x7f234006fd50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f235c1998f0 tx=0x7f234401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:45.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.588+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f234c056f10 con 0x7f235c102470 2024-02-28T23:30:45.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.898+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f235c1996d0 con 0x7f235c102470 2024-02-28T23:30:45.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.899+0000 7f2353400700 1 -- 172.21.15.57:0/188315975 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f234c05a5d0 con 0x7f235c102470 2024-02-28T23:30:45.897 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:30:45.897 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:30:45.897 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:30:45.898 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:30:45.899 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 msgr2=0x7f234006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 0x7f234006fd50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f235c1998f0 tx=0x7f234401a040 comp rx=0 tx=0).stop 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 msgr2=0x7f235c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f234c00d900 tx=0x7f234c00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 shutdown_connections 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f234006d8a0 0x7f234006fd50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f235c102470 0x7f235c1982d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 --2- 172.21.15.57:0/188315975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c108470 0x7f235c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.901+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 >> 172.21.15.57:0/188315975 conn(0x7f235c0fdf90 msgr2=0x7f235c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.902+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 shutdown_connections 2024-02-28T23:30:45.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:45.902+0000 7f2361800700 1 -- 172.21.15.57:0/188315975 wait complete. 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 -- 172.21.15.57:0/2750136763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8102470 msgr2=0x7f3bf81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 --2- 172.21.15.57:0/2750136763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8102470 0x7f3bf81028e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009910 tx=0x7f3be4009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 -- 172.21.15.57:0/2750136763 shutdown_connections 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 --2- 172.21.15.57:0/2750136763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8102470 0x7f3bf81028e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 --2- 172.21.15.57:0/2750136763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8108470 0x7f3bf8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 -- 172.21.15.57:0/2750136763 >> 172.21.15.57:0/2750136763 conn(0x7f3bf80fdf90 msgr2=0x7f3bf81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:46.063 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:45 smithi057 ceph-mon[89491]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:46.063 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:45 smithi057 ceph-mon[89491]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:46.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 -- 172.21.15.57:0/2750136763 shutdown_connections 2024-02-28T23:30:46.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.064+0000 7f3bfe600700 1 -- 172.21.15.57:0/2750136763 wait complete. 2024-02-28T23:30:46.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 Processor -- start 2024-02-28T23:30:46.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 -- start start 2024-02-28T23:30:46.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8102470 0x7f3bf8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8198e90 con 0x7f3bf8108470 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bfe600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf819cc20 con 0x7f3bf8102470 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.066+0000 7f3bf6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53386/0 (socket says 172.21.15.57:53386) 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 -- 172.21.15.57:0/710747322 learned_addr learned my addr 172.21.15.57:0/710747322 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:46.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf7400700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8102470 0x7f3bf8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 -- 172.21.15.57:0/710747322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8102470 msgr2=0x7f3bf8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8102470 0x7f3bf8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 -- 172.21.15.57:0/710747322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40095f0 con 0x7f3bf8108470 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.067+0000 7f3bf6a00700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3be40092e0 tx=0x7f3be400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.068+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3be401d070 con 0x7f3bf8108470 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.068+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3be40052a0 con 0x7f3bf8108470 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.068+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3be4017630 con 0x7f3bf8108470 2024-02-28T23:30:46.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.068+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf819cea0 con 0x7f3bf8108470 2024-02-28T23:30:46.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.068+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf819d390 con 0x7f3bf8108470 2024-02-28T23:30:46.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.069+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf810a970 con 0x7f3bf8108470 2024-02-28T23:30:46.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.070+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3be40178a0 con 0x7f3bf8108470 2024-02-28T23:30:46.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.071+0000 7f3befe00700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 0x7f3bdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.071+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3be408e8c0 con 0x7f3bf8108470 2024-02-28T23:30:46.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.072+0000 7f3bf7400700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 0x7f3bdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.072+0000 7f3bf7400700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 0x7f3bdc06fd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009bf0 tx=0x7f3be8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.076+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3be4058050 con 0x7f3bf8108470 2024-02-28T23:30:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:45 smithi084 ceph-mon[70440]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:45 smithi084 ceph-mon[70440]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:46.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.358+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3bf810a350 con 0x7f3bf8108470 2024-02-28T23:30:46.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.358+0000 7f3befe00700 1 -- 172.21.15.57:0/710747322 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f3be4003e60 con 0x7f3bf8108470 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:30:46.357 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:30:46.358 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:46.359 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:30:46.360 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:30:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 msgr2=0x7f3bdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 0x7f3bdc06fd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009bf0 tx=0x7f3be8009260 comp rx=0 tx=0).stop 2024-02-28T23:30:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 msgr2=0x7f3bf81987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3be40092e0 tx=0x7f3be400f460 comp rx=0 tx=0).stop 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 shutdown_connections 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3bdc06d8a0 0x7f3bdc06fd50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bf8102470 0x7f3bf8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 --2- 172.21.15.57:0/710747322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bf8108470 0x7f3bf81987b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 >> 172.21.15.57:0/710747322 conn(0x7f3bf80fdf90 msgr2=0x7f3bf80ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 shutdown_connections 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.361+0000 7f3bfe600700 1 -- 172.21.15.57:0/710747322 wait complete. 2024-02-28T23:30:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:30:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 -- 172.21.15.57:0/27454061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 msgr2=0x7fd8b41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/27454061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b41008b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009910 tx=0x7fd8a4009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 -- 172.21.15.57:0/27454061 shutdown_connections 2024-02-28T23:30:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/27454061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/27454061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b41008b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 -- 172.21.15.57:0/27454061 >> 172.21.15.57:0/27454061 conn(0x7fd8b40fbda0 msgr2=0x7fd8b40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 -- 172.21.15.57:0/27454061 shutdown_connections 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.511+0000 7fd8b8e00700 1 -- 172.21.15.57:0/27454061 wait complete. 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.512+0000 7fd8b8e00700 1 Processor -- start 2024-02-28T23:30:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b8e00700 1 -- start start 2024-02-28T23:30:46.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b8e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b8e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b8e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4198da0 con 0x7fd8b41004e0 2024-02-28T23:30:46.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b8e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b419cb30 con 0x7fd8b4100e80 2024-02-28T23:30:46.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53408/0 (socket says 172.21.15.57:53408) 2024-02-28T23:30:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.513+0000 7fd8b1e00700 1 -- 172.21.15.57:0/1829359231 learned_addr learned my addr 172.21.15.57:0/1829359231 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8b1e00700 1 -- 172.21.15.57:0/1829359231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 msgr2=0x7fd8b41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8abe00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8b1e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8b1e00700 1 -- 172.21.15.57:0/1829359231 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a40095f0 con 0x7fd8b41004e0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8abe00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8b1e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b5c0 tx=0x7fd8a400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a401d070 con 0x7fd8b41004e0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a400f460 con 0x7fd8b41004e0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a4017610 con 0x7fd8b41004e0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.514+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b419cdb0 con 0x7fd8b41004e0 2024-02-28T23:30:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.515+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b419d2a0 con 0x7fd8b41004e0 2024-02-28T23:30:46.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.516+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b41045d0 con 0x7fd8b41004e0 2024-02-28T23:30:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.517+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd8a400f5d0 con 0x7fd8b41004e0 2024-02-28T23:30:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.518+0000 7fd8aaa00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 0x7fd89806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.518+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd8a408eb50 con 0x7fd8b41004e0 2024-02-28T23:30:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.518+0000 7fd8abe00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 0x7fd89806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.519+0000 7fd8abe00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 0x7fd89806fd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8b41997a0 tx=0x7fd89c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.522+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8a40582e0 con 0x7fd8b41004e0 2024-02-28T23:30:46.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.762+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b402cca0 con 0x7fd89806d8a0 2024-02-28T23:30:46.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.765+0000 7fd8aaa00700 1 -- 172.21.15.57:0/1829359231 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd8b402cca0 con 0x7fd89806d8a0 2024-02-28T23:30:46.763 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:30:46.763 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:30:46.764 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.768+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 msgr2=0x7fd89806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.768+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 0x7fd89806fd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8b41997a0 tx=0x7fd89c008040 comp rx=0 tx=0).stop 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.768+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 msgr2=0x7fd8b4198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.768+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b5c0 tx=0x7fd8a400be00 comp rx=0 tx=0).stop 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 shutdown_connections 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd89806d8a0 0x7fd89806fd50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8b41004e0 0x7fd8b4198180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 --2- 172.21.15.57:0/1829359231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8b4100e80 0x7fd8b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 >> 172.21.15.57:0/1829359231 conn(0x7fd8b40fbda0 msgr2=0x7fd8b40fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 shutdown_connections 2024-02-28T23:30:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.769+0000 7fd8b8e00700 1 -- 172.21.15.57:0/1829359231 wait complete. 2024-02-28T23:30:46.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 -- 172.21.15.57:0/201067128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d40730f0 msgr2=0x7f80d40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 --2- 172.21.15.57:0/201067128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d40730f0 0x7f80d40734c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009910 tx=0x7f80c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 -- 172.21.15.57:0/201067128 shutdown_connections 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 --2- 172.21.15.57:0/201067128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d4073a00 0x7f80d4110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 --2- 172.21.15.57:0/201067128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d40730f0 0x7f80d40734c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 -- 172.21.15.57:0/201067128 >> 172.21.15.57:0/201067128 conn(0x7f80d40fbda0 msgr2=0x7f80d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.925+0000 7f80d8a00700 1 -- 172.21.15.57:0/201067128 shutdown_connections 2024-02-28T23:30:46.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.926+0000 7f80d8a00700 1 -- 172.21.15.57:0/201067128 wait complete. 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.926+0000 7f80d8a00700 1 Processor -- start 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d8a00700 1 -- start start 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d8a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d40730f0 0x7f80d419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d8a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d8a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419d020 con 0x7f80d4073a00 2024-02-28T23:30:46.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d8a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41a0db0 con 0x7f80d40730f0 2024-02-28T23:30:46.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53430/0 (socket says 172.21.15.57:53430) 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.927+0000 7f80d1400700 1 -- 172.21.15.57:0/262756641 learned_addr learned my addr 172.21.15.57:0/262756641 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d1e00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d40730f0 0x7f80d419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d1400700 1 -- 172.21.15.57:0/262756641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d40730f0 msgr2=0x7f80d419c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d1400700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d40730f0 0x7f80d419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d1400700 1 -- 172.21.15.57:0/262756641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40095f0 con 0x7f80d4073a00 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d1400700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d900 tx=0x7f80c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c0004a00 con 0x7f80d4073a00 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80c00054e0 con 0x7f80d4073a00 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d41a1090 con 0x7f80d4073a00 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.928+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c0009d60 con 0x7f80d4073a00 2024-02-28T23:30:46.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.929+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d41a15e0 con 0x7f80d4073a00 2024-02-28T23:30:46.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.930+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d410e4d0 con 0x7f80d4073a00 2024-02-28T23:30:46.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.931+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f80c0005020 con 0x7f80d4073a00 2024-02-28T23:30:46.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.931+0000 7f80caa00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 0x7f80b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:30:46.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.932+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80c008d3b0 con 0x7f80d4073a00 2024-02-28T23:30:46.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.932+0000 7f80d1e00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 0x7f80b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:30:46.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.932+0000 7f80d1e00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 0x7f80b806fd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80c4006010 tx=0x7f80c400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:30:46.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:46.936+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80c0056be0 con 0x7f80d4073a00 2024-02-28T23:30:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:46 smithi057 ceph-mon[89491]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:46 smithi057 ceph-mon[89491]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:46 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/188315975' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:30:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:46 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/710747322' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:30:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:46 smithi084 ceph-mon[70440]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:46 smithi084 ceph-mon[70440]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:46 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/188315975' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:30:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:46 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/710747322' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:30:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.246+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f80d404f2a0 con 0x7f80d4073a00 2024-02-28T23:30:47.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.247+0000 7f80caa00700 1 -- 172.21.15.57:0/262756641 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f80c005a2a0 con 0x7f80d4073a00 2024-02-28T23:30:47.246 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:30:47.246 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:30:47.246 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:30:47.246 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:30:47.247 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 msgr2=0x7f80b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 0x7f80b806fd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80c4006010 tx=0x7f80c400b540 comp rx=0 tx=0).stop 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 msgr2=0x7f80d419c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d900 tx=0x7f80c000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 shutdown_connections 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f80b806d8a0 0x7f80b806fd50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80d40730f0 0x7f80d419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 --2- 172.21.15.57:0/262756641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80d4073a00 0x7f80d419c940 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:30:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 >> 172.21.15.57:0/262756641 conn(0x7f80d40fbda0 msgr2=0x7f80d4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:30:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 shutdown_connections 2024-02-28T23:30:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:30:47.250+0000 7f80d8a00700 1 -- 172.21.15.57:0/262756641 wait complete. 2024-02-28T23:30:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:47 smithi057 ceph-mon[89491]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:47 smithi057 ceph-mon[89491]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:47 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/262756641' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:30:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:47 smithi084 ceph-mon[70440]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:47 smithi084 ceph-mon[70440]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:30:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:47 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/262756641' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:30:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:30:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:49 smithi057 ceph-mon[89491]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:49 smithi084 ceph-mon[70440]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:51 smithi057 ceph-mon[89491]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:51 smithi084 ceph-mon[70440]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:53 smithi057 ceph-mon[89491]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:53 smithi084 ceph-mon[70440]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:56.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:55 smithi057 ceph-mon[89491]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:55 smithi084 ceph-mon[70440]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:57 smithi084 ceph-mon[70440]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:30:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:57 smithi057 ceph-mon[89491]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:30:59 smithi084 ceph-mon[70440]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:00.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:30:59 smithi057 ceph-mon[89491]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:02.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:01 smithi057 ceph-mon[89491]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:01 smithi084 ceph-mon[70440]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:03 smithi057 ceph-mon[89491]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:03 smithi084 ceph-mon[70440]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:05 smithi057 ceph-mon[89491]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:05 smithi084 ceph-mon[70440]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:07 smithi057 ceph-mon[89491]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:08.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:07 smithi084 ceph-mon[70440]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:09 smithi057 ceph-mon[89491]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:10.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:09 smithi084 ceph-mon[70440]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:12.132 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:11 smithi084 ceph-mon[70440]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:12.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:11 smithi057 ceph-mon[89491]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:13 smithi057 ceph-mon[89491]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:14.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:13 smithi084 ceph-mon[70440]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:16.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:15 smithi057 ceph-mon[89491]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:16.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:15 smithi084 ceph-mon[70440]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.427+0000 7f4e97c00700 1 -- 172.21.15.57:0/3532870825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e90106b80 msgr2=0x7f4e90106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.427+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3532870825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e90106b80 0x7f4e90106ff0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009910 tx=0x7f4e7c009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 -- 172.21.15.57:0/3532870825 shutdown_connections 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3532870825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e90106b80 0x7f4e90106ff0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3532870825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e9010c8f0 0x7f4e9010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 -- 172.21.15.57:0/3532870825 >> 172.21.15.57:0/3532870825 conn(0x7f4e90076060 msgr2=0x7f4e90078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 -- 172.21.15.57:0/3532870825 shutdown_connections 2024-02-28T23:31:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.428+0000 7f4e97c00700 1 -- 172.21.15.57:0/3532870825 wait complete. 2024-02-28T23:31:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.429+0000 7f4e97c00700 1 Processor -- start 2024-02-28T23:31:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.429+0000 7f4e97c00700 1 -- start start 2024-02-28T23:31:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e97c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 0x7f4e9019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e97c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e97c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019d060 con 0x7f4e9010c8f0 2024-02-28T23:31:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e97c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e901a0df0 con 0x7f4e90106b80 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e8fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e8fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48388/0 (socket says 172.21.15.57:48388) 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.430+0000 7f4e8fe00700 1 -- 172.21.15.57:0/3191906325 learned_addr learned my addr 172.21.15.57:0/3191906325 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e95200700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 0x7f4e9019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e8fe00700 1 -- 172.21.15.57:0/3191906325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 msgr2=0x7f4e9019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e8fe00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 0x7f4e9019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e8fe00700 1 -- 172.21.15.57:0/3191906325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0095f0 con 0x7f4e9010c8f0 2024-02-28T23:31:17.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e95200700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 0x7f4e9019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.431+0000 7f4e8fe00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006010 tx=0x7f4e7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.432+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7c01d070 con 0x7f4e9010c8f0 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.432+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e7c00f460 con 0x7f4e9010c8f0 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.432+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e901a1070 con 0x7f4e9010c8f0 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.432+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7c017630 con 0x7f4e9010c8f0 2024-02-28T23:31:17.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.432+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e901a1560 con 0x7f4e9010c8f0 2024-02-28T23:31:17.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.433+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9010ecc0 con 0x7f4e9010c8f0 2024-02-28T23:31:17.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.434+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4e7c00f5d0 con 0x7f4e9010c8f0 2024-02-28T23:31:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.436+0000 7f4e8d600700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 0x7f4e7806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.436+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e7c08ea40 con 0x7f4e9010c8f0 2024-02-28T23:31:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.436+0000 7f4e95200700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 0x7f4e7806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.437+0000 7f4e95200700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 0x7f4e7806fd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005950 tx=0x7f4e840058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:17.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.441+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e7c058150 con 0x7f4e9010c8f0 2024-02-28T23:31:17.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.690+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e901a1840 con 0x7f4e7806d8a0 2024-02-28T23:31:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.692+0000 7f4e8d600700 1 -- 172.21.15.57:0/3191906325 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e901a1840 con 0x7f4e7806d8a0 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 msgr2=0x7f4e7806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 0x7f4e7806fd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005950 tx=0x7f4e840058e0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 msgr2=0x7f4e9019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006010 tx=0x7f4e7c00be20 comp rx=0 tx=0).stop 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 shutdown_connections 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.695+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4e7806d8a0 0x7f4e7806fd50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.696+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e90106b80 0x7f4e9019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.696+0000 7f4e97c00700 1 --2- 172.21.15.57:0/3191906325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9010c8f0 0x7f4e9019c980 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.696+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 >> 172.21.15.57:0/3191906325 conn(0x7f4e90076060 msgr2=0x7f4e90077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.696+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 shutdown_connections 2024-02-28T23:31:17.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.696+0000 7f4e97c00700 1 -- 172.21.15.57:0/3191906325 wait complete. 2024-02-28T23:31:17.712 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 -- 172.21.15.57:0/3766779576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 msgr2=0x7f65300734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 --2- 172.21.15.57:0/3766779576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f65300734c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6524009910 tx=0x7f6524009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 -- 172.21.15.57:0/3766779576 shutdown_connections 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 --2- 172.21.15.57:0/3766779576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 0x7f653010c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 --2- 172.21.15.57:0/3766779576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f65300734c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 -- 172.21.15.57:0/3766779576 >> 172.21.15.57:0/3766779576 conn(0x7f65300fbb60 msgr2=0x7f65300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 -- 172.21.15.57:0/3766779576 shutdown_connections 2024-02-28T23:31:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.865+0000 7f6535e00700 1 -- 172.21.15.57:0/3766779576 wait complete. 2024-02-28T23:31:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.866+0000 7f6535e00700 1 Processor -- start 2024-02-28T23:31:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.866+0000 7f6535e00700 1 -- start start 2024-02-28T23:31:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6535e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6535e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 0x7f65301983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6535e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019c6f0 con 0x7f65300730f0 2024-02-28T23:31:17.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6535e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019c860 con 0x7f6530073a00 2024-02-28T23:31:17.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6534a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6534a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48400/0 (socket says 172.21.15.57:48400) 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f6534a00700 1 -- 172.21.15.57:0/1824270409 learned_addr learned my addr 172.21.15.57:0/1824270409 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.867+0000 7f652fe00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 0x7f65301983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6534a00700 1 -- 172.21.15.57:0/1824270409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 msgr2=0x7f65301983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6534a00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 0x7f65301983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:17.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6534a00700 1 -- 172.21.15.57:0/1824270409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240095f0 con 0x7f65300730f0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6534a00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f652400b5c0 tx=0x7f652400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652401d070 con 0x7f65300730f0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6524005e00 con 0x7f65300730f0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653019cb40 con 0x7f65300730f0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6524017400 con 0x7f65300730f0 2024-02-28T23:31:17.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.868+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653019d090 con 0x7f65300730f0 2024-02-28T23:31:17.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.870+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530109a60 con 0x7f65300730f0 2024-02-28T23:31:17.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.871+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f652400f460 con 0x7f65300730f0 2024-02-28T23:31:17.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.872+0000 7f652d600700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 0x7f651806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:17.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.872+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f652408e940 con 0x7f65300730f0 2024-02-28T23:31:17.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.872+0000 7f652fe00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 0x7f651806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:17.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.873+0000 7f652fe00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 0x7f651806fd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f651c005950 tx=0x7f651c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:17.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:17.876+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6524058170 con 0x7f65300730f0 2024-02-28T23:31:18.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:17 smithi057 ceph-mon[89491]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:18.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.133+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65301990a0 con 0x7f651806d8a0 2024-02-28T23:31:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.144+0000 7f652d600700 1 -- 172.21.15.57:0/1824270409 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f65301990a0 con 0x7f651806d8a0 2024-02-28T23:31:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 msgr2=0x7f651806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 0x7f651806fd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f651c005950 tx=0x7f651c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 msgr2=0x7f6530197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f652400b5c0 tx=0x7f652400be20 comp rx=0 tx=0).stop 2024-02-28T23:31:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 shutdown_connections 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f651806d8a0 0x7f651806fd50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65300730f0 0x7f6530197d40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 --2- 172.21.15.57:0/1824270409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6530073a00 0x7f65301983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 >> 172.21.15.57:0/1824270409 conn(0x7f65300fbb60 msgr2=0x7f6530106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 shutdown_connections 2024-02-28T23:31:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.148+0000 7f6535e00700 1 -- 172.21.15.57:0/1824270409 wait complete. 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.315+0000 7f4a7ba00700 1 -- 172.21.15.57:0/2415459742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 msgr2=0x7f4a740734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.315+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/2415459742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a740734c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009910 tx=0x7f4a64009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 -- 172.21.15.57:0/2415459742 shutdown_connections 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/2415459742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a7410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/2415459742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a740734c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 -- 172.21.15.57:0/2415459742 >> 172.21.15.57:0/2415459742 conn(0x7f4a740fbda0 msgr2=0x7f4a740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 -- 172.21.15.57:0/2415459742 shutdown_connections 2024-02-28T23:31:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.316+0000 7f4a7ba00700 1 -- 172.21.15.57:0/2415459742 wait complete. 2024-02-28T23:31:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.317+0000 7f4a7ba00700 1 Processor -- start 2024-02-28T23:31:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.317+0000 7f4a7ba00700 1 -- start start 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a7ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a7ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a74198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a7ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419c970 con 0x7f4a740730f0 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a7ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419cae0 con 0x7f4a74073a00 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a79000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a79000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48432/0 (socket says 172.21.15.57:48432) 2024-02-28T23:31:18.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a79000700 1 -- 172.21.15.57:0/100907282 learned_addr learned my addr 172.21.15.57:0/100907282 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:18.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a79000700 1 -- 172.21.15.57:0/100907282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 msgr2=0x7f4a74198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.318+0000 7f4a6be00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a74198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a79000700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a74198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a79000700 1 -- 172.21.15.57:0/100907282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a640095f0 con 0x7f4a740730f0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a6be00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a74198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a79000700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4a640092e0 tx=0x7f4a6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6401d070 con 0x7f4a740730f0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a64005e00 con 0x7f4a740730f0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a64017400 con 0x7f4a740730f0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.319+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7419cdc0 con 0x7f4a740730f0 2024-02-28T23:31:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.320+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7419d310 con 0x7f4a740730f0 2024-02-28T23:31:18.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.321+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a74109c80 con 0x7f4a740730f0 2024-02-28T23:31:18.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.322+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a6400f460 con 0x7f4a740730f0 2024-02-28T23:31:18.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.323+0000 7f4a72000700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.323+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a6408ea90 con 0x7f4a740730f0 2024-02-28T23:31:18.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.324+0000 7f4a6be00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:18.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.324+0000 7f4a6be00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005fd0 tx=0x7f4a5c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:18.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.329+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a640581a0 con 0x7f4a740730f0 2024-02-28T23:31:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:17 smithi084 ceph-mon[70440]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:18.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.566+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a74199320 con 0x7f4a5806d8a0 2024-02-28T23:31:18.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.599+0000 7f4a72000700 1 -- 172.21.15.57:0/100907282 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4a74199320 con 0x7f4a5806d8a0 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (20m) 12m ago 34m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (31m) 12m ago 35m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (33m) 11m ago 33m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (17m) 12m ago 35m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (16m) 11m ago 33m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (20m) 12m ago 34m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:31:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (12m) 12m ago 27m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (12m) 12m ago 27m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (27m) 11m ago 27m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (12m) 11m ago 27m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (23m) 12m ago 36m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (22m) 11m ago 33m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (18m) 12m ago 36m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (18m) 11m ago 33m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (22m) 12m ago 34m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (21m) 11m ago 33m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (16m) 12m ago 32m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (15m) 12m ago 31m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (15m) 12m ago 31m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (14m) 11m ago 30m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:31:18.600 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (14m) 11m ago 29m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:31:18.601 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (13m) 11m ago 29m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:31:18.601 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (21m) 12m ago 34m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.602+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 msgr2=0x7f4a5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.602+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005fd0 tx=0x7f4a5c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.602+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 msgr2=0x7f4a74197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.602+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4a640092e0 tx=0x7f4a6400be20 comp rx=0 tx=0).stop 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 shutdown_connections 2024-02-28T23:31:18.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5806d8a0 0x7f4a5806fd50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a740730f0 0x7f4a74197fc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 --2- 172.21.15.57:0/100907282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a74073a00 0x7f4a74198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 >> 172.21.15.57:0/100907282 conn(0x7f4a740fbda0 msgr2=0x7f4a74106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:18.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 shutdown_connections 2024-02-28T23:31:18.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.603+0000 7f4a7ba00700 1 -- 172.21.15.57:0/100907282 wait complete. 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 -- 172.21.15.57:0/761103074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073980 msgr2=0x7f38e410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 --2- 172.21.15.57:0/761103074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073980 0x7f38e410c600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009940 tx=0x7f38d0009c50 comp rx=0 tx=0).stop 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 -- 172.21.15.57:0/761103074 shutdown_connections 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 --2- 172.21.15.57:0/761103074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073980 0x7f38e410c600 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 --2- 172.21.15.57:0/761103074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e4073070 0x7f38e4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 -- 172.21.15.57:0/761103074 >> 172.21.15.57:0/761103074 conn(0x7f38e40fbda0 msgr2=0x7f38e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 -- 172.21.15.57:0/761103074 shutdown_connections 2024-02-28T23:31:18.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.768+0000 7f38ebe00700 1 -- 172.21.15.57:0/761103074 wait complete. 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.769+0000 7f38ebe00700 1 Processor -- start 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38ebe00700 1 -- start start 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38ebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38ebe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e4073980 0x7f38e41a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38ebe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410b9b0 con 0x7f38e4073070 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38ebe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410bb20 con 0x7f38e4073980 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38e9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38e9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48460/0 (socket says 172.21.15.57:48460) 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.770+0000 7f38e9400700 1 -- 172.21.15.57:0/3597837437 learned_addr learned my addr 172.21.15.57:0/3597837437 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:18.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e9400700 1 -- 172.21.15.57:0/3597837437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e4073980 msgr2=0x7f38e41a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:18.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e9400700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e4073980 0x7f38e41a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:18.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e9400700 1 -- 172.21.15.57:0/3597837437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00095f0 con 0x7f38e4073070 2024-02-28T23:31:18.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e9400700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f38d800b4b0 tx=0x7f38d800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:18.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d8011800 con 0x7f38e4073070 2024-02-28T23:31:18.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.771+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d8011e20 con 0x7f38e4073070 2024-02-28T23:31:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.772+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e410be00 con 0x7f38e4073070 2024-02-28T23:31:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.772+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d800f2e0 con 0x7f38e4073070 2024-02-28T23:31:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.772+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e410c350 con 0x7f38e4073070 2024-02-28T23:31:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.773+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e404f2a0 con 0x7f38e4073070 2024-02-28T23:31:18.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.774+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f38d8010380 con 0x7f38e4073070 2024-02-28T23:31:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.775+0000 7f38e2000700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 0x7f38cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.775+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f38d808d0b0 con 0x7f38e4073070 2024-02-28T23:31:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.776+0000 7f38e8a00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 0x7f38cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.777+0000 7f38e8a00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 0x7f38cc06fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006010 tx=0x7f38d0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:18.780+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38d8056840 con 0x7f38e4073070 2024-02-28T23:31:19.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.101+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38e410c620 con 0x7f38e4073070 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.103+0000 7f38e2000700 1 -- 172.21.15.57:0/3597837437 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f38d8059f00 con 0x7f38e4073070 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:31:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:31:19.103 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:31:19.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 msgr2=0x7f38cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 0x7f38cc06fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006010 tx=0x7f38d0000bc0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 msgr2=0x7f38e41a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f38d800b4b0 tx=0x7f38d800b870 comp rx=0 tx=0).stop 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 shutdown_connections 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f38cc06d8a0 0x7f38cc06fd50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38e4073070 0x7f38e41a2360 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 --2- 172.21.15.57:0/3597837437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e4073980 0x7f38e41a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.106+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 >> 172.21.15.57:0/3597837437 conn(0x7f38e40fbda0 msgr2=0x7f38e41021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.107+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 shutdown_connections 2024-02-28T23:31:19.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.107+0000 7f38ebe00700 1 -- 172.21.15.57:0/3597837437 wait complete. 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 -- 172.21.15.57:0/2164225569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 msgr2=0x7f70680689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 --2- 172.21.15.57:0/2164225569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f70680689b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7054009910 tx=0x7f7054009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 -- 172.21.15.57:0/2164225569 shutdown_connections 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 --2- 172.21.15.57:0/2164225569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f70680689b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 --2- 172.21.15.57:0/2164225569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.272+0000 7f706d800700 1 -- 172.21.15.57:0/2164225569 >> 172.21.15.57:0/2164225569 conn(0x7f7068075130 msgr2=0x7f7068075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:19.272 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:18 smithi057 ceph-mon[89491]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:19.272 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.273+0000 7f706d800700 1 -- 172.21.15.57:0/2164225569 shutdown_connections 2024-02-28T23:31:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.273+0000 7f706d800700 1 -- 172.21.15.57:0/2164225569 wait complete. 2024-02-28T23:31:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.275+0000 7f706d800700 1 Processor -- start 2024-02-28T23:31:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.275+0000 7f706d800700 1 -- start start 2024-02-28T23:31:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.276+0000 7f706d800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.276+0000 7f706d800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.276+0000 7f706d800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068196a80 con 0x7f7068068540 2024-02-28T23:31:19.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.276+0000 7f706d800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706819a810 con 0x7f7068106320 2024-02-28T23:31:19.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48482/0 (socket says 172.21.15.57:48482) 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 -- 172.21.15.57:0/1837738073 learned_addr learned my addr 172.21.15.57:0/1837738073 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066000700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 -- 172.21.15.57:0/1837738073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 msgr2=0x7f70681963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066a00700 1 -- 172.21.15.57:0/1837738073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540095f0 con 0x7f7068068540 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.277+0000 7f7066000700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:19.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f7066a00700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f705800d8d0 tx=0x7f705800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7058009940 con 0x7f7068068540 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7058010460 con 0x7f7068068540 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706819aaf0 con 0x7f7068068540 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706819b040 con 0x7f7068068540 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.278+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f705800f5f0 con 0x7f7068068540 2024-02-28T23:31:19.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.279+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7048005320 con 0x7f7068068540 2024-02-28T23:31:19.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.281+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7058010ad0 con 0x7f7068068540 2024-02-28T23:31:19.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.282+0000 7f705f400700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 0x7f704c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.282+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f705808d420 con 0x7f7068068540 2024-02-28T23:31:19.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.282+0000 7f7066000700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 0x7f704c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.283+0000 7f7066000700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 0x7f704c06fd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7054006010 tx=0x7f705400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:19.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.287+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7058056c50 con 0x7f7068068540 2024-02-28T23:31:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:18 smithi084 ceph-mon[70440]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.563+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7048006200 con 0x7f7068068540 2024-02-28T23:31:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.564+0000 7f705f400700 1 -- 172.21.15.57:0/1837738073 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f7058016740 con 0x7f7068068540 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:31:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:31:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:31:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 msgr2=0x7f704c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 0x7f704c06fd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7054006010 tx=0x7f705400b540 comp rx=0 tx=0).stop 2024-02-28T23:31:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 msgr2=0x7f7068195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f705800d8d0 tx=0x7f705800dc90 comp rx=0 tx=0).stop 2024-02-28T23:31:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 shutdown_connections 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f704c06d8a0 0x7f704c06fd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7068068540 0x7f7068195e60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 --2- 172.21.15.57:0/1837738073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7068106320 0x7f70681963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 >> 172.21.15.57:0/1837738073 conn(0x7f7068075130 msgr2=0x7f70680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 shutdown_connections 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.569+0000 7f706d800700 1 -- 172.21.15.57:0/1837738073 wait complete. 2024-02-28T23:31:19.569 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 -- 172.21.15.57:0/1428852910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260108470 msgr2=0x7f9260108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 --2- 172.21.15.57:0/1428852910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260108470 0x7f9260108840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9250009910 tx=0x7f9250009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 -- 172.21.15.57:0/1428852910 shutdown_connections 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 --2- 172.21.15.57:0/1428852910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260102470 0x7f92601028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 --2- 172.21.15.57:0/1428852910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260108470 0x7f9260108840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 -- 172.21.15.57:0/1428852910 >> 172.21.15.57:0/1428852910 conn(0x7f92600fdf90 msgr2=0x7f92601003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 -- 172.21.15.57:0/1428852910 shutdown_connections 2024-02-28T23:31:19.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.734+0000 7f9266c00700 1 -- 172.21.15.57:0/1428852910 wait complete. 2024-02-28T23:31:19.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.735+0000 7f9266c00700 1 Processor -- start 2024-02-28T23:31:19.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.735+0000 7f9266c00700 1 -- start start 2024-02-28T23:31:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f9266c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f9266c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260108470 0x7f9260198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f9266c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9260198e10 con 0x7f9260102470 2024-02-28T23:31:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f9266c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019cba0 con 0x7f9260108470 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48506/0 (socket says 172.21.15.57:48506) 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 -- 172.21.15.57:0/1876487517 learned_addr learned my addr 172.21.15.57:0/1876487517 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925f400700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260108470 0x7f9260198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 -- 172.21.15.57:0/1876487517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260108470 msgr2=0x7f9260198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260108470 0x7f9260198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.736+0000 7f925fe00700 1 -- 172.21.15.57:0/1876487517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500095f0 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f925fe00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f925000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925001d070 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9250004fb0 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9250022ea0 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926019ce20 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.737+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926019d310 con 0x7f9260102470 2024-02-28T23:31:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.739+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926010a9e0 con 0x7f9260102470 2024-02-28T23:31:19.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.740+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9250022470 con 0x7f9260102470 2024-02-28T23:31:19.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.740+0000 7f925cc00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 0x7f924406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:19.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.741+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f925008eca0 con 0x7f9260102470 2024-02-28T23:31:19.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.741+0000 7f925f400700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 0x7f924406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:19.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.742+0000 7f925f400700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 0x7f924406fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f924c005fd0 tx=0x7f924c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.746+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9250058580 con 0x7f9260102470 2024-02-28T23:31:19.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.992+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9260199690 con 0x7f924406d8a0 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.995+0000 7f925cc00700 1 -- 172.21.15.57:0/1876487517 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f924c00a2d0 con 0x7f924406d8a0 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:31:19.994 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:31:19.995 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:31:19.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 msgr2=0x7f924406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 0x7f924406fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f924c005fd0 tx=0x7f924c00a560 comp rx=0 tx=0).stop 2024-02-28T23:31:19.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 msgr2=0x7f92601981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f925000be20 comp rx=0 tx=0).stop 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 shutdown_connections 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f924406d8a0 0x7f924406fd50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9260102470 0x7f92601981f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 --2- 172.21.15.57:0/1876487517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9260108470 0x7f9260198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.998+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 >> 172.21.15.57:0/1876487517 conn(0x7f92600fdf90 msgr2=0x7f92601002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.999+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 shutdown_connections 2024-02-28T23:31:19.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:19.999+0000 7f9266c00700 1 -- 172.21.15.57:0/1876487517 wait complete. 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 -- 172.21.15.57:0/971127276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 msgr2=0x7fdd681028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 --2- 172.21.15.57:0/971127276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd681028a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009910 tx=0x7fdd5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 -- 172.21.15.57:0/971127276 shutdown_connections 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 --2- 172.21.15.57:0/971127276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd68102e70 0x7fdd68106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 --2- 172.21.15.57:0/971127276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd681028a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 -- 172.21.15.57:0/971127276 >> 172.21.15.57:0/971127276 conn(0x7fdd680fdd50 msgr2=0x7fdd68100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:20.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 -- 172.21.15.57:0/971127276 shutdown_connections 2024-02-28T23:31:20.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.152+0000 7fdd6e800700 1 -- 172.21.15.57:0/971127276 wait complete. 2024-02-28T23:31:20.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.153+0000 7fdd6e800700 1 Processor -- start 2024-02-28T23:31:20.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.153+0000 7fdd6e800700 1 -- start start 2024-02-28T23:31:20.153 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:19 smithi057 ceph-mon[89491]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:20.153 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:19 smithi057 ceph-mon[89491]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:20.153 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:19 smithi057 ceph-mon[89491]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:20.153 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:19 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3597837437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:31:20.153 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:19 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1837738073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:31:20.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6e800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd68102e70 0x7fdd68198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6e800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68198a60 con 0x7fdd681024d0 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6e800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6819c7f0 con 0x7fdd68102e70 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48530/0 (socket says 172.21.15.57:48530) 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 -- 172.21.15.57:0/1542562713 learned_addr learned my addr 172.21.15.57:0/1542562713 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 -- 172.21.15.57:0/1542562713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd68102e70 msgr2=0x7fdd68198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd68102e70 0x7fdd68198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.154+0000 7fdd6d400700 1 -- 172.21.15.57:0/1542562713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0095f0 con 0x7fdd681024d0 2024-02-28T23:31:20.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd6d400700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0092e0 tx=0x7fdd5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5c01d070 con 0x7fdd681024d0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd5c005e00 con 0x7fdd681024d0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5c017400 con 0x7fdd681024d0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6819ca70 con 0x7fdd681024d0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.155+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6819cf60 con 0x7fdd681024d0 2024-02-28T23:31:20.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.156+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6810a7e0 con 0x7fdd681024d0 2024-02-28T23:31:20.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.158+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdd5c00f460 con 0x7fdd681024d0 2024-02-28T23:31:20.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.159+0000 7fdd66000700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 0x7fdd5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:20.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.160+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdd5c08ea30 con 0x7fdd681024d0 2024-02-28T23:31:20.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.160+0000 7fdd6ca00700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 0x7fdd5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:20.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.160+0000 7fdd6ca00700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 0x7fdd5006fd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd58005950 tx=0x7fdd580058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:20.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.165+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd5c0581c0 con 0x7fdd681024d0 2024-02-28T23:31:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:19 smithi084 ceph-mon[70440]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:19 smithi084 ceph-mon[70440]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:19 smithi084 ceph-mon[70440]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:19 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3597837437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:31:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:19 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1837738073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:31:20.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.474+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdd6819d350 con 0x7fdd681024d0 2024-02-28T23:31:20.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.475+0000 7fdd66000700 1 -- 172.21.15.57:0/1542562713 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fdd5c05b880 con 0x7fdd681024d0 2024-02-28T23:31:20.474 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:31:20.474 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:31:20.474 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:31:20.474 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:31:20.474 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:31:20.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 msgr2=0x7fdd5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:20.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 0x7fdd5006fd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd58005950 tx=0x7fdd580058e0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 msgr2=0x7fdd68197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:20.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0092e0 tx=0x7fdd5c00be20 comp rx=0 tx=0).stop 2024-02-28T23:31:20.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 shutdown_connections 2024-02-28T23:31:20.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdd5006d8a0 0x7fdd5006fd50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd681024d0 0x7fdd68197e40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 --2- 172.21.15.57:0/1542562713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd68102e70 0x7fdd68198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 >> 172.21.15.57:0/1542562713 conn(0x7fdd680fdd50 msgr2=0x7fdd68100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 shutdown_connections 2024-02-28T23:31:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:20.479+0000 7fdd6e800700 1 -- 172.21.15.57:0/1542562713 wait complete. 2024-02-28T23:31:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:20 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1542562713' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:31:21.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:20 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1542562713' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:31:22.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:21 smithi057 ceph-mon[89491]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:22.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:21 smithi057 ceph-mon[89491]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:22.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:21 smithi084 ceph-mon[70440]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:22.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:21 smithi084 ceph-mon[70440]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:24.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:23 smithi057 ceph-mon[89491]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:24.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:23 smithi084 ceph-mon[70440]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:25 smithi057 ceph-mon[89491]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:26.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:25 smithi084 ceph-mon[70440]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:28.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:27 smithi057 ceph-mon[89491]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:28.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:27 smithi084 ceph-mon[70440]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:30.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:29 smithi057 ceph-mon[89491]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:30.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:29 smithi084 ceph-mon[70440]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:31 smithi057 ceph-mon[89491]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:32.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:31 smithi084 ceph-mon[70440]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:33 smithi057 ceph-mon[89491]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:33 smithi084 ceph-mon[70440]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:35 smithi057 ceph-mon[89491]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:36.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:35 smithi084 ceph-mon[70440]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:36 smithi057 ceph-mon[89491]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:36 smithi084 ceph-mon[70440]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:40.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:39 smithi057 ceph-mon[89491]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:39 smithi084 ceph-mon[70440]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:41 smithi057 ceph-mon[89491]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:41 smithi084 ceph-mon[70440]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:43 smithi057 ceph-mon[89491]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:43 smithi084 ceph-mon[70440]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:45 smithi057 ceph-mon[89491]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:45 smithi084 ceph-mon[70440]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:47 smithi057 ceph-mon[89491]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:48.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:47 smithi084 ceph-mon[70440]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:31:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:49 smithi057 ceph-mon[89491]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:49 smithi084 ceph-mon[70440]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 -- 172.21.15.57:0/993803466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f410c8f0 msgr2=0x7f93f410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 --2- 172.21.15.57:0/993803466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f410c8f0 0x7f93f410ccc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 -- 172.21.15.57:0/993803466 shutdown_connections 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 --2- 172.21.15.57:0/993803466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f4106b80 0x7f93f4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 --2- 172.21.15.57:0/993803466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f410c8f0 0x7f93f410ccc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.655+0000 7f93fa600700 1 -- 172.21.15.57:0/993803466 >> 172.21.15.57:0/993803466 conn(0x7f93f4076060 msgr2=0x7f93f4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:50.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.656+0000 7f93fa600700 1 -- 172.21.15.57:0/993803466 shutdown_connections 2024-02-28T23:31:50.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.656+0000 7f93fa600700 1 -- 172.21.15.57:0/993803466 wait complete. 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 Processor -- start 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 -- start start 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f410c8f0 0x7f93f419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419d120 con 0x7f93f4106b80 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93fa600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41a0eb0 con 0x7f93f410c8f0 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.657+0000 7f93f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:50.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49732/0 (socket says 172.21.15.57:49732) 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 -- 172.21.15.57:0/398325052 learned_addr learned my addr 172.21.15.57:0/398325052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f2a00700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f410c8f0 0x7f93f419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 -- 172.21.15.57:0/398325052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f410c8f0 msgr2=0x7f93f419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f410c8f0 0x7f93f419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 -- 172.21.15.57:0/398325052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40095f0 con 0x7f93f4106b80 2024-02-28T23:31:50.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.658+0000 7f93f3400700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f690 tx=0x7f93e400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.659+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e401d070 con 0x7f93f4106b80 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.659+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93e40050d0 con 0x7f93f4106b80 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.659+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e4022470 con 0x7f93f4106b80 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.659+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f41a1130 con 0x7f93f4106b80 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.659+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41a1620 con 0x7f93f4106b80 2024-02-28T23:31:50.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.660+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4109b20 con 0x7f93f4106b80 2024-02-28T23:31:50.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.661+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f93e4003680 con 0x7f93f4106b80 2024-02-28T23:31:50.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.662+0000 7f93ebe00700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 0x7f93d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:50.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.662+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f93e408e970 con 0x7f93f4106b80 2024-02-28T23:31:50.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.662+0000 7f93f2a00700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 0x7f93d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:50.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.663+0000 7f93f2a00700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 0x7f93d806fd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93f419db20 tx=0x7f93e0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:50.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.667+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f93e4058100 con 0x7f93f4106b80 2024-02-28T23:31:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.919+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f419d970 con 0x7f93d806d8a0 2024-02-28T23:31:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.921+0000 7f93ebe00700 1 -- 172.21.15.57:0/398325052 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f93f419d970 con 0x7f93d806d8a0 2024-02-28T23:31:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 msgr2=0x7f93d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 0x7f93d806fd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93f419db20 tx=0x7f93e0009260 comp rx=0 tx=0).stop 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 msgr2=0x7f93f419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f690 tx=0x7f93e400f770 comp rx=0 tx=0).stop 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 shutdown_connections 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f93d806d8a0 0x7f93d806fd50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106b80 0x7f93f419c500 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 --2- 172.21.15.57:0/398325052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f410c8f0 0x7f93f419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 >> 172.21.15.57:0/398325052 conn(0x7f93f4076060 msgr2=0x7f93f4077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.924+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 shutdown_connections 2024-02-28T23:31:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:50.925+0000 7f93fa600700 1 -- 172.21.15.57:0/398325052 wait complete. 2024-02-28T23:31:50.939 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.089+0000 7f555f000700 1 -- 172.21.15.57:0/3442151738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 msgr2=0x7f55581028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.089+0000 7f555f000700 1 --2- 172.21.15.57:0/3442151738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f55581028a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f554c009910 tx=0x7f554c009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.089+0000 7f555f000700 1 -- 172.21.15.57:0/3442151738 shutdown_connections 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.089+0000 7f555f000700 1 --2- 172.21.15.57:0/3442151738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f5558106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.089+0000 7f555f000700 1 --2- 172.21.15.57:0/3442151738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f55581028a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.090+0000 7f555f000700 1 -- 172.21.15.57:0/3442151738 >> 172.21.15.57:0/3442151738 conn(0x7f55580fdd90 msgr2=0x7f55581001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.090+0000 7f555f000700 1 -- 172.21.15.57:0/3442151738 shutdown_connections 2024-02-28T23:31:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.090+0000 7f555f000700 1 -- 172.21.15.57:0/3442151738 wait complete. 2024-02-28T23:31:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 Processor -- start 2024-02-28T23:31:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 -- start start 2024-02-28T23:31:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f55581983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558198a80 con 0x7f55581024d0 2024-02-28T23:31:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.091+0000 7f555f000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555819c810 con 0x7f5558102e70 2024-02-28T23:31:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49756/0 (socket says 172.21.15.57:49756) 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 -- 172.21.15.57:0/1027326644 learned_addr learned my addr 172.21.15.57:0/1027326644 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555d200700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f55581983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 -- 172.21.15.57:0/1027326644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 msgr2=0x7f55581983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f55581983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555dc00700 1 -- 172.21.15.57:0/1027326644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0095f0 con 0x7f55581024d0 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.092+0000 7f555d200700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f55581983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f555dc00700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f554c0092e0 tx=0x7f554c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554c01d070 con 0x7f55581024d0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555819ca90 con 0x7f55581024d0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f554c005e00 con 0x7f55581024d0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554c017400 con 0x7f55581024d0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.093+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555819cf80 con 0x7f55581024d0 2024-02-28T23:31:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.094+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555810a7e0 con 0x7f55581024d0 2024-02-28T23:31:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.095+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f554c00f460 con 0x7f55581024d0 2024-02-28T23:31:51.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.096+0000 7f5556000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 0x7f554006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.096+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f554c08eb00 con 0x7f55581024d0 2024-02-28T23:31:51.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.097+0000 7f555d200700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 0x7f554006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.097+0000 7f555d200700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 0x7f554006fd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f5544005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.101+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f554c058210 con 0x7f55581024d0 2024-02-28T23:31:51.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.351+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f555819d260 con 0x7f554006d8a0 2024-02-28T23:31:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.354+0000 7f5556000700 1 -- 172.21.15.57:0/1027326644 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f555819d260 con 0x7f554006d8a0 2024-02-28T23:31:51.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 msgr2=0x7f554006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 0x7f554006fd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f5544005ee0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 msgr2=0x7f5558197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f554c0092e0 tx=0x7f554c00be20 comp rx=0 tx=0).stop 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 shutdown_connections 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f554006d8a0 0x7f554006fd50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55581024d0 0x7f5558197e60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 --2- 172.21.15.57:0/1027326644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5558102e70 0x7f55581983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 >> 172.21.15.57:0/1027326644 conn(0x7f55580fdd90 msgr2=0x7f5558100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 shutdown_connections 2024-02-28T23:31:51.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.357+0000 7f555f000700 1 -- 172.21.15.57:0/1027326644 wait complete. 2024-02-28T23:31:51.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 -- 172.21.15.57:0/2201562422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101d40 msgr2=0x7fcb1810a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/2201562422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101d40 0x7fcb1810a240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009910 tx=0x7fcb04009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 -- 172.21.15.57:0/2201562422 shutdown_connections 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/2201562422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101d40 0x7fcb1810a240 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/2201562422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101430 0x7fcb18101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 -- 172.21.15.57:0/2201562422 >> 172.21.15.57:0/2201562422 conn(0x7fcb180fac70 msgr2=0x7fcb180fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 -- 172.21.15.57:0/2201562422 shutdown_connections 2024-02-28T23:31:51.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.515+0000 7fcb1cc00700 1 -- 172.21.15.57:0/2201562422 wait complete. 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.516+0000 7fcb1cc00700 1 Processor -- start 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb1cc00700 1 -- start start 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb1cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb1cc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101d40 0x7fcb18198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb1cc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18198c60 con 0x7fcb18101430 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb1cc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1819c9f0 con 0x7fcb18101d40 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb15e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb15e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49784/0 (socket says 172.21.15.57:49784) 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.517+0000 7fcb15e00700 1 -- 172.21.15.57:0/3547072806 learned_addr learned my addr 172.21.15.57:0/3547072806 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb15400700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101d40 0x7fcb18198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb15e00700 1 -- 172.21.15.57:0/3547072806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101d40 msgr2=0x7fcb18198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb15e00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101d40 0x7fcb18198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb15e00700 1 -- 172.21.15.57:0/3547072806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040095f0 con 0x7fcb18101430 2024-02-28T23:31:51.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb15e00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800c8a0 tx=0x7fcb0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.518+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb0800cea0 con 0x7fcb18101430 2024-02-28T23:31:51.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.519+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb08004830 con 0x7fcb18101430 2024-02-28T23:31:51.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.519+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1819ccd0 con 0x7fcb18101430 2024-02-28T23:31:51.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.519+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb080055b0 con 0x7fcb18101430 2024-02-28T23:31:51.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.519+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1819d220 con 0x7fcb18101430 2024-02-28T23:31:51.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.520+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb18109d10 con 0x7fcb18101430 2024-02-28T23:31:51.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.521+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcb080049a0 con 0x7fcb18101430 2024-02-28T23:31:51.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.522+0000 7fcb0ea00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 0x7fcafc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.522+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb0808d260 con 0x7fcb18101430 2024-02-28T23:31:51.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.522+0000 7fcb15400700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 0x7fcafc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.523+0000 7fcb15400700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 0x7fcafc06fd50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb18199660 tx=0x7fcb0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.526+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb08056970 con 0x7fcb18101430 2024-02-28T23:31:51.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.759+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb1819d4a0 con 0x7fcafc06d8a0 2024-02-28T23:31:51.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.772+0000 7fcb0ea00700 1 -- 172.21.15.57:0/3547072806 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcb1819d4a0 con 0x7fcafc06d8a0 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (21m) 12m ago 35m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (31m) 12m ago 35m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (34m) 12m ago 34m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (17m) 12m ago 35m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (17m) 12m ago 34m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (20m) 12m ago 34m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (13m) 12m ago 28m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (13m) 12m ago 27m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (27m) 12m ago 27m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:31:51.772 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (12m) 12m ago 28m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (23m) 12m ago 37m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (22m) 12m ago 33m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (19m) 12m ago 37m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (18m) 12m ago 33m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (22m) 12m ago 35m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (22m) 12m ago 33m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (16m) 12m ago 32m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (16m) 12m ago 32m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (15m) 12m ago 31m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (15m) 12m ago 31m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (14m) 12m ago 30m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (13m) 12m ago 29m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:31:51.773 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (21m) 12m ago 34m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 msgr2=0x7fcafc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 0x7fcafc06fd50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb18199660 tx=0x7fcb0401a040 comp rx=0 tx=0).stop 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 msgr2=0x7fcb18198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800c8a0 tx=0x7fcb0800cc60 comp rx=0 tx=0).stop 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 shutdown_connections 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcafc06d8a0 0x7fcafc06fd50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb18101430 0x7fcb18198040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 --2- 172.21.15.57:0/3547072806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb18101d40 0x7fcb18198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.775+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 >> 172.21.15.57:0/3547072806 conn(0x7fcb180fac70 msgr2=0x7fcb180ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:51.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.776+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 shutdown_connections 2024-02-28T23:31:51.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.776+0000 7fcb1cc00700 1 -- 172.21.15.57:0/3547072806 wait complete. 2024-02-28T23:31:51.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 -- 172.21.15.57:0/3525897237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 msgr2=0x7f8cf4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 --2- 172.21.15.57:0/3525897237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf4106ff0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009910 tx=0x7f8cdc009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 -- 172.21.15.57:0/3525897237 shutdown_connections 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 --2- 172.21.15.57:0/3525897237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf4106ff0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 --2- 172.21.15.57:0/3525897237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 -- 172.21.15.57:0/3525897237 >> 172.21.15.57:0/3525897237 conn(0x7f8cf4076060 msgr2=0x7f8cf4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 -- 172.21.15.57:0/3525897237 shutdown_connections 2024-02-28T23:31:51.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.930+0000 7f8cf9400700 1 -- 172.21.15.57:0/3525897237 wait complete. 2024-02-28T23:31:51.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.931+0000 7f8cf9400700 1 Processor -- start 2024-02-28T23:31:51.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.931+0000 7f8cf9400700 1 -- start start 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf9400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf9400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf419d060 con 0x7f8cf4106b80 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf9400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a0df0 con 0x7f8cf410c8f0 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf2800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf2800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49802/0 (socket says 172.21.15.57:49802) 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf2800700 1 -- 172.21.15.57:0/550465565 learned_addr learned my addr 172.21.15.57:0/550465565 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8ce9e00700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf2800700 1 -- 172.21.15.57:0/550465565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 msgr2=0x7f8cf419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.932+0000 7f8cf2800700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cf2800700 1 -- 172.21.15.57:0/550465565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0095f0 con 0x7f8cf4106b80 2024-02-28T23:31:51.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8ce9e00700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:51.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cf2800700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400ada0 tx=0x7f8ce400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce400f070 con 0x7f8cf4106b80 2024-02-28T23:31:51.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce4007bf0 con 0x7f8cf4106b80 2024-02-28T23:31:51.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce4005690 con 0x7f8cf4106b80 2024-02-28T23:31:51.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf41a10d0 con 0x7f8cf4106b80 2024-02-28T23:31:51.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.933+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41a1620 con 0x7f8cf4106b80 2024-02-28T23:31:51.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.935+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf410ecc0 con 0x7f8cf4106b80 2024-02-28T23:31:51.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.936+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ce4005850 con 0x7f8cf4106b80 2024-02-28T23:31:51.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.936+0000 7f8cebe00700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 0x7f8cd806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:51.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.937+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ce408ce80 con 0x7f8cf4106b80 2024-02-28T23:31:51.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.937+0000 7f8ce9e00700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 0x7f8cd806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:51.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.937+0000 7f8ce9e00700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 0x7f8cd806fd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc006010 tx=0x7f8cdc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:51.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:51.941+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ce40566b0 con 0x7f8cf4106b80 2024-02-28T23:31:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:51 smithi057 ceph-mon[89491]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:51 smithi057 ceph-mon[89491]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:51 smithi084 ceph-mon[70440]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:51 smithi084 ceph-mon[70440]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:52.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.253+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8cf404f2a0 con 0x7f8cf4106b80 2024-02-28T23:31:52.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.254+0000 7f8cebe00700 1 -- 172.21.15.57:0/550465565 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f8ce4059d70 con 0x7f8cf4106b80 2024-02-28T23:31:52.253 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:31:52.253 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:31:52.253 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:31:52.254 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:31:52.255 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:31:52.255 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.256+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 msgr2=0x7f8cd806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.256+0000 7f8cf9400700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 0x7f8cd806fd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc006010 tx=0x7f8cdc00b540 comp rx=0 tx=0).stop 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.256+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 msgr2=0x7f8cf419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.256+0000 7f8cf9400700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400ada0 tx=0x7f8ce400c330 comp rx=0 tx=0).stop 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 shutdown_connections 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8cd806d8a0 0x7f8cd806fd50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cf4106b80 0x7f8cf419c440 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 --2- 172.21.15.57:0/550465565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cf410c8f0 0x7f8cf419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 >> 172.21.15.57:0/550465565 conn(0x7f8cf4076060 msgr2=0x7f8cf4077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:52.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 shutdown_connections 2024-02-28T23:31:52.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.257+0000 7f8cf9400700 1 -- 172.21.15.57:0/550465565 wait complete. 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.412+0000 7efe90400700 1 -- 172.21.15.57:0/3671083564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe8810c8f0 msgr2=0x7efe8810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.412+0000 7efe90400700 1 --2- 172.21.15.57:0/3671083564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe8810c8f0 0x7efe8810ccc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009910 tx=0x7efe7c009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 -- 172.21.15.57:0/3671083564 shutdown_connections 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 --2- 172.21.15.57:0/3671083564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88106b80 0x7efe88106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 --2- 172.21.15.57:0/3671083564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe8810c8f0 0x7efe8810ccc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 -- 172.21.15.57:0/3671083564 >> 172.21.15.57:0/3671083564 conn(0x7efe88076060 msgr2=0x7efe88078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 -- 172.21.15.57:0/3671083564 shutdown_connections 2024-02-28T23:31:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.413+0000 7efe90400700 1 -- 172.21.15.57:0/3671083564 wait complete. 2024-02-28T23:31:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.414+0000 7efe90400700 1 Processor -- start 2024-02-28T23:31:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.414+0000 7efe90400700 1 -- start start 2024-02-28T23:31:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe90400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe90400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 0x7efe8819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe90400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8819d040 con 0x7efe88106b80 2024-02-28T23:31:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe90400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881a0dd0 con 0x7efe8810c8f0 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49826/0 (socket says 172.21.15.57:49826) 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 -- 172.21.15.57:0/3966468978 learned_addr learned my addr 172.21.15.57:0/3966468978 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8d000700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 0x7efe8819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 -- 172.21.15.57:0/3966468978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 msgr2=0x7efe8819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 0x7efe8819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8da00700 1 -- 172.21.15.57:0/3966468978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0095f0 con 0x7efe88106b80 2024-02-28T23:31:52.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.415+0000 7efe8d000700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 0x7efe8819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe8da00700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00f690 tx=0x7efe7c00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7c01d070 con 0x7efe88106b80 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe7c0050d0 con 0x7efe88106b80 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7c022470 con 0x7efe88106b80 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe881a1050 con 0x7efe88106b80 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.416+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe881a14f0 con 0x7efe88106b80 2024-02-28T23:31:52.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.417+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe88109b20 con 0x7efe88106b80 2024-02-28T23:31:52.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.418+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efe7c003680 con 0x7efe88106b80 2024-02-28T23:31:52.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.419+0000 7efe86000700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 0x7efe7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.419+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efe7c08ea10 con 0x7efe88106b80 2024-02-28T23:31:52.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.419+0000 7efe8d000700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 0x7efe7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.420+0000 7efe8d000700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 0x7efe7006fd50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe8819da40 tx=0x7efe74009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:52.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.424+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe7c058120 con 0x7efe88106b80 2024-02-28T23:31:52.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.689+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe8802cc30 con 0x7efe88106b80 2024-02-28T23:31:52.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.690+0000 7efe86000700 1 -- 172.21.15.57:0/3966468978 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7efe7c05b7e0 con 0x7efe88106b80 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:31:52.690 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:31:52.691 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:52.692 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:31:52.693 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:31:52.693 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:52.693 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:52.693 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:31:52.693 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.692+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 msgr2=0x7efe7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.692+0000 7efe90400700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 0x7efe7006fd50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe8819da40 tx=0x7efe74009260 comp rx=0 tx=0).stop 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.692+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 msgr2=0x7efe8819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.692+0000 7efe90400700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00f690 tx=0x7efe7c00f770 comp rx=0 tx=0).stop 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 shutdown_connections 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe7006d8a0 0x7efe7006fd50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe88106b80 0x7efe8819c420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 --2- 172.21.15.57:0/3966468978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe8810c8f0 0x7efe8819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 >> 172.21.15.57:0/3966468978 conn(0x7efe88076060 msgr2=0x7efe88077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 shutdown_connections 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.693+0000 7efe90400700 1 -- 172.21.15.57:0/3966468978 wait complete. 2024-02-28T23:31:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.854+0000 7f112da00700 1 -- 172.21.15.57:0/694961807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11281004e0 msgr2=0x7f11281008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.854+0000 7f112da00700 1 --2- 172.21.15.57:0/694961807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11281004e0 0x7f11281008b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1118009910 tx=0x7f1118009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 -- 172.21.15.57:0/694961807 shutdown_connections 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 --2- 172.21.15.57:0/694961807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1128100e80 0x7f1128104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 --2- 172.21.15.57:0/694961807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11281004e0 0x7f11281008b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 -- 172.21.15.57:0/694961807 >> 172.21.15.57:0/694961807 conn(0x7f11280fbda0 msgr2=0x7f11280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:52.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 -- 172.21.15.57:0/694961807 shutdown_connections 2024-02-28T23:31:52.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.855+0000 7f112da00700 1 -- 172.21.15.57:0/694961807 wait complete. 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 Processor -- start 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 -- start start 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 0x7f11281980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128198c80 con 0x7f1128100e80 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.856+0000 7f112da00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128198dc0 con 0x7f11281004e0 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49856/0 (socket says 172.21.15.57:49856) 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 -- 172.21.15.57:0/2772464962 learned_addr learned my addr 172.21.15.57:0/2772464962 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f1126a00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 0x7f11281980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 -- 172.21.15.57:0/2772464962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 msgr2=0x7f11281980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 0x7f11281980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 -- 172.21.15.57:0/2772464962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11180095f0 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f1126a00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 0x7f11281980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.857+0000 7f111fe00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f111000c960 tx=0x7f111000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.858+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1110007a70 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.858+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112819cc10 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.858+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1110007bd0 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.858+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1110018660 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.858+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112819d160 con 0x7f1128100e80 2024-02-28T23:31:52.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.859+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11281045d0 con 0x7f1128100e80 2024-02-28T23:31:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.860+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f111000faa0 con 0x7f1128100e80 2024-02-28T23:31:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.861+0000 7f111f400700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 0x7f110c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.861+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f111008d5f0 con 0x7f1128100e80 2024-02-28T23:31:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.862+0000 7f1126a00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 0x7f110c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.862+0000 7f1126a00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 0x7f110c06fda0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1118006010 tx=0x7f111800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:52.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:52.866+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1110056d00 con 0x7f1128100e80 2024-02-28T23:31:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:52 smithi057 ceph-mon[89491]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:52 smithi057 ceph-mon[89491]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:52 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/550465565' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:31:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:52 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3966468978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:31:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:52 smithi084 ceph-mon[70440]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:52 smithi084 ceph-mon[70440]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:52 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/550465565' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:31:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:52 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3966468978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:31:53.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.105+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1128199560 con 0x7f110c06d8f0 2024-02-28T23:31:53.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.107+0000 7f111f400700 1 -- 172.21.15.57:0/2772464962 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1128199560 con 0x7f110c06d8f0 2024-02-28T23:31:53.106 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:31:53.106 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:31:53.106 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:31:53.107 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:31:53.108 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:31:53.108 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:31:53.108 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:31:53.108 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 msgr2=0x7f110c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 0x7f110c06fda0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1118006010 tx=0x7f111800b540 comp rx=0 tx=0).stop 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 msgr2=0x7f1128198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f111000c960 tx=0x7f111000cd20 comp rx=0 tx=0).stop 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 shutdown_connections 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f110c06d8f0 0x7f110c06fda0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11281004e0 0x7f11281980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 --2- 172.21.15.57:0/2772464962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1128100e80 0x7f1128198630 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.109+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 >> 172.21.15.57:0/2772464962 conn(0x7f11280fbda0 msgr2=0x7f11280fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:53.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.110+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 shutdown_connections 2024-02-28T23:31:53.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.110+0000 7f112da00700 1 -- 172.21.15.57:0/2772464962 wait complete. 2024-02-28T23:31:53.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.264+0000 7f4905e00700 1 -- 172.21.15.57:0/2829669570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 msgr2=0x7f49000ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.264+0000 7f4905e00700 1 --2- 172.21.15.57:0/2829669570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f49000ff790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009910 tx=0x7f48f0009c20 comp rx=0 tx=0).stop 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 -- 172.21.15.57:0/2829669570 shutdown_connections 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 --2- 172.21.15.57:0/2829669570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f4900103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 --2- 172.21.15.57:0/2829669570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f49000ff790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 -- 172.21.15.57:0/2829669570 >> 172.21.15.57:0/2829669570 conn(0x7f49000fac60 msgr2=0x7f49000fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 -- 172.21.15.57:0/2829669570 shutdown_connections 2024-02-28T23:31:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.265+0000 7f4905e00700 1 -- 172.21.15.57:0/2829669570 wait complete. 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.266+0000 7f4905e00700 1 Processor -- start 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.266+0000 7f4905e00700 1 -- start start 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.266+0000 7f4905e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f4905e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f49001985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f4905e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4900198ca0 con 0x7f49000ff3c0 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f4905e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490019ca30 con 0x7f49000ffd60 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49878/0 (socket says 172.21.15.57:49878) 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 -- 172.21.15.57:0/4034080593 learned_addr learned my addr 172.21.15.57:0/4034080593 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fe000700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f49001985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 -- 172.21.15.57:0/4034080593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 msgr2=0x7f49001985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f49001985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fea00700 1 -- 172.21.15.57:0/4034080593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00095f0 con 0x7f49000ff3c0 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.267+0000 7f48fe000700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f49001985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f48fea00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006010 tx=0x7f48f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f001d070 con 0x7f49000ff3c0 2024-02-28T23:31:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48f000f460 con 0x7f49000ff3c0 2024-02-28T23:31:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f0017630 con 0x7f49000ff3c0 2024-02-28T23:31:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490019ccb0 con 0x7f49000ff3c0 2024-02-28T23:31:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.268+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490019d1a0 con 0x7f49000ff3c0 2024-02-28T23:31:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.269+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490004f2a0 con 0x7f49000ff3c0 2024-02-28T23:31:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.271+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f48f000f5d0 con 0x7f49000ff3c0 2024-02-28T23:31:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.271+0000 7f4904a00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 0x7f48e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:31:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.272+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f48f008ea30 con 0x7f49000ff3c0 2024-02-28T23:31:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.272+0000 7f48fe000700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 0x7f48e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:31:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.273+0000 7f48fe000700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 0x7f48e406fd50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f48ec005fd0 tx=0x7f48ec005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:31:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.277+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48f00581c0 con 0x7f49000ff3c0 2024-02-28T23:31:53.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.599+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f49001993e0 con 0x7f49000ff3c0 2024-02-28T23:31:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.600+0000 7f4904a00700 1 -- 172.21.15.57:0/4034080593 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f48f005b880 con 0x7f49000ff3c0 2024-02-28T23:31:53.599 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:31:53.600 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:31:53.600 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:31:53.600 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:31:53.600 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.602+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 msgr2=0x7f48e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.602+0000 7f4905e00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 0x7f48e406fd50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f48ec005fd0 tx=0x7f48ec005ee0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 msgr2=0x7f4900198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006010 tx=0x7f48f000be20 comp rx=0 tx=0).stop 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 shutdown_connections 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f48e406d8a0 0x7f48e406fd50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49000ff3c0 0x7f4900198080 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 --2- 172.21.15.57:0/4034080593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000ffd60 0x7f49001985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:31:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 >> 172.21.15.57:0/4034080593 conn(0x7f49000fac60 msgr2=0x7f4900068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:31:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 shutdown_connections 2024-02-28T23:31:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:31:53.603+0000 7f4905e00700 1 -- 172.21.15.57:0/4034080593 wait complete. 2024-02-28T23:31:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:53 smithi057 ceph-mon[89491]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:53 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4034080593' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:31:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:53 smithi084 ceph-mon[70440]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:53 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4034080593' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:31:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:54 smithi057 ceph-mon[89491]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:55.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:54 smithi084 ceph-mon[70440]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:31:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:55 smithi057 ceph-mon[89491]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:55 smithi084 ceph-mon[70440]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:57 smithi084 ceph-mon[70440]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:31:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:57 smithi057 ceph-mon[89491]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:00.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:31:59 smithi057 ceph-mon[89491]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:00.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:31:59 smithi084 ceph-mon[70440]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:02.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:01 smithi057 ceph-mon[89491]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:01 smithi084 ceph-mon[70440]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:04.288 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:03 smithi084 ceph-mon[70440]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:04.289 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:03 smithi057 ceph-mon[89491]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:05 smithi057 ceph-mon[89491]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:06.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:05 smithi084 ceph-mon[70440]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:07 smithi057 ceph-mon[89491]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:08.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:07 smithi084 ceph-mon[70440]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:09 smithi057 ceph-mon[89491]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:10.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:09 smithi084 ceph-mon[70440]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:12.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:11 smithi057 ceph-mon[89491]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:12.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:11 smithi084 ceph-mon[70440]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:13 smithi057 ceph-mon[89491]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:14.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:13 smithi084 ceph-mon[70440]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:16.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:15 smithi057 ceph-mon[89491]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:16.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:15 smithi084 ceph-mon[70440]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:18.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:17 smithi057 ceph-mon[89491]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:17 smithi084 ceph-mon[70440]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:20.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:19 smithi084 ceph-mon[70440]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:19 smithi057 ceph-mon[89491]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:22.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:21 smithi057 ceph-mon[89491]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:22.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:21 smithi084 ceph-mon[70440]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 -- 172.21.15.57:0/3863300015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 msgr2=0x7fce201028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 --2- 172.21.15.57:0/3863300015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201028e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009910 tx=0x7fce0c009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 -- 172.21.15.57:0/3863300015 shutdown_connections 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 --2- 172.21.15.57:0/3863300015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201028e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 --2- 172.21.15.57:0/3863300015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 0x7fce20108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.779+0000 7fce28200700 1 -- 172.21.15.57:0/3863300015 >> 172.21.15.57:0/3863300015 conn(0x7fce200fdf90 msgr2=0x7fce201003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:23.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.780+0000 7fce28200700 1 -- 172.21.15.57:0/3863300015 shutdown_connections 2024-02-28T23:32:23.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.780+0000 7fce28200700 1 -- 172.21.15.57:0/3863300015 wait complete. 2024-02-28T23:32:23.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.781+0000 7fce28200700 1 Processor -- start 2024-02-28T23:32:23.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.781+0000 7fce28200700 1 -- start start 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce28200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce28200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 0x7fce20198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce28200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20198e10 con 0x7fce20102470 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce28200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2019cba0 con 0x7fce20108470 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44712/0 (socket says 172.21.15.57:44712) 2024-02-28T23:32:23.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 -- 172.21.15.57:0/2798892184 learned_addr learned my addr 172.21.15.57:0/2798892184 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:23.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce24e00700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 0x7fce20198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:23.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 -- 172.21.15.57:0/2798892184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 msgr2=0x7fce20198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:23.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 0x7fce20198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:23.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.782+0000 7fce25800700 1 -- 172.21.15.57:0/2798892184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0095f0 con 0x7fce20102470 2024-02-28T23:32:23.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce25800700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d8d0 tx=0x7fce1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce14009940 con 0x7fce20102470 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce14010460 con 0x7fce20102470 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1400f5f0 con 0x7fce20102470 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2019ce80 con 0x7fce20102470 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.783+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2019d3d0 con 0x7fce20102470 2024-02-28T23:32:23.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.784+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2010a970 con 0x7fce20102470 2024-02-28T23:32:23.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.785+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fce140105d0 con 0x7fce20102470 2024-02-28T23:32:23.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.786+0000 7fce1e000700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 0x7fce0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:23.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.786+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fce1408d4c0 con 0x7fce20102470 2024-02-28T23:32:23.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.787+0000 7fce24e00700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 0x7fce0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:23.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.787+0000 7fce24e00700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 0x7fce0806fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce20199810 tx=0x7fce0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:23.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:23.792+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce14056c50 con 0x7fce20102470 2024-02-28T23:32:24.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.047+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce20199690 con 0x7fce0806d8a0 2024-02-28T23:32:24.047 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:23 smithi057 ceph-mon[89491]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:24.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.049+0000 7fce1e000700 1 -- 172.21.15.57:0/2798892184 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fce0c0055e0 con 0x7fce0806d8a0 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 msgr2=0x7fce0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 0x7fce0806fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce20199810 tx=0x7fce0c01a040 comp rx=0 tx=0).stop 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 msgr2=0x7fce201981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d8d0 tx=0x7fce1400dc90 comp rx=0 tx=0).stop 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 shutdown_connections 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fce0806d8a0 0x7fce0806fd50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fce20102470 0x7fce201981f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 --2- 172.21.15.57:0/2798892184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce20108470 0x7fce20198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.052+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 >> 172.21.15.57:0/2798892184 conn(0x7fce200fdf90 msgr2=0x7fce200ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:24.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.053+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 shutdown_connections 2024-02-28T23:32:24.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.053+0000 7fce28200700 1 -- 172.21.15.57:0/2798892184 wait complete. 2024-02-28T23:32:24.069 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.215+0000 7f0588800700 1 -- 172.21.15.57:0/744267788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05801004e0 msgr2=0x7f05801008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.215+0000 7f0588800700 1 --2- 172.21.15.57:0/744267788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05801004e0 0x7f05801008b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0574009910 tx=0x7f0574009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 -- 172.21.15.57:0/744267788 shutdown_connections 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 --2- 172.21.15.57:0/744267788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0580100e80 0x7f0580104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 --2- 172.21.15.57:0/744267788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05801004e0 0x7f05801008b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 -- 172.21.15.57:0/744267788 >> 172.21.15.57:0/744267788 conn(0x7f05800fbda0 msgr2=0x7f05800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:24.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 -- 172.21.15.57:0/744267788 shutdown_connections 2024-02-28T23:32:24.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.216+0000 7f0588800700 1 -- 172.21.15.57:0/744267788 wait complete. 2024-02-28T23:32:24.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.218+0000 7f0588800700 1 Processor -- start 2024-02-28T23:32:24.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.218+0000 7f0588800700 1 -- start start 2024-02-28T23:32:24.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0588800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 0x7f0580195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0588800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0588800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580196a80 con 0x7f0580100e80 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0588800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580196bc0 con 0x7f05801004e0 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0585400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0585400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44728/0 (socket says 172.21.15.57:44728) 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0585400700 1 -- 172.21.15.57:0/4140897160 learned_addr learned my addr 172.21.15.57:0/4140897160 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:24.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.219+0000 7f0585e00700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 0x7f0580195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.220+0000 7f0585400700 1 -- 172.21.15.57:0/4140897160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 msgr2=0x7f0580195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.220+0000 7f0585400700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 0x7f0580195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.220+0000 7f0585400700 1 -- 172.21.15.57:0/4140897160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740095f0 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.220+0000 7f0585e00700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 0x7f0580195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.220+0000 7f0585400700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f057000c960 tx=0x7f057000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.221+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0570007a70 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.221+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0570007bd0 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.221+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058019aa10 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.221+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05700186c0 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.221+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058019af60 con 0x7f0580100e80 2024-02-28T23:32:24.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.222+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05801087f0 con 0x7f0580100e80 2024-02-28T23:32:24.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.223+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f057000faa0 con 0x7f0580100e80 2024-02-28T23:32:24.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.224+0000 7f057aa00700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 0x7f056806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.224+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f057008d4c0 con 0x7f0580100e80 2024-02-28T23:32:24.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.225+0000 7f0585e00700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 0x7f056806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.225+0000 7f0585e00700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 0x7f056806fd50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0574005100 tx=0x7f057401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:24.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.229+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f057005a820 con 0x7f0580100e80 2024-02-28T23:32:24.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:23 smithi084 ceph-mon[70440]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.480+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f058002cca0 con 0x7f056806d8a0 2024-02-28T23:32:24.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.483+0000 7f057aa00700 1 -- 172.21.15.57:0/4140897160 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f058002cca0 con 0x7f056806d8a0 2024-02-28T23:32:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 msgr2=0x7f056806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 0x7f056806fd50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0574005100 tx=0x7f057401a040 comp rx=0 tx=0).stop 2024-02-28T23:32:24.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 msgr2=0x7f0580196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f057000c960 tx=0x7f057000cd20 comp rx=0 tx=0).stop 2024-02-28T23:32:24.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 shutdown_connections 2024-02-28T23:32:24.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f056806d8a0 0x7f056806fd50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05801004e0 0x7f0580195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 --2- 172.21.15.57:0/4140897160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0580100e80 0x7f0580196430 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 >> 172.21.15.57:0/4140897160 conn(0x7f05800fbda0 msgr2=0x7f05800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 shutdown_connections 2024-02-28T23:32:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.486+0000 7f0588800700 1 -- 172.21.15.57:0/4140897160 wait complete. 2024-02-28T23:32:24.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.653+0000 7f0ad6e00700 1 -- 172.21.15.57:0/2732319848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 msgr2=0x7f0ad00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.653+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/2732319848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad00734c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009910 tx=0x7f0ac4009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 -- 172.21.15.57:0/2732319848 shutdown_connections 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/2732319848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/2732319848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad00734c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 -- 172.21.15.57:0/2732319848 >> 172.21.15.57:0/2732319848 conn(0x7f0ad00fbb10 msgr2=0x7f0ad00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 -- 172.21.15.57:0/2732319848 shutdown_connections 2024-02-28T23:32:24.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.654+0000 7f0ad6e00700 1 -- 172.21.15.57:0/2732319848 wait complete. 2024-02-28T23:32:24.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 Processor -- start 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 -- start start 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0197b30 con 0x7f0ad00730f0 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.655+0000 7f0ad6e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0197ca0 con 0x7f0ad0073a00 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44746/0 (socket says 172.21.15.57:44746) 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 -- 172.21.15.57:0/3241518556 learned_addr learned my addr 172.21.15.57:0/3241518556 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5000700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 -- 172.21.15.57:0/3241518556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 msgr2=0x7f0ad019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5a00700 1 -- 172.21.15.57:0/3241518556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40095f0 con 0x7f0ad00730f0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.656+0000 7f0ad5000700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ad5a00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4006010 tx=0x7f0ac400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac401d070 con 0x7f0ad00730f0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ac400f460 con 0x7f0ad00730f0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4017630 con 0x7f0ad00730f0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad0197f80 con 0x7f0ad00730f0 2024-02-28T23:32:24.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.657+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad01984d0 con 0x7f0ad00730f0 2024-02-28T23:32:24.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.658+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad0109a30 con 0x7f0ad00730f0 2024-02-28T23:32:24.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.659+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0ac400f5d0 con 0x7f0ad00730f0 2024-02-28T23:32:24.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.660+0000 7f0ace000700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:24.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.660+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ac408ea20 con 0x7f0ad00730f0 2024-02-28T23:32:24.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.661+0000 7f0ad5000700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:24.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.661+0000 7f0ad5000700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:24.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.665+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ac405bc60 con 0x7f0ad00730f0 2024-02-28T23:32:24.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.919+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad010f330 con 0x7f0ab806d8a0 2024-02-28T23:32:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.939+0000 7f0ace000700 1 -- 172.21.15.57:0/3241518556 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0ad010f330 con 0x7f0ab806d8a0 2024-02-28T23:32:24.939 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:32:24.939 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (22m) 13m ago 35m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:32:24.939 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (32m) 13m ago 36m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:32:24.939 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (34m) 13m ago 34m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (18m) 13m ago 36m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (17m) 13m ago 34m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (21m) 13m ago 35m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (14m) 13m ago 28m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (13m) 13m ago 28m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (28m) 13m ago 28m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (13m) 13m ago 28m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:32:24.940 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (24m) 13m ago 37m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (23m) 13m ago 34m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (19m) 13m ago 38m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (19m) 13m ago 34m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (23m) 13m ago 36m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (22m) 13m ago 34m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (17m) 13m ago 33m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (16m) 13m ago 32m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:32:24.941 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (16m) 13m ago 32m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:32:24.942 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (15m) 13m ago 31m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:32:24.942 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (15m) 13m ago 31m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:32:24.942 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (14m) 13m ago 30m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:32:24.942 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (22m) 13m ago 35m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.942+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 msgr2=0x7f0ab806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.942+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005ee0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.942+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 msgr2=0x7f0ad019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.942+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4006010 tx=0x7f0ac400be20 comp rx=0 tx=0).stop 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 shutdown_connections 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad00730f0 0x7f0ad019dab0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 --2- 172.21.15.57:0/3241518556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0073a00 0x7f0ad019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:24.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 >> 172.21.15.57:0/3241518556 conn(0x7f0ad00fbb10 msgr2=0x7f0ad0106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:24.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 shutdown_connections 2024-02-28T23:32:24.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:24.944+0000 7f0ad6e00700 1 -- 172.21.15.57:0/3241518556 wait complete. 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.106+0000 7f9092a00700 1 -- 172.21.15.57:0/4037996556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 msgr2=0x7f908c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.106+0000 7f9092a00700 1 --2- 172.21.15.57:0/4037996556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c075650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 -- 172.21.15.57:0/4037996556 shutdown_connections 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 --2- 172.21.15.57:0/4037996556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 0x7f908c110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 --2- 172.21.15.57:0/4037996556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c075650 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 -- 172.21.15.57:0/4037996556 >> 172.21.15.57:0/4037996556 conn(0x7f908c0fdf40 msgr2=0x7f908c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:25.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 -- 172.21.15.57:0/4037996556 shutdown_connections 2024-02-28T23:32:25.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.107+0000 7f9092a00700 1 -- 172.21.15.57:0/4037996556 wait complete. 2024-02-28T23:32:25.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.108+0000 7f9092a00700 1 Processor -- start 2024-02-28T23:32:25.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.109+0000 7f9092a00700 1 -- start start 2024-02-28T23:32:25.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.109+0000 7f9092a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.109+0000 7f9092a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 0x7f908c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.109+0000 7f9092a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19d130 con 0x7f908c075280 2024-02-28T23:32:25.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.109+0000 7f9092a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c1a0ec0 con 0x7f908c075b90 2024-02-28T23:32:25.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40986/0 (socket says 172.21.15.57:40986) 2024-02-28T23:32:25.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 -- 172.21.15.57:0/3725768070 learned_addr learned my addr 172.21.15.57:0/3725768070 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908b400700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 0x7f908c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 -- 172.21.15.57:0/3725768070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 msgr2=0x7f908c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 0x7f908c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.110+0000 7f908be00700 1 -- 172.21.15.57:0/3725768070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c0095f0 con 0x7f908c075280 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f908be00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f907c0092e0 tx=0x7f907c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c01d070 con 0x7f908c075280 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f907c005dc0 con 0x7f908c075280 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c1a1140 con 0x7f908c075280 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c021410 con 0x7f908c075280 2024-02-28T23:32:25.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.111+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c1a1690 con 0x7f908c075280 2024-02-28T23:32:25.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.112+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c10e310 con 0x7f908c075280 2024-02-28T23:32:25.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.114+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f907c003a60 con 0x7f908c075280 2024-02-28T23:32:25.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.115+0000 7f9088c00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 0x7f907006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.115+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f907c08e750 con 0x7f908c075280 2024-02-28T23:32:25.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.115+0000 7f908b400700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 0x7f907006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.116+0000 7f908b400700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 0x7f907006fd50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f908c19db30 tx=0x7f90780094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:25.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.120+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f907c057e60 con 0x7f908c075280 2024-02-28T23:32:25.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.453+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f908c19d870 con 0x7f908c075280 2024-02-28T23:32:25.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.454+0000 7f9088c00700 1 -- 172.21.15.57:0/3725768070 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f907c05b520 con 0x7f908c075280 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:32:25.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:32:25.454 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:32:25.455 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:32:25.455 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:32:25.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.457+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 msgr2=0x7f907006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.457+0000 7f9092a00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 0x7f907006fd50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f908c19db30 tx=0x7f90780094f0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.457+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 msgr2=0x7f908c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.457+0000 7f9092a00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f907c0092e0 tx=0x7f907c0056a0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 shutdown_connections 2024-02-28T23:32:25.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f907006d8a0 0x7f907006fd50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908c075280 0x7f908c19c510 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 --2- 172.21.15.57:0/3725768070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c075b90 0x7f908c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 >> 172.21.15.57:0/3725768070 conn(0x7f908c0fdf40 msgr2=0x7f908c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:25.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 shutdown_connections 2024-02-28T23:32:25.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.458+0000 7f9092a00700 1 -- 172.21.15.57:0/3725768070 wait complete. 2024-02-28T23:32:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 -- 172.21.15.57:0/1427959436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc068540 msgr2=0x7fcddc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 --2- 172.21.15.57:0/1427959436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc068540 0x7fcddc0689b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009910 tx=0x7fcdc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 -- 172.21.15.57:0/1427959436 shutdown_connections 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 --2- 172.21.15.57:0/1427959436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc068540 0x7fcddc0689b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 --2- 172.21.15.57:0/1427959436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc105230 0x7fcddc105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 -- 172.21.15.57:0/1427959436 >> 172.21.15.57:0/1427959436 conn(0x7fcddc075130 msgr2=0x7fcddc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 -- 172.21.15.57:0/1427959436 shutdown_connections 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.623+0000 7fcde1a00700 1 -- 172.21.15.57:0/1427959436 wait complete. 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.624+0000 7fcde1a00700 1 Processor -- start 2024-02-28T23:32:25.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcde1a00700 1 -- start start 2024-02-28T23:32:25.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcde1a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 0x7fcddc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcde1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcde1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc198bc0 con 0x7fcddc105230 2024-02-28T23:32:25.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcde1a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc19c950 con 0x7fcddc068540 2024-02-28T23:32:25.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcdda000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcdda000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41010/0 (socket says 172.21.15.57:41010) 2024-02-28T23:32:25.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.625+0000 7fcdda000700 1 -- 172.21.15.57:0/2596323376 learned_addr learned my addr 172.21.15.57:0/2596323376 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcddaa00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 0x7fcddc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdda000700 1 -- 172.21.15.57:0/2596323376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 msgr2=0x7fcddc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdda000700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 0x7fcddc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdda000700 1 -- 172.21.15.57:0/2596323376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80095f0 con 0x7fcddc105230 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcddaa00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 0x7fcddc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdda000700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800f690 tx=0x7fcdc800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc801d070 con 0x7fcddc105230 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdc80050d0 con 0x7fcddc105230 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc19cbd0 con 0x7fcddc105230 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.626+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc8022470 con 0x7fcddc105230 2024-02-28T23:32:25.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.627+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc19d0c0 con 0x7fcddc105230 2024-02-28T23:32:25.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.628+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc101160 con 0x7fcddc105230 2024-02-28T23:32:25.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.629+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcdc8003680 con 0x7fcddc105230 2024-02-28T23:32:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.630+0000 7fcdd3400700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 0x7fcdc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.630+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcdc808ea10 con 0x7fcddc105230 2024-02-28T23:32:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.630+0000 7fcddaa00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 0x7fcdc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.631+0000 7fcddaa00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 0x7fcdc006fd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009be0 tx=0x7fcdcc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:25.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.635+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdc8058120 con 0x7fcddc105230 2024-02-28T23:32:25.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.907+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcddc04f2a0 con 0x7fcddc105230 2024-02-28T23:32:25.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.908+0000 7fcdd3400700 1 -- 172.21.15.57:0/2596323376 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7fcdc805b7e0 con 0x7fcddc105230 2024-02-28T23:32:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:32:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:32:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:32:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:32:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:32:25.910 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 msgr2=0x7fcdc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 0x7fcdc006fd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009be0 tx=0x7fcdcc009260 comp rx=0 tx=0).stop 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 msgr2=0x7fcddc1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800f690 tx=0x7fcdc800f770 comp rx=0 tx=0).stop 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 shutdown_connections 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcdc006d8a0 0x7fcdc006fd50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcddc068540 0x7fcddc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 --2- 172.21.15.57:0/2596323376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcddc105230 0x7fcddc1984e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 >> 172.21.15.57:0/2596323376 conn(0x7fcddc075130 msgr2=0x7fcddc104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:25.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 shutdown_connections 2024-02-28T23:32:25.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:25.911+0000 7fcde1a00700 1 -- 172.21.15.57:0/2596323376 wait complete. 2024-02-28T23:32:25.912 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:32:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 -- 172.21.15.57:0/1089773218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 msgr2=0x7f87b4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 --2- 172.21.15.57:0/1089773218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b4068b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009910 tx=0x7f87a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 -- 172.21.15.57:0/1089773218 shutdown_connections 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 --2- 172.21.15.57:0/1089773218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 0x7f87b4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 --2- 172.21.15.57:0/1089773218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b4068b70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.078+0000 7f87bc400700 1 -- 172.21.15.57:0/1089773218 >> 172.21.15.57:0/1089773218 conn(0x7f87b4075130 msgr2=0x7f87b4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.079+0000 7f87bc400700 1 -- 172.21.15.57:0/1089773218 shutdown_connections 2024-02-28T23:32:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.079+0000 7f87bc400700 1 -- 172.21.15.57:0/1089773218 wait complete. 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 Processor -- start 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 -- start start 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 0x7f87b41001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4100720 con 0x7f87b40687a0 2024-02-28T23:32:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.080+0000 7f87bc400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4100860 con 0x7f87b4069140 2024-02-28T23:32:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41022/0 (socket says 172.21.15.57:41022) 2024-02-28T23:32:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 -- 172.21.15.57:0/3663521823 learned_addr learned my addr 172.21.15.57:0/3663521823 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9000700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 0x7f87b41001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 -- 172.21.15.57:0/3663521823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 msgr2=0x7f87b41001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 0x7f87b41001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 -- 172.21.15.57:0/3663521823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80095f0 con 0x7f87b40687a0 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.081+0000 7f87b9a00700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f87a800b5c0 tx=0x7f87a800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.082+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a801d070 con 0x7f87b40687a0 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.082+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87a800f460 con 0x7f87b40687a0 2024-02-28T23:32:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.082+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b4100ae0 con 0x7f87b40687a0 2024-02-28T23:32:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.082+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a8017610 con 0x7f87b40687a0 2024-02-28T23:32:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.082+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b41a2390 con 0x7f87b40687a0 2024-02-28T23:32:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.083+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4108790 con 0x7f87b40687a0 2024-02-28T23:32:26.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.084+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f87a800f5d0 con 0x7f87b40687a0 2024-02-28T23:32:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.085+0000 7f87b2000700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 0x7f879c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.085+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f87a808eb50 con 0x7f87b40687a0 2024-02-28T23:32:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.085+0000 7f87b9000700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 0x7f879c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.086+0000 7f87b9000700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 0x7f879c06fd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87b4101490 tx=0x7f87a0005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.090+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87a80582e0 con 0x7f87b40687a0 2024-02-28T23:32:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3725768070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:32:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2596323376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:32:26.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:26.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:26.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3725768070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:32:26.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2596323376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:32:26.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.339+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87b4061190 con 0x7f879c06d8a0 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.342+0000 7f87b2000700 1 -- 172.21.15.57:0/3663521823 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f87b4061190 con 0x7f879c06d8a0 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:32:26.340 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:32:26.341 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.344+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 msgr2=0x7f879c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.344+0000 7f87bc400700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 0x7f879c06fd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87b4101490 tx=0x7f87a0005d50 comp rx=0 tx=0).stop 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 msgr2=0x7f87b40ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f87a800b5c0 tx=0x7f87a800be00 comp rx=0 tx=0).stop 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 shutdown_connections 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f879c06d8a0 0x7f879c06fd50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87b40687a0 0x7f87b40ffca0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 --2- 172.21.15.57:0/3663521823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87b4069140 0x7f87b41001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 >> 172.21.15.57:0/3663521823 conn(0x7f87b4075130 msgr2=0x7f87b40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:26.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 shutdown_connections 2024-02-28T23:32:26.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.345+0000 7f87bc400700 1 -- 172.21.15.57:0/3663521823 wait complete. 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.499+0000 7f7b41a00700 1 -- 172.21.15.57:0/1514393861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 msgr2=0x7f7b3c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.499+0000 7f7b41a00700 1 --2- 172.21.15.57:0/1514393861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c10ccc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009910 tx=0x7f7b2c009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 -- 172.21.15.57:0/1514393861 shutdown_connections 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 --2- 172.21.15.57:0/1514393861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 0x7f7b3c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 --2- 172.21.15.57:0/1514393861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c10ccc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 -- 172.21.15.57:0/1514393861 >> 172.21.15.57:0/1514393861 conn(0x7f7b3c076060 msgr2=0x7f7b3c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 -- 172.21.15.57:0/1514393861 shutdown_connections 2024-02-28T23:32:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.500+0000 7f7b41a00700 1 -- 172.21.15.57:0/1514393861 wait complete. 2024-02-28T23:32:26.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.501+0000 7f7b41a00700 1 Processor -- start 2024-02-28T23:32:26.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.501+0000 7f7b41a00700 1 -- start start 2024-02-28T23:32:26.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b41a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 0x7f7b3c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b41a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b41a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c19d120 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b41a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a0eb0 con 0x7f7b3c106b80 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b3a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b3a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41044/0 (socket says 172.21.15.57:41044) 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b3a000700 1 -- 172.21.15.57:0/3634854136 learned_addr learned my addr 172.21.15.57:0/3634854136 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.502+0000 7f7b3aa00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 0x7f7b3c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b3a000700 1 -- 172.21.15.57:0/3634854136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 msgr2=0x7f7b3c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b3a000700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 0x7f7b3c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b3a000700 1 -- 172.21.15.57:0/3634854136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0095f0 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b3a000700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e920 tx=0x7f7b2800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b28009790 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b2800cba0 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b28018650 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c1a1190 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.503+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c1a16e0 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.505+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c109b20 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.506+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7b2800cd10 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.507+0000 7f7b33400700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 0x7f7b2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.507+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7b28014070 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.507+0000 7f7b3aa00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 0x7f7b2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:26.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.508+0000 7f7b3aa00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 0x7f7b2006fd50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b760 tx=0x7f7b2c009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.511+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b28052d20 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.832+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7b3c04f2a0 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.833+0000 7f7b33400700 1 -- 172.21.15.57:0/3634854136 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7b2805abd0 con 0x7f7b3c10c8f0 2024-02-28T23:32:26.831 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:32:26.831 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:32:26.831 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:32:26.831 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:32:26.832 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:32:26.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.835+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 msgr2=0x7f7b2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.835+0000 7f7b41a00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 0x7f7b2006fd50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b760 tx=0x7f7b2c009f90 comp rx=0 tx=0).stop 2024-02-28T23:32:26.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.835+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 msgr2=0x7f7b3c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:26.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.835+0000 7f7b41a00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e920 tx=0x7f7b2800ece0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 shutdown_connections 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7b2006d8a0 0x7f7b2006fd50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3c106b80 0x7f7b3c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 --2- 172.21.15.57:0/3634854136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c10c8f0 0x7f7b3c19ca40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 >> 172.21.15.57:0/3634854136 conn(0x7f7b3c076060 msgr2=0x7f7b3c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 shutdown_connections 2024-02-28T23:32:26.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:26.836+0000 7f7b41a00700 1 -- 172.21.15.57:0/3634854136 wait complete. 2024-02-28T23:32:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:26 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3634854136' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:32:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:26 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3634854136' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:32:28.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:27 smithi057 ceph-mon[89491]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:28.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:27 smithi057 ceph-mon[89491]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:27 smithi084 ceph-mon[70440]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:27 smithi084 ceph-mon[70440]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:30.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:29 smithi057 ceph-mon[89491]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:30.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:29 smithi084 ceph-mon[70440]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:31 smithi057 ceph-mon[89491]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:32.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:31 smithi084 ceph-mon[70440]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:33 smithi057 ceph-mon[89491]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:33 smithi084 ceph-mon[70440]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:35.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:35 smithi057 ceph-mon[89491]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:35 smithi084 ceph-mon[70440]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:38.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:37 smithi057 ceph-mon[89491]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:38.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:37 smithi084 ceph-mon[70440]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:40.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:39 smithi057 ceph-mon[89491]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:40.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:39 smithi084 ceph-mon[70440]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:42.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:41 smithi057 ceph-mon[89491]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:41 smithi084 ceph-mon[70440]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:43 smithi057 ceph-mon[89491]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:43 smithi084 ceph-mon[70440]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:45.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:45 smithi084 ceph-mon[70440]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:45 smithi057 ceph-mon[89491]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:47 smithi057 ceph-mon[89491]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:47 smithi084 ceph-mon[70440]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:32:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:49 smithi057 ceph-mon[89491]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:50.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:49 smithi084 ceph-mon[70440]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:52.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:51 smithi057 ceph-mon[89491]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:51 smithi084 ceph-mon[70440]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:53 smithi057 ceph-mon[89491]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:54.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:53 smithi084 ceph-mon[70440]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:56.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:55 smithi057 ceph-mon[89491]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:56.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:55 smithi084 ceph-mon[70440]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:57.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 -- 172.21.15.57:0/2413911137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 msgr2=0x7f83c8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 --2- 172.21.15.57:0/2413911137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c8106e40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009910 tx=0x7f83b0009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 -- 172.21.15.57:0/2413911137 shutdown_connections 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 --2- 172.21.15.57:0/2413911137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c8106e40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 --2- 172.21.15.57:0/2413911137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 0x7f83c81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.015+0000 7f83cd600700 1 -- 172.21.15.57:0/2413911137 >> 172.21.15.57:0/2413911137 conn(0x7f83c80fdd90 msgr2=0x7f83c81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.016+0000 7f83cd600700 1 -- 172.21.15.57:0/2413911137 shutdown_connections 2024-02-28T23:32:57.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.016+0000 7f83cd600700 1 -- 172.21.15.57:0/2413911137 wait complete. 2024-02-28T23:32:57.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 Processor -- start 2024-02-28T23:32:57.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 -- start start 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 0x7f83c8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8198ba0 con 0x7f83c8102e70 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.017+0000 7f83cd600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c819c930 con 0x7f83c81024d0 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48674/0 (socket says 172.21.15.57:48674) 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 -- 172.21.15.57:0/1813742116 learned_addr learned my addr 172.21.15.57:0/1813742116 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:57.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83c7e00700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 0x7f83c8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 -- 172.21.15.57:0/1813742116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 msgr2=0x7f83c8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 0x7f83c8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.018+0000 7f83bfe00700 1 -- 172.21.15.57:0/1813742116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00095f0 con 0x7f83c8102e70 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83bfe00700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f83b000bda0 tx=0x7f83b000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b001d070 con 0x7f83c8102e70 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83b0004fb0 con 0x7f83c8102e70 2024-02-28T23:32:57.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b000f480 con 0x7f83c8102e70 2024-02-28T23:32:57.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c819cbb0 con 0x7f83c8102e70 2024-02-28T23:32:57.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.019+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c819d0a0 con 0x7f83c8102e70 2024-02-28T23:32:57.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.020+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c810a790 con 0x7f83c8102e70 2024-02-28T23:32:57.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.021+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f83b0022870 con 0x7f83c8102e70 2024-02-28T23:32:57.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.022+0000 7f83c5600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.022+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f83b008e9a0 con 0x7f83c8102e70 2024-02-28T23:32:57.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.023+0000 7f83c7e00700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.023+0000 7f83c7e00700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 0x7f83ac06fd50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83b8006fd0 tx=0x7f83b8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.027+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83b0058280 con 0x7f83c8102e70 2024-02-28T23:32:57.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.277+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83c819d440 con 0x7f83ac06d8a0 2024-02-28T23:32:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.280+0000 7f83c5600700 1 -- 172.21.15.57:0/1813742116 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f83c819d440 con 0x7f83ac06d8a0 2024-02-28T23:32:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.283+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 msgr2=0x7f83ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.283+0000 7f83cd600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 0x7f83ac06fd50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83b8006fd0 tx=0x7f83b8008040 comp rx=0 tx=0).stop 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.283+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 msgr2=0x7f83c81984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.283+0000 7f83cd600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f83b000bda0 tx=0x7f83b000be80 comp rx=0 tx=0).stop 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 shutdown_connections 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c81024d0 0x7f83c8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 --2- 172.21.15.57:0/1813742116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83c8102e70 0x7f83c81984c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 >> 172.21.15.57:0/1813742116 conn(0x7f83c80fdd90 msgr2=0x7f83c80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 shutdown_connections 2024-02-28T23:32:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.284+0000 7f83cd600700 1 -- 172.21.15.57:0/1813742116 wait complete. 2024-02-28T23:32:57.300 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:32:57.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 -- 172.21.15.57:0/2356595297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96000739a0 msgr2=0x7f960010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 --2- 172.21.15.57:0/2356595297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96000739a0 0x7f960010c5e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009940 tx=0x7f95ec009c50 comp rx=0 tx=0).stop 2024-02-28T23:32:57.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 -- 172.21.15.57:0/2356595297 shutdown_connections 2024-02-28T23:32:57.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 --2- 172.21.15.57:0/2356595297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96000739a0 0x7f960010c5e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 --2- 172.21.15.57:0/2356595297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9600073090 0x7f9600073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 -- 172.21.15.57:0/2356595297 >> 172.21.15.57:0/2356595297 conn(0x7f96000fbda0 msgr2=0x7f96000fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:57.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 -- 172.21.15.57:0/2356595297 shutdown_connections 2024-02-28T23:32:57.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.453+0000 7f9608000700 1 -- 172.21.15.57:0/2356595297 wait complete. 2024-02-28T23:32:57.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.454+0000 7f9608000700 1 Processor -- start 2024-02-28T23:32:57.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.454+0000 7f9608000700 1 -- start start 2024-02-28T23:32:57.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9608000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9608000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96000739a0 0x7f96001986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9608000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600198dd0 con 0x7f9600073090 2024-02-28T23:32:57.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9608000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960019cb60 con 0x7f96000739a0 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48692/0 (socket says 172.21.15.57:48692) 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 -- 172.21.15.57:0/3019725058 learned_addr learned my addr 172.21.15.57:0/3019725058 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9604c00700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96000739a0 0x7f96001986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 -- 172.21.15.57:0/3019725058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96000739a0 msgr2=0x7f96001986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96000739a0 0x7f96001986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.455+0000 7f9605600700 1 -- 172.21.15.57:0/3019725058 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0095f0 con 0x7f9600073090 2024-02-28T23:32:57.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f9605600700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d8d0 tx=0x7f95f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f4009940 con 0x7f9600073090 2024-02-28T23:32:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f4010460 con 0x7f9600073090 2024-02-28T23:32:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f400f5f0 con 0x7f9600073090 2024-02-28T23:32:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960019ce40 con 0x7f9600073090 2024-02-28T23:32:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.456+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960019d390 con 0x7f9600073090 2024-02-28T23:32:57.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.458+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9600109ce0 con 0x7f9600073090 2024-02-28T23:32:57.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.458+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f95f40105d0 con 0x7f9600073090 2024-02-28T23:32:57.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.459+0000 7f95fe000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 0x7f95e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.460+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f95f408d4c0 con 0x7f9600073090 2024-02-28T23:32:57.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.460+0000 7f9604c00700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 0x7f95e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.460+0000 7f9604c00700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 0x7f95e806fd50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96001997d0 tx=0x7f95ec005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.464+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95f4056c50 con 0x7f9600073090 2024-02-28T23:32:57.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.715+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9600199650 con 0x7f95e806d8a0 2024-02-28T23:32:57.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.718+0000 7f95fe000700 1 -- 172.21.15.57:0/3019725058 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f95ec01e040 con 0x7f95e806d8a0 2024-02-28T23:32:57.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 msgr2=0x7f95e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 0x7f95e806fd50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96001997d0 tx=0x7f95ec005ab0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 msgr2=0x7f96001981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d8d0 tx=0x7f95f400dc90 comp rx=0 tx=0).stop 2024-02-28T23:32:57.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 shutdown_connections 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f95e806d8a0 0x7f95e806fd50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9600073090 0x7f96001981b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 --2- 172.21.15.57:0/3019725058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96000739a0 0x7f96001986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 >> 172.21.15.57:0/3019725058 conn(0x7f96000fbda0 msgr2=0x7f9600106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 shutdown_connections 2024-02-28T23:32:57.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.721+0000 7f9608000700 1 -- 172.21.15.57:0/3019725058 wait complete. 2024-02-28T23:32:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.888+0000 7f8945000700 1 -- 172.21.15.57:0/613845913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8940100e80 msgr2=0x7f8940104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.888+0000 7f8945000700 1 --2- 172.21.15.57:0/613845913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8940100e80 0x7f8940104e50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f892c009910 tx=0x7f892c009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 -- 172.21.15.57:0/613845913 shutdown_connections 2024-02-28T23:32:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 --2- 172.21.15.57:0/613845913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8940100e80 0x7f8940104e50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 --2- 172.21.15.57:0/613845913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89401004e0 0x7f89401008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 -- 172.21.15.57:0/613845913 >> 172.21.15.57:0/613845913 conn(0x7f89400fbda0 msgr2=0x7f89400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 -- 172.21.15.57:0/613845913 shutdown_connections 2024-02-28T23:32:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.889+0000 7f8945000700 1 -- 172.21.15.57:0/613845913 wait complete. 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 Processor -- start 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 -- start start 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 0x7f8940198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940198d50 con 0x7f89401004e0 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.890+0000 7f8945000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894019cae0 con 0x7f8940100e80 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48716/0 (socket says 172.21.15.57:48716) 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 -- 172.21.15.57:0/2822687115 learned_addr learned my addr 172.21.15.57:0/2822687115 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893d400700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 0x7f8940198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 -- 172.21.15.57:0/2822687115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 msgr2=0x7f8940198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 0x7f8940198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893de00700 1 -- 172.21.15.57:0/2822687115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0095f0 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.891+0000 7f893d400700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 0x7f8940198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f893de00700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f893000d8d0 tx=0x7f893000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8930009940 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8930010460 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f893000f5f0 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894019cdc0 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.892+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894019d310 con 0x7f89401004e0 2024-02-28T23:32:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.893+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89401045d0 con 0x7f89401004e0 2024-02-28T23:32:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.894+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f89300105d0 con 0x7f89401004e0 2024-02-28T23:32:57.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.895+0000 7f8936a00700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 0x7f892406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.895+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f893008d4c0 con 0x7f89401004e0 2024-02-28T23:32:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.896+0000 7f893d400700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 0x7f892406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.896+0000 7f893d400700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 0x7f892406fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8940199750 tx=0x7f892c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:57.900+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8930056c50 con 0x7f89401004e0 2024-02-28T23:32:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:57 smithi057 ceph-mon[89491]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:57 smithi084 ceph-mon[70440]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:32:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.141+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f894019d580 con 0x7f892406d8a0 2024-02-28T23:32:58.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.155+0000 7f8936a00700 1 -- 172.21.15.57:0/2822687115 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f894019d580 con 0x7f892406d8a0 2024-02-28T23:32:58.154 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:32:58.154 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (22m) 14m ago 36m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:32:58.154 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (32m) 14m ago 36m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (35m) 13m ago 35m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (18m) 14m ago 36m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (18m) 13m ago 35m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (21m) 14m ago 36m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (14m) 14m ago 29m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (14m) 14m ago 29m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (29m) 13m ago 29m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (13m) 13m ago 29m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (25m) 14m ago 38m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (24m) 13m ago 35m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (20m) 14m ago 38m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (19m) 13m ago 34m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (23m) 14m ago 36m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (23m) 13m ago 35m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (18m) 14m ago 33m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:32:58.155 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (17m) 14m ago 33m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:32:58.156 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (16m) 14m ago 32m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:32:58.156 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (16m) 13m ago 32m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:32:58.156 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (15m) 13m ago 31m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:32:58.156 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (15m) 13m ago 31m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:32:58.156 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (23m) 14m ago 35m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 msgr2=0x7f892406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 0x7f892406fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8940199750 tx=0x7f892c01a040 comp rx=0 tx=0).stop 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 msgr2=0x7f8940198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f893000d8d0 tx=0x7f893000dc90 comp rx=0 tx=0).stop 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 shutdown_connections 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f892406d8a0 0x7f892406fd50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89401004e0 0x7f8940198130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 --2- 172.21.15.57:0/2822687115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8940100e80 0x7f8940198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 >> 172.21.15.57:0/2822687115 conn(0x7f89400fbda0 msgr2=0x7f89400fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:58.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 shutdown_connections 2024-02-28T23:32:58.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.158+0000 7f8945000700 1 -- 172.21.15.57:0/2822687115 wait complete. 2024-02-28T23:32:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 -- 172.21.15.57:0/1962308973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0730b0 msgr2=0x7f7afc073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 --2- 172.21.15.57:0/1962308973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0730b0 0x7f7afc073480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009940 tx=0x7f7aec009c50 comp rx=0 tx=0).stop 2024-02-28T23:32:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 -- 172.21.15.57:0/1962308973 shutdown_connections 2024-02-28T23:32:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 --2- 172.21.15.57:0/1962308973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0739c0 0x7f7afc10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 --2- 172.21.15.57:0/1962308973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0730b0 0x7f7afc073480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 -- 172.21.15.57:0/1962308973 >> 172.21.15.57:0/1962308973 conn(0x7f7afc0fbda0 msgr2=0x7f7afc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 -- 172.21.15.57:0/1962308973 shutdown_connections 2024-02-28T23:32:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.318+0000 7f7b01800700 1 -- 172.21.15.57:0/1962308973 wait complete. 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.319+0000 7f7b01800700 1 Processor -- start 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.319+0000 7f7b01800700 1 -- start start 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7b01800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 0x7f7afc198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7b01800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7b01800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc198e30 con 0x7f7afc0739c0 2024-02-28T23:32:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7b01800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc19cbc0 con 0x7f7afc0730b0 2024-02-28T23:32:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7afa000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7afa000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48744/0 (socket says 172.21.15.57:48744) 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.320+0000 7f7afa000700 1 -- 172.21.15.57:0/3126285069 learned_addr learned my addr 172.21.15.57:0/3126285069 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afaa00700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 0x7f7afc198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afa000700 1 -- 172.21.15.57:0/3126285069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 msgr2=0x7f7afc198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afa000700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 0x7f7afc198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afa000700 1 -- 172.21.15.57:0/3126285069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0095f0 con 0x7f7afc0739c0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afaa00700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 0x7f7afc198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7afa000700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d8d0 tx=0x7f7ae800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae8009940 con 0x7f7afc0739c0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ae8010460 con 0x7f7afc0739c0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc19cea0 con 0x7f7afc0739c0 2024-02-28T23:32:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae800f5f0 con 0x7f7afc0739c0 2024-02-28T23:32:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.321+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc19d3f0 con 0x7f7afc0739c0 2024-02-28T23:32:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.323+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc109cc0 con 0x7f7afc0739c0 2024-02-28T23:32:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.324+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7ae80105d0 con 0x7f7afc0739c0 2024-02-28T23:32:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.325+0000 7f7af3400700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 0x7f7ae006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.325+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7ae808d5d0 con 0x7f7afc0739c0 2024-02-28T23:32:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.325+0000 7f7afaa00700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 0x7f7ae006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.326+0000 7f7afaa00700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 0x7f7ae006fd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009910 tx=0x7f7aec005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.330+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ae8056ce0 con 0x7f7afc0739c0 2024-02-28T23:32:58.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.649+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7afc19d030 con 0x7f7afc0739c0 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.650+0000 7f7af3400700 1 -- 172.21.15.57:0/3126285069 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f7afc19d030 con 0x7f7afc0739c0 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:32:58.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:32:58.651 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 msgr2=0x7f7ae006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 0x7f7ae006fd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009910 tx=0x7f7aec005fd0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 msgr2=0x7f7afc198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d8d0 tx=0x7f7ae800dc90 comp rx=0 tx=0).stop 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 shutdown_connections 2024-02-28T23:32:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7ae006d8a0 0x7f7ae006fd50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7afc0730b0 0x7f7afc198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 --2- 172.21.15.57:0/3126285069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc0739c0 0x7f7afc198750 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 >> 172.21.15.57:0/3126285069 conn(0x7f7afc0fbda0 msgr2=0x7f7afc106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 shutdown_connections 2024-02-28T23:32:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.654+0000 7f7b01800700 1 -- 172.21.15.57:0/3126285069 wait complete. 2024-02-28T23:32:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 -- 172.21.15.57:0/2089646669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a681004e0 msgr2=0x7f9a681008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 --2- 172.21.15.57:0/2089646669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a681004e0 0x7f9a681008b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800b0a0 tx=0x7f9a5800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 -- 172.21.15.57:0/2089646669 shutdown_connections 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 --2- 172.21.15.57:0/2089646669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a68100e80 0x7f9a68104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 --2- 172.21.15.57:0/2089646669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a681004e0 0x7f9a681008b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 -- 172.21.15.57:0/2089646669 >> 172.21.15.57:0/2089646669 conn(0x7f9a680fbda0 msgr2=0x7f9a680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 -- 172.21.15.57:0/2089646669 shutdown_connections 2024-02-28T23:32:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.818+0000 7f9a6d600700 1 -- 172.21.15.57:0/2089646669 wait complete. 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.819+0000 7f9a6d600700 1 Processor -- start 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a6d600700 1 -- start start 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a681004e0 0x7f9a68198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a6d600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68198d90 con 0x7f9a68100e80 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a6d600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6819cb20 con 0x7f9a681004e0 2024-02-28T23:32:58.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a66000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a66000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48762/0 (socket says 172.21.15.57:48762) 2024-02-28T23:32:58.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.820+0000 7f9a66000700 1 -- 172.21.15.57:0/1748198080 learned_addr learned my addr 172.21.15.57:0/1748198080 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a66a00700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a681004e0 0x7f9a68198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a66000700 1 -- 172.21.15.57:0/1748198080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a681004e0 msgr2=0x7f9a68198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a66000700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a681004e0 0x7f9a68198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a66000700 1 -- 172.21.15.57:0/1748198080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58009d00 con 0x7f9a68100e80 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a66000700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400e8f0 tx=0x7f9a5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a54009790 con 0x7f9a68100e80 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a54004cf0 con 0x7f9a68100e80 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6819ce00 con 0x7f9a68100e80 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.821+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a54005690 con 0x7f9a68100e80 2024-02-28T23:32:58.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.822+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6819d350 con 0x7f9a68100e80 2024-02-28T23:32:58.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.823+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a681045d0 con 0x7f9a68100e80 2024-02-28T23:32:58.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.824+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9a54004750 con 0x7f9a68100e80 2024-02-28T23:32:58.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.824+0000 7f9a5f400700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 0x7f9a4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:58.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.825+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a54014070 con 0x7f9a68100e80 2024-02-28T23:32:58.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.825+0000 7f9a66a00700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 0x7f9a4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:58.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.826+0000 7f9a66a00700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 0x7f9a4c06fd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009480 tx=0x7f9a5800b740 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:58.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:58.829+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a540570d0 con 0x7f9a68100e80 2024-02-28T23:32:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:58 smithi057 ceph-mon[89491]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:58 smithi057 ceph-mon[89491]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:58 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3126285069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:32:59.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:58 smithi084 ceph-mon[70440]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:58 smithi084 ceph-mon[70440]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:32:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:58 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3126285069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:32:59.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.106+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9a6819d5d0 con 0x7f9a68100e80 2024-02-28T23:32:59.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.107+0000 7f9a5f400700 1 -- 172.21.15.57:0/1748198080 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f9a5405a790 con 0x7f9a68100e80 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:32:59.108 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:32:59.109 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:32:59.110 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:32:59.111 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:59.111 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:59.111 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:32:59.111 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 msgr2=0x7f9a4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 0x7f9a4c06fd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009480 tx=0x7f9a5800b740 comp rx=0 tx=0).stop 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 msgr2=0x7f9a681986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400e8f0 tx=0x7f9a5400ecb0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 shutdown_connections 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9a4c06d8a0 0x7f9a4c06fd50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a681004e0 0x7f9a68198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 --2- 172.21.15.57:0/1748198080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a68100e80 0x7f9a681986b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.110+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 >> 172.21.15.57:0/1748198080 conn(0x7f9a680fbda0 msgr2=0x7f9a680fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.111+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 shutdown_connections 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.111+0000 7f9a6d600700 1 -- 172.21.15.57:0/1748198080 wait complete. 2024-02-28T23:32:59.112 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 -- 172.21.15.57:0/404369922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 msgr2=0x7f592c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 --2- 172.21.15.57:0/404369922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1028e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5918009910 tx=0x7f5918009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 -- 172.21.15.57:0/404369922 shutdown_connections 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 --2- 172.21.15.57:0/404369922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1028e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 --2- 172.21.15.57:0/404369922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 0x7f592c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 -- 172.21.15.57:0/404369922 >> 172.21.15.57:0/404369922 conn(0x7f592c0fdf90 msgr2=0x7f592c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 -- 172.21.15.57:0/404369922 shutdown_connections 2024-02-28T23:32:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.265+0000 7f5932000700 1 -- 172.21.15.57:0/404369922 wait complete. 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.266+0000 7f5932000700 1 Processor -- start 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.266+0000 7f5932000700 1 -- start start 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f5932000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f5932000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 0x7f592c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f5932000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c198e10 con 0x7f592c102470 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f5932000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19cba0 con 0x7f592c108470 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f592b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f592b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48792/0 (socket says 172.21.15.57:48792) 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.267+0000 7f592b400700 1 -- 172.21.15.57:0/1332118356 learned_addr learned my addr 172.21.15.57:0/1332118356 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f592aa00700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 0x7f592c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f592b400700 1 -- 172.21.15.57:0/1332118356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 msgr2=0x7f592c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f592b400700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 0x7f592c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f592b400700 1 -- 172.21.15.57:0/1332118356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180095f0 con 0x7f592c102470 2024-02-28T23:32:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f592b400700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d8d0 tx=0x7f591c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591c009940 con 0x7f592c102470 2024-02-28T23:32:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f591c010460 con 0x7f592c102470 2024-02-28T23:32:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19ce80 con 0x7f592c102470 2024-02-28T23:32:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591c00f5f0 con 0x7f592c102470 2024-02-28T23:32:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.268+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c19d3d0 con 0x7f592c102470 2024-02-28T23:32:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.269+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c10a970 con 0x7f592c102470 2024-02-28T23:32:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.271+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f591c0105d0 con 0x7f592c102470 2024-02-28T23:32:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.271+0000 7f5923e00700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 0x7f591006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.271+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f591c08d5d0 con 0x7f592c102470 2024-02-28T23:32:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.272+0000 7f592aa00700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 0x7f591006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.272+0000 7f592aa00700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 0x7f591006fd50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f591800b5c0 tx=0x7f5918005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.276+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f591c056ce0 con 0x7f592c102470 2024-02-28T23:32:59.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.516+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f592c19d5f0 con 0x7f591006d8a0 2024-02-28T23:32:59.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.519+0000 7f5923e00700 1 -- 172.21.15.57:0/1332118356 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f592c19d5f0 con 0x7f591006d8a0 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:32:59.518 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:32:59.519 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:32:59.519 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:32:59.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.521+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 msgr2=0x7f591006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.521+0000 7f5932000700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 0x7f591006fd50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f591800b5c0 tx=0x7f5918005960 comp rx=0 tx=0).stop 2024-02-28T23:32:59.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.521+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 msgr2=0x7f592c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.521+0000 7f5932000700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d8d0 tx=0x7f591c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 shutdown_connections 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f591006d8a0 0x7f591006fd50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f592c102470 0x7f592c1981f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 --2- 172.21.15.57:0/1332118356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f592c108470 0x7f592c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 >> 172.21.15.57:0/1332118356 conn(0x7f592c0fdf90 msgr2=0x7f592c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 shutdown_connections 2024-02-28T23:32:59.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.522+0000 7f5932000700 1 -- 172.21.15.57:0/1332118356 wait complete. 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 -- 172.21.15.57:0/1129854207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c68100c60 msgr2=0x7f4c68104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 --2- 172.21.15.57:0/1129854207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c68100c60 0x7f4c68104c30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009910 tx=0x7f4c54009c20 comp rx=0 tx=0).stop 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 -- 172.21.15.57:0/1129854207 shutdown_connections 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 --2- 172.21.15.57:0/1129854207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c68100c60 0x7f4c68104c30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 --2- 172.21.15.57:0/1129854207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c681002c0 0x7f4c68100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.671+0000 7f4c6e200700 1 -- 172.21.15.57:0/1129854207 >> 172.21.15.57:0/1129854207 conn(0x7f4c680fbb60 msgr2=0x7f4c680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.672+0000 7f4c6e200700 1 -- 172.21.15.57:0/1129854207 shutdown_connections 2024-02-28T23:32:59.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.672+0000 7f4c6e200700 1 -- 172.21.15.57:0/1129854207 wait complete. 2024-02-28T23:32:59.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 Processor -- start 2024-02-28T23:32:59.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 -- start start 2024-02-28T23:32:59.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c68100c60 0x7f4c68073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68073ad0 con 0x7f4c681002c0 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6e200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68073c10 con 0x7f4c68100c60 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48806/0 (socket says 172.21.15.57:48806) 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.673+0000 7f4c6ce00700 1 -- 172.21.15.57:0/720743105 learned_addr learned my addr 172.21.15.57:0/720743105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c6ce00700 1 -- 172.21.15.57:0/720743105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c68100c60 msgr2=0x7f4c68073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c6ce00700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c68100c60 0x7f4c68073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c6ce00700 1 -- 172.21.15.57:0/720743105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540095f0 con 0x7f4c681002c0 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c6ce00700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00e920 tx=0x7f4c5c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c009790 con 0x7f4c681002c0 2024-02-28T23:32:59.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c5c00cba0 con 0x7f4c681002c0 2024-02-28T23:32:59.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c018650 con 0x7f4c681002c0 2024-02-28T23:32:59.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.674+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c681a1de0 con 0x7f4c681002c0 2024-02-28T23:32:59.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.675+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c681a21f0 con 0x7f4c681002c0 2024-02-28T23:32:59.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.676+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68108630 con 0x7f4c681002c0 2024-02-28T23:32:59.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.677+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4c5c00cd10 con 0x7f4c681002c0 2024-02-28T23:32:59.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.678+0000 7f4c65600700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 0x7f4c5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:32:59.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.678+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4c5c014070 con 0x7f4c681002c0 2024-02-28T23:32:59.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.678+0000 7f4c67e00700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 0x7f4c5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:32:59.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.679+0000 7f4c67e00700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 0x7f4c5006fd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b760 tx=0x7f4c54000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:32:59.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.682+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c5c052d20 con 0x7f4c681002c0 2024-02-28T23:32:59.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.993+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4c6802ccd0 con 0x7f4c681002c0 2024-02-28T23:32:59.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.994+0000 7f4c65600700 1 -- 172.21.15.57:0/720743105 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4c5c05abd0 con 0x7f4c681002c0 2024-02-28T23:32:59.994 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:32:59.994 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:32:59.994 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:32:59.994 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:32:59.994 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.997+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 msgr2=0x7f4c5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.997+0000 7f4c6e200700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 0x7f4c5006fd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b760 tx=0x7f4c54000bc0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 msgr2=0x7f4c68073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00e920 tx=0x7f4c5c00ece0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 shutdown_connections 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4c5006d8a0 0x7f4c5006fd50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c681002c0 0x7f4c68073050 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 --2- 172.21.15.57:0/720743105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c68100c60 0x7f4c68073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 >> 172.21.15.57:0/720743105 conn(0x7f4c680fbb60 msgr2=0x7f4c680fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:32:59.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 shutdown_connections 2024-02-28T23:32:59.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:32:59.998+0000 7f4c6e200700 1 -- 172.21.15.57:0/720743105 wait complete. 2024-02-28T23:33:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:59 smithi057 ceph-mon[89491]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:59 smithi057 ceph-mon[89491]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:32:59 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1748198080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:33:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:59 smithi084 ceph-mon[70440]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:00.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:59 smithi084 ceph-mon[70440]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:00.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:32:59 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1748198080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:33:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:00 smithi084 ceph-mon[70440]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:00 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/720743105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:33:01.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:00 smithi057 ceph-mon[89491]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:01.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:00 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/720743105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:33:02.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:01 smithi057 ceph-mon[89491]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:01 smithi084 ceph-mon[70440]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:03 smithi057 ceph-mon[89491]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:03 smithi084 ceph-mon[70440]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:06.143 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:05 smithi084 ceph-mon[70440]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:05 smithi057 ceph-mon[89491]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:07 smithi057 ceph-mon[89491]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:08.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:07 smithi084 ceph-mon[70440]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:09 smithi057 ceph-mon[89491]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:10.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:09 smithi084 ceph-mon[70440]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:12.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:11 smithi057 ceph-mon[89491]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:12.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:11 smithi084 ceph-mon[70440]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:14.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:13 smithi057 ceph-mon[89491]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:14.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:13 smithi084 ceph-mon[70440]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:16.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:15 smithi057 ceph-mon[89491]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:16.133 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:15 smithi084 ceph-mon[70440]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:18.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:17 smithi057 ceph-mon[89491]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:18.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:17 smithi084 ceph-mon[70440]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:19 smithi057 ceph-mon[89491]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:20.354 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:19 smithi084 ceph-mon[70440]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:22.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:21 smithi057 ceph-mon[89491]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:22.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:21 smithi084 ceph-mon[70440]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:24.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:23 smithi057 ceph-mon[89491]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:24.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:23 smithi084 ceph-mon[70440]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:26.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:25 smithi057 ceph-mon[89491]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:26.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:25 smithi084 ceph-mon[70440]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:28.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:27 smithi057 ceph-mon[89491]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:28.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:27 smithi084 ceph-mon[70440]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 -- 172.21.15.57:0/3777247916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 msgr2=0x7fd7501028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 --2- 172.21.15.57:0/3777247916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501028e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009910 tx=0x7fd73c009c20 comp rx=0 tx=0).stop 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 -- 172.21.15.57:0/3777247916 shutdown_connections 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 --2- 172.21.15.57:0/3777247916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501028e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 --2- 172.21.15.57:0/3777247916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 -- 172.21.15.57:0/3777247916 >> 172.21.15.57:0/3777247916 conn(0x7fd7500fdf90 msgr2=0x7fd7501003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:30.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 -- 172.21.15.57:0/3777247916 shutdown_connections 2024-02-28T23:33:30.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.175+0000 7fd756800700 1 -- 172.21.15.57:0/3777247916 wait complete. 2024-02-28T23:33:30.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 Processor -- start 2024-02-28T23:33:30.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 -- start start 2024-02-28T23:33:30.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd750198e10 con 0x7fd750102470 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.177+0000 7fd756800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75019cba0 con 0x7fd750108470 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54498/0 (socket says 172.21.15.57:54498) 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 -- 172.21.15.57:0/3247927157 learned_addr learned my addr 172.21.15.57:0/3247927157 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74ea00700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 -- 172.21.15.57:0/3247927157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 msgr2=0x7fd750198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74f400700 1 -- 172.21.15.57:0/3247927157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0095f0 con 0x7fd750102470 2024-02-28T23:33:30.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.178+0000 7fd74ea00700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:33:30.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd74f400700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d8d0 tx=0x7fd74000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:30.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd740009940 con 0x7fd750102470 2024-02-28T23:33:30.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd740010460 con 0x7fd750102470 2024-02-28T23:33:30.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd74000f5f0 con 0x7fd750102470 2024-02-28T23:33:30.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75019ce80 con 0x7fd750102470 2024-02-28T23:33:30.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.179+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75019d3d0 con 0x7fd750102470 2024-02-28T23:33:30.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.180+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75010a970 con 0x7fd750102470 2024-02-28T23:33:30.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.181+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd7400105d0 con 0x7fd750102470 2024-02-28T23:33:30.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.182+0000 7fd747e00700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 0x7fd73406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.182+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd74008d4c0 con 0x7fd750102470 2024-02-28T23:33:30.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.183+0000 7fd74ea00700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 0x7fd73406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.183+0000 7fd74ea00700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 0x7fd73406fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:30.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.187+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd740056c50 con 0x7fd750102470 2024-02-28T23:33:30.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:29 smithi057 ceph-mon[89491]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:30.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:29 smithi084 ceph-mon[70440]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.432+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd750199690 con 0x7fd73406d8a0 2024-02-28T23:33:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.436+0000 7fd747e00700 1 -- 172.21.15.57:0/3247927157 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd73c005480 con 0x7fd73406d8a0 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 msgr2=0x7fd73406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 0x7fd73406fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c00b540 comp rx=0 tx=0).stop 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 msgr2=0x7fd7501981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d8d0 tx=0x7fd74000dc90 comp rx=0 tx=0).stop 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 shutdown_connections 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd73406d8a0 0x7fd73406fd50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd750102470 0x7fd7501981f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 --2- 172.21.15.57:0/3247927157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd750108470 0x7fd750198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.438+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 >> 172.21.15.57:0/3247927157 conn(0x7fd7500fdf90 msgr2=0x7fd7500ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.439+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 shutdown_connections 2024-02-28T23:33:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.439+0000 7fd756800700 1 -- 172.21.15.57:0/3247927157 wait complete. 2024-02-28T23:33:30.452 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.602+0000 7f81ed600700 1 -- 172.21.15.57:0/4089409549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e8106b80 msgr2=0x7f81e8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.602+0000 7f81ed600700 1 --2- 172.21.15.57:0/4089409549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e8106b80 0x7f81e8106ff0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009910 tx=0x7f81d0009c20 comp rx=0 tx=0).stop 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.602+0000 7f81ed600700 1 -- 172.21.15.57:0/4089409549 shutdown_connections 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.602+0000 7f81ed600700 1 --2- 172.21.15.57:0/4089409549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e8106b80 0x7f81e8106ff0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.603+0000 7f81ed600700 1 --2- 172.21.15.57:0/4089409549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e810c8f0 0x7f81e810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.603+0000 7f81ed600700 1 -- 172.21.15.57:0/4089409549 >> 172.21.15.57:0/4089409549 conn(0x7f81e8076060 msgr2=0x7f81e8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.603+0000 7f81ed600700 1 -- 172.21.15.57:0/4089409549 shutdown_connections 2024-02-28T23:33:30.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.603+0000 7f81ed600700 1 -- 172.21.15.57:0/4089409549 wait complete. 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 Processor -- start 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 -- start start 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 0x7f81e819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e819cfd0 con 0x7f81e810c8f0 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.604+0000 7f81ed600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e819d110 con 0x7f81e8106b80 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54506/0 (socket says 172.21.15.57:54506) 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 -- 172.21.15.57:0/3107940316 learned_addr learned my addr 172.21.15.57:0/3107940316 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81e6a00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 0x7f81e819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 -- 172.21.15.57:0/3107940316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 msgr2=0x7f81e819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 0x7f81e819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81dfe00700 1 -- 172.21.15.57:0/3107940316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00095f0 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.605+0000 7f81e6a00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 0x7f81e819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81dfe00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f81d000b5c0 tx=0x7f81d000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d001d070 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81d000f460 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d0017610 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e81a0f00 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.606+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e81a13a0 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.607+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e810ecc0 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.608+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f81d000f5d0 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.609+0000 7f81df400700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 0x7f81cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.609+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f81d008ec00 con 0x7f81e810c8f0 2024-02-28T23:33:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.609+0000 7f81e6a00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 0x7f81cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.610+0000 7f81e6a00700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 0x7f81cc06fd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005fd0 tx=0x7f81d8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.614+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81d0058310 con 0x7f81e810c8f0 2024-02-28T23:33:30.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.879+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81e81a16d0 con 0x7f81cc06d8a0 2024-02-28T23:33:30.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.881+0000 7f81df400700 1 -- 172.21.15.57:0/3107940316 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f81e81a16d0 con 0x7f81cc06d8a0 2024-02-28T23:33:30.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.884+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 msgr2=0x7f81cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.884+0000 7f81ed600700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 0x7f81cc06fd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005fd0 tx=0x7f81d8005e20 comp rx=0 tx=0).stop 2024-02-28T23:33:30.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.884+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 msgr2=0x7f81e819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:30.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.884+0000 7f81ed600700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f81d000b5c0 tx=0x7f81d000be00 comp rx=0 tx=0).stop 2024-02-28T23:33:30.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 shutdown_connections 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f81cc06d8a0 0x7f81cc06fd50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e8106b80 0x7f81e819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 --2- 172.21.15.57:0/3107940316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e810c8f0 0x7f81e819c980 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 >> 172.21.15.57:0/3107940316 conn(0x7f81e8076060 msgr2=0x7f81e8077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 shutdown_connections 2024-02-28T23:33:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:30.885+0000 7f81ed600700 1 -- 172.21.15.57:0/3107940316 wait complete. 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 -- 172.21.15.57:0/2098027820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c100e80 msgr2=0x7f165c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 --2- 172.21.15.57:0/2098027820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c100e80 0x7f165c104e50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1644009910 tx=0x7f1644009c20 comp rx=0 tx=0).stop 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 -- 172.21.15.57:0/2098027820 shutdown_connections 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 --2- 172.21.15.57:0/2098027820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c100e80 0x7f165c104e50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 --2- 172.21.15.57:0/2098027820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c1004e0 0x7f165c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 -- 172.21.15.57:0/2098027820 >> 172.21.15.57:0/2098027820 conn(0x7f165c0fbda0 msgr2=0x7f165c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 -- 172.21.15.57:0/2098027820 shutdown_connections 2024-02-28T23:33:31.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.040+0000 7f1663400700 1 -- 172.21.15.57:0/2098027820 wait complete. 2024-02-28T23:33:31.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.041+0000 7f1663400700 1 Processor -- start 2024-02-28T23:33:31.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.041+0000 7f1663400700 1 -- start start 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1663400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1663400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 0x7f165c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1663400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c073ad0 con 0x7f165c1004e0 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1663400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c073c10 con 0x7f165c100e80 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54524/0 (socket says 172.21.15.57:54524) 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 -- 172.21.15.57:0/3470346296 learned_addr learned my addr 172.21.15.57:0/3470346296 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f165be00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 0x7f165c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 -- 172.21.15.57:0/3470346296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 msgr2=0x7f165c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 0x7f165c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f1660a00700 1 -- 172.21.15.57:0/3470346296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440095f0 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.042+0000 7f165be00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 0x7f165c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1660a00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f164c00ef10 tx=0x7f164c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164c00ca50 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f164c00f040 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165c073e90 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164c0146b0 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.043+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165c1a2490 con 0x7f165c1004e0 2024-02-28T23:33:31.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.044+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165c108850 con 0x7f165c1004e0 2024-02-28T23:33:31.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.045+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f164c00cbb0 con 0x7f165c1004e0 2024-02-28T23:33:31.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.046+0000 7f1659600700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 0x7f164006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.046+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f164c08d950 con 0x7f165c1004e0 2024-02-28T23:33:31.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.046+0000 7f165be00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 0x7f164006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.047+0000 7f165be00700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 0x7f164006fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f164400b760 tx=0x7f1644005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:31.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.051+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f164c057100 con 0x7f165c1004e0 2024-02-28T23:33:31.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.291+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f165c061190 con 0x7f164006d8a0 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.330+0000 7f1659600700 1 -- 172.21.15.57:0/3470346296 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f165c061190 con 0x7f164006d8a0 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (23m) 14m ago 36m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (33m) 14m ago 37m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (35m) 14m ago 35m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (19m) 14m ago 37m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:33:31.329 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (19m) 14m ago 35m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (22m) 14m ago 36m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (15m) 14m ago 29m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (14m) 14m ago 29m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (29m) 14m ago 29m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (14m) 14m ago 29m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (25m) 14m ago 38m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (24m) 14m ago 35m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (20m) 14m ago 39m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (20m) 14m ago 35m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (24m) 14m ago 37m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (24m) 14m ago 35m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (18m) 14m ago 34m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (18m) 14m ago 33m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:33:31.330 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (17m) 14m ago 33m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:33:31.331 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (16m) 14m ago 32m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:33:31.331 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (16m) 14m ago 32m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:33:31.331 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (15m) 14m ago 31m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:33:31.331 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (23m) 14m ago 36m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 msgr2=0x7f164006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 0x7f164006fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f164400b760 tx=0x7f1644005fb0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 msgr2=0x7f165c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f164c00ef10 tx=0x7f164c00c3a0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 shutdown_connections 2024-02-28T23:33:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.333+0000 7f1663400700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f164006d8a0 0x7f164006fd50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.334+0000 7f1663400700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f165c1004e0 0x7f165c073050 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.334+0000 7f1663400700 1 --2- 172.21.15.57:0/3470346296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f165c100e80 0x7f165c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.334+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 >> 172.21.15.57:0/3470346296 conn(0x7f165c0fbda0 msgr2=0x7f165c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.334+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 shutdown_connections 2024-02-28T23:33:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.334+0000 7f1663400700 1 -- 172.21.15.57:0/3470346296 wait complete. 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 -- 172.21.15.57:0/3563279383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0108470 msgr2=0x7f7de0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 --2- 172.21.15.57:0/3563279383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0108470 0x7f7de0108840 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b0a0 tx=0x7f7dd000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 -- 172.21.15.57:0/3563279383 shutdown_connections 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 --2- 172.21.15.57:0/3563279383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0102470 0x7f7de01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 --2- 172.21.15.57:0/3563279383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0108470 0x7f7de0108840 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.495+0000 7f7de5000700 1 -- 172.21.15.57:0/3563279383 >> 172.21.15.57:0/3563279383 conn(0x7f7de00fdf90 msgr2=0x7f7de01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.496+0000 7f7de5000700 1 -- 172.21.15.57:0/3563279383 shutdown_connections 2024-02-28T23:33:31.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.496+0000 7f7de5000700 1 -- 172.21.15.57:0/3563279383 wait complete. 2024-02-28T23:33:31.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 Processor -- start 2024-02-28T23:33:31.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 -- start start 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0108470 0x7f7de0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0198e50 con 0x7f7de0102470 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.497+0000 7f7de5000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de019cbe0 con 0x7f7de0108470 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54542/0 (socket says 172.21.15.57:54542) 2024-02-28T23:33:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 -- 172.21.15.57:0/2567038253 learned_addr learned my addr 172.21.15.57:0/2567038253 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:31.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7dd5400700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0108470 0x7f7de0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 -- 172.21.15.57:0/2567038253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0108470 msgr2=0x7f7de0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0108470 0x7f7de0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.498+0000 7f7ddde00700 1 -- 172.21.15.57:0/2567038253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009d00 con 0x7f7de0102470 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7ddde00700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b070 tx=0x7f7dd000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd000e040 con 0x7f7de0102470 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dd0003de0 con 0x7f7de0102470 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd001d8e0 con 0x7f7de0102470 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de019ce60 con 0x7f7de0102470 2024-02-28T23:33:31.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.499+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de019d350 con 0x7f7de0102470 2024-02-28T23:33:31.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.500+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de010a9e0 con 0x7f7de0102470 2024-02-28T23:33:31.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.501+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7dd001da40 con 0x7f7de0102470 2024-02-28T23:33:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.502+0000 7f7dd7400700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 0x7f7dc406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.503+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7dd008f320 con 0x7f7de0102470 2024-02-28T23:33:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.503+0000 7f7dd5400700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 0x7f7dc406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.504+0000 7f7dd5400700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 0x7f7dc406fd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005950 tx=0x7f7dc80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:31.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.507+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7dd0058b30 con 0x7f7de0102470 2024-02-28T23:33:31.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.826+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7de004f2a0 con 0x7f7de0102470 2024-02-28T23:33:31.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.827+0000 7f7dd7400700 1 -- 172.21.15.57:0/2567038253 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f7dd005c1f0 con 0x7f7de0102470 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:33:31.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:33:31.827 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:33:31.828 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:33:31.828 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 msgr2=0x7f7dc406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 0x7f7dc406fd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005950 tx=0x7f7dc80058e0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 msgr2=0x7f7de0198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b070 tx=0x7f7dd000cdd0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 shutdown_connections 2024-02-28T23:33:31.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7dc406d8a0 0x7f7dc406fd50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7de0102470 0x7f7de0198230 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 --2- 172.21.15.57:0/2567038253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7de0108470 0x7f7de0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.830+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 >> 172.21.15.57:0/2567038253 conn(0x7f7de00fdf90 msgr2=0x7f7de0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:31.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.831+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 shutdown_connections 2024-02-28T23:33:31.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.831+0000 7f7de5000700 1 -- 172.21.15.57:0/2567038253 wait complete. 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 -- 172.21.15.57:0/1063052882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073070 msgr2=0x7f8438073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 --2- 172.21.15.57:0/1063052882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073070 0x7f8438073440 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8428009940 tx=0x7f8428009c50 comp rx=0 tx=0).stop 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 -- 172.21.15.57:0/1063052882 shutdown_connections 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 --2- 172.21.15.57:0/1063052882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073980 0x7f843810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 --2- 172.21.15.57:0/1063052882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073070 0x7f8438073440 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 -- 172.21.15.57:0/1063052882 >> 172.21.15.57:0/1063052882 conn(0x7f84380fbda0 msgr2=0x7f84380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.993+0000 7f843ce00700 1 -- 172.21.15.57:0/1063052882 shutdown_connections 2024-02-28T23:33:31.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.994+0000 7f843ce00700 1 -- 172.21.15.57:0/1063052882 wait complete. 2024-02-28T23:33:31.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.995+0000 7f843ce00700 1 Processor -- start 2024-02-28T23:33:31.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.995+0000 7f843ce00700 1 -- start start 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f843ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073070 0x7f8438198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f843ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f843ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438198db0 con 0x7f8438073980 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f843ce00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843819caf0 con 0x7f8438073070 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f8435400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f8435400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54546/0 (socket says 172.21.15.57:54546) 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.996+0000 7f8435400700 1 -- 172.21.15.57:0/216144748 learned_addr learned my addr 172.21.15.57:0/216144748 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f8435e00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073070 0x7f8438198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:31.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f8435400700 1 -- 172.21.15.57:0/216144748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073070 msgr2=0x7f8438198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:31.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f8435400700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073070 0x7f8438198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:31.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f8435400700 1 -- 172.21.15.57:0/216144748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280095f0 con 0x7f8438073980 2024-02-28T23:33:31.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f8435400700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f842400b4b0 tx=0x7f842400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8424011800 con 0x7f8438073980 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8424011e20 con 0x7f8438073980 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843819cdd0 con 0x7f8438073980 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842400f2e0 con 0x7f8438073980 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.997+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843819d320 con 0x7f8438073980 2024-02-28T23:33:31.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:31.999+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8438109d00 con 0x7f8438073980 2024-02-28T23:33:31.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.000+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8424010380 con 0x7f8438073980 2024-02-28T23:33:31.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.001+0000 7f842ea00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 0x7f841c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:31.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.001+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f842408d0f0 con 0x7f8438073980 2024-02-28T23:33:31.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.001+0000 7f8435e00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 0x7f841c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.002+0000 7f8435e00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 0x7f841c06fd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f8428005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:32.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.006+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8424056930 con 0x7f8438073980 2024-02-28T23:33:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.286+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f843804f2a0 con 0x7f8438073980 2024-02-28T23:33:32.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:31 smithi057 ceph-mon[89491]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:32.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:31 smithi057 ceph-mon[89491]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:32.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:31 smithi057 ceph-mon[89491]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:32.285 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:31 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2567038253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:33:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.287+0000 7f842ea00700 1 -- 172.21.15.57:0/216144748 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f8424059ff0 con 0x7f8438073980 2024-02-28T23:33:32.287 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:33:32.287 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:33:32.287 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:33:32.287 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:33:32.288 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:33:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:33:32.290 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:33:32.291 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:33:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 msgr2=0x7f841c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 0x7f841c06fd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f8428005a90 comp rx=0 tx=0).stop 2024-02-28T23:33:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 msgr2=0x7f84381986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f842400b4b0 tx=0x7f842400b870 comp rx=0 tx=0).stop 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 shutdown_connections 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f841c06d8a0 0x7f841c06fd50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8438073070 0x7f8438198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 --2- 172.21.15.57:0/216144748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8438073980 0x7f84381986d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.290+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 >> 172.21.15.57:0/216144748 conn(0x7f84380fbda0 msgr2=0x7f8438106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.291+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 shutdown_connections 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.291+0000 7f843ce00700 1 -- 172.21.15.57:0/216144748 wait complete. 2024-02-28T23:33:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:33:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:31 smithi084 ceph-mon[70440]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:31 smithi084 ceph-mon[70440]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:31 smithi084 ceph-mon[70440]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:31 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2567038253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 -- 172.21.15.57:0/285942645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c0730f0 msgr2=0x7f891c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 --2- 172.21.15.57:0/285942645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c0730f0 0x7f891c0734c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8904009910 tx=0x7f8904009c20 comp rx=0 tx=0).stop 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 -- 172.21.15.57:0/285942645 shutdown_connections 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 --2- 172.21.15.57:0/285942645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c073a00 0x7f891c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 --2- 172.21.15.57:0/285942645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c0730f0 0x7f891c0734c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 -- 172.21.15.57:0/285942645 >> 172.21.15.57:0/285942645 conn(0x7f891c0fbda0 msgr2=0x7f891c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:32.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 -- 172.21.15.57:0/285942645 shutdown_connections 2024-02-28T23:33:32.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.457+0000 7f8921800700 1 -- 172.21.15.57:0/285942645 wait complete. 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.458+0000 7f8921800700 1 Processor -- start 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8921800700 1 -- start start 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8921800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 0x7f891c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8921800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8921800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19c970 con 0x7f891c073a00 2024-02-28T23:33:32.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8921800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19cae0 con 0x7f891c0730f0 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8912000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8912000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54572/0 (socket says 172.21.15.57:54572) 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.459+0000 7f8912000700 1 -- 172.21.15.57:0/3541028061 learned_addr learned my addr 172.21.15.57:0/3541028061 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f891aa00700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 0x7f891c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f8912000700 1 -- 172.21.15.57:0/3541028061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 msgr2=0x7f891c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f8912000700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 0x7f891c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f8912000700 1 -- 172.21.15.57:0/3541028061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040095f0 con 0x7f891c073a00 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f891aa00700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 0x7f891c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:33:32.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f8912000700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d8d0 tx=0x7f890c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:32.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.460+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c009940 con 0x7f891c073a00 2024-02-28T23:33:32.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.461+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f890c010460 con 0x7f891c073a00 2024-02-28T23:33:32.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.461+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c00f5f0 con 0x7f891c073a00 2024-02-28T23:33:32.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.461+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c19cdc0 con 0x7f891c073a00 2024-02-28T23:33:32.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.461+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c19d310 con 0x7f891c073a00 2024-02-28T23:33:32.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.462+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c109c80 con 0x7f891c073a00 2024-02-28T23:33:32.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.463+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f890c0105d0 con 0x7f891c073a00 2024-02-28T23:33:32.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.464+0000 7f8913e00700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 0x7f890006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.464+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f890c08d5d0 con 0x7f891c073a00 2024-02-28T23:33:32.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.464+0000 7f891aa00700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 0x7f890006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.465+0000 7f891aa00700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 0x7f890006fd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f890400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:32.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.468+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f890c056ce0 con 0x7f891c073a00 2024-02-28T23:33:32.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.718+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f891c199390 con 0x7f890006d8a0 2024-02-28T23:33:32.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.721+0000 7f8913e00700 1 -- 172.21.15.57:0/3541028061 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f891c199390 con 0x7f890006d8a0 2024-02-28T23:33:32.719 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:33:32.719 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:33:32.720 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:33:32.721 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:33:32.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 msgr2=0x7f890006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 0x7f890006fd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f890400b540 comp rx=0 tx=0).stop 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 msgr2=0x7f891c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d8d0 tx=0x7f890c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 shutdown_connections 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f890006d8a0 0x7f890006fd50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f891c0730f0 0x7f891c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 --2- 172.21.15.57:0/3541028061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f891c073a00 0x7f891c198630 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 >> 172.21.15.57:0/3541028061 conn(0x7f891c0fbda0 msgr2=0x7f891c106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.724+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 shutdown_connections 2024-02-28T23:33:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.725+0000 7f8921800700 1 -- 172.21.15.57:0/3541028061 wait complete. 2024-02-28T23:33:32.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 -- 172.21.15.57:0/3147145730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a0102e70 msgr2=0x7fb8a0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/3147145730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a0102e70 0x7fb8a0106e40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009910 tx=0x7fb88c009c20 comp rx=0 tx=0).stop 2024-02-28T23:33:32.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 -- 172.21.15.57:0/3147145730 shutdown_connections 2024-02-28T23:33:32.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/3147145730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a0102e70 0x7fb8a0106e40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/3147145730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a01024d0 0x7fb8a01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.885+0000 7fb8a6a00700 1 -- 172.21.15.57:0/3147145730 >> 172.21.15.57:0/3147145730 conn(0x7fb8a00fdd90 msgr2=0x7fb8a01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:32.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.886+0000 7fb8a6a00700 1 -- 172.21.15.57:0/3147145730 shutdown_connections 2024-02-28T23:33:32.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.886+0000 7fb8a6a00700 1 -- 172.21.15.57:0/3147145730 wait complete. 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 Processor -- start 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 -- start start 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a0102e70 0x7fb8a0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a0198a30 con 0x7fb8a01024d0 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.887+0000 7fb8a6a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a019c7c0 con 0x7fb8a0102e70 2024-02-28T23:33:32.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54584/0 (socket says 172.21.15.57:54584) 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 -- 172.21.15.57:0/47536255 learned_addr learned my addr 172.21.15.57:0/47536255 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a4c00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a0102e70 0x7fb8a0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 -- 172.21.15.57:0/47536255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a0102e70 msgr2=0x7fb8a0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a0102e70 0x7fb8a0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 -- 172.21.15.57:0/47536255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0095f0 con 0x7fb8a01024d0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.888+0000 7fb8a5600700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb89400d8d0 tx=0x7fb89400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.889+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb894009940 con 0x7fb8a01024d0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.889+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb894010460 con 0x7fb8a01024d0 2024-02-28T23:33:32.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.889+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a019caa0 con 0x7fb8a01024d0 2024-02-28T23:33:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.889+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89400f5f0 con 0x7fb8a01024d0 2024-02-28T23:33:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.889+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a019cff0 con 0x7fb8a01024d0 2024-02-28T23:33:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.890+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a010a840 con 0x7fb8a01024d0 2024-02-28T23:33:32.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.891+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb8940105d0 con 0x7fb8a01024d0 2024-02-28T23:33:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.892+0000 7fb89e000700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 0x7fb88806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:33:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.892+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb89408d4c0 con 0x7fb8a01024d0 2024-02-28T23:33:32.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.892+0000 7fb8a4c00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 0x7fb88806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:33:32.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.893+0000 7fb8a4c00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 0x7fb88806fd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0199430 tx=0x7fb88c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:33:32.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:32.897+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb894056c50 con 0x7fb8a01024d0 2024-02-28T23:33:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.221+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb8a0062380 con 0x7fb8a01024d0 2024-02-28T23:33:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.222+0000 7fb89e000700 1 -- 172.21.15.57:0/47536255 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb89405a310 con 0x7fb8a01024d0 2024-02-28T23:33:33.221 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:33:33.221 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:33:33.221 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:33:33.221 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:33:33.222 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 msgr2=0x7fb88806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 0x7fb88806fd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0199430 tx=0x7fb88c01a040 comp rx=0 tx=0).stop 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 msgr2=0x7fb8a0197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb89400d8d0 tx=0x7fb89400dc90 comp rx=0 tx=0).stop 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 shutdown_connections 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb88806d8a0 0x7fb88806fd50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8a01024d0 0x7fb8a0197e10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 --2- 172.21.15.57:0/47536255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8a0102e70 0x7fb8a0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 >> 172.21.15.57:0/47536255 conn(0x7fb8a00fdd90 msgr2=0x7fb8a0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:33:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.225+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 shutdown_connections 2024-02-28T23:33:33.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:33:33.226+0000 7fb8a6a00700 1 -- 172.21.15.57:0/47536255 wait complete. 2024-02-28T23:33:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:32 smithi057 ceph-mon[89491]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:32 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/216144748' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:33:33.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:32 smithi084 ceph-mon[70440]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:33.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:32 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/216144748' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:33:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:33 smithi057 ceph-mon[89491]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:33 smithi057 ceph-mon[89491]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:33 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/47536255' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:33:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:33 smithi084 ceph-mon[70440]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:33 smithi084 ceph-mon[70440]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:33:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:33 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/47536255' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:33:36.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:35 smithi057 ceph-mon[89491]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:36.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:35 smithi084 ceph-mon[70440]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:38.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:38 smithi057 ceph-mon[89491]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:38.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:38 smithi084 ceph-mon[70440]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:39.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:39 smithi057 ceph-mon[89491]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:39.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:39 smithi084 ceph-mon[70440]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:41.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:41 smithi057 ceph-mon[89491]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:42.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:41 smithi084 ceph-mon[70440]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:43 smithi057 ceph-mon[89491]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:44.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:43 smithi084 ceph-mon[70440]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:45 smithi057 ceph-mon[89491]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:46.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:45 smithi084 ceph-mon[70440]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:48.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:47 smithi057 ceph-mon[89491]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:48.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:47 smithi084 ceph-mon[70440]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:33:50.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:49 smithi057 ceph-mon[89491]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:50.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:49 smithi084 ceph-mon[70440]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:52.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:51 smithi057 ceph-mon[89491]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:52.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:51 smithi084 ceph-mon[70440]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:54.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:53 smithi057 ceph-mon[89491]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:54.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:53 smithi084 ceph-mon[70440]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:56.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:55 smithi084 ceph-mon[70440]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:56.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:55 smithi057 ceph-mon[89491]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:57 smithi084 ceph-mon[70440]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:33:58.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:57 smithi057 ceph-mon[89491]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:00.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:33:59 smithi057 ceph-mon[89491]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:00.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:33:59 smithi084 ceph-mon[70440]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:02.177 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:01 smithi057 ceph-mon[89491]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:02.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:01 smithi084 ceph-mon[70440]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 -- 172.21.15.57:0/3794441177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144102470 msgr2=0x7fe1441028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 --2- 172.21.15.57:0/3794441177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144102470 0x7fe1441028e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe130009910 tx=0x7fe130009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 -- 172.21.15.57:0/3794441177 shutdown_connections 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 --2- 172.21.15.57:0/3794441177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144102470 0x7fe1441028e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 --2- 172.21.15.57:0/3794441177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144108470 0x7fe144108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 -- 172.21.15.57:0/3794441177 >> 172.21.15.57:0/3794441177 conn(0x7fe1440fdf90 msgr2=0x7fe1441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 -- 172.21.15.57:0/3794441177 shutdown_connections 2024-02-28T23:34:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.404+0000 7fe149e00700 1 -- 172.21.15.57:0/3794441177 wait complete. 2024-02-28T23:34:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.405+0000 7fe149e00700 1 Processor -- start 2024-02-28T23:34:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.405+0000 7fe149e00700 1 -- start start 2024-02-28T23:34:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe149e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 0x7fe1441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe149e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe149e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144198e10 con 0x7fe144108470 2024-02-28T23:34:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe149e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14419cba0 con 0x7fe144102470 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe142000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe142000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37988/0 (socket says 172.21.15.57:37988) 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe142000700 1 -- 172.21.15.57:0/2507604122 learned_addr learned my addr 172.21.15.57:0/2507604122 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.406+0000 7fe142a00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 0x7fe1441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe142000700 1 -- 172.21.15.57:0/2507604122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 msgr2=0x7fe1441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe142000700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 0x7fe1441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe142000700 1 -- 172.21.15.57:0/2507604122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300095f0 con 0x7fe144108470 2024-02-28T23:34:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe142a00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 0x7fe1441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe142000700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b5c0 tx=0x7fe13000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe13001d070 con 0x7fe144108470 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe130005e00 con 0x7fe144108470 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe130017400 con 0x7fe144108470 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.407+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14419ce20 con 0x7fe144108470 2024-02-28T23:34:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.408+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14419d310 con 0x7fe144108470 2024-02-28T23:34:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.409+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14410a970 con 0x7fe144108470 2024-02-28T23:34:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.410+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe13000f460 con 0x7fe144108470 2024-02-28T23:34:03.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.411+0000 7fe148a00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 0x7fe12806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.411+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe13008ea10 con 0x7fe144108470 2024-02-28T23:34:03.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.411+0000 7fe142a00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 0x7fe12806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.412+0000 7fe142a00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 0x7fe12806fd50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe134005950 tx=0x7fe1340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:03.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.416+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1300581a0 con 0x7fe144108470 2024-02-28T23:34:03.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.668+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe144199660 con 0x7fe12806d8a0 2024-02-28T23:34:03.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.671+0000 7fe148a00700 1 -- 172.21.15.57:0/2507604122 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe144199660 con 0x7fe12806d8a0 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 msgr2=0x7fe12806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 0x7fe12806fd50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe134005950 tx=0x7fe1340058e0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 msgr2=0x7fe144198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b5c0 tx=0x7fe13000be20 comp rx=0 tx=0).stop 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 shutdown_connections 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe12806d8a0 0x7fe12806fd50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe144102470 0x7fe1441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 --2- 172.21.15.57:0/2507604122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe144108470 0x7fe144198730 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.674+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 >> 172.21.15.57:0/2507604122 conn(0x7fe1440fdf90 msgr2=0x7fe1440ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.675+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 shutdown_connections 2024-02-28T23:34:03.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.675+0000 7fe149e00700 1 -- 172.21.15.57:0/2507604122 wait complete. 2024-02-28T23:34:03.690 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:34:03.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 -- 172.21.15.57:0/569614621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930102470 msgr2=0x7f99301028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 --2- 172.21.15.57:0/569614621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930102470 0x7f99301028e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f991c009910 tx=0x7f991c009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:03.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 -- 172.21.15.57:0/569614621 shutdown_connections 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 --2- 172.21.15.57:0/569614621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930102470 0x7f99301028e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 --2- 172.21.15.57:0/569614621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930108470 0x7f9930108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 -- 172.21.15.57:0/569614621 >> 172.21.15.57:0/569614621 conn(0x7f99300fdf90 msgr2=0x7f99301003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 -- 172.21.15.57:0/569614621 shutdown_connections 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.850+0000 7f9934c00700 1 -- 172.21.15.57:0/569614621 wait complete. 2024-02-28T23:34:03.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.851+0000 7f9934c00700 1 Processor -- start 2024-02-28T23:34:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.851+0000 7f9934c00700 1 -- start start 2024-02-28T23:34:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f9934c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930102470 0x7f99301981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f9934c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f9934c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930198e10 con 0x7f9930108470 2024-02-28T23:34:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f9934c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993019cba0 con 0x7f9930102470 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38004/0 (socket says 172.21.15.57:38004) 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 -- 172.21.15.57:0/4210980872 learned_addr learned my addr 172.21.15.57:0/4210980872 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992de00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930102470 0x7f99301981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 -- 172.21.15.57:0/4210980872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930102470 msgr2=0x7f99301981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930102470 0x7f99301981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.852+0000 7f992d400700 1 -- 172.21.15.57:0/4210980872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0095f0 con 0x7f9930108470 2024-02-28T23:34:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.853+0000 7f992d400700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f991c0092e0 tx=0x7f991c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.854+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f991c01d070 con 0x7f9930108470 2024-02-28T23:34:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.854+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f991c005e00 con 0x7f9930108470 2024-02-28T23:34:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.854+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f991c017400 con 0x7f9930108470 2024-02-28T23:34:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.854+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993019ce20 con 0x7f9930108470 2024-02-28T23:34:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.854+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993019d310 con 0x7f9930108470 2024-02-28T23:34:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.855+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993010a970 con 0x7f9930108470 2024-02-28T23:34:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.856+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f991c00f460 con 0x7f9930108470 2024-02-28T23:34:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.857+0000 7f9926a00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 0x7f991406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.857+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f991c08ea30 con 0x7f9930108470 2024-02-28T23:34:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.858+0000 7f992de00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 0x7f991406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.858+0000 7f992de00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 0x7f991406fd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9920005950 tx=0x7f99200058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:03.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:03.864+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f991c0581c0 con 0x7f9930108470 2024-02-28T23:34:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:03 smithi057 ceph-mon[89491]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:04.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.112+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f993019d580 con 0x7f991406d8a0 2024-02-28T23:34:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.114+0000 7f9926a00700 1 -- 172.21.15.57:0/4210980872 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f993019d580 con 0x7f991406d8a0 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 msgr2=0x7f991406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 0x7f991406fd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9920005950 tx=0x7f99200058e0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 msgr2=0x7f9930198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f991c0092e0 tx=0x7f991c00be20 comp rx=0 tx=0).stop 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 shutdown_connections 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f991406d8a0 0x7f991406fd50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9930102470 0x7f99301981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 --2- 172.21.15.57:0/4210980872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9930108470 0x7f9930198730 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 >> 172.21.15.57:0/4210980872 conn(0x7f99300fdf90 msgr2=0x7f99300ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 shutdown_connections 2024-02-28T23:34:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.117+0000 7f9934c00700 1 -- 172.21.15.57:0/4210980872 wait complete. 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 -- 172.21.15.57:0/1834944449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 msgr2=0x7f4ad810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 --2- 172.21.15.57:0/1834944449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad810ccc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009910 tx=0x7f4ac8009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 -- 172.21.15.57:0/1834944449 shutdown_connections 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 --2- 172.21.15.57:0/1834944449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 0x7f4ad8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 --2- 172.21.15.57:0/1834944449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad810ccc0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 -- 172.21.15.57:0/1834944449 >> 172.21.15.57:0/1834944449 conn(0x7f4ad8076060 msgr2=0x7f4ad8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 -- 172.21.15.57:0/1834944449 shutdown_connections 2024-02-28T23:34:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.267+0000 7f4ade600700 1 -- 172.21.15.57:0/1834944449 wait complete. 2024-02-28T23:34:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.268+0000 7f4ade600700 1 Processor -- start 2024-02-28T23:34:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.268+0000 7f4ade600700 1 -- start start 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4ade600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 0x7f4ad819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4ade600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4ade600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad819d120 con 0x7f4ad810c8f0 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4ade600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad81a0eb0 con 0x7f4ad8106b80 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38018/0 (socket says 172.21.15.57:38018) 2024-02-28T23:34:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 -- 172.21.15.57:0/2282776799 learned_addr learned my addr 172.21.15.57:0/2282776799 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:04.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4ad7400700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 0x7f4ad819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 -- 172.21.15.57:0/2282776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 msgr2=0x7f4ad819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 0x7f4ad819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.269+0000 7f4acea00700 1 -- 172.21.15.57:0/2282776799 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80095f0 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4acea00700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000ada0 tx=0x7f4ac000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac000f070 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad81a1190 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac0007bf0 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac0005690 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.270+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad81a16e0 con 0x7f4ad810c8f0 2024-02-28T23:34:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.271+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8109b20 con 0x7f4ad810c8f0 2024-02-28T23:34:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.272+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4ac0005900 con 0x7f4ad810c8f0 2024-02-28T23:34:04.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.273+0000 7f4ad4c00700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 0x7f4abc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.273+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4ac008cfe0 con 0x7f4ad810c8f0 2024-02-28T23:34:04.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.274+0000 7f4ad7400700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 0x7f4abc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.274+0000 7f4ad7400700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 0x7f4abc06fd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000c00 tx=0x7f4ac8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.278+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ac00c0d20 con 0x7f4ad810c8f0 2024-02-28T23:34:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:03 smithi084 ceph-mon[70440]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:04.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.507+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad81a1900 con 0x7f4abc06d8a0 2024-02-28T23:34:04.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.536+0000 7f4ad4c00700 1 -- 172.21.15.57:0/2282776799 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4ad81a1900 con 0x7f4abc06d8a0 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (23m) 15m ago 37m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (33m) 15m ago 37m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (36m) 14m ago 36m 9319k - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (19m) 15m ago 37m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (19m) 14m ago 36m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (23m) 15m ago 37m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (15m) 15m ago 30m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (15m) 15m ago 30m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:34:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (30m) 14m ago 30m 12.0M - 18.2.0 dc2bc1663786 dae74dd3ce37 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (14m) 14m ago 30m 11.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (26m) 15m ago 39m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (25m) 14m ago 36m 452M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (21m) 15m ago 39m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (20m) 14m ago 36m 46.6M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (24m) 15m ago 37m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (24m) 14m ago 36m 8032k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (19m) 15m ago 35m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (18m) 15m ago 34m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (18m) 15m ago 33m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (17m) 14m ago 33m 108M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (16m) 14m ago 32m 101M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (16m) 14m ago 32m 87.8M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:34:04.537 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (24m) 15m ago 36m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.539+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 msgr2=0x7f4abc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.539+0000 7f4ade600700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 0x7f4abc06fd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000c00 tx=0x7f4ac8005fd0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.539+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 msgr2=0x7f4ad819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.539+0000 7f4ade600700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000ada0 tx=0x7f4ac000c330 comp rx=0 tx=0).stop 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 shutdown_connections 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4abc06d8a0 0x7f4abc06fd50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad8106b80 0x7f4ad819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 --2- 172.21.15.57:0/2282776799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad810c8f0 0x7f4ad819ca40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 >> 172.21.15.57:0/2282776799 conn(0x7f4ad8076060 msgr2=0x7f4ad8077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:04.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 shutdown_connections 2024-02-28T23:34:04.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.540+0000 7f4ade600700 1 -- 172.21.15.57:0/2282776799 wait complete. 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.701+0000 7f6090c00700 1 -- 172.21.15.57:0/3043510894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c10c860 msgr2=0x7f608c10cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.701+0000 7f6090c00700 1 --2- 172.21.15.57:0/3043510894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c10c860 0x7f608c10cc30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f607c009910 tx=0x7f607c009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 -- 172.21.15.57:0/3043510894 shutdown_connections 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 --2- 172.21.15.57:0/3043510894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c106b80 0x7f608c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 --2- 172.21.15.57:0/3043510894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c10c860 0x7f608c10cc30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 -- 172.21.15.57:0/3043510894 >> 172.21.15.57:0/3043510894 conn(0x7f608c075f50 msgr2=0x7f608c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 -- 172.21.15.57:0/3043510894 shutdown_connections 2024-02-28T23:34:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.702+0000 7f6090c00700 1 -- 172.21.15.57:0/3043510894 wait complete. 2024-02-28T23:34:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.703+0000 7f6090c00700 1 Processor -- start 2024-02-28T23:34:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6090c00700 1 -- start start 2024-02-28T23:34:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6090c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6090c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c10c860 0x7f608c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6090c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c19ce60 con 0x7f608c106b80 2024-02-28T23:34:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6090c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c1a0bf0 con 0x7f608c10c860 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6089e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6089e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38036/0 (socket says 172.21.15.57:38036) 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.704+0000 7f6089e00700 1 -- 172.21.15.57:0/646072027 learned_addr learned my addr 172.21.15.57:0/646072027 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.705+0000 7f6089e00700 1 -- 172.21.15.57:0/646072027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c10c860 msgr2=0x7f608c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.705+0000 7f6089400700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c10c860 0x7f608c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.705+0000 7f6089e00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c10c860 0x7f608c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.705+0000 7f6089e00700 1 -- 172.21.15.57:0/646072027 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0095f0 con 0x7f608c106b80 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.705+0000 7f6089e00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f607c006010 tx=0x7f607c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.706+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c01d070 con 0x7f608c106b80 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.706+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f607c00f460 con 0x7f608c106b80 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.706+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c017630 con 0x7f608c106b80 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.706+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c1a0e70 con 0x7f608c106b80 2024-02-28T23:34:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.706+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c1a1390 con 0x7f608c106b80 2024-02-28T23:34:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.707+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c04f2a0 con 0x7f608c106b80 2024-02-28T23:34:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.708+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f607c00f5d0 con 0x7f608c106b80 2024-02-28T23:34:04.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.709+0000 7f6082a00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.709+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f607c08ea90 con 0x7f608c106b80 2024-02-28T23:34:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.709+0000 7f6089400700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.710+0000 7f6089400700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 0x7f607006fd50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6078005950 tx=0x7f60780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:04.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:04.714+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f607c0581a0 con 0x7f608c106b80 2024-02-28T23:34:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.036+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f608c1a1850 con 0x7f608c106b80 2024-02-28T23:34:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.037+0000 7f6082a00700 1 -- 172.21.15.57:0/646072027 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f607c05b860 con 0x7f608c106b80 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 3 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 13 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:34:05.038 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:34:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 msgr2=0x7f607006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 0x7f607006fd50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6078005950 tx=0x7f60780058e0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 msgr2=0x7f608c19c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f607c006010 tx=0x7f607c00be20 comp rx=0 tx=0).stop 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 shutdown_connections 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c106b80 0x7f608c19c240 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 --2- 172.21.15.57:0/646072027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c10c860 0x7f608c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 >> 172.21.15.57:0/646072027 conn(0x7f608c075f50 msgr2=0x7f608c076720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.040+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 shutdown_connections 2024-02-28T23:34:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.041+0000 7f6090c00700 1 -- 172.21.15.57:0/646072027 wait complete. 2024-02-28T23:34:05.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 -- 172.21.15.57:0/1253963130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c069140 msgr2=0x7f5b6c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 --2- 172.21.15.57:0/1253963130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c069140 0x7f5b6c104eb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009910 tx=0x7f5b54009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:05.187 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:04 smithi057 ceph-mon[89491]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:05.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 -- 172.21.15.57:0/1253963130 shutdown_connections 2024-02-28T23:34:05.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 --2- 172.21.15.57:0/1253963130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c069140 0x7f5b6c104eb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 --2- 172.21.15.57:0/1253963130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.187+0000 7f5b70c00700 1 -- 172.21.15.57:0/1253963130 >> 172.21.15.57:0/1253963130 conn(0x7f5b6c075130 msgr2=0x7f5b6c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.188+0000 7f5b70c00700 1 -- 172.21.15.57:0/1253963130 shutdown_connections 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.188+0000 7f5b70c00700 1 -- 172.21.15.57:0/1253963130 wait complete. 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 Processor -- start 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 -- start start 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c069140 0x7f5b6c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c100720 con 0x7f5b6c0687a0 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.189+0000 7f5b70c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c100860 con 0x7f5b6c069140 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37008/0 (socket says 172.21.15.57:37008) 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 -- 172.21.15.57:0/2263554247 learned_addr learned my addr 172.21.15.57:0/2263554247 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b61400700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c069140 0x7f5b6c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 -- 172.21.15.57:0/2263554247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c069140 msgr2=0x7f5b6c1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c069140 0x7f5b6c1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 -- 172.21.15.57:0/2263554247 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540095f0 con 0x7f5b6c0687a0 2024-02-28T23:34:05.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.190+0000 7f5b69e00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00cfd0 tx=0x7f5b5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.191+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c00ad70 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.191+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b5c0128e0 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.191+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c1a2030 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.191+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c011880 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.191+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c1a2390 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.192+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6c108750 con 0x7f5b6c0687a0 2024-02-28T23:34:05.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.194+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5b5c012420 con 0x7f5b6c0687a0 2024-02-28T23:34:05.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.194+0000 7f5b63400700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 0x7f5b5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.194+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5b5c08d660 con 0x7f5b6c0687a0 2024-02-28T23:34:05.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.195+0000 7f5b61400700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 0x7f5b5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.195+0000 7f5b61400700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 0x7f5b5006fd50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c101490 tx=0x7f5b5401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:05.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.199+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b5c056df0 con 0x7f5b6c0687a0 2024-02-28T23:34:05.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:04 smithi084 ceph-mon[70440]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:05.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.466+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b6c04f2a0 con 0x7f5b6c0687a0 2024-02-28T23:34:05.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.467+0000 7f5b63400700 1 -- 172.21.15.57:0/2263554247 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f5b5c05a4b0 con 0x7f5b6c0687a0 2024-02-28T23:34:05.467 INFO:teuthology.orchestra.run.smithi057.stdout:e31 2024-02-28T23:34:05.467 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:34:05.467 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:34:05.467 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:34:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:34:05.468 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:34:05.469 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:34:05.470 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/369172854,v1:172.21.15.84:6827/369172854] compat {c=[1],r=[1],i=[7ff]}] 2024-02-28T23:34:05.471 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.470+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 msgr2=0x7f5b5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.470+0000 7f5b70c00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 0x7f5b5006fd50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c101490 tx=0x7f5b5401a040 comp rx=0 tx=0).stop 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.470+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 msgr2=0x7f5b6c0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.470+0000 7f5b70c00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00cfd0 tx=0x7f5b5c00a340 comp rx=0 tx=0).stop 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 shutdown_connections 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b5006d8a0 0x7f5b5006fd50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b6c0687a0 0x7f5b6c0ffca0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 --2- 172.21.15.57:0/2263554247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c069140 0x7f5b6c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 >> 172.21.15.57:0/2263554247 conn(0x7f5b6c075130 msgr2=0x7f5b6c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:05.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 shutdown_connections 2024-02-28T23:34:05.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.471+0000 7f5b70c00700 1 -- 172.21.15.57:0/2263554247 wait complete. 2024-02-28T23:34:05.474 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 -- 172.21.15.57:0/2768395200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068ee0 msgr2=0x7f2210110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 --2- 172.21.15.57:0/2768395200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068ee0 0x7f2210110a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009940 tx=0x7f21fc009c50 comp rx=0 tx=0).stop 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 -- 172.21.15.57:0/2768395200 shutdown_connections 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 --2- 172.21.15.57:0/2768395200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068ee0 0x7f2210110a10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 --2- 172.21.15.57:0/2768395200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068540 0x7f2210068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 -- 172.21.15.57:0/2768395200 >> 172.21.15.57:0/2768395200 conn(0x7f2210075130 msgr2=0x7f2210075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 -- 172.21.15.57:0/2768395200 shutdown_connections 2024-02-28T23:34:05.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.626+0000 7f2216600700 1 -- 172.21.15.57:0/2768395200 wait complete. 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.627+0000 7f2216600700 1 Processor -- start 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.628+0000 7f2216600700 1 -- start start 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.628+0000 7f2216600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.628+0000 7f2216600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068ee0 0x7f221019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.628+0000 7f2216600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221019d080 con 0x7f2210068540 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.628+0000 7f2216600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22101a0e10 con 0x7f2210068ee0 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37038/0 (socket says 172.21.15.57:37038) 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 -- 172.21.15.57:0/3800930512 learned_addr learned my addr 172.21.15.57:0/3800930512 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220ea00700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068ee0 0x7f221019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 -- 172.21.15.57:0/3800930512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068ee0 msgr2=0x7f221019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068ee0 0x7f221019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 -- 172.21.15.57:0/3800930512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0095f0 con 0x7f2210068540 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.629+0000 7f220f400700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f220000d8d0 tx=0x7f220000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.630+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2200009940 con 0x7f2210068540 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.630+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2200010460 con 0x7f2210068540 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.630+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22101a10f0 con 0x7f2210068540 2024-02-28T23:34:05.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.630+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f220000f5f0 con 0x7f2210068540 2024-02-28T23:34:05.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.630+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22101a1640 con 0x7f2210068540 2024-02-28T23:34:05.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.631+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2210101410 con 0x7f2210068540 2024-02-28T23:34:05.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.632+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f22000105d0 con 0x7f2210068540 2024-02-28T23:34:05.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.633+0000 7f2207e00700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 0x7f21f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:05.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.633+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f220008d4c0 con 0x7f2210068540 2024-02-28T23:34:05.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.634+0000 7f220ea00700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 0x7f21f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:05.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.634+0000 7f220ea00700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 0x7f21f406fd50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f221019da80 tx=0x7f21fc005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:05.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.638+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2200056c50 con 0x7f2210068540 2024-02-28T23:34:05.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.888+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22101a18b0 con 0x7f21f406d8a0 2024-02-28T23:34:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.891+0000 7f2207e00700 1 -- 172.21.15.57:0/3800930512 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f22101a18b0 con 0x7f21f406d8a0 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr", 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:34:05.891 INFO:teuthology.orchestra.run.smithi057.stdout: "osd" 2024-02-28T23:34:05.892 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:34:05.892 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "15/23 daemons upgraded", 2024-02-28T23:34:05.892 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:34:05.892 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:34:05.892 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:34:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 msgr2=0x7f21f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 0x7f21f406fd50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f221019da80 tx=0x7f21fc005ab0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 msgr2=0x7f221019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f220000d8d0 tx=0x7f220000dc90 comp rx=0 tx=0).stop 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 shutdown_connections 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f21f406d8a0 0x7f21f406fd50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2210068540 0x7f221019c460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 --2- 172.21.15.57:0/3800930512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2210068ee0 0x7f221019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 >> 172.21.15.57:0/3800930512 conn(0x7f2210075130 msgr2=0x7f221010eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 shutdown_connections 2024-02-28T23:34:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:05.894+0000 7f2216600700 1 -- 172.21.15.57:0/3800930512 wait complete. 2024-02-28T23:34:06.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 -- 172.21.15.57:0/1571693142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 msgr2=0x7f5d54106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:06.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1571693142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d54106e40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009910 tx=0x7f5d40009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 -- 172.21.15.57:0/1571693142 shutdown_connections 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1571693142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d54106e40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1571693142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.054+0000 7f5d59e00700 1 -- 172.21.15.57:0/1571693142 >> 172.21.15.57:0/1571693142 conn(0x7f5d540fdd90 msgr2=0x7f5d541001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.055+0000 7f5d59e00700 1 -- 172.21.15.57:0/1571693142 shutdown_connections 2024-02-28T23:34:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.055+0000 7f5d59e00700 1 -- 172.21.15.57:0/1571693142 wait complete. 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 Processor -- start 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 -- start start 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d54197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54198ba0 con 0x7f5d54102e70 2024-02-28T23:34:06.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d59e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419c930 con 0x7f5d541024d0 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37054/0 (socket says 172.21.15.57:37054) 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.056+0000 7f5d53e00700 1 -- 172.21.15.57:0/1376484559 learned_addr learned my addr 172.21.15.57:0/1376484559 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d53e00700 1 -- 172.21.15.57:0/1376484559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 msgr2=0x7f5d54197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d58a00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d54197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d53e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d54197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d53e00700 1 -- 172.21.15.57:0/1376484559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400095f0 con 0x7f5d54102e70 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d58a00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d54197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:34:06.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.057+0000 7f5d53e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000bda0 tx=0x7f5d4000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.058+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4001d070 con 0x7f5d54102e70 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.058+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d40004fb0 con 0x7f5d54102e70 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.058+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4000f460 con 0x7f5d54102e70 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.058+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5419cbb0 con 0x7f5d54102e70 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.058+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5419d0a0 con 0x7f5d54102e70 2024-02-28T23:34:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.059+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5410a790 con 0x7f5d54102e70 2024-02-28T23:34:06.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.060+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5d40022800 con 0x7f5d54102e70 2024-02-28T23:34:06.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.061+0000 7f5d51600700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 0x7f5d3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:06.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.061+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d4008ea30 con 0x7f5d54102e70 2024-02-28T23:34:06.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.061+0000 7f5d58a00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 0x7f5d3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:06.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.062+0000 7f5d58a00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 0x7f5d3c06fd50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007900 tx=0x7f5d48008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:06.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.066+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d40058390 con 0x7f5d54102e70 2024-02-28T23:34:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:05 smithi057 ceph-mon[89491]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:05 smithi057 ceph-mon[89491]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:05 smithi057 ceph-mon[89491]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:05 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/646072027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:06.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:05 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2263554247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:34:06.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:05 smithi084 ceph-mon[70440]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:05 smithi084 ceph-mon[70440]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:05 smithi084 ceph-mon[70440]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:05 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/646072027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:05 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2263554247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:34:06.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.373+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d54062380 con 0x7f5d54102e70 2024-02-28T23:34:06.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.373+0000 7f5d51600700 1 -- 172.21.15.57:0/1376484559 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5d40027100 con 0x7f5d54102e70 2024-02-28T23:34:06.373 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:34:06.373 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:34:06.373 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:34:06.373 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:34:06.373 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 msgr2=0x7f5d3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 0x7f5d3c06fd50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007900 tx=0x7f5d48008040 comp rx=0 tx=0).stop 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 msgr2=0x7f5d541984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000bda0 tx=0x7f5d4000be80 comp rx=0 tx=0).stop 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 shutdown_connections 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5d3c06d8a0 0x7f5d3c06fd50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d541024d0 0x7f5d54197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 --2- 172.21.15.57:0/1376484559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d54102e70 0x7f5d541984c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.376+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 >> 172.21.15.57:0/1376484559 conn(0x7f5d540fdd90 msgr2=0x7f5d540ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.377+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 shutdown_connections 2024-02-28T23:34:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:06.377+0000 7f5d59e00700 1 -- 172.21.15.57:0/1376484559 wait complete. 2024-02-28T23:34:07.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:06 smithi057 ceph-mon[89491]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:07.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:06 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1376484559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:34:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:06 smithi084 ceph-mon[70440]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:06 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1376484559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:34:08.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:07 smithi057 ceph-mon[89491]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:08.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:07 smithi084 ceph-mon[70440]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:10.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:09 smithi057 ceph-mon[89491]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:10.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:09 smithi084 ceph-mon[70440]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:12.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:11 smithi057 ceph-mon[89491]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:12.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:11 smithi084 ceph-mon[70440]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:14.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:13 smithi057 ceph-mon[89491]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:14.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:13 smithi084 ceph-mon[70440]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:15 smithi057 ceph-mon[89491]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:16.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:15 smithi084 ceph-mon[70440]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:18.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:17 smithi057 ceph-mon[89491]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:17 smithi084 ceph-mon[70440]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:34:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:20.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:20 smithi057 ceph-mon[89491]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:20 smithi057 ceph-mon[89491]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-28T23:34:21.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:20 smithi084 ceph-mon[70440]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:20 smithi084 ceph-mon[70440]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-28T23:34:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:23.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: Upgrade: Updating mds.cephfs.smithi084.dadqfu 2024-02-28T23:34:23.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:23.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:34:23.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:34:23.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:22 smithi084 ceph-mon[70440]: Deploying daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:34:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: Upgrade: Updating mds.cephfs.smithi084.dadqfu 2024-02-28T23:34:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi084.dadqfu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-28T23:34:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:34:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:22 smithi057 ceph-mon[89491]: Deploying daemon mds.cephfs.smithi084.dadqfu on smithi084 2024-02-28T23:34:25.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:24 smithi057 ceph-mon[89491]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:24 smithi084 ceph-mon[70440]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:26 smithi057 ceph-mon[89491]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:26 smithi057 ceph-mon[89491]: osdmap e87: 6 total, 6 up, 6 in 2024-02-28T23:34:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:26 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:34:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:26 smithi084 ceph-mon[70440]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:26 smithi084 ceph-mon[70440]: osdmap e87: 6 total, 6 up, 6 in 2024-02-28T23:34:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:26 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-28T23:34:28.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:34:28.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:34:29.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:28 smithi057 ceph-mon[89491]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:29.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:28 smithi084 ceph-mon[70440]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:30 smithi057 ceph-mon[89491]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:31.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:30 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:34:31.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:30 smithi084 ceph-mon[70440]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 984 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:31.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:31.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:31.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:30 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] up:boot 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] up:boot 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:33 smithi057 ceph-mon[89491]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:33.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:33 smithi084 ceph-mon[70440]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:34 smithi084 ceph-mon[70440]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:34.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:34 smithi057 ceph-mon[89491]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 -- 172.21.15.57:0/695038057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82200687a0 msgr2=0x7f8220068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 --2- 172.21.15.57:0/695038057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82200687a0 0x7f8220068b70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8210009910 tx=0x7f8210009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 -- 172.21.15.57:0/695038057 shutdown_connections 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 --2- 172.21.15.57:0/695038057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8220069140 0x7f8220104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 --2- 172.21.15.57:0/695038057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82200687a0 0x7f8220068b70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 -- 172.21.15.57:0/695038057 >> 172.21.15.57:0/695038057 conn(0x7f8220075130 msgr2=0x7f8220075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 -- 172.21.15.57:0/695038057 shutdown_connections 2024-02-28T23:34:36.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.555+0000 7f8226a00700 1 -- 172.21.15.57:0/695038057 wait complete. 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 Processor -- start 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 -- start start 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82200687a0 0x7f82201980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220198cd0 con 0x7f8220069140 2024-02-28T23:34:36.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.557+0000 7f8226a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019ca60 con 0x7f82200687a0 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41018/0 (socket says 172.21.15.57:41018) 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 -- 172.21.15.57:0/3332761271 learned_addr learned my addr 172.21.15.57:0/3332761271 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821fe00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82200687a0 0x7f82201980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 -- 172.21.15.57:0/3332761271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82200687a0 msgr2=0x7f82201980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82200687a0 0x7f82201980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.558+0000 7f821f400700 1 -- 172.21.15.57:0/3332761271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100095f0 con 0x7f8220069140 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f821f400700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d900 tx=0x7f820c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f820c009900 con 0x7f8220069140 2024-02-28T23:34:36.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f820c010460 con 0x7f8220069140 2024-02-28T23:34:36.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822019cd40 con 0x7f8220069140 2024-02-28T23:34:36.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f820c00f5f0 con 0x7f8220069140 2024-02-28T23:34:36.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.559+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822019d290 con 0x7f8220069140 2024-02-28T23:34:36.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.561+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004f2a0 con 0x7f8220069140 2024-02-28T23:34:36.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.562+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f820c0105d0 con 0x7f8220069140 2024-02-28T23:34:36.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.563+0000 7f821cc00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 0x7f820406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:36.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.563+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f820c08d0d0 con 0x7f8220069140 2024-02-28T23:34:36.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.563+0000 7f821fe00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 0x7f820406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:36.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.564+0000 7f821fe00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 0x7f820406fd50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8210006010 tx=0x7f821001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:36.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.569+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f820c056880 con 0x7f8220069140 2024-02-28T23:34:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:36 smithi057 ceph-mon[89491]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:36.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:36 smithi084 ceph-mon[70440]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:36.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:36.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:36.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.832+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f822019d520 con 0x7f820406d8a0 2024-02-28T23:34:36.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.834+0000 7f821cc00700 1 -- 172.21.15.57:0/3332761271 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f822019d520 con 0x7f820406d8a0 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 msgr2=0x7f820406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 0x7f820406fd50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8210006010 tx=0x7f821001a040 comp rx=0 tx=0).stop 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 msgr2=0x7f82201985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d900 tx=0x7f820c00dc10 comp rx=0 tx=0).stop 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 shutdown_connections 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f820406d8a0 0x7f820406fd50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82200687a0 0x7f82201980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 --2- 172.21.15.57:0/3332761271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8220069140 0x7f82201985f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.837+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 >> 172.21.15.57:0/3332761271 conn(0x7f8220075130 msgr2=0x7f8220101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.838+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 shutdown_connections 2024-02-28T23:34:36.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:36.838+0000 7f8226a00700 1 -- 172.21.15.57:0/3332761271 wait complete. 2024-02-28T23:34:36.852 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.006+0000 7f0d63400700 1 -- 172.21.15.57:0/3667046994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 msgr2=0x7f0d5c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.006+0000 7f0d63400700 1 --2- 172.21.15.57:0/3667046994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c106ff0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009910 tx=0x7f0d48009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 -- 172.21.15.57:0/3667046994 shutdown_connections 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 --2- 172.21.15.57:0/3667046994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c106ff0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 --2- 172.21.15.57:0/3667046994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 0x7f0d5c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 -- 172.21.15.57:0/3667046994 >> 172.21.15.57:0/3667046994 conn(0x7f0d5c076060 msgr2=0x7f0d5c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 -- 172.21.15.57:0/3667046994 shutdown_connections 2024-02-28T23:34:37.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.007+0000 7f0d63400700 1 -- 172.21.15.57:0/3667046994 wait complete. 2024-02-28T23:34:37.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.008+0000 7f0d63400700 1 Processor -- start 2024-02-28T23:34:37.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.008+0000 7f0d63400700 1 -- start start 2024-02-28T23:34:37.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d63400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d63400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 0x7f0d5c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d63400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c19d060 con 0x7f0d5c106b80 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d63400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1a0df0 con 0x7f0d5c10c8f0 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d60a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d60a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41026/0 (socket says 172.21.15.57:41026) 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d60a00700 1 -- 172.21.15.57:0/2349432022 learned_addr learned my addr 172.21.15.57:0/2349432022 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.009+0000 7f0d5be00700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 0x7f0d5c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d60a00700 1 -- 172.21.15.57:0/2349432022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 msgr2=0x7f0d5c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d60a00700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 0x7f0d5c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d60a00700 1 -- 172.21.15.57:0/2349432022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480095f0 con 0x7f0d5c106b80 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d60a00700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e920 tx=0x7f0d5000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d50009790 con 0x7f0d5c106b80 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c1a10d0 con 0x7f0d5c106b80 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d5000cba0 con 0x7f0d5c106b80 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d50018650 con 0x7f0d5c106b80 2024-02-28T23:34:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.010+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c1a1620 con 0x7f0d5c106b80 2024-02-28T23:34:37.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.012+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c10ecc0 con 0x7f0d5c106b80 2024-02-28T23:34:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.013+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0d5000cd10 con 0x7f0d5c106b80 2024-02-28T23:34:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.014+0000 7f0d59600700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 0x7f0d4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.014+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d50014070 con 0x7f0d5c106b80 2024-02-28T23:34:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.014+0000 7f0d5be00700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 0x7f0d4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.015+0000 7f0d5be00700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 0x7f0d4406fd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b760 tx=0x7f0d48005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.019+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d50052d20 con 0x7f0d5c106b80 2024-02-28T23:34:37.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.282+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d5c1a1890 con 0x7f0d4406d8a0 2024-02-28T23:34:37.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.284+0000 7f0d59600700 1 -- 172.21.15.57:0/2349432022 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0d5c1a1890 con 0x7f0d4406d8a0 2024-02-28T23:34:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 msgr2=0x7f0d4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 0x7f0d4406fd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b760 tx=0x7f0d48005fd0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 msgr2=0x7f0d5c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e920 tx=0x7f0d5000ece0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 shutdown_connections 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4406d8a0 0x7f0d4406fd50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d5c106b80 0x7f0d5c19c440 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 --2- 172.21.15.57:0/2349432022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d5c10c8f0 0x7f0d5c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.287+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 >> 172.21.15.57:0/2349432022 conn(0x7f0d5c076060 msgr2=0x7f0d5c077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.288+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 shutdown_connections 2024-02-28T23:34:37.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.288+0000 7f0d63400700 1 -- 172.21.15.57:0/2349432022 wait complete. 2024-02-28T23:34:37.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 -- 172.21.15.57:0/2158619113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09040730f0 msgr2=0x7f09040734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 --2- 172.21.15.57:0/2158619113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09040730f0 0x7f09040734c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f08f400b0a0 tx=0x7f08f400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 -- 172.21.15.57:0/2158619113 shutdown_connections 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 --2- 172.21.15.57:0/2158619113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0904073a00 0x7f090410c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 --2- 172.21.15.57:0/2158619113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09040730f0 0x7f09040734c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 -- 172.21.15.57:0/2158619113 >> 172.21.15.57:0/2158619113 conn(0x7f09040fbb50 msgr2=0x7f09040fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 -- 172.21.15.57:0/2158619113 shutdown_connections 2024-02-28T23:34:37.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.447+0000 7f0908a00700 1 -- 172.21.15.57:0/2158619113 wait complete. 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.448+0000 7f0908a00700 1 Processor -- start 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.448+0000 7f0908a00700 1 -- start start 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f0908a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09040730f0 0x7f0904197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f0908a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f0908a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090419c6d0 con 0x7f0904073a00 2024-02-28T23:34:37.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f0908a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090419c840 con 0x7f09040730f0 2024-02-28T23:34:37.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f08fbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f08fbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41052/0 (socket says 172.21.15.57:41052) 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f08fbe00700 1 -- 172.21.15.57:0/2709080086 learned_addr learned my addr 172.21.15.57:0/2709080086 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.449+0000 7f0903400700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09040730f0 0x7f0904197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f08fbe00700 1 -- 172.21.15.57:0/2709080086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09040730f0 msgr2=0x7f0904197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f08fbe00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09040730f0 0x7f0904197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f08fbe00700 1 -- 172.21.15.57:0/2709080086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009d00 con 0x7f0904073a00 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f08fbe00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d8d0 tx=0x7f08ec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ec0098a0 con 0x7f0904073a00 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08ec010460 con 0x7f0904073a00 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090419cb20 con 0x7f0904073a00 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.450+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ec00f5f0 con 0x7f0904073a00 2024-02-28T23:34:37.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.451+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090419d070 con 0x7f0904073a00 2024-02-28T23:34:37.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.452+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0904109a50 con 0x7f0904073a00 2024-02-28T23:34:37.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.453+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f08ec009a00 con 0x7f0904073a00 2024-02-28T23:34:37.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.453+0000 7f0900c00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 0x7f08e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.454+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f08ec08d490 con 0x7f0904073a00 2024-02-28T23:34:37.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.454+0000 7f0903400700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 0x7f08e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.455+0000 7f0903400700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 0x7f08e806fd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009480 tx=0x7f08f400b7b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.459+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f08ec056ba0 con 0x7f0904073a00 2024-02-28T23:34:37.567 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:37 smithi057 ceph-mon[89491]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:37.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.713+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09041991c0 con 0x7f08e806d8a0 2024-02-28T23:34:37.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.735+0000 7f0900c00700 1 -- 172.21.15.57:0/2709080086 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f09041991c0 con 0x7f08e806d8a0 2024-02-28T23:34:37.733 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (24m) 15m ago 38m 14.0M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (34m) 15m ago 38m 9403k - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (36m) 2s ago 36m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (20m) 15m ago 38m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (20m) 2s ago 36m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (23m) 15m ago 37m 74.3M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (16m) 15m ago 30m 11.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (15m) 15m ago 30m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (7s) 2s ago 30m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (15m) 2s ago 30m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (26m) 15m ago 39m 507M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (25m) 2s ago 36m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:34:37.734 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (22m) 15m ago 40m 55.0M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (21m) 2s ago 36m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (25m) 15m ago 38m 8056k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (25m) 2s ago 36m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (19m) 15m ago 35m 92.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (19m) 15m ago 34m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (18m) 15m ago 34m 83.0M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (18m) 2s ago 33m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (17m) 2s ago 33m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (16m) 2s ago 32m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:34:37.735 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (24m) 15m ago 37m 70.1M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 msgr2=0x7f08e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 0x7f08e806fd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009480 tx=0x7f08f400b7b0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 msgr2=0x7f0904198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d8d0 tx=0x7f08ec00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 shutdown_connections 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f08e806d8a0 0x7f08e806fd50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09040730f0 0x7f0904197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 --2- 172.21.15.57:0/2709080086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0904073a00 0x7f0904198390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.739+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 >> 172.21.15.57:0/2709080086 conn(0x7f09040fbb50 msgr2=0x7f0904106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.740+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 shutdown_connections 2024-02-28T23:34:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.740+0000 7f0908a00700 1 -- 172.21.15.57:0/2709080086 wait complete. 2024-02-28T23:34:37.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:37 smithi084 ceph-mon[70440]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 -- 172.21.15.57:0/3268615065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05041061b0 msgr2=0x7f0504106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 --2- 172.21.15.57:0/3268615065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05041061b0 0x7f0504106580 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009910 tx=0x7f04f8009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 -- 172.21.15.57:0/3268615065 shutdown_connections 2024-02-28T23:34:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 --2- 172.21.15.57:0/3268615065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0504068540 0x7f05040689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 --2- 172.21.15.57:0/3268615065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05041061b0 0x7f0504106580 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 -- 172.21.15.57:0/3268615065 >> 172.21.15.57:0/3268615065 conn(0x7f0504074fa0 msgr2=0x7f05040753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 -- 172.21.15.57:0/3268615065 shutdown_connections 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.892+0000 7f050a600700 1 -- 172.21.15.57:0/3268615065 wait complete. 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.893+0000 7f050a600700 1 Processor -- start 2024-02-28T23:34:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f050a600700 1 -- start start 2024-02-28T23:34:37.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f050a600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f050a600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05041061b0 0x7f05040ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f050a600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0504100530 con 0x7f0504068540 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f050a600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0504100670 con 0x7f05041061b0 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f0509200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f0509200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41068/0 (socket says 172.21.15.57:41068) 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f0509200700 1 -- 172.21.15.57:0/2215402594 learned_addr learned my addr 172.21.15.57:0/2215402594 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.894+0000 7f0503e00700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05041061b0 0x7f05040ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0509200700 1 -- 172.21.15.57:0/2215402594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05041061b0 msgr2=0x7f05040ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0509200700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05041061b0 0x7f05040ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:37.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0509200700 1 -- 172.21.15.57:0/2215402594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f80095f0 con 0x7f0504068540 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0509200700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f04f8006010 tx=0x7f04f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f801d070 con 0x7f0504068540 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05041008f0 con 0x7f0504068540 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f800f460 con 0x7f0504068540 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.895+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f8017630 con 0x7f0504068540 2024-02-28T23:34:37.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.896+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05041a2110 con 0x7f0504068540 2024-02-28T23:34:37.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.897+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0504062380 con 0x7f0504068540 2024-02-28T23:34:37.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.898+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f04f800f5d0 con 0x7f0504068540 2024-02-28T23:34:37.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.899+0000 7f0501600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 0x7f04ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:37.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.899+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f04f808ea80 con 0x7f0504068540 2024-02-28T23:34:37.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.900+0000 7f0503e00700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 0x7f04ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:37.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.900+0000 7f0503e00700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 0x7f04ec06fd50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005950 tx=0x7f04f00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:37.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:37.906+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04f8058190 con 0x7f0504068540 2024-02-28T23:34:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.312+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0504100fe0 con 0x7f0504068540 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.313+0000 7f0501600700 1 -- 172.21.15.57:0/2215402594 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f04f805b850 con 0x7f0504068540 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:34:38.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:34:38.312 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 msgr2=0x7f04ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 0x7f04ec06fd50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005950 tx=0x7f04f00058e0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 msgr2=0x7f05040ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f04f8006010 tx=0x7f04f800be20 comp rx=0 tx=0).stop 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 shutdown_connections 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f04ec06d8a0 0x7f04ec06fd50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0504068540 0x7f05040ffab0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 --2- 172.21.15.57:0/2215402594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05041061b0 0x7f05040ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 >> 172.21.15.57:0/2215402594 conn(0x7f0504074fa0 msgr2=0x7f05040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:38.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 shutdown_connections 2024-02-28T23:34:38.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.316+0000 7f050a600700 1 -- 172.21.15.57:0/2215402594 wait complete. 2024-02-28T23:34:38.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 -- 172.21.15.57:0/3679277846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 msgr2=0x7fa53c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 --2- 172.21.15.57:0/3679277846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c1008b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009910 tx=0x7fa52c009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 -- 172.21.15.57:0/3679277846 shutdown_connections 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 --2- 172.21.15.57:0/3679277846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 0x7fa53c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 --2- 172.21.15.57:0/3679277846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c1008b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 -- 172.21.15.57:0/3679277846 >> 172.21.15.57:0/3679277846 conn(0x7fa53c0fbda0 msgr2=0x7fa53c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 -- 172.21.15.57:0/3679277846 shutdown_connections 2024-02-28T23:34:38.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.485+0000 7fa542600700 1 -- 172.21.15.57:0/3679277846 wait complete. 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.486+0000 7fa542600700 1 Processor -- start 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa542600700 1 -- start start 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa542600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa542600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 0x7fa53c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa542600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c196b10 con 0x7fa53c1004e0 2024-02-28T23:34:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa542600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c19a8a0 con 0x7fa53c100e80 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa53b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa53b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41082/0 (socket says 172.21.15.57:41082) 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.487+0000 7fa53b400700 1 -- 172.21.15.57:0/2635038183 learned_addr learned my addr 172.21.15.57:0/2635038183 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa532a00700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 0x7fa53c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa53b400700 1 -- 172.21.15.57:0/2635038183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 msgr2=0x7fa53c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa53b400700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 0x7fa53c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa53b400700 1 -- 172.21.15.57:0/2635038183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0095f0 con 0x7fa53c1004e0 2024-02-28T23:34:38.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa53b400700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006010 tx=0x7fa52c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:38.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52c01d070 con 0x7fa53c1004e0 2024-02-28T23:34:38.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52c00f460 con 0x7fa53c1004e0 2024-02-28T23:34:38.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c19ab20 con 0x7fa53c1004e0 2024-02-28T23:34:38.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.488+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52c017610 con 0x7fa53c1004e0 2024-02-28T23:34:38.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.489+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c19b010 con 0x7fa53c1004e0 2024-02-28T23:34:38.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.490+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c1087f0 con 0x7fa53c1004e0 2024-02-28T23:34:38.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.491+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa52c00f5d0 con 0x7fa53c1004e0 2024-02-28T23:34:38.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.492+0000 7fa538c00700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 0x7fa52006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.492+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa52c08ea70 con 0x7fa53c1004e0 2024-02-28T23:34:38.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.492+0000 7fa532a00700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 0x7fa52006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.493+0000 7fa532a00700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 0x7fa52006fd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa524005950 tx=0x7fa5240058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.497+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa52c058240 con 0x7fa53c1004e0 2024-02-28T23:34:38.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.769+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa53c197350 con 0x7fa53c1004e0 2024-02-28T23:34:38.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.770+0000 7fa538c00700 1 -- 172.21.15.57:0/2635038183 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fa52c05b900 con 0x7fa53c1004e0 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:34:38.769 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:34:38.770 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:38.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:38.772 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:34:38.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.772+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 msgr2=0x7fa52006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.772+0000 7fa542600700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 0x7fa52006fd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa524005950 tx=0x7fa5240058e0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.772+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 msgr2=0x7fa53c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.772+0000 7fa542600700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006010 tx=0x7fa52c00be20 comp rx=0 tx=0).stop 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 shutdown_connections 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa52006d8a0 0x7fa52006fd50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53c1004e0 0x7fa53c195ef0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 --2- 172.21.15.57:0/2635038183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53c100e80 0x7fa53c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 >> 172.21.15.57:0/2635038183 conn(0x7fa53c0fbda0 msgr2=0x7fa53c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 shutdown_connections 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.773+0000 7fa542600700 1 -- 172.21.15.57:0/2635038183 wait complete. 2024-02-28T23:34:38.773 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:38 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2215402594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:38.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:38 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2215402594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.927+0000 7fd1fb000700 1 -- 172.21.15.57:0/1629553915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f4100e80 msgr2=0x7fd1f4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.927+0000 7fd1fb000700 1 --2- 172.21.15.57:0/1629553915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f4100e80 0x7fd1f4104e50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009910 tx=0x7fd1e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 -- 172.21.15.57:0/1629553915 shutdown_connections 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 --2- 172.21.15.57:0/1629553915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f4100e80 0x7fd1f4104e50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 --2- 172.21.15.57:0/1629553915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f41004e0 0x7fd1f41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 -- 172.21.15.57:0/1629553915 >> 172.21.15.57:0/1629553915 conn(0x7fd1f40fbda0 msgr2=0x7fd1f40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 -- 172.21.15.57:0/1629553915 shutdown_connections 2024-02-28T23:34:38.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.928+0000 7fd1fb000700 1 -- 172.21.15.57:0/1629553915 wait complete. 2024-02-28T23:34:38.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 Processor -- start 2024-02-28T23:34:38.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 -- start start 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f4100e80 0x7fd1f4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4196b50 con 0x7fd1f41004e0 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.929+0000 7fd1fb000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f419a8e0 con 0x7fd1f4100e80 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41094/0 (socket says 172.21.15.57:41094) 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 -- 172.21.15.57:0/3004927227 learned_addr learned my addr 172.21.15.57:0/3004927227 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:38.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3400700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f4100e80 0x7fd1f4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 -- 172.21.15.57:0/3004927227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f4100e80 msgr2=0x7fd1f4196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f4100e80 0x7fd1f4196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 -- 172.21.15.57:0/3004927227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00095f0 con 0x7fd1f41004e0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.930+0000 7fd1f3e00700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e920 tx=0x7fd1e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.931+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e400c9d0 con 0x7fd1f41004e0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.931+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1e4010430 con 0x7fd1f41004e0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.931+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f419abc0 con 0x7fd1f41004e0 2024-02-28T23:34:38.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.931+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e4018630 con 0x7fd1f41004e0 2024-02-28T23:34:38.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.931+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f419b0e0 con 0x7fd1f41004e0 2024-02-28T23:34:38.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.932+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f4108850 con 0x7fd1f41004e0 2024-02-28T23:34:38.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.933+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd1e40105a0 con 0x7fd1f41004e0 2024-02-28T23:34:38.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.934+0000 7fd1f0c00700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 0x7fd1d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:38.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.934+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd1e4014070 con 0x7fd1f41004e0 2024-02-28T23:34:38.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.934+0000 7fd1f3400700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 0x7fd1d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:38.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.935+0000 7fd1f3400700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 0x7fd1d806fd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0005b40 tx=0x7fd1e0005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:38.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:38.939+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1e4052d20 con 0x7fd1f41004e0 2024-02-28T23:34:39.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.177+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1f419b300 con 0x7fd1d806d8a0 2024-02-28T23:34:39.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.178+0000 7fd1f0c00700 1 -- 172.21.15.57:0/3004927227 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd1f419b300 con 0x7fd1d806d8a0 2024-02-28T23:34:39.176 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:34:39.176 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:34:39.176 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:34:39.177 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:34:39.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 msgr2=0x7fd1d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 0x7fd1d806fd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0005b40 tx=0x7fd1e0005a90 comp rx=0 tx=0).stop 2024-02-28T23:34:39.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 msgr2=0x7fd1f4195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e920 tx=0x7fd1e400ec30 comp rx=0 tx=0).stop 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 shutdown_connections 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd1d806d8a0 0x7fd1d806fd50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f41004e0 0x7fd1f4195f30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.181+0000 7fd1fb000700 1 --2- 172.21.15.57:0/3004927227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1f4100e80 0x7fd1f4196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.182+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 >> 172.21.15.57:0/3004927227 conn(0x7fd1f40fbda0 msgr2=0x7fd1f40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.182+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 shutdown_connections 2024-02-28T23:34:39.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.182+0000 7fd1fb000700 1 -- 172.21.15.57:0/3004927227 wait complete. 2024-02-28T23:34:39.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 -- 172.21.15.57:0/2033278078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca500730f0 msgr2=0x7fca500734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 --2- 172.21.15.57:0/2033278078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca500730f0 0x7fca500734c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fca40009910 tx=0x7fca40009c20 comp rx=0 tx=0).stop 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 -- 172.21.15.57:0/2033278078 shutdown_connections 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 --2- 172.21.15.57:0/2033278078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca50073a00 0x7fca50110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 --2- 172.21.15.57:0/2033278078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca500730f0 0x7fca500734c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 -- 172.21.15.57:0/2033278078 >> 172.21.15.57:0/2033278078 conn(0x7fca500fbda0 msgr2=0x7fca500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 -- 172.21.15.57:0/2033278078 shutdown_connections 2024-02-28T23:34:39.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.331+0000 7fca56a00700 1 -- 172.21.15.57:0/2033278078 wait complete. 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.332+0000 7fca56a00700 1 Processor -- start 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca56a00700 1 -- start start 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca56a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca500730f0 0x7fca5019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca56a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca56a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5019d150 con 0x7fca50073a00 2024-02-28T23:34:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca56a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca501a0ee0 con 0x7fca500730f0 2024-02-28T23:34:39.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca4f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:39.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca4f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41116/0 (socket says 172.21.15.57:41116) 2024-02-28T23:34:39.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.333+0000 7fca4f400700 1 -- 172.21.15.57:0/1595040816 learned_addr learned my addr 172.21.15.57:0/1595040816 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4fe00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca500730f0 0x7fca5019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4f400700 1 -- 172.21.15.57:0/1595040816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca500730f0 msgr2=0x7fca5019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4f400700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca500730f0 0x7fca5019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4f400700 1 -- 172.21.15.57:0/1595040816 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400095f0 con 0x7fca50073a00 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4f400700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00b4b0 tx=0x7fca3c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c011800 con 0x7fca50073a00 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca3c011e20 con 0x7fca50073a00 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.334+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c00f2e0 con 0x7fca50073a00 2024-02-28T23:34:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.335+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca501a11c0 con 0x7fca50073a00 2024-02-28T23:34:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.335+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca501a1710 con 0x7fca50073a00 2024-02-28T23:34:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.336+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5010e4d0 con 0x7fca50073a00 2024-02-28T23:34:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.337+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fca3c010380 con 0x7fca50073a00 2024-02-28T23:34:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.338+0000 7fca4cc00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 0x7fca3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:34:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.338+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca3c08d0f0 con 0x7fca50073a00 2024-02-28T23:34:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.338+0000 7fca4fe00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 0x7fca3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:34:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.339+0000 7fca4fe00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 0x7fca3406fd50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:34:39.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.347+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca3c056930 con 0x7fca50073a00 2024-02-28T23:34:39.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:39 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:39.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:39 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2635038183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:34:39.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.655+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca5004f2a0 con 0x7fca50073a00 2024-02-28T23:34:39.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.656+0000 7fca4cc00700 1 -- 172.21.15.57:0/1595040816 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fca3c059ff0 con 0x7fca50073a00 2024-02-28T23:34:39.654 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:34:39.654 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:34:39.654 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:34:39.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:34:39.654 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:34:39.655 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:34:39.656 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:34:39.656 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:34:39.656 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:34:39.656 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 msgr2=0x7fca3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 0x7fca3406fd50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca4001a040 comp rx=0 tx=0).stop 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 msgr2=0x7fca5019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00b4b0 tx=0x7fca3c00b870 comp rx=0 tx=0).stop 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 shutdown_connections 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fca3406d8a0 0x7fca3406fd50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca500730f0 0x7fca5019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 --2- 172.21.15.57:0/1595040816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca50073a00 0x7fca5019ca70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 >> 172.21.15.57:0/1595040816 conn(0x7fca500fbda0 msgr2=0x7fca50102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:34:39.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 shutdown_connections 2024-02-28T23:34:39.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:34:39.659+0000 7fca56a00700 1 -- 172.21.15.57:0/1595040816 wait complete. 2024-02-28T23:34:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:39 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:34:39.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:39 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2635038183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:34:40.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:40 smithi057 ceph-mon[89491]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:40.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:40 smithi057 ceph-mon[89491]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:40.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:40 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1595040816' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:34:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:40 smithi084 ceph-mon[70440]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:34:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:40 smithi084 ceph-mon[70440]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:40 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1595040816' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:34:42.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:42 smithi057 ceph-mon[89491]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:42.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:42 smithi084 ceph-mon[70440]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:44 smithi057 ceph-mon[89491]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:44.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:44 smithi084 ceph-mon[70440]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:46.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:46 smithi084 ceph-mon[70440]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:46 smithi057 ceph-mon[89491]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:48 smithi057 ceph-mon[89491]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:48 smithi084 ceph-mon[70440]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:34:50.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:50 smithi057 ceph-mon[89491]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:51.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:50 smithi084 ceph-mon[70440]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:52 smithi057 ceph-mon[89491]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:53.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:52 smithi084 ceph-mon[70440]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:54 smithi057 ceph-mon[89491]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:55.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:54 smithi084 ceph-mon[70440]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:57.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:56 smithi057 ceph-mon[89491]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:57.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:56 smithi084 ceph-mon[70440]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:34:58 smithi057 ceph-mon[89491]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:34:59.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:34:58 smithi084 ceph-mon[70440]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:00.904 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:00 smithi057 ceph-mon[89491]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:00 smithi084 ceph-mon[70440]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:03.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:02 smithi057 ceph-mon[89491]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:03.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:02 smithi084 ceph-mon[70440]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:03.992 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:05.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:04 smithi057 ceph-mon[89491]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:05.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:04 smithi084 ceph-mon[70440]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:06 smithi057 ceph-mon[89491]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:07.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:06 smithi084 ceph-mon[70440]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:08 smithi057 ceph-mon[89491]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:09.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:08 smithi084 ceph-mon[70440]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:09.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 -- 172.21.15.57:0/2228667772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac108470 msgr2=0x7facac108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 --2- 172.21.15.57:0/2228667772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac108470 0x7facac108840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00b0a0 tx=0x7fac9c00b3b0 comp rx=0 tx=0).stop 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 -- 172.21.15.57:0/2228667772 shutdown_connections 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 --2- 172.21.15.57:0/2228667772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac102470 0x7facac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 --2- 172.21.15.57:0/2228667772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac108470 0x7facac108840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 -- 172.21.15.57:0/2228667772 >> 172.21.15.57:0/2228667772 conn(0x7facac0fdf90 msgr2=0x7facac1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 -- 172.21.15.57:0/2228667772 shutdown_connections 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.833+0000 7facb1600700 1 -- 172.21.15.57:0/2228667772 wait complete. 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 Processor -- start 2024-02-28T23:35:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 -- start start 2024-02-28T23:35:09.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 0x7facac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac198e50 con 0x7facac102470 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.835+0000 7facb1600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac19cbe0 con 0x7facac108470 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32884/0 (socket says 172.21.15.57:32884) 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 -- 172.21.15.57:0/1419338274 learned_addr learned my addr 172.21.15.57:0/1419338274 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7faca2000700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 0x7facac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 -- 172.21.15.57:0/1419338274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 msgr2=0x7facac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 0x7facac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7facaaa00700 1 -- 172.21.15.57:0/1419338274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c009d00 con 0x7facac102470 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.836+0000 7faca2000700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 0x7facac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:35:09.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7facaaa00700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00b070 tx=0x7fac9c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c00e040 con 0x7facac102470 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac9c003de0 con 0x7facac102470 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c01d8e0 con 0x7facac102470 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac19ce60 con 0x7facac102470 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.837+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac19d350 con 0x7facac102470 2024-02-28T23:35:09.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.838+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac10a9e0 con 0x7facac102470 2024-02-28T23:35:09.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.839+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fac9c004040 con 0x7facac102470 2024-02-28T23:35:09.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.840+0000 7faca3e00700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 0x7fac9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:09.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.840+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fac9c08ed10 con 0x7facac102470 2024-02-28T23:35:09.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.841+0000 7faca2000700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 0x7fac9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:09.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.842+0000 7faca2000700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 0x7fac9006fd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac94005fd0 tx=0x7fac94005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:09.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:09.848+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac9c0584a0 con 0x7facac102470 2024-02-28T23:35:10.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.097+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facac1996a0 con 0x7fac9006d8a0 2024-02-28T23:35:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.099+0000 7faca3e00700 1 -- 172.21.15.57:0/1419338274 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7facac1996a0 con 0x7fac9006d8a0 2024-02-28T23:35:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 msgr2=0x7fac9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 0x7fac9006fd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac94005fd0 tx=0x7fac94005ee0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 msgr2=0x7facac198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00b070 tx=0x7fac9c00cdd0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 shutdown_connections 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fac9006d8a0 0x7fac9006fd50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7facac102470 0x7facac198230 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 --2- 172.21.15.57:0/1419338274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac108470 0x7facac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.102+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 >> 172.21.15.57:0/1419338274 conn(0x7facac0fdf90 msgr2=0x7facac100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:10.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.103+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 shutdown_connections 2024-02-28T23:35:10.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.103+0000 7facb1600700 1 -- 172.21.15.57:0/1419338274 wait complete. 2024-02-28T23:35:10.119 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:35:10.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 -- 172.21.15.57:0/1988733423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 msgr2=0x7fb3cc103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/1988733423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc103e50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009940 tx=0x7fb3b4009c50 comp rx=0 tx=0).stop 2024-02-28T23:35:10.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 -- 172.21.15.57:0/1988733423 shutdown_connections 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/1988733423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc103e50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/1988733423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 0x7fb3cc0ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 -- 172.21.15.57:0/1988733423 >> 172.21.15.57:0/1988733423 conn(0x7fb3cc0faad0 msgr2=0x7fb3cc0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 -- 172.21.15.57:0/1988733423 shutdown_connections 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.272+0000 7fb3cbe00700 1 -- 172.21.15.57:0/1988733423 wait complete. 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.273+0000 7fb3cbe00700 1 Processor -- start 2024-02-28T23:35:10.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3cbe00700 1 -- start start 2024-02-28T23:35:10.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 0x7fb3cc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3cbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3cbe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19cfe0 con 0x7fb3cc0ff710 2024-02-28T23:35:10.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3cbe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19d120 con 0x7fb3cc0fee00 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.274+0000 7fb3ca000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32910/0 (socket says 172.21.15.57:32910) 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 -- 172.21.15.57:0/2818173652 learned_addr learned my addr 172.21.15.57:0/2818173652 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3caa00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 0x7fb3cc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 -- 172.21.15.57:0/2818173652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 msgr2=0x7fb3cc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 0x7fb3cc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 -- 172.21.15.57:0/2818173652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40095f0 con 0x7fb3cc0ff710 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.275+0000 7fb3ca000700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400b870 tx=0x7fb3b40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.276+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b401d070 con 0x7fb3cc0ff710 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.276+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3b400f460 con 0x7fb3cc0ff710 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.276+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b4021640 con 0x7fb3cc0ff710 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.276+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc1a0f10 con 0x7fb3cc0ff710 2024-02-28T23:35:10.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.276+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1a1400 con 0x7fb3cc0ff710 2024-02-28T23:35:10.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.277+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc1015b0 con 0x7fb3cc0ff710 2024-02-28T23:35:10.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.278+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb3b400fa40 con 0x7fb3cc0ff710 2024-02-28T23:35:10.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.279+0000 7fb3c3400700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 0x7fb3b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.279+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb3b408e800 con 0x7fb3cc0ff710 2024-02-28T23:35:10.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.279+0000 7fb3caa00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 0x7fb3b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.280+0000 7fb3caa00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 0x7fb3b006fd50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e270 tx=0x7fb3bc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:10.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.284+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3b4057f10 con 0x7fb3cc0ff710 2024-02-28T23:35:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.540+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3cc19d870 con 0x7fb3b006d8a0 2024-02-28T23:35:10.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.543+0000 7fb3c3400700 1 -- 172.21.15.57:0/2818173652 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb3cc19d870 con 0x7fb3b006d8a0 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.546+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 msgr2=0x7fb3b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.546+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 0x7fb3b006fd50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e270 tx=0x7fb3bc009040 comp rx=0 tx=0).stop 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.546+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 msgr2=0x7fb3cc19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.546+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400b870 tx=0x7fb3b40056a0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.546+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 shutdown_connections 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3b006d8a0 0x7fb3b006fd50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3cc0fee00 0x7fb3cc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 --2- 172.21.15.57:0/2818173652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3cc0ff710 0x7fb3cc19c990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 >> 172.21.15.57:0/2818173652 conn(0x7fb3cc0faad0 msgr2=0x7fb3cc1104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:10.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 shutdown_connections 2024-02-28T23:35:10.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.547+0000 7fb3cbe00700 1 -- 172.21.15.57:0/2818173652 wait complete. 2024-02-28T23:35:10.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 -- 172.21.15.57:0/3219576004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ffb70 msgr2=0x7ff6f4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 --2- 172.21.15.57:0/3219576004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ffb70 0x7ff6f4103b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009910 tx=0x7ff6e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 -- 172.21.15.57:0/3219576004 shutdown_connections 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 --2- 172.21.15.57:0/3219576004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ffb70 0x7ff6f4103b40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 --2- 172.21.15.57:0/3219576004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f40ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 -- 172.21.15.57:0/3219576004 >> 172.21.15.57:0/3219576004 conn(0x7ff6f40faa90 msgr2=0x7ff6f40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 -- 172.21.15.57:0/3219576004 shutdown_connections 2024-02-28T23:35:10.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.707+0000 7ff6f3200700 1 -- 172.21.15.57:0/3219576004 wait complete. 2024-02-28T23:35:10.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 Processor -- start 2024-02-28T23:35:10.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 -- start start 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ffb70 0x7ff6f419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419c610 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.709+0000 7ff6f3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4195ac0 con 0x7ff6f40ffb70 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32932/0 (socket says 172.21.15.57:32932) 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 -- 172.21.15.57:0/1073178667 learned_addr learned my addr 172.21.15.57:0/1073178667 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1400700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ffb70 0x7ff6f419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 -- 172.21.15.57:0/1073178667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ffb70 msgr2=0x7ff6f419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ffb70 0x7ff6f419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 -- 172.21.15.57:0/1073178667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00095f0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.710+0000 7ff6f1e00700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400d8d0 tx=0x7ff6e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:10.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.711+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e4009940 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.711+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6e4010460 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.711+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e400f5f0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.711+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4195da0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.711+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f41962f0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.712+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f4062380 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.713+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff6e40105d0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.714+0000 7ff6eaa00700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 0x7ff6d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:10.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.714+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff6e408d510 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.714+0000 7ff6f1400700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 0x7ff6d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:10.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.715+0000 7ff6f1400700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 0x7ff6d806fd50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40fb120 tx=0x7ff6e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:10.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.719+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6e405a7d0 con 0x7ff6f40ff1d0 2024-02-28T23:35:10.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:10 smithi057 ceph-mon[89491]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:10.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.956+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f4196690 con 0x7ff6d806d8a0 2024-02-28T23:35:10.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.988+0000 7ff6eaa00700 1 -- 172.21.15.57:0/1073178667 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff6f4196690 con 0x7ff6d806d8a0 2024-02-28T23:35:10.987 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:35:10.987 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (24m) 32s ago 38m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:35:10.987 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (34m) 32s ago 39m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:35:10.987 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (37m) 35s ago 37m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (20m) 32s ago 38m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (20m) 35s ago 37m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (24m) 32s ago 38m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (16m) 32s ago 31m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (16m) 32s ago 31m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (41s) 35s ago 31m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (15m) 35s ago 31m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (27m) 32s ago 40m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (26m) 35s ago 37m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (22m) 32s ago 40m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (21m) 35s ago 37m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (25m) 32s ago 38m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:35:10.988 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (25m) 35s ago 37m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (20m) 32s ago 36m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (19m) 32s ago 35m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (19m) 32s ago 34m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (18m) 35s ago 34m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (17m) 35s ago 33m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (17m) 35s ago 33m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:35:10.989 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (25m) 32s ago 37m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:35:10.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 msgr2=0x7ff6d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 0x7ff6d806fd50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40fb120 tx=0x7ff6e001a040 comp rx=0 tx=0).stop 2024-02-28T23:35:10.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 msgr2=0x7ff6f419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400d8d0 tx=0x7ff6e400dc90 comp rx=0 tx=0).stop 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 shutdown_connections 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff6d806d8a0 0x7ff6d806fd50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f40ff1d0 0x7ff6f419ba40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 --2- 172.21.15.57:0/1073178667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6f40ffb70 0x7ff6f419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.991+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 >> 172.21.15.57:0/1073178667 conn(0x7ff6f40faa90 msgr2=0x7ff6f40fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.992+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 shutdown_connections 2024-02-28T23:35:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:10.992+0000 7ff6f3200700 1 -- 172.21.15.57:0/1073178667 wait complete. 2024-02-28T23:35:11.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:10 smithi084 ceph-mon[70440]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 -- 172.21.15.57:0/4058465097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0102460 msgr2=0x7fdad01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 --2- 172.21.15.57:0/4058465097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0102460 0x7fdad01028d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009910 tx=0x7fdac0009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 -- 172.21.15.57:0/4058465097 shutdown_connections 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 --2- 172.21.15.57:0/4058465097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0102460 0x7fdad01028d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 --2- 172.21.15.57:0/4058465097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0108460 0x7fdad0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.145+0000 7fdad8800700 1 -- 172.21.15.57:0/4058465097 >> 172.21.15.57:0/4058465097 conn(0x7fdad00fdf60 msgr2=0x7fdad0100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.146+0000 7fdad8800700 1 -- 172.21.15.57:0/4058465097 shutdown_connections 2024-02-28T23:35:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.146+0000 7fdad8800700 1 -- 172.21.15.57:0/4058465097 wait complete. 2024-02-28T23:35:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 Processor -- start 2024-02-28T23:35:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 -- start start 2024-02-28T23:35:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 0x7fdad01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0198df0 con 0x7fdad0108460 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad8800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad019cb80 con 0x7fdad0102460 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32942/0 (socket says 172.21.15.57:32942) 2024-02-28T23:35:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.147+0000 7fdad5400700 1 -- 172.21.15.57:0/2461653262 learned_addr learned my addr 172.21.15.57:0/2461653262 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5e00700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 0x7fdad01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5400700 1 -- 172.21.15.57:0/2461653262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 msgr2=0x7fdad01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5400700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 0x7fdad01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5400700 1 -- 172.21.15.57:0/2461653262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac00095f0 con 0x7fdad0108460 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5e00700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 0x7fdad01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.148+0000 7fdad5400700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006010 tx=0x7fdac000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.149+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac001d070 con 0x7fdad0108460 2024-02-28T23:35:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.149+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdac0004fb0 con 0x7fdad0108460 2024-02-28T23:35:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.149+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac0022ea0 con 0x7fdad0108460 2024-02-28T23:35:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.149+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad019ce00 con 0x7fdad0108460 2024-02-28T23:35:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.149+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad019d2f0 con 0x7fdad0108460 2024-02-28T23:35:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.150+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad010a9f0 con 0x7fdad0108460 2024-02-28T23:35:11.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.151+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdac0022470 con 0x7fdad0108460 2024-02-28T23:35:11.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.152+0000 7fdacaa00700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 0x7fdab806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.152+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdac008ea50 con 0x7fdad0108460 2024-02-28T23:35:11.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.153+0000 7fdad5e00700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 0x7fdab806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.153+0000 7fdad5e00700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 0x7fdab806fd50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007900 tx=0x7fdac4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:11.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.157+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdac0058290 con 0x7fdad0108460 2024-02-28T23:35:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.468+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdad004f2a0 con 0x7fdad0108460 2024-02-28T23:35:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.470+0000 7fdacaa00700 1 -- 172.21.15.57:0/2461653262 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdac005b950 con 0x7fdad0108460 2024-02-28T23:35:11.469 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:35:11.469 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:35:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:35:11.471 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:35:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 msgr2=0x7fdab806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 0x7fdab806fd50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007900 tx=0x7fdac4008040 comp rx=0 tx=0).stop 2024-02-28T23:35:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 msgr2=0x7fdad0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006010 tx=0x7fdac000be20 comp rx=0 tx=0).stop 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 shutdown_connections 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdab806d8a0 0x7fdab806fd50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdad0102460 0x7fdad01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 --2- 172.21.15.57:0/2461653262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdad0108460 0x7fdad0198710 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.473+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 >> 172.21.15.57:0/2461653262 conn(0x7fdad00fdf60 msgr2=0x7fdad00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.474+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 shutdown_connections 2024-02-28T23:35:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.474+0000 7fdad8800700 1 -- 172.21.15.57:0/2461653262 wait complete. 2024-02-28T23:35:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 -- 172.21.15.57:0/1592625374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a4069140 msgr2=0x7f47a4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 --2- 172.21.15.57:0/1592625374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a4069140 0x7f47a4104eb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4790009910 tx=0x7f4790009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 -- 172.21.15.57:0/1592625374 shutdown_connections 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 --2- 172.21.15.57:0/1592625374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a4069140 0x7f47a4104eb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 --2- 172.21.15.57:0/1592625374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a40687a0 0x7f47a4068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.621+0000 7f47ac400700 1 -- 172.21.15.57:0/1592625374 >> 172.21.15.57:0/1592625374 conn(0x7f47a4075130 msgr2=0x7f47a4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.622+0000 7f47ac400700 1 -- 172.21.15.57:0/1592625374 shutdown_connections 2024-02-28T23:35:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.622+0000 7f47ac400700 1 -- 172.21.15.57:0/1592625374 wait complete. 2024-02-28T23:35:11.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 Processor -- start 2024-02-28T23:35:11.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 -- start start 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a4069140 0x7f47a4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a4198bd0 con 0x7f47a40687a0 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.623+0000 7f47ac400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a4198d10 con 0x7f47a4069140 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32958/0 (socket says 172.21.15.57:32958) 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 -- 172.21.15.57:0/2746160824 learned_addr learned my addr 172.21.15.57:0/2746160824 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9000700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a4069140 0x7f47a4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 -- 172.21.15.57:0/2746160824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a4069140 msgr2=0x7f47a4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a4069140 0x7f47a4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 -- 172.21.15.57:0/2746160824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900095f0 con 0x7f47a40687a0 2024-02-28T23:35:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.624+0000 7f47a9a00700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f479800c910 tx=0x7f479800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.625+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47980041f0 con 0x7f47a40687a0 2024-02-28T23:35:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.625+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4798007dc0 con 0x7f47a40687a0 2024-02-28T23:35:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.625+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4798003e70 con 0x7f47a40687a0 2024-02-28T23:35:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.625+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a419cb60 con 0x7f47a40687a0 2024-02-28T23:35:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.625+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a419d0b0 con 0x7f47a40687a0 2024-02-28T23:35:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.626+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a404f2a0 con 0x7f47a40687a0 2024-02-28T23:35:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.627+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4798004350 con 0x7f47a40687a0 2024-02-28T23:35:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.628+0000 7f47a2000700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 0x7f478c06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.628+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f479808d350 con 0x7f47a40687a0 2024-02-28T23:35:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.629+0000 7f47a9000700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 0x7f478c06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.629+0000 7f47a9000700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 0x7f478c06fd00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4790005f50 tx=0x7f479001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:11.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.633+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4798056a60 con 0x7f47a40687a0 2024-02-28T23:35:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:11 smithi057 ceph-mon[89491]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:11 smithi057 ceph-mon[89491]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:11 smithi057 ceph-mon[89491]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:11.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:11 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2461653262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:35:11.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.904+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47a419d3d0 con 0x7f47a40687a0 2024-02-28T23:35:11.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.905+0000 7f47a2000700 1 -- 172.21.15.57:0/2746160824 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f479805a120 con 0x7f47a40687a0 2024-02-28T23:35:11.905 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:35:11.905 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:35:11.905 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:35:11.905 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:35:11.905 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:35:11.906 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:35:11.907 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 msgr2=0x7f478c06fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 0x7f478c06fd00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4790005f50 tx=0x7f479001a040 comp rx=0 tx=0).stop 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 msgr2=0x7f47a4198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f479800c910 tx=0x7f479800ccd0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 shutdown_connections 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f478c06d850 0x7f478c06fd00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a40687a0 0x7f47a4198040 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 --2- 172.21.15.57:0/2746160824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a4069140 0x7f47a4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:11.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.909+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 >> 172.21.15.57:0/2746160824 conn(0x7f47a4075130 msgr2=0x7f47a40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:11.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.910+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 shutdown_connections 2024-02-28T23:35:11.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:11.910+0000 7f47ac400700 1 -- 172.21.15.57:0/2746160824 wait complete. 2024-02-28T23:35:11.910 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:35:12.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 -- 172.21.15.57:0/3062411555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 msgr2=0x7ff63010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 --2- 172.21.15.57:0/3062411555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff63010c580 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009910 tx=0x7ff61c009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:12.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 -- 172.21.15.57:0/3062411555 shutdown_connections 2024-02-28T23:35:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 --2- 172.21.15.57:0/3062411555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff63010c580 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 --2- 172.21.15.57:0/3062411555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 0x7ff6300734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 -- 172.21.15.57:0/3062411555 >> 172.21.15.57:0/3062411555 conn(0x7ff6300fbda0 msgr2=0x7ff6300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 -- 172.21.15.57:0/3062411555 shutdown_connections 2024-02-28T23:35:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.061+0000 7ff635000700 1 -- 172.21.15.57:0/3062411555 wait complete. 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.062+0000 7ff635000700 1 Processor -- start 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.062+0000 7ff635000700 1 -- start start 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff635000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 0x7ff630197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff635000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff635000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63019c930 con 0x7ff630073a00 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff635000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63019caa0 con 0x7ff6300730f0 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32970/0 (socket says 172.21.15.57:32970) 2024-02-28T23:35:12.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 -- 172.21.15.57:0/806354099 learned_addr learned my addr 172.21.15.57:0/806354099 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:12.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62de00700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 0x7ff630197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 -- 172.21.15.57:0/806354099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 msgr2=0x7ff630197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 0x7ff630197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.063+0000 7ff62d400700 1 -- 172.21.15.57:0/806354099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0095f0 con 0x7ff630073a00 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff62d400700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b5c0 tx=0x7ff61c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff61c01d070 con 0x7ff630073a00 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff61c005e00 con 0x7ff630073a00 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63019cd80 con 0x7ff630073a00 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff61c017400 con 0x7ff630073a00 2024-02-28T23:35:12.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.064+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63019d2d0 con 0x7ff630073a00 2024-02-28T23:35:12.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.065+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff630109c80 con 0x7ff630073a00 2024-02-28T23:35:12.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.067+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff61c00f460 con 0x7ff630073a00 2024-02-28T23:35:12.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.067+0000 7ff626a00700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 0x7ff61406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.068+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff61c08ea90 con 0x7ff630073a00 2024-02-28T23:35:12.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.068+0000 7ff62de00700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 0x7ff61406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.068+0000 7ff62de00700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 0x7ff61406fd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff620005950 tx=0x7ff6200058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:12.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.072+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff61c0581a0 con 0x7ff630073a00 2024-02-28T23:35:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:11 smithi084 ceph-mon[70440]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:11 smithi084 ceph-mon[70440]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:11 smithi084 ceph-mon[70440]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:12.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:11 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2461653262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:35:12.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.310+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff630199350 con 0x7ff61406d8a0 2024-02-28T23:35:12.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.312+0000 7ff626a00700 1 -- 172.21.15.57:0/806354099 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff630199350 con 0x7ff61406d8a0 2024-02-28T23:35:12.311 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:35:12.311 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:35:12.311 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:35:12.312 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.314+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 msgr2=0x7ff61406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.314+0000 7ff635000700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 0x7ff61406fd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff620005950 tx=0x7ff6200058e0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.314+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 msgr2=0x7ff6301985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.314+0000 7ff635000700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b5c0 tx=0x7ff61c00be20 comp rx=0 tx=0).stop 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 shutdown_connections 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff61406d8a0 0x7ff61406fd50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6300730f0 0x7ff630197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 --2- 172.21.15.57:0/806354099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff630073a00 0x7ff6301985f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 >> 172.21.15.57:0/806354099 conn(0x7ff6300fbda0 msgr2=0x7ff630106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 shutdown_connections 2024-02-28T23:35:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.315+0000 7ff635000700 1 -- 172.21.15.57:0/806354099 wait complete. 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.478+0000 7fe6ab400700 1 -- 172.21.15.57:0/3689521209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a4100e80 msgr2=0x7fe6a4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.478+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3689521209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a4100e80 0x7fe6a4104e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe690009910 tx=0x7fe690009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 -- 172.21.15.57:0/3689521209 shutdown_connections 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3689521209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a4100e80 0x7fe6a4104e50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3689521209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a41004e0 0x7fe6a41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 -- 172.21.15.57:0/3689521209 >> 172.21.15.57:0/3689521209 conn(0x7fe6a40fbda0 msgr2=0x7fe6a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 -- 172.21.15.57:0/3689521209 shutdown_connections 2024-02-28T23:35:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.479+0000 7fe6ab400700 1 -- 172.21.15.57:0/3689521209 wait complete. 2024-02-28T23:35:12.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.480+0000 7fe6ab400700 1 Processor -- start 2024-02-28T23:35:12.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.480+0000 7fe6ab400700 1 -- start start 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.480+0000 7fe6ab400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6ab400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 0x7fe6a4193120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6ab400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4196d70 con 0x7fe6a41004e0 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6ab400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4193660 con 0x7fe6a4100e80 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32990/0 (socket says 172.21.15.57:32990) 2024-02-28T23:35:12.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 -- 172.21.15.57:0/3446791926 learned_addr learned my addr 172.21.15.57:0/3446791926 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a3e00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 0x7fe6a4193120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 -- 172.21.15.57:0/3446791926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 msgr2=0x7fe6a4193120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 0x7fe6a4193120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a8a00700 1 -- 172.21.15.57:0/3446791926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900095f0 con 0x7fe6a41004e0 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.481+0000 7fe6a3e00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 0x7fe6a4193120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:35:12.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6a8a00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe69800d8d0 tx=0x7fe69800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe698009940 con 0x7fe6a41004e0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe698010460 con 0x7fe6a41004e0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a4193940 con 0x7fe6a41004e0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69800f5f0 con 0x7fe6a41004e0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.482+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41a6920 con 0x7fe6a41004e0 2024-02-28T23:35:12.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.483+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a41045d0 con 0x7fe6a41004e0 2024-02-28T23:35:12.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.485+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe6980105d0 con 0x7fe6a41004e0 2024-02-28T23:35:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.486+0000 7fe6a1600700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 0x7fe68c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.486+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe69808cf20 con 0x7fe6a41004e0 2024-02-28T23:35:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.486+0000 7fe6a3e00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 0x7fe68c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.487+0000 7fe6a3e00700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 0x7fe68c06fd50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe690006010 tx=0x7fe69000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:12.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.490+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6980566b0 con 0x7fe6a41004e0 2024-02-28T23:35:12.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.812+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe6a404f2a0 con 0x7fe6a41004e0 2024-02-28T23:35:12.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.813+0000 7fe6a1600700 1 -- 172.21.15.57:0/3446791926 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fe698059d70 con 0x7fe6a41004e0 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:35:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:35:12.814 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:35:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 msgr2=0x7fe68c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 0x7fe68c06fd50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe690006010 tx=0x7fe69000b540 comp rx=0 tx=0).stop 2024-02-28T23:35:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 msgr2=0x7fe6a4192be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe69800d8d0 tx=0x7fe69800dc90 comp rx=0 tx=0).stop 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 shutdown_connections 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe68c06d8a0 0x7fe68c06fd50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6a41004e0 0x7fe6a4192be0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 --2- 172.21.15.57:0/3446791926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a4100e80 0x7fe6a4193120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.816+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 >> 172.21.15.57:0/3446791926 conn(0x7fe6a40fbda0 msgr2=0x7fe6a40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.817+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 shutdown_connections 2024-02-28T23:35:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:12.817+0000 7fe6ab400700 1 -- 172.21.15.57:0/3446791926 wait complete. 2024-02-28T23:35:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:12 smithi057 ceph-mon[89491]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:12 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2746160824' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:35:13.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:12 smithi084 ceph-mon[70440]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:13.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:12 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2746160824' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:35:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:13 smithi057 ceph-mon[89491]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:14.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:13 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3446791926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:35:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:13 smithi084 ceph-mon[70440]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:14.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:13 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3446791926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:35:15.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:14 smithi057 ceph-mon[89491]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:15.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:14 smithi084 ceph-mon[70440]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:16 smithi057 ceph-mon[89491]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:17.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:16 smithi084 ceph-mon[70440]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:18 smithi057 ceph-mon[89491]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:18 smithi084 ceph-mon[70440]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:21.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:20 smithi057 ceph-mon[89491]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:21.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:20 smithi084 ceph-mon[70440]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:22 smithi057 ceph-mon[89491]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:23.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:22 smithi084 ceph-mon[70440]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:25.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:24 smithi057 ceph-mon[89491]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:25.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:24 smithi084 ceph-mon[70440]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:27.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:26 smithi057 ceph-mon[89491]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:27.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:26 smithi084 ceph-mon[70440]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:29.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:28 smithi057 ceph-mon[89491]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:29.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:28 smithi084 ceph-mon[70440]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:31.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:30 smithi057 ceph-mon[89491]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:31.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:30 smithi084 ceph-mon[70440]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:33.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:32 smithi057 ceph-mon[89491]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:33.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:32 smithi084 ceph-mon[70440]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:34 smithi057 ceph-mon[89491]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:34 smithi084 ceph-mon[70440]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:36 smithi057 ceph-mon[89491]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:36 smithi084 ceph-mon[70440]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:39.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:38 smithi057 ceph-mon[89491]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:39.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:38 smithi084 ceph-mon[70440]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:40 smithi057 ceph-mon[89491]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:40 smithi084 ceph-mon[70440]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:42.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 -- 172.21.15.57:0/3913423137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0103e50 msgr2=0x7fdcd01042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:42.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/3913423137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0103e50 0x7fdcd01042c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009910 tx=0x7fdcb8009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:42.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 -- 172.21.15.57:0/3913423137 shutdown_connections 2024-02-28T23:35:42.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/3913423137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd0103e50 0x7fdcd01042c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:42.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/3913423137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01025f0 0x7fdcd01029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:42.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 -- 172.21.15.57:0/3913423137 >> 172.21.15.57:0/3913423137 conn(0x7fdcd00fddb0 msgr2=0x7fdcd01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:42.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 -- 172.21.15.57:0/3913423137 shutdown_connections 2024-02-28T23:35:42.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.992+0000 7fdcd4e00700 1 -- 172.21.15.57:0/3913423137 wait complete. 2024-02-28T23:35:42.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 Processor -- start 2024-02-28T23:35:42.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 -- start start 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0103e50 0x7fdcd0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0198c00 con 0x7fdcd01025f0 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.994+0000 7fdcd4e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019c990 con 0x7fdcd0103e50 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59360/0 (socket says 172.21.15.57:59360) 2024-02-28T23:35:42.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 -- 172.21.15.57:0/2638607805 learned_addr learned my addr 172.21.15.57:0/2638607805 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdcc7e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0103e50 0x7fdcd0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 -- 172.21.15.57:0/2638607805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0103e50 msgr2=0x7fdcd0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0103e50 0x7fdcd0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 -- 172.21.15.57:0/2638607805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80095f0 con 0x7fdcd01025f0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.995+0000 7fdccf400700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000b4b0 tx=0x7fdcc000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.996+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc0011800 con 0x7fdcd01025f0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.996+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcc0011e20 con 0x7fdcd01025f0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.996+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd019cc70 con 0x7fdcd01025f0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.996+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc000f2e0 con 0x7fdcd01025f0 2024-02-28T23:35:42.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.996+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd019d240 con 0x7fdcd01025f0 2024-02-28T23:35:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.997+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd010a8b0 con 0x7fdcd01025f0 2024-02-28T23:35:42.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.998+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdcc0010380 con 0x7fdcd01025f0 2024-02-28T23:35:42.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.999+0000 7fdcccc00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:42.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.999+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdcc008d0f0 con 0x7fdcd01025f0 2024-02-28T23:35:42.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:42.999+0000 7fdcc7e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:42.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.000+0000 7fdcc7e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0199600 tx=0x7fdcb801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.004+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdcc0056930 con 0x7fdcd01025f0 2024-02-28T23:35:43.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:42 smithi084 ceph-mon[70440]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:43.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.250+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcd019d510 con 0x7fdcb406d8a0 2024-02-28T23:35:43.249 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:42 smithi057 ceph-mon[89491]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:43.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.253+0000 7fdcccc00700 1 -- 172.21.15.57:0/2638607805 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdcd019d510 con 0x7fdcb406d8a0 2024-02-28T23:35:43.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 msgr2=0x7fdcb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0199600 tx=0x7fdcb801a040 comp rx=0 tx=0).stop 2024-02-28T23:35:43.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 msgr2=0x7fdcd0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000b4b0 tx=0x7fdcc000b870 comp rx=0 tx=0).stop 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 shutdown_connections 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdcb406d8a0 0x7fdcb406fd50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcd01025f0 0x7fdcd0197fe0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 --2- 172.21.15.57:0/2638607805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd0103e50 0x7fdcd0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 >> 172.21.15.57:0/2638607805 conn(0x7fdcd00fddb0 msgr2=0x7fdcd00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 shutdown_connections 2024-02-28T23:35:43.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.257+0000 7fdcd4e00700 1 -- 172.21.15.57:0/2638607805 wait complete. 2024-02-28T23:35:43.272 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.423+0000 7f4954000700 1 -- 172.21.15.57:0/3424951091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c106b80 msgr2=0x7f494c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 --2- 172.21.15.57:0/3424951091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c106b80 0x7f494c106ff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4938009910 tx=0x7f4938009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 -- 172.21.15.57:0/3424951091 shutdown_connections 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 --2- 172.21.15.57:0/3424951091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c106b80 0x7f494c106ff0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 --2- 172.21.15.57:0/3424951091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c10c8f0 0x7f494c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 -- 172.21.15.57:0/3424951091 >> 172.21.15.57:0/3424951091 conn(0x7f494c076060 msgr2=0x7f494c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:43.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 -- 172.21.15.57:0/3424951091 shutdown_connections 2024-02-28T23:35:43.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.424+0000 7f4954000700 1 -- 172.21.15.57:0/3424951091 wait complete. 2024-02-28T23:35:43.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.425+0000 7f4954000700 1 Processor -- start 2024-02-28T23:35:43.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.425+0000 7f4954000700 1 -- start start 2024-02-28T23:35:43.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4954000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c106b80 0x7f494c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4954000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4954000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19cfc0 con 0x7f494c10c8f0 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4954000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1a0d50 con 0x7f494c106b80 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59368/0 (socket says 172.21.15.57:59368) 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 -- 172.21.15.57:0/2755663339 learned_addr learned my addr 172.21.15.57:0/2755663339 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4951600700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c106b80 0x7f494c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 -- 172.21.15.57:0/2755663339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c106b80 msgr2=0x7f494c19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c106b80 0x7f494c19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.426+0000 7f4950c00700 1 -- 172.21.15.57:0/2755663339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380095f0 con 0x7f494c10c8f0 2024-02-28T23:35:43.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f4950c00700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f493800b5c0 tx=0x7f493800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493801d070 con 0x7f494c10c8f0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4938005e00 con 0x7f494c10c8f0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4938017400 con 0x7f494c10c8f0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c1a0fd0 con 0x7f494c10c8f0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.427+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a14c0 con 0x7f494c10c8f0 2024-02-28T23:35:43.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.428+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c10ecc0 con 0x7f494c10c8f0 2024-02-28T23:35:43.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.429+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f493800f460 con 0x7f494c10c8f0 2024-02-28T23:35:43.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.430+0000 7f494a000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 0x7f493406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.430+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f493808ea10 con 0x7f494c10c8f0 2024-02-28T23:35:43.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.431+0000 7f4951600700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 0x7f493406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.432+0000 7f4951600700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 0x7f493406fd50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4940005950 tx=0x7f49400058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:43.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.435+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49380581a0 con 0x7f494c10c8f0 2024-02-28T23:35:43.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.691+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f494c02cca0 con 0x7f493406d8a0 2024-02-28T23:35:43.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.693+0000 7f494a000700 1 -- 172.21.15.57:0/2755663339 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f494c02cca0 con 0x7f493406d8a0 2024-02-28T23:35:43.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.696+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 msgr2=0x7f493406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.696+0000 7f4954000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 0x7f493406fd50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4940005950 tx=0x7f49400058e0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.696+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 msgr2=0x7f494c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.696+0000 7f4954000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f493800b5c0 tx=0x7f493800be20 comp rx=0 tx=0).stop 2024-02-28T23:35:43.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.696+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 shutdown_connections 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f493406d8a0 0x7f493406fd50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f494c106b80 0x7f494c19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 --2- 172.21.15.57:0/2755663339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494c10c8f0 0x7f494c19c8e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 >> 172.21.15.57:0/2755663339 conn(0x7f494c076060 msgr2=0x7f494c077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 shutdown_connections 2024-02-28T23:35:43.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.697+0000 7f4954000700 1 -- 172.21.15.57:0/2755663339 wait complete. 2024-02-28T23:35:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.861+0000 7fbed3e00700 1 -- 172.21.15.57:0/441992902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 msgr2=0x7fbecc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.861+0000 7fbed3e00700 1 --2- 172.21.15.57:0/441992902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc1008b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009910 tx=0x7fbec0009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 -- 172.21.15.57:0/441992902 shutdown_connections 2024-02-28T23:35:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 --2- 172.21.15.57:0/441992902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 0x7fbecc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 --2- 172.21.15.57:0/441992902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc1008b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 -- 172.21.15.57:0/441992902 >> 172.21.15.57:0/441992902 conn(0x7fbecc0fbda0 msgr2=0x7fbecc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:43.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 -- 172.21.15.57:0/441992902 shutdown_connections 2024-02-28T23:35:43.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.862+0000 7fbed3e00700 1 -- 172.21.15.57:0/441992902 wait complete. 2024-02-28T23:35:43.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.863+0000 7fbed3e00700 1 Processor -- start 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.863+0000 7fbed3e00700 1 -- start start 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.863+0000 7fbed3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 0x7fbecc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc073ad0 con 0x7fbecc1004e0 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc073c10 con 0x7fbecc100e80 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59394/0 (socket says 172.21.15.57:59394) 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 -- 172.21.15.57:0/3462639362 learned_addr learned my addr 172.21.15.57:0/3462639362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:43.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed0a00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 0x7fbecc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 -- 172.21.15.57:0/3462639362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 msgr2=0x7fbecc073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:43.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 0x7fbecc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:43.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.864+0000 7fbed1400700 1 -- 172.21.15.57:0/3462639362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00095f0 con 0x7fbecc1004e0 2024-02-28T23:35:43.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbed1400700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006010 tx=0x7fbec000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec001d070 con 0x7fbecc1004e0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbec000f460 con 0x7fbecc1004e0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc073e90 con 0x7fbecc1004e0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec0017630 con 0x7fbecc1004e0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.865+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc1a23f0 con 0x7fbecc1004e0 2024-02-28T23:35:43.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.866+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc1087f0 con 0x7fbecc1004e0 2024-02-28T23:35:43.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.867+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbec000f5d0 con 0x7fbecc1004e0 2024-02-28T23:35:43.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.869+0000 7fbeca000700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 0x7fbeb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:43.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.869+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbec008ea80 con 0x7fbecc1004e0 2024-02-28T23:35:43.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.869+0000 7fbed0a00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 0x7fbeb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:43.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.870+0000 7fbed0a00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 0x7fbeb406fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005950 tx=0x7fbeb80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:43.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:43.874+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbec0058190 con 0x7fbecc1004e0 2024-02-28T23:35:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.122+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbecc061190 con 0x7fbeb406d8a0 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.155+0000 7fbeca000700 1 -- 172.21.15.57:0/3462639362 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbecc061190 con 0x7fbeb406d8a0 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (25m) 65s ago 39m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (35m) 65s ago 39m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (38m) 68s ago 38m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:35:44.153 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (21m) 65s ago 39m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (21m) 68s ago 38m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (24m) 65s ago 38m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (17m) 65s ago 32m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (16m) 65s ago 31m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (74s) 68s ago 31m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (16m) 68s ago 31m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (27m) 65s ago 41m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (26m) 68s ago 37m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (23m) 65s ago 41m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (22m) 68s ago 37m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (26m) 65s ago 39m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (26m) 68s ago 37m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (20m) 65s ago 36m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (20m) 65s ago 36m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:35:44.154 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (19m) 65s ago 35m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:35:44.155 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (19m) 68s ago 34m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:35:44.155 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (18m) 68s ago 34m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:35:44.155 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (17m) 68s ago 33m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:35:44.155 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (25m) 65s ago 38m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 msgr2=0x7fbeb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 0x7fbeb406fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005950 tx=0x7fbeb80058e0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 msgr2=0x7fbecc073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006010 tx=0x7fbec000be20 comp rx=0 tx=0).stop 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 shutdown_connections 2024-02-28T23:35:44.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbeb406d8a0 0x7fbeb406fd50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbecc1004e0 0x7fbecc073050 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 --2- 172.21.15.57:0/3462639362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc100e80 0x7fbecc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 >> 172.21.15.57:0/3462639362 conn(0x7fbecc0fbda0 msgr2=0x7fbecc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:44.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 shutdown_connections 2024-02-28T23:35:44.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.158+0000 7fbed3e00700 1 -- 172.21.15.57:0/3462639362 wait complete. 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.324+0000 7fae52e00700 1 -- 172.21.15.57:0/1423781143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c101c40 msgr2=0x7fae4c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.324+0000 7fae52e00700 1 --2- 172.21.15.57:0/1423781143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c101c40 0x7fae4c1020b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fae38009910 tx=0x7fae38009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 -- 172.21.15.57:0/1423781143 shutdown_connections 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 --2- 172.21.15.57:0/1423781143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c101c40 0x7fae4c1020b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 --2- 172.21.15.57:0/1423781143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c1003e0 0x7fae4c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 -- 172.21.15.57:0/1423781143 >> 172.21.15.57:0/1423781143 conn(0x7fae4c0fbbc0 msgr2=0x7fae4c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 -- 172.21.15.57:0/1423781143 shutdown_connections 2024-02-28T23:35:44.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.325+0000 7fae52e00700 1 -- 172.21.15.57:0/1423781143 wait complete. 2024-02-28T23:35:44.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.326+0000 7fae52e00700 1 Processor -- start 2024-02-28T23:35:44.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.326+0000 7fae52e00700 1 -- start start 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae52e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae52e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c101c40 0x7fae4c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae52e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c198b30 con 0x7fae4c1003e0 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae52e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c19c8c0 con 0x7fae4c101c40 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59410/0 (socket says 172.21.15.57:59410) 2024-02-28T23:35:44.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 -- 172.21.15.57:0/2200981144 learned_addr learned my addr 172.21.15.57:0/2200981144 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51000700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c101c40 0x7fae4c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 -- 172.21.15.57:0/2200981144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c101c40 msgr2=0x7fae4c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c101c40 0x7fae4c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.327+0000 7fae51a00700 1 -- 172.21.15.57:0/2200981144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380095f0 con 0x7fae4c1003e0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae51a00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d8d0 tx=0x7fae4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae40009940 con 0x7fae4c1003e0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae40010460 con 0x7fae4c1003e0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c19cba0 con 0x7fae4c1003e0 2024-02-28T23:35:44.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae4000f5f0 con 0x7fae4c1003e0 2024-02-28T23:35:44.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.328+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c19d0f0 con 0x7fae4c1003e0 2024-02-28T23:35:44.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.329+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c062380 con 0x7fae4c1003e0 2024-02-28T23:35:44.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.331+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fae400105d0 con 0x7fae4c1003e0 2024-02-28T23:35:44.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.331+0000 7fae4a000700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 0x7fae3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.332+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fae4008d4c0 con 0x7fae4c1003e0 2024-02-28T23:35:44.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.332+0000 7fae51000700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 0x7fae3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.333+0000 7fae51000700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 0x7fae3406fd50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b5c0 tx=0x7fae38005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:44.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.337+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae40056c50 con 0x7fae4c1003e0 2024-02-28T23:35:44.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.649+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae4c199270 con 0x7fae4c1003e0 2024-02-28T23:35:44.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.650+0000 7fae4a000700 1 -- 172.21.15.57:0/2200981144 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fae4005a310 con 0x7fae4c1003e0 2024-02-28T23:35:44.648 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:35:44.649 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:35:44.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:35:44.650 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:35:44.650 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:35:44.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 msgr2=0x7fae3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 0x7fae3406fd50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b5c0 tx=0x7fae38005960 comp rx=0 tx=0).stop 2024-02-28T23:35:44.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 msgr2=0x7fae4c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d8d0 tx=0x7fae4000dc90 comp rx=0 tx=0).stop 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 shutdown_connections 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fae3406d8a0 0x7fae3406fd50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae4c1003e0 0x7fae4c197f10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 --2- 172.21.15.57:0/2200981144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae4c101c40 0x7fae4c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 >> 172.21.15.57:0/2200981144 conn(0x7fae4c0fbbc0 msgr2=0x7fae4c0fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.653+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 shutdown_connections 2024-02-28T23:35:44.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.654+0000 7fae52e00700 1 -- 172.21.15.57:0/2200981144 wait complete. 2024-02-28T23:35:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 -- 172.21.15.57:0/3388604655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 msgr2=0x7fb40c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 --2- 172.21.15.57:0/3388604655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c104e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009910 tx=0x7fb3f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 -- 172.21.15.57:0/3388604655 shutdown_connections 2024-02-28T23:35:44.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 --2- 172.21.15.57:0/3388604655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c104e50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 --2- 172.21.15.57:0/3388604655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 -- 172.21.15.57:0/3388604655 >> 172.21.15.57:0/3388604655 conn(0x7fb40c0fbda0 msgr2=0x7fb40c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 -- 172.21.15.57:0/3388604655 shutdown_connections 2024-02-28T23:35:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.816+0000 7fb411600700 1 -- 172.21.15.57:0/3388604655 wait complete. 2024-02-28T23:35:44.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.817+0000 7fb411600700 1 Processor -- start 2024-02-28T23:35:44.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb411600700 1 -- start start 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb411600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb411600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb411600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c198d00 con 0x7fb40c100e80 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb411600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c19ca90 con 0x7fb40c1004e0 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.818+0000 7fb402000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59430/0 (socket says 172.21.15.57:59430) 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 -- 172.21.15.57:0/140876103 learned_addr learned my addr 172.21.15.57:0/140876103 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:44.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb40aa00700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 -- 172.21.15.57:0/140876103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 msgr2=0x7fb40c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 -- 172.21.15.57:0/140876103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40095f0 con 0x7fb40c100e80 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb40aa00700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.819+0000 7fb402000700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400bda0 tx=0x7fb3f400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.820+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3f401d070 con 0x7fb40c100e80 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.820+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3f4004fb0 con 0x7fb40c100e80 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.820+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3f400f460 con 0x7fb40c100e80 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.820+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c19cd10 con 0x7fb40c100e80 2024-02-28T23:35:44.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.820+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c19d200 con 0x7fb40c100e80 2024-02-28T23:35:44.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.821+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c1045d0 con 0x7fb40c100e80 2024-02-28T23:35:44.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.822+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb3f4022800 con 0x7fb40c100e80 2024-02-28T23:35:44.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.823+0000 7fb403e00700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 0x7fb3f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:44.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.823+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb3f408eb00 con 0x7fb40c100e80 2024-02-28T23:35:44.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.823+0000 7fb40aa00700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 0x7fb3f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:44.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.824+0000 7fb40aa00700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 0x7fb3f006fd50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005fd0 tx=0x7fb3fc005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:44.828+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3f40583e0 con 0x7fb40c100e80 2024-02-28T23:35:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:44 smithi057 ceph-mon[89491]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:44 smithi057 ceph-mon[89491]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:44 smithi057 ceph-mon[89491]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:44 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2200981144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:35:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.105+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb40c04f2a0 con 0x7fb40c100e80 2024-02-28T23:35:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.105+0000 7fb403e00700 1 -- 172.21.15.57:0/140876103 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fb3f4027020 con 0x7fb40c100e80 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:45.104 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:35:45.105 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:35:45.106 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:45.107 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:35:45.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:35:45.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:45.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:45.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:45.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:35:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.108+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 msgr2=0x7fb3f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.108+0000 7fb411600700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 0x7fb3f006fd50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005fd0 tx=0x7fb3fc005dc0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.108+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 msgr2=0x7fb40c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.108+0000 7fb411600700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400bda0 tx=0x7fb3f400be80 comp rx=0 tx=0).stop 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 shutdown_connections 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb3f006d8a0 0x7fb3f006fd50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb40c1004e0 0x7fb40c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 --2- 172.21.15.57:0/140876103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb40c100e80 0x7fb40c198620 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 >> 172.21.15.57:0/140876103 conn(0x7fb40c0fbda0 msgr2=0x7fb40c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 shutdown_connections 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.109+0000 7fb411600700 1 -- 172.21.15.57:0/140876103 wait complete. 2024-02-28T23:35:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 -- 172.21.15.57:0/3220362556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 msgr2=0x7fc8400ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 --2- 172.21.15.57:0/3220362556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc8400ff720 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc830009910 tx=0x7fc830009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 -- 172.21.15.57:0/3220362556 shutdown_connections 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 --2- 172.21.15.57:0/3220362556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 0x7fc84010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 --2- 172.21.15.57:0/3220362556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc8400ff720 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 -- 172.21.15.57:0/3220362556 >> 172.21.15.57:0/3220362556 conn(0x7fc8400748b0 msgr2=0x7fc840074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 -- 172.21.15.57:0/3220362556 shutdown_connections 2024-02-28T23:35:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.262+0000 7fc845200700 1 -- 172.21.15.57:0/3220362556 wait complete. 2024-02-28T23:35:45.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.263+0000 7fc845200700 1 Processor -- start 2024-02-28T23:35:45.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.263+0000 7fc845200700 1 -- start start 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc845200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc845200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 0x7fc840198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc845200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc840198d20 con 0x7fc8400ff350 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc845200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019cab0 con 0x7fc8400ffc60 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc83de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc83de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47256/0 (socket says 172.21.15.57:47256) 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc83de00700 1 -- 172.21.15.57:0/954309023 learned_addr learned my addr 172.21.15.57:0/954309023 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.264+0000 7fc83d400700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 0x7fc840198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc83de00700 1 -- 172.21.15.57:0/954309023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 msgr2=0x7fc840198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc83de00700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 0x7fc840198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc83de00700 1 -- 172.21.15.57:0/954309023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300095f0 con 0x7fc8400ff350 2024-02-28T23:35:45.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc83de00700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc83000bda0 tx=0x7fc83000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:45.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83001d070 con 0x7fc8400ff350 2024-02-28T23:35:45.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84019cd30 con 0x7fc8400ff350 2024-02-28T23:35:45.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc830004fb0 con 0x7fc8400ff350 2024-02-28T23:35:45.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83000f460 con 0x7fc8400ff350 2024-02-28T23:35:45.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.265+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84019d220 con 0x7fc8400ff350 2024-02-28T23:35:45.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.266+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84010a230 con 0x7fc8400ff350 2024-02-28T23:35:45.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.268+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc830022800 con 0x7fc8400ff350 2024-02-28T23:35:45.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.268+0000 7fc836a00700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 0x7fc82406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.269+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc83008eb00 con 0x7fc8400ff350 2024-02-28T23:35:45.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.269+0000 7fc83d400700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 0x7fc82406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.269+0000 7fc83d400700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 0x7fc82406fd50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc82c006fd0 tx=0x7fc82c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:45.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.273+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8300583e0 con 0x7fc8400ff350 2024-02-28T23:35:45.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:44 smithi084 ceph-mon[70440]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:45.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:44 smithi084 ceph-mon[70440]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:44 smithi084 ceph-mon[70440]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:44 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2200981144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:35:45.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.525+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc84019d500 con 0x7fc82406d8a0 2024-02-28T23:35:45.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.527+0000 7fc836a00700 1 -- 172.21.15.57:0/954309023 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc84019d500 con 0x7fc82406d8a0 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:35:45.526 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:35:45.527 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 msgr2=0x7fc82406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 0x7fc82406fd50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc82c006fd0 tx=0x7fc82c008040 comp rx=0 tx=0).stop 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 msgr2=0x7fc840198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc83000bda0 tx=0x7fc83000be80 comp rx=0 tx=0).stop 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 shutdown_connections 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc82406d8a0 0x7fc82406fd50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8400ff350 0x7fc840198100 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 --2- 172.21.15.57:0/954309023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8400ffc60 0x7fc840198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.530+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 >> 172.21.15.57:0/954309023 conn(0x7fc8400748b0 msgr2=0x7fc840100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:45.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.531+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 shutdown_connections 2024-02-28T23:35:45.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.531+0000 7fc845200700 1 -- 172.21.15.57:0/954309023 wait complete. 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 -- 172.21.15.57:0/4156837213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a00ffbd0 msgr2=0x7f35a0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 --2- 172.21.15.57:0/4156837213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f358c009910 tx=0x7f358c009c20 comp rx=0 tx=0).stop 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 -- 172.21.15.57:0/4156837213 shutdown_connections 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 --2- 172.21.15.57:0/4156837213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100040 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 --2- 172.21.15.57:0/4156837213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a0104130 0x7f35a0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.679+0000 7f35a6000700 1 -- 172.21.15.57:0/4156837213 >> 172.21.15.57:0/4156837213 conn(0x7f35a0075130 msgr2=0x7f35a0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.680+0000 7f35a6000700 1 -- 172.21.15.57:0/4156837213 shutdown_connections 2024-02-28T23:35:45.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.680+0000 7f35a6000700 1 -- 172.21.15.57:0/4156837213 wait complete. 2024-02-28T23:35:45.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.681+0000 7f35a6000700 1 Processor -- start 2024-02-28T23:35:45.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.681+0000 7f35a6000700 1 -- start start 2024-02-28T23:35:45.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f35a6000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f35a6000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f35a6000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a01018b0 con 0x7f35a0104130 2024-02-28T23:35:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f35a6000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a01019f0 con 0x7f35a00ffbd0 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f359ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f359ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47272/0 (socket says 172.21.15.57:47272) 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f359ea00700 1 -- 172.21.15.57:0/3428084818 learned_addr learned my addr 172.21.15.57:0/3428084818 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.682+0000 7f359f400700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f359ea00700 1 -- 172.21.15.57:0/3428084818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a00ffbd0 msgr2=0x7f35a0100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f359ea00700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f359ea00700 1 -- 172.21.15.57:0/3428084818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0095f0 con 0x7f35a0104130 2024-02-28T23:35:45.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f359ea00700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358c01d070 con 0x7f35a0104130 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f358c004fb0 con 0x7f35a0104130 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a0101c70 con 0x7f35a0104130 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358c00f480 con 0x7f35a0104130 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.683+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a01a2320 con 0x7f35a0104130 2024-02-28T23:35:45.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.684+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a01086a0 con 0x7f35a0104130 2024-02-28T23:35:45.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.686+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f358c022890 con 0x7f35a0104130 2024-02-28T23:35:45.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.686+0000 7f3597e00700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 0x7f358406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:35:45.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.687+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f358c08ee80 con 0x7f35a0104130 2024-02-28T23:35:45.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.687+0000 7f359f400700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 0x7f358406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:35:45.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.688+0000 7f359f400700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 0x7f358406fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:35:45.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:45.692+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f358c058760 con 0x7f35a0104130 2024-02-28T23:35:46.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.017+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f35a0108080 con 0x7f35a0104130 2024-02-28T23:35:46.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.018+0000 7f3597e00700 1 -- 172.21.15.57:0/3428084818 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f358c027090 con 0x7f35a0104130 2024-02-28T23:35:46.016 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:35:46.016 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:35:46.016 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:35:46.016 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:35:46.017 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:35:46.018 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.020+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 msgr2=0x7f358406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.020+0000 7f35a6000700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 0x7f358406fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000a560 comp rx=0 tx=0).stop 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.020+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 msgr2=0x7f35a0101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.020+0000 7f35a6000700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c00be20 comp rx=0 tx=0).stop 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 shutdown_connections 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f358406d8a0 0x7f358406fd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35a00ffbd0 0x7f35a0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 --2- 172.21.15.57:0/3428084818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35a0104130 0x7f35a0101370 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:35:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 >> 172.21.15.57:0/3428084818 conn(0x7f35a0075130 msgr2=0x7f35a00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:35:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 shutdown_connections 2024-02-28T23:35:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:35:46.021+0000 7f35a6000700 1 -- 172.21.15.57:0/3428084818 wait complete. 2024-02-28T23:35:46.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:45 smithi057 ceph-mon[89491]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:46.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:45 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/140876103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:35:46.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:45 smithi084 ceph-mon[70440]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:46.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:45 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/140876103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:35:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:46 smithi057 ceph-mon[89491]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:46 smithi057 ceph-mon[89491]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:46 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3428084818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:35:47.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:46 smithi084 ceph-mon[70440]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:46 smithi084 ceph-mon[70440]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:35:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:46 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3428084818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:35:49.121 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:48 smithi057 ceph-mon[89491]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:49.126 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:49.132 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:48 smithi084 ceph-mon[70440]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:49.133 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:35:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:50 smithi057 ceph-mon[89491]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:51.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:50 smithi084 ceph-mon[70440]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:53.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:52 smithi057 ceph-mon[89491]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:53.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:52 smithi084 ceph-mon[70440]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:55.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:54 smithi057 ceph-mon[89491]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:55.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:54 smithi084 ceph-mon[70440]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:56 smithi057 ceph-mon[89491]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:57.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:56 smithi084 ceph-mon[70440]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:59.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:35:58 smithi057 ceph-mon[89491]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:35:59.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:35:58 smithi084 ceph-mon[70440]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:00 smithi057 ceph-mon[89491]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:01.320 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:00 smithi084 ceph-mon[70440]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:03.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:02 smithi057 ceph-mon[89491]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:03.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:02 smithi084 ceph-mon[70440]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:04.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:04.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:04 smithi057 ceph-mon[89491]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:05.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:04 smithi084 ceph-mon[70440]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:07.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:06 smithi057 ceph-mon[89491]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:06 smithi084 ceph-mon[70440]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:09.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:08 smithi057 ceph-mon[89491]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:09.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:08 smithi084 ceph-mon[70440]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:11.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:10 smithi057 ceph-mon[89491]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:11.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:10 smithi084 ceph-mon[70440]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:13.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:12 smithi057 ceph-mon[89491]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:13.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:12 smithi084 ceph-mon[70440]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:15.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:14 smithi057 ceph-mon[89491]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:15.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:14 smithi084 ceph-mon[70440]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:16.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 -- 172.21.15.57:0/1938929286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 msgr2=0x7fa110068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 --2- 172.21.15.57:0/1938929286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110068b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa100009910 tx=0x7fa100009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:16.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 -- 172.21.15.57:0/1938929286 shutdown_connections 2024-02-28T23:36:16.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 --2- 172.21.15.57:0/1938929286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa110104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 --2- 172.21.15.57:0/1938929286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110068b70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.198+0000 7fa115400700 1 -- 172.21.15.57:0/1938929286 >> 172.21.15.57:0/1938929286 conn(0x7fa110075130 msgr2=0x7fa110075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:16.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.199+0000 7fa115400700 1 -- 172.21.15.57:0/1938929286 shutdown_connections 2024-02-28T23:36:16.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.199+0000 7fa115400700 1 -- 172.21.15.57:0/1938929286 wait complete. 2024-02-28T23:36:16.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.200+0000 7fa115400700 1 Processor -- start 2024-02-28T23:36:16.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.200+0000 7fa115400700 1 -- start start 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa115400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa115400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa1101964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa115400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110196b80 con 0x7fa1100687a0 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa115400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11019a910 con 0x7fa110069140 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42102/0 (socket says 172.21.15.57:42102) 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 -- 172.21.15.57:0/2240182647 learned_addr learned my addr 172.21.15.57:0/2240182647 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa105e00700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa1101964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 -- 172.21.15.57:0/2240182647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 msgr2=0x7fa1101964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa1101964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa10e800700 1 -- 172.21.15.57:0/2240182647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1000095f0 con 0x7fa1100687a0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.201+0000 7fa105e00700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa1101964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa10e800700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa100006010 tx=0x7fa10000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10001d070 con 0x7fa1100687a0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11019ab90 con 0x7fa1100687a0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa100004fb0 con 0x7fa1100687a0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10000f690 con 0x7fa1100687a0 2024-02-28T23:36:16.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.202+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11019b080 con 0x7fa1100687a0 2024-02-28T23:36:16.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.203+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa110108790 con 0x7fa1100687a0 2024-02-28T23:36:16.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.204+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa1000228a0 con 0x7fa1100687a0 2024-02-28T23:36:16.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.205+0000 7fa107e00700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 0x7fa0f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.205+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa10008ee70 con 0x7fa1100687a0 2024-02-28T23:36:16.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.206+0000 7fa105e00700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 0x7fa0f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.206+0000 7fa105e00700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 0x7fa0f406fd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa110197580 tx=0x7fa0f8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:16.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.210+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa100058580 con 0x7fa1100687a0 2024-02-28T23:36:16.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.477+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11002cca0 con 0x7fa0f406d8a0 2024-02-28T23:36:16.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.480+0000 7fa107e00700 1 -- 172.21.15.57:0/2240182647 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa11002cca0 con 0x7fa0f406d8a0 2024-02-28T23:36:16.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 msgr2=0x7fa0f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 0x7fa0f406fd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa110197580 tx=0x7fa0f8008040 comp rx=0 tx=0).stop 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 msgr2=0x7fa110195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa100006010 tx=0x7fa10000be20 comp rx=0 tx=0).stop 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 shutdown_connections 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa0f406d8a0 0x7fa0f406fd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1100687a0 0x7fa110195f60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 --2- 172.21.15.57:0/2240182647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa110069140 0x7fa1101964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.483+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 >> 172.21.15.57:0/2240182647 conn(0x7fa110075130 msgr2=0x7fa1100fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.484+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 shutdown_connections 2024-02-28T23:36:16.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.484+0000 7fa115400700 1 -- 172.21.15.57:0/2240182647 wait complete. 2024-02-28T23:36:16.499 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.651+0000 7f58e5000700 1 -- 172.21.15.57:0/956208689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 msgr2=0x7f58e0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.651+0000 7f58e5000700 1 --2- 172.21.15.57:0/956208689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e0104eb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009910 tx=0x7f58cc009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 -- 172.21.15.57:0/956208689 shutdown_connections 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 --2- 172.21.15.57:0/956208689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e0104eb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 --2- 172.21.15.57:0/956208689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 -- 172.21.15.57:0/956208689 >> 172.21.15.57:0/956208689 conn(0x7f58e0075130 msgr2=0x7f58e0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:16.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 -- 172.21.15.57:0/956208689 shutdown_connections 2024-02-28T23:36:16.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.652+0000 7f58e5000700 1 -- 172.21.15.57:0/956208689 wait complete. 2024-02-28T23:36:16.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 Processor -- start 2024-02-28T23:36:16.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 -- start start 2024-02-28T23:36:16.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0198d10 con 0x7f58e0069140 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.653+0000 7f58e5000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0198e50 con 0x7f58e00687a0 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42118/0 (socket says 172.21.15.57:42118) 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 -- 172.21.15.57:0/2680699697 learned_addr learned my addr 172.21.15.57:0/2680699697 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dde00700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 -- 172.21.15.57:0/2680699697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 msgr2=0x7f58e0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dd400700 1 -- 172.21.15.57:0/2680699697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc0095f0 con 0x7f58e0069140 2024-02-28T23:36:16.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.654+0000 7f58dde00700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58dd400700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00bda0 tx=0x7f58cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58cc01d070 con 0x7f58e0069140 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58cc004fb0 con 0x7f58e0069140 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58cc00f480 con 0x7f58e0069140 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e019cc40 con 0x7f58e0069140 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.655+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e019d0e0 con 0x7f58e0069140 2024-02-28T23:36:16.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.656+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e004f2a0 con 0x7f58e0069140 2024-02-28T23:36:16.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.657+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f58cc022870 con 0x7f58e0069140 2024-02-28T23:36:16.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.658+0000 7f58d6a00700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 0x7f58c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:16.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.658+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f58cc08ed40 con 0x7f58e0069140 2024-02-28T23:36:16.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.659+0000 7f58dde00700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 0x7f58c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:16.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.659+0000 7f58dde00700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 0x7f58c406fd50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005fd0 tx=0x7f58d0005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:16.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.663+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58cc058620 con 0x7f58e0069140 2024-02-28T23:36:16.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.916+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58e019d410 con 0x7f58c406d8a0 2024-02-28T23:36:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.919+0000 7f58d6a00700 1 -- 172.21.15.57:0/2680699697 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f58e019d410 con 0x7f58c406d8a0 2024-02-28T23:36:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 msgr2=0x7f58c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 0x7f58c406fd50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005fd0 tx=0x7f58d0005e20 comp rx=0 tx=0).stop 2024-02-28T23:36:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 msgr2=0x7f58e01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00bda0 tx=0x7f58cc00be80 comp rx=0 tx=0).stop 2024-02-28T23:36:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 shutdown_connections 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f58c406d8a0 0x7f58c406fd50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58e00687a0 0x7f58e0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 --2- 172.21.15.57:0/2680699697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e0069140 0x7f58e01986c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.922+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 >> 172.21.15.57:0/2680699697 conn(0x7f58e0075130 msgr2=0x7f58e0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.923+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 shutdown_connections 2024-02-28T23:36:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:16.923+0000 7f58e5000700 1 -- 172.21.15.57:0/2680699697 wait complete. 2024-02-28T23:36:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:16 smithi084 ceph-mon[70440]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 -- 172.21.15.57:0/1815426120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c100e80 msgr2=0x7f9d2c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 --2- 172.21.15.57:0/1815426120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c100e80 0x7f9d2c104e50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009910 tx=0x7f9d14009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 -- 172.21.15.57:0/1815426120 shutdown_connections 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 --2- 172.21.15.57:0/1815426120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c100e80 0x7f9d2c104e50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 --2- 172.21.15.57:0/1815426120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.082+0000 7f9d31200700 1 -- 172.21.15.57:0/1815426120 >> 172.21.15.57:0/1815426120 conn(0x7f9d2c0fbda0 msgr2=0x7f9d2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.083+0000 7f9d31200700 1 -- 172.21.15.57:0/1815426120 shutdown_connections 2024-02-28T23:36:17.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.083+0000 7f9d31200700 1 -- 172.21.15.57:0/1815426120 wait complete. 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 Processor -- start 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 -- start start 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 0x7f9d2c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c196b50 con 0x7f9d2c1004e0 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.084+0000 7f9d31200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c19a8e0 con 0x7f9d2c100e80 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42146/0 (socket says 172.21.15.57:42146) 2024-02-28T23:36:17.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 -- 172.21.15.57:0/1648893434 learned_addr learned my addr 172.21.15.57:0/1648893434 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d23e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 0x7f9d2c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 -- 172.21.15.57:0/1648893434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 msgr2=0x7f9d2c196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 0x7f9d2c196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 -- 172.21.15.57:0/1648893434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140095f0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d23e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 0x7f9d2c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.085+0000 7f9d29e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c8a0 tx=0x7f9d1c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.086+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1c00cea0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.086+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c19abc0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.086+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d1c004830 con 0x7f9d2c1004e0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.086+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1c0055b0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.086+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c19b110 con 0x7f9d2c1004e0 2024-02-28T23:36:17.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.087+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c108850 con 0x7f9d2c1004e0 2024-02-28T23:36:17.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.088+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9d1c0049a0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.089+0000 7f9d22a00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 0x7f9d1006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.089+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9d1c08d0e0 con 0x7f9d2c1004e0 2024-02-28T23:36:17.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.090+0000 7f9d23e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 0x7f9d1006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.090+0000 7f9d23e00700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 0x7f9d1006fd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006010 tx=0x7f9d14005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:17.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.094+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d1c056910 con 0x7f9d2c1004e0 2024-02-28T23:36:17.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:16 smithi057 ceph-mon[89491]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:17.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.331+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d2c19b3e0 con 0x7f9d1006d8a0 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.364+0000 7f9d22a00700 1 -- 172.21.15.57:0/1648893434 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9d2c19b3e0 con 0x7f9d1006d8a0 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (25m) 98s ago 39m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (36m) 98s ago 40m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (38m) 101s ago 38m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:36:17.363 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (21m) 98s ago 40m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (21m) 101s ago 38m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (25m) 98s ago 39m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (17m) 98s ago 32m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (17m) 98s ago 32m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (107s) 101s ago 32m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (17m) 101s ago 32m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (28m) 98s ago 41m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (27m) 101s ago 38m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (23m) 98s ago 41m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (23m) 101s ago 38m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (26m) 98s ago 39m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:36:17.364 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (26m) 101s ago 38m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (21m) 98s ago 37m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (20m) 98s ago 36m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (20m) 98s ago 36m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (19m) 101s ago 35m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (19m) 101s ago 34m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (18m) 101s ago 34m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:36:17.365 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (26m) 98s ago 39m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:36:17.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 msgr2=0x7f9d1006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 0x7f9d1006fd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006010 tx=0x7f9d14005960 comp rx=0 tx=0).stop 2024-02-28T23:36:17.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 msgr2=0x7f9d2c195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c8a0 tx=0x7f9d1c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 shutdown_connections 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d1006d8a0 0x7f9d1006fd50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d2c1004e0 0x7f9d2c195f30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 --2- 172.21.15.57:0/1648893434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2c100e80 0x7f9d2c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 >> 172.21.15.57:0/1648893434 conn(0x7f9d2c0fbda0 msgr2=0x7f9d2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 shutdown_connections 2024-02-28T23:36:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.367+0000 7f9d31200700 1 -- 172.21.15.57:0/1648893434 wait complete. 2024-02-28T23:36:17.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 -- 172.21.15.57:0/2712324650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ff3c0 msgr2=0x7ff94c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 --2- 172.21.15.57:0/2712324650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c0ff790 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff940009910 tx=0x7ff940009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 -- 172.21.15.57:0/2712324650 shutdown_connections 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 --2- 172.21.15.57:0/2712324650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ffd60 0x7ff94c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 --2- 172.21.15.57:0/2712324650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c0ff790 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 -- 172.21.15.57:0/2712324650 >> 172.21.15.57:0/2712324650 conn(0x7ff94c0fac80 msgr2=0x7ff94c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 -- 172.21.15.57:0/2712324650 shutdown_connections 2024-02-28T23:36:17.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.522+0000 7ff953800700 1 -- 172.21.15.57:0/2712324650 wait complete. 2024-02-28T23:36:17.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.523+0000 7ff953800700 1 Processor -- start 2024-02-28T23:36:17.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.523+0000 7ff953800700 1 -- start start 2024-02-28T23:36:17.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff953800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff953800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff953800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c196c40 con 0x7ff94c0ffd60 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff953800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c19a9d0 con 0x7ff94c0ff3c0 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff94be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff94be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42164/0 (socket says 172.21.15.57:42164) 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff94be00700 1 -- 172.21.15.57:0/3804360651 learned_addr learned my addr 172.21.15.57:0/3804360651 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.524+0000 7ff950e00700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff94be00700 1 -- 172.21.15.57:0/3804360651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ff3c0 msgr2=0x7ff94c196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff94be00700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff94be00700 1 -- 172.21.15.57:0/3804360651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9400095f0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff94be00700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b480 tx=0x7ff93800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9380118c0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff938010380 con 0x7ff94c0ffd60 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c19acb0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93800f4f0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.525+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c19b200 con 0x7ff94c0ffd60 2024-02-28T23:36:17.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.527+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c04f2a0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.528+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff9380104f0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.529+0000 7ff949600700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 0x7ff93406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:17.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.529+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff93808d210 con 0x7ff94c0ffd60 2024-02-28T23:36:17.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.529+0000 7ff950e00700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 0x7ff93406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:17.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.530+0000 7ff950e00700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 0x7ff93406fd50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff940006010 tx=0x7ff94001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.534+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff938056920 con 0x7ff94c0ffd60 2024-02-28T23:36:17.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.855+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff94c19b4b0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.856+0000 7ff949600700 1 -- 172.21.15.57:0/3804360651 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff938059fe0 con 0x7ff94c0ffd60 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:36:17.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:36:17.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 msgr2=0x7ff93406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 0x7ff93406fd50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff940006010 tx=0x7ff94001a040 comp rx=0 tx=0).stop 2024-02-28T23:36:17.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 msgr2=0x7ff94c196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:17.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b480 tx=0x7ff93800b840 comp rx=0 tx=0).stop 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 shutdown_connections 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff93406d8a0 0x7ff93406fd50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff94c0ff3c0 0x7ff94c196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 --2- 172.21.15.57:0/3804360651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff94c0ffd60 0x7ff94c196560 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 >> 172.21.15.57:0/3804360651 conn(0x7ff94c0fac80 msgr2=0x7ff94c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.859+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 shutdown_connections 2024-02-28T23:36:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:17.860+0000 7ff953800700 1 -- 172.21.15.57:0/3804360651 wait complete. 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 -- 172.21.15.57:0/1739239865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 msgr2=0x7fa6f810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/1739239865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f810c5e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009940 tx=0x7fa6e4009c50 comp rx=0 tx=0).stop 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 -- 172.21.15.57:0/1739239865 shutdown_connections 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/1739239865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f810c5e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/1739239865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f8073090 0x7fa6f8073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.019+0000 7fa6ffa00700 1 -- 172.21.15.57:0/1739239865 >> 172.21.15.57:0/1739239865 conn(0x7fa6f80fbda0 msgr2=0x7fa6f80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:18.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.020+0000 7fa6ffa00700 1 -- 172.21.15.57:0/1739239865 shutdown_connections 2024-02-28T23:36:18.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.020+0000 7fa6ffa00700 1 -- 172.21.15.57:0/1739239865 wait complete. 2024-02-28T23:36:18.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 Processor -- start 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 -- start start 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f8073090 0x7fa6f8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8198db0 con 0x7fa6f80739a0 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6ffa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8198ef0 con 0x7fa6f8073090 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6f7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6f7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42182/0 (socket says 172.21.15.57:42182) 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.021+0000 7fa6f7e00700 1 -- 172.21.15.57:0/2220796943 learned_addr learned my addr 172.21.15.57:0/2220796943 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f7e00700 1 -- 172.21.15.57:0/2220796943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f8073090 msgr2=0x7fa6f8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f7e00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f8073090 0x7fa6f8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f7e00700 1 -- 172.21.15.57:0/2220796943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40095f0 con 0x7fa6f80739a0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f7e00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b870 tx=0x7fa6e40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e401d070 con 0x7fa6f80739a0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6e400f460 con 0x7fa6f80739a0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e4021640 con 0x7fa6f80739a0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.022+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f819cce0 con 0x7fa6f80739a0 2024-02-28T23:36:18.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.023+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f819d1d0 con 0x7fa6f80739a0 2024-02-28T23:36:18.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.024+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8109ce0 con 0x7fa6f80739a0 2024-02-28T23:36:18.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.025+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa6e400fa60 con 0x7fa6f80739a0 2024-02-28T23:36:18.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.026+0000 7fa6f5600700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 0x7fa6e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.026+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa6e408e8b0 con 0x7fa6f80739a0 2024-02-28T23:36:18.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.026+0000 7fa6fd000700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 0x7fa6e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.027+0000 7fa6fd000700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 0x7fa6e006fd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005fd0 tx=0x7fa6ec005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.030+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6e4057fc0 con 0x7fa6f80739a0 2024-02-28T23:36:18.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.304+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6f804f2a0 con 0x7fa6f80739a0 2024-02-28T23:36:18.303 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:17 smithi057 ceph-mon[89491]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:18.303 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:17 smithi057 ceph-mon[89491]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:18.303 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3804360651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:36:18.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.305+0000 7fa6f5600700 1 -- 172.21.15.57:0/2220796943 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fa6e405b680 con 0x7fa6f80739a0 2024-02-28T23:36:18.305 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:36:18.305 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:36:18.305 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:36:18.305 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:36:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:36:18.307 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:18.308 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:18.309 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.307+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 msgr2=0x7fa6e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.307+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 0x7fa6e006fd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005fd0 tx=0x7fa6ec005d00 comp rx=0 tx=0).stop 2024-02-28T23:36:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.307+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 msgr2=0x7fa6f8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.307+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b870 tx=0x7fa6e40056a0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 shutdown_connections 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa6e006d8a0 0x7fa6e006fd50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f8073090 0x7fa6f8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 --2- 172.21.15.57:0/2220796943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f80739a0 0x7fa6f8198760 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 >> 172.21.15.57:0/2220796943 conn(0x7fa6f80fbda0 msgr2=0x7fa6f8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 shutdown_connections 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.308+0000 7fa6ffa00700 1 -- 172.21.15.57:0/2220796943 wait complete. 2024-02-28T23:36:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:36:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:17 smithi084 ceph-mon[70440]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:18.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:17 smithi084 ceph-mon[70440]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:18.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3804360651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:36:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.458+0000 7f5b3fe00700 1 -- 172.21.15.57:0/4092104902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b40100c60 msgr2=0x7f5b40104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.458+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/4092104902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b40100c60 0x7f5b40104c30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009910 tx=0x7f5b30009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 -- 172.21.15.57:0/4092104902 shutdown_connections 2024-02-28T23:36:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/4092104902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b40100c60 0x7f5b40104c30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/4092104902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b401002c0 0x7f5b40100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 -- 172.21.15.57:0/4092104902 >> 172.21.15.57:0/4092104902 conn(0x7f5b400fbb80 msgr2=0x7f5b400fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 -- 172.21.15.57:0/4092104902 shutdown_connections 2024-02-28T23:36:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.459+0000 7f5b3fe00700 1 -- 172.21.15.57:0/4092104902 wait complete. 2024-02-28T23:36:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.460+0000 7f5b3fe00700 1 Processor -- start 2024-02-28T23:36:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.460+0000 7f5b3fe00700 1 -- start start 2024-02-28T23:36:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3fe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b40100c60 0x7f5b40073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3fe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40073ad0 con 0x7f5b401002c0 2024-02-28T23:36:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3fe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40073c10 con 0x7f5b40100c60 2024-02-28T23:36:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42200/0 (socket says 172.21.15.57:42200) 2024-02-28T23:36:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 -- 172.21.15.57:0/2780336447 learned_addr learned my addr 172.21.15.57:0/2780336447 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:18.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b36000700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b40100c60 0x7f5b40073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 -- 172.21.15.57:0/2780336447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b40100c60 msgr2=0x7f5b40073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b40100c60 0x7f5b40073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.461+0000 7f5b3ea00700 1 -- 172.21.15.57:0/2780336447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300095f0 con 0x7f5b401002c0 2024-02-28T23:36:18.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b3ea00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c8a0 tx=0x7f5b2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2800cea0 con 0x7f5b401002c0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b28004830 con 0x7f5b401002c0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b280055b0 con 0x7f5b401002c0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b401a1e60 con 0x7f5b401002c0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.462+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b401a2270 con 0x7f5b401002c0 2024-02-28T23:36:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.463+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40062380 con 0x7f5b401002c0 2024-02-28T23:36:18.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.464+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5b280049a0 con 0x7f5b401002c0 2024-02-28T23:36:18.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.465+0000 7f5b37e00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 0x7f5b2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.465+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5b2808d190 con 0x7f5b401002c0 2024-02-28T23:36:18.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.466+0000 7f5b36000700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 0x7f5b2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.466+0000 7f5b36000700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 0x7f5b2006fd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b3000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.471+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b28056920 con 0x7f5b401002c0 2024-02-28T23:36:18.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.711+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b400fdeb0 con 0x7f5b2006d8a0 2024-02-28T23:36:18.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.713+0000 7f5b37e00700 1 -- 172.21.15.57:0/2780336447 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5b400fdeb0 con 0x7f5b2006d8a0 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:36:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:36:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:36:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:36:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:36:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:36:18.714 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:36:18.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 msgr2=0x7f5b2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 0x7f5b2006fd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b3000b540 comp rx=0 tx=0).stop 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 msgr2=0x7f5b40073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c8a0 tx=0x7f5b2800cc60 comp rx=0 tx=0).stop 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 shutdown_connections 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5b2006d8a0 0x7f5b2006fd50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b401002c0 0x7f5b40073050 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 --2- 172.21.15.57:0/2780336447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b40100c60 0x7f5b40073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 >> 172.21.15.57:0/2780336447 conn(0x7f5b400fbb80 msgr2=0x7f5b400fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 shutdown_connections 2024-02-28T23:36:18.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.716+0000 7f5b3fe00700 1 -- 172.21.15.57:0/2780336447 wait complete. 2024-02-28T23:36:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 -- 172.21.15.57:0/697321085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 msgr2=0x7f8e70106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 --2- 172.21.15.57:0/697321085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e70106ff0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009910 tx=0x7f8e5c009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:18.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 -- 172.21.15.57:0/697321085 shutdown_connections 2024-02-28T23:36:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 --2- 172.21.15.57:0/697321085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e70106ff0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 --2- 172.21.15.57:0/697321085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 0x7f8e7010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 -- 172.21.15.57:0/697321085 >> 172.21.15.57:0/697321085 conn(0x7f8e70076060 msgr2=0x7f8e70078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 -- 172.21.15.57:0/697321085 shutdown_connections 2024-02-28T23:36:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.879+0000 7f8e75200700 1 -- 172.21.15.57:0/697321085 wait complete. 2024-02-28T23:36:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.880+0000 7f8e75200700 1 Processor -- start 2024-02-28T23:36:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.880+0000 7f8e75200700 1 -- start start 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.880+0000 7f8e75200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.880+0000 7f8e75200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 0x7f8e7019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e75200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7019d060 con 0x7f8e70106b80 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e75200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e701a0df0 con 0x7f8e7010c8f0 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42212/0 (socket says 172.21.15.57:42212) 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 -- 172.21.15.57:0/1539818930 learned_addr learned my addr 172.21.15.57:0/1539818930 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6d400700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 0x7f8e7019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 -- 172.21.15.57:0/1539818930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 msgr2=0x7f8e7019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 0x7f8e7019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.881+0000 7f8e6de00700 1 -- 172.21.15.57:0/1539818930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0095f0 con 0x7f8e70106b80 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e6de00700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000d8d0 tx=0x7f8e6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e60009940 con 0x7f8e70106b80 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e60010460 con 0x7f8e70106b80 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e6000f5f0 con 0x7f8e70106b80 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e701a10d0 con 0x7f8e70106b80 2024-02-28T23:36:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.882+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e701a1620 con 0x7f8e70106b80 2024-02-28T23:36:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.883+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7010ecc0 con 0x7f8e70106b80 2024-02-28T23:36:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.885+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8e600105d0 con 0x7f8e70106b80 2024-02-28T23:36:18.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.885+0000 7f8e66a00700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 0x7f8e5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:18.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.886+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8e6008d4c0 con 0x7f8e70106b80 2024-02-28T23:36:18.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.886+0000 7f8e6d400700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 0x7f8e5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:18.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.887+0000 7f8e6d400700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 0x7f8e5406fd50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e7019da60 tx=0x7f8e5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:18.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:18.891+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e60056c50 con 0x7f8e70106b80 2024-02-28T23:36:19.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.215+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8e7004f2a0 con 0x7f8e70106b80 2024-02-28T23:36:19.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.215+0000 7f8e66a00700 1 -- 172.21.15.57:0/1539818930 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f8e6005a310 con 0x7f8e70106b80 2024-02-28T23:36:19.215 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:36:19.215 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:36:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:36:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:36:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:36:19.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.218+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 msgr2=0x7f8e5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.218+0000 7f8e75200700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 0x7f8e5406fd50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e7019da60 tx=0x7f8e5c01a040 comp rx=0 tx=0).stop 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.218+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 msgr2=0x7f8e7019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.218+0000 7f8e75200700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000d8d0 tx=0x7f8e6000dc90 comp rx=0 tx=0).stop 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 shutdown_connections 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8e5406d8a0 0x7f8e5406fd50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e70106b80 0x7f8e7019c440 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 --2- 172.21.15.57:0/1539818930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e7010c8f0 0x7f8e7019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:19.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 >> 172.21.15.57:0/1539818930 conn(0x7f8e70076060 msgr2=0x7f8e70077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:19.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 shutdown_connections 2024-02-28T23:36:19.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:19.219+0000 7f8e75200700 1 -- 172.21.15.57:0/1539818930 wait complete. 2024-02-28T23:36:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:18 smithi057 ceph-mon[89491]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:18 smithi057 ceph-mon[89491]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:19.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:18 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2220796943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:36:19.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:18 smithi084 ceph-mon[70440]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:18 smithi084 ceph-mon[70440]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:18 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2220796943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:36:20.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:20 smithi057 ceph-mon[89491]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:20.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:20 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1539818930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:36:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:20 smithi084 ceph-mon[70440]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:20.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:20 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1539818930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:36:21.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:21 smithi057 ceph-mon[89491]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:21.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:21 smithi084 ceph-mon[70440]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:23 smithi057 ceph-mon[89491]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:23.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:23 smithi084 ceph-mon[70440]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:25.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:25 smithi057 ceph-mon[89491]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:25.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:25 smithi084 ceph-mon[70440]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:27 smithi057 ceph-mon[89491]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:27.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:27 smithi084 ceph-mon[70440]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:28.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:28 smithi057 ceph-mon[89491]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:28.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:28 smithi084 ceph-mon[70440]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:30.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:30 smithi084 ceph-mon[70440]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:30.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:30 smithi057 ceph-mon[89491]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:32.579 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:32 smithi084 ceph-mon[70440]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:32.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:32 smithi057 ceph-mon[89491]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:33.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:34.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:34 smithi057 ceph-mon[89491]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:34.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:34 smithi084 ceph-mon[70440]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:36 smithi057 ceph-mon[89491]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:36.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:36 smithi084 ceph-mon[70440]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:38.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:38 smithi057 ceph-mon[89491]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:38.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:38 smithi084 ceph-mon[70440]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:40.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:40 smithi057 ceph-mon[89491]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:40.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:40 smithi084 ceph-mon[70440]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:42.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:42 smithi057 ceph-mon[89491]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:42.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:42 smithi084 ceph-mon[70440]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:44 smithi057 ceph-mon[89491]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:44.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:44 smithi084 ceph-mon[70440]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:46 smithi084 ceph-mon[70440]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:46 smithi057 ceph-mon[89491]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:48 smithi057 ceph-mon[89491]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:48.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:48 smithi084 ceph-mon[70440]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:48.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:36:49.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 -- 172.21.15.57:0/3688622745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 msgr2=0x7f635010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 --2- 172.21.15.57:0/3688622745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f635010c5e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f633c009940 tx=0x7f633c009c50 comp rx=0 tx=0).stop 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 -- 172.21.15.57:0/3688622745 shutdown_connections 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 --2- 172.21.15.57:0/3688622745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f635010c5e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 --2- 172.21.15.57:0/3688622745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f6350073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 -- 172.21.15.57:0/3688622745 >> 172.21.15.57:0/3688622745 conn(0x7f63500fbda0 msgr2=0x7f63500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 -- 172.21.15.57:0/3688622745 shutdown_connections 2024-02-28T23:36:49.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.399+0000 7f6357e00700 1 -- 172.21.15.57:0/3688622745 wait complete. 2024-02-28T23:36:49.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 Processor -- start 2024-02-28T23:36:49.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 -- start start 2024-02-28T23:36:49.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f63501981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6350198d40 con 0x7f63500739a0 2024-02-28T23:36:49.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.401+0000 7f6357e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6350198e80 con 0x7f6350073090 2024-02-28T23:36:49.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58912/0 (socket says 172.21.15.57:58912) 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 -- 172.21.15.57:0/3584647574 learned_addr learned my addr 172.21.15.57:0/3584647574 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6355400700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f63501981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 -- 172.21.15.57:0/3584647574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 msgr2=0x7f63501981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f63501981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6354a00700 1 -- 172.21.15.57:0/3584647574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0095f0 con 0x7f63500739a0 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.402+0000 7f6355400700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f63501981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f6354a00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f633c00b870 tx=0x7f633c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:49.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633c01d070 con 0x7f63500739a0 2024-02-28T23:36:49.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f633c00f460 con 0x7f63500739a0 2024-02-28T23:36:49.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635019cc70 con 0x7f63500739a0 2024-02-28T23:36:49.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635019d160 con 0x7f63500739a0 2024-02-28T23:36:49.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.403+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633c021640 con 0x7f63500739a0 2024-02-28T23:36:49.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.405+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6350109ce0 con 0x7f63500739a0 2024-02-28T23:36:49.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.405+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f633c00fa60 con 0x7f63500739a0 2024-02-28T23:36:49.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.406+0000 7f634e000700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 0x7f633806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.406+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f633c08e8e0 con 0x7f63500739a0 2024-02-28T23:36:49.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.407+0000 7f6355400700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 0x7f633806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.407+0000 7f6355400700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 0x7f633806fd50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6344005fd0 tx=0x7f6344005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.412+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f633c058090 con 0x7f63500739a0 2024-02-28T23:36:49.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.663+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6350199510 con 0x7f633806d8a0 2024-02-28T23:36:49.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.665+0000 7f634e000700 1 -- 172.21.15.57:0/3584647574 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6350199510 con 0x7f633806d8a0 2024-02-28T23:36:49.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 msgr2=0x7f633806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 0x7f633806fd50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6344005fd0 tx=0x7f6344005e30 comp rx=0 tx=0).stop 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 msgr2=0x7f63501986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f633c00b870 tx=0x7f633c0056a0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 shutdown_connections 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f633806d8a0 0x7f633806fd50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6350073090 0x7f63501981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 --2- 172.21.15.57:0/3584647574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63500739a0 0x7f63501986f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 >> 172.21.15.57:0/3584647574 conn(0x7f63500fbda0 msgr2=0x7f6350106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 shutdown_connections 2024-02-28T23:36:49.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.668+0000 7f6357e00700 1 -- 172.21.15.57:0/3584647574 wait complete. 2024-02-28T23:36:49.683 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 -- 172.21.15.57:0/3543970771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 msgr2=0x7f8ab410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 --2- 172.21.15.57:0/3543970771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab410ccc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009910 tx=0x7f8aa4009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 -- 172.21.15.57:0/3543970771 shutdown_connections 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 --2- 172.21.15.57:0/3543970771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 0x7f8ab4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 --2- 172.21.15.57:0/3543970771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab410ccc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 -- 172.21.15.57:0/3543970771 >> 172.21.15.57:0/3543970771 conn(0x7f8ab4076060 msgr2=0x7f8ab4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:49.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 -- 172.21.15.57:0/3543970771 shutdown_connections 2024-02-28T23:36:49.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.834+0000 7f8aba800700 1 -- 172.21.15.57:0/3543970771 wait complete. 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.835+0000 7f8aba800700 1 Processor -- start 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.835+0000 7f8aba800700 1 -- start start 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aba800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 0x7f8ab419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aba800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aba800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab419d120 con 0x7f8ab410c8f0 2024-02-28T23:36:49.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aba800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab41a0eb0 con 0x7f8ab4106b80 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58924/0 (socket says 172.21.15.57:58924) 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.836+0000 7f8aabe00700 1 -- 172.21.15.57:0/2060675441 learned_addr learned my addr 172.21.15.57:0/2060675441 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8ab3400700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 0x7f8ab419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8aabe00700 1 -- 172.21.15.57:0/2060675441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 msgr2=0x7f8ab419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8aabe00700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 0x7f8ab419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8aabe00700 1 -- 172.21.15.57:0/2060675441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40095f0 con 0x7f8ab410c8f0 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8aabe00700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00d8d0 tx=0x7f8a9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:49.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a9c009940 con 0x7f8ab410c8f0 2024-02-28T23:36:49.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a9c010460 con 0x7f8ab410c8f0 2024-02-28T23:36:49.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab41a1190 con 0x7f8ab410c8f0 2024-02-28T23:36:49.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.837+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a9c00f5f0 con 0x7f8ab410c8f0 2024-02-28T23:36:49.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.838+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab41a16e0 con 0x7f8ab410c8f0 2024-02-28T23:36:49.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.839+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab4109b20 con 0x7f8ab410c8f0 2024-02-28T23:36:49.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.840+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8a9c0105d0 con 0x7f8ab410c8f0 2024-02-28T23:36:49.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.841+0000 7f8ab0c00700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 0x7f8a9806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:49.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.841+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8a9c08d4c0 con 0x7f8ab410c8f0 2024-02-28T23:36:49.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.841+0000 7f8ab3400700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 0x7f8a9806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:49.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.842+0000 7f8ab3400700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 0x7f8a9806fd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b5c0 tx=0x7f8aa40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:49.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:49.846+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a9c056c50 con 0x7f8ab410c8f0 2024-02-28T23:36:50.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.098+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ab41a19c0 con 0x7f8a9806d8a0 2024-02-28T23:36:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.101+0000 7f8ab0c00700 1 -- 172.21.15.57:0/2060675441 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ab41a19c0 con 0x7f8a9806d8a0 2024-02-28T23:36:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 msgr2=0x7f8a9806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 0x7f8a9806fd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b5c0 tx=0x7f8aa40058e0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 msgr2=0x7f8ab419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00d8d0 tx=0x7f8a9c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 shutdown_connections 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8a9806d8a0 0x7f8a9806fd50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4106b80 0x7f8ab419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 --2- 172.21.15.57:0/2060675441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab410c8f0 0x7f8ab419ca40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 >> 172.21.15.57:0/2060675441 conn(0x7f8ab4076060 msgr2=0x7f8ab4077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.104+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 shutdown_connections 2024-02-28T23:36:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.105+0000 7f8aba800700 1 -- 172.21.15.57:0/2060675441 wait complete. 2024-02-28T23:36:50.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 -- 172.21.15.57:0/915979968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac0103e50 msgr2=0x7f8ac01042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/915979968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac0103e50 0x7f8ac01042c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009910 tx=0x7f8aac009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:50.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 -- 172.21.15.57:0/915979968 shutdown_connections 2024-02-28T23:36:50.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/915979968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac0103e50 0x7f8ac01042c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/915979968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ac01025f0 0x7f8ac01029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 -- 172.21.15.57:0/915979968 >> 172.21.15.57:0/915979968 conn(0x7f8ac00fddb0 msgr2=0x7f8ac01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 -- 172.21.15.57:0/915979968 shutdown_connections 2024-02-28T23:36:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.261+0000 7f8ac4a00700 1 -- 172.21.15.57:0/915979968 wait complete. 2024-02-28T23:36:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.262+0000 7f8ac4a00700 1 Processor -- start 2024-02-28T23:36:50.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8ac4a00700 1 -- start start 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8ac4a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8ac4a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ac0103e50 0x7f8ac01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8ac4a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0198ac0 con 0x7f8ac01025f0 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8ac4a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac019c850 con 0x7f8ac0103e50 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8abe800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:50.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8abe800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58944/0 (socket says 172.21.15.57:58944) 2024-02-28T23:36:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.263+0000 7f8abe800700 1 -- 172.21.15.57:0/1889020554 learned_addr learned my addr 172.21.15.57:0/1889020554 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8abe800700 1 -- 172.21.15.57:0/1889020554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ac0103e50 msgr2=0x7f8ac01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8abe800700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ac0103e50 0x7f8ac01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8abe800700 1 -- 172.21.15.57:0/1889020554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0095f0 con 0x7f8ac01025f0 2024-02-28T23:36:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8abe800700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000d8d0 tx=0x7f8ab000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ab0009940 con 0x7f8ac01025f0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ab0010460 con 0x7f8ac01025f0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ab000f5f0 con 0x7f8ac01025f0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.264+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac019cb30 con 0x7f8ac01025f0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.265+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac019d100 con 0x7f8ac01025f0 2024-02-28T23:36:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.266+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac0062380 con 0x7f8ac01025f0 2024-02-28T23:36:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.267+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ab00105d0 con 0x7f8ac01025f0 2024-02-28T23:36:50.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.268+0000 7f8ab7400700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 0x7f8aa406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.268+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ab008d5d0 con 0x7f8ac01025f0 2024-02-28T23:36:50.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.269+0000 7f8abde00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 0x7f8aa406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:50.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.269+0000 7f8abde00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 0x7f8aa406fd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006010 tx=0x7f8aac00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.273+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ab0056ce0 con 0x7f8ac01025f0 2024-02-28T23:36:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.504+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ac019d430 con 0x7f8aa406d8a0 2024-02-28T23:36:50.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.537+0000 7f8ab7400700 1 -- 172.21.15.57:0/1889020554 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8ac019d430 con 0x7f8aa406d8a0 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (26m) 2m ago 40m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (36m) 2m ago 40m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (39m) 2m ago 39m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (22m) 2m ago 40m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (22m) 2m ago 39m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (25m) 2m ago 39m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (18m) 2m ago 33m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (18m) 2m ago 32m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (2m) 2m ago 32m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:36:50.545 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (17m) 2m ago 33m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (28m) 2m ago 42m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (27m) 2m ago 38m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (24m) 2m ago 42m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (23m) 2m ago 38m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (27m) 2m ago 40m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (27m) 2m ago 38m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (21m) 2m ago 37m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (21m) 2m ago 37m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (20m) 2m ago 36m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (20m) 2m ago 36m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (19m) 2m ago 35m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (18m) 2m ago 34m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:36:50.546 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (26m) 2m ago 39m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 msgr2=0x7f8aa406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 0x7f8aa406fd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006010 tx=0x7f8aac00b540 comp rx=0 tx=0).stop 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 msgr2=0x7f8ac0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000d8d0 tx=0x7f8ab000dc90 comp rx=0 tx=0).stop 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 shutdown_connections 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8aa406d8a0 0x7f8aa406fd50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ac01025f0 0x7f8ac0197ea0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 --2- 172.21.15.57:0/1889020554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ac0103e50 0x7f8ac01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 >> 172.21.15.57:0/1889020554 conn(0x7f8ac00fddb0 msgr2=0x7f8ac00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:50.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 shutdown_connections 2024-02-28T23:36:50.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.541+0000 7f8ac4a00700 1 -- 172.21.15.57:0/1889020554 wait complete. 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 -- 172.21.15.57:0/2586281010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 msgr2=0x7f7a681028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 --2- 172.21.15.57:0/2586281010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a681028a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009910 tx=0x7f7a58009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 -- 172.21.15.57:0/2586281010 shutdown_connections 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 --2- 172.21.15.57:0/2586281010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 0x7f7a68106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 --2- 172.21.15.57:0/2586281010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a681028a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.686+0000 7f7a67e00700 1 -- 172.21.15.57:0/2586281010 >> 172.21.15.57:0/2586281010 conn(0x7f7a680fdd90 msgr2=0x7f7a681001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:50.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.687+0000 7f7a67e00700 1 -- 172.21.15.57:0/2586281010 shutdown_connections 2024-02-28T23:36:50.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.687+0000 7f7a67e00700 1 -- 172.21.15.57:0/2586281010 wait complete. 2024-02-28T23:36:50.686 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:50 smithi057 ceph-mon[89491]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:50.686 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:50 smithi057 ceph-mon[89491]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 Processor -- start 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 -- start start 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 0x7f7a681983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68198a80 con 0x7f7a681024d0 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a67e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6819c810 con 0x7f7a68102e70 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.688+0000 7f7a66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58966/0 (socket says 172.21.15.57:58966) 2024-02-28T23:36:50.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 -- 172.21.15.57:0/3872135017 learned_addr learned my addr 172.21.15.57:0/3872135017 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:50.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66000700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 0x7f7a681983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:50.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 -- 172.21.15.57:0/3872135017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 msgr2=0x7f7a681983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:50.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 0x7f7a681983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:50.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 -- 172.21.15.57:0/3872135017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580095f0 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.689+0000 7f7a66a00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a580092e0 tx=0x7f7a5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.690+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5801d070 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.690+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a58005e00 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.690+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a58017400 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.690+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6819ca90 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.690+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6819cf30 con 0x7f7a681024d0 2024-02-28T23:36:50.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.691+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6810a7e0 con 0x7f7a681024d0 2024-02-28T23:36:50.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.692+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7a5800f460 con 0x7f7a681024d0 2024-02-28T23:36:50.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.693+0000 7f7a5f400700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 0x7f7a4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:50.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.693+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7a5808eb00 con 0x7f7a681024d0 2024-02-28T23:36:50.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.693+0000 7f7a66000700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 0x7f7a4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:50.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.694+0000 7f7a66000700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 0x7f7a4c06fd50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005950 tx=0x7f7a540058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:50.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:50.698+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a58058210 con 0x7f7a681024d0 2024-02-28T23:36:50.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:50 smithi084 ceph-mon[70440]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:50.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:50 smithi084 ceph-mon[70440]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.009+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7a68062380 con 0x7f7a681024d0 2024-02-28T23:36:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.010+0000 7f7a5f400700 1 -- 172.21.15.57:0/3872135017 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7a5805b8d0 con 0x7f7a681024d0 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:36:51.009 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:36:51.010 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:36:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.013+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 msgr2=0x7f7a4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.013+0000 7f7a67e00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 0x7f7a4c06fd50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005950 tx=0x7f7a540058e0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.013+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 msgr2=0x7f7a68197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.013+0000 7f7a67e00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a580092e0 tx=0x7f7a5800be20 comp rx=0 tx=0).stop 2024-02-28T23:36:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.013+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 shutdown_connections 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7a4c06d8a0 0x7f7a4c06fd50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a681024d0 0x7f7a68197e60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 --2- 172.21.15.57:0/3872135017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a68102e70 0x7f7a681983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 >> 172.21.15.57:0/3872135017 conn(0x7f7a680fdd90 msgr2=0x7f7a68100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 shutdown_connections 2024-02-28T23:36:51.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.014+0000 7f7a67e00700 1 -- 172.21.15.57:0/3872135017 wait complete. 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 -- 172.21.15.57:0/624450176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2050100e80 msgr2=0x7f2050104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 --2- 172.21.15.57:0/624450176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2050100e80 0x7f2050104e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f203c009910 tx=0x7f203c009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 -- 172.21.15.57:0/624450176 shutdown_connections 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 --2- 172.21.15.57:0/624450176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2050100e80 0x7f2050104e50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 --2- 172.21.15.57:0/624450176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20501004e0 0x7f20501008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.178+0000 7f2057e00700 1 -- 172.21.15.57:0/624450176 >> 172.21.15.57:0/624450176 conn(0x7f20500fbda0 msgr2=0x7f20500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:51.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.179+0000 7f2057e00700 1 -- 172.21.15.57:0/624450176 shutdown_connections 2024-02-28T23:36:51.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.179+0000 7f2057e00700 1 -- 172.21.15.57:0/624450176 wait complete. 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 Processor -- start 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 -- start start 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 0x7f2050073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050073ad0 con 0x7f20501004e0 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2057e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050073c10 con 0x7f2050100e80 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.180+0000 7f2055400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58992/0 (socket says 172.21.15.57:58992) 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 -- 172.21.15.57:0/1450882663 learned_addr learned my addr 172.21.15.57:0/1450882663 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2054a00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 0x7f2050073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 -- 172.21.15.57:0/1450882663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 msgr2=0x7f2050073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 0x7f2050073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 -- 172.21.15.57:0/1450882663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0095f0 con 0x7f20501004e0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2054a00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 0x7f2050073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.181+0000 7f2055400700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f204400d8d0 tx=0x7f204400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.182+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2044009940 con 0x7f20501004e0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.182+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2044010460 con 0x7f20501004e0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.182+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f204400f5f0 con 0x7f20501004e0 2024-02-28T23:36:51.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.182+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501a2100 con 0x7f20501004e0 2024-02-28T23:36:51.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.182+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a2510 con 0x7f20501004e0 2024-02-28T23:36:51.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.183+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050108850 con 0x7f20501004e0 2024-02-28T23:36:51.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.184+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f20440105d0 con 0x7f20501004e0 2024-02-28T23:36:51.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.185+0000 7f204e000700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 0x7f203806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.185+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f204408d4c0 con 0x7f20501004e0 2024-02-28T23:36:51.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.185+0000 7f2054a00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 0x7f203806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.186+0000 7f2054a00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 0x7f203806fd50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:51.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.190+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2044056c50 con 0x7f20501004e0 2024-02-28T23:36:51.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.463+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f205004f2a0 con 0x7f20501004e0 2024-02-28T23:36:51.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.464+0000 7f204e000700 1 -- 172.21.15.57:0/1450882663 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f2044016740 con 0x7f20501004e0 2024-02-28T23:36:51.463 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:36:51.463 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:36:51.463 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:36:51.463 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:36:51.463 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:36:51.464 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:36:51.465 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:36:51.466 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:51.467 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:36:51.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.466+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 msgr2=0x7f203806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.466+0000 7f2057e00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 0x7f203806fd50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c0053a0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 msgr2=0x7f2050073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f204400d8d0 tx=0x7f204400dc90 comp rx=0 tx=0).stop 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 shutdown_connections 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f203806d8a0 0x7f203806fd50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20501004e0 0x7f2050073050 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 --2- 172.21.15.57:0/1450882663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2050100e80 0x7f2050073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 >> 172.21.15.57:0/1450882663 conn(0x7f20500fbda0 msgr2=0x7f20500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 shutdown_connections 2024-02-28T23:36:51.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.467+0000 7f2057e00700 1 -- 172.21.15.57:0/1450882663 wait complete. 2024-02-28T23:36:51.470 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:36:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 -- 172.21.15.57:0/832975005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b8106b80 msgr2=0x7fe6b8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 --2- 172.21.15.57:0/832975005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b8106b80 0x7fe6b8106ff0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009910 tx=0x7fe6a4009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 -- 172.21.15.57:0/832975005 shutdown_connections 2024-02-28T23:36:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 --2- 172.21.15.57:0/832975005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b8106b80 0x7fe6b8106ff0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 --2- 172.21.15.57:0/832975005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b810c8f0 0x7fe6b810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.615+0000 7fe6c0600700 1 -- 172.21.15.57:0/832975005 >> 172.21.15.57:0/832975005 conn(0x7fe6b8076060 msgr2=0x7fe6b8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.616+0000 7fe6c0600700 1 -- 172.21.15.57:0/832975005 shutdown_connections 2024-02-28T23:36:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.616+0000 7fe6c0600700 1 -- 172.21.15.57:0/832975005 wait complete. 2024-02-28T23:36:51.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 Processor -- start 2024-02-28T23:36:51.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 -- start start 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 0x7fe6b819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b819d100 con 0x7fe6b810c8f0 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6c0600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b81a0e90 con 0x7fe6b8106b80 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.617+0000 7fe6bd200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59010/0 (socket says 172.21.15.57:59010) 2024-02-28T23:36:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 -- 172.21.15.57:0/1073317659 learned_addr learned my addr 172.21.15.57:0/1073317659 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bdc00700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 0x7fe6b819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 -- 172.21.15.57:0/1073317659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 msgr2=0x7fe6b819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 0x7fe6b819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 -- 172.21.15.57:0/1073317659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40095f0 con 0x7fe6b810c8f0 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bdc00700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 0x7fe6b819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.618+0000 7fe6bd200700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b5c0 tx=0x7fe6a400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.619+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6a401d070 con 0x7fe6b810c8f0 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.619+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6a400f460 con 0x7fe6b810c8f0 2024-02-28T23:36:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.619+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6a4017610 con 0x7fe6b810c8f0 2024-02-28T23:36:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.619+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b81a1110 con 0x7fe6b810c8f0 2024-02-28T23:36:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.619+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81a1600 con 0x7fe6b810c8f0 2024-02-28T23:36:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.620+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b8109b20 con 0x7fe6b810c8f0 2024-02-28T23:36:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.622+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe6a400f5d0 con 0x7fe6b810c8f0 2024-02-28T23:36:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.623+0000 7fe6b6000700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 0x7fe6a006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.623+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe6a408eb30 con 0x7fe6b810c8f0 2024-02-28T23:36:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.623+0000 7fe6bdc00700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 0x7fe6a006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.624+0000 7fe6bdc00700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 0x7fe6a006fd50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac007900 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.628+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6a40582c0 con 0x7fe6b810c8f0 2024-02-28T23:36:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:51 smithi057 ceph-mon[89491]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:51 smithi057 ceph-mon[89491]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:51.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:51 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3872135017' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:36:51.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:51 smithi084 ceph-mon[70440]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:51 smithi084 ceph-mon[70440]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:51.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:51 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3872135017' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:36:51.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.868+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b81a18e0 con 0x7fe6a006d8a0 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.871+0000 7fe6b6000700 1 -- 172.21.15.57:0/1073317659 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe6b81a18e0 con 0x7fe6a006d8a0 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:36:51.869 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:36:51.870 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 msgr2=0x7fe6a006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 0x7fe6a006fd50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac007900 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 msgr2=0x7fe6b819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b5c0 tx=0x7fe6a400be00 comp rx=0 tx=0).stop 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 shutdown_connections 2024-02-28T23:36:51.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe6a006d8a0 0x7fe6a006fd50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b8106b80 0x7fe6b819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 --2- 172.21.15.57:0/1073317659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6b810c8f0 0x7fe6b819ca20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:51.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 >> 172.21.15.57:0/1073317659 conn(0x7fe6b8076060 msgr2=0x7fe6b8077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:51.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 shutdown_connections 2024-02-28T23:36:51.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:51.874+0000 7fe6c0600700 1 -- 172.21.15.57:0/1073317659 wait complete. 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 -- 172.21.15.57:0/1849308861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 msgr2=0x7fa7d010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 --2- 172.21.15.57:0/1849308861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d010ccc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009910 tx=0x7fa7c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 -- 172.21.15.57:0/1849308861 shutdown_connections 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 --2- 172.21.15.57:0/1849308861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 0x7fa7d0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 --2- 172.21.15.57:0/1849308861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d010ccc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 -- 172.21.15.57:0/1849308861 >> 172.21.15.57:0/1849308861 conn(0x7fa7d0076060 msgr2=0x7fa7d0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 -- 172.21.15.57:0/1849308861 shutdown_connections 2024-02-28T23:36:52.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.038+0000 7fa7d7600700 1 -- 172.21.15.57:0/1849308861 wait complete. 2024-02-28T23:36:52.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 Processor -- start 2024-02-28T23:36:52.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 -- start start 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 0x7fa7d019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d019d090 con 0x7fa7d010c8f0 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.039+0000 7fa7d7600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d019d1d0 con 0x7fa7d0106b80 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59022/0 (socket says 172.21.15.57:59022) 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 -- 172.21.15.57:0/2572646111 learned_addr learned my addr 172.21.15.57:0/2572646111 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:36:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7d4c00700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 0x7fa7d019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 -- 172.21.15.57:0/2572646111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 msgr2=0x7fa7d019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 0x7fa7d019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.040+0000 7fa7cfe00700 1 -- 172.21.15.57:0/2572646111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40095f0 con 0x7fa7d010c8f0 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7cfe00700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c930 tx=0x7fa7c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c0007ad0 con 0x7fa7d010c8f0 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7c0007c30 con 0x7fa7d010c8f0 2024-02-28T23:36:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c0018610 con 0x7fa7d010c8f0 2024-02-28T23:36:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d01a1020 con 0x7fa7d010c8f0 2024-02-28T23:36:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.041+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d01a1520 con 0x7fa7d010c8f0 2024-02-28T23:36:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.042+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d0109b20 con 0x7fa7d010c8f0 2024-02-28T23:36:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.044+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa7c000f450 con 0x7fa7d010c8f0 2024-02-28T23:36:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.045+0000 7fa7cd600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 0x7fa7b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:36:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.045+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa7c008d720 con 0x7fa7d010c8f0 2024-02-28T23:36:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.045+0000 7fa7d4c00700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 0x7fa7b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:36:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.046+0000 7fa7d4c00700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 0x7fa7b806fd50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400b5c0 tx=0x7fa7c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:36:52.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.050+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7c0056eb0 con 0x7fa7d010c8f0 2024-02-28T23:36:52.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.372+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa7d004f2a0 con 0x7fa7d010c8f0 2024-02-28T23:36:52.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.373+0000 7fa7cd600700 1 -- 172.21.15.57:0/2572646111 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fa7c005a570 con 0x7fa7d010c8f0 2024-02-28T23:36:52.371 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:36:52.371 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:36:52.371 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:36:52.372 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:36:52.373 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.376+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 msgr2=0x7fa7b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.376+0000 7fa7d7600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 0x7fa7b806fd50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400b5c0 tx=0x7fa7c401a040 comp rx=0 tx=0).stop 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 msgr2=0x7fa7d019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c930 tx=0x7fa7c000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 shutdown_connections 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7b806d8a0 0x7fa7b806fd50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0106b80 0x7fa7d019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 --2- 172.21.15.57:0/2572646111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d010c8f0 0x7fa7d019ca40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:36:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 >> 172.21.15.57:0/2572646111 conn(0x7fa7d0076060 msgr2=0x7fa7d0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:36:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 shutdown_connections 2024-02-28T23:36:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:36:52.377+0000 7fa7d7600700 1 -- 172.21.15.57:0/2572646111 wait complete. 2024-02-28T23:36:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:52 smithi057 ceph-mon[89491]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:52 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1450882663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:36:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:52 smithi057 ceph-mon[89491]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:52 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2572646111' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:36:52.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:52 smithi084 ceph-mon[70440]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:52.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:52 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1450882663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:36:52.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:52 smithi084 ceph-mon[70440]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:36:52.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:52 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2572646111' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:36:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:54 smithi057 ceph-mon[89491]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:54.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:54 smithi084 ceph-mon[70440]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:56.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:56 smithi057 ceph-mon[89491]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:56.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:56 smithi084 ceph-mon[70440]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:58.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:36:58 smithi057 ceph-mon[89491]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:36:58.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:36:58 smithi084 ceph-mon[70440]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:00.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:00 smithi057 ceph-mon[89491]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:00.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:00 smithi084 ceph-mon[70440]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:02.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:02 smithi057 ceph-mon[89491]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:02.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:02 smithi084 ceph-mon[70440]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:03.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:03.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:04.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:04 smithi057 ceph-mon[89491]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:04.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:04 smithi084 ceph-mon[70440]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:06.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:06 smithi057 ceph-mon[89491]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:06.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:06 smithi084 ceph-mon[70440]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:08.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:08 smithi057 ceph-mon[89491]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:08.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:08 smithi084 ceph-mon[70440]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:10.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:10 smithi057 ceph-mon[89491]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:10.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:10 smithi084 ceph-mon[70440]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:12.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:12 smithi057 ceph-mon[89491]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:12.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:12 smithi084 ceph-mon[70440]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:14.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:14 smithi057 ceph-mon[89491]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:14.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:14 smithi084 ceph-mon[70440]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:16.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:16 smithi057 ceph-mon[89491]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:16.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:16 smithi084 ceph-mon[70440]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:18 smithi057 ceph-mon[89491]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:18.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:18.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:18 smithi084 ceph-mon[70440]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:18.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:20.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:20 smithi057 ceph-mon[89491]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:20.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:20 smithi084 ceph-mon[70440]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 -- 172.21.15.57:0/4179336119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde1010c8f0 msgr2=0x7fde1010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 --2- 172.21.15.57:0/4179336119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde1010c8f0 0x7fde1010ccc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fde04009910 tx=0x7fde04009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 -- 172.21.15.57:0/4179336119 shutdown_connections 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 --2- 172.21.15.57:0/4179336119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde10106b80 0x7fde10106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 --2- 172.21.15.57:0/4179336119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde1010c8f0 0x7fde1010ccc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 -- 172.21.15.57:0/4179336119 >> 172.21.15.57:0/4179336119 conn(0x7fde10076060 msgr2=0x7fde10078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 -- 172.21.15.57:0/4179336119 shutdown_connections 2024-02-28T23:37:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.548+0000 7fde17600700 1 -- 172.21.15.57:0/4179336119 wait complete. 2024-02-28T23:37:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.549+0000 7fde17600700 1 Processor -- start 2024-02-28T23:37:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde17600700 1 -- start start 2024-02-28T23:37:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde17600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde17600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 0x7fde1019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde17600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1019d120 con 0x7fde10106b80 2024-02-28T23:37:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde17600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde101a0eb0 con 0x7fde1010c8f0 2024-02-28T23:37:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.550+0000 7fde14c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde14c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41350/0 (socket says 172.21.15.57:41350) 2024-02-28T23:37:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde14c00700 1 -- 172.21.15.57:0/3583397687 learned_addr learned my addr 172.21.15.57:0/3583397687 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde0fe00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 0x7fde1019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde14c00700 1 -- 172.21.15.57:0/3583397687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 msgr2=0x7fde1019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde14c00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 0x7fde1019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde14c00700 1 -- 172.21.15.57:0/3583397687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde040095f0 con 0x7fde10106b80 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.551+0000 7fde0fe00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 0x7fde1019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde14c00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fde040092e0 tx=0x7fde0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0401d070 con 0x7fde10106b80 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde04005e00 con 0x7fde10106b80 2024-02-28T23:37:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde101a1130 con 0x7fde10106b80 2024-02-28T23:37:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde04017400 con 0x7fde10106b80 2024-02-28T23:37:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.552+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde101a1620 con 0x7fde10106b80 2024-02-28T23:37:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.553+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10109b20 con 0x7fde10106b80 2024-02-28T23:37:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.554+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fde0400f460 con 0x7fde10106b80 2024-02-28T23:37:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.555+0000 7fde0d600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 0x7fddf806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.555+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde0408eb00 con 0x7fde10106b80 2024-02-28T23:37:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.556+0000 7fde0fe00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 0x7fddf806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.556+0000 7fde0fe00700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 0x7fddf806fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde00005950 tx=0x7fde000058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.561+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde04058210 con 0x7fde10106b80 2024-02-28T23:37:22.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.813+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1019d970 con 0x7fddf806d8a0 2024-02-28T23:37:22.813 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:22 smithi057 ceph-mon[89491]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:22.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.815+0000 7fde0d600700 1 -- 172.21.15.57:0/3583397687 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fde1019d970 con 0x7fddf806d8a0 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 msgr2=0x7fddf806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 0x7fddf806fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde00005950 tx=0x7fde000058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 msgr2=0x7fde1019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fde040092e0 tx=0x7fde0400be20 comp rx=0 tx=0).stop 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 shutdown_connections 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fddf806d8a0 0x7fddf806fd50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde10106b80 0x7fde1019c500 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.818+0000 7fde17600700 1 --2- 172.21.15.57:0/3583397687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde1010c8f0 0x7fde1019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.819+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 >> 172.21.15.57:0/3583397687 conn(0x7fde10076060 msgr2=0x7fde10077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.819+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 shutdown_connections 2024-02-28T23:37:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.819+0000 7fde17600700 1 -- 172.21.15.57:0/3583397687 wait complete. 2024-02-28T23:37:22.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:22 smithi084 ceph-mon[70440]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:22.834 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:37:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 -- 172.21.15.57:0/2412916837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101b40 msgr2=0x7fa38010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 --2- 172.21.15.57:0/2412916837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101b40 0x7fa38010a040 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009910 tx=0x7fa36c009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 -- 172.21.15.57:0/2412916837 shutdown_connections 2024-02-28T23:37:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 --2- 172.21.15.57:0/2412916837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101b40 0x7fa38010a040 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 --2- 172.21.15.57:0/2412916837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101230 0x7fa380101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 -- 172.21.15.57:0/2412916837 >> 172.21.15.57:0/2412916837 conn(0x7fa3800faa90 msgr2=0x7fa3800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 -- 172.21.15.57:0/2412916837 shutdown_connections 2024-02-28T23:37:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.976+0000 7fa386800700 1 -- 172.21.15.57:0/2412916837 wait complete. 2024-02-28T23:37:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 Processor -- start 2024-02-28T23:37:22.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 -- start start 2024-02-28T23:37:22.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101b40 0x7fa380198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380198a30 con 0x7fa380101230 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.978+0000 7fa386800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019c7c0 con 0x7fa380101b40 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41366/0 (socket says 172.21.15.57:41366) 2024-02-28T23:37:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 -- 172.21.15.57:0/179724240 learned_addr learned my addr 172.21.15.57:0/179724240 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa384a00700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101b40 0x7fa380198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 -- 172.21.15.57:0/179724240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101b40 msgr2=0x7fa380198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101b40 0x7fa380198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.979+0000 7fa385400700 1 -- 172.21.15.57:0/179724240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0095f0 con 0x7fa380101230 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa385400700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c8f0 tx=0x7fa37400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa374007a70 con 0x7fa380101230 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa374004cf0 con 0x7fa380101230 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa374005610 con 0x7fa380101230 2024-02-28T23:37:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38019caa0 con 0x7fa380101230 2024-02-28T23:37:22.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.980+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38019cff0 con 0x7fa380101230 2024-02-28T23:37:22.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.981+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa380109b10 con 0x7fa380101230 2024-02-28T23:37:22.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.982+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa374004750 con 0x7fa380101230 2024-02-28T23:37:22.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.983+0000 7fa37e000700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 0x7fa36806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:22.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.983+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa374013070 con 0x7fa380101230 2024-02-28T23:37:22.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.983+0000 7fa384a00700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 0x7fa36806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:22.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.984+0000 7fa384a00700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 0x7fa36806fd50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa36c005f50 tx=0x7fa36c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:22.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:22.987+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa374056af0 con 0x7fa380101230 2024-02-28T23:37:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.229+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa38019d330 con 0x7fa36806d8a0 2024-02-28T23:37:23.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.232+0000 7fa37e000700 1 -- 172.21.15.57:0/179724240 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa38019d330 con 0x7fa36806d8a0 2024-02-28T23:37:23.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.234+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 msgr2=0x7fa36806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.234+0000 7fa386800700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 0x7fa36806fd50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa36c005f50 tx=0x7fa36c01a040 comp rx=0 tx=0).stop 2024-02-28T23:37:23.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.234+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 msgr2=0x7fa380197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.234+0000 7fa386800700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c8f0 tx=0x7fa37400ccb0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 shutdown_connections 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa36806d8a0 0x7fa36806fd50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380101230 0x7fa380197e10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 --2- 172.21.15.57:0/179724240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380101b40 0x7fa380198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 >> 172.21.15.57:0/179724240 conn(0x7fa3800faa90 msgr2=0x7fa3800ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:23.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 shutdown_connections 2024-02-28T23:37:23.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.235+0000 7fa386800700 1 -- 172.21.15.57:0/179724240 wait complete. 2024-02-28T23:37:23.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.389+0000 7f8bb1200700 1 -- 172.21.15.57:0/949149102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 msgr2=0x7f8bac104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.389+0000 7f8bb1200700 1 --2- 172.21.15.57:0/949149102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac104e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009910 tx=0x7f8b98009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.389+0000 7f8bb1200700 1 -- 172.21.15.57:0/949149102 shutdown_connections 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.389+0000 7f8bb1200700 1 --2- 172.21.15.57:0/949149102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac104e50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.390+0000 7f8bb1200700 1 --2- 172.21.15.57:0/949149102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 0x7f8bac1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.390+0000 7f8bb1200700 1 -- 172.21.15.57:0/949149102 >> 172.21.15.57:0/949149102 conn(0x7f8bac0fbda0 msgr2=0x7f8bac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.390+0000 7f8bb1200700 1 -- 172.21.15.57:0/949149102 shutdown_connections 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.390+0000 7f8bb1200700 1 -- 172.21.15.57:0/949149102 wait complete. 2024-02-28T23:37:23.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 Processor -- start 2024-02-28T23:37:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 -- start start 2024-02-28T23:37:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 0x7f8bac195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac196b50 con 0x7f8bac100e80 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.391+0000 7f8bb1200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac19a8e0 con 0x7f8bac1004e0 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41382/0 (socket says 172.21.15.57:41382) 2024-02-28T23:37:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 -- 172.21.15.57:0/3208779793 learned_addr learned my addr 172.21.15.57:0/3208779793 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9e00700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 0x7f8bac195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 -- 172.21.15.57:0/3208779793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 msgr2=0x7f8bac195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 0x7f8bac195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.392+0000 7f8ba9400700 1 -- 172.21.15.57:0/3208779793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b980095f0 con 0x7f8bac100e80 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8ba9400700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bd70 tx=0x7f8b9800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9801d070 con 0x7f8bac100e80 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b98005dc0 con 0x7f8bac100e80 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac19ab60 con 0x7f8bac100e80 2024-02-28T23:37:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b98017400 con 0x7f8bac100e80 2024-02-28T23:37:23.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.393+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac19b050 con 0x7f8bac100e80 2024-02-28T23:37:23.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.394+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac108850 con 0x7f8bac100e80 2024-02-28T23:37:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.395+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8b9800f7f0 con 0x7f8bac100e80 2024-02-28T23:37:23.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.396+0000 7f8ba2a00700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 0x7f8b9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.396+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8b9808e9f0 con 0x7f8bac100e80 2024-02-28T23:37:23.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.397+0000 7f8ba9e00700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 0x7f8b9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.397+0000 7f8ba9e00700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 0x7f8b9006fd50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005950 tx=0x7f8b9c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.401+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b98058180 con 0x7f8bac100e80 2024-02-28T23:37:23.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.647+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bac02cca0 con 0x7f8b9006d8a0 2024-02-28T23:37:23.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.666+0000 7f8ba2a00700 1 -- 172.21.15.57:0/3208779793 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8bac02cca0 con 0x7f8b9006d8a0 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (27m) 2m ago 40m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (37m) 2m ago 41m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (39m) 2m ago 39m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (23m) 2m ago 41m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (22m) 2m ago 39m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (26m) 2m ago 40m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (19m) 2m ago 33m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (18m) 2m ago 33m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:37:23.666 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (2m) 2m ago 33m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (18m) 2m ago 33m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (29m) 2m ago 42m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (28m) 2m ago 39m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (24m) 2m ago 43m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (24m) 2m ago 39m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (28m) 2m ago 41m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (27m) 2m ago 39m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (22m) 2m ago 38m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (21m) 2m ago 37m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (21m) 2m ago 37m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (20m) 2m ago 36m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (20m) 2m ago 36m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:37:23.667 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (19m) 2m ago 35m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:37:23.668 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (27m) 2m ago 40m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:37:23.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 msgr2=0x7f8b9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 0x7f8b9006fd50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005950 tx=0x7f8b9c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 msgr2=0x7f8bac196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bd70 tx=0x7f8b9800be50 comp rx=0 tx=0).stop 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 shutdown_connections 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8b9006d8a0 0x7f8b9006fd50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bac1004e0 0x7f8bac195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 --2- 172.21.15.57:0/3208779793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bac100e80 0x7f8bac196470 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 >> 172.21.15.57:0/3208779793 conn(0x7f8bac0fbda0 msgr2=0x7f8bac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 shutdown_connections 2024-02-28T23:37:23.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.669+0000 7f8bb1200700 1 -- 172.21.15.57:0/3208779793 wait complete. 2024-02-28T23:37:23.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:23 smithi057 ceph-mon[89491]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.826+0000 7f8b18800700 1 -- 172.21.15.57:0/1090874889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b100687a0 msgr2=0x7f8b10068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.826+0000 7f8b18800700 1 --2- 172.21.15.57:0/1090874889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b100687a0 0x7f8b10068b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009910 tx=0x7f8b04009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 -- 172.21.15.57:0/1090874889 shutdown_connections 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 --2- 172.21.15.57:0/1090874889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b10069140 0x7f8b10104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 --2- 172.21.15.57:0/1090874889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b100687a0 0x7f8b10068b70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 -- 172.21.15.57:0/1090874889 >> 172.21.15.57:0/1090874889 conn(0x7f8b10075130 msgr2=0x7f8b10075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 -- 172.21.15.57:0/1090874889 shutdown_connections 2024-02-28T23:37:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.827+0000 7f8b18800700 1 -- 172.21.15.57:0/1090874889 wait complete. 2024-02-28T23:37:23.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 Processor -- start 2024-02-28T23:37:23.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 -- start start 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b100687a0 0x7f8b10198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10198c90 con 0x7f8b10069140 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.828+0000 7f8b18800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1019ca20 con 0x7f8b100687a0 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41394/0 (socket says 172.21.15.57:41394) 2024-02-28T23:37:23.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 -- 172.21.15.57:0/1429488748 learned_addr learned my addr 172.21.15.57:0/1429488748 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15e00700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b100687a0 0x7f8b10198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 -- 172.21.15.57:0/1429488748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b100687a0 msgr2=0x7f8b10198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b100687a0 0x7f8b10198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.829+0000 7f8b15400700 1 -- 172.21.15.57:0/1429488748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040095f0 con 0x7f8b10069140 2024-02-28T23:37:23.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:23 smithi084 ceph-mon[70440]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b15400700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d8d0 tx=0x7f8b0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b00009940 con 0x7f8b10069140 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b00010460 con 0x7f8b10069140 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0000f5f0 con 0x7f8b10069140 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1019cd00 con 0x7f8b10069140 2024-02-28T23:37:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.830+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1019d250 con 0x7f8b10069140 2024-02-28T23:37:23.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.831+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1004f2a0 con 0x7f8b10069140 2024-02-28T23:37:23.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.832+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8b000105d0 con 0x7f8b10069140 2024-02-28T23:37:23.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.833+0000 7f8b0aa00700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 0x7f8af806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:23.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.833+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8b0008d4c0 con 0x7f8b10069140 2024-02-28T23:37:23.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.834+0000 7f8b15e00700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 0x7f8af806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:23.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.834+0000 7f8b15e00700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 0x7f8af806fd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005b40 tx=0x7f8b0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:23.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:23.838+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b00056c50 con 0x7f8b10069140 2024-02-28T23:37:24.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.163+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b101993d0 con 0x7f8b10069140 2024-02-28T23:37:24.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.164+0000 7f8b0aa00700 1 -- 172.21.15.57:0/1429488748 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8b0005a310 con 0x7f8b10069140 2024-02-28T23:37:24.163 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:37:24.163 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:37:24.163 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:37:24.163 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:37:24.164 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:37:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 msgr2=0x7f8af806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 0x7f8af806fd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005b40 tx=0x7f8b0401a040 comp rx=0 tx=0).stop 2024-02-28T23:37:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 msgr2=0x7f8b101985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d8d0 tx=0x7f8b0000dc90 comp rx=0 tx=0).stop 2024-02-28T23:37:24.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 shutdown_connections 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8af806d8a0 0x7f8af806fd50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b100687a0 0x7f8b10198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 --2- 172.21.15.57:0/1429488748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b10069140 0x7f8b101985b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 >> 172.21.15.57:0/1429488748 conn(0x7f8b10075130 msgr2=0x7f8b10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 shutdown_connections 2024-02-28T23:37:24.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.167+0000 7f8b18800700 1 -- 172.21.15.57:0/1429488748 wait complete. 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 -- 172.21.15.57:0/2512249811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe044106b80 msgr2=0x7fe044106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 --2- 172.21.15.57:0/2512249811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe044106b80 0x7fe044106ff0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe030009910 tx=0x7fe030009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 -- 172.21.15.57:0/2512249811 shutdown_connections 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 --2- 172.21.15.57:0/2512249811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe044106b80 0x7fe044106ff0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 --2- 172.21.15.57:0/2512249811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe04410c8f0 0x7fe04410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 -- 172.21.15.57:0/2512249811 >> 172.21.15.57:0/2512249811 conn(0x7fe044076060 msgr2=0x7fe044078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 -- 172.21.15.57:0/2512249811 shutdown_connections 2024-02-28T23:37:24.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.323+0000 7fe049000700 1 -- 172.21.15.57:0/2512249811 wait complete. 2024-02-28T23:37:24.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.324+0000 7fe049000700 1 Processor -- start 2024-02-28T23:37:24.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.324+0000 7fe049000700 1 -- start start 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe049000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 0x7fe04419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe049000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe049000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04419cfd0 con 0x7fe04410c8f0 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe049000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04419d110 con 0x7fe044106b80 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41402/0 (socket says 172.21.15.57:41402) 2024-02-28T23:37:24.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 -- 172.21.15.57:0/1161759666 learned_addr learned my addr 172.21.15.57:0/1161759666 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041e00700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 0x7fe04419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 -- 172.21.15.57:0/1161759666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 msgr2=0x7fe04419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 0x7fe04419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041400700 1 -- 172.21.15.57:0/1161759666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300095f0 con 0x7fe04410c8f0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.325+0000 7fe041e00700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 0x7fe04419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe041400700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe030006010 tx=0x7fe03000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03001d070 con 0x7fe04410c8f0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe030004fb0 con 0x7fe04410c8f0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe030022ea0 con 0x7fe04410c8f0 2024-02-28T23:37:24.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0441a0f00 con 0x7fe04410c8f0 2024-02-28T23:37:24.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.326+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0441a13f0 con 0x7fe04410c8f0 2024-02-28T23:37:24.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.327+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04410ecc0 con 0x7fe04410c8f0 2024-02-28T23:37:24.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.328+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe030022470 con 0x7fe04410c8f0 2024-02-28T23:37:24.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.329+0000 7fe03aa00700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 0x7fe02806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.330+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe03008ea50 con 0x7fe04410c8f0 2024-02-28T23:37:24.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.330+0000 7fe041e00700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 0x7fe02806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:24.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.330+0000 7fe041e00700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 0x7fe02806fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe034005fd0 tx=0x7fe034005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:24.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.334+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe030058290 con 0x7fe04410c8f0 2024-02-28T23:37:24.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.603+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe04404f2a0 con 0x7fe04410c8f0 2024-02-28T23:37:24.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.604+0000 7fe03aa00700 1 -- 172.21.15.57:0/1161759666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fe030022720 con 0x7fe04410c8f0 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:37:24.604 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:37:24.605 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:37:24.606 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:24.607 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:24.607 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:24.607 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:24.607 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:24.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 msgr2=0x7fe02806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 0x7fe02806fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe034005fd0 tx=0x7fe034005e20 comp rx=0 tx=0).stop 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 msgr2=0x7fe04419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe030006010 tx=0x7fe03000be20 comp rx=0 tx=0).stop 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 shutdown_connections 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe02806d8a0 0x7fe02806fd50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe044106b80 0x7fe04419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 --2- 172.21.15.57:0/1161759666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe04410c8f0 0x7fe04419c980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 >> 172.21.15.57:0/1161759666 conn(0x7fe044076060 msgr2=0x7fe044077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 shutdown_connections 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.607+0000 7fe049000700 1 -- 172.21.15.57:0/1161759666 wait complete. 2024-02-28T23:37:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:37:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 -- 172.21.15.57:0/4015750907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068ee0 msgr2=0x7f3404110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 --2- 172.21.15.57:0/4015750907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068ee0 0x7f3404110a10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009940 tx=0x7f33f0009c50 comp rx=0 tx=0).stop 2024-02-28T23:37:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 -- 172.21.15.57:0/4015750907 shutdown_connections 2024-02-28T23:37:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 --2- 172.21.15.57:0/4015750907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068ee0 0x7f3404110a10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 --2- 172.21.15.57:0/4015750907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404068540 0x7f3404068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 -- 172.21.15.57:0/4015750907 >> 172.21.15.57:0/4015750907 conn(0x7f3404075130 msgr2=0x7f3404075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 -- 172.21.15.57:0/4015750907 shutdown_connections 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.771+0000 7f340aa00700 1 -- 172.21.15.57:0/4015750907 wait complete. 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.772+0000 7f340aa00700 1 Processor -- start 2024-02-28T23:37:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f340aa00700 1 -- start start 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f340aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f340aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404068ee0 0x7f340419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f340aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340419d080 con 0x7f3404068540 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f340aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34041a0e10 con 0x7f3404068ee0 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f3403e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f3403e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41414/0 (socket says 172.21.15.57:41414) 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.773+0000 7f3403e00700 1 -- 172.21.15.57:0/1967762092 learned_addr learned my addr 172.21.15.57:0/1967762092 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3403e00700 1 -- 172.21.15.57:0/1967762092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404068ee0 msgr2=0x7f340419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3403e00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404068ee0 0x7f340419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3403e00700 1 -- 172.21.15.57:0/1967762092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00095f0 con 0x7f3404068540 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3403e00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d8d0 tx=0x7f33f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f4009940 con 0x7f3404068540 2024-02-28T23:37:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33f4010460 con 0x7f3404068540 2024-02-28T23:37:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34041a10f0 con 0x7f3404068540 2024-02-28T23:37:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f400f5f0 con 0x7f3404068540 2024-02-28T23:37:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.774+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34041a1640 con 0x7f3404068540 2024-02-28T23:37:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.776+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3404101410 con 0x7f3404068540 2024-02-28T23:37:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.777+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f33f40105d0 con 0x7f3404068540 2024-02-28T23:37:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.777+0000 7f3400c00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 0x7f33e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:24.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.778+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f33f408d530 con 0x7f3404068540 2024-02-28T23:37:24.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.778+0000 7f3403400700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 0x7f33e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:24.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.779+0000 7f3403400700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 0x7f33e806fd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006010 tx=0x7f33f0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:24.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:24.782+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33f4056ce0 con 0x7f3404068540 2024-02-28T23:37:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:24 smithi057 ceph-mon[89491]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:24 smithi057 ceph-mon[89491]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:24 smithi057 ceph-mon[89491]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:24 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1429488748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:37:24.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:24 smithi084 ceph-mon[70440]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:24.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:24 smithi084 ceph-mon[70440]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:24.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:24 smithi084 ceph-mon[70440]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:24.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:24 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1429488748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:37:25.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.028+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34041a1860 con 0x7f33e806d8a0 2024-02-28T23:37:25.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.031+0000 7f3400c00700 1 -- 172.21.15.57:0/1967762092 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f34041a1860 con 0x7f33e806d8a0 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:37:25.030 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:37:25.031 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 msgr2=0x7f33e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 0x7f33e806fd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006010 tx=0x7f33f0000bc0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 msgr2=0x7f340419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d8d0 tx=0x7f33f400dc90 comp rx=0 tx=0).stop 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 shutdown_connections 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d8a0 0x7f33e806fd50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404068540 0x7f340419c460 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 --2- 172.21.15.57:0/1967762092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404068ee0 0x7f340419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 >> 172.21.15.57:0/1967762092 conn(0x7f3404075130 msgr2=0x7f340410eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:25.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 shutdown_connections 2024-02-28T23:37:25.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.034+0000 7f340aa00700 1 -- 172.21.15.57:0/1967762092 wait complete. 2024-02-28T23:37:25.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 -- 172.21.15.57:0/3031293625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee40106b80 msgr2=0x7fee40106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 --2- 172.21.15.57:0/3031293625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee40106b80 0x7fee40106ff0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009910 tx=0x7fee2c009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 -- 172.21.15.57:0/3031293625 shutdown_connections 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 --2- 172.21.15.57:0/3031293625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee40106b80 0x7fee40106ff0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 --2- 172.21.15.57:0/3031293625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4010c8f0 0x7fee4010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.196+0000 7fee48000700 1 -- 172.21.15.57:0/3031293625 >> 172.21.15.57:0/3031293625 conn(0x7fee40076060 msgr2=0x7fee40078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.197+0000 7fee48000700 1 -- 172.21.15.57:0/3031293625 shutdown_connections 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.197+0000 7fee48000700 1 -- 172.21.15.57:0/3031293625 wait complete. 2024-02-28T23:37:25.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 Processor -- start 2024-02-28T23:37:25.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 -- start start 2024-02-28T23:37:25.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee40106b80 0x7fee4019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:25.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:25.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4019d060 con 0x7fee4010c8f0 2024-02-28T23:37:25.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.198+0000 7fee48000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a0df0 con 0x7fee40106b80 2024-02-28T23:37:25.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:25.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39438/0 (socket says 172.21.15.57:39438) 2024-02-28T23:37:25.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 -- 172.21.15.57:0/3548165460 learned_addr learned my addr 172.21.15.57:0/3548165460 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee45600700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee40106b80 0x7fee4019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 -- 172.21.15.57:0/3548165460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee40106b80 msgr2=0x7fee4019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee40106b80 0x7fee4019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.199+0000 7fee44c00700 1 -- 172.21.15.57:0/3548165460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c0095f0 con 0x7fee4010c8f0 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee44c00700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006010 tx=0x7fee2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:25.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2c01d070 con 0x7fee4010c8f0 2024-02-28T23:37:25.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2c00f460 con 0x7fee4010c8f0 2024-02-28T23:37:25.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2c017630 con 0x7fee4010c8f0 2024-02-28T23:37:25.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee401a1070 con 0x7fee4010c8f0 2024-02-28T23:37:25.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.200+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee401a1560 con 0x7fee4010c8f0 2024-02-28T23:37:25.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.201+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4010ecc0 con 0x7fee4010c8f0 2024-02-28T23:37:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.202+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fee2c00f5d0 con 0x7fee4010c8f0 2024-02-28T23:37:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.203+0000 7fee3e000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 0x7fee2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.204+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fee2c08ea30 con 0x7fee4010c8f0 2024-02-28T23:37:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.204+0000 7fee45600700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 0x7fee2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:25.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.205+0000 7fee45600700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 0x7fee2806fd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee34005950 tx=0x7fee340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:25.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.208+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee2c0581c0 con 0x7fee4010c8f0 2024-02-28T23:37:25.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.527+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fee4004f2a0 con 0x7fee4010c8f0 2024-02-28T23:37:25.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.528+0000 7fee3e000700 1 -- 172.21.15.57:0/3548165460 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fee2c05b880 con 0x7fee4010c8f0 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:37:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:37:25.529 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:37:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:37:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 msgr2=0x7fee2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 0x7fee2806fd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee34005950 tx=0x7fee340058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 msgr2=0x7fee4019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006010 tx=0x7fee2c00be20 comp rx=0 tx=0).stop 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 shutdown_connections 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fee2806d8a0 0x7fee2806fd50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee40106b80 0x7fee4019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 --2- 172.21.15.57:0/3548165460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee4010c8f0 0x7fee4019c980 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.531+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 >> 172.21.15.57:0/3548165460 conn(0x7fee40076060 msgr2=0x7fee40077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.532+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 shutdown_connections 2024-02-28T23:37:25.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:25.532+0000 7fee48000700 1 -- 172.21.15.57:0/3548165460 wait complete. 2024-02-28T23:37:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1161759666' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:37:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:25 smithi057 ceph-mon[89491]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:25.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3548165460' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:37:25.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1161759666' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:37:25.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:25 smithi084 ceph-mon[70440]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:25.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3548165460' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:37:26.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:26 smithi057 ceph-mon[89491]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:26.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:26 smithi084 ceph-mon[70440]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:28 smithi057 ceph-mon[89491]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:28.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:28 smithi084 ceph-mon[70440]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:30 smithi084 ceph-mon[70440]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:30.845 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:30 smithi057 ceph-mon[89491]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:32.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:32 smithi057 ceph-mon[89491]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:33.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:32 smithi084 ceph-mon[70440]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:34.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:34 smithi057 ceph-mon[89491]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:35.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:34 smithi084 ceph-mon[70440]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:36 smithi057 ceph-mon[89491]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:37.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:36 smithi084 ceph-mon[70440]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:38 smithi057 ceph-mon[89491]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:39.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:38 smithi084 ceph-mon[70440]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:40 smithi057 ceph-mon[89491]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:41.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:40 smithi084 ceph-mon[70440]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:43.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:42 smithi057 ceph-mon[89491]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:43.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:42 smithi084 ceph-mon[70440]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:44.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:44 smithi084 ceph-mon[70440]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:45.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:44 smithi057 ceph-mon[89491]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:47.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:46 smithi057 ceph-mon[89491]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:47.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:46 smithi084 ceph-mon[70440]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:48 smithi057 ceph-mon[89491]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:49.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:48 smithi084 ceph-mon[70440]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:49.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:37:50.981 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:50 smithi084 ceph-mon[70440]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:51.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:50 smithi057 ceph-mon[89491]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:53.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:52 smithi057 ceph-mon[89491]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:53.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:52 smithi084 ceph-mon[70440]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:54 smithi057 ceph-mon[89491]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:55.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:54 smithi084 ceph-mon[70440]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 -- 172.21.15.57:0/1885476829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 msgr2=0x7ff9dc106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 --2- 172.21.15.57:0/1885476829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc106e40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009910 tx=0x7ff9c8009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 -- 172.21.15.57:0/1885476829 shutdown_connections 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 --2- 172.21.15.57:0/1885476829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc106e40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 --2- 172.21.15.57:0/1885476829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 0x7ff9dc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 -- 172.21.15.57:0/1885476829 >> 172.21.15.57:0/1885476829 conn(0x7ff9dc0fdd90 msgr2=0x7ff9dc1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 -- 172.21.15.57:0/1885476829 shutdown_connections 2024-02-28T23:37:55.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.708+0000 7ff9e2400700 1 -- 172.21.15.57:0/1885476829 wait complete. 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.709+0000 7ff9e2400700 1 Processor -- start 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9e2400700 1 -- start start 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9e2400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 0x7ff9dc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9e2400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9e2400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc198a70 con 0x7ff9dc102e70 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9e2400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc19c800 con 0x7ff9dc1024d0 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9dbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38536/0 (socket says 172.21.15.57:38536) 2024-02-28T23:37:55.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.710+0000 7ff9dbe00700 1 -- 172.21.15.57:0/717651751 learned_addr learned my addr 172.21.15.57:0/717651751 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.711+0000 7ff9e1000700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 0x7ff9dc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.711+0000 7ff9dbe00700 1 -- 172.21.15.57:0/717651751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 msgr2=0x7ff9dc197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.711+0000 7ff9dbe00700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 0x7ff9dc197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.711+0000 7ff9dbe00700 1 -- 172.21.15.57:0/717651751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80095f0 con 0x7ff9dc102e70 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.711+0000 7ff9dbe00700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800bd70 tx=0x7ff9c800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.712+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c801d070 con 0x7ff9dc102e70 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.712+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c8005dc0 con 0x7ff9dc102e70 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.712+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc19ca80 con 0x7ff9dc102e70 2024-02-28T23:37:55.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.712+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc19cf70 con 0x7ff9dc102e70 2024-02-28T23:37:55.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.712+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c8017400 con 0x7ff9dc102e70 2024-02-28T23:37:55.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.713+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc005320 con 0x7ff9dc102e70 2024-02-28T23:37:55.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.714+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff9c8003a60 con 0x7ff9dc102e70 2024-02-28T23:37:55.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.715+0000 7ff9d9600700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:55.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.715+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff9c808e910 con 0x7ff9dc102e70 2024-02-28T23:37:55.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.715+0000 7ff9e1000700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:55.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.716+0000 7ff9e1000700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005950 tx=0x7ff9d00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:55.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.720+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9c80580a0 con 0x7ff9dc102e70 2024-02-28T23:37:55.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.969+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9bc000bf0 con 0x7ff9c406d8a0 2024-02-28T23:37:55.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.973+0000 7ff9d9600700 1 -- 172.21.15.57:0/717651751 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff9bc000bf0 con 0x7ff9c406d8a0 2024-02-28T23:37:55.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.975+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 msgr2=0x7ff9c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:55.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.975+0000 7ff9e2400700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005950 tx=0x7ff9d00058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.975+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 msgr2=0x7ff9dc198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:55.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.975+0000 7ff9e2400700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800bd70 tx=0x7ff9c800be50 comp rx=0 tx=0).stop 2024-02-28T23:37:55.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 shutdown_connections 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1024d0 0x7ff9dc197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 --2- 172.21.15.57:0/717651751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc102e70 0x7ff9dc198390 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 >> 172.21.15.57:0/717651751 conn(0x7ff9dc0fdd90 msgr2=0x7ff9dc1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 shutdown_connections 2024-02-28T23:37:55.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:55.976+0000 7ff9e2400700 1 -- 172.21.15.57:0/717651751 wait complete. 2024-02-28T23:37:55.990 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:37:56.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.136+0000 7fec23800700 1 -- 172.21.15.57:0/3978338340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c100e80 msgr2=0x7fec1c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.136+0000 7fec23800700 1 --2- 172.21.15.57:0/3978338340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c100e80 0x7fec1c104e50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fec08009910 tx=0x7fec08009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 -- 172.21.15.57:0/3978338340 shutdown_connections 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 --2- 172.21.15.57:0/3978338340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c100e80 0x7fec1c104e50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 --2- 172.21.15.57:0/3978338340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c1004e0 0x7fec1c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 -- 172.21.15.57:0/3978338340 >> 172.21.15.57:0/3978338340 conn(0x7fec1c0fbda0 msgr2=0x7fec1c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 -- 172.21.15.57:0/3978338340 shutdown_connections 2024-02-28T23:37:56.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.137+0000 7fec23800700 1 -- 172.21.15.57:0/3978338340 wait complete. 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 Processor -- start 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 -- start start 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 0x7fec1c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c198d30 con 0x7fec1c1004e0 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.138+0000 7fec23800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c19cac0 con 0x7fec1c100e80 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38554/0 (socket says 172.21.15.57:38554) 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 -- 172.21.15.57:0/3999632062 learned_addr learned my addr 172.21.15.57:0/3999632062 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec1be00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 0x7fec1c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 -- 172.21.15.57:0/3999632062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 msgr2=0x7fec1c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 0x7fec1c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec20e00700 1 -- 172.21.15.57:0/3999632062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec080095f0 con 0x7fec1c1004e0 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.139+0000 7fec1be00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 0x7fec1c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec20e00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d8d0 tx=0x7fec1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:56.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec10009940 con 0x7fec1c1004e0 2024-02-28T23:37:56.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec10010460 con 0x7fec1c1004e0 2024-02-28T23:37:56.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c19cda0 con 0x7fec1c1004e0 2024-02-28T23:37:56.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec1000f5f0 con 0x7fec1c1004e0 2024-02-28T23:37:56.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.140+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c19d2f0 con 0x7fec1c1004e0 2024-02-28T23:37:56.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.141+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c1045d0 con 0x7fec1c1004e0 2024-02-28T23:37:56.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.142+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fec100105d0 con 0x7fec1c1004e0 2024-02-28T23:37:56.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.143+0000 7fec19600700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 0x7fec0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.143+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec1008d5d0 con 0x7fec1c1004e0 2024-02-28T23:37:56.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.143+0000 7fec1be00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 0x7fec0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.144+0000 7fec1be00700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 0x7fec0406fd50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec08006010 tx=0x7fec0800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:56.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.148+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec10056ce0 con 0x7fec1c1004e0 2024-02-28T23:37:56.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.398+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec1c19d510 con 0x7fec0406d8a0 2024-02-28T23:37:56.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.402+0000 7fec19600700 1 -- 172.21.15.57:0/3999632062 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fec1c19d510 con 0x7fec0406d8a0 2024-02-28T23:37:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 msgr2=0x7fec0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 0x7fec0406fd50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec08006010 tx=0x7fec0800b540 comp rx=0 tx=0).stop 2024-02-28T23:37:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 msgr2=0x7fec1c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d8d0 tx=0x7fec1000dc90 comp rx=0 tx=0).stop 2024-02-28T23:37:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 shutdown_connections 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fec0406d8a0 0x7fec0406fd50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec1c1004e0 0x7fec1c198110 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 --2- 172.21.15.57:0/3999632062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1c100e80 0x7fec1c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.404+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 >> 172.21.15.57:0/3999632062 conn(0x7fec1c0fbda0 msgr2=0x7fec1c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.405+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 shutdown_connections 2024-02-28T23:37:56.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.405+0000 7fec23800700 1 -- 172.21.15.57:0/3999632062 wait complete. 2024-02-28T23:37:56.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 -- 172.21.15.57:0/3974034908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0108470 msgr2=0x7f1ca0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 --2- 172.21.15.57:0/3974034908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0108470 0x7f1ca0108840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009910 tx=0x7f1c94009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 -- 172.21.15.57:0/3974034908 shutdown_connections 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 --2- 172.21.15.57:0/3974034908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0102470 0x7f1ca01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 --2- 172.21.15.57:0/3974034908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0108470 0x7f1ca0108840 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.566+0000 7f1ca8600700 1 -- 172.21.15.57:0/3974034908 >> 172.21.15.57:0/3974034908 conn(0x7f1ca00fdf90 msgr2=0x7f1ca01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.567+0000 7f1ca8600700 1 -- 172.21.15.57:0/3974034908 shutdown_connections 2024-02-28T23:37:56.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.567+0000 7f1ca8600700 1 -- 172.21.15.57:0/3974034908 wait complete. 2024-02-28T23:37:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 Processor -- start 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 -- start start 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0108470 0x7f1ca0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0198e10 con 0x7f1ca0102470 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca8600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca019cba0 con 0x7f1ca0108470 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.568+0000 7f1ca5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38576/0 (socket says 172.21.15.57:38576) 2024-02-28T23:37:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 -- 172.21.15.57:0/61861828 learned_addr learned my addr 172.21.15.57:0/61861828 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5200700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0108470 0x7f1ca0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 -- 172.21.15.57:0/61861828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0108470 msgr2=0x7f1ca0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0108470 0x7f1ca0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 -- 172.21.15.57:0/61861828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940095f0 con 0x7f1ca0102470 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.569+0000 7f1ca5c00700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1c940092e0 tx=0x7f1c9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.570+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9401d070 con 0x7f1ca0102470 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.570+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c94005e00 con 0x7f1ca0102470 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.570+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c94017400 con 0x7f1ca0102470 2024-02-28T23:37:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.570+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca019ce20 con 0x7f1ca0102470 2024-02-28T23:37:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.570+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca019d310 con 0x7f1ca0102470 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.571+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca010a9e0 con 0x7f1ca0102470 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.572+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1c9400f460 con 0x7f1ca0102470 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.573+0000 7f1c9e000700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 0x7f1c8806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.573+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1c9408ea30 con 0x7f1ca0102470 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.573+0000 7f1ca5200700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 0x7f1c8806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.574+0000 7f1ca5200700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 0x7f1c8806fd50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005950 tx=0x7f1c8c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:56.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.577+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c940581c0 con 0x7f1ca0102470 2024-02-28T23:37:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.817+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ca019d580 con 0x7f1c8806d8a0 2024-02-28T23:37:56.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.831+0000 7f1c9e000700 1 -- 172.21.15.57:0/61861828 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1ca019d580 con 0x7f1c8806d8a0 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (27m) 3m ago 41m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (37m) 3m ago 41m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (40m) 3m ago 40m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (23m) 3m ago 41m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (23m) 3m ago 40m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:37:56.830 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (26m) 3m ago 41m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (19m) 3m ago 34m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (19m) 3m ago 34m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (3m) 3m ago 34m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (18m) 3m ago 34m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (30m) 3m ago 43m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (28m) 3m ago 40m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (25m) 3m ago 43m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (24m) 3m ago 39m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (28m) 3m ago 41m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:37:56.831 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (28m) 3m ago 40m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (22m) 3m ago 38m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (22m) 3m ago 38m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (21m) 3m ago 37m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (21m) 3m ago 37m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (20m) 3m ago 36m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (20m) 3m ago 35m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:37:56.832 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (28m) 3m ago 40m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:37:56.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 msgr2=0x7f1c8806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 0x7f1c8806fd50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005950 tx=0x7f1c8c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 msgr2=0x7f1ca01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1c940092e0 tx=0x7f1c9400be20 comp rx=0 tx=0).stop 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 shutdown_connections 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1c8806d8a0 0x7f1c8806fd50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ca0102470 0x7f1ca01981f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 --2- 172.21.15.57:0/61861828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca0108470 0x7f1ca0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.834+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 >> 172.21.15.57:0/61861828 conn(0x7f1ca00fdf90 msgr2=0x7f1ca01002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.835+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 shutdown_connections 2024-02-28T23:37:56.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.835+0000 7f1ca8600700 1 -- 172.21.15.57:0/61861828 wait complete. 2024-02-28T23:37:56.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.993+0000 7fde4c400700 1 -- 172.21.15.57:0/160932891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44106320 msgr2=0x7fde441066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.993+0000 7fde4c400700 1 --2- 172.21.15.57:0/160932891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44106320 0x7fde441066f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fde38009910 tx=0x7fde38009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:56.992 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:56 smithi057 ceph-mon[89491]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:56.993 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:56 smithi057 ceph-mon[89491]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:56.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 -- 172.21.15.57:0/160932891 shutdown_connections 2024-02-28T23:37:56.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 --2- 172.21.15.57:0/160932891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44068540 0x7fde440689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 --2- 172.21.15.57:0/160932891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44106320 0x7fde441066f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 -- 172.21.15.57:0/160932891 >> 172.21.15.57:0/160932891 conn(0x7fde44075130 msgr2=0x7fde44075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 -- 172.21.15.57:0/160932891 shutdown_connections 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.994+0000 7fde4c400700 1 -- 172.21.15.57:0/160932891 wait complete. 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.995+0000 7fde4c400700 1 Processor -- start 2024-02-28T23:37:56.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.995+0000 7fde4c400700 1 -- start start 2024-02-28T23:37:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde4c400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde4c400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44106320 0x7fde441986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde4c400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44198d90 con 0x7fde44068540 2024-02-28T23:37:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde4c400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4419cb20 con 0x7fde44106320 2024-02-28T23:37:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde49a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde49a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38588/0 (socket says 172.21.15.57:38588) 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.996+0000 7fde49a00700 1 -- 172.21.15.57:0/3054153895 learned_addr learned my addr 172.21.15.57:0/3054153895 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde49000700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44106320 0x7fde441986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde49a00700 1 -- 172.21.15.57:0/3054153895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44106320 msgr2=0x7fde441986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde49a00700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44106320 0x7fde441986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde49a00700 1 -- 172.21.15.57:0/3054153895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380095f0 con 0x7fde44068540 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde49a00700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde38006010 tx=0x7fde3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.997+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3801d070 con 0x7fde44068540 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.998+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde38004fb0 con 0x7fde44068540 2024-02-28T23:37:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.998+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3800f690 con 0x7fde44068540 2024-02-28T23:37:56.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.998+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4419cda0 con 0x7fde44068540 2024-02-28T23:37:56.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.998+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4419d290 con 0x7fde44068540 2024-02-28T23:37:56.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:56.999+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4404f2a0 con 0x7fde44068540 2024-02-28T23:37:56.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.000+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fde380228a0 con 0x7fde44068540 2024-02-28T23:37:56.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.001+0000 7fde42000700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 0x7fde2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:56.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.001+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde3808edd0 con 0x7fde44068540 2024-02-28T23:37:56.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.001+0000 7fde49000700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 0x7fde2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.002+0000 7fde49000700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 0x7fde2c06fd50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fde30006fd0 tx=0x7fde30008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:57.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.006+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde38058580 con 0x7fde44068540 2024-02-28T23:37:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:56 smithi084 ceph-mon[70440]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:56 smithi084 ceph-mon[70440]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:57.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.314+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fde4419d4e0 con 0x7fde44068540 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.315+0000 7fde42000700 1 -- 172.21.15.57:0/3054153895 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fde38027280 con 0x7fde44068540 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:37:57.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:37:57.315 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:37:57.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 msgr2=0x7fde2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 0x7fde2c06fd50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fde30006fd0 tx=0x7fde30008040 comp rx=0 tx=0).stop 2024-02-28T23:37:57.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 msgr2=0x7fde44198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde38006010 tx=0x7fde3800be20 comp rx=0 tx=0).stop 2024-02-28T23:37:57.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 shutdown_connections 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fde2c06d8a0 0x7fde2c06fd50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde44068540 0x7fde44198170 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 --2- 172.21.15.57:0/3054153895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde44106320 0x7fde441986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 >> 172.21.15.57:0/3054153895 conn(0x7fde44075130 msgr2=0x7fde440fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 shutdown_connections 2024-02-28T23:37:57.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.318+0000 7fde4c400700 1 -- 172.21.15.57:0/3054153895 wait complete. 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 -- 172.21.15.57:0/1770017673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a8106b80 msgr2=0x7fd5a8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 --2- 172.21.15.57:0/1770017673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a8106b80 0x7fd5a8106ff0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd594009910 tx=0x7fd594009c20 comp rx=0 tx=0).stop 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 -- 172.21.15.57:0/1770017673 shutdown_connections 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 --2- 172.21.15.57:0/1770017673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a8106b80 0x7fd5a8106ff0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 --2- 172.21.15.57:0/1770017673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a810c8f0 0x7fd5a810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.477+0000 7fd5afa00700 1 -- 172.21.15.57:0/1770017673 >> 172.21.15.57:0/1770017673 conn(0x7fd5a8076060 msgr2=0x7fd5a8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:57.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.478+0000 7fd5afa00700 1 -- 172.21.15.57:0/1770017673 shutdown_connections 2024-02-28T23:37:57.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.478+0000 7fd5afa00700 1 -- 172.21.15.57:0/1770017673 wait complete. 2024-02-28T23:37:57.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.478+0000 7fd5afa00700 1 Processor -- start 2024-02-28T23:37:57.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5afa00700 1 -- start start 2024-02-28T23:37:57.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5afa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a8106b80 0x7fd5a819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5afa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5afa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a819d060 con 0x7fd5a810c8f0 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5afa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81a0df0 con 0x7fd5a8106b80 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38592/0 (socket says 172.21.15.57:38592) 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.479+0000 7fd5a7e00700 1 -- 172.21.15.57:0/687283585 learned_addr learned my addr 172.21.15.57:0/687283585 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.480+0000 7fd5ad000700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a8106b80 0x7fd5a819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.480+0000 7fd5a7e00700 1 -- 172.21.15.57:0/687283585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a8106b80 msgr2=0x7fd5a819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.480+0000 7fd5a7e00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a8106b80 0x7fd5a819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.480+0000 7fd5a7e00700 1 -- 172.21.15.57:0/687283585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940095f0 con 0x7fd5a810c8f0 2024-02-28T23:37:57.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.480+0000 7fd5a7e00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b5c0 tx=0x7fd59400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.481+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59401d070 con 0x7fd5a810c8f0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.481+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd594005e00 con 0x7fd5a810c8f0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.481+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a81a1070 con 0x7fd5a810c8f0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.481+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd594017400 con 0x7fd5a810c8f0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.481+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a81a1560 con 0x7fd5a810c8f0 2024-02-28T23:37:57.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.482+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a810ecc0 con 0x7fd5a810c8f0 2024-02-28T23:37:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.483+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd59400f460 con 0x7fd5a810c8f0 2024-02-28T23:37:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.484+0000 7fd5a5600700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 0x7fd59006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.484+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd59408ea10 con 0x7fd5a810c8f0 2024-02-28T23:37:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.484+0000 7fd5ad000700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 0x7fd59006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.485+0000 7fd5ad000700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 0x7fd59006fd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005950 tx=0x7fd59c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.489+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5940581a0 con 0x7fd5a810c8f0 2024-02-28T23:37:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.767+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd5a804f2a0 con 0x7fd5a810c8f0 2024-02-28T23:37:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.768+0000 7fd5a5600700 1 -- 172.21.15.57:0/687283585 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fd59405b860 con 0x7fd5a810c8f0 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:37:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:37:57.768 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:37:57.769 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:37:57.770 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:57.771 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 msgr2=0x7fd59006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 0x7fd59006fd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005950 tx=0x7fd59c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 msgr2=0x7fd5a819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b5c0 tx=0x7fd59400be20 comp rx=0 tx=0).stop 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 shutdown_connections 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd59006d8a0 0x7fd59006fd50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a8106b80 0x7fd5a819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 --2- 172.21.15.57:0/687283585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5a810c8f0 0x7fd5a819c980 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 >> 172.21.15.57:0/687283585 conn(0x7fd5a8076060 msgr2=0x7fd5a8077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:57.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 shutdown_connections 2024-02-28T23:37:57.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.771+0000 7fd5afa00700 1 -- 172.21.15.57:0/687283585 wait complete. 2024-02-28T23:37:57.773 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:37:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.930+0000 7fb121e00700 1 -- 172.21.15.57:0/2055031282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 msgr2=0x7fb11c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.930+0000 7fb121e00700 1 --2- 172.21.15.57:0/2055031282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c073460 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009940 tx=0x7fb10c009c50 comp rx=0 tx=0).stop 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 -- 172.21.15.57:0/2055031282 shutdown_connections 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 --2- 172.21.15.57:0/2055031282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 0x7fb11c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 --2- 172.21.15.57:0/2055031282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c073460 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 -- 172.21.15.57:0/2055031282 >> 172.21.15.57:0/2055031282 conn(0x7fb11c0fbda0 msgr2=0x7fb11c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 -- 172.21.15.57:0/2055031282 shutdown_connections 2024-02-28T23:37:57.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.931+0000 7fb121e00700 1 -- 172.21.15.57:0/2055031282 wait complete. 2024-02-28T23:37:57.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.932+0000 7fb121e00700 1 Processor -- start 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.932+0000 7fb121e00700 1 -- start start 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb121e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb121e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 0x7fb11c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb121e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c198da0 con 0x7fb11c073090 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb121e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c19cb30 con 0x7fb11c0739a0 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38616/0 (socket says 172.21.15.57:38616) 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 -- 172.21.15.57:0/3055547663 learned_addr learned my addr 172.21.15.57:0/3055547663 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:57.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb112000700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 0x7fb11c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 -- 172.21.15.57:0/3055547663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 msgr2=0x7fb11c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 0x7fb11c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.933+0000 7fb11aa00700 1 -- 172.21.15.57:0/3055547663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0095f0 con 0x7fb11c073090 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb11aa00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0055f0 tx=0x7fb10c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10c01d070 con 0x7fb11c073090 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb10c00f460 con 0x7fb11c073090 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10c021640 con 0x7fb11c073090 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c19cdb0 con 0x7fb11c073090 2024-02-28T23:37:57.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.934+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c19d2a0 con 0x7fb11c073090 2024-02-28T23:37:57.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.935+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c109ce0 con 0x7fb11c073090 2024-02-28T23:37:57.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.937+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb10c00fab0 con 0x7fb11c073090 2024-02-28T23:37:57.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.937+0000 7fb120a00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 0x7fb10006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:57.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.938+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb10c08e6f0 con 0x7fb11c073090 2024-02-28T23:37:57.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.938+0000 7fb112000700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 0x7fb10006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:57.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.938+0000 7fb112000700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 0x7fb10006fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb104005950 tx=0x7fb1040058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:57.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:57.942+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb10c057e80 con 0x7fb11c073090 2024-02-28T23:37:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:57 smithi057 ceph-mon[89491]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:57 smithi057 ceph-mon[89491]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:58.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:57 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3054153895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:37:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:57 smithi084 ceph-mon[70440]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:58.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:57 smithi084 ceph-mon[70440]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:37:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:57 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3054153895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:37:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.187+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb11c1995c0 con 0x7fb10006d8a0 2024-02-28T23:37:58.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.190+0000 7fb120a00700 1 -- 172.21.15.57:0/3055547663 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb11c1995c0 con 0x7fb10006d8a0 2024-02-28T23:37:58.188 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:37:58.189 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 msgr2=0x7fb10006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 0x7fb10006fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb104005950 tx=0x7fb1040058e0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 msgr2=0x7fb11c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0055f0 tx=0x7fb10c0056d0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 shutdown_connections 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb10006d8a0 0x7fb10006fd50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c073090 0x7fb11c198180 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 --2- 172.21.15.57:0/3055547663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb11c0739a0 0x7fb11c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.193+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 >> 172.21.15.57:0/3055547663 conn(0x7fb11c0fbda0 msgr2=0x7fb11c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:58.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.194+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 shutdown_connections 2024-02-28T23:37:58.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.194+0000 7fb121e00700 1 -- 172.21.15.57:0/3055547663 wait complete. 2024-02-28T23:37:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 -- 172.21.15.57:0/1238373563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c073090 msgr2=0x7f4a9c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1238373563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c073090 0x7f4a9c073460 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009940 tx=0x7f4a90009c50 comp rx=0 tx=0).stop 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 -- 172.21.15.57:0/1238373563 shutdown_connections 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1238373563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1238373563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c073090 0x7f4a9c073460 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 -- 172.21.15.57:0/1238373563 >> 172.21.15.57:0/1238373563 conn(0x7f4a9c0fbda0 msgr2=0x7f4a9c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.348+0000 7f4aa4000700 1 -- 172.21.15.57:0/1238373563 shutdown_connections 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.349+0000 7f4aa4000700 1 -- 172.21.15.57:0/1238373563 wait complete. 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 Processor -- start 2024-02-28T23:37:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 -- start start 2024-02-28T23:37:58.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c073090 0x7f4a9c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:58.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:58.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c198da0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa4000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19cb30 con 0x7f4a9c073090 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38642/0 (socket says 172.21.15.57:38642) 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.350+0000 7f4aa0c00700 1 -- 172.21.15.57:0/1100312767 learned_addr learned my addr 172.21.15.57:0/1100312767 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.351+0000 7f4aa1600700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c073090 0x7f4a9c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.351+0000 7f4aa0c00700 1 -- 172.21.15.57:0/1100312767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c073090 msgr2=0x7f4a9c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.351+0000 7f4aa0c00700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c073090 0x7f4a9c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.351+0000 7f4aa0c00700 1 -- 172.21.15.57:0/1100312767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900095f0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.351+0000 7f4aa0c00700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e920 tx=0x7f4a8800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:58.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.352+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a88009790 con 0x7f4a9c0739a0 2024-02-28T23:37:58.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.352+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a8800cba0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.352+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a88018650 con 0x7f4a9c0739a0 2024-02-28T23:37:58.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.352+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c19ce10 con 0x7f4a9c0739a0 2024-02-28T23:37:58.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.352+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c19d360 con 0x7f4a9c0739a0 2024-02-28T23:37:58.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.353+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c109ce0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.354+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a8800cd10 con 0x7f4a9c0739a0 2024-02-28T23:37:58.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.355+0000 7f4a9a000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 0x7f4a8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:37:58.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.355+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a88014070 con 0x7f4a9c0739a0 2024-02-28T23:37:58.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.355+0000 7f4aa1600700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 0x7f4a8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:37:58.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.356+0000 7f4aa1600700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 0x7f4a8406fd50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005b40 tx=0x7f4a90005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:37:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.360+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a88052d20 con 0x7f4a9c0739a0 2024-02-28T23:37:58.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.682+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a9c04f2a0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.683+0000 7f4a9a000700 1 -- 172.21.15.57:0/1100312767 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f4a8805abd0 con 0x7f4a9c0739a0 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:37:58.682 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:37:58.683 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:37:58.684 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:37:58.684 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:37:58.684 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:37:58.684 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 msgr2=0x7f4a8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 0x7f4a8406fd50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005b40 tx=0x7f4a90005a90 comp rx=0 tx=0).stop 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 msgr2=0x7f4a9c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e920 tx=0x7f4a8800ece0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 shutdown_connections 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a8406d8a0 0x7f4a8406fd50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9c073090 0x7f4a9c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 --2- 172.21.15.57:0/1100312767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a9c0739a0 0x7f4a9c1986c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 >> 172.21.15.57:0/1100312767 conn(0x7f4a9c0fbda0 msgr2=0x7f4a9c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 shutdown_connections 2024-02-28T23:37:58.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:37:58.686+0000 7f4aa4000700 1 -- 172.21.15.57:0/1100312767 wait complete. 2024-02-28T23:37:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:58 smithi057 ceph-mon[89491]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:58 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/687283585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:37:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:58 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1100312767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:37:59.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:58 smithi084 ceph-mon[70440]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:37:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:58 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/687283585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:37:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:58 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1100312767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:37:59.990 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:37:59 smithi084 ceph-mon[70440]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:00.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:37:59 smithi057 ceph-mon[89491]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:01.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:00 smithi057 ceph-mon[89491]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:01.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:00 smithi084 ceph-mon[70440]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:03.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:02 smithi057 ceph-mon[89491]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:03.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:02 smithi084 ceph-mon[70440]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:04.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:04.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:04.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:04 smithi057 ceph-mon[89491]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:05.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:04 smithi084 ceph-mon[70440]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:06 smithi057 ceph-mon[89491]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:07.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:06 smithi084 ceph-mon[70440]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:09.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:08 smithi057 ceph-mon[89491]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:09.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:08 smithi084 ceph-mon[70440]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:11.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:10 smithi057 ceph-mon[89491]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:11.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:10 smithi084 ceph-mon[70440]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:13.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:12 smithi057 ceph-mon[89491]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:13.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:12 smithi084 ceph-mon[70440]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:15.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:14 smithi057 ceph-mon[89491]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:15.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:14 smithi084 ceph-mon[70440]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:17.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:16 smithi057 ceph-mon[89491]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:17.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:16 smithi084 ceph-mon[70440]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:18 smithi057 ceph-mon[89491]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:18 smithi084 ceph-mon[70440]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:19.079 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:20.883 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:20 smithi084 ceph-mon[70440]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:21.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:20 smithi057 ceph-mon[89491]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:22 smithi084 ceph-mon[70440]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:23.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:22 smithi057 ceph-mon[89491]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:24 smithi084 ceph-mon[70440]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:25.207 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:24 smithi057 ceph-mon[89491]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:27.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:26 smithi057 ceph-mon[89491]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:27.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:26 smithi084 ceph-mon[70440]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:28.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 -- 172.21.15.57:0/2033014308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 msgr2=0x7f07b8108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 --2- 172.21.15.57:0/2033014308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8108840 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009910 tx=0x7f07a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 -- 172.21.15.57:0/2033014308 shutdown_connections 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 --2- 172.21.15.57:0/2033014308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 --2- 172.21.15.57:0/2033014308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8108840 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 -- 172.21.15.57:0/2033014308 >> 172.21.15.57:0/2033014308 conn(0x7f07b80fdf90 msgr2=0x7f07b81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 -- 172.21.15.57:0/2033014308 shutdown_connections 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.861+0000 7f07bd200700 1 -- 172.21.15.57:0/2033014308 wait complete. 2024-02-28T23:38:28.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.862+0000 7f07bd200700 1 Processor -- start 2024-02-28T23:38:28.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07bd200700 1 -- start start 2024-02-28T23:38:28.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07bd200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:28.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07bd200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:28.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07bd200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8198e10 con 0x7f07b8108470 2024-02-28T23:38:28.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07bd200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b819cba0 con 0x7f07b8102470 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07b5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07b5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55112/0 (socket says 172.21.15.57:55112) 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.863+0000 7f07b5400700 1 -- 172.21.15.57:0/1958062661 learned_addr learned my addr 172.21.15.57:0/1958062661 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5e00700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5400700 1 -- 172.21.15.57:0/1958062661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 msgr2=0x7f07b81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5400700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5400700 1 -- 172.21.15.57:0/1958062661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a80095f0 con 0x7f07b8108470 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5e00700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.864+0000 7f07b5400700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d8d0 tx=0x7f07a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.865+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07a4009940 con 0x7f07b8108470 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.865+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07a4010460 con 0x7f07b8108470 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.865+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b819ce80 con 0x7f07b8108470 2024-02-28T23:38:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.865+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07a400f5f0 con 0x7f07b8108470 2024-02-28T23:38:28.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.865+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b819d3d0 con 0x7f07b8108470 2024-02-28T23:38:28.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.866+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b810a9e0 con 0x7f07b8108470 2024-02-28T23:38:28.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.867+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f07a40105d0 con 0x7f07b8108470 2024-02-28T23:38:28.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.868+0000 7f07aea00700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 0x7f079c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:28.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.868+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f07a408d4c0 con 0x7f07b8108470 2024-02-28T23:38:28.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.868+0000 7f07b5e00700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 0x7f079c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:28.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.869+0000 7f07b5e00700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 0x7f079c06fd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f07a8006010 tx=0x7f07a800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:28.873+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07a4056c50 con 0x7f07b8108470 2024-02-28T23:38:29.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.121+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07b8199660 con 0x7f079c06d8a0 2024-02-28T23:38:29.121 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:28 smithi057 ceph-mon[89491]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:29.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.124+0000 7f07aea00700 1 -- 172.21.15.57:0/1958062661 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f07b8199660 con 0x7f079c06d8a0 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 msgr2=0x7f079c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 0x7f079c06fd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f07a8006010 tx=0x7f07a800b540 comp rx=0 tx=0).stop 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 msgr2=0x7f07b8198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d8d0 tx=0x7f07a400dc90 comp rx=0 tx=0).stop 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 shutdown_connections 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f079c06d8a0 0x7f079c06fd50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07b8102470 0x7f07b81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 --2- 172.21.15.57:0/1958062661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b8108470 0x7f07b8198730 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 >> 172.21.15.57:0/1958062661 conn(0x7f07b80fdf90 msgr2=0x7f07b81002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 shutdown_connections 2024-02-28T23:38:29.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.128+0000 7f07bd200700 1 -- 172.21.15.57:0/1958062661 wait complete. 2024-02-28T23:38:29.144 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 -- 172.21.15.57:0/76151391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 msgr2=0x7f0788100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 --2- 172.21.15.57:0/76151391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788100890 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f077c009910 tx=0x7f077c009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 -- 172.21.15.57:0/76151391 shutdown_connections 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 --2- 172.21.15.57:0/76151391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 0x7f0788104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 --2- 172.21.15.57:0/76151391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788100890 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.297+0000 7f078f600700 1 -- 172.21.15.57:0/76151391 >> 172.21.15.57:0/76151391 conn(0x7f07880fbd80 msgr2=0x7f07880fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.298+0000 7f078f600700 1 -- 172.21.15.57:0/76151391 shutdown_connections 2024-02-28T23:38:29.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.298+0000 7f078f600700 1 -- 172.21.15.57:0/76151391 wait complete. 2024-02-28T23:38:29.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 Processor -- start 2024-02-28T23:38:29.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 -- start start 2024-02-28T23:38:29.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 0x7f0788198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788198c90 con 0x7f07881004c0 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.299+0000 7f078f600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788198dd0 con 0x7f0788100e60 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55126/0 (socket says 172.21.15.57:55126) 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 -- 172.21.15.57:0/1609372798 learned_addr learned my addr 172.21.15.57:0/1609372798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f0787e00700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 0x7f0788198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 -- 172.21.15.57:0/1609372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 msgr2=0x7f0788198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 0x7f0788198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 -- 172.21.15.57:0/1609372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0095f0 con 0x7f07881004c0 2024-02-28T23:38:29.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.300+0000 7f078cc00700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b5c0 tx=0x7f077c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.301+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077c01d070 con 0x7f07881004c0 2024-02-28T23:38:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.301+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f077c00f460 con 0x7f07881004c0 2024-02-28T23:38:29.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.301+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078819cbc0 con 0x7f07881004c0 2024-02-28T23:38:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.301+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077c017610 con 0x7f07881004c0 2024-02-28T23:38:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.301+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078819d0b0 con 0x7f07881004c0 2024-02-28T23:38:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.302+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07881045b0 con 0x7f07881004c0 2024-02-28T23:38:29.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.303+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f077c00f5d0 con 0x7f07881004c0 2024-02-28T23:38:29.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.304+0000 7f0785600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 0x7f077006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.304+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f077c08eb30 con 0x7f07881004c0 2024-02-28T23:38:29.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.305+0000 7f0787e00700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 0x7f077006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.305+0000 7f0787e00700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 0x7f077006fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0774005ea0 tx=0x7f0774005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.309+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f077c0582c0 con 0x7f07881004c0 2024-02-28T23:38:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:28 smithi084 ceph-mon[70440]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:29.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.553+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f078819d3e0 con 0x7f077006d8a0 2024-02-28T23:38:29.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.557+0000 7f0785600700 1 -- 172.21.15.57:0/1609372798 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f078819d3e0 con 0x7f077006d8a0 2024-02-28T23:38:29.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.560+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 msgr2=0x7f077006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.560+0000 7f078f600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 0x7f077006fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0774005ea0 tx=0x7f0774005e30 comp rx=0 tx=0).stop 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.560+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 msgr2=0x7f0788198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.560+0000 7f078f600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b5c0 tx=0x7f077c00be00 comp rx=0 tx=0).stop 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 shutdown_connections 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f077006d8a0 0x7f077006fd50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07881004c0 0x7f0788198100 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 --2- 172.21.15.57:0/1609372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0788100e60 0x7f0788198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 >> 172.21.15.57:0/1609372798 conn(0x7f07880fbd80 msgr2=0x7f07880fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 shutdown_connections 2024-02-28T23:38:29.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.561+0000 7f078f600700 1 -- 172.21.15.57:0/1609372798 wait complete. 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 -- 172.21.15.57:0/2400474302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9304073a00 msgr2=0x7f930410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 --2- 172.21.15.57:0/2400474302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9304073a00 0x7f930410c580 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009910 tx=0x7f92f0009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 -- 172.21.15.57:0/2400474302 shutdown_connections 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 --2- 172.21.15.57:0/2400474302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9304073a00 0x7f930410c580 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 --2- 172.21.15.57:0/2400474302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93040730f0 0x7f93040734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 -- 172.21.15.57:0/2400474302 >> 172.21.15.57:0/2400474302 conn(0x7f93040fbda0 msgr2=0x7f93040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 -- 172.21.15.57:0/2400474302 shutdown_connections 2024-02-28T23:38:29.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.708+0000 7f9309c00700 1 -- 172.21.15.57:0/2400474302 wait complete. 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.709+0000 7f9309c00700 1 Processor -- start 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9309c00700 1 -- start start 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9309c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9309c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9304073a00 0x7f9304198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9309c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930419c970 con 0x7f93040730f0 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9309c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930419cae0 con 0x7f9304073a00 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9302a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9302a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55142/0 (socket says 172.21.15.57:55142) 2024-02-28T23:38:29.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.710+0000 7f9302a00700 1 -- 172.21.15.57:0/3972792280 learned_addr learned my addr 172.21.15.57:0/3972792280 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9302a00700 1 -- 172.21.15.57:0/3972792280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9304073a00 msgr2=0x7f9304198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9302000700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9304073a00 0x7f9304198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9302a00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9304073a00 0x7f9304198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9302a00700 1 -- 172.21.15.57:0/3972792280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00095f0 con 0x7f93040730f0 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9302a00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d8d0 tx=0x7f92f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f4009940 con 0x7f93040730f0 2024-02-28T23:38:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930419cdc0 con 0x7f93040730f0 2024-02-28T23:38:29.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92f4010460 con 0x7f93040730f0 2024-02-28T23:38:29.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f400f5f0 con 0x7f93040730f0 2024-02-28T23:38:29.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.711+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930419d310 con 0x7f93040730f0 2024-02-28T23:38:29.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.713+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304109c80 con 0x7f93040730f0 2024-02-28T23:38:29.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.714+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f92f40105d0 con 0x7f93040730f0 2024-02-28T23:38:29.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.714+0000 7f92fb400700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 0x7f92e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:29.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.715+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92f408d5d0 con 0x7f93040730f0 2024-02-28T23:38:29.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.715+0000 7f9302000700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 0x7f92e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:29.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.716+0000 7f9302000700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 0x7f92e806fd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:29.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.719+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92f4056ce0 con 0x7f93040730f0 2024-02-28T23:38:29.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.957+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93041993c0 con 0x7f92e806d8a0 2024-02-28T23:38:29.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.990+0000 7f92fb400700 1 -- 172.21.15.57:0/3972792280 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f93041993c0 con 0x7f92e806d8a0 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (28m) 3m ago 41m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (38m) 3m ago 42m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (40m) 3m ago 40m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (24m) 3m ago 42m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (24m) 3m ago 40m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (27m) 3m ago 41m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (20m) 3m ago 34m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (19m) 3m ago 34m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (4m) 3m ago 34m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:38:29.991 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (19m) 3m ago 34m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (30m) 3m ago 43m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (29m) 3m ago 40m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (25m) 3m ago 44m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (25m) 3m ago 40m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (29m) 3m ago 42m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (29m) 3m ago 40m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (23m) 3m ago 39m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (23m) 3m ago 38m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (22m) 3m ago 38m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (21m) 3m ago 37m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (21m) 3m ago 37m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (20m) 3m ago 36m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:38:29.992 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (28m) 3m ago 41m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:38:29.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 msgr2=0x7f92e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 0x7f92e806fd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0005fd0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 msgr2=0x7f9304197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d8d0 tx=0x7f92f400dc90 comp rx=0 tx=0).stop 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 shutdown_connections 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f92e806d8a0 0x7f92e806fd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93040730f0 0x7f9304197fc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 --2- 172.21.15.57:0/3972792280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9304073a00 0x7f9304198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 >> 172.21.15.57:0/3972792280 conn(0x7f93040fbda0 msgr2=0x7f9304106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 shutdown_connections 2024-02-28T23:38:29.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:29.994+0000 7f9309c00700 1 -- 172.21.15.57:0/3972792280 wait complete. 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 -- 172.21.15.57:0/4035239680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 msgr2=0x7f2304068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 --2- 172.21.15.57:0/4035239680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f2304068b70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009910 tx=0x7f22f4009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 -- 172.21.15.57:0/4035239680 shutdown_connections 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 --2- 172.21.15.57:0/4035239680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 0x7f2304104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 --2- 172.21.15.57:0/4035239680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f2304068b70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 -- 172.21.15.57:0/4035239680 >> 172.21.15.57:0/4035239680 conn(0x7f2304075130 msgr2=0x7f2304075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 -- 172.21.15.57:0/4035239680 shutdown_connections 2024-02-28T23:38:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.144+0000 7f2309e00700 1 -- 172.21.15.57:0/4035239680 wait complete. 2024-02-28T23:38:30.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.145+0000 7f2309e00700 1 Processor -- start 2024-02-28T23:38:30.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2309e00700 1 -- start start 2024-02-28T23:38:30.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2309e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2309e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 0x7f2304198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2309e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304198d00 con 0x7f23040687a0 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2309e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230419ca90 con 0x7f2304069140 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2302a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2302a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55160/0 (socket says 172.21.15.57:55160) 2024-02-28T23:38:30.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.146+0000 7f2302a00700 1 -- 172.21.15.57:0/3260226976 learned_addr learned my addr 172.21.15.57:0/3260226976 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f22fbe00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 0x7f2304198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2302a00700 1 -- 172.21.15.57:0/3260226976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 msgr2=0x7f2304198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2302a00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 0x7f2304198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2302a00700 1 -- 172.21.15.57:0/3260226976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40095f0 con 0x7f23040687a0 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2302a00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b5c0 tx=0x7f22f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f401d070 con 0x7f23040687a0 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22f4005e00 con 0x7f23040687a0 2024-02-28T23:38:30.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.147+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230419cd10 con 0x7f23040687a0 2024-02-28T23:38:30.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.148+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f4017400 con 0x7f23040687a0 2024-02-28T23:38:30.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.148+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230419d200 con 0x7f23040687a0 2024-02-28T23:38:30.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.149+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230404f2a0 con 0x7f23040687a0 2024-02-28T23:38:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.150+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f22f400f460 con 0x7f23040687a0 2024-02-28T23:38:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.151+0000 7f2308a00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.151+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f22f408ea10 con 0x7f23040687a0 2024-02-28T23:38:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.151+0000 7f22fbe00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.152+0000 7f22fbe00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005950 tx=0x7f22ec0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:30.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.155+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22f40581a0 con 0x7f23040687a0 2024-02-28T23:38:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.488+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f230419d4a0 con 0x7f23040687a0 2024-02-28T23:38:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.489+0000 7f2308a00700 1 -- 172.21.15.57:0/3260226976 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f22f405b860 con 0x7f23040687a0 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:38:30.489 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:38:30.490 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 msgr2=0x7f22e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005950 tx=0x7f22ec0058e0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 msgr2=0x7f23041980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b5c0 tx=0x7f22f400be20 comp rx=0 tx=0).stop 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 shutdown_connections 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f22e806d8a0 0x7f22e806fd50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23040687a0 0x7f23041980e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 --2- 172.21.15.57:0/3260226976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2304069140 0x7f2304198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 >> 172.21.15.57:0/3260226976 conn(0x7f2304075130 msgr2=0x7f23040fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 shutdown_connections 2024-02-28T23:38:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.492+0000 7f2309e00700 1 -- 172.21.15.57:0/3260226976 wait complete. 2024-02-28T23:38:30.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.652+0000 7fe8d3200700 1 -- 172.21.15.57:0/4011689329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 msgr2=0x7fe8d410ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.652+0000 7fe8d3200700 1 --2- 172.21.15.57:0/4011689329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d410ca30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009910 tx=0x7fe8c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:30.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.652+0000 7fe8d3200700 1 -- 172.21.15.57:0/4011689329 shutdown_connections 2024-02-28T23:38:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.653+0000 7fe8d3200700 1 --2- 172.21.15.57:0/4011689329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.653+0000 7fe8d3200700 1 --2- 172.21.15.57:0/4011689329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d410ca30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.653+0000 7fe8d3200700 1 -- 172.21.15.57:0/4011689329 >> 172.21.15.57:0/4011689329 conn(0x7fe8d4075e90 msgr2=0x7fe8d40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.653+0000 7fe8d3200700 1 -- 172.21.15.57:0/4011689329 shutdown_connections 2024-02-28T23:38:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.653+0000 7fe8d3200700 1 -- 172.21.15.57:0/4011689329 wait complete. 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.654+0000 7fe8d3200700 1 Processor -- start 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.654+0000 7fe8d3200700 1 -- start start 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8d3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8d3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8d3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d419cea0 con 0x7fe8d410c660 2024-02-28T23:38:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8d3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41a0bc0 con 0x7fe8d4106b80 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8c9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8c9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55176/0 (socket says 172.21.15.57:55176) 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8c9400700 1 -- 172.21.15.57:0/2524797130 learned_addr learned my addr 172.21.15.57:0/2524797130 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.655+0000 7fe8d1e00700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8c9400700 1 -- 172.21.15.57:0/2524797130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 msgr2=0x7fe8d419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8c9400700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8c9400700 1 -- 172.21.15.57:0/2524797130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40095f0 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8d1e00700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8c9400700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d900 tx=0x7fe8bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8bc0041f0 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8bc004cf0 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.656+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8bc00b750 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.657+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d41a0ea0 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.657+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d41a13f0 con 0x7fe8d410c660 2024-02-28T23:38:30.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.658+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d410eb40 con 0x7fe8d410c660 2024-02-28T23:38:30.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.659+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe8bc004350 con 0x7fe8d410c660 2024-02-28T23:38:30.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.660+0000 7fe8cb400700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 0x7fe8b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:30.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.660+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe8bc08d640 con 0x7fe8d410c660 2024-02-28T23:38:30.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.660+0000 7fe8d1e00700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 0x7fe8b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:30.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.661+0000 7fe8d1e00700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 0x7fe8b406fd50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400b5c0 tx=0x7fe8c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:30.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.665+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8bc056dd0 con 0x7fe8d410c660 2024-02-28T23:38:30.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.955+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8d4062380 con 0x7fe8d410c660 2024-02-28T23:38:30.954 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:30 smithi057 ceph-mon[89491]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:30.955 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:30 smithi057 ceph-mon[89491]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:30.955 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:30 smithi057 ceph-mon[89491]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:30.955 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:30 smithi057 ceph-mon[89491]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:30.955 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3260226976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:38:30.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.956+0000 7fe8cb400700 1 -- 172.21.15.57:0/2524797130 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fe8bc016740 con 0x7fe8d410c660 2024-02-28T23:38:30.956 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:38:30.957 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:38:30.958 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:38:30.959 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:38:30.960 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:38:30.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:38:30.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 msgr2=0x7fe8b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 0x7fe8b406fd50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400b5c0 tx=0x7fe8c401a040 comp rx=0 tx=0).stop 2024-02-28T23:38:30.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 msgr2=0x7fe8d419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d900 tx=0x7fe8bc00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 shutdown_connections 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe8b406d8a0 0x7fe8b406fd50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8d4106b80 0x7fe8d419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 --2- 172.21.15.57:0/2524797130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8d410c660 0x7fe8d419c7c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 >> 172.21.15.57:0/2524797130 conn(0x7fe8d4075e90 msgr2=0x7fe8d4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 shutdown_connections 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:30.959+0000 7fe8d3200700 1 -- 172.21.15.57:0/2524797130 wait complete. 2024-02-28T23:38:30.963 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:38:31.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.116+0000 7f3869200700 1 -- 172.21.15.57:0/2370930766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3864106b80 msgr2=0x7f3864106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.116+0000 7f3869200700 1 --2- 172.21.15.57:0/2370930766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3864106b80 0x7f3864106ff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3850009910 tx=0x7f3850009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 -- 172.21.15.57:0/2370930766 shutdown_connections 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 --2- 172.21.15.57:0/2370930766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3864106b80 0x7f3864106ff0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 --2- 172.21.15.57:0/2370930766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386410c8f0 0x7f386410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 -- 172.21.15.57:0/2370930766 >> 172.21.15.57:0/2370930766 conn(0x7f3864076060 msgr2=0x7f3864078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 -- 172.21.15.57:0/2370930766 shutdown_connections 2024-02-28T23:38:31.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.117+0000 7f3869200700 1 -- 172.21.15.57:0/2370930766 wait complete. 2024-02-28T23:38:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 Processor -- start 2024-02-28T23:38:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 -- start start 2024-02-28T23:38:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864106b80 0x7f386419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386419d100 con 0x7f386410c8f0 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.118+0000 7f3869200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38641a0e90 con 0x7f3864106b80 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55194/0 (socket says 172.21.15.57:55194) 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 -- 172.21.15.57:0/1501169493 learned_addr learned my addr 172.21.15.57:0/1501169493 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861e00700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864106b80 0x7f386419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 -- 172.21.15.57:0/1501169493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864106b80 msgr2=0x7f386419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864106b80 0x7f386419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.119+0000 7f3861400700 1 -- 172.21.15.57:0/1501169493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500095f0 con 0x7f386410c8f0 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f3861400700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f385000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385001d070 con 0x7f386410c8f0 2024-02-28T23:38:31.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3850004fb0 con 0x7f386410c8f0 2024-02-28T23:38:31.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38641a1110 con 0x7f386410c8f0 2024-02-28T23:38:31.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385000f690 con 0x7f386410c8f0 2024-02-28T23:38:31.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.120+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38641a1600 con 0x7f386410c8f0 2024-02-28T23:38:31.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.121+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3864109b20 con 0x7f386410c8f0 2024-02-28T23:38:31.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.122+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f38500228a0 con 0x7f386410c8f0 2024-02-28T23:38:31.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.123+0000 7f385aa00700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 0x7f384806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.123+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f385008ede0 con 0x7f386410c8f0 2024-02-28T23:38:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.123+0000 7f3861e00700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 0x7f384806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.124+0000 7f3861e00700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 0x7f384806fd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3854006fd0 tx=0x7f3854008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:31.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.128+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3850058570 con 0x7f386410c8f0 2024-02-28T23:38:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:30 smithi084 ceph-mon[70440]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:30 smithi084 ceph-mon[70440]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:30 smithi084 ceph-mon[70440]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:30 smithi084 ceph-mon[70440]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3260226976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:38:31.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.374+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f386402cca0 con 0x7f384806d8a0 2024-02-28T23:38:31.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.376+0000 7f385aa00700 1 -- 172.21.15.57:0/1501169493 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f386402cca0 con 0x7f384806d8a0 2024-02-28T23:38:31.375 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:38:31.376 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:38:31.377 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:38:31.377 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:38:31.377 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:38:31.377 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.378+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 msgr2=0x7f384806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.378+0000 7f3869200700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 0x7f384806fd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3854006fd0 tx=0x7f3854008040 comp rx=0 tx=0).stop 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.378+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 msgr2=0x7f386419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.378+0000 7f3869200700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f385000be20 comp rx=0 tx=0).stop 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 shutdown_connections 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f384806d8a0 0x7f384806fd50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864106b80 0x7f386419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 --2- 172.21.15.57:0/1501169493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f386410c8f0 0x7f386419ca20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 >> 172.21.15.57:0/1501169493 conn(0x7f3864076060 msgr2=0x7f3864077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:31.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 shutdown_connections 2024-02-28T23:38:31.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.379+0000 7f3869200700 1 -- 172.21.15.57:0/1501169493 wait complete. 2024-02-28T23:38:31.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 -- 172.21.15.57:0/1464244543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 msgr2=0x7f1190108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 --2- 172.21.15.57:0/1464244543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190108980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1184009910 tx=0x7f1184009c20 comp rx=0 tx=0).stop 2024-02-28T23:38:31.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 -- 172.21.15.57:0/1464244543 shutdown_connections 2024-02-28T23:38:31.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 --2- 172.21.15.57:0/1464244543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 0x7f11900689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 --2- 172.21.15.57:0/1464244543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190108980 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 -- 172.21.15.57:0/1464244543 >> 172.21.15.57:0/1464244543 conn(0x7f1190075130 msgr2=0x7f1190075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:31.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 -- 172.21.15.57:0/1464244543 shutdown_connections 2024-02-28T23:38:31.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.528+0000 7f1198400700 1 -- 172.21.15.57:0/1464244543 wait complete. 2024-02-28T23:38:31.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.529+0000 7f1198400700 1 Processor -- start 2024-02-28T23:38:31.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.529+0000 7f1198400700 1 -- start start 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1198400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 0x7f1190198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1198400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1198400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1190198d50 con 0x7f11901085b0 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1198400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019cae0 con 0x7f1190068540 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1195000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1195000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55206/0 (socket says 172.21.15.57:55206) 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.530+0000 7f1195000700 1 -- 172.21.15.57:0/1901352571 learned_addr learned my addr 172.21.15.57:0/1901352571 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1195a00700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 0x7f1190198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1195000700 1 -- 172.21.15.57:0/1901352571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 msgr2=0x7f1190198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1195000700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 0x7f1190198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1195000700 1 -- 172.21.15.57:0/1901352571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840095f0 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1195000700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b4b0 tx=0x7f117c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f117c011800 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f117c011e20 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119019cdc0 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f117c00f2e0 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.531+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119019d310 con 0x7f11901085b0 2024-02-28T23:38:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.532+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119010aa00 con 0x7f11901085b0 2024-02-28T23:38:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.533+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f117c010380 con 0x7f11901085b0 2024-02-28T23:38:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.535+0000 7f118e000700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 0x7f117806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:38:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.535+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f117c08d0f0 con 0x7f11901085b0 2024-02-28T23:38:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.535+0000 7f1195a00700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 0x7f117806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:38:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.536+0000 7f1195a00700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 0x7f117806fda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1184000c00 tx=0x7f1184005050 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:38:31.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.540+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f117c056930 con 0x7f11901085b0 2024-02-28T23:38:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.870+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f119004f2a0 con 0x7f11901085b0 2024-02-28T23:38:31.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.873+0000 7f118e000700 1 -- 172.21.15.57:0/1901352571 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f117c059ff0 con 0x7f11901085b0 2024-02-28T23:38:31.873 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:38:31.873 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:38:31.873 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:38:31.873 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:38:31.874 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:38:31.875 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:38:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 msgr2=0x7f117806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 0x7f117806fda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1184000c00 tx=0x7f1184005050 comp rx=0 tx=0).stop 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 msgr2=0x7f1190198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b4b0 tx=0x7f117c00b870 comp rx=0 tx=0).stop 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 shutdown_connections 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f117806d8f0 0x7f117806fda0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1190068540 0x7f1190198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 --2- 172.21.15.57:0/1901352571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11901085b0 0x7f1190198670 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.876+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 >> 172.21.15.57:0/1901352571 conn(0x7f1190075130 msgr2=0x7f1190106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.877+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 shutdown_connections 2024-02-28T23:38:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:38:31.877+0000 7f1198400700 1 -- 172.21.15.57:0/1901352571 wait complete. 2024-02-28T23:38:32.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:31 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2524797130' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:38:32.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:31 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2524797130' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:38:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:32 smithi057 ceph-mon[89491]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:33.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:32 smithi057 ceph-mon[89491]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:33.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:32 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1901352571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:38:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:32 smithi084 ceph-mon[70440]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:32 smithi084 ceph-mon[70440]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:38:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:32 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1901352571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:38:34.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:34.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:35.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:34 smithi057 ceph-mon[89491]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:34 smithi084 ceph-mon[70440]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:37.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:36 smithi057 ceph-mon[89491]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:37.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:36 smithi084 ceph-mon[70440]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:39.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:38 smithi057 ceph-mon[89491]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:39.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:38 smithi084 ceph-mon[70440]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:41.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:40 smithi057 ceph-mon[89491]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:41.255 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:40 smithi084 ceph-mon[70440]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:43.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:42 smithi057 ceph-mon[89491]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:43.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:42 smithi084 ceph-mon[70440]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:45.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:44 smithi057 ceph-mon[89491]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:45.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:44 smithi084 ceph-mon[70440]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:47.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:46 smithi057 ceph-mon[89491]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:47.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:46 smithi084 ceph-mon[70440]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:49.131 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:48 smithi084 ceph-mon[70440]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:49.131 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:48 smithi057 ceph-mon[89491]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:38:51.121 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:50 smithi057 ceph-mon[89491]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:50 smithi084 ceph-mon[70440]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:53.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:52 smithi057 ceph-mon[89491]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:52 smithi084 ceph-mon[70440]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:55.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:54 smithi057 ceph-mon[89491]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:54 smithi084 ceph-mon[70440]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:56 smithi084 ceph-mon[70440]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:57.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:56 smithi057 ceph-mon[89491]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:59.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:38:58 smithi057 ceph-mon[89491]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:38:59.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:38:58 smithi084 ceph-mon[70440]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:01.315 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:00 smithi057 ceph-mon[89491]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:01.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:00 smithi084 ceph-mon[70440]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:02.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 -- 172.21.15.57:0/4175819412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 msgr2=0x7f5754106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 --2- 172.21.15.57:0/4175819412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f5754106ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5740009910 tx=0x7f5740009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 -- 172.21.15.57:0/4175819412 shutdown_connections 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 --2- 172.21.15.57:0/4175819412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f5754106ff0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 --2- 172.21.15.57:0/4175819412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 -- 172.21.15.57:0/4175819412 >> 172.21.15.57:0/4175819412 conn(0x7f5754076060 msgr2=0x7f5754078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.054+0000 7f5759c00700 1 -- 172.21.15.57:0/4175819412 shutdown_connections 2024-02-28T23:39:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.055+0000 7f5759c00700 1 -- 172.21.15.57:0/4175819412 wait complete. 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.056+0000 7f5759c00700 1 Processor -- start 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.056+0000 7f5759c00700 1 -- start start 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5759c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5759c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5759c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575419d060 con 0x7f5754106b80 2024-02-28T23:39:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5759c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57541a0df0 con 0x7f575410c8f0 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47440/0 (socket says 172.21.15.57:47440) 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 -- 172.21.15.57:0/4125049066 learned_addr learned my addr 172.21.15.57:0/4125049066 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752000700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 -- 172.21.15.57:0/4125049066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 msgr2=0x7f575419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752a00700 1 -- 172.21.15.57:0/4125049066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400095f0 con 0x7f5754106b80 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.057+0000 7f5752000700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:39:02.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f5752a00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f574400d8d0 tx=0x7f574400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5744009940 con 0x7f5754106b80 2024-02-28T23:39:02.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5744010460 con 0x7f5754106b80 2024-02-28T23:39:02.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57541a10d0 con 0x7f5754106b80 2024-02-28T23:39:02.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574400f5f0 con 0x7f5754106b80 2024-02-28T23:39:02.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.058+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57541a1620 con 0x7f5754106b80 2024-02-28T23:39:02.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.059+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575410ecc0 con 0x7f5754106b80 2024-02-28T23:39:02.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.061+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f57440105d0 con 0x7f5754106b80 2024-02-28T23:39:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.062+0000 7f574b400700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 0x7f573806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.062+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f574408d420 con 0x7f5754106b80 2024-02-28T23:39:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.062+0000 7f5752000700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 0x7f573806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.063+0000 7f5752000700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 0x7f573806fd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f574000b5c0 tx=0x7f5740005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.067+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5744056c50 con 0x7f5754106b80 2024-02-28T23:39:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.327+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57541a1890 con 0x7f573806d8a0 2024-02-28T23:39:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.330+0000 7f574b400700 1 -- 172.21.15.57:0/4125049066 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f57541a1890 con 0x7f573806d8a0 2024-02-28T23:39:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 msgr2=0x7f573806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 0x7f573806fd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f574000b5c0 tx=0x7f5740005960 comp rx=0 tx=0).stop 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 msgr2=0x7f575419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f574400d8d0 tx=0x7f574400dc90 comp rx=0 tx=0).stop 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 shutdown_connections 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f573806d8a0 0x7f573806fd50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5754106b80 0x7f575419c440 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 --2- 172.21.15.57:0/4125049066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f575410c8f0 0x7f575419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.333+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 >> 172.21.15.57:0/4125049066 conn(0x7f5754076060 msgr2=0x7f5754077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.334+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 shutdown_connections 2024-02-28T23:39:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.334+0000 7f5759c00700 1 -- 172.21.15.57:0/4125049066 wait complete. 2024-02-28T23:39:02.348 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:39:02.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 -- 172.21.15.57:0/3926106064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec1025f0 msgr2=0x7f12ec1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 --2- 172.21.15.57:0/3926106064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec1025f0 0x7f12ec1029c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009910 tx=0x7f12dc009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 -- 172.21.15.57:0/3926106064 shutdown_connections 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 --2- 172.21.15.57:0/3926106064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec103e50 0x7f12ec1042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 --2- 172.21.15.57:0/3926106064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec1025f0 0x7f12ec1029c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 -- 172.21.15.57:0/3926106064 >> 172.21.15.57:0/3926106064 conn(0x7f12ec0fddb0 msgr2=0x7f12ec1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 -- 172.21.15.57:0/3926106064 shutdown_connections 2024-02-28T23:39:02.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.498+0000 7f12eb200700 1 -- 172.21.15.57:0/3926106064 wait complete. 2024-02-28T23:39:02.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.499+0000 7f12eb200700 1 Processor -- start 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.499+0000 7f12eb200700 1 -- start start 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12eb200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec1025f0 0x7f12ec197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12eb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12eb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec198b40 con 0x7f12ec103e50 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12eb200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec19c8d0 con 0x7f12ec1025f0 2024-02-28T23:39:02.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47470/0 (socket says 172.21.15.57:47470) 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 -- 172.21.15.57:0/4138118981 learned_addr learned my addr 172.21.15.57:0/4138118981 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9e00700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec1025f0 0x7f12ec197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 -- 172.21.15.57:0/4138118981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec1025f0 msgr2=0x7f12ec197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec1025f0 0x7f12ec197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.500+0000 7f12e9400700 1 -- 172.21.15.57:0/4138118981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0095f0 con 0x7f12ec103e50 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12e9400700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f12d800d8d0 tx=0x7f12d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d8009940 con 0x7f12ec103e50 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d8010460 con 0x7f12ec103e50 2024-02-28T23:39:02.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d800f5f0 con 0x7f12ec103e50 2024-02-28T23:39:02.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec19cbb0 con 0x7f12ec103e50 2024-02-28T23:39:02.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.501+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec19d180 con 0x7f12ec103e50 2024-02-28T23:39:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.502+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec062380 con 0x7f12ec103e50 2024-02-28T23:39:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.503+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f12d80105d0 con 0x7f12ec103e50 2024-02-28T23:39:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.504+0000 7f12e2a00700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 0x7f12d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.504+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f12d808d4c0 con 0x7f12ec103e50 2024-02-28T23:39:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.505+0000 7f12e9e00700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 0x7f12d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.506+0000 7f12e9e00700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 0x7f12d006fd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f12dc006010 tx=0x7f12dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.509+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12d8056c50 con 0x7f12ec103e50 2024-02-28T23:39:02.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.755+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12ec19d490 con 0x7f12d006d8a0 2024-02-28T23:39:02.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.757+0000 7f12e2a00700 1 -- 172.21.15.57:0/4138118981 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f12ec19d490 con 0x7f12d006d8a0 2024-02-28T23:39:02.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 msgr2=0x7f12d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 0x7f12d006fd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f12dc006010 tx=0x7f12dc01a040 comp rx=0 tx=0).stop 2024-02-28T23:39:02.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 msgr2=0x7f12ec198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f12d800d8d0 tx=0x7f12d800dc90 comp rx=0 tx=0).stop 2024-02-28T23:39:02.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 shutdown_connections 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f12d006d8a0 0x7f12d006fd50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ec1025f0 0x7f12ec197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 --2- 172.21.15.57:0/4138118981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12ec103e50 0x7f12ec198460 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 >> 172.21.15.57:0/4138118981 conn(0x7f12ec0fddb0 msgr2=0x7f12ec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 shutdown_connections 2024-02-28T23:39:02.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.760+0000 7f12eb200700 1 -- 172.21.15.57:0/4138118981 wait complete. 2024-02-28T23:39:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.916+0000 7fdc66c00700 1 -- 172.21.15.57:0/730309166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 msgr2=0x7fdc601008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.916+0000 7fdc66c00700 1 --2- 172.21.15.57:0/730309166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc601008b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009910 tx=0x7fdc50009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 -- 172.21.15.57:0/730309166 shutdown_connections 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 --2- 172.21.15.57:0/730309166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 0x7fdc60104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 --2- 172.21.15.57:0/730309166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc601008b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 -- 172.21.15.57:0/730309166 >> 172.21.15.57:0/730309166 conn(0x7fdc600fbda0 msgr2=0x7fdc600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 -- 172.21.15.57:0/730309166 shutdown_connections 2024-02-28T23:39:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.917+0000 7fdc66c00700 1 -- 172.21.15.57:0/730309166 wait complete. 2024-02-28T23:39:02.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.918+0000 7fdc66c00700 1 Processor -- start 2024-02-28T23:39:02.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.918+0000 7fdc66c00700 1 -- start start 2024-02-28T23:39:02.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc66c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc66c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 0x7fdc60196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc66c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60196b10 con 0x7fdc601004e0 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc66c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6019a8a0 con 0x7fdc60100e80 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc5fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc5fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47482/0 (socket says 172.21.15.57:47482) 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.919+0000 7fdc5fe00700 1 -- 172.21.15.57:0/1931424032 learned_addr learned my addr 172.21.15.57:0/1931424032 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5f400700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 0x7fdc60196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5fe00700 1 -- 172.21.15.57:0/1931424032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 msgr2=0x7fdc60196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:02.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5fe00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 0x7fdc60196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:02.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5fe00700 1 -- 172.21.15.57:0/1931424032 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500095f0 con 0x7fdc601004e0 2024-02-28T23:39:02.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5fe00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006010 tx=0x7fdc5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.920+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc5001d070 con 0x7fdc601004e0 2024-02-28T23:39:02.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.921+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc5000f460 con 0x7fdc601004e0 2024-02-28T23:39:02.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.921+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6019ab20 con 0x7fdc601004e0 2024-02-28T23:39:02.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.921+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc50017630 con 0x7fdc601004e0 2024-02-28T23:39:02.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.921+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6019b010 con 0x7fdc601004e0 2024-02-28T23:39:02.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.922+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc601087f0 con 0x7fdc601004e0 2024-02-28T23:39:02.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.923+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdc5000f5d0 con 0x7fdc601004e0 2024-02-28T23:39:02.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.923+0000 7fdc5cc00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 0x7fdc4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:02.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.924+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc5008ea10 con 0x7fdc601004e0 2024-02-28T23:39:02.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.924+0000 7fdc5f400700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 0x7fdc4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:02.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.924+0000 7fdc5f400700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 0x7fdc4406fd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005950 tx=0x7fdc4c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:02.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:02.928+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc500581a0 con 0x7fdc601004e0 2024-02-28T23:39:03.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:02 smithi057 ceph-mon[89491]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:03.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.157+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc6019b230 con 0x7fdc4406d8a0 2024-02-28T23:39:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.180+0000 7fdc5cc00700 1 -- 172.21.15.57:0/1931424032 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdc6019b230 con 0x7fdc4406d8a0 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (28m) 4m ago 42m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (38m) 4m ago 42m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (41m) 4m ago 41m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (24m) 4m ago 42m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (24m) 4m ago 41m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (28m) 4m ago 42m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (20m) 4m ago 35m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:39:03.180 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (20m) 4m ago 35m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (4m) 4m ago 35m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (19m) 4m ago 35m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (31m) 4m ago 44m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (30m) 4m ago 41m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (26m) 4m ago 44m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (25m) 4m ago 41m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (29m) 4m ago 42m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (29m) 4m ago 41m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:39:03.181 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (24m) 4m ago 40m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (23m) 4m ago 39m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (23m) 4m ago 38m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (22m) 4m ago 38m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (21m) 4m ago 37m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (21m) 4m ago 37m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:39:03.182 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (29m) 4m ago 41m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:39:03.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 msgr2=0x7fdc4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 0x7fdc4406fd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005950 tx=0x7fdc4c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 msgr2=0x7fdc60195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006010 tx=0x7fdc5000be20 comp rx=0 tx=0).stop 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 shutdown_connections 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc4406d8a0 0x7fdc4406fd50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc601004e0 0x7fdc60195ef0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 --2- 172.21.15.57:0/1931424032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc60100e80 0x7fdc60196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.183+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 >> 172.21.15.57:0/1931424032 conn(0x7fdc600fbda0 msgr2=0x7fdc600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.184+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 shutdown_connections 2024-02-28T23:39:03.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.184+0000 7fdc66c00700 1 -- 172.21.15.57:0/1931424032 wait complete. 2024-02-28T23:39:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 -- 172.21.15.57:0/1815535898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 msgr2=0x7f1940107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 --2- 172.21.15.57:0/1815535898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940107010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1930009910 tx=0x7f1930009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 -- 172.21.15.57:0/1815535898 shutdown_connections 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 --2- 172.21.15.57:0/1815535898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940107010 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 --2- 172.21.15.57:0/1815535898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 0x7f1940102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 -- 172.21.15.57:0/1815535898 >> 172.21.15.57:0/1815535898 conn(0x7f19400fdf40 msgr2=0x7f1940100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 -- 172.21.15.57:0/1815535898 shutdown_connections 2024-02-28T23:39:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.329+0000 7f1948800700 1 -- 172.21.15.57:0/1815535898 wait complete. 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 Processor -- start 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 -- start start 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 0x7f1940198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940198c70 con 0x7f1940103040 2024-02-28T23:39:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.330+0000 7f1948800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194019ca00 con 0x7f19401026a0 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47502/0 (socket says 172.21.15.57:47502) 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 -- 172.21.15.57:0/1403870922 learned_addr learned my addr 172.21.15.57:0/1403870922 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945e00700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 0x7f1940198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 -- 172.21.15.57:0/1403870922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 msgr2=0x7f1940198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 0x7f1940198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.331+0000 7f1945400700 1 -- 172.21.15.57:0/1403870922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300095f0 con 0x7f1940103040 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f1945400700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f193000bda0 tx=0x7f193000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f193001d070 con 0x7f1940103040 2024-02-28T23:39:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f193000f460 con 0x7f1940103040 2024-02-28T23:39:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194019cc80 con 0x7f1940103040 2024-02-28T23:39:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1930017610 con 0x7f1940103040 2024-02-28T23:39:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.332+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194019d170 con 0x7f1940103040 2024-02-28T23:39:03.334 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:02 smithi084 ceph-mon[70440]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.333+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194010aa10 con 0x7f1940103040 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.334+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f193000fa40 con 0x7f1940103040 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.335+0000 7f193aa00700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 0x7f192806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.335+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f193008ea50 con 0x7f1940103040 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.335+0000 7f1945e00700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 0x7f192806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.336+0000 7f1945e00700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 0x7f192806fd50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1934005950 tx=0x7f19340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.340+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1930058160 con 0x7f1940103040 2024-02-28T23:39:03.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.661+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f194004f2a0 con 0x7f1940103040 2024-02-28T23:39:03.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.662+0000 7f193aa00700 1 -- 172.21.15.57:0/1403870922 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f193005b820 con 0x7f1940103040 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:39:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:39:03.692 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:39:03.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 msgr2=0x7f192806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 0x7f192806fd50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1934005950 tx=0x7f19340058e0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 msgr2=0x7f1940198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f193000bda0 tx=0x7f193000be80 comp rx=0 tx=0).stop 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 shutdown_connections 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f192806d8a0 0x7f192806fd50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19401026a0 0x7f1940198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 --2- 172.21.15.57:0/1403870922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1940103040 0x7f1940198590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 >> 172.21.15.57:0/1403870922 conn(0x7f19400fdf40 msgr2=0x7f1940100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 shutdown_connections 2024-02-28T23:39:03.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.666+0000 7f1948800700 1 -- 172.21.15.57:0/1403870922 wait complete. 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1173228674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 msgr2=0x7fd0dc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1173228674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc104e50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009910 tx=0x7fd0c8009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1173228674 shutdown_connections 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1173228674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc104e50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1173228674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0dc1004e0 0x7fd0dc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1173228674 >> 172.21.15.57:0/1173228674 conn(0x7fd0dc0fbda0 msgr2=0x7fd0dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1173228674 shutdown_connections 2024-02-28T23:39:03.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.826+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1173228674 wait complete. 2024-02-28T23:39:03.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.827+0000 7fd0e2a00700 1 Processor -- start 2024-02-28T23:39:03.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0e2a00700 1 -- start start 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0e2a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0dc1004e0 0x7fd0dc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0e2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0e2a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc073ad0 con 0x7fd0dc100e80 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0e2a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc073c10 con 0x7fd0dc1004e0 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0db400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0db400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47524/0 (socket says 172.21.15.57:47524) 2024-02-28T23:39:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.828+0000 7fd0db400700 1 -- 172.21.15.57:0/1391026301 learned_addr learned my addr 172.21.15.57:0/1391026301 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0db400700 1 -- 172.21.15.57:0/1391026301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0dc1004e0 msgr2=0x7fd0dc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0db400700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0dc1004e0 0x7fd0dc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0db400700 1 -- 172.21.15.57:0/1391026301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80095f0 con 0x7fd0dc100e80 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0db400700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800bda0 tx=0x7fd0c800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c801d070 con 0x7fd0dc100e80 2024-02-28T23:39:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c8004fb0 con 0x7fd0dc100e80 2024-02-28T23:39:03.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c800f460 con 0x7fd0dc100e80 2024-02-28T23:39:03.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.829+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc073e90 con 0x7fd0dc100e80 2024-02-28T23:39:03.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.830+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc1a23c0 con 0x7fd0dc100e80 2024-02-28T23:39:03.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.831+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc108850 con 0x7fd0dc100e80 2024-02-28T23:39:03.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.832+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd0c8022800 con 0x7fd0dc100e80 2024-02-28T23:39:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.833+0000 7fd0d8c00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 0x7fd0c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.833+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd0c808ea30 con 0x7fd0dc100e80 2024-02-28T23:39:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.833+0000 7fd0dbe00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 0x7fd0c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:03.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.834+0000 7fd0dbe00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 0x7fd0c006fd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005fd0 tx=0x7fd0cc005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:03.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:03.838+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0c8058390 con 0x7fd0dc100e80 2024-02-28T23:39:04.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.106+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0dc04f2a0 con 0x7fd0dc100e80 2024-02-28T23:39:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.106+0000 7fd0d8c00700 1 -- 172.21.15.57:0/1391026301 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fd0c8027740 con 0x7fd0dc100e80 2024-02-28T23:39:04.105 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:39:04.105 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:39:04.105 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:39:04.105 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:39:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:39:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:04.108 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:04.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.109+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 msgr2=0x7fd0c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.109+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 0x7fd0c006fd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005fd0 tx=0x7fd0cc005d00 comp rx=0 tx=0).stop 2024-02-28T23:39:04.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.109+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 msgr2=0x7fd0dc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.109+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800bda0 tx=0x7fd0c800be80 comp rx=0 tx=0).stop 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 shutdown_connections 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd0c006d8a0 0x7fd0c006fd50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0dc1004e0 0x7fd0dc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 --2- 172.21.15.57:0/1391026301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0dc100e80 0x7fd0dc073590 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 >> 172.21.15.57:0/1391026301 conn(0x7fd0dc0fbda0 msgr2=0x7fd0dc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 shutdown_connections 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.110+0000 7fd0e2a00700 1 -- 172.21.15.57:0/1391026301 wait complete. 2024-02-28T23:39:04.110 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:39:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 -- 172.21.15.57:0/2681669881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c0102e70 msgr2=0x7f16c0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 --2- 172.21.15.57:0/2681669881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c0102e70 0x7f16c0106e40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009910 tx=0x7f16ac009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 -- 172.21.15.57:0/2681669881 shutdown_connections 2024-02-28T23:39:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 --2- 172.21.15.57:0/2681669881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c0102e70 0x7f16c0106e40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 --2- 172.21.15.57:0/2681669881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c01024d0 0x7f16c01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 -- 172.21.15.57:0/2681669881 >> 172.21.15.57:0/2681669881 conn(0x7f16c00fdd90 msgr2=0x7f16c01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 -- 172.21.15.57:0/2681669881 shutdown_connections 2024-02-28T23:39:04.271 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:04 smithi057 ceph-mon[89491]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:04.271 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:04 smithi057 ceph-mon[89491]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:04.271 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:04.272 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:04 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1403870922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.272+0000 7f16c5000700 1 -- 172.21.15.57:0/2681669881 wait complete. 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.273+0000 7f16c5000700 1 Processor -- start 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.273+0000 7f16c5000700 1 -- start start 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16c5000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16c5000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c0102e70 0x7f16c01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16c5000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0198ba0 con 0x7f16c01024d0 2024-02-28T23:39:04.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16c5000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c019c930 con 0x7f16c0102e70 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16bf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16bf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47550/0 (socket says 172.21.15.57:47550) 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.274+0000 7f16bf400700 1 -- 172.21.15.57:0/256928342 learned_addr learned my addr 172.21.15.57:0/256928342 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.275+0000 7f16bea00700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c0102e70 0x7f16c01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.275+0000 7f16bf400700 1 -- 172.21.15.57:0/256928342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c0102e70 msgr2=0x7f16c01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.275+0000 7f16bf400700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c0102e70 0x7f16c01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.275+0000 7f16bf400700 1 -- 172.21.15.57:0/256928342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0095f0 con 0x7f16c01024d0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.275+0000 7f16bf400700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f16b000b4b0 tx=0x7f16b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.276+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b0011800 con 0x7f16c01024d0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.276+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c019cc10 con 0x7f16c01024d0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.276+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b0011e20 con 0x7f16c01024d0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.276+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c019d160 con 0x7f16c01024d0 2024-02-28T23:39:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.276+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b000f2e0 con 0x7f16c01024d0 2024-02-28T23:39:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.277+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0005320 con 0x7f16c01024d0 2024-02-28T23:39:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.278+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f16b0011960 con 0x7f16c01024d0 2024-02-28T23:39:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.279+0000 7f16b7e00700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 0x7f16a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.279+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f16b008d0b0 con 0x7f16c01024d0 2024-02-28T23:39:04.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.280+0000 7f16bea00700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 0x7f16a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.280+0000 7f16bea00700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 0x7f16a406fd50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f16c01995a0 tx=0x7f16ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:04.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.285+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16b00567c0 con 0x7f16c01024d0 2024-02-28T23:39:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:04 smithi084 ceph-mon[70440]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:04 smithi084 ceph-mon[70440]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:04 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1403870922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:39:04.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.533+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16a0000bf0 con 0x7f16a406d8a0 2024-02-28T23:39:04.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.536+0000 7f16b7e00700 1 -- 172.21.15.57:0/256928342 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f16a0000bf0 con 0x7f16a406d8a0 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:39:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:39:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:39:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:39:04.536 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:39:04.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 msgr2=0x7f16a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 0x7f16a406fd50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f16c01995a0 tx=0x7f16ac01a040 comp rx=0 tx=0).stop 2024-02-28T23:39:04.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 msgr2=0x7f16c0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f16b000b4b0 tx=0x7f16b000b870 comp rx=0 tx=0).stop 2024-02-28T23:39:04.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 shutdown_connections 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f16a406d8a0 0x7f16a406fd50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16c01024d0 0x7f16c0197f80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 --2- 172.21.15.57:0/256928342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c0102e70 0x7f16c01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.539+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 >> 172.21.15.57:0/256928342 conn(0x7f16c00fdd90 msgr2=0x7f16c00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.540+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 shutdown_connections 2024-02-28T23:39:04.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.540+0000 7f16c5000700 1 -- 172.21.15.57:0/256928342 wait complete. 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.700+0000 7fdc39200700 1 -- 172.21.15.57:0/3821585203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc34100e80 msgr2=0x7fdc34104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.700+0000 7fdc39200700 1 --2- 172.21.15.57:0/3821585203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc34100e80 0x7fdc34104e50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009910 tx=0x7fdc1c009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 -- 172.21.15.57:0/3821585203 shutdown_connections 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 --2- 172.21.15.57:0/3821585203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc34100e80 0x7fdc34104e50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 --2- 172.21.15.57:0/3821585203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc341004e0 0x7fdc341008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 -- 172.21.15.57:0/3821585203 >> 172.21.15.57:0/3821585203 conn(0x7fdc340fbda0 msgr2=0x7fdc340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 -- 172.21.15.57:0/3821585203 shutdown_connections 2024-02-28T23:39:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.701+0000 7fdc39200700 1 -- 172.21.15.57:0/3821585203 wait complete. 2024-02-28T23:39:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 Processor -- start 2024-02-28T23:39:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 -- start start 2024-02-28T23:39:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc34100e80 0x7fdc34196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34196b50 con 0x7fdc341004e0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.702+0000 7fdc39200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3419a8e0 con 0x7fdc34100e80 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47562/0 (socket says 172.21.15.57:47562) 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 -- 172.21.15.57:0/733946550 learned_addr learned my addr 172.21.15.57:0/733946550 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc29400700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc34100e80 0x7fdc34196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 -- 172.21.15.57:0/733946550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc34100e80 msgr2=0x7fdc34196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc34100e80 0x7fdc34196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 -- 172.21.15.57:0/733946550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0095f0 con 0x7fdc341004e0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.703+0000 7fdc31e00700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d8d0 tx=0x7fdc2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.704+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc24009940 con 0x7fdc341004e0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.704+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc24010460 con 0x7fdc341004e0 2024-02-28T23:39:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.704+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3419abc0 con 0x7fdc341004e0 2024-02-28T23:39:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.704+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2400f5f0 con 0x7fdc341004e0 2024-02-28T23:39:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.704+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3419b110 con 0x7fdc341004e0 2024-02-28T23:39:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.705+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34108850 con 0x7fdc341004e0 2024-02-28T23:39:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.706+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdc240105d0 con 0x7fdc341004e0 2024-02-28T23:39:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.707+0000 7fdc2b400700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 0x7fdc1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.708+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc2408d4c0 con 0x7fdc341004e0 2024-02-28T23:39:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.708+0000 7fdc29400700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 0x7fdc1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.708+0000 7fdc29400700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 0x7fdc1806fd50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c000c00 tx=0x7fdc1c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:04.712+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc24056c50 con 0x7fdc341004e0 2024-02-28T23:39:05.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.026+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdc3404f2a0 con 0x7fdc341004e0 2024-02-28T23:39:05.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.027+0000 7fdc2b400700 1 -- 172.21.15.57:0/733946550 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fdc2405a310 con 0x7fdc341004e0 2024-02-28T23:39:05.026 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:39:05.026 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:39:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:39:05.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:39:05.026 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:39:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:39:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:39:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 msgr2=0x7fdc1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 0x7fdc1806fd50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c000c00 tx=0x7fdc1c005fd0 comp rx=0 tx=0).stop 2024-02-28T23:39:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 msgr2=0x7fdc34195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d8d0 tx=0x7fdc2400dc90 comp rx=0 tx=0).stop 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 shutdown_connections 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdc1806d8a0 0x7fdc1806fd50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc341004e0 0x7fdc34195f30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 --2- 172.21.15.57:0/733946550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc34100e80 0x7fdc34196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.030+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 >> 172.21.15.57:0/733946550 conn(0x7fdc340fbda0 msgr2=0x7fdc340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.031+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 shutdown_connections 2024-02-28T23:39:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:05.031+0000 7fdc39200700 1 -- 172.21.15.57:0/733946550 wait complete. 2024-02-28T23:39:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:05 smithi057 ceph-mon[89491]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:05 smithi057 ceph-mon[89491]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:05.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:05 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1391026301' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:39:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:05 smithi084 ceph-mon[70440]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:05 smithi084 ceph-mon[70440]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:05 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1391026301' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:39:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:06 smithi057 ceph-mon[89491]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:06 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/733946550' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:39:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:06 smithi084 ceph-mon[70440]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:06.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:06 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/733946550' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:39:07.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:07 smithi057 ceph-mon[89491]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:07.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:07 smithi084 ceph-mon[70440]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:09.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:09 smithi057 ceph-mon[89491]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:09 smithi084 ceph-mon[70440]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:11.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:11 smithi057 ceph-mon[89491]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:11.329 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:11 smithi084 ceph-mon[70440]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:13 smithi057 ceph-mon[89491]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:13 smithi084 ceph-mon[70440]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:15 smithi057 ceph-mon[89491]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:15 smithi084 ceph-mon[70440]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:17 smithi084 ceph-mon[70440]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:17.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:17 smithi057 ceph-mon[89491]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:18.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:18 smithi057 ceph-mon[89491]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:18.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:18 smithi084 ceph-mon[70440]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:18.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:20.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:20 smithi057 ceph-mon[89491]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:20.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:20 smithi084 ceph-mon[70440]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:22.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:22 smithi057 ceph-mon[89491]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:22.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:22 smithi084 ceph-mon[70440]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:24.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:24 smithi057 ceph-mon[89491]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:24.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:24 smithi084 ceph-mon[70440]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:26.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:26 smithi057 ceph-mon[89491]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:26.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:26 smithi084 ceph-mon[70440]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:28.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:28 smithi057 ceph-mon[89491]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:28.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:28 smithi084 ceph-mon[70440]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:30.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:30 smithi057 ceph-mon[89491]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:30.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:30 smithi084 ceph-mon[70440]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:32.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:32 smithi057 ceph-mon[89491]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:32.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:32 smithi084 ceph-mon[70440]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:33.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:33.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:34.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:34 smithi057 ceph-mon[89491]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:34.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:34 smithi084 ceph-mon[70440]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.203+0000 7fc8afe00700 1 -- 172.21.15.57:0/2364374600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b01025d0 msgr2=0x7fc8b01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.203+0000 7fc8afe00700 1 --2- 172.21.15.57:0/2364374600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b01025d0 0x7fc8b01029a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b0a0 tx=0x7fc8a000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 -- 172.21.15.57:0/2364374600 shutdown_connections 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 --2- 172.21.15.57:0/2364374600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b0103e30 0x7fc8b01042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 --2- 172.21.15.57:0/2364374600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b01025d0 0x7fc8b01029a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 -- 172.21.15.57:0/2364374600 >> 172.21.15.57:0/2364374600 conn(0x7fc8b00fddb0 msgr2=0x7fc8b01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 -- 172.21.15.57:0/2364374600 shutdown_connections 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.204+0000 7fc8afe00700 1 -- 172.21.15.57:0/2364374600 wait complete. 2024-02-28T23:39:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.205+0000 7fc8afe00700 1 Processor -- start 2024-02-28T23:39:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.205+0000 7fc8afe00700 1 -- start start 2024-02-28T23:39:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8afe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b01025d0 0x7fc8b0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8afe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8afe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0198b60 con 0x7fc8b0103e30 2024-02-28T23:39:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8afe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019c8f0 con 0x7fc8b01025d0 2024-02-28T23:39:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8a7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8a7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34684/0 (socket says 172.21.15.57:34684) 2024-02-28T23:39:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.206+0000 7fc8a7e00700 1 -- 172.21.15.57:0/1813073848 learned_addr learned my addr 172.21.15.57:0/1813073848 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8aea00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b01025d0 0x7fc8b0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7e00700 1 -- 172.21.15.57:0/1813073848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b01025d0 msgr2=0x7fc8b0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7e00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b01025d0 0x7fc8b0197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7e00700 1 -- 172.21.15.57:0/1813073848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0009d00 con 0x7fc8b0103e30 2024-02-28T23:39:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7e00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc89800d8d0 tx=0x7fc89800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc898009940 con 0x7fc8b0103e30 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc898010460 con 0x7fc8b0103e30 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019cbd0 con 0x7fc8b0103e30 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.207+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc89800f5f0 con 0x7fc8b0103e30 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.208+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b019d120 con 0x7fc8b0103e30 2024-02-28T23:39:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.209+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b0062380 con 0x7fc8b0103e30 2024-02-28T23:39:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.210+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc8980105d0 con 0x7fc8b0103e30 2024-02-28T23:39:35.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.210+0000 7fc8a7400700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 0x7fc89406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.211+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc89808d4c0 con 0x7fc8b0103e30 2024-02-28T23:39:35.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.211+0000 7fc8aea00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 0x7fc89406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.212+0000 7fc8aea00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 0x7fc89406fd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b830 tx=0x7fc8a0006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:35.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.216+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc898056c50 con 0x7fc8b0103e30 2024-02-28T23:39:35.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.468+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b019d400 con 0x7fc89406d8a0 2024-02-28T23:39:35.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.471+0000 7fc8a7400700 1 -- 172.21.15.57:0/1813073848 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc8b019d400 con 0x7fc89406d8a0 2024-02-28T23:39:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 msgr2=0x7fc89406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 0x7fc89406fd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b830 tx=0x7fc8a0006000 comp rx=0 tx=0).stop 2024-02-28T23:39:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 msgr2=0x7fc8b0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc89800d8d0 tx=0x7fc89800dc90 comp rx=0 tx=0).stop 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 shutdown_connections 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc89406d8a0 0x7fc89406fd50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b01025d0 0x7fc8b0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 --2- 172.21.15.57:0/1813073848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b0103e30 0x7fc8b0198480 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 >> 172.21.15.57:0/1813073848 conn(0x7fc8b00fddb0 msgr2=0x7fc8b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 shutdown_connections 2024-02-28T23:39:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.474+0000 7fc8afe00700 1 -- 172.21.15.57:0/1813073848 wait complete. 2024-02-28T23:39:35.490 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 -- 172.21.15.57:0/887207391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e010c8f0 msgr2=0x7f13e010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 --2- 172.21.15.57:0/887207391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e010c8f0 0x7f13e010ccc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f13d000b0a0 tx=0x7f13d000b3b0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 -- 172.21.15.57:0/887207391 shutdown_connections 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 --2- 172.21.15.57:0/887207391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e0106b80 0x7f13e0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 --2- 172.21.15.57:0/887207391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e010c8f0 0x7f13e010ccc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 -- 172.21.15.57:0/887207391 >> 172.21.15.57:0/887207391 conn(0x7f13e0076060 msgr2=0x7f13e0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:35.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 -- 172.21.15.57:0/887207391 shutdown_connections 2024-02-28T23:39:35.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.642+0000 7f13e5e00700 1 -- 172.21.15.57:0/887207391 wait complete. 2024-02-28T23:39:35.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.643+0000 7f13e5e00700 1 Processor -- start 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13e5e00700 1 -- start start 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13e5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13e5e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 0x7f13e019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13e5e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e019d090 con 0x7f13e0106b80 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13e5e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e019d1d0 con 0x7f13e010c8f0 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.644+0000 7f13dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34700/0 (socket says 172.21.15.57:34700) 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 -- 172.21.15.57:0/517828246 learned_addr learned my addr 172.21.15.57:0/517828246 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13d6000700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 0x7f13e019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 -- 172.21.15.57:0/517828246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 msgr2=0x7f13e019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 0x7f13e019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 -- 172.21.15.57:0/517828246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009d00 con 0x7f13e0106b80 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13d6000700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 0x7f13e019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.645+0000 7f13dea00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000f80 tx=0x7f13d000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.646+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d000e040 con 0x7f13e0106b80 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.646+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13d0003e60 con 0x7f13e0106b80 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.646+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e01a0fc0 con 0x7f13e0106b80 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.646+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d001d910 con 0x7f13e0106b80 2024-02-28T23:39:35.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.646+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e01a14b0 con 0x7f13e0106b80 2024-02-28T23:39:35.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.647+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e0109b20 con 0x7f13e0106b80 2024-02-28T23:39:35.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.648+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f13d0004820 con 0x7f13e0106b80 2024-02-28T23:39:35.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.649+0000 7f13e4a00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 0x7f13c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:35.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.649+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f13d008ee00 con 0x7f13e0106b80 2024-02-28T23:39:35.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.649+0000 7f13d6000700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 0x7f13c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:35.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.650+0000 7f13d6000700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 0x7f13c406fd50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:35.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.654+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13d0058590 con 0x7f13e0106b80 2024-02-28T23:39:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.903+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e01a17e0 con 0x7f13c406d8a0 2024-02-28T23:39:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.904+0000 7f13e4a00700 1 -- 172.21.15.57:0/517828246 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f13e01a17e0 con 0x7f13c406d8a0 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.907+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 msgr2=0x7f13c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.907+0000 7f13e5e00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 0x7f13c406fd50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8005e20 comp rx=0 tx=0).stop 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.907+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 msgr2=0x7f13e019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.907+0000 7f13e5e00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000f80 tx=0x7f13d000cdd0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 shutdown_connections 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f13c406d8a0 0x7f13c406fd50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e0106b80 0x7f13e019c500 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 --2- 172.21.15.57:0/517828246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e010c8f0 0x7f13e019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 >> 172.21.15.57:0/517828246 conn(0x7f13e0076060 msgr2=0x7f13e0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 shutdown_connections 2024-02-28T23:39:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:35.908+0000 7f13e5e00700 1 -- 172.21.15.57:0/517828246 wait complete. 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f8543e00700 1 -- 172.21.15.57:0/2072931744 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853000bc20 con 0x7f85440739a0 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 -- 172.21.15.57:0/2072931744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 msgr2=0x7f854410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 --2- 172.21.15.57:0/2072931744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f854410c5e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8530009940 tx=0x7f8530009c50 comp rx=0 tx=0).stop 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 -- 172.21.15.57:0/2072931744 shutdown_connections 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 --2- 172.21.15.57:0/2072931744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f854410c5e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 --2- 172.21.15.57:0/2072931744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 0x7f8544073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 -- 172.21.15.57:0/2072931744 >> 172.21.15.57:0/2072931744 conn(0x7f85440fbda0 msgr2=0x7f85440fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.073+0000 7f854c600700 1 -- 172.21.15.57:0/2072931744 shutdown_connections 2024-02-28T23:39:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.074+0000 7f854c600700 1 -- 172.21.15.57:0/2072931744 wait complete. 2024-02-28T23:39:36.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.074+0000 7f854c600700 1 Processor -- start 2024-02-28T23:39:36.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f854c600700 1 -- start start 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f854c600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 0x7f85441981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f854c600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f854c600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544198dd0 con 0x7f85440739a0 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f854c600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854419cb60 con 0x7f8544073090 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f8549200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f8549200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34714/0 (socket says 172.21.15.57:34714) 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.075+0000 7f8549200700 1 -- 172.21.15.57:0/2103143061 learned_addr learned my addr 172.21.15.57:0/2103143061 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:36.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8549c00700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 0x7f85441981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8549200700 1 -- 172.21.15.57:0/2103143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 msgr2=0x7f85441981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8549200700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 0x7f85441981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8549200700 1 -- 172.21.15.57:0/2103143061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300095f0 con 0x7f85440739a0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8549200700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f85300055f0 tx=0x7f85300056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.076+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853001d070 con 0x7f85440739a0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.077+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854419cde0 con 0x7f85440739a0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.077+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f853000f460 con 0x7f85440739a0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.077+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8530021620 con 0x7f85440739a0 2024-02-28T23:39:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.077+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854419d2d0 con 0x7f85440739a0 2024-02-28T23:39:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.078+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8544109ce0 con 0x7f85440739a0 2024-02-28T23:39:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.079+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f853000fa60 con 0x7f85440739a0 2024-02-28T23:39:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.080+0000 7f8542000700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 0x7f852c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.080+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f853008e8e0 con 0x7f85440739a0 2024-02-28T23:39:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.080+0000 7f8549c00700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 0x7f852c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.081+0000 7f8549c00700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 0x7f852c06fd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8538005fd0 tx=0x7f853800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.085+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8530057ff0 con 0x7f85440739a0 2024-02-28T23:39:36.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.323+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8544199620 con 0x7f852c06d8a0 2024-02-28T23:39:36.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.355+0000 7f8542000700 1 -- 172.21.15.57:0/2103143061 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8544199620 con 0x7f852c06d8a0 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (29m) 4m ago 43m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (39m) 4m ago 43m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (41m) 5m ago 41m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (25m) 4m ago 43m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:39:36.355 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (25m) 5m ago 41m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (28m) 4m ago 42m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (21m) 4m ago 35m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (20m) 4m ago 35m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (5m) 5m ago 35m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (20m) 5m ago 35m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (31m) 4m ago 44m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (30m) 5m ago 41m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:39:36.356 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (26m) 4m ago 45m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (26m) 5m ago 41m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (30m) 4m ago 43m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (30m) 5m ago 41m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (24m) 4m ago 40m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (24m) 4m ago 39m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (23m) 4m ago 39m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (23m) 5m ago 38m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (22m) 5m ago 38m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:39:36.357 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (21m) 5m ago 37m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:39:36.358 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (29m) 4m ago 42m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 msgr2=0x7f852c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 0x7f852c06fd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8538005fd0 tx=0x7f853800a560 comp rx=0 tx=0).stop 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 msgr2=0x7f85441986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f85300055f0 tx=0x7f85300056d0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 shutdown_connections 2024-02-28T23:39:36.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f852c06d8a0 0x7f852c06fd50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544073090 0x7f85441981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 --2- 172.21.15.57:0/2103143061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85440739a0 0x7f85441986f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 >> 172.21.15.57:0/2103143061 conn(0x7f85440fbda0 msgr2=0x7f8544106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:36.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 shutdown_connections 2024-02-28T23:39:36.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.358+0000 7f854c600700 1 -- 172.21.15.57:0/2103143061 wait complete. 2024-02-28T23:39:36.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 -- 172.21.15.57:0/1800187549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80108470 msgr2=0x7f2f80108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 --2- 172.21.15.57:0/1800187549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80108470 0x7f2f80108840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009910 tx=0x7f2f70009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 -- 172.21.15.57:0/1800187549 shutdown_connections 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 --2- 172.21.15.57:0/1800187549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 0x7f2f801028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 --2- 172.21.15.57:0/1800187549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80108470 0x7f2f80108840 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.520+0000 7f2f86200700 1 -- 172.21.15.57:0/1800187549 >> 172.21.15.57:0/1800187549 conn(0x7f2f800fdf90 msgr2=0x7f2f801003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.521+0000 7f2f86200700 1 -- 172.21.15.57:0/1800187549 shutdown_connections 2024-02-28T23:39:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.521+0000 7f2f86200700 1 -- 172.21.15.57:0/1800187549 wait complete. 2024-02-28T23:39:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 Processor -- start 2024-02-28T23:39:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 -- start start 2024-02-28T23:39:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 0x7f2f80198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80198f70 con 0x7f2f80198960 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.522+0000 7f2f86200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801990e0 con 0x7f2f80102470 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34728/0 (socket says 172.21.15.57:34728) 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 -- 172.21.15.57:0/1373428180 learned_addr learned my addr 172.21.15.57:0/1373428180 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7f400700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 0x7f2f80198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 -- 172.21.15.57:0/1373428180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 msgr2=0x7f2f80198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 0x7f2f80198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.523+0000 7f2f7ea00700 1 -- 172.21.15.57:0/1373428180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700095f0 con 0x7f2f80198960 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f7ea00700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00b4e0 tx=0x7f2f6c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c011800 con 0x7f2f80198960 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f6c011e20 con 0x7f2f80198960 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c00f2e0 con 0x7f2f80198960 2024-02-28T23:39:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8019d3d0 con 0x7f2f80198960 2024-02-28T23:39:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.524+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8019d920 con 0x7f2f80198960 2024-02-28T23:39:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.525+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8010a9e0 con 0x7f2f80198960 2024-02-28T23:39:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.527+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2f6c010380 con 0x7f2f80198960 2024-02-28T23:39:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.527+0000 7f2f77e00700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 0x7f2f6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.528+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f6c08d1c0 con 0x7f2f80198960 2024-02-28T23:39:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.528+0000 7f2f7f400700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 0x7f2f6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.528+0000 7f2f7f400700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 0x7f2f6406fd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009fd0 tx=0x7f2f7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:36.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.532+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f6c056970 con 0x7f2f80198960 2024-02-28T23:39:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:36 smithi057 ceph-mon[89491]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:36 smithi057 ceph-mon[89491]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:36.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:36 smithi057 ceph-mon[89491]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:36 smithi084 ceph-mon[70440]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:36 smithi084 ceph-mon[70440]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:36.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:36 smithi084 ceph-mon[70440]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:36.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.853+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2f8004f2a0 con 0x7f2f80198960 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.854+0000 7f2f77e00700 1 -- 172.21.15.57:0/1373428180 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2f6c05a030 con 0x7f2f80198960 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:36.854 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:39:36.855 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 msgr2=0x7f2f6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 0x7f2f6406fd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009fd0 tx=0x7f2f7001a040 comp rx=0 tx=0).stop 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 msgr2=0x7f2f8019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00b4e0 tx=0x7f2f6c00b8a0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 shutdown_connections 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f6406d8a0 0x7f2f6406fd50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f80102470 0x7f2f80198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 --2- 172.21.15.57:0/1373428180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f80198960 0x7f2f8019cdd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:36.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.857+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 >> 172.21.15.57:0/1373428180 conn(0x7f2f800fdf90 msgr2=0x7f2f800ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:36.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.858+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 shutdown_connections 2024-02-28T23:39:36.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:36.858+0000 7f2f86200700 1 -- 172.21.15.57:0/1373428180 wait complete. 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 -- 172.21.15.57:0/2540414498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770102470 msgr2=0x7f77701028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 --2- 172.21.15.57:0/2540414498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770102470 0x7f77701028e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f775c009910 tx=0x7f775c009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 -- 172.21.15.57:0/2540414498 shutdown_connections 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 --2- 172.21.15.57:0/2540414498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770102470 0x7f77701028e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 --2- 172.21.15.57:0/2540414498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770108470 0x7f7770108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 -- 172.21.15.57:0/2540414498 >> 172.21.15.57:0/2540414498 conn(0x7f77700fdf90 msgr2=0x7f77701003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 -- 172.21.15.57:0/2540414498 shutdown_connections 2024-02-28T23:39:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.019+0000 7f7777e00700 1 -- 172.21.15.57:0/2540414498 wait complete. 2024-02-28T23:39:37.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.020+0000 7f7777e00700 1 Processor -- start 2024-02-28T23:39:37.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.020+0000 7f7777e00700 1 -- start start 2024-02-28T23:39:37.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7777e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 0x7f7770198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7777e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7777e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7770198e20 con 0x7f7770108470 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7777e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777019cb60 con 0x7f7770102470 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7774a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7774a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34740/0 (socket says 172.21.15.57:34740) 2024-02-28T23:39:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.021+0000 7f7774a00700 1 -- 172.21.15.57:0/2631485701 learned_addr learned my addr 172.21.15.57:0/2631485701 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7775400700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 0x7f7770198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7774a00700 1 -- 172.21.15.57:0/2631485701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 msgr2=0x7f7770198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7774a00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 0x7f7770198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7774a00700 1 -- 172.21.15.57:0/2631485701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0095f0 con 0x7f7770108470 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7775400700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 0x7f7770198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7774a00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f775c00b5c0 tx=0x7f775c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c01d070 con 0x7f7770108470 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f775c00f460 con 0x7f7770108470 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777019cde0 con 0x7f7770108470 2024-02-28T23:39:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c017610 con 0x7f7770108470 2024-02-28T23:39:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.022+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777019d2d0 con 0x7f7770108470 2024-02-28T23:39:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.024+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777010aa00 con 0x7f7770108470 2024-02-28T23:39:37.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.025+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f775c00f5d0 con 0x7f7770108470 2024-02-28T23:39:37.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.025+0000 7f776e000700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 0x7f775806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.026+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f775c08eb50 con 0x7f7770108470 2024-02-28T23:39:37.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.026+0000 7f7775400700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 0x7f775806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.026+0000 7f7775400700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 0x7f775806fd50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7764007900 tx=0x7f7764008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.031+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f775c0582e0 con 0x7f7770108470 2024-02-28T23:39:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.311+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f777010a3e0 con 0x7f7770108470 2024-02-28T23:39:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.312+0000 7f776e000700 1 -- 172.21.15.57:0/2631485701 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f775c05b9a0 con 0x7f7770108470 2024-02-28T23:39:37.313 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:39:37.314 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:39:37.315 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:37.316 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:37.317 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:37.317 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.315+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 msgr2=0x7f775806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.315+0000 7f7777e00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 0x7f775806fd50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7764007900 tx=0x7f7764008040 comp rx=0 tx=0).stop 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.315+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 msgr2=0x7f7770198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.315+0000 7f7777e00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f775c00b5c0 tx=0x7f775c00be00 comp rx=0 tx=0).stop 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 shutdown_connections 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f775806d8a0 0x7f775806fd50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7770102470 0x7f7770198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 --2- 172.21.15.57:0/2631485701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7770108470 0x7f7770198740 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 >> 172.21.15.57:0/2631485701 conn(0x7f77700fdf90 msgr2=0x7f77700febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 shutdown_connections 2024-02-28T23:39:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.316+0000 7f7777e00700 1 -- 172.21.15.57:0/2631485701 wait complete. 2024-02-28T23:39:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:39:37.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 -- 172.21.15.57:0/2286261052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101bf0 msgr2=0x7f1b24106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 --2- 172.21.15.57:0/2286261052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101bf0 0x7f1b24106040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009910 tx=0x7f1b10009c20 comp rx=0 tx=0).stop 2024-02-28T23:39:37.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 -- 172.21.15.57:0/2286261052 shutdown_connections 2024-02-28T23:39:37.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 --2- 172.21.15.57:0/2286261052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101bf0 0x7f1b24106040 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 --2- 172.21.15.57:0/2286261052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101250 0x7f1b24101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 -- 172.21.15.57:0/2286261052 >> 172.21.15.57:0/2286261052 conn(0x7f1b240faad0 msgr2=0x7f1b240fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:37.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.490+0000 7f1b29a00700 1 -- 172.21.15.57:0/2286261052 shutdown_connections 2024-02-28T23:39:37.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.491+0000 7f1b29a00700 1 -- 172.21.15.57:0/2286261052 wait complete. 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 Processor -- start 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 -- start start 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 0x7f1b24198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24198c60 con 0x7f1b24101250 2024-02-28T23:39:37.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.492+0000 7f1b29a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2419c9a0 con 0x7f1b24101bf0 2024-02-28T23:39:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34758/0 (socket says 172.21.15.57:34758) 2024-02-28T23:39:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 -- 172.21.15.57:0/563418951 learned_addr learned my addr 172.21.15.57:0/563418951 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23400700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 0x7f1b24198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 -- 172.21.15.57:0/563418951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 msgr2=0x7f1b24198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 0x7f1b24198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23e00700 1 -- 172.21.15.57:0/563418951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100095f0 con 0x7f1b24101250 2024-02-28T23:39:37.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.493+0000 7f1b23400700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 0x7f1b24198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:39:37.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.494+0000 7f1b23e00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d8d0 tx=0x7f1b1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.497+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b14009940 con 0x7f1b24101250 2024-02-28T23:39:37.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.497+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b14010460 con 0x7f1b24101250 2024-02-28T23:39:37.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.497+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1400f5f0 con 0x7f1b24101250 2024-02-28T23:39:37.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.497+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2419cc80 con 0x7f1b24101250 2024-02-28T23:39:37.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.497+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2419d1d0 con 0x7f1b24101250 2024-02-28T23:39:37.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.499+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2410a770 con 0x7f1b24101250 2024-02-28T23:39:37.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.499+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1b140105d0 con 0x7f1b24101250 2024-02-28T23:39:37.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.501+0000 7f1b20c00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 0x7f1b0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.501+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1b1408d4c0 con 0x7f1b24101250 2024-02-28T23:39:37.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.501+0000 7f1b23400700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 0x7f1b0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.502+0000 7f1b23400700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 0x7f1b0806fd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b24199610 tx=0x7f1b1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.506+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b14056c50 con 0x7f1b24101250 2024-02-28T23:39:37.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:37 smithi057 ceph-mon[89491]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:37.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1373428180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:39:37.568 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2631485701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:39:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.763+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b241993c0 con 0x7f1b0806d8a0 2024-02-28T23:39:37.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.766+0000 7f1b20c00700 1 -- 172.21.15.57:0/563418951 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1b241993c0 con 0x7f1b0806d8a0 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:39:37.766 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:39:37.767 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:39:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.768+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 msgr2=0x7f1b0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.768+0000 7f1b29a00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 0x7f1b0806fd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b24199610 tx=0x7f1b1001a040 comp rx=0 tx=0).stop 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.768+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 msgr2=0x7f1b24198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.768+0000 7f1b29a00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d8d0 tx=0x7f1b1400dc90 comp rx=0 tx=0).stop 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 shutdown_connections 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1b0806d8a0 0x7f1b0806fd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b24101250 0x7f1b24198040 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 --2- 172.21.15.57:0/563418951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b24101bf0 0x7f1b24198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 >> 172.21.15.57:0/563418951 conn(0x7f1b240faad0 msgr2=0x7f1b241078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 shutdown_connections 2024-02-28T23:39:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.769+0000 7f1b29a00700 1 -- 172.21.15.57:0/563418951 wait complete. 2024-02-28T23:39:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:37 smithi084 ceph-mon[70440]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1373428180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:39:37.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2631485701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:39:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.924+0000 7f5494000700 1 -- 172.21.15.57:0/2498157737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 msgr2=0x7f548c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.924+0000 7f5494000700 1 --2- 172.21.15.57:0/2498157737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c073460 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f548000b0d0 tx=0x7f548000b3e0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 -- 172.21.15.57:0/2498157737 shutdown_connections 2024-02-28T23:39:37.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 --2- 172.21.15.57:0/2498157737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0739a0 0x7f548c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 --2- 172.21.15.57:0/2498157737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c073460 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 -- 172.21.15.57:0/2498157737 >> 172.21.15.57:0/2498157737 conn(0x7f548c0fbda0 msgr2=0x7f548c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:37.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 -- 172.21.15.57:0/2498157737 shutdown_connections 2024-02-28T23:39:37.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.925+0000 7f5494000700 1 -- 172.21.15.57:0/2498157737 wait complete. 2024-02-28T23:39:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 Processor -- start 2024-02-28T23:39:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 -- start start 2024-02-28T23:39:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0739a0 0x7f548c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c198d10 con 0x7f548c073090 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.926+0000 7f5494000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c198e50 con 0x7f548c0739a0 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34774/0 (socket says 172.21.15.57:34774) 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 -- 172.21.15.57:0/962591373 learned_addr learned my addr 172.21.15.57:0/962591373 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 -- 172.21.15.57:0/962591373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0739a0 msgr2=0x7f548c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0739a0 0x7f548c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 -- 172.21.15.57:0/962591373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009d00 con 0x7f548c073090 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.927+0000 7f5491600700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f548000b0a0 tx=0x7f5480003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.928+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548000e040 con 0x7f548c073090 2024-02-28T23:39:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.928+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5480007d60 con 0x7f548c073090 2024-02-28T23:39:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.928+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c19cc40 con 0x7f548c073090 2024-02-28T23:39:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.928+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548001d870 con 0x7f548c073090 2024-02-28T23:39:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.928+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c19d130 con 0x7f548c073090 2024-02-28T23:39:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.929+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c109ce0 con 0x7f548c073090 2024-02-28T23:39:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.930+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5480004790 con 0x7f548c073090 2024-02-28T23:39:37.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.931+0000 7f548a000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 0x7f547406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:39:37.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.931+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f548008ee20 con 0x7f548c073090 2024-02-28T23:39:37.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.931+0000 7f5490c00700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 0x7f547406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:39:37.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.932+0000 7f5490c00700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 0x7f547406fd50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5478005fd0 tx=0x7f5478005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:39:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:37.936+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54800585b0 con 0x7f548c073090 2024-02-28T23:39:38.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.251+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f548c04f2a0 con 0x7f548c073090 2024-02-28T23:39:38.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.251+0000 7f548a000700 1 -- 172.21.15.57:0/962591373 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f548005bc70 con 0x7f548c073090 2024-02-28T23:39:38.251 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:39:38.251 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:39:38.251 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:39:38.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:39:38.252 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:39:38.253 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:39:38.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.254+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 msgr2=0x7f547406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:38.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.254+0000 7f5494000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 0x7f547406fd50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5478005fd0 tx=0x7f5478005d00 comp rx=0 tx=0).stop 2024-02-28T23:39:38.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.254+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 msgr2=0x7f548c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:39:38.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.254+0000 7f5494000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f548000b0a0 tx=0x7f5480003ce0 comp rx=0 tx=0).stop 2024-02-28T23:39:38.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 shutdown_connections 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f547406d8a0 0x7f547406fd50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f548c073090 0x7f548c198180 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 --2- 172.21.15.57:0/962591373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0739a0 0x7f548c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 >> 172.21.15.57:0/962591373 conn(0x7f548c0fbda0 msgr2=0x7f548c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 shutdown_connections 2024-02-28T23:39:38.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:39:38.255+0000 7f5494000700 1 -- 172.21.15.57:0/962591373 wait complete. 2024-02-28T23:39:38.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:38 smithi057 ceph-mon[89491]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:38.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:38 smithi057 ceph-mon[89491]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:38.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:38 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/962591373' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:39:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:38 smithi084 ceph-mon[70440]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:38 smithi084 ceph-mon[70440]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:39:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:38 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/962591373' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:39:40.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:40 smithi057 ceph-mon[89491]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:40.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:40 smithi084 ceph-mon[70440]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:42.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:42 smithi057 ceph-mon[89491]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:42.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:42 smithi084 ceph-mon[70440]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:44.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:44 smithi057 ceph-mon[89491]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:44.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:44 smithi084 ceph-mon[70440]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:46.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:46 smithi057 ceph-mon[89491]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:46.829 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:46 smithi084 ceph-mon[70440]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:48 smithi084 ceph-mon[70440]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:48 smithi057 ceph-mon[89491]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:48.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:39:50.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:50 smithi057 ceph-mon[89491]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:50.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:50 smithi084 ceph-mon[70440]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:52.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:52 smithi057 ceph-mon[89491]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:52.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:52 smithi084 ceph-mon[70440]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:54.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:54 smithi057 ceph-mon[89491]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:54 smithi084 ceph-mon[70440]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:56.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:56 smithi084 ceph-mon[70440]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:57.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:56 smithi057 ceph-mon[89491]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:59.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:39:58 smithi057 ceph-mon[89491]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:39:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:39:58 smithi084 ceph-mon[70440]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:40:00.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:39:59.999+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:40:00.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:40:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:40:00.000+0000 7f864d400700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-28T23:40:00.847 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: /usr/bin/podman: timeout after 895 seconds 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: Traceback (most recent call last): 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: return _run_code(code, main_globals, None, 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: exec(code, run_globals) 2024-02-28T23:40:00.848 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: fs cephfs is degraded 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:40:00.849 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:00 smithi057 ceph-mon[89491]: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: /usr/bin/podman: timeout after 895 seconds 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: Traceback (most recent call last): 2024-02-28T23:40:01.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: return _run_code(code, main_globals, None, 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: exec(code, run_globals) 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:01.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:40:01.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: fs cephfs is degraded 2024-02-28T23:40:01.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:40:01.083 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:00 smithi084 ceph-mon[70440]: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:40:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:02 smithi057 ceph-mon[89491]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:02 smithi084 ceph-mon[70440]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:05.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:04 smithi057 ceph-mon[89491]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:04 smithi084 ceph-mon[70440]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:07.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:06 smithi057 ceph-mon[89491]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:06 smithi084 ceph-mon[70440]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 -- 172.21.15.57:0/2434389388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b810c8f0 msgr2=0x7f93b810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 --2- 172.21.15.57:0/2434389388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b810c8f0 0x7f93b810ccc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009910 tx=0x7f93a8009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 -- 172.21.15.57:0/2434389388 shutdown_connections 2024-02-28T23:40:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 --2- 172.21.15.57:0/2434389388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b8106b80 0x7f93b8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 --2- 172.21.15.57:0/2434389388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b810c8f0 0x7f93b810ccc0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 -- 172.21.15.57:0/2434389388 >> 172.21.15.57:0/2434389388 conn(0x7f93b8076060 msgr2=0x7f93b8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.438+0000 7f93bda00700 1 -- 172.21.15.57:0/2434389388 shutdown_connections 2024-02-28T23:40:08.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.439+0000 7f93bda00700 1 -- 172.21.15.57:0/2434389388 wait complete. 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 Processor -- start 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 -- start start 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b810c8f0 0x7f93b819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819d090 con 0x7f93b8106b80 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.440+0000 7f93bda00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819d1d0 con 0x7f93b810c8f0 2024-02-28T23:40:08.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54924/0 (socket says 172.21.15.57:54924) 2024-02-28T23:40:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 -- 172.21.15.57:0/717856160 learned_addr learned my addr 172.21.15.57:0/717856160 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:08.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93afe00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b810c8f0 0x7f93b819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 -- 172.21.15.57:0/717856160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b810c8f0 msgr2=0x7f93b819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b810c8f0 0x7f93b819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.441+0000 7f93b6a00700 1 -- 172.21.15.57:0/717856160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80095f0 con 0x7f93b8106b80 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93b6a00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b5c0 tx=0x7f93a800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a801d070 con 0x7f93b8106b80 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b81a0fc0 con 0x7f93b8106b80 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93a800f460 con 0x7f93b8106b80 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a8017630 con 0x7f93b8106b80 2024-02-28T23:40:08.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.442+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81a14b0 con 0x7f93b8106b80 2024-02-28T23:40:08.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.444+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b804f2a0 con 0x7f93b8106b80 2024-02-28T23:40:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.445+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f93a800f5d0 con 0x7f93b8106b80 2024-02-28T23:40:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.446+0000 7f93af400700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 0x7f939c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.446+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f93a808eb50 con 0x7f93b8106b80 2024-02-28T23:40:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.446+0000 7f93afe00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 0x7f939c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.447+0000 7f93afe00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 0x7f939c06fd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f93b819db20 tx=0x7f93a000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:08.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.452+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f93a8058260 con 0x7f93b8106b80 2024-02-28T23:40:08.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.706+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93b8109570 con 0x7f939c06d8a0 2024-02-28T23:40:08.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.709+0000 7f93af400700 1 -- 172.21.15.57:0/717856160 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f93b8109570 con 0x7f939c06d8a0 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 msgr2=0x7f939c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 0x7f939c06fd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f93b819db20 tx=0x7f93a000a380 comp rx=0 tx=0).stop 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 msgr2=0x7f93b819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b5c0 tx=0x7f93a800bd50 comp rx=0 tx=0).stop 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 shutdown_connections 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f939c06d8a0 0x7f939c06fd50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93b8106b80 0x7f93b819c500 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 --2- 172.21.15.57:0/717856160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b810c8f0 0x7f93b819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.712+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 >> 172.21.15.57:0/717856160 conn(0x7f93b8076060 msgr2=0x7f93b8077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:08.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.713+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 shutdown_connections 2024-02-28T23:40:08.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.713+0000 7f93bda00700 1 -- 172.21.15.57:0/717856160 wait complete. 2024-02-28T23:40:08.727 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:40:08.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 -- 172.21.15.57:0/1034689187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 msgr2=0x7f18cc1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 --2- 172.21.15.57:0/1034689187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1028e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009910 tx=0x7f18b8009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:08.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 -- 172.21.15.57:0/1034689187 shutdown_connections 2024-02-28T23:40:08.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 --2- 172.21.15.57:0/1034689187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1028e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 --2- 172.21.15.57:0/1034689187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 0x7f18cc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.877+0000 7f18d1600700 1 -- 172.21.15.57:0/1034689187 >> 172.21.15.57:0/1034689187 conn(0x7f18cc0fdf90 msgr2=0x7f18cc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:08.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.878+0000 7f18d1600700 1 -- 172.21.15.57:0/1034689187 shutdown_connections 2024-02-28T23:40:08.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.878+0000 7f18d1600700 1 -- 172.21.15.57:0/1034689187 wait complete. 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 Processor -- start 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 -- start start 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 0x7f18cc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc198d80 con 0x7f18cc102470 2024-02-28T23:40:08.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.879+0000 7f18d1600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc198ec0 con 0x7f18cc108470 2024-02-28T23:40:08.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54942/0 (socket says 172.21.15.57:54942) 2024-02-28T23:40:08.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 -- 172.21.15.57:0/484503952 learned_addr learned my addr 172.21.15.57:0/484503952 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:08.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18ca000700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 0x7f18cc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 -- 172.21.15.57:0/484503952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 msgr2=0x7f18cc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 0x7f18cc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.880+0000 7f18caa00700 1 -- 172.21.15.57:0/484503952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80095f0 con 0x7f18cc102470 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18caa00700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00c930 tx=0x7f18bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc007ad0 con 0x7f18cc102470 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18bc00ce80 con 0x7f18cc102470 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc19cd10 con 0x7f18cc102470 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc018630 con 0x7f18cc102470 2024-02-28T23:40:08.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.881+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc19d260 con 0x7f18cc102470 2024-02-28T23:40:08.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.882+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc10a970 con 0x7f18cc102470 2024-02-28T23:40:08.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.883+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f18bc007c30 con 0x7f18cc102470 2024-02-28T23:40:08.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.884+0000 7f18c3400700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 0x7f18b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:08.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.884+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f18bc08d5f0 con 0x7f18cc102470 2024-02-28T23:40:08.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.884+0000 7f18ca000700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 0x7f18b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:08.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.885+0000 7f18ca000700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 0x7f18b006fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18cc199810 tx=0x7f18b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:08.884 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:08 smithi057 ceph-mon[89491]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:08.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:08.890+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18bc056d00 con 0x7f18cc102470 2024-02-28T23:40:08.941 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:08 smithi084 ceph-mon[70440]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:09.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.133+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18cc199690 con 0x7f18b006d8a0 2024-02-28T23:40:09.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.136+0000 7f18c3400700 1 -- 172.21.15.57:0/484503952 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f18b801e040 con 0x7f18b006d8a0 2024-02-28T23:40:09.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 msgr2=0x7f18b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 0x7f18b006fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18cc199810 tx=0x7f18b801a040 comp rx=0 tx=0).stop 2024-02-28T23:40:09.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 msgr2=0x7f18cc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00c930 tx=0x7f18bc00cc40 comp rx=0 tx=0).stop 2024-02-28T23:40:09.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 shutdown_connections 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f18b006d8a0 0x7f18b006fd50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18cc102470 0x7f18cc1981f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 --2- 172.21.15.57:0/484503952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18cc108470 0x7f18cc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 >> 172.21.15.57:0/484503952 conn(0x7f18cc0fdf90 msgr2=0x7f18cc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.139+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 shutdown_connections 2024-02-28T23:40:09.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.140+0000 7f18d1600700 1 -- 172.21.15.57:0/484503952 wait complete. 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 -- 172.21.15.57:0/3098605867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 msgr2=0x7f14d410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 --2- 172.21.15.57:0/3098605867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d410ccc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009910 tx=0x7f14c8009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 -- 172.21.15.57:0/3098605867 shutdown_connections 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 --2- 172.21.15.57:0/3098605867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 0x7f14d4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 --2- 172.21.15.57:0/3098605867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d410ccc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 -- 172.21.15.57:0/3098605867 >> 172.21.15.57:0/3098605867 conn(0x7f14d4076060 msgr2=0x7f14d4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:09.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.298+0000 7f14db800700 1 -- 172.21.15.57:0/3098605867 shutdown_connections 2024-02-28T23:40:09.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.299+0000 7f14db800700 1 -- 172.21.15.57:0/3098605867 wait complete. 2024-02-28T23:40:09.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.300+0000 7f14db800700 1 Processor -- start 2024-02-28T23:40:09.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.300+0000 7f14db800700 1 -- start start 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14db800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 0x7f14d419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14db800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14db800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d419d120 con 0x7f14d410c8f0 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14db800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41a0eb0 con 0x7f14d4106b80 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54964/0 (socket says 172.21.15.57:54964) 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d3e00700 1 -- 172.21.15.57:0/2248823080 learned_addr learned my addr 172.21.15.57:0/2248823080 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d8e00700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 0x7f14d419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d3e00700 1 -- 172.21.15.57:0/2248823080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 msgr2=0x7f14d419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.301+0000 7f14d3e00700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 0x7f14d419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14d3e00700 1 -- 172.21.15.57:0/2248823080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80095f0 con 0x7f14d410c8f0 2024-02-28T23:40:09.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14d3e00700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d8d0 tx=0x7f14c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:09.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c0009940 con 0x7f14d410c8f0 2024-02-28T23:40:09.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d41a1190 con 0x7f14d410c8f0 2024-02-28T23:40:09.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14c0010460 con 0x7f14d410c8f0 2024-02-28T23:40:09.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c001b620 con 0x7f14d410c8f0 2024-02-28T23:40:09.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.302+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d41a16e0 con 0x7f14d410c8f0 2024-02-28T23:40:09.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.304+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d4109b20 con 0x7f14d410c8f0 2024-02-28T23:40:09.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.305+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f14c00105d0 con 0x7f14d410c8f0 2024-02-28T23:40:09.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.305+0000 7f14d1600700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 0x7f14bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.306+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f14c008d8a0 con 0x7f14d410c8f0 2024-02-28T23:40:09.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.306+0000 7f14d8e00700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 0x7f14bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.307+0000 7f14d8e00700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 0x7f14bc06fd50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006010 tx=0x7f14c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:09.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.312+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14c0057050 con 0x7f14d410c8f0 2024-02-28T23:40:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.547+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f14d41a1900 con 0x7f14bc06d8a0 2024-02-28T23:40:09.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.580+0000 7f14d1600700 1 -- 172.21.15.57:0/2248823080 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f14d41a1900 con 0x7f14bc06d8a0 2024-02-28T23:40:09.579 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:40:09.579 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (29m) 5m ago 43m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:40:09.579 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (39m) 5m ago 44m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:40:09.579 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (42m) 5m ago 42m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (25m) 5m ago 43m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (25m) 5m ago 42m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (29m) 5m ago 43m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (21m) 5m ago 36m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (21m) 5m ago 36m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (5m) 5m ago 36m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (20m) 5m ago 36m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (32m) 5m ago 45m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (31m) 5m ago 42m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (27m) 5m ago 45m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (26m) 5m ago 42m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (30m) 5m ago 43m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (30m) 5m ago 42m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (25m) 5m ago 41m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:40:09.580 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (24m) 5m ago 40m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:40:09.581 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (24m) 5m ago 39m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:40:09.581 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (23m) 5m ago 39m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:40:09.581 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (22m) 5m ago 38m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:40:09.581 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (22m) 5m ago 38m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:40:09.581 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (30m) 5m ago 42m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 msgr2=0x7f14bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 0x7f14bc06fd50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006010 tx=0x7f14c801a040 comp rx=0 tx=0).stop 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 msgr2=0x7f14d419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d8d0 tx=0x7f14c000dc90 comp rx=0 tx=0).stop 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 shutdown_connections 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14bc06d8a0 0x7f14bc06fd50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106b80 0x7f14d419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 --2- 172.21.15.57:0/2248823080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d410c8f0 0x7f14d419ca40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 >> 172.21.15.57:0/2248823080 conn(0x7f14d4076060 msgr2=0x7f14d4077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.583+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 shutdown_connections 2024-02-28T23:40:09.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.584+0000 7f14db800700 1 -- 172.21.15.57:0/2248823080 wait complete. 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 -- 172.21.15.57:0/3225640657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068ee0 msgr2=0x7fbd94110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 --2- 172.21.15.57:0/3225640657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068ee0 0x7fbd94110a10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009940 tx=0x7fbd80009c50 comp rx=0 tx=0).stop 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 -- 172.21.15.57:0/3225640657 shutdown_connections 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 --2- 172.21.15.57:0/3225640657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068ee0 0x7fbd94110a10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 --2- 172.21.15.57:0/3225640657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068540 0x7fbd94068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 -- 172.21.15.57:0/3225640657 >> 172.21.15.57:0/3225640657 conn(0x7fbd94075130 msgr2=0x7fbd94075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.743+0000 7fbd9b600700 1 -- 172.21.15.57:0/3225640657 shutdown_connections 2024-02-28T23:40:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.744+0000 7fbd9b600700 1 -- 172.21.15.57:0/3225640657 wait complete. 2024-02-28T23:40:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 Processor -- start 2024-02-28T23:40:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 -- start start 2024-02-28T23:40:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 0x7fbd9419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9419d170 con 0x7fbd94068540 2024-02-28T23:40:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.745+0000 7fbd9b600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a0f00 con 0x7fbd94068ee0 2024-02-28T23:40:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54984/0 (socket says 172.21.15.57:54984) 2024-02-28T23:40:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 -- 172.21.15.57:0/2924508605 learned_addr learned my addr 172.21.15.57:0/2924508605 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd93e00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 0x7fbd9419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 -- 172.21.15.57:0/2924508605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 msgr2=0x7fbd9419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 0x7fbd9419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 -- 172.21.15.57:0/2924508605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800095f0 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd93e00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 0x7fbd9419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.746+0000 7fbd98c00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b4b0 tx=0x7fbd8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.747+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd88011800 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.747+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd88011e20 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.747+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd941a11e0 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.747+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd8800f2e0 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.747+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd941a1730 con 0x7fbd94068540 2024-02-28T23:40:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.748+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd94101410 con 0x7fbd94068540 2024-02-28T23:40:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.749+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbd88010380 con 0x7fbd94068540 2024-02-28T23:40:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.750+0000 7fbd91600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 0x7fbd7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:09.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.750+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd8808d0f0 con 0x7fbd94068540 2024-02-28T23:40:09.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.750+0000 7fbd93e00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 0x7fbd7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:09.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.751+0000 7fbd93e00700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 0x7fbd7c06fd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd80000c00 tx=0x7fbd80005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:09.755+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd88056930 con 0x7fbd94068540 2024-02-28T23:40:09.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:09 smithi057 ceph-mon[89491]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:10.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.078+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd9404f2a0 con 0x7fbd94068540 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.079+0000 7fbd91600700 1 -- 172.21.15.57:0/2924508605 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbd88059ff0 con 0x7fbd94068540 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:40:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:40:10.079 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:40:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:09 smithi084 ceph-mon[70440]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 msgr2=0x7fbd7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 0x7fbd7c06fd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd80000c00 tx=0x7fbd80005fb0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 msgr2=0x7fbd9419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b4b0 tx=0x7fbd8800b870 comp rx=0 tx=0).stop 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 shutdown_connections 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd7c06d8a0 0x7fbd7c06fd50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd94068540 0x7fbd9419c550 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 --2- 172.21.15.57:0/2924508605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd94068ee0 0x7fbd9419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 >> 172.21.15.57:0/2924508605 conn(0x7fbd94075130 msgr2=0x7fbd9410f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 shutdown_connections 2024-02-28T23:40:10.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.082+0000 7fbd9b600700 1 -- 172.21.15.57:0/2924508605 wait complete. 2024-02-28T23:40:10.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.229+0000 7f8f3d000700 1 -- 172.21.15.57:0/1722718976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38102470 msgr2=0x7f8f381028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.229+0000 7f8f3d000700 1 --2- 172.21.15.57:0/1722718976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38102470 0x7f8f381028e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009910 tx=0x7f8f24009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.229+0000 7f8f3d000700 1 -- 172.21.15.57:0/1722718976 shutdown_connections 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.229+0000 7f8f3d000700 1 --2- 172.21.15.57:0/1722718976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38102470 0x7f8f381028e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.230+0000 7f8f3d000700 1 --2- 172.21.15.57:0/1722718976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38108470 0x7f8f38108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.230+0000 7f8f3d000700 1 -- 172.21.15.57:0/1722718976 >> 172.21.15.57:0/1722718976 conn(0x7f8f380fdf90 msgr2=0x7f8f381003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.230+0000 7f8f3d000700 1 -- 172.21.15.57:0/1722718976 shutdown_connections 2024-02-28T23:40:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.230+0000 7f8f3d000700 1 -- 172.21.15.57:0/1722718976 wait complete. 2024-02-28T23:40:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 Processor -- start 2024-02-28T23:40:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 -- start start 2024-02-28T23:40:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38102470 0x7f8f38198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38198db0 con 0x7f8f38108470 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.231+0000 7f8f3d000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3819cb40 con 0x7f8f38102470 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55006/0 (socket says 172.21.15.57:55006) 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 -- 172.21.15.57:0/4145070122 learned_addr learned my addr 172.21.15.57:0/4145070122 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35e00700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38102470 0x7f8f38198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 -- 172.21.15.57:0/4145070122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38102470 msgr2=0x7f8f38198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38102470 0x7f8f38198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 -- 172.21.15.57:0/4145070122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240095f0 con 0x7f8f38108470 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.232+0000 7f8f35400700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b5c0 tx=0x7f8f2400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.233+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2401d070 con 0x7f8f38108470 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.233+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f2400f460 con 0x7f8f38108470 2024-02-28T23:40:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.233+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3819cdc0 con 0x7f8f38108470 2024-02-28T23:40:10.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.233+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f24017610 con 0x7f8f38108470 2024-02-28T23:40:10.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.233+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3819d2b0 con 0x7f8f38108470 2024-02-28T23:40:10.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.234+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3810aa00 con 0x7f8f38108470 2024-02-28T23:40:10.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.235+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8f2400f5d0 con 0x7f8f38108470 2024-02-28T23:40:10.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.236+0000 7f8f2ea00700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 0x7f8f1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.236+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8f2408eb50 con 0x7f8f38108470 2024-02-28T23:40:10.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.236+0000 7f8f35e00700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 0x7f8f1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.237+0000 7f8f35e00700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 0x7f8f1c06fd50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:10.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.241+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f240582e0 con 0x7f8f38108470 2024-02-28T23:40:10.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.507+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8f3810a3e0 con 0x7f8f38108470 2024-02-28T23:40:10.509 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:40:10.509 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:40:10.509 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:40:10.510 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:40:10.511 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:10.512 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:10.513 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:10.513 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:10.513 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.509+0000 7f8f2ea00700 1 -- 172.21.15.57:0/4145070122 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f8f2405b9a0 con 0x7f8f38108470 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 msgr2=0x7f8f1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 0x7f8f1c06fd50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28008040 comp rx=0 tx=0).stop 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 msgr2=0x7f8f381986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b5c0 tx=0x7f8f2400be00 comp rx=0 tx=0).stop 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 shutdown_connections 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f1c06d8a0 0x7f8f1c06fd50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f38102470 0x7f8f38198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 --2- 172.21.15.57:0/4145070122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38108470 0x7f8f381986d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 >> 172.21.15.57:0/4145070122 conn(0x7f8f380fdf90 msgr2=0x7f8f380feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:10.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 shutdown_connections 2024-02-28T23:40:10.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.512+0000 7f8f3d000700 1 -- 172.21.15.57:0/4145070122 wait complete. 2024-02-28T23:40:10.515 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:40:10.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 -- 172.21.15.57:0/629026028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ffd60 msgr2=0x7fc890103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 --2- 172.21.15.57:0/629026028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ffd60 0x7fc890103d30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009910 tx=0x7fc87c009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:10.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 -- 172.21.15.57:0/629026028 shutdown_connections 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 --2- 172.21.15.57:0/629026028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ffd60 0x7fc890103d30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 --2- 172.21.15.57:0/629026028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ff3c0 0x7fc8900ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 -- 172.21.15.57:0/629026028 >> 172.21.15.57:0/629026028 conn(0x7fc8900fac80 msgr2=0x7fc8900fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 -- 172.21.15.57:0/629026028 shutdown_connections 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.660+0000 7fc897400700 1 -- 172.21.15.57:0/629026028 wait complete. 2024-02-28T23:40:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.661+0000 7fc897400700 1 Processor -- start 2024-02-28T23:40:10.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.661+0000 7fc897400700 1 -- start start 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc897400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc897400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ffd60 0x7fc8901963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc897400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc890196a90 con 0x7fc8900ff3c0 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc897400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89019a820 con 0x7fc8900ffd60 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55020/0 (socket says 172.21.15.57:55020) 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 -- 172.21.15.57:0/1670689420 learned_addr learned my addr 172.21.15.57:0/1670689420 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc88fe00700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ffd60 0x7fc8901963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 -- 172.21.15.57:0/1670689420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ffd60 msgr2=0x7fc8901963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ffd60 0x7fc8901963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.662+0000 7fc894a00700 1 -- 172.21.15.57:0/1670689420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0095f0 con 0x7fc8900ff3c0 2024-02-28T23:40:10.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc894a00700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c8a0 tx=0x7fc88400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:10.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc88400cea0 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc884004830 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8840055b0 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89019ab00 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.663+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89019b050 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.664+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89004f2a0 con 0x7fc8900ff3c0 2024-02-28T23:40:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.665+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc8840049a0 con 0x7fc8900ff3c0 2024-02-28T23:40:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.666+0000 7fc88d600700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 0x7fc87806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.666+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc88408d190 con 0x7fc8900ff3c0 2024-02-28T23:40:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.666+0000 7fc88fe00700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 0x7fc87806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.667+0000 7fc88fe00700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 0x7fc87806fd50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc890197490 tx=0x7fc87c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:10.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.671+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc884056920 con 0x7fc8900ff3c0 2024-02-28T23:40:10.818 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:10 smithi057 ceph-mon[89491]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:10 smithi057 ceph-mon[89491]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:10 smithi057 ceph-mon[89491]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2924508605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:40:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4145070122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:40:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.920+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc89019b430 con 0x7fc87806d8a0 2024-02-28T23:40:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.923+0000 7fc88d600700 1 -- 172.21.15.57:0/1670689420 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc89019b430 con 0x7fc87806d8a0 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:40:10.922 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:40:10.923 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 msgr2=0x7fc87806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 0x7fc87806fd50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc890197490 tx=0x7fc87c01a040 comp rx=0 tx=0).stop 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 msgr2=0x7fc890195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c8a0 tx=0x7fc88400cc60 comp rx=0 tx=0).stop 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 shutdown_connections 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc87806d8a0 0x7fc87806fd50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8900ff3c0 0x7fc890195e70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 --2- 172.21.15.57:0/1670689420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8900ffd60 0x7fc8901963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:10.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 >> 172.21.15.57:0/1670689420 conn(0x7fc8900fac80 msgr2=0x7fc8900fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:10.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 shutdown_connections 2024-02-28T23:40:10.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:10.926+0000 7fc897400700 1 -- 172.21.15.57:0/1670689420 wait complete. 2024-02-28T23:40:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:10 smithi084 ceph-mon[70440]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:10 smithi084 ceph-mon[70440]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:10 smithi084 ceph-mon[70440]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2924508605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:40:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4145070122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 -- 172.21.15.57:0/3788901551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 msgr2=0x7fbe58073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 --2- 172.21.15.57:0/3788901551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58073480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009940 tx=0x7fbe48009c50 comp rx=0 tx=0).stop 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 -- 172.21.15.57:0/3788901551 shutdown_connections 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 --2- 172.21.15.57:0/3788901551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe5810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 --2- 172.21.15.57:0/3788901551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58073480 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 -- 172.21.15.57:0/3788901551 >> 172.21.15.57:0/3788901551 conn(0x7fbe580fbda0 msgr2=0x7fbe580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:11.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 -- 172.21.15.57:0/3788901551 shutdown_connections 2024-02-28T23:40:11.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.085+0000 7fbe5e200700 1 -- 172.21.15.57:0/3788901551 wait complete. 2024-02-28T23:40:11.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.086+0000 7fbe5e200700 1 Processor -- start 2024-02-28T23:40:11.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.086+0000 7fbe5e200700 1 -- start start 2024-02-28T23:40:11.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe5e200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:11.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe5e200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe58198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe5e200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58198e30 con 0x7fbe580730b0 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe5e200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819cbc0 con 0x7fbe580739c0 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe57400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe57400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55052/0 (socket says 172.21.15.57:55052) 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe57400700 1 -- 172.21.15.57:0/2547334012 learned_addr learned my addr 172.21.15.57:0/2547334012 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe4ea00700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe58198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe57400700 1 -- 172.21.15.57:0/2547334012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 msgr2=0x7fbe58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.087+0000 7fbe57400700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe58198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe57400700 1 -- 172.21.15.57:0/2547334012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480095f0 con 0x7fbe580730b0 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe4ea00700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:40:11.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe57400700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b580 tx=0x7fbe480056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:11.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4801d070 con 0x7fbe580730b0 2024-02-28T23:40:11.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe4800f460 con 0x7fbe580730b0 2024-02-28T23:40:11.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe48021640 con 0x7fbe580730b0 2024-02-28T23:40:11.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5819ce40 con 0x7fbe580730b0 2024-02-28T23:40:11.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.088+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5819d2e0 con 0x7fbe580730b0 2024-02-28T23:40:11.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.089+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe58109cc0 con 0x7fbe580730b0 2024-02-28T23:40:11.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.090+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbe4800fab0 con 0x7fbe580730b0 2024-02-28T23:40:11.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.091+0000 7fbe54c00700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 0x7fbe3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:11.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.091+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe4808e8e0 con 0x7fbe580730b0 2024-02-28T23:40:11.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.092+0000 7fbe4ea00700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 0x7fbe3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:11.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.092+0000 7fbe4ea00700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 0x7fbe3c06fd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005fd0 tx=0x7fbe40005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:11.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.096+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe48057ff0 con 0x7fbe580730b0 2024-02-28T23:40:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.403+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe5804f2a0 con 0x7fbe580730b0 2024-02-28T23:40:11.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.403+0000 7fbe54c00700 1 -- 172.21.15.57:0/2547334012 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fbe4805b6b0 con 0x7fbe580730b0 2024-02-28T23:40:11.402 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:40:11.402 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:40:11.402 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:40:11.403 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:40:11.404 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:40:11.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 msgr2=0x7fbe3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:11.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 0x7fbe3c06fd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005fd0 tx=0x7fbe40005ee0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 msgr2=0x7fbe58198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:11.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b580 tx=0x7fbe480056a0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 shutdown_connections 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe3c06d8a0 0x7fbe3c06fd50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe580730b0 0x7fbe58198210 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.406+0000 7fbe5e200700 1 --2- 172.21.15.57:0/2547334012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe580739c0 0x7fbe58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.407+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 >> 172.21.15.57:0/2547334012 conn(0x7fbe580fbda0 msgr2=0x7fbe58106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.407+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 shutdown_connections 2024-02-28T23:40:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:11.407+0000 7fbe5e200700 1 -- 172.21.15.57:0/2547334012 wait complete. 2024-02-28T23:40:12.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:11 smithi057 ceph-mon[89491]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:12.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:11 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2547334012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:40:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:11 smithi084 ceph-mon[70440]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:12.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:11 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2547334012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:40:13.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:12 smithi057 ceph-mon[89491]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:12 smithi084 ceph-mon[70440]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:14 smithi057 ceph-mon[89491]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:14 smithi084 ceph-mon[70440]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:17.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:16 smithi057 ceph-mon[89491]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:16 smithi084 ceph-mon[70440]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:19.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:18 smithi057 ceph-mon[89491]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:18 smithi084 ceph-mon[70440]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:21.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:20 smithi057 ceph-mon[89491]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:20 smithi084 ceph-mon[70440]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:23.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:22 smithi057 ceph-mon[89491]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:22 smithi084 ceph-mon[70440]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:24 smithi057 ceph-mon[89491]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:24 smithi084 ceph-mon[70440]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:27.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:26 smithi057 ceph-mon[89491]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:27.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:26 smithi084 ceph-mon[70440]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:28 smithi057 ceph-mon[89491]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:28 smithi084 ceph-mon[70440]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:30 smithi057 ceph-mon[89491]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:30 smithi084 ceph-mon[70440]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:32 smithi057 ceph-mon[89491]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:32 smithi084 ceph-mon[70440]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:34.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:35.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:34 smithi057 ceph-mon[89491]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:34 smithi084 ceph-mon[70440]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:37.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:36 smithi057 ceph-mon[89491]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:36 smithi084 ceph-mon[70440]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:39.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:38 smithi057 ceph-mon[89491]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:38 smithi084 ceph-mon[70440]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:40 smithi057 ceph-mon[89491]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:40 smithi084 ceph-mon[70440]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:41.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 -- 172.21.15.57:0/4268908733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 msgr2=0x7f6724101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:41.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 --2- 172.21.15.57:0/4268908733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724101440 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6714009910 tx=0x7f6714009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:41.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 -- 172.21.15.57:0/4268908733 shutdown_connections 2024-02-28T23:40:41.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 --2- 172.21.15.57:0/4268908733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 0x7f67240689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 --2- 172.21.15.57:0/4268908733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724101440 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 -- 172.21.15.57:0/4268908733 >> 172.21.15.57:0/4268908733 conn(0x7f6724075130 msgr2=0x7f6724075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:41.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 -- 172.21.15.57:0/4268908733 shutdown_connections 2024-02-28T23:40:41.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.580+0000 7f672aa00700 1 -- 172.21.15.57:0/4268908733 wait complete. 2024-02-28T23:40:41.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.581+0000 7f672aa00700 1 Processor -- start 2024-02-28T23:40:41.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.581+0000 7f672aa00700 1 -- start start 2024-02-28T23:40:41.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f672aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 0x7f6724198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:41.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f672aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f672aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6724198c70 con 0x7f6724101070 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f672aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672419ca00 con 0x7f6724068540 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f671b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f671b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40406/0 (socket says 172.21.15.57:40406) 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f671b400700 1 -- 172.21.15.57:0/3333736892 learned_addr learned my addr 172.21.15.57:0/3333736892 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.582+0000 7f6723e00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 0x7f6724198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f671b400700 1 -- 172.21.15.57:0/3333736892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 msgr2=0x7f6724198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f671b400700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 0x7f6724198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f671b400700 1 -- 172.21.15.57:0/3333736892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140095f0 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f671b400700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f670c00d8d0 tx=0x7f670c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c009940 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f670c010460 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c00f5f0 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.583+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672419cce0 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.584+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672419d230 con 0x7f6724101070 2024-02-28T23:40:41.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.585+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672404f2a0 con 0x7f6724101070 2024-02-28T23:40:41.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.586+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f670c0105d0 con 0x7f6724101070 2024-02-28T23:40:41.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.587+0000 7f6721600700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 0x7f670806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:41.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.587+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f670c08d4c0 con 0x7f6724101070 2024-02-28T23:40:41.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.587+0000 7f6723e00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 0x7f670806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:41.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.588+0000 7f6723e00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 0x7f670806fd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6714006010 tx=0x7f671401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:41.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.592+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f670c056c50 con 0x7f6724101070 2024-02-28T23:40:41.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.867+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f672419d580 con 0x7f670806d8a0 2024-02-28T23:40:41.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.869+0000 7f6721600700 1 -- 172.21.15.57:0/3333736892 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f672419d580 con 0x7f670806d8a0 2024-02-28T23:40:41.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 msgr2=0x7f670806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:41.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 0x7f670806fd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6714006010 tx=0x7f671401a040 comp rx=0 tx=0).stop 2024-02-28T23:40:41.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 msgr2=0x7f6724198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:41.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f670c00d8d0 tx=0x7f670c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 shutdown_connections 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f670806d8a0 0x7f670806fd50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6724068540 0x7f6724198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 --2- 172.21.15.57:0/3333736892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6724101070 0x7f6724198590 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 >> 172.21.15.57:0/3333736892 conn(0x7f6724075130 msgr2=0x7f67240fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.872+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 shutdown_connections 2024-02-28T23:40:41.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:41.873+0000 7f672aa00700 1 -- 172.21.15.57:0/3333736892 wait complete. 2024-02-28T23:40:41.888 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 -- 172.21.15.57:0/3210801971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f640730f0 msgr2=0x7f0f640734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3210801971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f640730f0 0x7f0f640734c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009910 tx=0x7f0f58009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 -- 172.21.15.57:0/3210801971 shutdown_connections 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3210801971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64073a00 0x7f0f6410c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3210801971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f640730f0 0x7f0f640734c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 -- 172.21.15.57:0/3210801971 >> 172.21.15.57:0/3210801971 conn(0x7f0f640fbb60 msgr2=0x7f0f640fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 -- 172.21.15.57:0/3210801971 shutdown_connections 2024-02-28T23:40:42.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.038+0000 7f0f6a800700 1 -- 172.21.15.57:0/3210801971 wait complete. 2024-02-28T23:40:42.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.039+0000 7f0f6a800700 1 Processor -- start 2024-02-28T23:40:42.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.039+0000 7f0f6a800700 1 -- start start 2024-02-28T23:40:42.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f6a800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 0x7f0f64197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f6a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f6a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f64198960 con 0x7f0f64073a00 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f6a800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6419c6f0 con 0x7f0f640730f0 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f68a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f68a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40422/0 (socket says 172.21.15.57:40422) 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f68a00700 1 -- 172.21.15.57:0/3296573733 learned_addr learned my addr 172.21.15.57:0/3296573733 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.040+0000 7f0f69400700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 0x7f0f64197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f68a00700 1 -- 172.21.15.57:0/3296573733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 msgr2=0x7f0f64197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f68a00700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 0x7f0f64197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f68a00700 1 -- 172.21.15.57:0/3296573733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580095f0 con 0x7f0f64073a00 2024-02-28T23:40:42.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f69400700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 0x7f0f64197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f68a00700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c930 tx=0x7f0f5000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f50007ad0 con 0x7f0f64073a00 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f50007c30 con 0x7f0f64073a00 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6419c9d0 con 0x7f0f64073a00 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.041+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f50018610 con 0x7f0f64073a00 2024-02-28T23:40:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.042+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6419cf20 con 0x7f0f64073a00 2024-02-28T23:40:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.043+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64109a60 con 0x7f0f64073a00 2024-02-28T23:40:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.044+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0f5000f450 con 0x7f0f64073a00 2024-02-28T23:40:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.045+0000 7f0f62000700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 0x7f0f4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.045+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f5008d7f0 con 0x7f0f64073a00 2024-02-28T23:40:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.046+0000 7f0f69400700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 0x7f0f4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.046+0000 7f0f69400700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 0x7f0f4c06fd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800b5c0 tx=0x7f0f5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.050+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f50056f00 con 0x7f0f64073a00 2024-02-28T23:40:42.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.301+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f6419d210 con 0x7f0f4c06d8a0 2024-02-28T23:40:42.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.304+0000 7f0f62000700 1 -- 172.21.15.57:0/3296573733 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0f6419d210 con 0x7f0f4c06d8a0 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 msgr2=0x7f0f4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 0x7f0f4c06fd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800b5c0 tx=0x7f0f5801a040 comp rx=0 tx=0).stop 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 msgr2=0x7f0f64198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c930 tx=0x7f0f5000ccf0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 shutdown_connections 2024-02-28T23:40:42.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0f4c06d8a0 0x7f0f4c06fd50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f640730f0 0x7f0f64197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 --2- 172.21.15.57:0/3296573733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f64073a00 0x7f0f64198280 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 >> 172.21.15.57:0/3296573733 conn(0x7f0f640fbb60 msgr2=0x7f0f64106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:42.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 shutdown_connections 2024-02-28T23:40:42.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.308+0000 7f0f6a800700 1 -- 172.21.15.57:0/3296573733 wait complete. 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.475+0000 7f1634e00700 1 -- 172.21.15.57:0/2178414094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630102470 msgr2=0x7f16301028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 --2- 172.21.15.57:0/2178414094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630102470 0x7f16301028e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f161c009910 tx=0x7f161c009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 -- 172.21.15.57:0/2178414094 shutdown_connections 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 --2- 172.21.15.57:0/2178414094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630102470 0x7f16301028e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 --2- 172.21.15.57:0/2178414094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630108470 0x7f1630108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 -- 172.21.15.57:0/2178414094 >> 172.21.15.57:0/2178414094 conn(0x7f16300fdf90 msgr2=0x7f16301003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:42.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 -- 172.21.15.57:0/2178414094 shutdown_connections 2024-02-28T23:40:42.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.476+0000 7f1634e00700 1 -- 172.21.15.57:0/2178414094 wait complete. 2024-02-28T23:40:42.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.477+0000 7f1634e00700 1 Processor -- start 2024-02-28T23:40:42.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.477+0000 7f1634e00700 1 -- start start 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f1634e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630102470 0x7f16301981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f1634e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f1634e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630198e10 con 0x7f1630108470 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f1634e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019cba0 con 0x7f1630102470 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40448/0 (socket says 172.21.15.57:40448) 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 -- 172.21.15.57:0/223886842 learned_addr learned my addr 172.21.15.57:0/223886842 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162de00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630102470 0x7f16301981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 -- 172.21.15.57:0/223886842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630102470 msgr2=0x7f16301981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630102470 0x7f16301981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.478+0000 7f162d400700 1 -- 172.21.15.57:0/223886842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0095f0 con 0x7f1630108470 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f162d400700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b5c0 tx=0x7f161c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161c01d070 con 0x7f1630108470 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f161c005e00 con 0x7f1630108470 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161c017400 con 0x7f1630108470 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163019ce20 con 0x7f1630108470 2024-02-28T23:40:42.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.479+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163019d310 con 0x7f1630108470 2024-02-28T23:40:42.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.480+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163010a970 con 0x7f1630108470 2024-02-28T23:40:42.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.481+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f161c00f460 con 0x7f1630108470 2024-02-28T23:40:42.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.483+0000 7f1626a00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 0x7f161406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.483+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f161c08ea10 con 0x7f1630108470 2024-02-28T23:40:42.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.483+0000 7f162de00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 0x7f161406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.484+0000 7f162de00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 0x7f161406fd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1620005950 tx=0x7f16200058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.488+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f161c0581a0 con 0x7f1630108470 2024-02-28T23:40:42.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.729+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1630199660 con 0x7f161406d8a0 2024-02-28T23:40:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.743+0000 7f1626a00700 1 -- 172.21.15.57:0/223886842 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1630199660 con 0x7f161406d8a0 2024-02-28T23:40:42.743 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:40:42.744 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (30m) 6m ago 44m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:40:42.744 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (40m) 6m ago 44m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:40:42.744 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (43m) 6m ago 43m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:40:42.744 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (26m) 6m ago 44m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:40:42.744 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (26m) 6m ago 42m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (29m) 6m ago 43m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (22m) 6m ago 36m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (21m) 6m ago 36m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (6m) 6m ago 36m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (21m) 6m ago 36m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:40:42.745 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (32m) 6m ago 46m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (31m) 6m ago 42m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (28m) 6m ago 46m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (27m) 6m ago 42m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (31m) 6m ago 44m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (31m) 6m ago 42m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (25m) 6m ago 41m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (25m) 6m ago 41m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (24m) 6m ago 40m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:40:42.746 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (24m) 6m ago 39m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:40:42.747 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (23m) 6m ago 39m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:40:42.747 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (22m) 6m ago 38m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:40:42.747 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (30m) 6m ago 43m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 msgr2=0x7f161406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 0x7f161406fd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1620005950 tx=0x7f16200058e0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 msgr2=0x7f1630198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b5c0 tx=0x7f161c00be20 comp rx=0 tx=0).stop 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 shutdown_connections 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f161406d8a0 0x7f161406fd50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1630102470 0x7f16301981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 --2- 172.21.15.57:0/223886842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1630108470 0x7f1630198730 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.745+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 >> 172.21.15.57:0/223886842 conn(0x7f16300fdf90 msgr2=0x7f16300ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:42.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.746+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 shutdown_connections 2024-02-28T23:40:42.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.746+0000 7f1634e00700 1 -- 172.21.15.57:0/223886842 wait complete. 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.898+0000 7ffb5f200700 1 -- 172.21.15.57:0/496232436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb6010c660 msgr2=0x7ffb6010ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.898+0000 7ffb5f200700 1 --2- 172.21.15.57:0/496232436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb6010c660 0x7ffb6010ca30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009910 tx=0x7ffb50009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 -- 172.21.15.57:0/496232436 shutdown_connections 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 --2- 172.21.15.57:0/496232436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb60106b80 0x7ffb60106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 --2- 172.21.15.57:0/496232436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb6010c660 0x7ffb6010ca30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 -- 172.21.15.57:0/496232436 >> 172.21.15.57:0/496232436 conn(0x7ffb60075e90 msgr2=0x7ffb600782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:42.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 -- 172.21.15.57:0/496232436 shutdown_connections 2024-02-28T23:40:42.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.899+0000 7ffb5f200700 1 -- 172.21.15.57:0/496232436 wait complete. 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 Processor -- start 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 -- start start 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb6010c660 0x7ffb6019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019ce10 con 0x7ffb60106b80 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.900+0000 7ffb5f200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019cf50 con 0x7ffb6010c660 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40470/0 (socket says 172.21.15.57:40470) 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 -- 172.21.15.57:0/4148510818 learned_addr learned my addr 172.21.15.57:0/4148510818 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5d400700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb6010c660 0x7ffb6019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 -- 172.21.15.57:0/4148510818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb6010c660 msgr2=0x7ffb6019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb6010c660 0x7ffb6019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 -- 172.21.15.57:0/4148510818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500095f0 con 0x7ffb60106b80 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.901+0000 7ffb5de00700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ffb50006010 tx=0x7ffb5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.902+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5001d070 con 0x7ffb60106b80 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.902+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb50004fb0 con 0x7ffb60106b80 2024-02-28T23:40:42.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.902+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb601a0cd0 con 0x7ffb60106b80 2024-02-28T23:40:42.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.902+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb50022ea0 con 0x7ffb60106b80 2024-02-28T23:40:42.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.902+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601a11c0 con 0x7ffb60106b80 2024-02-28T23:40:42.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.903+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6010eb60 con 0x7ffb60106b80 2024-02-28T23:40:42.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.904+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ffb50022470 con 0x7ffb60106b80 2024-02-28T23:40:42.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.905+0000 7ffb56a00700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 0x7ffb4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:42.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.905+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffb5008ea50 con 0x7ffb60106b80 2024-02-28T23:40:42.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.905+0000 7ffb5d400700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 0x7ffb4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:42.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.906+0000 7ffb5d400700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 0x7ffb4406fd50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005ea0 tx=0x7ffb4c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:42.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:42.909+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb50058290 con 0x7ffb60106b80 2024-02-28T23:40:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:42 smithi057 ceph-mon[89491]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:42 smithi057 ceph-mon[89491]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:42 smithi084 ceph-mon[70440]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:42 smithi084 ceph-mon[70440]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:43.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.219+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb6019d5e0 con 0x7ffb60106b80 2024-02-28T23:40:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.220+0000 7ffb56a00700 1 -- 172.21.15.57:0/4148510818 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ffb50022720 con 0x7ffb60106b80 2024-02-28T23:40:43.220 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:40:43.220 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:40:43.220 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:40:43.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:40:43.222 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:40:43.222 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:40:43.222 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:40:43.222 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 msgr2=0x7ffb4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 0x7ffb4406fd50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005ea0 tx=0x7ffb4c005e30 comp rx=0 tx=0).stop 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 msgr2=0x7ffb6019c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ffb50006010 tx=0x7ffb5000be20 comp rx=0 tx=0).stop 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 shutdown_connections 2024-02-28T23:40:43.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ffb4406d8a0 0x7ffb4406fd50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb60106b80 0x7ffb6019c280 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 --2- 172.21.15.57:0/4148510818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb6010c660 0x7ffb6019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 >> 172.21.15.57:0/4148510818 conn(0x7ffb60075e90 msgr2=0x7ffb60077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 shutdown_connections 2024-02-28T23:40:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.223+0000 7ffb5f200700 1 -- 172.21.15.57:0/4148510818 wait complete. 2024-02-28T23:40:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 -- 172.21.15.57:0/2634380771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 msgr2=0x7fcfd80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 --2- 172.21.15.57:0/2634380771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd80ff790 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 -- 172.21.15.57:0/2634380771 shutdown_connections 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 --2- 172.21.15.57:0/2634380771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 0x7fcfd8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 --2- 172.21.15.57:0/2634380771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd80ff790 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.387+0000 7fcfdca00700 1 -- 172.21.15.57:0/2634380771 >> 172.21.15.57:0/2634380771 conn(0x7fcfd80fac60 msgr2=0x7fcfd80fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.388+0000 7fcfdca00700 1 -- 172.21.15.57:0/2634380771 shutdown_connections 2024-02-28T23:40:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.388+0000 7fcfdca00700 1 -- 172.21.15.57:0/2634380771 wait complete. 2024-02-28T23:40:43.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.389+0000 7fcfdca00700 1 Processor -- start 2024-02-28T23:40:43.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.389+0000 7fcfdca00700 1 -- start start 2024-02-28T23:40:43.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfdca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfdca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 0x7fcfd81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfdca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8196a80 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfdca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd819a810 con 0x7fcfd80ffd60 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfd5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfd5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40486/0 (socket says 172.21.15.57:40486) 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfd5e00700 1 -- 172.21.15.57:0/567372798 learned_addr learned my addr 172.21.15.57:0/567372798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.390+0000 7fcfd5400700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 0x7fcfd81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfd5e00700 1 -- 172.21.15.57:0/567372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 msgr2=0x7fcfd81963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfd5e00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 0x7fcfd81963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfd5e00700 1 -- 172.21.15.57:0/567372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80095f0 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfd5e00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800f690 tx=0x7fcfc800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc801d070 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc80050d0 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd819aa90 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.391+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc8022470 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.392+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd819af80 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.393+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd81086e0 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.394+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcfc8003680 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.395+0000 7fcfcea00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.395+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcfc808e970 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.396+0000 7fcfd5400700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.396+0000 7fcfd5400700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8197480 tx=0x7fcfc4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:43.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.400+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcfc8058100 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.678+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcfd819b2b0 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.680+0000 7fcfcea00700 1 -- 172.21.15.57:0/567372798 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fcfc805b7c0 con 0x7fcfd80ff3c0 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:40:43.680 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:40:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:40:43.682 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:43.683 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:40:43.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 msgr2=0x7fcfbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8197480 tx=0x7fcfc4009260 comp rx=0 tx=0).stop 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 msgr2=0x7fcfd8195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800f690 tx=0x7fcfc800f770 comp rx=0 tx=0).stop 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 shutdown_connections 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcfbc06d8a0 0x7fcfbc06fd50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfd80ff3c0 0x7fcfd8195e60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 --2- 172.21.15.57:0/567372798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd80ffd60 0x7fcfd81963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 >> 172.21.15.57:0/567372798 conn(0x7fcfd80fac60 msgr2=0x7fcfd8069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 shutdown_connections 2024-02-28T23:40:43.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.683+0000 7fcfdca00700 1 -- 172.21.15.57:0/567372798 wait complete. 2024-02-28T23:40:43.686 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 -- 172.21.15.57:0/401293610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 msgr2=0x7fea18104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 --2- 172.21.15.57:0/401293610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea18104eb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fea04009910 tx=0x7fea04009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 -- 172.21.15.57:0/401293610 shutdown_connections 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 --2- 172.21.15.57:0/401293610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea18104eb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 --2- 172.21.15.57:0/401293610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 0x7fea18068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 -- 172.21.15.57:0/401293610 >> 172.21.15.57:0/401293610 conn(0x7fea18075130 msgr2=0x7fea18075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 -- 172.21.15.57:0/401293610 shutdown_connections 2024-02-28T23:40:43.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.849+0000 7fea1d800700 1 -- 172.21.15.57:0/401293610 wait complete. 2024-02-28T23:40:43.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.850+0000 7fea1d800700 1 Processor -- start 2024-02-28T23:40:43.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea1d800700 1 -- start start 2024-02-28T23:40:43.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea1d800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 0x7fea18195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea1d800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea1d800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea18196a80 con 0x7fea18069140 2024-02-28T23:40:43.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea1d800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1819a810 con 0x7fea180687a0 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea16000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea16000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40500/0 (socket says 172.21.15.57:40500) 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.851+0000 7fea16000700 1 -- 172.21.15.57:0/3604358272 learned_addr learned my addr 172.21.15.57:0/3604358272 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea16a00700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 0x7fea18195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea16000700 1 -- 172.21.15.57:0/3604358272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 msgr2=0x7fea18195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea16000700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 0x7fea18195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea16000700 1 -- 172.21.15.57:0/3604358272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040095f0 con 0x7fea18069140 2024-02-28T23:40:43.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea16000700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea0400bd70 tx=0x7fea0400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea0401d070 con 0x7fea18069140 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1819aa90 con 0x7fea18069140 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea04005dc0 con 0x7fea18069140 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.852+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea04017400 con 0x7fea18069140 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.853+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1819af80 con 0x7fea18069140 2024-02-28T23:40:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.854+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea18108750 con 0x7fea18069140 2024-02-28T23:40:43.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.855+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fea0400f7f0 con 0x7fea18069140 2024-02-28T23:40:43.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.856+0000 7fea0f400700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 0x7fe9fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:43.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.856+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fea0408ea10 con 0x7fea18069140 2024-02-28T23:40:43.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.856+0000 7fea16a00700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 0x7fe9fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:43.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.857+0000 7fea16a00700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 0x7fe9fc06fd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea08005950 tx=0x7fea080058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:43.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:43.861+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea040581a0 con 0x7fea18069140 2024-02-28T23:40:44.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:43 smithi057 ceph-mon[89491]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:44.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:43 smithi057 ceph-mon[89491]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:44.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4148510818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:40:44.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/567372798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:40:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:43 smithi084 ceph-mon[70440]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:43 smithi084 ceph-mon[70440]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4148510818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:40:44.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/567372798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:40:44.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.117+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea1802cca0 con 0x7fe9fc06d8a0 2024-02-28T23:40:44.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.120+0000 7fea0f400700 1 -- 172.21.15.57:0/3604358272 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fea1802cca0 con 0x7fe9fc06d8a0 2024-02-28T23:40:44.119 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:40:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:40:44.120 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:40:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 msgr2=0x7fe9fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 0x7fe9fc06fd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea08005950 tx=0x7fea080058e0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 msgr2=0x7fea181963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea0400bd70 tx=0x7fea0400be50 comp rx=0 tx=0).stop 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 shutdown_connections 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe9fc06d8a0 0x7fe9fc06fd50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea180687a0 0x7fea18195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 --2- 172.21.15.57:0/3604358272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea18069140 0x7fea181963a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 >> 172.21.15.57:0/3604358272 conn(0x7fea18075130 msgr2=0x7fea18104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 shutdown_connections 2024-02-28T23:40:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.123+0000 7fea1d800700 1 -- 172.21.15.57:0/3604358272 wait complete. 2024-02-28T23:40:44.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.283+0000 7f9ec3e00700 1 -- 172.21.15.57:0/1497692938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec410c660 msgr2=0x7f9ec410ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.283+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/1497692938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec410c660 0x7f9ec410ca30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009910 tx=0x7f9eb4009c20 comp rx=0 tx=0).stop 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 -- 172.21.15.57:0/1497692938 shutdown_connections 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/1497692938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec4106b80 0x7f9ec4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/1497692938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec410c660 0x7f9ec410ca30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 -- 172.21.15.57:0/1497692938 >> 172.21.15.57:0/1497692938 conn(0x7f9ec4075e90 msgr2=0x7f9ec40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 -- 172.21.15.57:0/1497692938 shutdown_connections 2024-02-28T23:40:44.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.284+0000 7f9ec3e00700 1 -- 172.21.15.57:0/1497692938 wait complete. 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 Processor -- start 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 -- start start 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 0x7f9ec419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec419cea0 con 0x7f9ec4106b80 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.285+0000 7f9ec3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41a0bc0 con 0x7f9ec410c660 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40516/0 (socket says 172.21.15.57:40516) 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 -- 172.21.15.57:0/3125673792 learned_addr learned my addr 172.21.15.57:0/3125673792 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2000700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 0x7f9ec419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 -- 172.21.15.57:0/3125673792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 msgr2=0x7f9ec419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 0x7f9ec419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 -- 172.21.15.57:0/3125673792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40095f0 con 0x7f9ec4106b80 2024-02-28T23:40:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2000700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 0x7f9ec419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:40:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.286+0000 7f9ec2a00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40092e0 tx=0x7f9eb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.287+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb401d070 con 0x7f9ec4106b80 2024-02-28T23:40:44.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.287+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9eb4005e00 con 0x7f9ec4106b80 2024-02-28T23:40:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.287+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb4017400 con 0x7f9ec4106b80 2024-02-28T23:40:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.287+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec41a0e40 con 0x7f9ec4106b80 2024-02-28T23:40:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.287+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec41a1330 con 0x7f9ec4106b80 2024-02-28T23:40:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.288+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec410eb60 con 0x7f9ec4106b80 2024-02-28T23:40:44.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.289+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9eb400f460 con 0x7f9ec4106b80 2024-02-28T23:40:44.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.290+0000 7f9ebb400700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 0x7f9ea806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:40:44.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.290+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9eb408ea30 con 0x7f9ec4106b80 2024-02-28T23:40:44.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.290+0000 7f9ec2000700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 0x7f9ea806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:40:44.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.291+0000 7f9ec2000700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 0x7f9ea806fd50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005fd0 tx=0x7f9eb0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:40:44.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.295+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9eb40581c0 con 0x7f9ec4106b80 2024-02-28T23:40:44.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.616+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9ec4062380 con 0x7f9ec4106b80 2024-02-28T23:40:44.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.617+0000 7f9ebb400700 1 -- 172.21.15.57:0/3125673792 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f9eb405b880 con 0x7f9ec4106b80 2024-02-28T23:40:44.617 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:40:44.617 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:40:44.617 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:40:44.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:40:44.618 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:40:44.619 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 msgr2=0x7f9ea806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 0x7f9ea806fd50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005fd0 tx=0x7f9eb0005ee0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 msgr2=0x7f9ec419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40092e0 tx=0x7f9eb400be20 comp rx=0 tx=0).stop 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 shutdown_connections 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ea806d8a0 0x7f9ea806fd50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ec4106b80 0x7f9ec419c280 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 --2- 172.21.15.57:0/3125673792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ec410c660 0x7f9ec419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:40:44.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 >> 172.21.15.57:0/3125673792 conn(0x7f9ec4075e90 msgr2=0x7f9ec4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:40:44.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 shutdown_connections 2024-02-28T23:40:44.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:40:44.621+0000 7f9ec3e00700 1 -- 172.21.15.57:0/3125673792 wait complete. 2024-02-28T23:40:45.055 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:44 smithi084 ceph-mon[70440]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:45.055 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:44 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3125673792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:40:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:44 smithi057 ceph-mon[89491]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:44 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3125673792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:40:46.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:45 smithi057 ceph-mon[89491]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:46.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:45 smithi084 ceph-mon[70440]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:40:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:46 smithi057 ceph-mon[89491]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:46 smithi084 ceph-mon[70440]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:48 smithi084 ceph-mon[70440]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:49.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:48 smithi057 ceph-mon[89491]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:40:51.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:50 smithi057 ceph-mon[89491]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:50 smithi084 ceph-mon[70440]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:53.120 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:52 smithi057 ceph-mon[89491]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:53.133 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:52 smithi084 ceph-mon[70440]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:55.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:54 smithi057 ceph-mon[89491]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:54 smithi084 ceph-mon[70440]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:56 smithi057 ceph-mon[89491]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:56 smithi084 ceph-mon[70440]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:59.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:40:58 smithi057 ceph-mon[89491]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:40:59.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:40:58 smithi084 ceph-mon[70440]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:00 smithi057 ceph-mon[89491]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:00 smithi084 ceph-mon[70440]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:02 smithi057 ceph-mon[89491]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:02 smithi084 ceph-mon[70440]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:04 smithi057 ceph-mon[89491]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:04 smithi084 ceph-mon[70440]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:06 smithi057 ceph-mon[89491]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:06 smithi084 ceph-mon[70440]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:08 smithi057 ceph-mon[89491]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:08 smithi084 ceph-mon[70440]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:10 smithi057 ceph-mon[89491]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:11.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:10 smithi084 ceph-mon[70440]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:12 smithi057 ceph-mon[89491]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:12 smithi084 ceph-mon[70440]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:14.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 -- 172.21.15.57:0/4749413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f101026a0 msgr2=0x7f4f10102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:14.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4749413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f101026a0 0x7f4f10102a70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009910 tx=0x7f4f04009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:14.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 -- 172.21.15.57:0/4749413 shutdown_connections 2024-02-28T23:41:14.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4749413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f10103040 0x7f4f10107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:14.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4749413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f101026a0 0x7f4f10102a70 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:14.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 -- 172.21.15.57:0/4749413 >> 172.21.15.57:0/4749413 conn(0x7f4f100fdf40 msgr2=0x7f4f10100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:14.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 -- 172.21.15.57:0/4749413 shutdown_connections 2024-02-28T23:41:14.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.798+0000 7f4f17c00700 1 -- 172.21.15.57:0/4749413 wait complete. 2024-02-28T23:41:14.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.799+0000 7f4f17c00700 1 Processor -- start 2024-02-28T23:41:14.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f17c00700 1 -- start start 2024-02-28T23:41:14.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f17c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f101026a0 0x7f4f10198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:14.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f17c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f17c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10198d50 con 0x7f4f10103040 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f17c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1019cae0 con 0x7f4f101026a0 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f0fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f0fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58394/0 (socket says 172.21.15.57:58394) 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.800+0000 7f4f0fe00700 1 -- 172.21.15.57:0/4261674779 learned_addr learned my addr 172.21.15.57:0/4261674779 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.801+0000 7f4f15200700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f101026a0 0x7f4f10198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.801+0000 7f4f0fe00700 1 -- 172.21.15.57:0/4261674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f101026a0 msgr2=0x7f4f10198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.801+0000 7f4f0fe00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f101026a0 0x7f4f10198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.801+0000 7f4f0fe00700 1 -- 172.21.15.57:0/4261674779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f040095f0 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.801+0000 7f4f0fe00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d900 tx=0x7f4efc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.802+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4efc009900 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.802+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4efc010460 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.802+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1019cdc0 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.802+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4efc00f5f0 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.802+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1019d310 con 0x7f4f10103040 2024-02-28T23:41:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.803+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1010a9b0 con 0x7f4f10103040 2024-02-28T23:41:14.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.804+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4efc0105d0 con 0x7f4f10103040 2024-02-28T23:41:14.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.805+0000 7f4f0d600700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 0x7f4ef806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:14.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.805+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4efc08d430 con 0x7f4f10103040 2024-02-28T23:41:14.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.805+0000 7f4f15200700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 0x7f4ef806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:14.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.806+0000 7f4f15200700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 0x7f4ef806fd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b5c0 tx=0x7f4f0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:14.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:14.810+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4efc056b40 con 0x7f4f10103040 2024-02-28T23:41:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.060+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f1019d530 con 0x7f4ef806d8a0 2024-02-28T23:41:15.059 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:14 smithi057 ceph-mon[89491]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.062+0000 7f4f0d600700 1 -- 172.21.15.57:0/4261674779 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4f1019d530 con 0x7f4ef806d8a0 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.065+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 msgr2=0x7f4ef806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.065+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 0x7f4ef806fd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b5c0 tx=0x7f4f0401a040 comp rx=0 tx=0).stop 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.065+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 msgr2=0x7f4f10198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.065+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d900 tx=0x7f4efc00dcc0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 shutdown_connections 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ef806d8a0 0x7f4ef806fd50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f101026a0 0x7f4f10198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 --2- 172.21.15.57:0/4261674779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f10103040 0x7f4f10198670 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 >> 172.21.15.57:0/4261674779 conn(0x7f4f100fdf40 msgr2=0x7f4f100ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 shutdown_connections 2024-02-28T23:41:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.066+0000 7f4f17c00700 1 -- 172.21.15.57:0/4261674779 wait complete. 2024-02-28T23:41:15.080 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:41:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 -- 172.21.15.57:0/3503242478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 msgr2=0x7f9f74106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 --2- 172.21.15.57:0/3503242478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f74106ff0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009910 tx=0x7f9f60009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 -- 172.21.15.57:0/3503242478 shutdown_connections 2024-02-28T23:41:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 --2- 172.21.15.57:0/3503242478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f74106ff0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 --2- 172.21.15.57:0/3503242478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 0x7f9f7410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 -- 172.21.15.57:0/3503242478 >> 172.21.15.57:0/3503242478 conn(0x7f9f74076060 msgr2=0x7f9f74078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 -- 172.21.15.57:0/3503242478 shutdown_connections 2024-02-28T23:41:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.231+0000 7f9f79e00700 1 -- 172.21.15.57:0/3503242478 wait complete. 2024-02-28T23:41:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 Processor -- start 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 -- start start 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 0x7f9f7419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7419d060 con 0x7f9f74106b80 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.233+0000 7f9f79e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f741a0df0 con 0x7f9f7410c8f0 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59632/0 (socket says 172.21.15.57:59632) 2024-02-28T23:41:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 -- 172.21.15.57:0/2254214956 learned_addr learned my addr 172.21.15.57:0/2254214956 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72000700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 0x7f9f7419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 -- 172.21.15.57:0/2254214956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 msgr2=0x7f9f7419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 0x7f9f7419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.234+0000 7f9f72a00700 1 -- 172.21.15.57:0/2254214956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600095f0 con 0x7f9f74106b80 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f72a00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400d8d0 tx=0x7f9f6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f64009940 con 0x7f9f74106b80 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f64010460 con 0x7f9f74106b80 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f741a10d0 con 0x7f9f74106b80 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f6400f5f0 con 0x7f9f74106b80 2024-02-28T23:41:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.235+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f741a1620 con 0x7f9f74106b80 2024-02-28T23:41:15.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.236+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7410ecc0 con 0x7f9f74106b80 2024-02-28T23:41:15.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.238+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9f640105d0 con 0x7f9f74106b80 2024-02-28T23:41:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.239+0000 7f9f78a00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 0x7f9f5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.239+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9f6408d4c0 con 0x7f9f74106b80 2024-02-28T23:41:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.239+0000 7f9f72000700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 0x7f9f5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.240+0000 7f9f72000700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 0x7f9f5806fd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005b40 tx=0x7f9f6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:15.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.244+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f64056c50 con 0x7f9f74106b80 2024-02-28T23:41:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:14 smithi084 ceph-mon[70440]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:15.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.486+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f741a1890 con 0x7f9f5806d8a0 2024-02-28T23:41:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.488+0000 7f9f78a00700 1 -- 172.21.15.57:0/2254214956 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9f741a1890 con 0x7f9f5806d8a0 2024-02-28T23:41:15.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 msgr2=0x7f9f5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 0x7f9f5806fd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005b40 tx=0x7f9f6001a040 comp rx=0 tx=0).stop 2024-02-28T23:41:15.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 msgr2=0x7f9f7419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400d8d0 tx=0x7f9f6400dc90 comp rx=0 tx=0).stop 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 shutdown_connections 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9f5806d8a0 0x7f9f5806fd50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f74106b80 0x7f9f7419c440 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 --2- 172.21.15.57:0/2254214956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f7410c8f0 0x7f9f7419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 >> 172.21.15.57:0/2254214956 conn(0x7f9f74076060 msgr2=0x7f9f74077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 shutdown_connections 2024-02-28T23:41:15.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.491+0000 7f9f79e00700 1 -- 172.21.15.57:0/2254214956 wait complete. 2024-02-28T23:41:15.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 -- 172.21.15.57:0/3523409124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e4100e80 msgr2=0x7f23e4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 --2- 172.21.15.57:0/3523409124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e4100e80 0x7f23e4104e50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009910 tx=0x7f23cc009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:15.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 -- 172.21.15.57:0/3523409124 shutdown_connections 2024-02-28T23:41:15.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 --2- 172.21.15.57:0/3523409124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e4100e80 0x7f23e4104e50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 --2- 172.21.15.57:0/3523409124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e41004e0 0x7f23e41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 -- 172.21.15.57:0/3523409124 >> 172.21.15.57:0/3523409124 conn(0x7f23e40fbda0 msgr2=0x7f23e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:15.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 -- 172.21.15.57:0/3523409124 shutdown_connections 2024-02-28T23:41:15.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.640+0000 7f23eb600700 1 -- 172.21.15.57:0/3523409124 wait complete. 2024-02-28T23:41:15.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.641+0000 7f23eb600700 1 Processor -- start 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23eb600700 1 -- start start 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23eb600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23eb600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e4100e80 0x7f23e4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23eb600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4198de0 con 0x7f23e41004e0 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23eb600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e419cb70 con 0x7f23e4100e80 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23e8c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23e8c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59654/0 (socket says 172.21.15.57:59654) 2024-02-28T23:41:15.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.642+0000 7f23e8c00700 1 -- 172.21.15.57:0/2786737814 learned_addr learned my addr 172.21.15.57:0/2786737814 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e8c00700 1 -- 172.21.15.57:0/2786737814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e4100e80 msgr2=0x7f23e4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23dbe00700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e4100e80 0x7f23e4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e8c00700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e4100e80 0x7f23e4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e8c00700 1 -- 172.21.15.57:0/2786737814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0095f0 con 0x7f23e41004e0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e8c00700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b4b0 tx=0x7f23d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d4011800 con 0x7f23e41004e0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23d4011e20 con 0x7f23e41004e0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d400f2e0 con 0x7f23e41004e0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.643+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e419ce50 con 0x7f23e41004e0 2024-02-28T23:41:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.644+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e419d3a0 con 0x7f23e41004e0 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.645+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e41045d0 con 0x7f23e41004e0 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.645+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f23d4010380 con 0x7f23e41004e0 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.646+0000 7f23e2000700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 0x7f23c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.647+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23d408d0f0 con 0x7f23e41004e0 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.647+0000 7f23dbe00700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 0x7f23c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:15.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.647+0000 7f23dbe00700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 0x7f23c806fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:15.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.651+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23d4056930 con 0x7f23e41004e0 2024-02-28T23:41:15.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.885+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e419d670 con 0x7f23c806d8a0 2024-02-28T23:41:15.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.918+0000 7f23e2000700 1 -- 172.21.15.57:0/2786737814 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f23e419d670 con 0x7f23c806d8a0 2024-02-28T23:41:15.917 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (30m) 6m ago 44m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (41m) 6m ago 45m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (43m) 6m ago 43m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (26m) 6m ago 45m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (26m) 6m ago 43m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (30m) 6m ago 44m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (22m) 6m ago 37m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (22m) 6m ago 37m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (6m) 6m ago 37m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:41:15.918 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (22m) 6m ago 37m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (33m) 6m ago 46m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (32m) 6m ago 43m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (28m) 6m ago 46m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (28m) 6m ago 43m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (31m) 6m ago 44m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (31m) 6m ago 43m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (26m) 6m ago 42m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (25m) 6m ago 41m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (25m) 6m ago 41m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (24m) 6m ago 40m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:41:15.919 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (24m) 6m ago 39m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:41:15.920 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (23m) 6m ago 39m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:41:15.920 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (31m) 6m ago 44m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:41:15.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 msgr2=0x7f23c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 0x7f23c806fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc005fd0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 msgr2=0x7f23e41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:15.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b4b0 tx=0x7f23d400b870 comp rx=0 tx=0).stop 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 shutdown_connections 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23c806d8a0 0x7f23c806fd50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23e41004e0 0x7f23e41981c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 --2- 172.21.15.57:0/2786737814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23e4100e80 0x7f23e4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 >> 172.21.15.57:0/2786737814 conn(0x7f23e40fbda0 msgr2=0x7f23e40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.922+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 shutdown_connections 2024-02-28T23:41:15.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:15.923+0000 7f23eb600700 1 -- 172.21.15.57:0/2786737814 wait complete. 2024-02-28T23:41:16.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 -- 172.21.15.57:0/3485488987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0106200 msgr2=0x7ff0f01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 --2- 172.21.15.57:0/3485488987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0106200 0x7ff0f01065d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009910 tx=0x7ff0e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 -- 172.21.15.57:0/3485488987 shutdown_connections 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 --2- 172.21.15.57:0/3485488987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0068540 0x7ff0f00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 --2- 172.21.15.57:0/3485488987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0106200 0x7ff0f01065d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 -- 172.21.15.57:0/3485488987 >> 172.21.15.57:0/3485488987 conn(0x7ff0f0074fa0 msgr2=0x7ff0f00753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 -- 172.21.15.57:0/3485488987 shutdown_connections 2024-02-28T23:41:16.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.071+0000 7ff0f5800700 1 -- 172.21.15.57:0/3485488987 wait complete. 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 Processor -- start 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 -- start start 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0106200 0x7ff0f0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0198960 con 0x7ff0f0068540 2024-02-28T23:41:16.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.073+0000 7ff0f5800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f019c6f0 con 0x7ff0f0106200 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59668/0 (socket says 172.21.15.57:59668) 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 -- 172.21.15.57:0/236769522 learned_addr learned my addr 172.21.15.57:0/236769522 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0ef400700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0106200 0x7ff0f0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 -- 172.21.15.57:0/236769522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0106200 msgr2=0x7ff0f0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0106200 0x7ff0f0198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 -- 172.21.15.57:0/236769522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00095f0 con 0x7ff0f0068540 2024-02-28T23:41:16.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.074+0000 7ff0efe00700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005f50 tx=0x7ff0e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.075+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e001d070 con 0x7ff0f0068540 2024-02-28T23:41:16.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.075+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0e000b610 con 0x7ff0f0068540 2024-02-28T23:41:16.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.075+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f019c970 con 0x7ff0f0068540 2024-02-28T23:41:16.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.075+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0022470 con 0x7ff0f0068540 2024-02-28T23:41:16.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.075+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f019cee0 con 0x7ff0f0068540 2024-02-28T23:41:16.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.076+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f0100ea0 con 0x7ff0f0068540 2024-02-28T23:41:16.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.077+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff0e000fca0 con 0x7ff0f0068540 2024-02-28T23:41:16.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.078+0000 7ff0ecc00700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 0x7ff0d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.078+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff0e008ea60 con 0x7ff0f0068540 2024-02-28T23:41:16.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.079+0000 7ff0ef400700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 0x7ff0d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.079+0000 7ff0ef400700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 0x7ff0d406fd50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0199360 tx=0x7ff0dc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.083+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0e0058170 con 0x7ff0f0068540 2024-02-28T23:41:16.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:15 smithi057 ceph-mon[89491]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:16.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:15 smithi084 ceph-mon[70440]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:16.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.391+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0f019d100 con 0x7ff0f0068540 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.392+0000 7ff0ecc00700 1 -- 172.21.15.57:0/236769522 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff0e005b830 con 0x7ff0f0068540 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:41:16.391 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:41:16.392 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 msgr2=0x7ff0d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 0x7ff0d406fd50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0199360 tx=0x7ff0dc009260 comp rx=0 tx=0).stop 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 msgr2=0x7ff0f0197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005f50 tx=0x7ff0e000f740 comp rx=0 tx=0).stop 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 shutdown_connections 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff0d406d8a0 0x7ff0d406fd50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0f0068540 0x7ff0f0197d40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 --2- 172.21.15.57:0/236769522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f0106200 0x7ff0f0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 >> 172.21.15.57:0/236769522 conn(0x7ff0f0074fa0 msgr2=0x7ff0f0107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 shutdown_connections 2024-02-28T23:41:16.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.396+0000 7ff0f5800700 1 -- 172.21.15.57:0/236769522 wait complete. 2024-02-28T23:41:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 -- 172.21.15.57:0/1833726404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 msgr2=0x7f841c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 --2- 172.21.15.57:0/1833726404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c1028a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f840c009910 tx=0x7f840c009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 -- 172.21.15.57:0/1833726404 shutdown_connections 2024-02-28T23:41:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 --2- 172.21.15.57:0/1833726404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 0x7f841c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 --2- 172.21.15.57:0/1833726404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c1028a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 -- 172.21.15.57:0/1833726404 >> 172.21.15.57:0/1833726404 conn(0x7f841c0fdd90 msgr2=0x7f841c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 -- 172.21.15.57:0/1833726404 shutdown_connections 2024-02-28T23:41:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.552+0000 7f8420c00700 1 -- 172.21.15.57:0/1833726404 wait complete. 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.553+0000 7f8420c00700 1 Processor -- start 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8420c00700 1 -- start start 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8420c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8420c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 0x7f841c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8420c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1989f0 con 0x7f841c1024d0 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8420c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c198b30 con 0x7f841c102e70 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f841b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f841b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59698/0 (socket says 172.21.15.57:59698) 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f841b400700 1 -- 172.21.15.57:0/2099980966 learned_addr learned my addr 172.21.15.57:0/2099980966 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.554+0000 7f8413e00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 0x7f841c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f841b400700 1 -- 172.21.15.57:0/2099980966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 msgr2=0x7f841c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f841b400700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 0x7f841c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f841b400700 1 -- 172.21.15.57:0/2099980966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0095f0 con 0x7f841c1024d0 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f841b400700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b5c0 tx=0x7f840c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840c01d070 con 0x7f841c1024d0 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c19c920 con 0x7f841c1024d0 2024-02-28T23:41:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f840c00f460 con 0x7f841c1024d0 2024-02-28T23:41:16.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840c017610 con 0x7f841c1024d0 2024-02-28T23:41:16.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.555+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c19ce10 con 0x7f841c1024d0 2024-02-28T23:41:16.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.556+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c10a7e0 con 0x7f841c1024d0 2024-02-28T23:41:16.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.557+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f840c00f5d0 con 0x7f841c1024d0 2024-02-28T23:41:16.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.558+0000 7f8418c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 0x7f840006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.558+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f840c08eb30 con 0x7f841c1024d0 2024-02-28T23:41:16.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.559+0000 7f8413e00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 0x7f840006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.559+0000 7f8413e00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 0x7f840006fd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f841c199480 tx=0x7f8404005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.563+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f840c0582c0 con 0x7f841c1024d0 2024-02-28T23:41:16.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.826+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f841c02cc30 con 0x7f841c1024d0 2024-02-28T23:41:16.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.827+0000 7f8418c00700 1 -- 172.21.15.57:0/2099980966 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f840c05b980 con 0x7f841c1024d0 2024-02-28T23:41:16.826 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:41:16.826 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:41:16.826 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:41:16.826 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:41:16.827 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:41:16.828 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:41:16.829 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:16.830 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:16.830 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:16.830 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:16.830 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 msgr2=0x7f840006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 0x7f840006fd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f841c199480 tx=0x7f8404005d50 comp rx=0 tx=0).stop 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 msgr2=0x7f841c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b5c0 tx=0x7f840c00be00 comp rx=0 tx=0).stop 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 shutdown_connections 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f840006d8a0 0x7f840006fd50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f841c1024d0 0x7f841c197e60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 --2- 172.21.15.57:0/2099980966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f841c102e70 0x7f841c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.829+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 >> 172.21.15.57:0/2099980966 conn(0x7f841c0fdd90 msgr2=0x7f841c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.830+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 shutdown_connections 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.830+0000 7f8420c00700 1 -- 172.21.15.57:0/2099980966 wait complete. 2024-02-28T23:41:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:41:16.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 -- 172.21.15.57:0/4068940288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 msgr2=0x7fbc0c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 --2- 172.21.15.57:0/4068940288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c10ccc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009910 tx=0x7fbbfc009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 -- 172.21.15.57:0/4068940288 shutdown_connections 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 --2- 172.21.15.57:0/4068940288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 0x7fbc0c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 --2- 172.21.15.57:0/4068940288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c10ccc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 -- 172.21.15.57:0/4068940288 >> 172.21.15.57:0/4068940288 conn(0x7fbc0c076060 msgr2=0x7fbc0c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 -- 172.21.15.57:0/4068940288 shutdown_connections 2024-02-28T23:41:16.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.979+0000 7fbc12a00700 1 -- 172.21.15.57:0/4068940288 wait complete. 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 Processor -- start 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 -- start start 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 0x7fbc0c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19d120 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.980+0000 7fbc12a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1a0eb0 con 0x7fbc0c106b80 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59714/0 (socket says 172.21.15.57:59714) 2024-02-28T23:41:16.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 -- 172.21.15.57:0/2757734088 learned_addr learned my addr 172.21.15.57:0/2757734088 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0be00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 0x7fbc0c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 -- 172.21.15.57:0/2757734088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 msgr2=0x7fbc0c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 0x7fbc0c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.981+0000 7fbc0b400700 1 -- 172.21.15.57:0/2757734088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0095f0 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc0b400700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800d8d0 tx=0x7fbbf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf8009940 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbf8010460 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf800f5f0 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c1a1190 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.982+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c1a16e0 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.983+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c109b20 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.984+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbbf80105d0 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.985+0000 7fbc08c00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 0x7fbbf006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:16.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.985+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbbf808d510 con 0x7fbc0c10c8f0 2024-02-28T23:41:16.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.986+0000 7fbc0be00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 0x7fbbf006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:16.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.986+0000 7fbc0be00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 0x7fbbf006fd50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:16.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:16.990+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbf8056ca0 con 0x7fbc0c10c8f0 2024-02-28T23:41:17.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.232+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc0c1a1a10 con 0x7fbbf006d8a0 2024-02-28T23:41:17.231 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:16 smithi057 ceph-mon[89491]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:17.231 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:16 smithi057 ceph-mon[89491]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:17.231 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:16 smithi057 ceph-mon[89491]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:17.231 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/236769522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:41:17.232 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2099980966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.234+0000 7fbc08c00700 1 -- 172.21.15.57:0/2757734088 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbc0c1a1a10 con 0x7fbbf006d8a0 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:41:17.234 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:41:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 msgr2=0x7fbbf006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 0x7fbbf006fd50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc01a040 comp rx=0 tx=0).stop 2024-02-28T23:41:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 msgr2=0x7fbc0c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800d8d0 tx=0x7fbbf800dc90 comp rx=0 tx=0).stop 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 shutdown_connections 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbbf006d8a0 0x7fbbf006fd50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc0c106b80 0x7fbc0c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 --2- 172.21.15.57:0/2757734088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c10c8f0 0x7fbc0c19ca40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 >> 172.21.15.57:0/2757734088 conn(0x7fbc0c076060 msgr2=0x7fbc0c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 shutdown_connections 2024-02-28T23:41:17.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.237+0000 7fbc12a00700 1 -- 172.21.15.57:0/2757734088 wait complete. 2024-02-28T23:41:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:16 smithi084 ceph-mon[70440]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:16 smithi084 ceph-mon[70440]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:16 smithi084 ceph-mon[70440]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/236769522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:41:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2099980966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:41:17.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 -- 172.21.15.57:0/847391400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 msgr2=0x7fbb8c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 --2- 172.21.15.57:0/847391400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c10a230 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009910 tx=0x7fbb74009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 -- 172.21.15.57:0/847391400 shutdown_connections 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 --2- 172.21.15.57:0/847391400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c10a230 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 --2- 172.21.15.57:0/847391400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 0x7fbb8c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.394+0000 7fbb91a00700 1 -- 172.21.15.57:0/847391400 >> 172.21.15.57:0/847391400 conn(0x7fbb8c0fac80 msgr2=0x7fbb8c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.395+0000 7fbb91a00700 1 -- 172.21.15.57:0/847391400 shutdown_connections 2024-02-28T23:41:17.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.395+0000 7fbb91a00700 1 -- 172.21.15.57:0/847391400 wait complete. 2024-02-28T23:41:17.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 Processor -- start 2024-02-28T23:41:17.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 -- start start 2024-02-28T23:41:17.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 0x7fbb8c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c198d10 con 0x7fbb8c101d30 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.396+0000 7fbb91a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c19caa0 con 0x7fbb8c101420 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59732/0 (socket says 172.21.15.57:59732) 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 -- 172.21.15.57:0/3195929078 learned_addr learned my addr 172.21.15.57:0/3195929078 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8aa00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 0x7fbb8c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 -- 172.21.15.57:0/3195929078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 msgr2=0x7fbb8c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 0x7fbb8c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.397+0000 7fbb8a000700 1 -- 172.21.15.57:0/3195929078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740095f0 con 0x7fbb8c101d30 2024-02-28T23:41:17.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb8a000700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b5c0 tx=0x7fbb7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7401d070 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb74005e00 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c19cd20 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb74017400 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.398+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c19d210 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.399+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c109d00 con 0x7fbb8c101d30 2024-02-28T23:41:17.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.400+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbb7400f460 con 0x7fbb8c101d30 2024-02-28T23:41:17.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.401+0000 7fbb83400700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 0x7fbb7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:17.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.401+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb7408ea10 con 0x7fbb8c101d30 2024-02-28T23:41:17.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.402+0000 7fbb8aa00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 0x7fbb7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:17.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.402+0000 7fbb8aa00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 0x7fbb7006fd50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009d10 tx=0x7fbb7c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:17.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.406+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb740581a0 con 0x7fbb8c101d30 2024-02-28T23:41:17.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.726+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbb8c1096e0 con 0x7fbb8c101d30 2024-02-28T23:41:17.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.726+0000 7fbb83400700 1 -- 172.21.15.57:0/3195929078 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fbb7405b860 con 0x7fbb8c101d30 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:41:17.726 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:41:17.727 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:41:17.728 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:41:17.728 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:41:17.728 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:41:17.728 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:41:17.728 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:41:17.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 msgr2=0x7fbb7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 0x7fbb7006fd50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009d10 tx=0x7fbb7c009040 comp rx=0 tx=0).stop 2024-02-28T23:41:17.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 msgr2=0x7fbb8c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:17.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b5c0 tx=0x7fbb7400be20 comp rx=0 tx=0).stop 2024-02-28T23:41:17.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 shutdown_connections 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbb7006d8a0 0x7fbb7006fd50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8c101420 0x7fbb8c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 --2- 172.21.15.57:0/3195929078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb8c101d30 0x7fbb8c198630 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.729+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 >> 172.21.15.57:0/3195929078 conn(0x7fbb8c0fac80 msgr2=0x7fbb8c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.730+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 shutdown_connections 2024-02-28T23:41:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:17.730+0000 7fbb91a00700 1 -- 172.21.15.57:0/3195929078 wait complete. 2024-02-28T23:41:18.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3195929078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:41:18.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3195929078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:41:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:18 smithi057 ceph-mon[89491]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:18 smithi057 ceph-mon[89491]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:18 smithi084 ceph-mon[70440]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:18 smithi084 ceph-mon[70440]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:20 smithi057 ceph-mon[89491]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:20 smithi084 ceph-mon[70440]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:22 smithi057 ceph-mon[89491]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:23.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:22 smithi084 ceph-mon[70440]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:24 smithi057 ceph-mon[89491]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:24 smithi084 ceph-mon[70440]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:27.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:26 smithi057 ceph-mon[89491]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:27.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:26 smithi084 ceph-mon[70440]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:29 smithi057 ceph-mon[89491]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:29 smithi084 ceph-mon[70440]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:31 smithi057 ceph-mon[89491]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:31 smithi084 ceph-mon[70440]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:33.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:33 smithi057 ceph-mon[89491]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:33 smithi084 ceph-mon[70440]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:35.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:35 smithi057 ceph-mon[89491]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:35 smithi084 ceph-mon[70440]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:37 smithi057 ceph-mon[89491]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:37 smithi084 ceph-mon[70440]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:39 smithi057 ceph-mon[89491]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:39 smithi084 ceph-mon[70440]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:41.217 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:41 smithi057 ceph-mon[89491]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:41 smithi084 ceph-mon[70440]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:43 smithi057 ceph-mon[89491]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:43.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:43 smithi084 ceph-mon[70440]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:45 smithi084 ceph-mon[70440]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:45 smithi057 ceph-mon[89491]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:47 smithi084 ceph-mon[70440]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:47.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:47 smithi057 ceph-mon[89491]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:47.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.905+0000 7f9de4c00700 1 -- 172.21.15.57:0/3953275009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 msgr2=0x7f9de0110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:47.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.905+0000 7f9de4c00700 1 --2- 172.21.15.57:0/3953275009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de0110bd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009910 tx=0x7f9dcc009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:47.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 -- 172.21.15.57:0/3953275009 shutdown_connections 2024-02-28T23:41:47.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 --2- 172.21.15.57:0/3953275009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de0110bd0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:47.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 --2- 172.21.15.57:0/3953275009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 0x7f9de0075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:47.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 -- 172.21.15.57:0/3953275009 >> 172.21.15.57:0/3953275009 conn(0x7f9de00fdf90 msgr2=0x7f9de01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:47.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 -- 172.21.15.57:0/3953275009 shutdown_connections 2024-02-28T23:41:47.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.906+0000 7f9de4c00700 1 -- 172.21.15.57:0/3953275009 wait complete. 2024-02-28T23:41:47.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 Processor -- start 2024-02-28T23:41:47.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 -- start start 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 0x7f9de019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de019d070 con 0x7f9de0075b90 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.907+0000 7f9de4c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de019d1b0 con 0x7f9de0075280 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57878/0 (socket says 172.21.15.57:57878) 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 -- 172.21.15.57:0/788955329 learned_addr learned my addr 172.21.15.57:0/788955329 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:47.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddde00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 0x7f9de019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 -- 172.21.15.57:0/788955329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 msgr2=0x7f9de019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 0x7f9de019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.908+0000 7f9ddd400700 1 -- 172.21.15.57:0/788955329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0095f0 con 0x7f9de0075b90 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9ddd400700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0092e0 tx=0x7f9dcc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dcc01d070 con 0x7f9de0075b90 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dcc005a00 con 0x7f9de0075b90 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01a0fa0 con 0x7f9de0075b90 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dcc021410 con 0x7f9de0075b90 2024-02-28T23:41:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.909+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de01a14f0 con 0x7f9de0075b90 2024-02-28T23:41:47.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.910+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de010e340 con 0x7f9de0075b90 2024-02-28T23:41:47.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.911+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9dcc00f460 con 0x7f9de0075b90 2024-02-28T23:41:47.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.912+0000 7f9dd6a00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 0x7f9dc406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:47.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.912+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9dcc08e8f0 con 0x7f9de0075b90 2024-02-28T23:41:47.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.913+0000 7f9ddde00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 0x7f9dc406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:47.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.914+0000 7f9ddde00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 0x7f9dc406fd50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005ea0 tx=0x7f9dd0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:47.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:47.917+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9dcc058000 con 0x7f9de0075b90 2024-02-28T23:41:48.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.168+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9de01a17d0 con 0x7f9dc406d8a0 2024-02-28T23:41:48.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.170+0000 7f9dd6a00700 1 -- 172.21.15.57:0/788955329 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9de01a17d0 con 0x7f9dc406d8a0 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 msgr2=0x7f9dc406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 0x7f9dc406fd50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005ea0 tx=0x7f9dd0005e30 comp rx=0 tx=0).stop 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 msgr2=0x7f9de019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0092e0 tx=0x7f9dcc0056a0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 shutdown_connections 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9dc406d8a0 0x7f9dc406fd50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de0075280 0x7f9de019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 --2- 172.21.15.57:0/788955329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9de0075b90 0x7f9de019ca20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.173+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 >> 172.21.15.57:0/788955329 conn(0x7f9de00fdf90 msgr2=0x7f9de00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.174+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 shutdown_connections 2024-02-28T23:41:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.174+0000 7f9de4c00700 1 -- 172.21.15.57:0/788955329 wait complete. 2024-02-28T23:41:48.190 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:41:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 -- 172.21.15.57:0/2046712228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e00730f0 msgr2=0x7fd5e00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 --2- 172.21.15.57:0/2046712228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e00730f0 0x7fd5e00734c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009910 tx=0x7fd5d0009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 -- 172.21.15.57:0/2046712228 shutdown_connections 2024-02-28T23:41:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 --2- 172.21.15.57:0/2046712228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e0073a00 0x7fd5e010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 --2- 172.21.15.57:0/2046712228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e00730f0 0x7fd5e00734c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 -- 172.21.15.57:0/2046712228 >> 172.21.15.57:0/2046712228 conn(0x7fd5e00fbda0 msgr2=0x7fd5e00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 -- 172.21.15.57:0/2046712228 shutdown_connections 2024-02-28T23:41:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.339+0000 7fd5e7600700 1 -- 172.21.15.57:0/2046712228 wait complete. 2024-02-28T23:41:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.341+0000 7fd5e7600700 1 Processor -- start 2024-02-28T23:41:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.341+0000 7fd5e7600700 1 -- start start 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e7600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e7600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e01987d0 0x7fd5e019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e7600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0198de0 con 0x7fd5e0073a00 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e7600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e019d0f0 con 0x7fd5e01987d0 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57896/0 (socket says 172.21.15.57:57896) 2024-02-28T23:41:48.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5e4c00700 1 -- 172.21.15.57:0/3779346087 learned_addr learned my addr 172.21.15.57:0/3779346087 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.342+0000 7fd5d7e00700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e01987d0 0x7fd5e019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5e4c00700 1 -- 172.21.15.57:0/3779346087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e01987d0 msgr2=0x7fd5e019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5e4c00700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e01987d0 0x7fd5e019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5e4c00700 1 -- 172.21.15.57:0/3779346087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00095f0 con 0x7fd5e0073a00 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5e4c00700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0000c00 tx=0x7fd5d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d001d070 con 0x7fd5e0073a00 2024-02-28T23:41:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5d0005dc0 con 0x7fd5e0073a00 2024-02-28T23:41:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d000fe90 con 0x7fd5e0073a00 2024-02-28T23:41:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.343+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e019d3d0 con 0x7fd5e0073a00 2024-02-28T23:41:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.344+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e019d920 con 0x7fd5e0073a00 2024-02-28T23:41:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.345+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e0109c80 con 0x7fd5e0073a00 2024-02-28T23:41:48.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.346+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd5d000f6d0 con 0x7fd5e0073a00 2024-02-28T23:41:48.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.347+0000 7fd5de000700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 0x7fd5c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.347+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd5d008eb00 con 0x7fd5e0073a00 2024-02-28T23:41:48.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.348+0000 7fd5d7e00700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 0x7fd5c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.348+0000 7fd5d7e00700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 0x7fd5c406fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005950 tx=0x7fd5c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:48.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.352+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5d0058210 con 0x7fd5e0073a00 2024-02-28T23:41:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.603+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e0199560 con 0x7fd5c406d8a0 2024-02-28T23:41:48.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.605+0000 7fd5de000700 1 -- 172.21.15.57:0/3779346087 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd5e0199560 con 0x7fd5c406d8a0 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 msgr2=0x7fd5c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 0x7fd5c406fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005950 tx=0x7fd5c80058e0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 msgr2=0x7fd5e0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0000c00 tx=0x7fd5d000be20 comp rx=0 tx=0).stop 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 shutdown_connections 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd5c406d8a0 0x7fd5c406fd50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5e0073a00 0x7fd5e0198200 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 --2- 172.21.15.57:0/3779346087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e01987d0 0x7fd5e019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 >> 172.21.15.57:0/3779346087 conn(0x7fd5e00fbda0 msgr2=0x7fd5e0106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:48.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 shutdown_connections 2024-02-28T23:41:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.609+0000 7fd5e7600700 1 -- 172.21.15.57:0/3779346087 wait complete. 2024-02-28T23:41:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 -- 172.21.15.57:0/702558098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 msgr2=0x7fd3701008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 --2- 172.21.15.57:0/702558098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd3701008b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd360009910 tx=0x7fd360009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 -- 172.21.15.57:0/702558098 shutdown_connections 2024-02-28T23:41:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 --2- 172.21.15.57:0/702558098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 --2- 172.21.15.57:0/702558098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd3701008b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 -- 172.21.15.57:0/702558098 >> 172.21.15.57:0/702558098 conn(0x7fd3700fbda0 msgr2=0x7fd3700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 -- 172.21.15.57:0/702558098 shutdown_connections 2024-02-28T23:41:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.767+0000 7fd376600700 1 -- 172.21.15.57:0/702558098 wait complete. 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.768+0000 7fd376600700 1 Processor -- start 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.768+0000 7fd376600700 1 -- start start 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd376600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd376600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd376600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370073ad0 con 0x7fd3701004e0 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd376600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370073c10 con 0x7fd370100e80 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57916/0 (socket says 172.21.15.57:57916) 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 -- 172.21.15.57:0/1478244401 learned_addr learned my addr 172.21.15.57:0/1478244401 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36ea00700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 -- 172.21.15.57:0/1478244401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 msgr2=0x7fd370073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36f400700 1 -- 172.21.15.57:0/1478244401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600095f0 con 0x7fd3701004e0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.769+0000 7fd36ea00700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd36f400700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd3600092e0 tx=0x7fd36000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36001d070 con 0x7fd3701004e0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd360005e00 con 0x7fd3701004e0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd360017400 con 0x7fd3701004e0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd370073e90 con 0x7fd3701004e0 2024-02-28T23:41:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.770+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3701a23f0 con 0x7fd3701004e0 2024-02-28T23:41:48.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.771+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3701087f0 con 0x7fd3701004e0 2024-02-28T23:41:48.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.772+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd36000f460 con 0x7fd3701004e0 2024-02-28T23:41:48.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.773+0000 7fd367e00700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 0x7fd35406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.773+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd36008ea30 con 0x7fd3701004e0 2024-02-28T23:41:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.773+0000 7fd36ea00700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 0x7fd35406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.774+0000 7fd36ea00700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 0x7fd35406fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:48.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:48.778+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3600581c0 con 0x7fd3701004e0 2024-02-28T23:41:49.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.016+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd370061190 con 0x7fd35406d8a0 2024-02-28T23:41:49.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.033+0000 7fd367e00700 1 -- 172.21.15.57:0/1478244401 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd370061190 con 0x7fd35406d8a0 2024-02-28T23:41:49.033 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (31m) 7m ago 45m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (41m) 7m ago 45m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (44m) 7m ago 44m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (27m) 7m ago 45m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (27m) 7m ago 44m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (30m) 7m ago 44m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (23m) 7m ago 38m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (23m) 7m ago 37m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (7m) 7m ago 37m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (22m) 7m ago 38m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (33m) 7m ago 47m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (32m) 7m ago 43m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (29m) 7m ago 47m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:41:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (28m) 7m ago 43m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (32m) 7m ago 45m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (32m) 7m ago 43m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (26m) 7m ago 42m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (26m) 7m ago 42m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (25m) 7m ago 41m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (25m) 7m ago 40m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (24m) 7m ago 40m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (23m) 7m ago 39m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:41:49.035 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (31m) 7m ago 44m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 msgr2=0x7fd35406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 0x7fd35406fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 msgr2=0x7fd370073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd3600092e0 tx=0x7fd36000be20 comp rx=0 tx=0).stop 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 shutdown_connections 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd35406d8a0 0x7fd35406fd50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3701004e0 0x7fd370073050 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 --2- 172.21.15.57:0/1478244401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370100e80 0x7fd370073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 >> 172.21.15.57:0/1478244401 conn(0x7fd3700fbda0 msgr2=0x7fd3700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 shutdown_connections 2024-02-28T23:41:49.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.037+0000 7fd376600700 1 -- 172.21.15.57:0/1478244401 wait complete. 2024-02-28T23:41:49.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 -- 172.21.15.57:0/3816256815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 msgr2=0x7f43c0100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 --2- 172.21.15.57:0/3816256815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0100890 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009910 tx=0x7f43b4009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:49.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 -- 172.21.15.57:0/3816256815 shutdown_connections 2024-02-28T23:41:49.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 --2- 172.21.15.57:0/3816256815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 0x7f43c0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 --2- 172.21.15.57:0/3816256815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0100890 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 -- 172.21.15.57:0/3816256815 >> 172.21.15.57:0/3816256815 conn(0x7f43c00fbd80 msgr2=0x7f43c00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 -- 172.21.15.57:0/3816256815 shutdown_connections 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.192+0000 7f43c7c00700 1 -- 172.21.15.57:0/3816256815 wait complete. 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.193+0000 7f43c7c00700 1 Processor -- start 2024-02-28T23:41:49.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c7c00700 1 -- start start 2024-02-28T23:41:49.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c7c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c7c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 0x7f43c01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c7c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0196ab0 con 0x7f43c01004c0 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c7c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019a840 con 0x7f43c0100e60 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57930/0 (socket says 172.21.15.57:57930) 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.194+0000 7f43c5200700 1 -- 172.21.15.57:0/2646180335 learned_addr learned my addr 172.21.15.57:0/2646180335 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c5200700 1 -- 172.21.15.57:0/2646180335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 msgr2=0x7f43c01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43bfe00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 0x7f43c01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c5200700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 0x7f43c01963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c5200700 1 -- 172.21.15.57:0/2646180335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40095f0 con 0x7f43c01004c0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c5200700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006010 tx=0x7f43b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b401d070 con 0x7f43c01004c0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b400f460 con 0x7f43c01004c0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b4017630 con 0x7f43c01004c0 2024-02-28T23:41:49.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c019aac0 con 0x7f43c01004c0 2024-02-28T23:41:49.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.195+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c019af60 con 0x7f43c01004c0 2024-02-28T23:41:49.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.197+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c01087d0 con 0x7f43c01004c0 2024-02-28T23:41:49.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.197+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f43b400f5d0 con 0x7f43c01004c0 2024-02-28T23:41:49.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.198+0000 7f43bd600700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 0x7f43a806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.198+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f43b408ea10 con 0x7f43c01004c0 2024-02-28T23:41:49.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.199+0000 7f43bfe00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 0x7f43a806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.199+0000 7f43bfe00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 0x7f43a806fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005950 tx=0x7f43ac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:49.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.203+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43b40581a0 con 0x7f43c01004c0 2024-02-28T23:41:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:49 smithi057 ceph-mon[89491]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.514+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43c004f2a0 con 0x7f43c01004c0 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.515+0000 7f43bd600700 1 -- 172.21.15.57:0/2646180335 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f43b405b860 con 0x7f43c01004c0 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:49.516 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:41:49.517 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:41:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 msgr2=0x7f43a806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 0x7f43a806fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005950 tx=0x7f43ac0058e0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 msgr2=0x7f43c0195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006010 tx=0x7f43b400be20 comp rx=0 tx=0).stop 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 shutdown_connections 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.518+0000 7f43c7c00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f43a806d8a0 0x7f43a806fd50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.519+0000 7f43c7c00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43c01004c0 0x7f43c0195e90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.519+0000 7f43c7c00700 1 --2- 172.21.15.57:0/2646180335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43c0100e60 0x7f43c01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.519+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 >> 172.21.15.57:0/2646180335 conn(0x7f43c00fbd80 msgr2=0x7f43c00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.519+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 shutdown_connections 2024-02-28T23:41:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.519+0000 7f43c7c00700 1 -- 172.21.15.57:0/2646180335 wait complete. 2024-02-28T23:41:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:49 smithi084 ceph-mon[70440]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:49.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:41:49.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.679+0000 7f9f18a00700 1 -- 172.21.15.57:0/1791652872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f141004e0 msgr2=0x7f9f141008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.679+0000 7f9f18a00700 1 --2- 172.21.15.57:0/1791652872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f141004e0 0x7f9f141008b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009910 tx=0x7f9f04009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 -- 172.21.15.57:0/1791652872 shutdown_connections 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 --2- 172.21.15.57:0/1791652872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f14100e80 0x7f9f14104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 --2- 172.21.15.57:0/1791652872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f141004e0 0x7f9f141008b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 -- 172.21.15.57:0/1791652872 >> 172.21.15.57:0/1791652872 conn(0x7f9f140fbda0 msgr2=0x7f9f140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 -- 172.21.15.57:0/1791652872 shutdown_connections 2024-02-28T23:41:49.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.680+0000 7f9f18a00700 1 -- 172.21.15.57:0/1791652872 wait complete. 2024-02-28T23:41:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.681+0000 7f9f18a00700 1 Processor -- start 2024-02-28T23:41:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.681+0000 7f9f18a00700 1 -- start start 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f18a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f141004e0 0x7f9f14073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f18a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f18a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14073ad0 con 0x7f9f14100e80 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f18a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14073c10 con 0x7f9f141004e0 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57952/0 (socket says 172.21.15.57:57952) 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11400700 1 -- 172.21.15.57:0/2309800548 learned_addr learned my addr 172.21.15.57:0/2309800548 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11e00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f141004e0 0x7f9f14073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11400700 1 -- 172.21.15.57:0/2309800548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f141004e0 msgr2=0x7f9f14073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.682+0000 7f9f11400700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f141004e0 0x7f9f14073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f11400700 1 -- 172.21.15.57:0/2309800548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040095f0 con 0x7f9f14100e80 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f11400700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e8f0 tx=0x7f9f0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f00009790 con 0x7f9f14100e80 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f00004cf0 con 0x7f9f14100e80 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f00005690 con 0x7f9f14100e80 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f141a20c0 con 0x7f9f14100e80 2024-02-28T23:41:49.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.683+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f141a24d0 con 0x7f9f14100e80 2024-02-28T23:41:49.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.685+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f141087f0 con 0x7f9f14100e80 2024-02-28T23:41:49.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.685+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9f00004750 con 0x7f9f14100e80 2024-02-28T23:41:49.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.686+0000 7f9f0aa00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 0x7f9ef806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:49.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.686+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9f00014070 con 0x7f9f14100e80 2024-02-28T23:41:49.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.687+0000 7f9f11e00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 0x7f9ef806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:49.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.687+0000 7f9f11e00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 0x7f9ef806fd50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400b760 tx=0x7f9f04005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:49.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.691+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f00057080 con 0x7f9f14100e80 2024-02-28T23:41:49.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.969+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f1404f2a0 con 0x7f9f14100e80 2024-02-28T23:41:49.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.970+0000 7f9f0aa00700 1 -- 172.21.15.57:0/2309800548 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f9f0005a740 con 0x7f9f14100e80 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:41:49.971 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:41:49.972 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:41:49.973 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:49.974 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.972+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 msgr2=0x7f9ef806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.972+0000 7f9f18a00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 0x7f9ef806fd50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400b760 tx=0x7f9f04005940 comp rx=0 tx=0).stop 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.972+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 msgr2=0x7f9f14073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.972+0000 7f9f18a00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e8f0 tx=0x7f9f0000ecb0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 shutdown_connections 2024-02-28T23:41:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9ef806d8a0 0x7f9ef806fd50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f141004e0 0x7f9f14073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 --2- 172.21.15.57:0/2309800548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f14100e80 0x7f9f14073590 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 >> 172.21.15.57:0/2309800548 conn(0x7f9f140fbda0 msgr2=0x7f9f140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 shutdown_connections 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:49.973+0000 7f9f18a00700 1 -- 172.21.15.57:0/2309800548 wait complete. 2024-02-28T23:41:49.976 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:41:50.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.142+0000 7f5ea7800700 1 -- 172.21.15.57:0/3385591431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea0073090 msgr2=0x7f5ea0073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.142+0000 7f5ea7800700 1 --2- 172.21.15.57:0/3385591431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea0073090 0x7f5ea0073460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009940 tx=0x7f5e94009c50 comp rx=0 tx=0).stop 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 -- 172.21.15.57:0/3385591431 shutdown_connections 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 --2- 172.21.15.57:0/3385591431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea00739a0 0x7f5ea010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 --2- 172.21.15.57:0/3385591431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea0073090 0x7f5ea0073460 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 -- 172.21.15.57:0/3385591431 >> 172.21.15.57:0/3385591431 conn(0x7f5ea00fbda0 msgr2=0x7f5ea00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 -- 172.21.15.57:0/3385591431 shutdown_connections 2024-02-28T23:41:50.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.143+0000 7f5ea7800700 1 -- 172.21.15.57:0/3385591431 wait complete. 2024-02-28T23:41:50.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.144+0000 7f5ea7800700 1 Processor -- start 2024-02-28T23:41:50.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.144+0000 7f5ea7800700 1 -- start start 2024-02-28T23:41:50.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5ea7800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea0073090 0x7f5ea0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5ea7800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5ea7800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0198da0 con 0x7f5ea00739a0 2024-02-28T23:41:50.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5ea7800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019cb30 con 0x7f5ea0073090 2024-02-28T23:41:50.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5e9fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5e9fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57968/0 (socket says 172.21.15.57:57968) 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.145+0000 7f5e9fe00700 1 -- 172.21.15.57:0/940694118 learned_addr learned my addr 172.21.15.57:0/940694118 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.146+0000 7f5ea4e00700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea0073090 0x7f5ea0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.146+0000 7f5e9fe00700 1 -- 172.21.15.57:0/940694118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea0073090 msgr2=0x7f5ea0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.146+0000 7f5e9fe00700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea0073090 0x7f5ea0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.146+0000 7f5e9fe00700 1 -- 172.21.15.57:0/940694118 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940095f0 con 0x7f5ea00739a0 2024-02-28T23:41:50.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.146+0000 7f5e9fe00700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00c8a0 tx=0x7f5e8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.147+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8c00cea0 con 0x7f5ea00739a0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.147+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea019ce10 con 0x7f5ea00739a0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.147+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e8c004830 con 0x7f5ea00739a0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.147+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8c0055b0 con 0x7f5ea00739a0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.147+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea019d360 con 0x7f5ea00739a0 2024-02-28T23:41:50.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.148+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0109ce0 con 0x7f5ea00739a0 2024-02-28T23:41:50.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.149+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5e8c0049a0 con 0x7f5ea00739a0 2024-02-28T23:41:50.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.150+0000 7f5e9d600700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 0x7f5e8806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.150+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e8c08d190 con 0x7f5ea00739a0 2024-02-28T23:41:50.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.151+0000 7f5ea4e00700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 0x7f5e8806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.151+0000 7f5ea4e00700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 0x7f5e8806fd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006010 tx=0x7f5e94005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:50.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.155+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e8c056920 con 0x7f5ea00739a0 2024-02-28T23:41:50.318 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:50 smithi057 ceph-mon[89491]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:50 smithi057 ceph-mon[89491]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:50 smithi057 ceph-mon[89491]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:50 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2646180335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:41:50.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:50 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2309800548' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:41:50.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.403+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ea019d630 con 0x7f5e8806d8a0 2024-02-28T23:41:50.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.406+0000 7f5e9d600700 1 -- 172.21.15.57:0/940694118 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ea019d630 con 0x7f5e8806d8a0 2024-02-28T23:41:50.406 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:41:50.406 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:41:50.407 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:41:50.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.409+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 msgr2=0x7f5e8806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.409+0000 7f5ea7800700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 0x7f5e8806fd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006010 tx=0x7f5e94005a90 comp rx=0 tx=0).stop 2024-02-28T23:41:50.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.409+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 msgr2=0x7f5ea01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.409+0000 7f5ea7800700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00c8a0 tx=0x7f5e8c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 shutdown_connections 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e8806d8a0 0x7f5e8806fd50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ea0073090 0x7f5ea0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 --2- 172.21.15.57:0/940694118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea00739a0 0x7f5ea01986c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 >> 172.21.15.57:0/940694118 conn(0x7f5ea00fbda0 msgr2=0x7f5ea0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 shutdown_connections 2024-02-28T23:41:50.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.410+0000 7f5ea7800700 1 -- 172.21.15.57:0/940694118 wait complete. 2024-02-28T23:41:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 -- 172.21.15.57:0/3292006784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b41004e0 msgr2=0x7f97b41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 --2- 172.21.15.57:0/3292006784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b41004e0 0x7f97b41008b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009910 tx=0x7f97a4009c20 comp rx=0 tx=0).stop 2024-02-28T23:41:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 -- 172.21.15.57:0/3292006784 shutdown_connections 2024-02-28T23:41:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 --2- 172.21.15.57:0/3292006784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b4100e80 0x7f97b4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 --2- 172.21.15.57:0/3292006784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b41004e0 0x7f97b41008b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 -- 172.21.15.57:0/3292006784 >> 172.21.15.57:0/3292006784 conn(0x7f97b40fbda0 msgr2=0x7f97b40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:50.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 -- 172.21.15.57:0/3292006784 shutdown_connections 2024-02-28T23:41:50.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.572+0000 7f97b8a00700 1 -- 172.21.15.57:0/3292006784 wait complete. 2024-02-28T23:41:50.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.573+0000 7f97b8a00700 1 Processor -- start 2024-02-28T23:41:50.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b8a00700 1 -- start start 2024-02-28T23:41:50.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b8a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b41004e0 0x7f97b41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b8a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b8a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4198d10 con 0x7f97b4100e80 2024-02-28T23:41:50.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b8a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b419caa0 con 0x7f97b41004e0 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57986/0 (socket says 172.21.15.57:57986) 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.574+0000 7f97b1400700 1 -- 172.21.15.57:0/293135682 learned_addr learned my addr 172.21.15.57:0/293135682 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97b1e00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b41004e0 0x7f97b41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97b1400700 1 -- 172.21.15.57:0/293135682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b41004e0 msgr2=0x7f97b41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97b1400700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b41004e0 0x7f97b41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97b1400700 1 -- 172.21.15.57:0/293135682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40095f0 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97b1400700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f97a000d900 tx=0x7f97a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.575+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a0009900 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.576+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97a0010460 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.576+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a000f5f0 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.576+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b419cd80 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.576+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b419d2d0 con 0x7f97b4100e80 2024-02-28T23:41:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.577+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b41045d0 con 0x7f97b4100e80 2024-02-28T23:41:50.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.578+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f97a0009c70 con 0x7f97b4100e80 2024-02-28T23:41:50.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.579+0000 7f97aaa00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 0x7f979806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:41:50.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.579+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f97a008d380 con 0x7f97b4100e80 2024-02-28T23:41:50.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.580+0000 7f97b1e00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 0x7f979806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:41:50.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.580+0000 7f97b1e00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 0x7f979806fd50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f97a4006010 tx=0x7f97a401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:41:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:50 smithi084 ceph-mon[70440]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:50 smithi084 ceph-mon[70440]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:50 smithi084 ceph-mon[70440]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:50 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2646180335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:41:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:50 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2309800548' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:41:50.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.583+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97a0056b90 con 0x7f97b4100e80 2024-02-28T23:41:50.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.892+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f97b404f2a0 con 0x7f97b4100e80 2024-02-28T23:41:50.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.893+0000 7f97aaa00700 1 -- 172.21.15.57:0/293135682 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f97a005a250 con 0x7f97b4100e80 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:41:50.893 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:41:50.894 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 msgr2=0x7f979806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 0x7f979806fd50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f97a4006010 tx=0x7f97a401a040 comp rx=0 tx=0).stop 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 msgr2=0x7f97b4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f97a000d900 tx=0x7f97a000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 shutdown_connections 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f979806d8a0 0x7f979806fd50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97b41004e0 0x7f97b41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 --2- 172.21.15.57:0/293135682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97b4100e80 0x7f97b4198630 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 >> 172.21.15.57:0/293135682 conn(0x7f97b40fbda0 msgr2=0x7f97b40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:41:50.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 shutdown_connections 2024-02-28T23:41:50.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:41:50.896+0000 7f97b8a00700 1 -- 172.21.15.57:0/293135682 wait complete. 2024-02-28T23:41:51.546 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:51 smithi057 ceph-mon[89491]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:51.546 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:51 smithi057 ceph-mon[89491]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:51.546 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:51 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/293135682' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:41:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:51 smithi084 ceph-mon[70440]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:51 smithi084 ceph-mon[70440]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:41:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:51 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/293135682' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:41:52.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:52 smithi057 ceph-mon[89491]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:52.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:52 smithi084 ceph-mon[70440]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:54 smithi057 ceph-mon[89491]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:54.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:54 smithi084 ceph-mon[70440]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:56.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:56 smithi057 ceph-mon[89491]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:56.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:56 smithi084 ceph-mon[70440]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:58.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:41:58 smithi057 ceph-mon[89491]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:41:58.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:41:58 smithi084 ceph-mon[70440]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:00.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:00 smithi057 ceph-mon[89491]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:00.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:00 smithi084 ceph-mon[70440]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:02.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:02 smithi057 ceph-mon[89491]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:02.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:02 smithi084 ceph-mon[70440]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:03.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:03.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:04.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:04 smithi057 ceph-mon[89491]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:04.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:04 smithi084 ceph-mon[70440]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:06.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:06 smithi057 ceph-mon[89491]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:06.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:06 smithi084 ceph-mon[70440]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:08 smithi057 ceph-mon[89491]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:08.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:08 smithi084 ceph-mon[70440]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:10.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:10 smithi057 ceph-mon[89491]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:10.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:10 smithi084 ceph-mon[70440]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:12.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:12 smithi057 ceph-mon[89491]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:12.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:12 smithi084 ceph-mon[70440]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:14.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:14 smithi084 ceph-mon[70440]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:14 smithi057 ceph-mon[89491]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:16.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:16 smithi084 ceph-mon[70440]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:17.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:16 smithi057 ceph-mon[89491]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:18 smithi057 ceph-mon[89491]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:18 smithi084 ceph-mon[70440]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:19.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:20.846 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:20 smithi057 ceph-mon[89491]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 -- 172.21.15.57:0/3583025916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83641004e0 msgr2=0x7f83641008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 --2- 172.21.15.57:0/3583025916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83641004e0 0x7f83641008b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8354009910 tx=0x7f8354009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 -- 172.21.15.57:0/3583025916 shutdown_connections 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 --2- 172.21.15.57:0/3583025916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8364100e80 0x7f8364104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 --2- 172.21.15.57:0/3583025916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83641004e0 0x7f83641008b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 -- 172.21.15.57:0/3583025916 >> 172.21.15.57:0/3583025916 conn(0x7f83640fbda0 msgr2=0x7f83640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 -- 172.21.15.57:0/3583025916 shutdown_connections 2024-02-28T23:42:21.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.073+0000 7f836b000700 1 -- 172.21.15.57:0/3583025916 wait complete. 2024-02-28T23:42:21.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.074+0000 7f836b000700 1 Processor -- start 2024-02-28T23:42:21.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.074+0000 7f836b000700 1 -- start start 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f836b000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83641004e0 0x7f83641980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f836b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f836b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364198cf0 con 0x7f8364100e80 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f836b000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836419ca80 con 0x7f83641004e0 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f835b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f835b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35114/0 (socket says 172.21.15.57:35114) 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f835b400700 1 -- 172.21.15.57:0/2440144532 learned_addr learned my addr 172.21.15.57:0/2440144532 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.075+0000 7f8363e00700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83641004e0 0x7f83641980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f835b400700 1 -- 172.21.15.57:0/2440144532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83641004e0 msgr2=0x7f83641980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f835b400700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83641004e0 0x7f83641980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f835b400700 1 -- 172.21.15.57:0/2440144532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540095f0 con 0x7f8364100e80 2024-02-28T23:42:21.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f835b400700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c8f0 tx=0x7f834c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c007a50 con 0x7f8364100e80 2024-02-28T23:42:21.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f834c007bb0 con 0x7f8364100e80 2024-02-28T23:42:21.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836419cd60 con 0x7f8364100e80 2024-02-28T23:42:21.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c018630 con 0x7f8364100e80 2024-02-28T23:42:21.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.076+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836419d2b0 con 0x7f8364100e80 2024-02-28T23:42:21.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.077+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83641045d0 con 0x7f8364100e80 2024-02-28T23:42:21.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.079+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f834c007d20 con 0x7f8364100e80 2024-02-28T23:42:21.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.079+0000 7f8361600700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 0x7f834806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.080+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f834c08d400 con 0x7f8364100e80 2024-02-28T23:42:21.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.080+0000 7f8363e00700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 0x7f834806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.081+0000 7f8363e00700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 0x7f834806fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8354006010 tx=0x7f835401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:20 smithi084 ceph-mon[70440]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:21.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.085+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f834c056c30 con 0x7f8364100e80 2024-02-28T23:42:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.335+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f836419d580 con 0x7f834806d8a0 2024-02-28T23:42:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.338+0000 7f8361600700 1 -- 172.21.15.57:0/2440144532 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f836419d580 con 0x7f834806d8a0 2024-02-28T23:42:21.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 msgr2=0x7f834806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 0x7f834806fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8354006010 tx=0x7f835401a040 comp rx=0 tx=0).stop 2024-02-28T23:42:21.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 msgr2=0x7f8364198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c8f0 tx=0x7f834c00ccb0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 shutdown_connections 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f834806d8a0 0x7f834806fd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83641004e0 0x7f83641980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 --2- 172.21.15.57:0/2440144532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8364100e80 0x7f8364198610 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 >> 172.21.15.57:0/2440144532 conn(0x7f83640fbda0 msgr2=0x7f83640fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 shutdown_connections 2024-02-28T23:42:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.341+0000 7f836b000700 1 -- 172.21.15.57:0/2440144532 wait complete. 2024-02-28T23:42:21.356 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:42:21.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.503+0000 7f21a0e00700 1 -- 172.21.15.57:0/2231271883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 msgr2=0x7f219c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.503+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2231271883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1028e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2188009910 tx=0x7f2188009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:21.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 -- 172.21.15.57:0/2231271883 shutdown_connections 2024-02-28T23:42:21.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2231271883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1028e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2231271883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 0x7f219c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 -- 172.21.15.57:0/2231271883 >> 172.21.15.57:0/2231271883 conn(0x7f219c0fdf90 msgr2=0x7f219c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:21.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 -- 172.21.15.57:0/2231271883 shutdown_connections 2024-02-28T23:42:21.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.504+0000 7f21a0e00700 1 -- 172.21.15.57:0/2231271883 wait complete. 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.505+0000 7f21a0e00700 1 Processor -- start 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.505+0000 7f21a0e00700 1 -- start start 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f21a0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f21a0e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 0x7f219c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f21a0e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c198e10 con 0x7f219c102470 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f21a0e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c19cba0 con 0x7f219c108470 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f2199e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f2199e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35138/0 (socket says 172.21.15.57:35138) 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.506+0000 7f2199e00700 1 -- 172.21.15.57:0/2493055125 learned_addr learned my addr 172.21.15.57:0/2493055125 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.507+0000 7f2199400700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 0x7f219c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.507+0000 7f2199e00700 1 -- 172.21.15.57:0/2493055125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 msgr2=0x7f219c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.507+0000 7f2199e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 0x7f219c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.507+0000 7f2199e00700 1 -- 172.21.15.57:0/2493055125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880095f0 con 0x7f219c102470 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.507+0000 7f2199e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ada0 tx=0x7f218c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.508+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c00f070 con 0x7f219c102470 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.508+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c19ce80 con 0x7f219c102470 2024-02-28T23:42:21.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.508+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f218c007bf0 con 0x7f219c102470 2024-02-28T23:42:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.508+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c005690 con 0x7f219c102470 2024-02-28T23:42:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.508+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c19d3d0 con 0x7f219c102470 2024-02-28T23:42:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.509+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c10a970 con 0x7f219c102470 2024-02-28T23:42:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.511+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f218c005830 con 0x7f219c102470 2024-02-28T23:42:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.512+0000 7f2192a00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 0x7f218006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.512+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f218c08ce90 con 0x7f219c102470 2024-02-28T23:42:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.512+0000 7f2199400700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 0x7f218006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.513+0000 7f2199400700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 0x7f218006fd50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f219c199810 tx=0x7f218801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.516+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f218c0566c0 con 0x7f219c102470 2024-02-28T23:42:21.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.768+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219c199690 con 0x7f218006d8a0 2024-02-28T23:42:21.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.772+0000 7f2192a00700 1 -- 172.21.15.57:0/2493055125 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2188005dc0 con 0x7f218006d8a0 2024-02-28T23:42:21.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 msgr2=0x7f218006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 0x7f218006fd50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f219c199810 tx=0x7f218801a040 comp rx=0 tx=0).stop 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 msgr2=0x7f219c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ada0 tx=0x7f218c00c330 comp rx=0 tx=0).stop 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 shutdown_connections 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f218006d8a0 0x7f218006fd50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f219c102470 0x7f219c1981f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 --2- 172.21.15.57:0/2493055125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f219c108470 0x7f219c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 >> 172.21.15.57:0/2493055125 conn(0x7f219c0fdf90 msgr2=0x7f219c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:21.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 shutdown_connections 2024-02-28T23:42:21.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.776+0000 7f21a0e00700 1 -- 172.21.15.57:0/2493055125 wait complete. 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 -- 172.21.15.57:0/3268116146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 msgr2=0x7f283c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 --2- 172.21.15.57:0/3268116146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c1020a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2828009910 tx=0x7f2828009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 -- 172.21.15.57:0/3268116146 shutdown_connections 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 --2- 172.21.15.57:0/3268116146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c1020a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 --2- 172.21.15.57:0/3268116146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 -- 172.21.15.57:0/3268116146 >> 172.21.15.57:0/3268116146 conn(0x7f283c0fbbd0 msgr2=0x7f283c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:21.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 -- 172.21.15.57:0/3268116146 shutdown_connections 2024-02-28T23:42:21.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.942+0000 7f2842800700 1 -- 172.21.15.57:0/3268116146 wait complete. 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.943+0000 7f2842800700 1 Processor -- start 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.943+0000 7f2842800700 1 -- start start 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2842800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2842800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2842800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c073ad0 con 0x7f283c101c30 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2842800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c073c10 con 0x7f283c1003d0 2024-02-28T23:42:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35152/0 (socket says 172.21.15.57:35152) 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 -- 172.21.15.57:0/289997095 learned_addr learned my addr 172.21.15.57:0/289997095 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2841400700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 -- 172.21.15.57:0/289997095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 msgr2=0x7f283c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2840a00700 1 -- 172.21.15.57:0/289997095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280095f0 con 0x7f283c101c30 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.944+0000 7f2841400700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f2840a00700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f282800b5c0 tx=0x7f282800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282801d070 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2828005e00 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c073e90 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2828017400 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.945+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c1a2220 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.947+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c062380 con 0x7f283c101c30 2024-02-28T23:42:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.947+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f282800f460 con 0x7f283c101c30 2024-02-28T23:42:21.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.948+0000 7f283a000700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 0x7f282406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:21.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.948+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f282808e970 con 0x7f283c101c30 2024-02-28T23:42:21.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.949+0000 7f2841400700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 0x7f282406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:21.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.949+0000 7f2841400700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 0x7f282406fd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2830005fd0 tx=0x7f2830005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:21.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:21.953+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28280581a0 con 0x7f283c101c30 2024-02-28T23:42:22.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.193+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f283c0fcae0 con 0x7f282406d8a0 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.206+0000 7f283a000700 1 -- 172.21.15.57:0/289997095 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f283c0fcae0 con 0x7f282406d8a0 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (32m) 7m ago 45m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (42m) 7m ago 46m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (44m) 7m ago 44m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (28m) 7m ago 46m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:42:22.206 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (27m) 7m ago 44m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (31m) 7m ago 45m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (24m) 7m ago 38m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (23m) 7m ago 38m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (7m) 7m ago 38m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (23m) 7m ago 38m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (34m) 7m ago 47m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (33m) 7m ago 44m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (29m) 7m ago 47m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (29m) 7m ago 44m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (33m) 7m ago 46m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (32m) 7m ago 44m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (27m) 7m ago 43m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (26m) 7m ago 42m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:42:22.207 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (26m) 7m ago 42m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:42:22.208 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (25m) 7m ago 41m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:42:22.208 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (25m) 7m ago 40m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:42:22.208 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (24m) 7m ago 40m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:42:22.208 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (32m) 7m ago 45m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:42:22.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.210+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 msgr2=0x7f282406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.210+0000 7f2842800700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 0x7f282406fd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2830005fd0 tx=0x7f2830005ee0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.210+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 msgr2=0x7f283c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.210+0000 7f2842800700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f282800b5c0 tx=0x7f282800be20 comp rx=0 tx=0).stop 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 shutdown_connections 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f282406d8a0 0x7f282406fd50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f283c1003d0 0x7f283c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 --2- 172.21.15.57:0/289997095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f283c101c30 0x7f283c073590 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 >> 172.21.15.57:0/289997095 conn(0x7f283c0fbbd0 msgr2=0x7f283c0fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 shutdown_connections 2024-02-28T23:42:22.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.211+0000 7f2842800700 1 -- 172.21.15.57:0/289997095 wait complete. 2024-02-28T23:42:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 -- 172.21.15.57:0/2735618273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98106b80 msgr2=0x7f1e98106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2735618273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98106b80 0x7f1e98106ff0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009910 tx=0x7f1e84009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 -- 172.21.15.57:0/2735618273 shutdown_connections 2024-02-28T23:42:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2735618273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98106b80 0x7f1e98106ff0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2735618273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 0x7f1e9810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.374+0000 7f1ea0000700 1 -- 172.21.15.57:0/2735618273 >> 172.21.15.57:0/2735618273 conn(0x7f1e98076060 msgr2=0x7f1e98078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.375+0000 7f1ea0000700 1 -- 172.21.15.57:0/2735618273 shutdown_connections 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.375+0000 7f1ea0000700 1 -- 172.21.15.57:0/2735618273 wait complete. 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 Processor -- start 2024-02-28T23:42:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 -- start start 2024-02-28T23:42:22.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 0x7f1e9819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9819d1a0 con 0x7f1e9819cb90 2024-02-28T23:42:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.376+0000 7f1ea0000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9819d310 con 0x7f1e9810c8f0 2024-02-28T23:42:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35170/0 (socket says 172.21.15.57:35170) 2024-02-28T23:42:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 -- 172.21.15.57:0/2183415668 learned_addr learned my addr 172.21.15.57:0/2183415668 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9d600700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 0x7f1e9819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 -- 172.21.15.57:0/2183415668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 msgr2=0x7f1e9819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 0x7f1e9819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.377+0000 7f1e9cc00700 1 -- 172.21.15.57:0/2183415668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840095f0 con 0x7f1e9819cb90 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1e9cc00700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1e84015000 tx=0x7f1e8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e8401d070 con 0x7f1e9819cb90 2024-02-28T23:42:22.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e8400f460 con 0x7f1e9819cb90 2024-02-28T23:42:22.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e981a15a0 con 0x7f1e9819cb90 2024-02-28T23:42:22.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e84017630 con 0x7f1e9819cb90 2024-02-28T23:42:22.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.378+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e981a1af0 con 0x7f1e9819cb90 2024-02-28T23:42:22.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.379+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98109b20 con 0x7f1e9819cb90 2024-02-28T23:42:22.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.380+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1e8400f5d0 con 0x7f1e9819cb90 2024-02-28T23:42:22.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.381+0000 7f1e96000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 0x7f1e8006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.382+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e8408eb50 con 0x7f1e9819cb90 2024-02-28T23:42:22.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.382+0000 7f1e9d600700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 0x7f1e8006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:22.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.383+0000 7f1e9d600700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 0x7f1e8006fda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0060b0 tx=0x7f1e8c007b80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:22.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.386+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e84058310 con 0x7f1e9819cb90 2024-02-28T23:42:22.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.705+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e980008d0 con 0x7f1e9819cb90 2024-02-28T23:42:22.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.706+0000 7f1e96000700 1 -- 172.21.15.57:0/2183415668 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1e8405b9d0 con 0x7f1e9819cb90 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:42:22.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:42:22.706 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.708+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 msgr2=0x7f1e8006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.708+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 0x7f1e8006fda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0060b0 tx=0x7f1e8c007b80 comp rx=0 tx=0).stop 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.708+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 msgr2=0x7f1e981a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.708+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1e84015000 tx=0x7f1e8400be20 comp rx=0 tx=0).stop 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 shutdown_connections 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1e8006d8f0 0x7f1e8006fda0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9810c8f0 0x7f1e9819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 --2- 172.21.15.57:0/2183415668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e9819cb90 0x7f1e981a1000 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 >> 172.21.15.57:0/2183415668 conn(0x7f1e98076060 msgr2=0x7f1e98077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:22.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 shutdown_connections 2024-02-28T23:42:22.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.709+0000 7f1ea0000700 1 -- 172.21.15.57:0/2183415668 wait complete. 2024-02-28T23:42:22.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:22 smithi057 ceph-mon[89491]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:22.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:22 smithi057 ceph-mon[89491]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:22.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:22 smithi057 ceph-mon[89491]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:22.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 -- 172.21.15.57:0/2397273134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 msgr2=0x7f82dc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 --2- 172.21.15.57:0/2397273134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc106ff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009910 tx=0x7f82c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:22.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 -- 172.21.15.57:0/2397273134 shutdown_connections 2024-02-28T23:42:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 --2- 172.21.15.57:0/2397273134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc106ff0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 --2- 172.21.15.57:0/2397273134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82dc10c8f0 0x7f82dc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.867+0000 7f82e1600700 1 -- 172.21.15.57:0/2397273134 >> 172.21.15.57:0/2397273134 conn(0x7f82dc076060 msgr2=0x7f82dc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.868+0000 7f82e1600700 1 -- 172.21.15.57:0/2397273134 shutdown_connections 2024-02-28T23:42:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.868+0000 7f82e1600700 1 -- 172.21.15.57:0/2397273134 wait complete. 2024-02-28T23:42:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 Processor -- start 2024-02-28T23:42:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 -- start start 2024-02-28T23:42:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82dc10c8f0 0x7f82dc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc19d060 con 0x7f82dc106b80 2024-02-28T23:42:22.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82e1600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc1a0df0 con 0x7f82dc10c8f0 2024-02-28T23:42:22.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:22.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82daa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35190/0 (socket says 172.21.15.57:35190) 2024-02-28T23:42:22.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.869+0000 7f82daa00700 1 -- 172.21.15.57:0/1108809830 learned_addr learned my addr 172.21.15.57:0/1108809830 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.870+0000 7f82daa00700 1 -- 172.21.15.57:0/1108809830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82dc10c8f0 msgr2=0x7f82dc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.870+0000 7f82daa00700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82dc10c8f0 0x7f82dc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.870+0000 7f82daa00700 1 -- 172.21.15.57:0/1108809830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40095f0 con 0x7f82dc106b80 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.870+0000 7f82daa00700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c8a0 tx=0x7f82cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.870+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc00cea0 con 0x7f82dc106b80 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.871+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82cc004830 con 0x7f82dc106b80 2024-02-28T23:42:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.871+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc0055b0 con 0x7f82dc106b80 2024-02-28T23:42:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.871+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc1a10d0 con 0x7f82dc106b80 2024-02-28T23:42:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.871+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc1a1620 con 0x7f82dc106b80 2024-02-28T23:42:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.872+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc10ecc0 con 0x7f82dc106b80 2024-02-28T23:42:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.873+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82cc0049a0 con 0x7f82dc106b80 2024-02-28T23:42:22.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.874+0000 7f82d3400700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 0x7f82bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:22.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.874+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f82cc08d190 con 0x7f82dc106b80 2024-02-28T23:42:22.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.875+0000 7f82d3e00700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 0x7f82bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.875+0000 7f82d3e00700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 0x7f82bc06fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006010 tx=0x7f82c400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:22.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:22.879+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82cc056920 con 0x7f82dc106b80 2024-02-28T23:42:23.061 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:22 smithi084 ceph-mon[70440]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:23.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:22 smithi084 ceph-mon[70440]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:23.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:22 smithi084 ceph-mon[70440]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:23.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.158+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f82dc1a18f0 con 0x7f82dc106b80 2024-02-28T23:42:23.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.158+0000 7f82d3400700 1 -- 172.21.15.57:0/1108809830 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f82cc01d740 con 0x7f82dc106b80 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:42:23.158 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:42:23.159 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:23.160 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:23.161 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:23.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 msgr2=0x7f82bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 0x7f82bc06fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006010 tx=0x7f82c400b540 comp rx=0 tx=0).stop 2024-02-28T23:42:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 msgr2=0x7f82dc19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c8a0 tx=0x7f82cc00cc60 comp rx=0 tx=0).stop 2024-02-28T23:42:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 shutdown_connections 2024-02-28T23:42:23.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f82bc06d8a0 0x7f82bc06fd50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82dc106b80 0x7f82dc19c440 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 --2- 172.21.15.57:0/1108809830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82dc10c8f0 0x7f82dc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 >> 172.21.15.57:0/1108809830 conn(0x7f82dc076060 msgr2=0x7f82dc077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 shutdown_connections 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.163+0000 7f82e1600700 1 -- 172.21.15.57:0/1108809830 wait complete. 2024-02-28T23:42:23.163 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:42:23.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.328+0000 7f883e600700 1 -- 172.21.15.57:0/1260717588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 msgr2=0x7f88381066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.328+0000 7f883e600700 1 --2- 172.21.15.57:0/1260717588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381066f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8828009910 tx=0x7f8828009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 -- 172.21.15.57:0/1260717588 shutdown_connections 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 --2- 172.21.15.57:0/1260717588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 0x7f88380689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 --2- 172.21.15.57:0/1260717588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381066f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 -- 172.21.15.57:0/1260717588 >> 172.21.15.57:0/1260717588 conn(0x7f8838075130 msgr2=0x7f8838075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 -- 172.21.15.57:0/1260717588 shutdown_connections 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.329+0000 7f883e600700 1 -- 172.21.15.57:0/1260717588 wait complete. 2024-02-28T23:42:23.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 Processor -- start 2024-02-28T23:42:23.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 -- start start 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 0x7f88381981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838198dc0 con 0x7f8838106320 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.330+0000 7f883e600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819cb50 con 0x7f8838068540 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35202/0 (socket says 172.21.15.57:35202) 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 -- 172.21.15.57:0/2170825063 learned_addr learned my addr 172.21.15.57:0/2170825063 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:23.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f8837400700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 0x7f88381981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 -- 172.21.15.57:0/2170825063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 msgr2=0x7f88381981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 0x7f88381981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.331+0000 7f882fe00700 1 -- 172.21.15.57:0/2170825063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280095f0 con 0x7f8838106320 2024-02-28T23:42:23.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f882fe00700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f882000b4b0 tx=0x7f882000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:23.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8820011800 con 0x7f8838106320 2024-02-28T23:42:23.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8820011e20 con 0x7f8838106320 2024-02-28T23:42:23.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882000f2e0 con 0x7f8838106320 2024-02-28T23:42:23.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883819ce30 con 0x7f8838106320 2024-02-28T23:42:23.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.332+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883819d380 con 0x7f8838106320 2024-02-28T23:42:23.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.333+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883804f2a0 con 0x7f8838106320 2024-02-28T23:42:23.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.334+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8820010380 con 0x7f8838106320 2024-02-28T23:42:23.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.335+0000 7f8834c00700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 0x7f881c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.335+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f882008d0f0 con 0x7f8838106320 2024-02-28T23:42:23.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.336+0000 7f8837400700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 0x7f881c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.336+0000 7f8837400700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 0x7f881c06fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8828006010 tx=0x7f882801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:23.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.340+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8820056930 con 0x7f8838106320 2024-02-28T23:42:23.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.583+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f883819d760 con 0x7f881c06d8a0 2024-02-28T23:42:23.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.586+0000 7f8834c00700 1 -- 172.21.15.57:0/2170825063 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f883819d760 con 0x7f881c06d8a0 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:42:23.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:42:23.587 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:42:23.587 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:42:23.587 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:42:23.587 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:42:23.587 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.589+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 msgr2=0x7f881c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.589+0000 7f883e600700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 0x7f881c06fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8828006010 tx=0x7f882801a040 comp rx=0 tx=0).stop 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.589+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 msgr2=0x7f88381986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.589+0000 7f883e600700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f882000b4b0 tx=0x7f882000b870 comp rx=0 tx=0).stop 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 shutdown_connections 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f881c06d8a0 0x7f881c06fd50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8838068540 0x7f88381981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 --2- 172.21.15.57:0/2170825063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8838106320 0x7f88381986e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 >> 172.21.15.57:0/2170825063 conn(0x7f8838075130 msgr2=0x7f88380fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 shutdown_connections 2024-02-28T23:42:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.590+0000 7f883e600700 1 -- 172.21.15.57:0/2170825063 wait complete. 2024-02-28T23:42:23.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 -- 172.21.15.57:0/2412461881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 msgr2=0x7f5398104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 --2- 172.21.15.57:0/2412461881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398104e50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5380009910 tx=0x7f5380009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:23.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 -- 172.21.15.57:0/2412461881 shutdown_connections 2024-02-28T23:42:23.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 --2- 172.21.15.57:0/2412461881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398104e50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 --2- 172.21.15.57:0/2412461881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 0x7f53981008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 -- 172.21.15.57:0/2412461881 >> 172.21.15.57:0/2412461881 conn(0x7f53980fbda0 msgr2=0x7f53980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:23.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 -- 172.21.15.57:0/2412461881 shutdown_connections 2024-02-28T23:42:23.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.753+0000 7f539ee00700 1 -- 172.21.15.57:0/2412461881 wait complete. 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.754+0000 7f539ee00700 1 Processor -- start 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.755+0000 7f539ee00700 1 -- start start 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.755+0000 7f539ee00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 0x7f53981980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.755+0000 7f539ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.755+0000 7f539ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5398198d00 con 0x7f5398100e80 2024-02-28T23:42:23.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.755+0000 7f539ee00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819ca90 con 0x7f53981004e0 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35232/0 (socket says 172.21.15.57:35232) 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 -- 172.21.15.57:0/3408064399 learned_addr learned my addr 172.21.15.57:0/3408064399 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f5397e00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 0x7f53981980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 -- 172.21.15.57:0/3408064399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 msgr2=0x7f53981980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 0x7f53981980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.756+0000 7f538f400700 1 -- 172.21.15.57:0/3408064399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800095f0 con 0x7f5398100e80 2024-02-28T23:42:23.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f538f400700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f53800092e0 tx=0x7f538000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:23.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f538001d070 con 0x7f5398100e80 2024-02-28T23:42:23.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5380005dc0 con 0x7f5398100e80 2024-02-28T23:42:23.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539819cd10 con 0x7f5398100e80 2024-02-28T23:42:23.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5380017400 con 0x7f5398100e80 2024-02-28T23:42:23.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.757+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539819d200 con 0x7f5398100e80 2024-02-28T23:42:23.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.758+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53981045d0 con 0x7f5398100e80 2024-02-28T23:42:23.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.759+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f538000f460 con 0x7f5398100e80 2024-02-28T23:42:23.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.761+0000 7f5395600700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 0x7f537c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:23.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.761+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f538008e9f0 con 0x7f5398100e80 2024-02-28T23:42:23.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.761+0000 7f5397e00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 0x7f537c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:23.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.762+0000 7f5397e00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 0x7f537c06fd50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5388009d10 tx=0x7f5388009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:23.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:23.766+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5380058180 con 0x7f5398100e80 2024-02-28T23:42:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:23 smithi057 ceph-mon[89491]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2183415668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:42:24.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1108809830' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:42:24.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:23 smithi084 ceph-mon[70440]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:24.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2183415668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:42:24.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1108809830' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:42:24.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.096+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f539802cc30 con 0x7f5398100e80 2024-02-28T23:42:24.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.097+0000 7f5395600700 1 -- 172.21.15.57:0/3408064399 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f538005b840 con 0x7f5398100e80 2024-02-28T23:42:24.096 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:42:24.096 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:42:24.096 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:42:24.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:42:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:42:24.098 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:42:24.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 msgr2=0x7f537c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:24.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 0x7f537c06fd50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5388009d10 tx=0x7f5388009040 comp rx=0 tx=0).stop 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 msgr2=0x7f5398198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f53800092e0 tx=0x7f538000be20 comp rx=0 tx=0).stop 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 shutdown_connections 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f537c06d8a0 0x7f537c06fd50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53981004e0 0x7f53981980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 --2- 172.21.15.57:0/3408064399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5398100e80 0x7f5398198620 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 >> 172.21.15.57:0/3408064399 conn(0x7f53980fbda0 msgr2=0x7f53980fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 shutdown_connections 2024-02-28T23:42:24.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:24.100+0000 7f539ee00700 1 -- 172.21.15.57:0/3408064399 wait complete. 2024-02-28T23:42:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:24 smithi057 ceph-mon[89491]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:25.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:24 smithi057 ceph-mon[89491]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:25.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:24 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3408064399' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:42:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:24 smithi084 ceph-mon[70440]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:24 smithi084 ceph-mon[70440]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:25.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:24 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3408064399' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:42:27.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:26 smithi057 ceph-mon[89491]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:27.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:26 smithi084 ceph-mon[70440]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:28 smithi057 ceph-mon[89491]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:28 smithi084 ceph-mon[70440]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:30 smithi057 ceph-mon[89491]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:30 smithi084 ceph-mon[70440]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:32 smithi057 ceph-mon[89491]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:32 smithi084 ceph-mon[70440]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:34 smithi057 ceph-mon[89491]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:34 smithi084 ceph-mon[70440]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:36 smithi057 ceph-mon[89491]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:36 smithi084 ceph-mon[70440]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:38.830 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:38 smithi084 ceph-mon[70440]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:39.068 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:38 smithi057 ceph-mon[89491]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:40 smithi057 ceph-mon[89491]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:40 smithi084 ceph-mon[70440]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:42 smithi057 ceph-mon[89491]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:42 smithi084 ceph-mon[70440]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:44 smithi057 ceph-mon[89491]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:44 smithi084 ceph-mon[70440]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:46 smithi057 ceph-mon[89491]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:46 smithi084 ceph-mon[70440]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:48 smithi057 ceph-mon[89491]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:48 smithi084 ceph-mon[70440]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:42:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:50 smithi057 ceph-mon[89491]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:50 smithi084 ceph-mon[70440]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:53.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:52 smithi057 ceph-mon[89491]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:52 smithi084 ceph-mon[70440]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:54.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 -- 172.21.15.57:0/4049414896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f4100e60 msgr2=0x7fb0f4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 --2- 172.21.15.57:0/4049414896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f4100e60 0x7fb0f4104e30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009910 tx=0x7fb0e0009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:54.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 -- 172.21.15.57:0/4049414896 shutdown_connections 2024-02-28T23:42:54.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 --2- 172.21.15.57:0/4049414896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f4100e60 0x7fb0f4104e30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 --2- 172.21.15.57:0/4049414896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0f41004c0 0x7fb0f4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.280+0000 7fb0fa800700 1 -- 172.21.15.57:0/4049414896 >> 172.21.15.57:0/4049414896 conn(0x7fb0f40fbd80 msgr2=0x7fb0f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.281+0000 7fb0fa800700 1 -- 172.21.15.57:0/4049414896 shutdown_connections 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.281+0000 7fb0fa800700 1 -- 172.21.15.57:0/4049414896 wait complete. 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.282+0000 7fb0fa800700 1 Processor -- start 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.282+0000 7fb0fa800700 1 -- start start 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0fa800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0fa800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0f4100e60 0x7fb0f4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0fa800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4196b10 con 0x7fb0f41004c0 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0fa800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419a8a0 con 0x7fb0f4100e60 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42788/0 (socket says 172.21.15.57:42788) 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.283+0000 7fb0f3400700 1 -- 172.21.15.57:0/367274861 learned_addr learned my addr 172.21.15.57:0/367274861 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.284+0000 7fb0f3400700 1 -- 172.21.15.57:0/367274861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0f4100e60 msgr2=0x7fb0f4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.284+0000 7fb0f3400700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0f4100e60 0x7fb0f4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.284+0000 7fb0f3400700 1 -- 172.21.15.57:0/367274861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00095f0 con 0x7fb0f41004c0 2024-02-28T23:42:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.284+0000 7fb0f3400700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400d8d0 tx=0x7fb0e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.285+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e4009940 con 0x7fb0f41004c0 2024-02-28T23:42:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.285+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0e4010460 con 0x7fb0f41004c0 2024-02-28T23:42:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.285+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f419ab80 con 0x7fb0f41004c0 2024-02-28T23:42:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.285+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e400f5f0 con 0x7fb0f41004c0 2024-02-28T23:42:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.285+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f419b0d0 con 0x7fb0f41004c0 2024-02-28T23:42:54.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.286+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4108830 con 0x7fb0f41004c0 2024-02-28T23:42:54.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.287+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb0e40105d0 con 0x7fb0f41004c0 2024-02-28T23:42:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.288+0000 7fb0ebe00700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 0x7fb0d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.288+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb0e408d5d0 con 0x7fb0f41004c0 2024-02-28T23:42:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.288+0000 7fb0f2a00700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 0x7fb0d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.289+0000 7fb0f2a00700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 0x7fb0d806fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b5c0 tx=0x7fb0e0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.293+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0e4056ce0 con 0x7fb0f41004c0 2024-02-28T23:42:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.543+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0f419b2f0 con 0x7fb0d806d8a0 2024-02-28T23:42:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.547+0000 7fb0ebe00700 1 -- 172.21.15.57:0/367274861 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb0f419b2f0 con 0x7fb0d806d8a0 2024-02-28T23:42:54.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 msgr2=0x7fb0d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 0x7fb0d806fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b5c0 tx=0x7fb0e0005fd0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 msgr2=0x7fb0f4195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400d8d0 tx=0x7fb0e400dc90 comp rx=0 tx=0).stop 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 shutdown_connections 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb0d806d8a0 0x7fb0d806fd50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f41004c0 0x7fb0f4195ef0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 --2- 172.21.15.57:0/367274861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0f4100e60 0x7fb0f4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 >> 172.21.15.57:0/367274861 conn(0x7fb0f40fbd80 msgr2=0x7fb0f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:54.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.550+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 shutdown_connections 2024-02-28T23:42:54.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.551+0000 7fb0fa800700 1 -- 172.21.15.57:0/367274861 wait complete. 2024-02-28T23:42:54.566 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:42:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 -- 172.21.15.57:0/2318903745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 msgr2=0x7fa47c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 --2- 172.21.15.57:0/2318903745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c10ccc0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa470009910 tx=0x7fa470009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 -- 172.21.15.57:0/2318903745 shutdown_connections 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 --2- 172.21.15.57:0/2318903745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 0x7fa47c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 --2- 172.21.15.57:0/2318903745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c10ccc0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 -- 172.21.15.57:0/2318903745 >> 172.21.15.57:0/2318903745 conn(0x7fa47c076060 msgr2=0x7fa47c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 -- 172.21.15.57:0/2318903745 shutdown_connections 2024-02-28T23:42:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.716+0000 7fa483e00700 1 -- 172.21.15.57:0/2318903745 wait complete. 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.717+0000 7fa483e00700 1 Processor -- start 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.717+0000 7fa483e00700 1 -- start start 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa483e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 0x7fa47c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa483e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa483e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c19d120 con 0x7fa47c10c8f0 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa483e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c1a0eb0 con 0x7fa47c106b80 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa480a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa480a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42816/0 (socket says 172.21.15.57:42816) 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa480a00700 1 -- 172.21.15.57:0/2687389906 learned_addr learned my addr 172.21.15.57:0/2687389906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.718+0000 7fa481400700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 0x7fa47c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa480a00700 1 -- 172.21.15.57:0/2687389906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 msgr2=0x7fa47c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa480a00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 0x7fa47c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa480a00700 1 -- 172.21.15.57:0/2687389906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700095f0 con 0x7fa47c10c8f0 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa480a00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa46800d8d0 tx=0x7fa46800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa468009940 con 0x7fa47c10c8f0 2024-02-28T23:42:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa468010460 con 0x7fa47c10c8f0 2024-02-28T23:42:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c1a1190 con 0x7fa47c10c8f0 2024-02-28T23:42:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46800f5f0 con 0x7fa47c10c8f0 2024-02-28T23:42:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.719+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c1a16e0 con 0x7fa47c10c8f0 2024-02-28T23:42:54.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.721+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c109b20 con 0x7fa47c10c8f0 2024-02-28T23:42:54.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.722+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa4680105d0 con 0x7fa47c10c8f0 2024-02-28T23:42:54.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.722+0000 7fa47a000700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 0x7fa46406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:54.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.723+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa46808d1d0 con 0x7fa47c10c8f0 2024-02-28T23:42:54.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.723+0000 7fa481400700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 0x7fa46406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.724+0000 7fa481400700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 0x7fa46406fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa470006010 tx=0x7fa47001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:54.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.728+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa468056a00 con 0x7fa47c10c8f0 2024-02-28T23:42:54.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:54 smithi057 ceph-mon[89491]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.981+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa47c1a1950 con 0x7fa46406d8a0 2024-02-28T23:42:54.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.983+0000 7fa47a000700 1 -- 172.21.15.57:0/2687389906 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa47c1a1950 con 0x7fa46406d8a0 2024-02-28T23:42:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 msgr2=0x7fa46406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 0x7fa46406fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa470006010 tx=0x7fa47001a040 comp rx=0 tx=0).stop 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 msgr2=0x7fa47c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa46800d8d0 tx=0x7fa46800dc90 comp rx=0 tx=0).stop 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 shutdown_connections 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa46406d8a0 0x7fa46406fd50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa47c106b80 0x7fa47c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 --2- 172.21.15.57:0/2687389906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa47c10c8f0 0x7fa47c19ca40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 >> 172.21.15.57:0/2687389906 conn(0x7fa47c076060 msgr2=0x7fa47c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.986+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 shutdown_connections 2024-02-28T23:42:54.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:54.987+0000 7fa483e00700 1 -- 172.21.15.57:0/2687389906 wait complete. 2024-02-28T23:42:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:54 smithi084 ceph-mon[70440]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:55.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 -- 172.21.15.57:0/13121192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 msgr2=0x7ff38c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 --2- 172.21.15.57:0/13121192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c1008b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009910 tx=0x7ff37c009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:55.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 -- 172.21.15.57:0/13121192 shutdown_connections 2024-02-28T23:42:55.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 --2- 172.21.15.57:0/13121192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 0x7ff38c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 --2- 172.21.15.57:0/13121192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c1008b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 -- 172.21.15.57:0/13121192 >> 172.21.15.57:0/13121192 conn(0x7ff38c0fbda0 msgr2=0x7ff38c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:55.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 -- 172.21.15.57:0/13121192 shutdown_connections 2024-02-28T23:42:55.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.141+0000 7ff392e00700 1 -- 172.21.15.57:0/13121192 wait complete. 2024-02-28T23:42:55.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.142+0000 7ff392e00700 1 Processor -- start 2024-02-28T23:42:55.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.142+0000 7ff392e00700 1 -- start start 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff392e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff392e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 0x7ff38c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff392e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c073ad0 con 0x7ff38c1004e0 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff392e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c073c10 con 0x7ff38c100e80 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff38be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff38be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58492/0 (socket says 172.21.15.57:58492) 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff38be00700 1 -- 172.21.15.57:0/4241518498 learned_addr learned my addr 172.21.15.57:0/4241518498 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.143+0000 7ff38b400700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 0x7ff38c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff38be00700 1 -- 172.21.15.57:0/4241518498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 msgr2=0x7ff38c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff38be00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 0x7ff38c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff38be00700 1 -- 172.21.15.57:0/4241518498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0095f0 con 0x7ff38c1004e0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff38be00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b5c0 tx=0x7ff37c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c01d070 con 0x7ff38c1004e0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff37c005e00 con 0x7ff38c1004e0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c017400 con 0x7ff38c1004e0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c073e90 con 0x7ff38c1004e0 2024-02-28T23:42:55.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.144+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c1a23f0 con 0x7ff38c1004e0 2024-02-28T23:42:55.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.146+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c1087f0 con 0x7ff38c1004e0 2024-02-28T23:42:55.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.147+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff37c00f460 con 0x7ff38c1004e0 2024-02-28T23:42:55.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.147+0000 7ff388c00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 0x7ff37006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.148+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff37c08e9e0 con 0x7ff38c1004e0 2024-02-28T23:42:55.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.148+0000 7ff38b400700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 0x7ff37006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.148+0000 7ff38b400700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 0x7ff37006fd50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff378005950 tx=0x7ff3780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:55.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.152+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff37c058170 con 0x7ff38c1004e0 2024-02-28T23:42:55.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.384+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38c061190 con 0x7ff37006d8a0 2024-02-28T23:42:55.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.416+0000 7ff388c00700 1 -- 172.21.15.57:0/4241518498 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff38c061190 con 0x7ff37006d8a0 2024-02-28T23:42:55.416 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:42:55.416 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (32m) 8m ago 46m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (42m) 8m ago 46m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (45m) 8m ago 45m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (28m) 8m ago 46m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (28m) 8m ago 45m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (31m) 8m ago 46m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (24m) 8m ago 39m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (24m) 8m ago 39m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (8m) 8m ago 38m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:42:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (23m) 8m ago 39m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (34m) 8m ago 48m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (33m) 8m ago 44m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (30m) 8m ago 48m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (29m) 8m ago 44m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (33m) 8m ago 46m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (33m) 8m ago 45m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (27m) 8m ago 43m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (27m) 8m ago 43m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:42:55.418 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (26m) 8m ago 42m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:42:55.419 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (26m) 8m ago 42m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:42:55.419 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (25m) 8m ago 41m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:42:55.419 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (24m) 8m ago 40m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:42:55.419 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (32m) 8m ago 45m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 msgr2=0x7ff37006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 0x7ff37006fd50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff378005950 tx=0x7ff3780058e0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 msgr2=0x7ff38c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b5c0 tx=0x7ff37c00be20 comp rx=0 tx=0).stop 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 shutdown_connections 2024-02-28T23:42:55.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff37006d8a0 0x7ff37006fd50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff38c1004e0 0x7ff38c073050 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 --2- 172.21.15.57:0/4241518498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff38c100e80 0x7ff38c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 >> 172.21.15.57:0/4241518498 conn(0x7ff38c0fbda0 msgr2=0x7ff38c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 shutdown_connections 2024-02-28T23:42:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.419+0000 7ff392e00700 1 -- 172.21.15.57:0/4241518498 wait complete. 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 -- 172.21.15.57:0/1506670402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 msgr2=0x7f67f010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 --2- 172.21.15.57:0/1506670402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f010c580 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009910 tx=0x7f67d8009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 -- 172.21.15.57:0/1506670402 shutdown_connections 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 --2- 172.21.15.57:0/1506670402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f010c580 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 --2- 172.21.15.57:0/1506670402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 0x7f67f00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.565+0000 7f67f7600700 1 -- 172.21.15.57:0/1506670402 >> 172.21.15.57:0/1506670402 conn(0x7f67f00fbda0 msgr2=0x7f67f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.566+0000 7f67f7600700 1 -- 172.21.15.57:0/1506670402 shutdown_connections 2024-02-28T23:42:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.566+0000 7f67f7600700 1 -- 172.21.15.57:0/1506670402 wait complete. 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.567+0000 7f67f7600700 1 Processor -- start 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.567+0000 7f67f7600700 1 -- start start 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.567+0000 7f67f7600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 0x7f67f0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67f7600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67f7600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f019c970 con 0x7f67f0073a00 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67f7600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f019cae0 con 0x7f67f00730f0 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58512/0 (socket says 172.21.15.57:58512) 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 -- 172.21.15.57:0/2171746568 learned_addr learned my addr 172.21.15.57:0/2171746568 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67f4c00700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 0x7f67f0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 -- 172.21.15.57:0/2171746568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 msgr2=0x7f67f0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 0x7f67f0197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 -- 172.21.15.57:0/2171746568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80095f0 con 0x7f67f0073a00 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.568+0000 7f67e7e00700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b5c0 tx=0x7f67d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.569+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d801d070 con 0x7f67f0073a00 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.569+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67d8005e00 con 0x7f67f0073a00 2024-02-28T23:42:55.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.569+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f019cdc0 con 0x7f67f0073a00 2024-02-28T23:42:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.569+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d8017400 con 0x7f67f0073a00 2024-02-28T23:42:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.569+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f019d310 con 0x7f67f0073a00 2024-02-28T23:42:55.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.570+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0109c80 con 0x7f67f0073a00 2024-02-28T23:42:55.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.571+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f67d800f460 con 0x7f67f0073a00 2024-02-28T23:42:55.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.572+0000 7f67ee000700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 0x7f67d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:55.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.572+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f67d808e940 con 0x7f67f0073a00 2024-02-28T23:42:55.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.572+0000 7f67f4c00700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 0x7f67d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:55.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.573+0000 7f67f4c00700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 0x7f67d406fd50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009c10 tx=0x7f67e0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:55.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.577+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67d8058170 con 0x7f67f0073a00 2024-02-28T23:42:55.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.896+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f67f0199480 con 0x7f67f0073a00 2024-02-28T23:42:55.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.897+0000 7f67ee000700 1 -- 172.21.15.57:0/2171746568 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f67d805b830 con 0x7f67f0073a00 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:42:55.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:42:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 msgr2=0x7f67d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 0x7f67d406fd50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009c10 tx=0x7f67e0009040 comp rx=0 tx=0).stop 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 msgr2=0x7f67f0198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b5c0 tx=0x7f67d800be20 comp rx=0 tx=0).stop 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 shutdown_connections 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f67d406d8a0 0x7f67d406fd50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f00730f0 0x7f67f0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.900+0000 7f67f7600700 1 --2- 172.21.15.57:0/2171746568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67f0073a00 0x7f67f0198630 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:55.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.901+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 >> 172.21.15.57:0/2171746568 conn(0x7f67f00fbda0 msgr2=0x7f67f0106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:55.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.901+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 shutdown_connections 2024-02-28T23:42:55.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:55.901+0000 7f67f7600700 1 -- 172.21.15.57:0/2171746568 wait complete. 2024-02-28T23:42:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 -- 172.21.15.57:0/215137454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 msgr2=0x7fdf881028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 --2- 172.21.15.57:0/215137454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf881028a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b0a0 tx=0x7fdf7800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:55 smithi057 ceph-mon[89491]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:56.049 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:55 smithi057 ceph-mon[89491]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 -- 172.21.15.57:0/215137454 shutdown_connections 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 --2- 172.21.15.57:0/215137454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf88106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 --2- 172.21.15.57:0/215137454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf881028a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 -- 172.21.15.57:0/215137454 >> 172.21.15.57:0/215137454 conn(0x7fdf880fdd90 msgr2=0x7fdf881001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.049+0000 7fdf87200700 1 -- 172.21.15.57:0/215137454 shutdown_connections 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.050+0000 7fdf87200700 1 -- 172.21.15.57:0/215137454 wait complete. 2024-02-28T23:42:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.050+0000 7fdf87200700 1 Processor -- start 2024-02-28T23:42:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf87200700 1 -- start start 2024-02-28T23:42:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf87200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf87200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf881983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf87200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88198a80 con 0x7fdf881024d0 2024-02-28T23:42:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf87200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8819c810 con 0x7fdf88102e70 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.051+0000 7fdf85e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58538/0 (socket says 172.21.15.57:58538) 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 -- 172.21.15.57:0/1481411363 learned_addr learned my addr 172.21.15.57:0/1481411363 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85400700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf881983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 -- 172.21.15.57:0/1481411363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 msgr2=0x7fdf881983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf881983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 -- 172.21.15.57:0/1481411363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009d00 con 0x7fdf881024d0 2024-02-28T23:42:56.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85400700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf881983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf85e00700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b070 tx=0x7fdf7800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.052+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7800e040 con 0x7fdf881024d0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.053+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf78003e60 con 0x7fdf881024d0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.053+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8819ca90 con 0x7fdf881024d0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.053+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7801d910 con 0x7fdf881024d0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.053+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8819cf80 con 0x7fdf881024d0 2024-02-28T23:42:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.054+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8810a7e0 con 0x7fdf881024d0 2024-02-28T23:42:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.055+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdf78004820 con 0x7fdf881024d0 2024-02-28T23:42:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.056+0000 7fdf7ea00700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 0x7fdf6c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.056+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdf7808eec0 con 0x7fdf881024d0 2024-02-28T23:42:56.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.056+0000 7fdf85400700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 0x7fdf6c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.057+0000 7fdf85400700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 0x7fdf6c06fd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdf74005fd0 tx=0x7fdf74005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.060+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf780585d0 con 0x7fdf881024d0 2024-02-28T23:42:56.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:55 smithi084 ceph-mon[70440]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:56.082 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:55 smithi084 ceph-mon[70440]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:56.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.325+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdf8819d260 con 0x7fdf881024d0 2024-02-28T23:42:56.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.326+0000 7fdf7ea00700 1 -- 172.21.15.57:0/1481411363 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fdf7805bc90 con 0x7fdf881024d0 2024-02-28T23:42:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:42:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:42:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:42:56.334 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:56.336 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:56.336 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:42:56.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 msgr2=0x7fdf6c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 0x7fdf6c06fd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdf74005fd0 tx=0x7fdf74005dc0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 msgr2=0x7fdf88197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b070 tx=0x7fdf7800cdd0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 shutdown_connections 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fdf6c06d8a0 0x7fdf6c06fd50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf881024d0 0x7fdf88197e60 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 --2- 172.21.15.57:0/1481411363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf88102e70 0x7fdf881983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.328+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 >> 172.21.15.57:0/1481411363 conn(0x7fdf880fdd90 msgr2=0x7fdf880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.329+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 shutdown_connections 2024-02-28T23:42:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.329+0000 7fdf87200700 1 -- 172.21.15.57:0/1481411363 wait complete. 2024-02-28T23:42:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:42:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 -- 172.21.15.57:0/537188919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075280 msgr2=0x7fc11c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 --2- 172.21.15.57:0/537188919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075280 0x7fc11c075650 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009910 tx=0x7fc10c009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 -- 172.21.15.57:0/537188919 shutdown_connections 2024-02-28T23:42:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 --2- 172.21.15.57:0/537188919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075b90 0x7fc11c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 --2- 172.21.15.57:0/537188919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075280 0x7fc11c075650 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 -- 172.21.15.57:0/537188919 >> 172.21.15.57:0/537188919 conn(0x7fc11c0fdf90 msgr2=0x7fc11c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:56.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 -- 172.21.15.57:0/537188919 shutdown_connections 2024-02-28T23:42:56.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.486+0000 7fc120a00700 1 -- 172.21.15.57:0/537188919 wait complete. 2024-02-28T23:42:56.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 Processor -- start 2024-02-28T23:42:56.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 -- start start 2024-02-28T23:42:56.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 0x7fc11c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19cfc0 con 0x7fc11c075b90 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.488+0000 7fc120a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19d100 con 0x7fc11c075280 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58558/0 (socket says 172.21.15.57:58558) 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 -- 172.21.15.57:0/902262478 learned_addr learned my addr 172.21.15.57:0/902262478 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119e00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 0x7fc11c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 -- 172.21.15.57:0/902262478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 msgr2=0x7fc11c19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 0x7fc11c19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 -- 172.21.15.57:0/902262478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119e00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 0x7fc11c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.489+0000 7fc119400700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c930 tx=0x7fc10800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.490+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc108007ad0 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.490+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc108007c30 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.490+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc108018610 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.490+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c1a0f50 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.490+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1a1400 con 0x7fc11c075b90 2024-02-28T23:42:56.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.491+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c10e340 con 0x7fc11c075b90 2024-02-28T23:42:56.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.492+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc10800f450 con 0x7fc11c075b90 2024-02-28T23:42:56.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.493+0000 7fc112a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.493+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc10808d810 con 0x7fc11c075b90 2024-02-28T23:42:56.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.494+0000 7fc119e00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.494+0000 7fc119e00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 0x7fc10006fd50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b850 tx=0x7fc10c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.498+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc108056f20 con 0x7fc11c075b90 2024-02-28T23:42:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.739+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c02cca0 con 0x7fc10006d8a0 2024-02-28T23:42:56.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.741+0000 7fc112a00700 1 -- 172.21.15.57:0/902262478 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc11c02cca0 con 0x7fc10006d8a0 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:42:56.742 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:42:56.743 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.744+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 msgr2=0x7fc10006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.744+0000 7fc120a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 0x7fc10006fd50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b850 tx=0x7fc10c00b540 comp rx=0 tx=0).stop 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.744+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 msgr2=0x7fc11c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.744+0000 7fc120a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c930 tx=0x7fc10800ccf0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 shutdown_connections 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc11c075280 0x7fc11c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 --2- 172.21.15.57:0/902262478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc11c075b90 0x7fc11c19c970 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 >> 172.21.15.57:0/902262478 conn(0x7fc11c0fdf90 msgr2=0x7fc11c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:56.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 shutdown_connections 2024-02-28T23:42:56.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.745+0000 7fc120a00700 1 -- 172.21.15.57:0/902262478 wait complete. 2024-02-28T23:42:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 -- 172.21.15.57:0/2160825958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3128100e80 msgr2=0x7f3128104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 --2- 172.21.15.57:0/2160825958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3128100e80 0x7f3128104e50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3110009910 tx=0x7f3110009c20 comp rx=0 tx=0).stop 2024-02-28T23:42:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 -- 172.21.15.57:0/2160825958 shutdown_connections 2024-02-28T23:42:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 --2- 172.21.15.57:0/2160825958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3128100e80 0x7f3128104e50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 --2- 172.21.15.57:0/2160825958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31281004e0 0x7f31281008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 -- 172.21.15.57:0/2160825958 >> 172.21.15.57:0/2160825958 conn(0x7f31280fbda0 msgr2=0x7f31280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 -- 172.21.15.57:0/2160825958 shutdown_connections 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.898+0000 7f312e200700 1 -- 172.21.15.57:0/2160825958 wait complete. 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.899+0000 7f312e200700 1 Processor -- start 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f312e200700 1 -- start start 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f312e200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f312e200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3128100e80 0x7f3128073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f312e200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128073ad0 con 0x7f31281004e0 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f312e200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128073c10 con 0x7f3128100e80 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f3127400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f3127400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58582/0 (socket says 172.21.15.57:58582) 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.900+0000 7f3127400700 1 -- 172.21.15.57:0/4011287940 learned_addr learned my addr 172.21.15.57:0/4011287940 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f311ea00700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3128100e80 0x7f3128073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3127400700 1 -- 172.21.15.57:0/4011287940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3128100e80 msgr2=0x7f3128073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3127400700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3128100e80 0x7f3128073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3127400700 1 -- 172.21.15.57:0/4011287940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100095f0 con 0x7f31281004e0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3127400700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3118007e90 tx=0x7f311800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3118017070 con 0x7f31281004e0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31281a2070 con 0x7f31281004e0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f311800f040 con 0x7f31281004e0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.901+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3118005970 con 0x7f31281004e0 2024-02-28T23:42:56.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.902+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31281a2480 con 0x7f31281004e0 2024-02-28T23:42:56.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.903+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128108760 con 0x7f31281004e0 2024-02-28T23:42:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.904+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3118005020 con 0x7f31281004e0 2024-02-28T23:42:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.905+0000 7f3124c00700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 0x7f310c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:42:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.905+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f311808d0c0 con 0x7f31281004e0 2024-02-28T23:42:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.905+0000 7f311ea00700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 0x7f310c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:42:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.906+0000 7f311ea00700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 0x7f310c06fd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3128074840 tx=0x7f311001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:42:56.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:56.909+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3118056850 con 0x7f31281004e0 2024-02-28T23:42:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:56 smithi057 ceph-mon[89491]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:56 smithi057 ceph-mon[89491]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:56 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2171746568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:42:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:56 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1481411363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:42:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:56 smithi084 ceph-mon[70440]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:56 smithi084 ceph-mon[70440]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:56 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2171746568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:42:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:56 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1481411363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:42:57.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.230+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f312804f2a0 con 0x7f31281004e0 2024-02-28T23:42:57.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.231+0000 7f3124c00700 1 -- 172.21.15.57:0/4011287940 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f3118059f10 con 0x7f31281004e0 2024-02-28T23:42:57.231 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:42:57.231 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:42:57.231 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:42:57.231 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:42:57.232 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:42:57.233 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:42:57.234 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:42:57.234 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:42:57.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.234+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 msgr2=0x7f310c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:57.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.234+0000 7f312e200700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 0x7f310c06fd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3128074840 tx=0x7f311001a040 comp rx=0 tx=0).stop 2024-02-28T23:42:57.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.234+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 msgr2=0x7f3128073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:42:57.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.234+0000 7f312e200700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3118007e90 tx=0x7f311800d3b0 comp rx=0 tx=0).stop 2024-02-28T23:42:57.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 shutdown_connections 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f310c06d8a0 0x7f310c06fd50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31281004e0 0x7f3128073050 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 --2- 172.21.15.57:0/4011287940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3128100e80 0x7f3128073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 >> 172.21.15.57:0/4011287940 conn(0x7f31280fbda0 msgr2=0x7f31281046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 shutdown_connections 2024-02-28T23:42:57.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:42:57.235+0000 7f312e200700 1 -- 172.21.15.57:0/4011287940 wait complete. 2024-02-28T23:42:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:57 smithi057 ceph-mon[89491]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:58.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:57 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4011287940' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:42:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:57 smithi084 ceph-mon[70440]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:42:58.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:57 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4011287940' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:42:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:42:58 smithi057 ceph-mon[89491]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:42:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:42:58 smithi084 ceph-mon[70440]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:00 smithi057 ceph-mon[89491]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:00 smithi084 ceph-mon[70440]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:02 smithi057 ceph-mon[89491]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:02 smithi084 ceph-mon[70440]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:04 smithi057 ceph-mon[89491]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:04 smithi084 ceph-mon[70440]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:06 smithi057 ceph-mon[89491]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:06 smithi084 ceph-mon[70440]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:08 smithi084 ceph-mon[70440]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:08 smithi057 ceph-mon[89491]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:10 smithi084 ceph-mon[70440]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:10 smithi057 ceph-mon[89491]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:12 smithi057 ceph-mon[89491]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:12 smithi084 ceph-mon[70440]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:14 smithi084 ceph-mon[70440]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:14 smithi057 ceph-mon[89491]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:16 smithi057 ceph-mon[89491]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:16 smithi084 ceph-mon[70440]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:19.290 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:18 smithi057 ceph-mon[89491]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:19.290 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:18 smithi084 ceph-mon[70440]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:20 smithi057 ceph-mon[89491]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:20 smithi084 ceph-mon[70440]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:22 smithi057 ceph-mon[89491]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:23.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:22 smithi084 ceph-mon[70440]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:24 smithi057 ceph-mon[89491]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:24 smithi084 ceph-mon[70440]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:26 smithi057 ceph-mon[89491]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:27.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:26 smithi084 ceph-mon[70440]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.412+0000 7f08b1e00700 1 -- 172.21.15.57:0/2223752495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac10c8f0 msgr2=0x7f08ac10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.412+0000 7f08b1e00700 1 --2- 172.21.15.57:0/2223752495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac10c8f0 0x7f08ac10ccc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f089c009910 tx=0x7f089c009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 -- 172.21.15.57:0/2223752495 shutdown_connections 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 --2- 172.21.15.57:0/2223752495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac106b80 0x7f08ac106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 --2- 172.21.15.57:0/2223752495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac10c8f0 0x7f08ac10ccc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 -- 172.21.15.57:0/2223752495 >> 172.21.15.57:0/2223752495 conn(0x7f08ac076060 msgr2=0x7f08ac078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 -- 172.21.15.57:0/2223752495 shutdown_connections 2024-02-28T23:43:27.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.413+0000 7f08b1e00700 1 -- 172.21.15.57:0/2223752495 wait complete. 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 Processor -- start 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 -- start start 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac10c8f0 0x7f08ac19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac19d120 con 0x7f08ac106b80 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.414+0000 7f08b1e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1a0eb0 con 0x7f08ac10c8f0 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50274/0 (socket says 172.21.15.57:50274) 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 -- 172.21.15.57:0/1955135246 learned_addr learned my addr 172.21.15.57:0/1955135246 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:27.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 -- 172.21.15.57:0/1955135246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac10c8f0 msgr2=0x7f08ac19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aa000700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac10c8f0 0x7f08ac19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac10c8f0 0x7f08ac19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 -- 172.21.15.57:0/1955135246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0095f0 con 0x7f08ac106b80 2024-02-28T23:43:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.415+0000 7f08aaa00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f089c00b5c0 tx=0x7f089c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:27.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.416+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f089c01d070 con 0x7f08ac106b80 2024-02-28T23:43:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.416+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f089c00f460 con 0x7f08ac106b80 2024-02-28T23:43:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.416+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f089c017610 con 0x7f08ac106b80 2024-02-28T23:43:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.416+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac1a1130 con 0x7f08ac106b80 2024-02-28T23:43:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.416+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac1a1620 con 0x7f08ac106b80 2024-02-28T23:43:27.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.417+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac109b20 con 0x7f08ac106b80 2024-02-28T23:43:27.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.418+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f089c00f5d0 con 0x7f08ac106b80 2024-02-28T23:43:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.419+0000 7f08b0a00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 0x7f089006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.419+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f089c08eb50 con 0x7f08ac106b80 2024-02-28T23:43:27.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.420+0000 7f08aa000700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 0x7f089006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.420+0000 7f08aa000700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 0x7f089006fd50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08ac19db20 tx=0x7f0898005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:27.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.424+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f089c0582e0 con 0x7f08ac106b80 2024-02-28T23:43:27.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.668+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08ac1a1890 con 0x7f089006d8a0 2024-02-28T23:43:27.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.671+0000 7f08b0a00700 1 -- 172.21.15.57:0/1955135246 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f08ac1a1890 con 0x7f089006d8a0 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 msgr2=0x7f089006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 0x7f089006fd50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08ac19db20 tx=0x7f0898005cf0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 msgr2=0x7f08ac19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f089c00b5c0 tx=0x7f089c00be00 comp rx=0 tx=0).stop 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 shutdown_connections 2024-02-28T23:43:27.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f089006d8a0 0x7f089006fd50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ac106b80 0x7f08ac19c500 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 --2- 172.21.15.57:0/1955135246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08ac10c8f0 0x7f08ac19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 >> 172.21.15.57:0/1955135246 conn(0x7f08ac076060 msgr2=0x7f08ac077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 shutdown_connections 2024-02-28T23:43:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.674+0000 7f08b1e00700 1 -- 172.21.15.57:0/1955135246 wait complete. 2024-02-28T23:43:27.689 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 -- 172.21.15.57:0/1974577606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 msgr2=0x7f91c4108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 --2- 172.21.15.57:0/1974577606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4108840 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009910 tx=0x7f91b8009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 -- 172.21.15.57:0/1974577606 shutdown_connections 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 --2- 172.21.15.57:0/1974577606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 0x7f91c41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 --2- 172.21.15.57:0/1974577606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4108840 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 -- 172.21.15.57:0/1974577606 >> 172.21.15.57:0/1974577606 conn(0x7f91c40fdf90 msgr2=0x7f91c41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 -- 172.21.15.57:0/1974577606 shutdown_connections 2024-02-28T23:43:27.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.836+0000 7f91cc000700 1 -- 172.21.15.57:0/1974577606 wait complete. 2024-02-28T23:43:27.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.837+0000 7f91cc000700 1 Processor -- start 2024-02-28T23:43:27.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.838+0000 7f91cc000700 1 -- start start 2024-02-28T23:43:27.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.838+0000 7f91cc000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 0x7f91c41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.838+0000 7f91cc000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.838+0000 7f91cc000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4198e10 con 0x7f91c4108470 2024-02-28T23:43:27.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.838+0000 7f91cc000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c419cba0 con 0x7f91c4102470 2024-02-28T23:43:27.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50298/0 (socket says 172.21.15.57:50298) 2024-02-28T23:43:27.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 -- 172.21.15.57:0/4061944103 learned_addr learned my addr 172.21.15.57:0/4061944103 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c9600700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 0x7f91c41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 -- 172.21.15.57:0/4061944103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 msgr2=0x7f91c41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 0x7f91c41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.839+0000 7f91c8c00700 1 -- 172.21.15.57:0/4061944103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80095f0 con 0x7f91c4108470 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91c8c00700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c8a0 tx=0x7f91b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91b000cea0 con 0x7f91c4108470 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91b0004830 con 0x7f91c4108470 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91b00055b0 con 0x7f91c4108470 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c419ce80 con 0x7f91c4108470 2024-02-28T23:43:27.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.840+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c419d3d0 con 0x7f91c4108470 2024-02-28T23:43:27.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.841+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c410a9e0 con 0x7f91c4108470 2024-02-28T23:43:27.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.842+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f91b00049a0 con 0x7f91c4108470 2024-02-28T23:43:27.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.843+0000 7f91c2000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 0x7f91ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:27.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.843+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f91b008d260 con 0x7f91c4108470 2024-02-28T23:43:27.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.843+0000 7f91c9600700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 0x7f91ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:27.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.844+0000 7f91c9600700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 0x7f91ac06fd50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006010 tx=0x7f91b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:27.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:27.848+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91b0056970 con 0x7f91c4108470 2024-02-28T23:43:28.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.103+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91c419d650 con 0x7f91ac06d8a0 2024-02-28T23:43:28.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.105+0000 7f91c2000700 1 -- 172.21.15.57:0/4061944103 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f91c419d650 con 0x7f91ac06d8a0 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 msgr2=0x7f91ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 0x7f91ac06fd50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006010 tx=0x7f91b801a040 comp rx=0 tx=0).stop 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 msgr2=0x7f91c4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c8a0 tx=0x7f91b000cc60 comp rx=0 tx=0).stop 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 shutdown_connections 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f91ac06d8a0 0x7f91ac06fd50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c4102470 0x7f91c41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 --2- 172.21.15.57:0/4061944103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c4108470 0x7f91c4198730 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 >> 172.21.15.57:0/4061944103 conn(0x7f91c40fdf90 msgr2=0x7f91c41002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 shutdown_connections 2024-02-28T23:43:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.108+0000 7f91cc000700 1 -- 172.21.15.57:0/4061944103 wait complete. 2024-02-28T23:43:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 -- 172.21.15.57:0/1207670584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc1004c0 msgr2=0x7ff9dc100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 --2- 172.21.15.57:0/1207670584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc100890 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009910 tx=0x7ff9d0009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 -- 172.21.15.57:0/1207670584 shutdown_connections 2024-02-28T23:43:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 --2- 172.21.15.57:0/1207670584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc100e60 0x7ff9dc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 --2- 172.21.15.57:0/1207670584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc100890 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 -- 172.21.15.57:0/1207670584 >> 172.21.15.57:0/1207670584 conn(0x7ff9dc0fbd80 msgr2=0x7ff9dc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 -- 172.21.15.57:0/1207670584 shutdown_connections 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.260+0000 7ff9e4400700 1 -- 172.21.15.57:0/1207670584 wait complete. 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 Processor -- start 2024-02-28T23:43:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 -- start start 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc198d20 con 0x7ff9dc100e60 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.262+0000 7ff9e4400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc19cab0 con 0x7ff9dc1004c0 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50320/0 (socket says 172.21.15.57:50320) 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 -- 172.21.15.57:0/2566569400 learned_addr learned my addr 172.21.15.57:0/2566569400 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1a00700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 -- 172.21.15.57:0/2566569400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1004c0 msgr2=0x7ff9dc198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.263+0000 7ff9e1000700 1 -- 172.21.15.57:0/2566569400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00095f0 con 0x7ff9dc100e60 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9e1000700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c8f0 tx=0x7ff9c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c8007a50 con 0x7ff9dc100e60 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c8007bb0 con 0x7ff9dc100e60 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c8018630 con 0x7ff9dc100e60 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc19cd90 con 0x7ff9dc100e60 2024-02-28T23:43:28.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.264+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc19d2e0 con 0x7ff9dc100e60 2024-02-28T23:43:28.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.265+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc1045b0 con 0x7ff9dc100e60 2024-02-28T23:43:28.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.266+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff9c8007d20 con 0x7ff9dc100e60 2024-02-28T23:43:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.267+0000 7ff9da000700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.267+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff9c808d5a0 con 0x7ff9dc100e60 2024-02-28T23:43:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.267+0000 7ff9e1a00700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.268+0000 7ff9e1a00700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.271+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9c8056cb0 con 0x7ff9dc100e60 2024-02-28T23:43:28.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.501+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9dc19d560 con 0x7ff9c406d8a0 2024-02-28T23:43:28.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.534+0000 7ff9da000700 1 -- 172.21.15.57:0/2566569400 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff9dc19d560 con 0x7ff9c406d8a0 2024-02-28T23:43:28.533 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:43:28.533 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (33m) 8m ago 46m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:43:28.533 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (43m) 8m ago 47m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:43:28.533 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (45m) 8m ago 45m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (29m) 8m ago 47m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (29m) 8m ago 45m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (32m) 8m ago 46m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (25m) 8m ago 39m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (24m) 8m ago 39m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (8m) 8m ago 39m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (24m) 8m ago 39m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (35m) 8m ago 48m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (34m) 8m ago 45m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (30m) 8m ago 49m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (30m) 8m ago 45m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (34m) 8m ago 47m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:43:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (34m) 8m ago 45m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (28m) 8m ago 44m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (27m) 8m ago 43m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (27m) 8m ago 43m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (26m) 8m ago 42m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (26m) 8m ago 42m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (25m) 8m ago 41m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:43:28.535 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (33m) 8m ago 46m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 msgr2=0x7ff9c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d001a040 comp rx=0 tx=0).stop 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 msgr2=0x7ff9dc198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c8f0 tx=0x7ff9c800ccb0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 shutdown_connections 2024-02-28T23:43:28.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff9c406d8a0 0x7ff9c406fd50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9dc1004c0 0x7ff9dc198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 --2- 172.21.15.57:0/2566569400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9dc100e60 0x7ff9dc198640 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 >> 172.21.15.57:0/2566569400 conn(0x7ff9dc0fbd80 msgr2=0x7ff9dc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:28.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 shutdown_connections 2024-02-28T23:43:28.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.538+0000 7ff9e4400700 1 -- 172.21.15.57:0/2566569400 wait complete. 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 -- 172.21.15.57:0/2328478018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17981004e0 msgr2=0x7f17981008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 --2- 172.21.15.57:0/2328478018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17981004e0 0x7f17981008b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f178800b0a0 tx=0x7f178800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 -- 172.21.15.57:0/2328478018 shutdown_connections 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 --2- 172.21.15.57:0/2328478018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798100e80 0x7f1798104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 --2- 172.21.15.57:0/2328478018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17981004e0 0x7f17981008b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.694+0000 7f179d600700 1 -- 172.21.15.57:0/2328478018 >> 172.21.15.57:0/2328478018 conn(0x7f17980fbda0 msgr2=0x7f17980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.695+0000 7f179d600700 1 -- 172.21.15.57:0/2328478018 shutdown_connections 2024-02-28T23:43:28.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.695+0000 7f179d600700 1 -- 172.21.15.57:0/2328478018 wait complete. 2024-02-28T23:43:28.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 Processor -- start 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 -- start start 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17981004e0 0x7f1798198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1798198d40 con 0x7f1798100e80 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f179d600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179819cad0 con 0x7f17981004e0 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f178e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f178e000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50338/0 (socket says 172.21.15.57:50338) 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.696+0000 7f178e000700 1 -- 172.21.15.57:0/1462141192 learned_addr learned my addr 172.21.15.57:0/1462141192 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178e000700 1 -- 172.21.15.57:0/1462141192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17981004e0 msgr2=0x7f1798198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178e000700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17981004e0 0x7f1798198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178e000700 1 -- 172.21.15.57:0/1462141192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009d00 con 0x7f1798100e80 2024-02-28T23:43:28.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178e000700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f178000c8f0 tx=0x7f178000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:28.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1780007a50 con 0x7f1798100e80 2024-02-28T23:43:28.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1780007bb0 con 0x7f1798100e80 2024-02-28T23:43:28.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1780018630 con 0x7f1798100e80 2024-02-28T23:43:28.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179819cdb0 con 0x7f1798100e80 2024-02-28T23:43:28.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.697+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179819d300 con 0x7f1798100e80 2024-02-28T23:43:28.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.698+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17981045d0 con 0x7f1798100e80 2024-02-28T23:43:28.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.699+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1780007d20 con 0x7f1798100e80 2024-02-28T23:43:28.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.700+0000 7f178fe00700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 0x7f177c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:28.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.700+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f178008d4a0 con 0x7f1798100e80 2024-02-28T23:43:28.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.701+0000 7f1796a00700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 0x7f177c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:28.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.701+0000 7f1796a00700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 0x7f177c06fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1788015040 tx=0x7f1788006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:28.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:28.706+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1780056c30 con 0x7f1798100e80 2024-02-28T23:43:29.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.014+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f179819d5d0 con 0x7f1798100e80 2024-02-28T23:43:29.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.015+0000 7f178fe00700 1 -- 172.21.15.57:0/1462141192 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f178005a2f0 con 0x7f1798100e80 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:43:29.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:43:29.017 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:43:29.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:43:29.017 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:43:29.017 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 msgr2=0x7f177c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 0x7f177c06fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1788015040 tx=0x7f1788006000 comp rx=0 tx=0).stop 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 msgr2=0x7f1798198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f178000c8f0 tx=0x7f178000ccb0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 shutdown_connections 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f177c06d8a0 0x7f177c06fd50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17981004e0 0x7f1798198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 --2- 172.21.15.57:0/1462141192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1798100e80 0x7f1798198660 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.018+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 >> 172.21.15.57:0/1462141192 conn(0x7f17980fbda0 msgr2=0x7f17980fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:29.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.019+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 shutdown_connections 2024-02-28T23:43:29.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.019+0000 7f179d600700 1 -- 172.21.15.57:0/1462141192 wait complete. 2024-02-28T23:43:29.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 -- 172.21.15.57:0/4067944144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 msgr2=0x7f5988104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 --2- 172.21.15.57:0/4067944144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988104500 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5978009910 tx=0x7f5978009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:29.181 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:28 smithi057 ceph-mon[89491]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:29.181 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:28 smithi057 ceph-mon[89491]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:29.181 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:28 smithi057 ceph-mon[89491]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 -- 172.21.15.57:0/4067944144 shutdown_connections 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 --2- 172.21.15.57:0/4067944144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 --2- 172.21.15.57:0/4067944144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988104500 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 -- 172.21.15.57:0/4067944144 >> 172.21.15.57:0/4067944144 conn(0x7f5988075130 msgr2=0x7f5988075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 -- 172.21.15.57:0/4067944144 shutdown_connections 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.181+0000 7f598cc00700 1 -- 172.21.15.57:0/4067944144 wait complete. 2024-02-28T23:43:29.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.182+0000 7f598cc00700 1 Processor -- start 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.182+0000 7f598cc00700 1 -- start start 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f598cc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f598cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f598cc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988198bd0 con 0x7f5988104130 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f598cc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988198d10 con 0x7f59880ffbd0 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f5985400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f5985400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50352/0 (socket says 172.21.15.57:50352) 2024-02-28T23:43:29.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.183+0000 7f5985400700 1 -- 172.21.15.57:0/567878021 learned_addr learned my addr 172.21.15.57:0/567878021 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:29.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985e00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985400700 1 -- 172.21.15.57:0/567878021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 msgr2=0x7f5988198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985400700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985400700 1 -- 172.21.15.57:0/567878021 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780095f0 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985e00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f5985400700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f597400c930 tx=0x7f597400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.184+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5974007ad0 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.185+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5974007c30 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.185+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5974018610 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.185+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598819cb60 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.185+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598819d0b0 con 0x7f5988104130 2024-02-28T23:43:29.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.186+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598804f2a0 con 0x7f5988104130 2024-02-28T23:43:29.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.187+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f597400f450 con 0x7f5988104130 2024-02-28T23:43:29.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.188+0000 7f597ea00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 0x7f596c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.188+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f597408d6d0 con 0x7f5988104130 2024-02-28T23:43:29.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.188+0000 7f5985e00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 0x7f596c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.189+0000 7f5985e00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 0x7f596c06fd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5978006010 tx=0x7f597800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.193+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5974056e60 con 0x7f5988104130 2024-02-28T23:43:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:28 smithi084 ceph-mon[70440]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:28 smithi084 ceph-mon[70440]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:28 smithi084 ceph-mon[70440]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:29.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.471+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f59881993a0 con 0x7f5988104130 2024-02-28T23:43:29.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.472+0000 7f597ea00700 1 -- 172.21.15.57:0/567878021 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f597405a520 con 0x7f5988104130 2024-02-28T23:43:29.472 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:43:29.472 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:43:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:43:29.474 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:43:29.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.476+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 msgr2=0x7f596c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.476+0000 7f598cc00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 0x7f596c06fd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5978006010 tx=0x7f597800b540 comp rx=0 tx=0).stop 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.476+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 msgr2=0x7f5988198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.476+0000 7f598cc00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f597400c930 tx=0x7f597400ccf0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 shutdown_connections 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f596c06d8a0 0x7f596c06fd50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880ffbd0 0x7f5988198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 --2- 172.21.15.57:0/567878021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5988104130 0x7f5988198580 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 >> 172.21.15.57:0/567878021 conn(0x7f5988075130 msgr2=0x7f5988068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:29.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 shutdown_connections 2024-02-28T23:43:29.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.477+0000 7f598cc00700 1 -- 172.21.15.57:0/567878021 wait complete. 2024-02-28T23:43:29.478 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 -- 172.21.15.57:0/774471049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 msgr2=0x7fe3480734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 --2- 172.21.15.57:0/774471049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe3480734c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009910 tx=0x7fe33c009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 -- 172.21.15.57:0/774471049 shutdown_connections 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 --2- 172.21.15.57:0/774471049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 0x7fe348110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 --2- 172.21.15.57:0/774471049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe3480734c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.641+0000 7fe34de00700 1 -- 172.21.15.57:0/774471049 >> 172.21.15.57:0/774471049 conn(0x7fe3480fbb10 msgr2=0x7fe3480fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.642+0000 7fe34de00700 1 -- 172.21.15.57:0/774471049 shutdown_connections 2024-02-28T23:43:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.642+0000 7fe34de00700 1 -- 172.21.15.57:0/774471049 wait complete. 2024-02-28T23:43:29.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 Processor -- start 2024-02-28T23:43:29.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 -- start start 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 0x7fe34819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819ce90 con 0x7fe3480730f0 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.643+0000 7fe34de00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3481a0c20 con 0x7fe348073a00 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50364/0 (socket says 172.21.15.57:50364) 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 -- 172.21.15.57:0/3623881190 learned_addr learned my addr 172.21.15.57:0/3623881190 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe347e00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 0x7fe34819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 -- 172.21.15.57:0/3623881190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 msgr2=0x7fe34819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 0x7fe34819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 -- 172.21.15.57:0/3623881190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0095f0 con 0x7fe3480730f0 2024-02-28T23:43:29.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.644+0000 7fe34ca00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006010 tx=0x7fe33c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.645+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33c01d070 con 0x7fe3480730f0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.645+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3481a0f00 con 0x7fe3480730f0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.645+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe33c004fb0 con 0x7fe3480730f0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.645+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33c022ea0 con 0x7fe3480730f0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.645+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481a1450 con 0x7fe3480730f0 2024-02-28T23:43:29.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.647+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34810e280 con 0x7fe3480730f0 2024-02-28T23:43:29.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.648+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe33c022470 con 0x7fe3480730f0 2024-02-28T23:43:29.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.648+0000 7fe345600700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 0x7fe33006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:29.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.649+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe33c08eb50 con 0x7fe3480730f0 2024-02-28T23:43:29.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.649+0000 7fe347e00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 0x7fe33006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:29.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.650+0000 7fe347e00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 0x7fe33006fd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe334006fd0 tx=0x7fe334008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:29.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.654+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe33c0584b0 con 0x7fe3480730f0 2024-02-28T23:43:29.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.905+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3481a1780 con 0x7fe33006d8a0 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.907+0000 7fe345600700 1 -- 172.21.15.57:0/3623881190 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe3481a1780 con 0x7fe33006d8a0 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:43:29.908 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:43:29.909 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:43:29.909 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:43:29.909 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:43:29.909 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:43:29.909 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.910+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 msgr2=0x7fe33006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.910+0000 7fe34de00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 0x7fe33006fd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe334006fd0 tx=0x7fe334008040 comp rx=0 tx=0).stop 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.910+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 msgr2=0x7fe34819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.910+0000 7fe34de00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006010 tx=0x7fe33c00be20 comp rx=0 tx=0).stop 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 shutdown_connections 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe33006d8a0 0x7fe33006fd50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3480730f0 0x7fe34819c270 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 --2- 172.21.15.57:0/3623881190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe348073a00 0x7fe34819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 >> 172.21.15.57:0/3623881190 conn(0x7fe3480fbb10 msgr2=0x7fe348102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 shutdown_connections 2024-02-28T23:43:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:29.911+0000 7fe34de00700 1 -- 172.21.15.57:0/3623881190 wait complete. 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.067+0000 7f1749600700 1 -- 172.21.15.57:0/3318752532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 msgr2=0x7f17441029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.067+0000 7f1749600700 1 --2- 172.21.15.57:0/3318752532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f17441029a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1734009910 tx=0x7f1734009c20 comp rx=0 tx=0).stop 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 -- 172.21.15.57:0/3318752532 shutdown_connections 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 --2- 172.21.15.57:0/3318752532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 0x7f17441042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 --2- 172.21.15.57:0/3318752532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f17441029a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 -- 172.21.15.57:0/3318752532 >> 172.21.15.57:0/3318752532 conn(0x7f17440fddb0 msgr2=0x7f17441001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 -- 172.21.15.57:0/3318752532 shutdown_connections 2024-02-28T23:43:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.068+0000 7f1749600700 1 -- 172.21.15.57:0/3318752532 wait complete. 2024-02-28T23:43:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.069+0000 7f1749600700 1 Processor -- start 2024-02-28T23:43:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.069+0000 7f1749600700 1 -- start start 2024-02-28T23:43:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.069+0000 7f1749600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1749600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 0x7f1744198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1749600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744198ad0 con 0x7f17441025d0 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1749600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744198c10 con 0x7f1744103e30 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50388/0 (socket says 172.21.15.57:50388) 2024-02-28T23:43:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 -- 172.21.15.57:0/2722670219 learned_addr learned my addr 172.21.15.57:0/2722670219 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743400700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 0x7f1744198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 -- 172.21.15.57:0/2722670219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 msgr2=0x7f1744198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 0x7f1744198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.070+0000 7f1743e00700 1 -- 172.21.15.57:0/2722670219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340095f0 con 0x7f17441025d0 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1743e00700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f173400b5c0 tx=0x7f173400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173401d070 con 0x7f17441025d0 2024-02-28T23:43:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173400f460 con 0x7f17441025d0 2024-02-28T23:43:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174419ca00 con 0x7f17441025d0 2024-02-28T23:43:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1734017610 con 0x7f17441025d0 2024-02-28T23:43:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.071+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174419cef0 con 0x7f17441025d0 2024-02-28T23:43:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.072+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1744062380 con 0x7f17441025d0 2024-02-28T23:43:30.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.073+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f173400f5d0 con 0x7f17441025d0 2024-02-28T23:43:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.074+0000 7f1740c00700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 0x7f172806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:43:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.074+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f173408eb50 con 0x7f17441025d0 2024-02-28T23:43:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.075+0000 7f1743400700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 0x7f172806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:43:30.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.075+0000 7f1743400700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 0x7f172806fd50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1744199560 tx=0x7f1730005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:43:30.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.079+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17340582e0 con 0x7f17441025d0 2024-02-28T23:43:30.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:29 smithi057 ceph-mon[89491]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:30.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1462141192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:43:30.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/567878021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:43:30.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:29 smithi084 ceph-mon[70440]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:30.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1462141192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:43:30.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/567878021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:43:30.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.386+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f17441992a0 con 0x7f17441025d0 2024-02-28T23:43:30.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.387+0000 7f1740c00700 1 -- 172.21.15.57:0/2722670219 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f173405b9a0 con 0x7f17441025d0 2024-02-28T23:43:30.387 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:43:30.388 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:43:30.389 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:43:30.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 msgr2=0x7f172806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:30.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 0x7f172806fd50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1744199560 tx=0x7f1730005d50 comp rx=0 tx=0).stop 2024-02-28T23:43:30.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 msgr2=0x7f1744197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:43:30.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f173400b5c0 tx=0x7f173400be00 comp rx=0 tx=0).stop 2024-02-28T23:43:30.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 shutdown_connections 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f172806d8a0 0x7f172806fd50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17441025d0 0x7f1744197f40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 --2- 172.21.15.57:0/2722670219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744103e30 0x7f1744198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 >> 172.21.15.57:0/2722670219 conn(0x7f17440fddb0 msgr2=0x7f17440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 shutdown_connections 2024-02-28T23:43:30.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:43:30.390+0000 7f1749600700 1 -- 172.21.15.57:0/2722670219 wait complete. 2024-02-28T23:43:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:30 smithi057 ceph-mon[89491]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:30 smithi057 ceph-mon[89491]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2722670219' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:43:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:30 smithi084 ceph-mon[70440]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:30 smithi084 ceph-mon[70440]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:43:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2722670219' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:43:33.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:32 smithi057 ceph-mon[89491]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:32 smithi084 ceph-mon[70440]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:34 smithi057 ceph-mon[89491]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:35.278 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:34 smithi084 ceph-mon[70440]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:36 smithi057 ceph-mon[89491]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:36 smithi084 ceph-mon[70440]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:38 smithi057 ceph-mon[89491]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:38 smithi084 ceph-mon[70440]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:41.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:40 smithi057 ceph-mon[89491]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:40 smithi084 ceph-mon[70440]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:42 smithi057 ceph-mon[89491]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:43.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:42 smithi084 ceph-mon[70440]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:45.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:44 smithi057 ceph-mon[89491]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:44 smithi084 ceph-mon[70440]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:46 smithi057 ceph-mon[89491]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:46 smithi084 ceph-mon[70440]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:48 smithi057 ceph-mon[89491]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:48 smithi084 ceph-mon[70440]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:43:51.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:51 smithi084 ceph-mon[70440]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:51 smithi057 ceph-mon[89491]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:53.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:53 smithi057 ceph-mon[89491]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:53 smithi084 ceph-mon[70440]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:55 smithi057 ceph-mon[89491]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:55 smithi084 ceph-mon[70440]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:57 smithi057 ceph-mon[89491]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:57.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:57 smithi084 ceph-mon[70440]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:59.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:43:59 smithi057 ceph-mon[89491]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:43:59.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:43:59 smithi084 ceph-mon[70440]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 -- 172.21.15.57:0/1238450530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101420 msgr2=0x7fa0401017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 --2- 172.21.15.57:0/1238450530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101420 0x7fa0401017f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa030009910 tx=0x7fa030009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 -- 172.21.15.57:0/1238450530 shutdown_connections 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 --2- 172.21.15.57:0/1238450530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101d30 0x7fa04010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 --2- 172.21.15.57:0/1238450530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101420 0x7fa0401017f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 -- 172.21.15.57:0/1238450530 >> 172.21.15.57:0/1238450530 conn(0x7fa0400fac80 msgr2=0x7fa0400fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 -- 172.21.15.57:0/1238450530 shutdown_connections 2024-02-28T23:44:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.564+0000 7fa045200700 1 -- 172.21.15.57:0/1238450530 wait complete. 2024-02-28T23:44:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 Processor -- start 2024-02-28T23:44:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 -- start start 2024-02-28T23:44:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101420 0x7fa0400ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400ffb00 con 0x7fa040101d30 2024-02-28T23:44:00.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.566+0000 7fa045200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400ffc40 con 0x7fa040101420 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56266/0 (socket says 172.21.15.57:56266) 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 -- 172.21.15.57:0/498321488 learned_addr learned my addr 172.21.15.57:0/498321488 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03de00700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101420 0x7fa0400ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 -- 172.21.15.57:0/498321488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101420 msgr2=0x7fa0400ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101420 0x7fa0400ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.567+0000 7fa03d400700 1 -- 172.21.15.57:0/498321488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300095f0 con 0x7fa040101d30 2024-02-28T23:44:00.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa03d400700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00c930 tx=0x7fa02c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa02c007ad0 con 0x7fa040101d30 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0401a2040 con 0x7fa040101d30 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa02c00ce80 con 0x7fa040101d30 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa02c018630 con 0x7fa040101d30 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.568+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0401a23a0 con 0x7fa040101d30 2024-02-28T23:44:00.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.570+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040107900 con 0x7fa040101d30 2024-02-28T23:44:00.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.571+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa02c007c30 con 0x7fa040101d30 2024-02-28T23:44:00.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.572+0000 7fa036a00700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 0x7fa02406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:00.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.572+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa02c08d5f0 con 0x7fa040101d30 2024-02-28T23:44:00.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.572+0000 7fa03de00700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 0x7fa02406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:00.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.573+0000 7fa03de00700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 0x7fa02406fd50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa030006010 tx=0x7fa03001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:00.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.577+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa02c056d00 con 0x7fa040101d30 2024-02-28T23:44:00.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.827+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa040061190 con 0x7fa02406d8a0 2024-02-28T23:44:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.829+0000 7fa036a00700 1 -- 172.21.15.57:0/498321488 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa040061190 con 0x7fa02406d8a0 2024-02-28T23:44:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 msgr2=0x7fa02406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 0x7fa02406fd50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa030006010 tx=0x7fa03001a040 comp rx=0 tx=0).stop 2024-02-28T23:44:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 msgr2=0x7fa0400ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00c930 tx=0x7fa02c00cc40 comp rx=0 tx=0).stop 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 shutdown_connections 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa02406d8a0 0x7fa02406fd50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa040101420 0x7fa0400ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 --2- 172.21.15.57:0/498321488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa040101d30 0x7fa0400ff5c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 >> 172.21.15.57:0/498321488 conn(0x7fa0400fac80 msgr2=0x7fa040104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 shutdown_connections 2024-02-28T23:44:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.832+0000 7fa045200700 1 -- 172.21.15.57:0/498321488 wait complete. 2024-02-28T23:44:00.847 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:44:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 -- 172.21.15.57:0/2009788574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 msgr2=0x7f996c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 --2- 172.21.15.57:0/2009788574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c106ff0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f995400b0a0 tx=0x7f995400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 -- 172.21.15.57:0/2009788574 shutdown_connections 2024-02-28T23:44:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 --2- 172.21.15.57:0/2009788574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c106ff0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 --2- 172.21.15.57:0/2009788574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 0x7f996c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 -- 172.21.15.57:0/2009788574 >> 172.21.15.57:0/2009788574 conn(0x7f996c075e90 msgr2=0x7f996c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 -- 172.21.15.57:0/2009788574 shutdown_connections 2024-02-28T23:44:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.995+0000 7f996be00700 1 -- 172.21.15.57:0/2009788574 wait complete. 2024-02-28T23:44:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.996+0000 7f996be00700 1 Processor -- start 2024-02-28T23:44:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.996+0000 7f996be00700 1 -- start start 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996be00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 0x7f996c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996be00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c19cf20 con 0x7f996c106b80 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996be00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1a0cb0 con 0x7f996c10c660 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56288/0 (socket says 172.21.15.57:56288) 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f996aa00700 1 -- 172.21.15.57:0/1577005512 learned_addr learned my addr 172.21.15.57:0/1577005512 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.997+0000 7f9962000700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 0x7f996c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996aa00700 1 -- 172.21.15.57:0/1577005512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 msgr2=0x7f996c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996aa00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 0x7f996c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996aa00700 1 -- 172.21.15.57:0/1577005512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954009d00 con 0x7f996c106b80 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996aa00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f995c00d8d0 tx=0x7f995c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c009940 con 0x7f996c106b80 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c1a0f90 con 0x7f996c106b80 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995c010460 con 0x7f996c106b80 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c00f5f0 con 0x7f996c106b80 2024-02-28T23:44:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.998+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c1a14e0 con 0x7f996c106b80 2024-02-28T23:44:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:00.999+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c10eb40 con 0x7f996c106b80 2024-02-28T23:44:01.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.000+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f995c0105d0 con 0x7f996c106b80 2024-02-28T23:44:01.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.001+0000 7f9963e00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 0x7f995006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.002+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f995c08d4c0 con 0x7f996c106b80 2024-02-28T23:44:01.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.002+0000 7f9962000700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 0x7f995006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.002+0000 7f9962000700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 0x7f995006fd50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9954015040 tx=0x7f9954000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.006+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f995c056c50 con 0x7f996c106b80 2024-02-28T23:44:01.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.255+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f996c1a1750 con 0x7f995006d8a0 2024-02-28T23:44:01.255 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:01 smithi057 ceph-mon[89491]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:01.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.258+0000 7f9963e00700 1 -- 172.21.15.57:0/1577005512 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f996c1a1750 con 0x7f995006d8a0 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.261+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 msgr2=0x7f995006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.261+0000 7f996be00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 0x7f995006fd50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9954015040 tx=0x7f9954000f40 comp rx=0 tx=0).stop 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.261+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 msgr2=0x7f996c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.261+0000 7f996be00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f995c00d8d0 tx=0x7f995c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 shutdown_connections 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f995006d8a0 0x7f995006fd50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f996c106b80 0x7f996c19c300 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 --2- 172.21.15.57:0/1577005512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f996c10c660 0x7f996c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 >> 172.21.15.57:0/1577005512 conn(0x7f996c075e90 msgr2=0x7f996c077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 shutdown_connections 2024-02-28T23:44:01.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.262+0000 7f996be00700 1 -- 172.21.15.57:0/1577005512 wait complete. 2024-02-28T23:44:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:01 smithi084 ceph-mon[70440]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.422+0000 7f705ba00700 1 -- 172.21.15.57:0/585482089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7054069140 msgr2=0x7f7054104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.422+0000 7f705ba00700 1 --2- 172.21.15.57:0/585482089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7054069140 0x7f7054104eb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7040009910 tx=0x7f7040009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 -- 172.21.15.57:0/585482089 shutdown_connections 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 --2- 172.21.15.57:0/585482089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7054069140 0x7f7054104eb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 --2- 172.21.15.57:0/585482089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70540687a0 0x7f7054068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 -- 172.21.15.57:0/585482089 >> 172.21.15.57:0/585482089 conn(0x7f7054075130 msgr2=0x7f7054075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:01.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 -- 172.21.15.57:0/585482089 shutdown_connections 2024-02-28T23:44:01.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.423+0000 7f705ba00700 1 -- 172.21.15.57:0/585482089 wait complete. 2024-02-28T23:44:01.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 Processor -- start 2024-02-28T23:44:01.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 -- start start 2024-02-28T23:44:01.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 0x7f70541986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054198dc0 con 0x7f70540687a0 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.424+0000 7f705ba00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705419cb50 con 0x7f7054069140 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56314/0 (socket says 172.21.15.57:56314) 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 -- 172.21.15.57:0/1546702568 learned_addr learned my addr 172.21.15.57:0/1546702568 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7053e00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 0x7f70541986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 -- 172.21.15.57:0/1546702568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 msgr2=0x7f70541986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 0x7f70541986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 -- 172.21.15.57:0/1546702568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400095f0 con 0x7f70540687a0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7053e00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 0x7f70541986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.425+0000 7f7059000700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f704800b4b0 tx=0x7f704800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.426+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7048011800 con 0x7f70540687a0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.426+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7048011e20 con 0x7f70540687a0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.426+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705419ce30 con 0x7f70540687a0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.426+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704800f2e0 con 0x7f70540687a0 2024-02-28T23:44:01.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.426+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705419d380 con 0x7f70540687a0 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.427+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705404f2a0 con 0x7f70540687a0 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.429+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7048010380 con 0x7f70540687a0 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.429+0000 7f7051600700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 0x7f703c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.429+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f704808d1d0 con 0x7f70540687a0 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.429+0000 7f7053e00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 0x7f703c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.430+0000 7f7053e00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 0x7f703c06fd50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f704000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.434+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70480568e0 con 0x7f70540687a0 2024-02-28T23:44:01.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.690+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f705419d6d0 con 0x7f703c06d8a0 2024-02-28T23:44:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.703+0000 7f7051600700 1 -- 172.21.15.57:0/1546702568 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f705419d6d0 con 0x7f703c06d8a0 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (33m) 9m ago 47m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (43m) 9m ago 47m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (46m) 9m ago 46m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (29m) 9m ago 47m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:44:01.704 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (29m) 9m ago 46m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (33m) 9m ago 47m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (25m) 9m ago 40m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (25m) 9m ago 40m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (9m) 9m ago 40m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (24m) 9m ago 40m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (36m) 9m ago 49m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (35m) 9m ago 46m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (31m) 9m ago 49m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (30m) 9m ago 45m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:44:01.705 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (34m) 9m ago 47m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (34m) 9m ago 46m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (29m) 9m ago 45m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (28m) 9m ago 44m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (28m) 9m ago 43m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (27m) 9m ago 43m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (26m) 9m ago 42m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:44:01.706 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (26m) 9m ago 42m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:44:01.707 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (34m) 9m ago 46m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 msgr2=0x7f703c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 0x7f703c06fd50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f704000b540 comp rx=0 tx=0).stop 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 msgr2=0x7f70541981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f704800b4b0 tx=0x7f704800b870 comp rx=0 tx=0).stop 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 shutdown_connections 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f703c06d8a0 0x7f703c06fd50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70540687a0 0x7f70541981a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 --2- 172.21.15.57:0/1546702568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7054069140 0x7f70541986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 >> 172.21.15.57:0/1546702568 conn(0x7f7054075130 msgr2=0x7f70540fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:01.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 shutdown_connections 2024-02-28T23:44:01.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.707+0000 7f705ba00700 1 -- 172.21.15.57:0/1546702568 wait complete. 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.855+0000 7fa3cf600700 1 -- 172.21.15.57:0/3714642739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c810c8f0 msgr2=0x7fa3c810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.855+0000 7fa3cf600700 1 --2- 172.21.15.57:0/3714642739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c810c8f0 0x7fa3c810ccc0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009910 tx=0x7fa3bc009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 -- 172.21.15.57:0/3714642739 shutdown_connections 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 --2- 172.21.15.57:0/3714642739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c8106b80 0x7fa3c8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 --2- 172.21.15.57:0/3714642739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c810c8f0 0x7fa3c810ccc0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 -- 172.21.15.57:0/3714642739 >> 172.21.15.57:0/3714642739 conn(0x7fa3c8076060 msgr2=0x7fa3c8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 -- 172.21.15.57:0/3714642739 shutdown_connections 2024-02-28T23:44:01.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.856+0000 7fa3cf600700 1 -- 172.21.15.57:0/3714642739 wait complete. 2024-02-28T23:44:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 Processor -- start 2024-02-28T23:44:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 -- start start 2024-02-28T23:44:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 0x7fa3c819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c819d100 con 0x7fa3c8106b80 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3cf600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a0e90 con 0x7fa3c810c8f0 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3ccc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3ccc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56328/0 (socket says 172.21.15.57:56328) 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.857+0000 7fa3ccc00700 1 -- 172.21.15.57:0/927502138 learned_addr learned my addr 172.21.15.57:0/927502138 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3c7e00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 0x7fa3c819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3ccc00700 1 -- 172.21.15.57:0/927502138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 msgr2=0x7fa3c819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3ccc00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 0x7fa3c819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3ccc00700 1 -- 172.21.15.57:0/927502138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0095f0 con 0x7fa3c8106b80 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3c7e00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 0x7fa3c819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3ccc00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b5c0 tx=0x7fa3bc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.858+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3bc01d070 con 0x7fa3c8106b80 2024-02-28T23:44:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.859+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3bc00f460 con 0x7fa3c8106b80 2024-02-28T23:44:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.859+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c81a1110 con 0x7fa3c8106b80 2024-02-28T23:44:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.859+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3bc017610 con 0x7fa3c8106b80 2024-02-28T23:44:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.859+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c81a1600 con 0x7fa3c8106b80 2024-02-28T23:44:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.860+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c8109b20 con 0x7fa3c8106b80 2024-02-28T23:44:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.861+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa3bc00f5d0 con 0x7fa3c8106b80 2024-02-28T23:44:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.862+0000 7fa3c5600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 0x7fa3b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:01.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.862+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3bc08ebe0 con 0x7fa3c8106b80 2024-02-28T23:44:01.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.862+0000 7fa3c7e00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 0x7fa3b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:01.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.863+0000 7fa3c7e00700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 0x7fa3b006fd50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa3c819db00 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:01.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:01.866+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3bc0582f0 con 0x7fa3c8106b80 2024-02-28T23:44:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.173+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa3c804f2a0 con 0x7fa3c8106b80 2024-02-28T23:44:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.173+0000 7fa3c5600700 1 -- 172.21.15.57:0/927502138 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa3bc05b9b0 con 0x7fa3c8106b80 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:44:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:44:02.174 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:44:02.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.175+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 msgr2=0x7fa3b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.175+0000 7fa3cf600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 0x7fa3b006fd50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa3c819db00 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.175+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 msgr2=0x7fa3c819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b5c0 tx=0x7fa3bc00be00 comp rx=0 tx=0).stop 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 shutdown_connections 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa3b006d8a0 0x7fa3b006fd50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3c8106b80 0x7fa3c819c4e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 --2- 172.21.15.57:0/927502138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3c810c8f0 0x7fa3c819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 >> 172.21.15.57:0/927502138 conn(0x7fa3c8076060 msgr2=0x7fa3c8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 shutdown_connections 2024-02-28T23:44:02.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.176+0000 7fa3cf600700 1 -- 172.21.15.57:0/927502138 wait complete. 2024-02-28T23:44:02.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:02 smithi057 ceph-mon[89491]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:02.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:02 smithi084 ceph-mon[70440]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 -- 172.21.15.57:0/554876827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 msgr2=0x7f3e58110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 --2- 172.21.15.57:0/554876827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e58110d50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009910 tx=0x7f3e44009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 -- 172.21.15.57:0/554876827 shutdown_connections 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 --2- 172.21.15.57:0/554876827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e58110d50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 --2- 172.21.15.57:0/554876827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 0x7f3e580734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 -- 172.21.15.57:0/554876827 >> 172.21.15.57:0/554876827 conn(0x7f3e580fbda0 msgr2=0x7f3e580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 -- 172.21.15.57:0/554876827 shutdown_connections 2024-02-28T23:44:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.338+0000 7f3e5e600700 1 -- 172.21.15.57:0/554876827 wait complete. 2024-02-28T23:44:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.339+0000 7f3e5e600700 1 Processor -- start 2024-02-28T23:44:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e5e600700 1 -- start start 2024-02-28T23:44:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e5e600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 0x7f3e5819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e5e600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e5e600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5819d050 con 0x7f3e58073a00 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e5e600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e581a0de0 con 0x7f3e580730f0 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e56a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e56a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56348/0 (socket says 172.21.15.57:56348) 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.340+0000 7f3e56a00700 1 -- 172.21.15.57:0/698967704 learned_addr learned my addr 172.21.15.57:0/698967704 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e57400700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 0x7f3e5819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e56a00700 1 -- 172.21.15.57:0/698967704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 msgr2=0x7f3e5819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e56a00700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 0x7f3e5819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e56a00700 1 -- 172.21.15.57:0/698967704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e440095f0 con 0x7f3e58073a00 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e56a00700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006010 tx=0x7f3e4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:02.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.341+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4401d070 con 0x7f3e58073a00 2024-02-28T23:44:02.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.342+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e44004fb0 con 0x7f3e58073a00 2024-02-28T23:44:02.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.342+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e44022ea0 con 0x7f3e58073a00 2024-02-28T23:44:02.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.342+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e581a10c0 con 0x7f3e58073a00 2024-02-28T23:44:02.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.342+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e581a1610 con 0x7f3e58073a00 2024-02-28T23:44:02.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.343+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5810e4d0 con 0x7f3e58073a00 2024-02-28T23:44:02.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.344+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3e44022470 con 0x7f3e58073a00 2024-02-28T23:44:02.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.345+0000 7f3e4fe00700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 0x7f3e3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.345+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e4408eb50 con 0x7f3e58073a00 2024-02-28T23:44:02.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.346+0000 7f3e57400700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 0x7f3e3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.346+0000 7f3e57400700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 0x7f3e3c06fd50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005fd0 tx=0x7f3e4800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:02.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.350+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e440584b0 con 0x7f3e58073a00 2024-02-28T23:44:02.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.630+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e5804f2a0 con 0x7f3e58073a00 2024-02-28T23:44:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.631+0000 7f3e4fe00700 1 -- 172.21.15.57:0/698967704 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f3e44027020 con 0x7f3e58073a00 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:44:02.632 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:44:02.633 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:44:02.634 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:02.635 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:44:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:02.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 msgr2=0x7f3e3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 0x7f3e3c06fd50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005fd0 tx=0x7f3e4800a560 comp rx=0 tx=0).stop 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 msgr2=0x7f3e5819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006010 tx=0x7f3e4400be20 comp rx=0 tx=0).stop 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 shutdown_connections 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3e3c06d8a0 0x7f3e3c06fd50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e580730f0 0x7f3e5819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 --2- 172.21.15.57:0/698967704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e58073a00 0x7f3e5819c970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 >> 172.21.15.57:0/698967704 conn(0x7f3e580fbda0 msgr2=0x7f3e58102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 shutdown_connections 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.635+0000 7f3e5e600700 1 -- 172.21.15.57:0/698967704 wait complete. 2024-02-28T23:44:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:44:02.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 -- 172.21.15.57:0/2695882375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 msgr2=0x7f4a041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/2695882375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a041028a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009910 tx=0x7f49f8009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 -- 172.21.15.57:0/2695882375 shutdown_connections 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/2695882375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a04106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/2695882375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a041028a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 -- 172.21.15.57:0/2695882375 >> 172.21.15.57:0/2695882375 conn(0x7f4a040fdd90 msgr2=0x7f4a041001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 -- 172.21.15.57:0/2695882375 shutdown_connections 2024-02-28T23:44:02.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.798+0000 7f4a0aa00700 1 -- 172.21.15.57:0/2695882375 wait complete. 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.799+0000 7f4a0aa00700 1 Processor -- start 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.799+0000 7f4a0aa00700 1 -- start start 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a0aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a0aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a041983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a0aa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04198a80 con 0x7f4a041024d0 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a0aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0419c810 con 0x7f4a04102e70 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a09600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a09600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56362/0 (socket says 172.21.15.57:56362) 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.800+0000 7f4a09600700 1 -- 172.21.15.57:0/585294815 learned_addr learned my addr 172.21.15.57:0/585294815 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a09600700 1 -- 172.21.15.57:0/585294815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 msgr2=0x7f4a041983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a08c00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a041983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a09600700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a041983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a09600700 1 -- 172.21.15.57:0/585294815 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80095f0 con 0x7f4a041024d0 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a08c00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a041983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a09600700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f49f8006010 tx=0x7f49f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f801d070 con 0x7f4a041024d0 2024-02-28T23:44:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49f8004fb0 con 0x7f4a041024d0 2024-02-28T23:44:02.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f8022ea0 con 0x7f4a041024d0 2024-02-28T23:44:02.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0419ca90 con 0x7f4a041024d0 2024-02-28T23:44:02.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.801+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0419cf80 con 0x7f4a041024d0 2024-02-28T23:44:02.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.803+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0410a7e0 con 0x7f4a041024d0 2024-02-28T23:44:02.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.803+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f49f8022470 con 0x7f4a041024d0 2024-02-28T23:44:02.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.804+0000 7f4a02000700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 0x7f49ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:02.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.804+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f49f808ea50 con 0x7f4a041024d0 2024-02-28T23:44:02.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.805+0000 7f4a08c00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 0x7f49ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:02.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.805+0000 7f4a08c00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 0x7f49ec06fd50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:02.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:02.809+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49f8058290 con 0x7f4a041024d0 2024-02-28T23:44:03.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.050+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0419d2b0 con 0x7f49ec06d8a0 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.054+0000 7f4a02000700 1 -- 172.21.15.57:0/585294815 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4a0419d2b0 con 0x7f49ec06d8a0 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:44:03.053 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:44:03.054 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:44:03.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 msgr2=0x7f49ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 0x7f49ec06fd50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0005dc0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 msgr2=0x7f4a04197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f49f8006010 tx=0x7f49f800be20 comp rx=0 tx=0).stop 2024-02-28T23:44:03.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 shutdown_connections 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f49ec06d8a0 0x7f49ec06fd50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a041024d0 0x7f4a04197e60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 --2- 172.21.15.57:0/585294815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a04102e70 0x7f4a041983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 >> 172.21.15.57:0/585294815 conn(0x7f4a040fdd90 msgr2=0x7f4a04100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 shutdown_connections 2024-02-28T23:44:03.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.058+0000 7f4a0aa00700 1 -- 172.21.15.57:0/585294815 wait complete. 2024-02-28T23:44:03.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 -- 172.21.15.57:0/2610990122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8080730f0 msgr2=0x7fa8080734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 --2- 172.21.15.57:0/2610990122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8080730f0 0x7fa8080734c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009910 tx=0x7fa7fc009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:03.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 -- 172.21.15.57:0/2610990122 shutdown_connections 2024-02-28T23:44:03.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 --2- 172.21.15.57:0/2610990122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa808073a00 0x7fa80810c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 --2- 172.21.15.57:0/2610990122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8080730f0 0x7fa8080734c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 -- 172.21.15.57:0/2610990122 >> 172.21.15.57:0/2610990122 conn(0x7fa8080fbd80 msgr2=0x7fa8080fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:03.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 -- 172.21.15.57:0/2610990122 shutdown_connections 2024-02-28T23:44:03.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.223+0000 7fa80f800700 1 -- 172.21.15.57:0/2610990122 wait complete. 2024-02-28T23:44:03.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.224+0000 7fa80f800700 1 Processor -- start 2024-02-28T23:44:03.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.224+0000 7fa80f800700 1 -- start start 2024-02-28T23:44:03.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80f800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 0x7fa808197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80f800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80f800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808198bc0 con 0x7fa808073a00 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80f800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80819c900 con 0x7fa8080730f0 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56372/0 (socket says 172.21.15.57:56372) 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 -- 172.21.15.57:0/3920888303 learned_addr learned my addr 172.21.15.57:0/3920888303 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80ce00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 0x7fa808197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:03.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 -- 172.21.15.57:0/3920888303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 msgr2=0x7fa808197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 0x7fa808197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa807e00700 1 -- 172.21.15.57:0/3920888303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0095f0 con 0x7fa808073a00 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.225+0000 7fa80ce00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 0x7fa808197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa807e00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c930 tx=0x7fa7f400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f4007ad0 con 0x7fa808073a00 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7f4007c30 con 0x7fa808073a00 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f4018610 con 0x7fa808073a00 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80819cbe0 con 0x7fa808073a00 2024-02-28T23:44:03.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.226+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80819d0e0 con 0x7fa808073a00 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.227+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa808109c60 con 0x7fa808073a00 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.228+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa7f400f450 con 0x7fa808073a00 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.229+0000 7fa805600700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 0x7fa7f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.229+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa7f408d720 con 0x7fa808073a00 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.230+0000 7fa80ce00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 0x7fa7f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:03.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.230+0000 7fa80ce00700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 0x7fa7f006fd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006010 tx=0x7fa7fc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:03.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.234+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7f4056eb0 con 0x7fa808073a00 2024-02-28T23:44:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:03 smithi057 ceph-mon[89491]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:03 smithi057 ceph-mon[89491]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:03 smithi057 ceph-mon[89491]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:03 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/927502138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:44:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:03 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/698967704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:44:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:03 smithi084 ceph-mon[70440]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:03 smithi084 ceph-mon[70440]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:03 smithi084 ceph-mon[70440]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:03 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/927502138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:44:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:03 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/698967704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:44:03.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.558+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa80804f2a0 con 0x7fa808073a00 2024-02-28T23:44:03.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.558+0000 7fa805600700 1 -- 172.21.15.57:0/3920888303 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fa7f405a570 con 0x7fa808073a00 2024-02-28T23:44:03.558 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:44:03.558 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:44:03.558 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:44:03.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:44:03.559 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:44:03.560 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:44:03.561 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:44:03.561 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 msgr2=0x7fa7f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 0x7fa7f006fd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006010 tx=0x7fa7fc00b540 comp rx=0 tx=0).stop 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 msgr2=0x7fa8081984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c930 tx=0x7fa7f400ccf0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 shutdown_connections 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa7f006d8a0 0x7fa7f006fd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8080730f0 0x7fa808197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 --2- 172.21.15.57:0/3920888303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa808073a00 0x7fa8081984e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.561+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 >> 172.21.15.57:0/3920888303 conn(0x7fa8080fbd80 msgr2=0x7fa808106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.562+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 shutdown_connections 2024-02-28T23:44:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:03.562+0000 7fa80f800700 1 -- 172.21.15.57:0/3920888303 wait complete. 2024-02-28T23:44:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:04 smithi057 ceph-mon[89491]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:04 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3920888303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:44:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:04 smithi084 ceph-mon[70440]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:04 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3920888303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:44:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:05 smithi057 ceph-mon[89491]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:05 smithi084 ceph-mon[70440]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:07 smithi084 ceph-mon[70440]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:07 smithi057 ceph-mon[89491]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:09.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:09 smithi057 ceph-mon[89491]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:09 smithi084 ceph-mon[70440]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:11.391 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:11 smithi084 ceph-mon[70440]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:11 smithi057 ceph-mon[89491]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:13 smithi057 ceph-mon[89491]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:13 smithi084 ceph-mon[70440]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:15 smithi057 ceph-mon[89491]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:15.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:15 smithi084 ceph-mon[70440]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:17 smithi057 ceph-mon[89491]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:17.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:17 smithi084 ceph-mon[70440]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:19 smithi057 ceph-mon[89491]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:19 smithi084 ceph-mon[70440]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:21 smithi057 ceph-mon[89491]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:21 smithi084 ceph-mon[70440]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:22 smithi057 ceph-mon[89491]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:22.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:22 smithi084 ceph-mon[70440]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:24.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:24 smithi084 ceph-mon[70440]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:24 smithi057 ceph-mon[89491]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:26.863 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:26 smithi084 ceph-mon[70440]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:27.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:26 smithi057 ceph-mon[89491]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:29.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:28 smithi057 ceph-mon[89491]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:28 smithi084 ceph-mon[70440]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:30 smithi057 ceph-mon[89491]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:30 smithi084 ceph-mon[70440]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:32 smithi057 ceph-mon[89491]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:32 smithi084 ceph-mon[70440]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.741+0000 7f0d6ae00700 1 -- 172.21.15.57:0/652594974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 msgr2=0x7f0d64110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.741+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/652594974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d64110d50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009910 tx=0x7f0d50009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 -- 172.21.15.57:0/652594974 shutdown_connections 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/652594974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d64110d50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/652594974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d640734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 -- 172.21.15.57:0/652594974 >> 172.21.15.57:0/652594974 conn(0x7f0d640fbda0 msgr2=0x7f0d640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 -- 172.21.15.57:0/652594974 shutdown_connections 2024-02-28T23:44:33.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.742+0000 7f0d6ae00700 1 -- 172.21.15.57:0/652594974 wait complete. 2024-02-28T23:44:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 Processor -- start 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 -- start start 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d6419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6419d050 con 0x7f0d64073a00 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.743+0000 7f0d6ae00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641a0de0 con 0x7f0d640730f0 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43456/0 (socket says 172.21.15.57:43456) 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 -- 172.21.15.57:0/77472865 learned_addr learned my addr 172.21.15.57:0/77472865 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63e00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d6419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 -- 172.21.15.57:0/77472865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 msgr2=0x7f0d6419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d6419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63400700 1 -- 172.21.15.57:0/77472865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500095f0 con 0x7f0d64073a00 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.744+0000 7f0d63e00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d6419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d63400700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0d500092e0 tx=0x7f0d5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5001d070 con 0x7f0d64073a00 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d50005e00 con 0x7f0d64073a00 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d50017400 con 0x7f0d64073a00 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d641a10c0 con 0x7f0d64073a00 2024-02-28T23:44:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.745+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d641a1610 con 0x7f0d64073a00 2024-02-28T23:44:33.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.747+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6410e4d0 con 0x7f0d64073a00 2024-02-28T23:44:33.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.747+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0d5000f460 con 0x7f0d64073a00 2024-02-28T23:44:33.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.748+0000 7f0d60c00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 0x7f0d4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:33.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.748+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d5008e940 con 0x7f0d64073a00 2024-02-28T23:44:33.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.749+0000 7f0d63e00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 0x7f0d4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:33.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.749+0000 7f0d63e00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 0x7f0d4806fd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0d540094f0 tx=0x7f0d54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:33.753+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d50058170 con 0x7f0d64073a00 2024-02-28T23:44:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.003+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d641a18f0 con 0x7f0d4806d8a0 2024-02-28T23:44:34.003 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:34.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.005+0000 7f0d60c00700 1 -- 172.21.15.57:0/77472865 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0d641a18f0 con 0x7f0d4806d8a0 2024-02-28T23:44:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 msgr2=0x7f0d4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 0x7f0d4806fd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0d540094f0 tx=0x7f0d54009040 comp rx=0 tx=0).stop 2024-02-28T23:44:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 msgr2=0x7f0d6419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0d500092e0 tx=0x7f0d5000be20 comp rx=0 tx=0).stop 2024-02-28T23:44:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 shutdown_connections 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0d4806d8a0 0x7f0d4806fd50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d640730f0 0x7f0d6419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 --2- 172.21.15.57:0/77472865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d64073a00 0x7f0d6419c970 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.008+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 >> 172.21.15.57:0/77472865 conn(0x7f0d640fbda0 msgr2=0x7f0d64102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.009+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 shutdown_connections 2024-02-28T23:44:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.009+0000 7f0d6ae00700 1 -- 172.21.15.57:0/77472865 wait complete. 2024-02-28T23:44:34.025 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:44:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:34.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.175+0000 7fbebd000700 1 -- 172.21.15.57:0/1979309717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8108470 msgr2=0x7fbeb8108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 --2- 172.21.15.57:0/1979309717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8108470 0x7fbeb8108840 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009910 tx=0x7fbea8009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 -- 172.21.15.57:0/1979309717 shutdown_connections 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 --2- 172.21.15.57:0/1979309717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8102470 0x7fbeb81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 --2- 172.21.15.57:0/1979309717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8108470 0x7fbeb8108840 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 -- 172.21.15.57:0/1979309717 >> 172.21.15.57:0/1979309717 conn(0x7fbeb80fdf90 msgr2=0x7fbeb81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 -- 172.21.15.57:0/1979309717 shutdown_connections 2024-02-28T23:44:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.176+0000 7fbebd000700 1 -- 172.21.15.57:0/1979309717 wait complete. 2024-02-28T23:44:34.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.177+0000 7fbebd000700 1 Processor -- start 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.177+0000 7fbebd000700 1 -- start start 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbebd000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbebd000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8108470 0x7fbeb8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbebd000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8198e10 con 0x7fbeb8102470 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbebd000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb819cba0 con 0x7fbeb8108470 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43470/0 (socket says 172.21.15.57:43470) 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 -- 172.21.15.57:0/2252093725 learned_addr learned my addr 172.21.15.57:0/2252093725 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:34.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbead400700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8108470 0x7fbeb8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 -- 172.21.15.57:0/2252093725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8108470 msgr2=0x7fbeb8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8108470 0x7fbeb8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.178+0000 7fbeb5e00700 1 -- 172.21.15.57:0/2252093725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80095f0 con 0x7fbeb8102470 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbeb5e00700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbea800b5c0 tx=0x7fbea800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea801d070 con 0x7fbeb8102470 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb819ce20 con 0x7fbeb8102470 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbea800f460 con 0x7fbeb8102470 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea8017610 con 0x7fbeb8102470 2024-02-28T23:44:34.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.179+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb819d310 con 0x7fbeb8102470 2024-02-28T23:44:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.180+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb810a9e0 con 0x7fbeb8102470 2024-02-28T23:44:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.181+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbea800f5d0 con 0x7fbeb8102470 2024-02-28T23:44:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.182+0000 7fbeaf400700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 0x7fbe9806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.182+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbea808ec00 con 0x7fbeb8102470 2024-02-28T23:44:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.183+0000 7fbead400700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 0x7fbe9806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.184+0000 7fbead400700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 0x7fbe9806fd50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005fd0 tx=0x7fbea000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:34.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.188+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbea8058310 con 0x7fbeb8102470 2024-02-28T23:44:34.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.440+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbeb8199690 con 0x7fbe9806d8a0 2024-02-28T23:44:34.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.443+0000 7fbeaf400700 1 -- 172.21.15.57:0/2252093725 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbea000a2d0 con 0x7fbe9806d8a0 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 msgr2=0x7fbe9806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 0x7fbe9806fd50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005fd0 tx=0x7fbea000a560 comp rx=0 tx=0).stop 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 msgr2=0x7fbeb81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbea800b5c0 tx=0x7fbea800be00 comp rx=0 tx=0).stop 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 shutdown_connections 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe9806d8a0 0x7fbe9806fd50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbeb8102470 0x7fbeb81981f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 --2- 172.21.15.57:0/2252093725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb8108470 0x7fbeb8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 >> 172.21.15.57:0/2252093725 conn(0x7fbeb80fdf90 msgr2=0x7fbeb81002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:34.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 shutdown_connections 2024-02-28T23:44:34.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.446+0000 7fbebd000700 1 -- 172.21.15.57:0/2252093725 wait complete. 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 -- 172.21.15.57:0/2504440988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7638100e80 msgr2=0x7f7638104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 --2- 172.21.15.57:0/2504440988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7638100e80 0x7f7638104e50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7624009910 tx=0x7f7624009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 -- 172.21.15.57:0/2504440988 shutdown_connections 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 --2- 172.21.15.57:0/2504440988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7638100e80 0x7f7638104e50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 --2- 172.21.15.57:0/2504440988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76381004e0 0x7f76381008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 -- 172.21.15.57:0/2504440988 >> 172.21.15.57:0/2504440988 conn(0x7f76380fbda0 msgr2=0x7f76380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:34.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 -- 172.21.15.57:0/2504440988 shutdown_connections 2024-02-28T23:44:34.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.613+0000 7f763ca00700 1 -- 172.21.15.57:0/2504440988 wait complete. 2024-02-28T23:44:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 Processor -- start 2024-02-28T23:44:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 -- start start 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 0x7f7638198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638198ce0 con 0x7f76381004e0 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f763ca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763819ca70 con 0x7f7638100e80 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f7635e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f7635e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43490/0 (socket says 172.21.15.57:43490) 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.615+0000 7f7635e00700 1 -- 172.21.15.57:0/2458818741 learned_addr learned my addr 172.21.15.57:0/2458818741 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635400700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 0x7f7638198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635e00700 1 -- 172.21.15.57:0/2458818741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 msgr2=0x7f7638198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635e00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 0x7f7638198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635e00700 1 -- 172.21.15.57:0/2458818741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240095f0 con 0x7f76381004e0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635400700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 0x7f7638198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f7635e00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f762800d900 tx=0x7f762800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.616+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7628004a00 con 0x7f76381004e0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.617+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76280054e0 con 0x7f76381004e0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.617+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7628009d60 con 0x7f76381004e0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.617+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763819cd50 con 0x7f76381004e0 2024-02-28T23:44:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.617+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763819d2a0 con 0x7f76381004e0 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.618+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638104ce0 con 0x7f76381004e0 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.619+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7628005020 con 0x7f76381004e0 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.620+0000 7f762ea00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 0x7f761c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.620+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f762808d800 con 0x7f76381004e0 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.620+0000 7f7635400700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 0x7f761c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:34.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.621+0000 7f7635400700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 0x7f761c06fd50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7624006010 tx=0x7f762400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:34.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.625+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7628056f10 con 0x7f76381004e0 2024-02-28T23:44:34.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:34 smithi057 ceph-mon[89491]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:34.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:34 smithi057 ceph-mon[89491]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:34.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.864+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f763819d4c0 con 0x7f761c06d8a0 2024-02-28T23:44:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.877+0000 7f762ea00700 1 -- 172.21.15.57:0/2458818741 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f763819d4c0 con 0x7f761c06d8a0 2024-02-28T23:44:34.877 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (34m) 9m ago 48m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (44m) 9m ago 48m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (46m) 9m ago 46m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (30m) 9m ago 48m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (30m) 9m ago 46m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (33m) 9m ago 47m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (26m) 9m ago 40m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:44:34.878 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (25m) 9m ago 40m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (10m) 9m ago 40m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (25m) 9m ago 40m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (36m) 9m ago 49m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (35m) 9m ago 46m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (31m) 9m ago 50m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (31m) 9m ago 46m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:44:34.879 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (35m) 9m ago 48m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (35m) 9m ago 46m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (29m) 9m ago 45m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (29m) 9m ago 44m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (28m) 9m ago 44m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (28m) 9m ago 43m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:44:34.880 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (27m) 9m ago 43m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:44:34.881 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (26m) 9m ago 42m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:44:34.881 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (34m) 9m ago 47m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:44:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 msgr2=0x7f761c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 0x7f761c06fd50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7624006010 tx=0x7f762400b540 comp rx=0 tx=0).stop 2024-02-28T23:44:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 msgr2=0x7f76381980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f762800d900 tx=0x7f762800dcc0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 shutdown_connections 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f761c06d8a0 0x7f761c06fd50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76381004e0 0x7f76381980c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 --2- 172.21.15.57:0/2458818741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7638100e80 0x7f7638198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.880+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 >> 172.21.15.57:0/2458818741 conn(0x7f76380fbda0 msgr2=0x7f7638101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.881+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 shutdown_connections 2024-02-28T23:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:34.881+0000 7f763ca00700 1 -- 172.21.15.57:0/2458818741 wait complete. 2024-02-28T23:44:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 -- 172.21.15.57:0/4042672454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c106b80 msgr2=0x7fc46c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 --2- 172.21.15.57:0/4042672454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c106b80 0x7fc46c106ff0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc458009910 tx=0x7fc458009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:35.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 -- 172.21.15.57:0/4042672454 shutdown_connections 2024-02-28T23:44:35.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 --2- 172.21.15.57:0/4042672454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c106b80 0x7fc46c106ff0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 --2- 172.21.15.57:0/4042672454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c10c8f0 0x7fc46c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 -- 172.21.15.57:0/4042672454 >> 172.21.15.57:0/4042672454 conn(0x7fc46c076060 msgr2=0x7fc46c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:35.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 -- 172.21.15.57:0/4042672454 shutdown_connections 2024-02-28T23:44:35.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.037+0000 7fc473200700 1 -- 172.21.15.57:0/4042672454 wait complete. 2024-02-28T23:44:35.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.038+0000 7fc473200700 1 Processor -- start 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.038+0000 7fc473200700 1 -- start start 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.038+0000 7fc473200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c106b80 0x7fc46c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.038+0000 7fc473200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc473200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19d100 con 0x7fc46c10c8f0 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc473200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a0e90 con 0x7fc46c106b80 2024-02-28T23:44:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35300/0 (socket says 172.21.15.57:35300) 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 -- 172.21.15.57:0/94425906 learned_addr learned my addr 172.21.15.57:0/94425906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46be00700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c106b80 0x7fc46c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 -- 172.21.15.57:0/94425906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c106b80 msgr2=0x7fc46c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c106b80 0x7fc46c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.039+0000 7fc46b400700 1 -- 172.21.15.57:0/94425906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580095f0 con 0x7fc46c10c8f0 2024-02-28T23:44:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc46b400700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc458006010 tx=0x7fc45800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc45801d070 con 0x7fc46c10c8f0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc458004fb0 con 0x7fc46c10c8f0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc45800f690 con 0x7fc46c10c8f0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c1a1110 con 0x7fc46c10c8f0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.040+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c1a1600 con 0x7fc46c10c8f0 2024-02-28T23:44:35.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.041+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c109b20 con 0x7fc46c10c8f0 2024-02-28T23:44:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.042+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc4580228a0 con 0x7fc46c10c8f0 2024-02-28T23:44:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.043+0000 7fc468c00700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.043+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc45808ed40 con 0x7fc46c10c8f0 2024-02-28T23:44:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.044+0000 7fc46be00700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.044+0000 7fc46be00700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc45c006fd0 tx=0x7fc45c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.048+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc458058570 con 0x7fc46c10c8f0 2024-02-28T23:44:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:34 smithi084 ceph-mon[70440]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:34 smithi084 ceph-mon[70440]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:35.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.369+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc46c1a1930 con 0x7fc46c10c8f0 2024-02-28T23:44:35.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.370+0000 7fc468c00700 1 -- 172.21.15.57:0/94425906 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc458027280 con 0x7fc46c10c8f0 2024-02-28T23:44:35.370 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:44:35.370 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:44:35.370 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:44:35.370 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:44:35.371 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:44:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:44:35.372 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:44:35.372 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 msgr2=0x7fc45006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc45c006fd0 tx=0x7fc45c008040 comp rx=0 tx=0).stop 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 msgr2=0x7fc46c19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc458006010 tx=0x7fc45800be20 comp rx=0 tx=0).stop 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 shutdown_connections 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46c106b80 0x7fc46c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 --2- 172.21.15.57:0/94425906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c10c8f0 0x7fc46c19ca20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.372+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 >> 172.21.15.57:0/94425906 conn(0x7fc46c076060 msgr2=0x7fc46c077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:35.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.373+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 shutdown_connections 2024-02-28T23:44:35.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.373+0000 7fc473200700 1 -- 172.21.15.57:0/94425906 wait complete. 2024-02-28T23:44:35.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 -- 172.21.15.57:0/1310541756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 msgr2=0x7f2810110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 --2- 172.21.15.57:0/1310541756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f2810110b00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009910 tx=0x7f27fc009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:35.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 -- 172.21.15.57:0/1310541756 shutdown_connections 2024-02-28T23:44:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 --2- 172.21.15.57:0/1310541756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f2810110b00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 --2- 172.21.15.57:0/1310541756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 0x7f28100734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 -- 172.21.15.57:0/1310541756 >> 172.21.15.57:0/1310541756 conn(0x7f28100fbb10 msgr2=0x7f28100fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 -- 172.21.15.57:0/1310541756 shutdown_connections 2024-02-28T23:44:35.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.529+0000 7f2816200700 1 -- 172.21.15.57:0/1310541756 wait complete. 2024-02-28T23:44:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.530+0000 7f2816200700 1 Processor -- start 2024-02-28T23:44:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f2816200700 1 -- start start 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f2816200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 0x7f281019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f2816200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f2816200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281019cec0 con 0x7f2810073a00 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f2816200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101a0c50 con 0x7f28100730f0 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f280fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f280fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35316/0 (socket says 172.21.15.57:35316) 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.531+0000 7f280fe00700 1 -- 172.21.15.57:0/2130253788 learned_addr learned my addr 172.21.15.57:0/2130253788 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f2814e00700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 0x7f281019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f280fe00700 1 -- 172.21.15.57:0/2130253788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 msgr2=0x7f281019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f280fe00700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 0x7f281019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f280fe00700 1 -- 172.21.15.57:0/2130253788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0095f0 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f280fe00700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.532+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27fc01d070 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.533+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27fc00f460 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.533+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101a0f30 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.533+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27fc017630 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.533+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28101a1480 con 0x7f2810073a00 2024-02-28T23:44:35.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.534+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281010e280 con 0x7f2810073a00 2024-02-28T23:44:35.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.535+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f27fc00f5d0 con 0x7f2810073a00 2024-02-28T23:44:35.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.536+0000 7f280d600700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 0x7f27f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.536+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f27fc08eb40 con 0x7f2810073a00 2024-02-28T23:44:35.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.536+0000 7f2814e00700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 0x7f27f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.537+0000 7f2814e00700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 0x7f27f806fd50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2804006fd0 tx=0x7f2804008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.541+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27fc058250 con 0x7f2810073a00 2024-02-28T23:44:35.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.817+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f28101a10c0 con 0x7f2810073a00 2024-02-28T23:44:35.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.818+0000 7f280d600700 1 -- 172.21.15.57:0/2130253788 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f28101a10c0 con 0x7f2810073a00 2024-02-28T23:44:35.819 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:44:35.819 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:44:35.819 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:44:35.819 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:44:35.820 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:44:35.821 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:44:35.822 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:35.823 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:35.824 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:35.824 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 msgr2=0x7f27f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 0x7f27f806fd50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2804006fd0 tx=0x7f2804008040 comp rx=0 tx=0).stop 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 msgr2=0x7f281019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc00be00 comp rx=0 tx=0).stop 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 shutdown_connections 2024-02-28T23:44:35.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f27f806d8a0 0x7f27f806fd50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28100730f0 0x7f281019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 --2- 172.21.15.57:0/2130253788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810073a00 0x7f281019c7e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 >> 172.21.15.57:0/2130253788 conn(0x7f28100fbb10 msgr2=0x7f2810102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 shutdown_connections 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.821+0000 7f2816200700 1 -- 172.21.15.57:0/2130253788 wait complete. 2024-02-28T23:44:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:44:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 -- 172.21.15.57:0/2475046731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ffc70 msgr2=0x7feeec10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 --2- 172.21.15.57:0/2475046731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ffc70 0x7feeec10a6e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feed8009910 tx=0x7feed8009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:35.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:35 smithi057 ceph-mon[89491]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:35.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:35 smithi057 ceph-mon[89491]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:35.979 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/94425906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 -- 172.21.15.57:0/2475046731 shutdown_connections 2024-02-28T23:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 --2- 172.21.15.57:0/2475046731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ffc70 0x7feeec10a6e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 --2- 172.21.15.57:0/2475046731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ff360 0x7feeec0ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 -- 172.21.15.57:0/2475046731 >> 172.21.15.57:0/2475046731 conn(0x7feeec0748c0 msgr2=0x7feeec074cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 -- 172.21.15.57:0/2475046731 shutdown_connections 2024-02-28T23:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.979+0000 7feef3000700 1 -- 172.21.15.57:0/2475046731 wait complete. 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 Processor -- start 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 -- start start 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 0x7feeec100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec100fc0 con 0x7feeec0ff360 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.981+0000 7feef3000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec101100 con 0x7feeec0ffc70 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35338/0 (socket says 172.21.15.57:35338) 2024-02-28T23:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 -- 172.21.15.57:0/2569370892 learned_addr learned my addr 172.21.15.57:0/2569370892 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeeb400700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 0x7feeec100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 -- 172.21.15.57:0/2569370892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 msgr2=0x7feeec100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 0x7feeec100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeebe00700 1 -- 172.21.15.57:0/2569370892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80095f0 con 0x7feeec0ff360 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.982+0000 7feeeb400700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 0x7feeec100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feeebe00700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c8a0 tx=0x7feedc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feedc00cea0 con 0x7feeec0ff360 2024-02-28T23:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec1a2010 con 0x7feeec0ff360 2024-02-28T23:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec1a2420 con 0x7feeec0ff360 2024-02-28T23:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feedc004830 con 0x7feeec0ff360 2024-02-28T23:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.983+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feedc0055b0 con 0x7feeec0ff360 2024-02-28T23:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.985+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feecc005320 con 0x7feeec0ff360 2024-02-28T23:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.986+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7feedc004e10 con 0x7feeec0ff360 2024-02-28T23:44:35.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.987+0000 7feee8c00700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 0x7feed006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:35.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.987+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feedc08d1f0 con 0x7feeec0ff360 2024-02-28T23:44:35.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.988+0000 7feeeb400700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 0x7feed006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:35.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.988+0000 7feeeb400700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 0x7feed006fd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feed8005b40 tx=0x7feed801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:35.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:35.992+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feedc056900 con 0x7feeec0ff360 2024-02-28T23:44:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:35 smithi084 ceph-mon[70440]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:36.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:35 smithi084 ceph-mon[70440]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:36.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/94425906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:44:36.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.238+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feecc000bf0 con 0x7feed006d8a0 2024-02-28T23:44:36.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.241+0000 7feee8c00700 1 -- 172.21.15.57:0/2569370892 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feecc000bf0 con 0x7feed006d8a0 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:44:36.241 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:44:36.242 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:44:36.242 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:44:36.242 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.243+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 msgr2=0x7feed006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.243+0000 7feef3000700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 0x7feed006fd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feed8005b40 tx=0x7feed801a040 comp rx=0 tx=0).stop 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.243+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 msgr2=0x7feeec100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.243+0000 7feef3000700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c8a0 tx=0x7feedc00cc60 comp rx=0 tx=0).stop 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 shutdown_connections 2024-02-28T23:44:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed006d8a0 0x7feed006fd50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec0ff360 0x7feeec100540 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 --2- 172.21.15.57:0/2569370892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec0ffc70 0x7feeec100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 >> 172.21.15.57:0/2569370892 conn(0x7feeec0748c0 msgr2=0x7feeec104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 shutdown_connections 2024-02-28T23:44:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.244+0000 7feef3000700 1 -- 172.21.15.57:0/2569370892 wait complete. 2024-02-28T23:44:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 -- 172.21.15.57:0/121310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c102e70 msgr2=0x7f467c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 --2- 172.21.15.57:0/121310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c102e70 0x7f467c106e40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4664009910 tx=0x7f4664009c20 comp rx=0 tx=0).stop 2024-02-28T23:44:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 -- 172.21.15.57:0/121310259 shutdown_connections 2024-02-28T23:44:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 --2- 172.21.15.57:0/121310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c102e70 0x7f467c106e40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 --2- 172.21.15.57:0/121310259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c1024d0 0x7f467c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 -- 172.21.15.57:0/121310259 >> 172.21.15.57:0/121310259 conn(0x7f467c0fdd90 msgr2=0x7f467c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:36.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 -- 172.21.15.57:0/121310259 shutdown_connections 2024-02-28T23:44:36.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.404+0000 7f4681a00700 1 -- 172.21.15.57:0/121310259 wait complete. 2024-02-28T23:44:36.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 Processor -- start 2024-02-28T23:44:36.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 -- start start 2024-02-28T23:44:36.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:36.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 0x7f467c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c198a30 con 0x7f467c1024d0 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.405+0000 7f4681a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c19c7c0 con 0x7f467c102e70 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35344/0 (socket says 172.21.15.57:35344) 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 -- 172.21.15.57:0/3873257335 learned_addr learned my addr 172.21.15.57:0/3873257335 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f4673400700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 0x7f467c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 -- 172.21.15.57:0/3873257335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 msgr2=0x7f467c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 0x7f467c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f467be00700 1 -- 172.21.15.57:0/3873257335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46640095f0 con 0x7f467c1024d0 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.406+0000 7f4673400700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 0x7f467c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:44:36.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f467be00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f466c0060b0 tx=0x7f466c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466c00dda0 con 0x7f467c1024d0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f466c00f040 con 0x7f467c1024d0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466c014790 con 0x7f467c1024d0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c19caa0 con 0x7f467c1024d0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.407+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c19cff0 con 0x7f467c1024d0 2024-02-28T23:44:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.408+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c10a840 con 0x7f467c1024d0 2024-02-28T23:44:36.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.409+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f466c0148f0 con 0x7f467c1024d0 2024-02-28T23:44:36.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.410+0000 7f4679600700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 0x7f466006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:44:36.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.410+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f466c08cf90 con 0x7f467c1024d0 2024-02-28T23:44:36.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.410+0000 7f4673400700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 0x7f466006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:44:36.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.411+0000 7f4673400700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 0x7f466006fd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4664006010 tx=0x7f4664005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:44:36.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.415+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f466c0567d0 con 0x7f467c1024d0 2024-02-28T23:44:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.738+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f467c062380 con 0x7f467c1024d0 2024-02-28T23:44:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.739+0000 7f4679600700 1 -- 172.21.15.57:0/3873257335 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f466c01a090 con 0x7f467c1024d0 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:44:36.739 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:44:36.740 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:44:36.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.741+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 msgr2=0x7f466006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.741+0000 7f4681a00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 0x7f466006fd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4664006010 tx=0x7f4664005960 comp rx=0 tx=0).stop 2024-02-28T23:44:36.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.741+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 msgr2=0x7f467c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.741+0000 7f4681a00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f466c0060b0 tx=0x7f466c00d3b0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 shutdown_connections 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f466006d8a0 0x7f466006fd50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f467c1024d0 0x7f467c197e10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 --2- 172.21.15.57:0/3873257335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c102e70 0x7f467c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:44:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 >> 172.21.15.57:0/3873257335 conn(0x7f467c0fdd90 msgr2=0x7f467c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:44:36.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 shutdown_connections 2024-02-28T23:44:36.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:44:36.742+0000 7f4681a00700 1 -- 172.21.15.57:0/3873257335 wait complete. 2024-02-28T23:44:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:36 smithi057 ceph-mon[89491]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2130253788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:44:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:36 smithi084 ceph-mon[70440]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2130253788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:44:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:37 smithi057 ceph-mon[89491]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:38.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3873257335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:44:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:37 smithi084 ceph-mon[70440]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:44:38.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3873257335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:44:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:38 smithi057 ceph-mon[89491]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:38 smithi084 ceph-mon[70440]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:41.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:40 smithi057 ceph-mon[89491]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:40 smithi084 ceph-mon[70440]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:42 smithi057 ceph-mon[89491]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:42 smithi084 ceph-mon[70440]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:44 smithi057 ceph-mon[89491]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:44 smithi084 ceph-mon[70440]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:47.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:46 smithi057 ceph-mon[89491]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:46 smithi084 ceph-mon[70440]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:48 smithi057 ceph-mon[89491]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:49.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:48 smithi084 ceph-mon[70440]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:44:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:50 smithi057 ceph-mon[89491]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:50 smithi084 ceph-mon[70440]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:53.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:52 smithi057 ceph-mon[89491]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:52 smithi084 ceph-mon[70440]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:55.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:54 smithi057 ceph-mon[89491]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:54 smithi084 ceph-mon[70440]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:56 smithi057 ceph-mon[89491]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:56 smithi084 ceph-mon[70440]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:44:58 smithi057 ceph-mon[89491]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:44:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:44:58 smithi084 ceph-mon[70440]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:00 smithi057 ceph-mon[89491]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:00 smithi084 ceph-mon[70440]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:02 smithi057 ceph-mon[89491]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:02 smithi084 ceph-mon[70440]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:04 smithi057 ceph-mon[89491]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:05.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:04 smithi084 ceph-mon[70440]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:06.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 -- 172.21.15.57:0/2343420178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc468106b80 msgr2=0x7fc468106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 --2- 172.21.15.57:0/2343420178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc468106b80 0x7fc468106ff0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc454009910 tx=0x7fc454009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 -- 172.21.15.57:0/2343420178 shutdown_connections 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 --2- 172.21.15.57:0/2343420178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc468106b80 0x7fc468106ff0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 --2- 172.21.15.57:0/2343420178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46810c8f0 0x7fc46810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 -- 172.21.15.57:0/2343420178 >> 172.21.15.57:0/2343420178 conn(0x7fc468076060 msgr2=0x7fc468078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.924+0000 7fc46fe00700 1 -- 172.21.15.57:0/2343420178 shutdown_connections 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.925+0000 7fc46fe00700 1 -- 172.21.15.57:0/2343420178 wait complete. 2024-02-28T23:45:06.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.925+0000 7fc46fe00700 1 Processor -- start 2024-02-28T23:45:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46fe00700 1 -- start start 2024-02-28T23:45:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46fe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc468106b80 0x7fc46819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46fe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46819d060 con 0x7fc46810c8f0 2024-02-28T23:45:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46fe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4681a0df0 con 0x7fc468106b80 2024-02-28T23:45:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:06.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46ca00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54644/0 (socket says 172.21.15.57:54644) 2024-02-28T23:45:06.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.926+0000 7fc46ca00700 1 -- 172.21.15.57:0/1173271175 learned_addr learned my addr 172.21.15.57:0/1173271175 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.927+0000 7fc46d400700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc468106b80 0x7fc46819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.927+0000 7fc46ca00700 1 -- 172.21.15.57:0/1173271175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc468106b80 msgr2=0x7fc46819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.927+0000 7fc46ca00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc468106b80 0x7fc46819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.927+0000 7fc46ca00700 1 -- 172.21.15.57:0/1173271175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4540095f0 con 0x7fc46810c8f0 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.927+0000 7fc46ca00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc4540092e0 tx=0x7fc45400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.928+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc45401d070 con 0x7fc46810c8f0 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.928+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc454005e00 con 0x7fc46810c8f0 2024-02-28T23:45:06.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.928+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4681a1070 con 0x7fc46810c8f0 2024-02-28T23:45:06.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.928+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc454017400 con 0x7fc46810c8f0 2024-02-28T23:45:06.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.928+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4681a1560 con 0x7fc46810c8f0 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.929+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46810ecc0 con 0x7fc46810c8f0 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.930+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc45400f460 con 0x7fc46810c8f0 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.931+0000 7fc466000700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.931+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc45408ea30 con 0x7fc46810c8f0 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.931+0000 7fc46d400700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.932+0000 7fc46d400700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009d10 tx=0x7fc45c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:06.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:06.936+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4540581c0 con 0x7fc46810c8f0 2024-02-28T23:45:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:06 smithi057 ceph-mon[89491]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:06 smithi084 ceph-mon[70440]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:07.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.191+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc46802cca0 con 0x7fc45006d8a0 2024-02-28T23:45:07.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.193+0000 7fc466000700 1 -- 172.21.15.57:0/1173271175 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc46802cca0 con 0x7fc45006d8a0 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 msgr2=0x7fc45006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009d10 tx=0x7fc45c009040 comp rx=0 tx=0).stop 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 msgr2=0x7fc46819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc4540092e0 tx=0x7fc45400be20 comp rx=0 tx=0).stop 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 shutdown_connections 2024-02-28T23:45:07.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc45006d8a0 0x7fc45006fd50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc468106b80 0x7fc46819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 --2- 172.21.15.57:0/1173271175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46810c8f0 0x7fc46819c980 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.196+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 >> 172.21.15.57:0/1173271175 conn(0x7fc468076060 msgr2=0x7fc468077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.197+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 shutdown_connections 2024-02-28T23:45:07.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.197+0000 7fc46fe00700 1 -- 172.21.15.57:0/1173271175 wait complete. 2024-02-28T23:45:07.212 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:45:07.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 -- 172.21.15.57:0/1261600187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075b90 msgr2=0x7f4090110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 --2- 172.21.15.57:0/1261600187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075b90 0x7f4090110bd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f407c009910 tx=0x7f407c009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:07.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 -- 172.21.15.57:0/1261600187 shutdown_connections 2024-02-28T23:45:07.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 --2- 172.21.15.57:0/1261600187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075b90 0x7f4090110bd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 --2- 172.21.15.57:0/1261600187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075280 0x7f4090075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.362+0000 7f4096000700 1 -- 172.21.15.57:0/1261600187 >> 172.21.15.57:0/1261600187 conn(0x7f40900fdf90 msgr2=0x7f40901003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:07.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.363+0000 7f4096000700 1 -- 172.21.15.57:0/1261600187 shutdown_connections 2024-02-28T23:45:07.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.363+0000 7f4096000700 1 -- 172.21.15.57:0/1261600187 wait complete. 2024-02-28T23:45:07.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 Processor -- start 2024-02-28T23:45:07.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 -- start start 2024-02-28T23:45:07.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 0x7f409019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409019d1f0 con 0x7f4090075280 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.364+0000 7f4096000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901a0f80 con 0x7f4090075b90 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54674/0 (socket says 172.21.15.57:54674) 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 -- 172.21.15.57:0/3200205727 learned_addr learned my addr 172.21.15.57:0/3200205727 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408ea00700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 0x7f409019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 -- 172.21.15.57:0/3200205727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 msgr2=0x7f409019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 0x7f409019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408f400700 1 -- 172.21.15.57:0/3200205727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0095f0 con 0x7f4090075280 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.365+0000 7f408ea00700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 0x7f409019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f408f400700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f408000b4b0 tx=0x7f408000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4080011800 con 0x7f4090075280 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4080011e20 con 0x7f4090075280 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40901a1260 con 0x7f4090075280 2024-02-28T23:45:07.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408000f2e0 con 0x7f4090075280 2024-02-28T23:45:07.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.366+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40901a17b0 con 0x7f4090075280 2024-02-28T23:45:07.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.367+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409010e340 con 0x7f4090075280 2024-02-28T23:45:07.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.368+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4080010380 con 0x7f4090075280 2024-02-28T23:45:07.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.370+0000 7f4087e00700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 0x7f407406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.370+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f408008d180 con 0x7f4090075280 2024-02-28T23:45:07.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.370+0000 7f408ea00700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 0x7f407406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.371+0000 7f408ea00700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 0x7f407406fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b5c0 tx=0x7f407c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:07.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.374+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4080056890 con 0x7f4090075280 2024-02-28T23:45:07.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.638+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40901a1a30 con 0x7f407406d8a0 2024-02-28T23:45:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.640+0000 7f4087e00700 1 -- 172.21.15.57:0/3200205727 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f40901a1a30 con 0x7f407406d8a0 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 msgr2=0x7f407406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 0x7f407406fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b5c0 tx=0x7f407c005960 comp rx=0 tx=0).stop 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 msgr2=0x7f409019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f408000b4b0 tx=0x7f408000b870 comp rx=0 tx=0).stop 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 shutdown_connections 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f407406d8a0 0x7f407406fd50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4090075280 0x7f409019c5d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 --2- 172.21.15.57:0/3200205727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4090075b90 0x7f409019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.643+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 >> 172.21.15.57:0/3200205727 conn(0x7f40900fdf90 msgr2=0x7f40900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:07.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.644+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 shutdown_connections 2024-02-28T23:45:07.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.644+0000 7f4096000700 1 -- 172.21.15.57:0/3200205727 wait complete. 2024-02-28T23:45:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 -- 172.21.15.57:0/4000739334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd8068540 msgr2=0x7f7fd80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 --2- 172.21.15.57:0/4000739334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd8068540 0x7f7fd80689b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009910 tx=0x7f7fc4009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 -- 172.21.15.57:0/4000739334 shutdown_connections 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 --2- 172.21.15.57:0/4000739334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd8068540 0x7f7fd80689b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 --2- 172.21.15.57:0/4000739334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd81061b0 0x7f7fd8106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 -- 172.21.15.57:0/4000739334 >> 172.21.15.57:0/4000739334 conn(0x7f7fd8074fa0 msgr2=0x7f7fd80753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 -- 172.21.15.57:0/4000739334 shutdown_connections 2024-02-28T23:45:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.796+0000 7f7fdd600700 1 -- 172.21.15.57:0/4000739334 wait complete. 2024-02-28T23:45:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.797+0000 7f7fdd600700 1 Processor -- start 2024-02-28T23:45:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.797+0000 7f7fdd600700 1 -- start start 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fdd600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 0x7f7fd819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fdd600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fdd600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd819c520 con 0x7f7fd81061b0 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fdd600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81959d0 con 0x7f7fd8068540 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fd7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54696/0 (socket says 172.21.15.57:54696) 2024-02-28T23:45:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.798+0000 7f7fd7400700 1 -- 172.21.15.57:0/2568767109 learned_addr learned my addr 172.21.15.57:0/2568767109 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7e00700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 0x7f7fd819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7400700 1 -- 172.21.15.57:0/2568767109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 msgr2=0x7f7fd819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7400700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 0x7f7fd819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7400700 1 -- 172.21.15.57:0/2568767109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40095f0 con 0x7f7fd81061b0 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7e00700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 0x7f7fd819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd7400700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006010 tx=0x7f7fc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc401d070 con 0x7f7fd81061b0 2024-02-28T23:45:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fc4004fb0 con 0x7f7fd81061b0 2024-02-28T23:45:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.799+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc4022ea0 con 0x7f7fd81061b0 2024-02-28T23:45:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.800+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd8195c50 con 0x7f7fd81061b0 2024-02-28T23:45:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.800+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd81961c0 con 0x7f7fd81061b0 2024-02-28T23:45:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.801+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd8062380 con 0x7f7fd81061b0 2024-02-28T23:45:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.802+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7fc4022470 con 0x7f7fd81061b0 2024-02-28T23:45:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.803+0000 7f7fd4c00700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.803+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7fc408eab0 con 0x7f7fd81061b0 2024-02-28T23:45:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.803+0000 7f7fd7e00700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.804+0000 7f7fd7e00700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005fd0 tx=0x7f7fc8005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:07.809+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fc4090e60 con 0x7f7fd81061b0 2024-02-28T23:45:08.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.036+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fd8196530 con 0x7f7fbc06d8a0 2024-02-28T23:45:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.066+0000 7f7fd4c00700 1 -- 172.21.15.57:0/2568767109 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7fd8196530 con 0x7f7fbc06d8a0 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (34m) 10m ago 48m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (44m) 10m ago 48m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (47m) 10m ago 47m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (30m) 10m ago 48m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:45:08.066 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (30m) 10m ago 47m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (34m) 10m ago 48m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (26m) 10m ago 41m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (26m) 10m ago 41m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (10m) 10m ago 41m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (25m) 10m ago 41m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (37m) 10m ago 50m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (36m) 10m ago 47m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (32m) 10m ago 50m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:45:08.067 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (31m) 10m ago 47m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (35m) 10m ago 48m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (35m) 10m ago 47m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (30m) 10m ago 46m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (29m) 10m ago 45m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (29m) 10m ago 44m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (28m) 10m ago 44m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (27m) 10m ago 43m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (27m) 10m ago 43m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:45:08.068 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (35m) 10m ago 47m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:45:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 msgr2=0x7f7fbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005fd0 tx=0x7f7fc8005dc0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 msgr2=0x7f7fd819be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006010 tx=0x7f7fc400be20 comp rx=0 tx=0).stop 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 shutdown_connections 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fd8068540 0x7f7fd819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 --2- 172.21.15.57:0/2568767109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fd81061b0 0x7f7fd819be90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 >> 172.21.15.57:0/2568767109 conn(0x7f7fd8074fa0 msgr2=0x7f7fd80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 shutdown_connections 2024-02-28T23:45:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.071+0000 7f7fdd600700 1 -- 172.21.15.57:0/2568767109 wait complete. 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 -- 172.21.15.57:0/2251315993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075280 msgr2=0x7f09c8075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 --2- 172.21.15.57:0/2251315993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075280 0x7f09c8075650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009910 tx=0x7f09bc009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 -- 172.21.15.57:0/2251315993 shutdown_connections 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 --2- 172.21.15.57:0/2251315993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075b90 0x7f09c8110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 --2- 172.21.15.57:0/2251315993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075280 0x7f09c8075650 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 -- 172.21.15.57:0/2251315993 >> 172.21.15.57:0/2251315993 conn(0x7f09c80fdf60 msgr2=0x7f09c8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:08.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 -- 172.21.15.57:0/2251315993 shutdown_connections 2024-02-28T23:45:08.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.228+0000 7f09cf400700 1 -- 172.21.15.57:0/2251315993 wait complete. 2024-02-28T23:45:08.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.229+0000 7f09cf400700 1 Processor -- start 2024-02-28T23:45:08.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.229+0000 7f09cf400700 1 -- start start 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09cf400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075280 0x7f09c819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09cf400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c819d170 con 0x7f09c8075b90 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09cf400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c81a0f00 con 0x7f09c8075280 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09c7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09c7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54714/0 (socket says 172.21.15.57:54714) 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.230+0000 7f09c7e00700 1 -- 172.21.15.57:0/1497962079 learned_addr learned my addr 172.21.15.57:0/1497962079 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c7e00700 1 -- 172.21.15.57:0/1497962079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075280 msgr2=0x7f09c819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09cca00700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075280 0x7f09c819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c7e00700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075280 0x7f09c819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c7e00700 1 -- 172.21.15.57:0/1497962079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0095f0 con 0x7f09c8075b90 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c7e00700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b4b0 tx=0x7f09b400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09b4011800 con 0x7f09c8075b90 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09b4011e20 con 0x7f09c8075b90 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c81a11e0 con 0x7f09c8075b90 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09b400f2e0 con 0x7f09c8075b90 2024-02-28T23:45:08.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.231+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c81a1730 con 0x7f09c8075b90 2024-02-28T23:45:08.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.232+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c810e330 con 0x7f09c8075b90 2024-02-28T23:45:08.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.233+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f09b4010380 con 0x7f09c8075b90 2024-02-28T23:45:08.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.234+0000 7f09c5600700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 0x7f09b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.234+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f09b408d180 con 0x7f09c8075b90 2024-02-28T23:45:08.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.235+0000 7f09cca00700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 0x7f09b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.235+0000 7f09cca00700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 0x7f09b006fd50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b5c0 tx=0x7f09bc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:08.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.239+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09b4056890 con 0x7f09c8075b90 2024-02-28T23:45:08.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.550+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09c804f2a0 con 0x7f09c8075b90 2024-02-28T23:45:08.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.551+0000 7f09c5600700 1 -- 172.21.15.57:0/1497962079 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f09b4059f50 con 0x7f09c8075b90 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:45:08.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:45:08.552 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 msgr2=0x7f09b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 0x7f09b006fd50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b5c0 tx=0x7f09bc005fd0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 msgr2=0x7f09c819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b4b0 tx=0x7f09b400b870 comp rx=0 tx=0).stop 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 shutdown_connections 2024-02-28T23:45:08.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f09b006d8a0 0x7f09b006fd50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c8075280 0x7f09c819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 --2- 172.21.15.57:0/1497962079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09c8075b90 0x7f09c819ca90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 >> 172.21.15.57:0/1497962079 conn(0x7f09c80fdf60 msgr2=0x7f09c80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:08.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 shutdown_connections 2024-02-28T23:45:08.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.554+0000 7f09cf400700 1 -- 172.21.15.57:0/1497962079 wait complete. 2024-02-28T23:45:08.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 -- 172.21.15.57:0/3114859030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 msgr2=0x7fd770075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 --2- 172.21.15.57:0/3114859030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd770075650 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd760009910 tx=0x7fd760009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 -- 172.21.15.57:0/3114859030 shutdown_connections 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 --2- 172.21.15.57:0/3114859030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 0x7fd770110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 --2- 172.21.15.57:0/3114859030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd770075650 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 -- 172.21.15.57:0/3114859030 >> 172.21.15.57:0/3114859030 conn(0x7fd7700fdf90 msgr2=0x7fd7701003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.715+0000 7fd775c00700 1 -- 172.21.15.57:0/3114859030 shutdown_connections 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.716+0000 7fd775c00700 1 -- 172.21.15.57:0/3114859030 wait complete. 2024-02-28T23:45:08.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.716+0000 7fd775c00700 1 Processor -- start 2024-02-28T23:45:08.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd775c00700 1 -- start start 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd775c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd775c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 0x7fd77019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd775c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77019d050 con 0x7fd770075280 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd775c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7701a0de0 con 0x7fd770075b90 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd76ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd76ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54730/0 (socket says 172.21.15.57:54730) 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.717+0000 7fd76ea00700 1 -- 172.21.15.57:0/2003253033 learned_addr learned my addr 172.21.15.57:0/2003253033 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.718+0000 7fd76ea00700 1 -- 172.21.15.57:0/2003253033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 msgr2=0x7fd77019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.718+0000 7fd76e000700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 0x7fd77019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.718+0000 7fd76ea00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 0x7fd77019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.718+0000 7fd76ea00700 1 -- 172.21.15.57:0/2003253033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600095f0 con 0x7fd770075280 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.718+0000 7fd76ea00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c010 tx=0x7fd76000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.719+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd76001d070 con 0x7fd770075280 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.719+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7600052a0 con 0x7fd770075280 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.719+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd760022470 con 0x7fd770075280 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.719+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7701a1060 con 0x7fd770075280 2024-02-28T23:45:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.719+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7701a1550 con 0x7fd770075280 2024-02-28T23:45:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.720+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77010e340 con 0x7fd770075280 2024-02-28T23:45:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.721+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd76000f820 con 0x7fd770075280 2024-02-28T23:45:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.722+0000 7fd767400700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.722+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd76008e9c0 con 0x7fd770075280 2024-02-28T23:45:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.722+0000 7fd76e000700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.723+0000 7fd76e000700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd77019da50 tx=0x7fd75c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:08.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:08.727+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd760058150 con 0x7fd770075280 2024-02-28T23:45:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:08 smithi057 ceph-mon[89491]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:08 smithi057 ceph-mon[89491]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:08 smithi057 ceph-mon[89491]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:08 smithi057 ceph-mon[89491]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1497962079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:45:09.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.004+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd77010dd20 con 0x7fd770075280 2024-02-28T23:45:09.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.004+0000 7fd767400700 1 -- 172.21.15.57:0/2003253033 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fd76005b810 con 0x7fd770075280 2024-02-28T23:45:09.004 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:45:09.004 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:45:09.005 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:45:09.006 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:09.007 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 msgr2=0x7fd75406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd77019da50 tx=0x7fd75c009260 comp rx=0 tx=0).stop 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 msgr2=0x7fd77019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c010 tx=0x7fd76000b660 comp rx=0 tx=0).stop 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 shutdown_connections 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd75406d8a0 0x7fd75406fd50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.007+0000 7fd775c00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd770075280 0x7fd77019c430 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.008+0000 7fd775c00700 1 --2- 172.21.15.57:0/2003253033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd770075b90 0x7fd77019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.008+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 >> 172.21.15.57:0/2003253033 conn(0x7fd7700fdf90 msgr2=0x7fd7700ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:09.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.008+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 shutdown_connections 2024-02-28T23:45:09.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.008+0000 7fd775c00700 1 -- 172.21.15.57:0/2003253033 wait complete. 2024-02-28T23:45:09.009 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:45:09.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:08 smithi084 ceph-mon[70440]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:09.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:08 smithi084 ceph-mon[70440]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:09.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:08 smithi084 ceph-mon[70440]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:09.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:08 smithi084 ceph-mon[70440]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:09.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1497962079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:45:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 -- 172.21.15.57:0/2483831591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8241004e0 msgr2=0x7fc8241008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 --2- 172.21.15.57:0/2483831591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8241004e0 0x7fc8241008b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc818009910 tx=0x7fc818009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 -- 172.21.15.57:0/2483831591 shutdown_connections 2024-02-28T23:45:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 --2- 172.21.15.57:0/2483831591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc824100e80 0x7fc824104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 --2- 172.21.15.57:0/2483831591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8241004e0 0x7fc8241008b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 -- 172.21.15.57:0/2483831591 >> 172.21.15.57:0/2483831591 conn(0x7fc8240fbda0 msgr2=0x7fc8240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:09.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 -- 172.21.15.57:0/2483831591 shutdown_connections 2024-02-28T23:45:09.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.166+0000 7fc82bc00700 1 -- 172.21.15.57:0/2483831591 wait complete. 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.167+0000 7fc82bc00700 1 Processor -- start 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.167+0000 7fc82bc00700 1 -- start start 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc82bc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 0x7fc824073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc82bc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc82bc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824073ad0 con 0x7fc824100e80 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc82bc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824073c10 con 0x7fc8241004e0 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54746/0 (socket says 172.21.15.57:54746) 2024-02-28T23:45:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 -- 172.21.15.57:0/1583854175 learned_addr learned my addr 172.21.15.57:0/1583854175 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc829200700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 0x7fc824073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 -- 172.21.15.57:0/1583854175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 msgr2=0x7fc824073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 0x7fc824073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc823e00700 1 -- 172.21.15.57:0/1583854175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180095f0 con 0x7fc824100e80 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.168+0000 7fc829200700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 0x7fc824073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc823e00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e8f0 tx=0x7fc81000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc810009790 con 0x7fc824100e80 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc810004cf0 con 0x7fc824100e80 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc810005690 con 0x7fc824100e80 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241a20c0 con 0x7fc824100e80 2024-02-28T23:45:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.169+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8241a2470 con 0x7fc824100e80 2024-02-28T23:45:09.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.170+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8241087f0 con 0x7fc824100e80 2024-02-28T23:45:09.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.171+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc810004750 con 0x7fc824100e80 2024-02-28T23:45:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.172+0000 7fc821600700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 0x7fc80c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.172+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc810014070 con 0x7fc824100e80 2024-02-28T23:45:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.173+0000 7fc829200700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 0x7fc80c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.173+0000 7fc829200700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 0x7fc80c06fd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b760 tx=0x7fc818009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:09.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.177+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc810057080 con 0x7fc824100e80 2024-02-28T23:45:09.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.423+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc82402cca0 con 0x7fc80c06d8a0 2024-02-28T23:45:09.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.426+0000 7fc821600700 1 -- 172.21.15.57:0/1583854175 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc82402cca0 con 0x7fc80c06d8a0 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:45:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:45:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:45:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:45:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:45:09.427 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.429+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 msgr2=0x7fc80c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.429+0000 7fc82bc00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 0x7fc80c06fd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b760 tx=0x7fc818009f70 comp rx=0 tx=0).stop 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.429+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 msgr2=0x7fc824073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.429+0000 7fc82bc00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e8f0 tx=0x7fc81000ecb0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 shutdown_connections 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc80c06d8a0 0x7fc80c06fd50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8241004e0 0x7fc824073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 --2- 172.21.15.57:0/1583854175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc824100e80 0x7fc824073590 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 >> 172.21.15.57:0/1583854175 conn(0x7fc8240fbda0 msgr2=0x7fc8240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:09.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 shutdown_connections 2024-02-28T23:45:09.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.430+0000 7fc82bc00700 1 -- 172.21.15.57:0/1583854175 wait complete. 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.588+0000 7f2499000700 1 -- 172.21.15.57:0/1855223241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 msgr2=0x7f249410c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.588+0000 7f2499000700 1 --2- 172.21.15.57:0/1855223241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f249410c3d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2480009940 tx=0x7f2480009c50 comp rx=0 tx=0).stop 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 -- 172.21.15.57:0/1855223241 shutdown_connections 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 --2- 172.21.15.57:0/1855223241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f249410c3d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 --2- 172.21.15.57:0/1855223241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 -- 172.21.15.57:0/1855223241 >> 172.21.15.57:0/1855223241 conn(0x7f24940fbbb0 msgr2=0x7f24940fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 -- 172.21.15.57:0/1855223241 shutdown_connections 2024-02-28T23:45:09.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.589+0000 7f2499000700 1 -- 172.21.15.57:0/1855223241 wait complete. 2024-02-28T23:45:09.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 Processor -- start 2024-02-28T23:45:09.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 -- start start 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494198a50 con 0x7f24940739a0 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.590+0000 7f2499000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249419c7e0 con 0x7f2494073090 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54772/0 (socket says 172.21.15.57:54772) 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 -- 172.21.15.57:0/1636256119 learned_addr learned my addr 172.21.15.57:0/1636256119 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2493400700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 -- 172.21.15.57:0/1636256119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 msgr2=0x7f2494197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 -- 172.21.15.57:0/1636256119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800095f0 con 0x7f24940739a0 2024-02-28T23:45:09.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2493400700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:09.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.591+0000 7f2492a00700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f248000b870 tx=0x7f24800056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:09.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.592+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248001d070 con 0x7f24940739a0 2024-02-28T23:45:09.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.592+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248000f460 con 0x7f24940739a0 2024-02-28T23:45:09.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.592+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2480021640 con 0x7f24940739a0 2024-02-28T23:45:09.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.592+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249419ca60 con 0x7f24940739a0 2024-02-28T23:45:09.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.592+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249419cf50 con 0x7f24940739a0 2024-02-28T23:45:09.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.593+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2494109ad0 con 0x7f24940739a0 2024-02-28T23:45:09.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.594+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f248000fa60 con 0x7f24940739a0 2024-02-28T23:45:09.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.595+0000 7f248be00700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 0x7f247806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:09.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.595+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f248008e810 con 0x7f24940739a0 2024-02-28T23:45:09.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.596+0000 7f2493400700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 0x7f247806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:09.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.596+0000 7f2493400700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 0x7f247806fd50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2484005fd0 tx=0x7f2484005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:09.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.600+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2480057fc0 con 0x7f24940739a0 2024-02-28T23:45:09.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.921+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2494062380 con 0x7f24940739a0 2024-02-28T23:45:09.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.922+0000 7f248be00700 1 -- 172.21.15.57:0/1636256119 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f248005b680 con 0x7f24940739a0 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:45:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:45:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:45:09.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 msgr2=0x7f247806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 0x7f247806fd50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2484005fd0 tx=0x7f2484005dc0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 msgr2=0x7f2494198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:09.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f248000b870 tx=0x7f24800056a0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 shutdown_connections 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f247806d8a0 0x7f247806fd50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2494073090 0x7f2494197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 --2- 172.21.15.57:0/1636256119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24940739a0 0x7f2494198370 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 >> 172.21.15.57:0/1636256119 conn(0x7f24940fbbb0 msgr2=0x7f2494106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 shutdown_connections 2024-02-28T23:45:09.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:09.926+0000 7f2499000700 1 -- 172.21.15.57:0/1636256119 wait complete. 2024-02-28T23:45:10.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:09 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2003253033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:45:10.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:09 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2003253033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:45:11.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:10 smithi057 ceph-mon[89491]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:11.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:10 smithi057 ceph-mon[89491]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:11.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:10 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1636256119' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:45:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:10 smithi084 ceph-mon[70440]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:10 smithi084 ceph-mon[70440]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:10 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1636256119' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:45:13.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:12 smithi057 ceph-mon[89491]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:12 smithi084 ceph-mon[70440]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:14 smithi084 ceph-mon[70440]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:14 smithi057 ceph-mon[89491]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:16 smithi057 ceph-mon[89491]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:17.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:16 smithi084 ceph-mon[70440]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:18 smithi057 ceph-mon[89491]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:18 smithi084 ceph-mon[70440]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:19.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:20 smithi057 ceph-mon[89491]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:20 smithi084 ceph-mon[70440]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:22 smithi057 ceph-mon[89491]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:23.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:22 smithi084 ceph-mon[70440]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:24 smithi057 ceph-mon[89491]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:24 smithi084 ceph-mon[70440]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:27.133 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:26 smithi084 ceph-mon[70440]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:26 smithi057 ceph-mon[89491]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:29.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:28 smithi084 ceph-mon[70440]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:28 smithi057 ceph-mon[89491]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:30 smithi057 ceph-mon[89491]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:30 smithi084 ceph-mon[70440]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:33.308 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:32 smithi057 ceph-mon[89491]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:32 smithi084 ceph-mon[70440]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:35.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:34 smithi057 ceph-mon[89491]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:34 smithi084 ceph-mon[70440]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:36 smithi057 ceph-mon[89491]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:36 smithi084 ceph-mon[70440]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:38 smithi057 ceph-mon[89491]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:38 smithi084 ceph-mon[70440]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.102+0000 7f7157200700 1 -- 172.21.15.57:0/3316462987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7150004d20 msgr2=0x7f7150005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.102+0000 7f7157200700 1 --2- 172.21.15.57:0/3316462987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7150004d20 0x7f7150005190 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7144009910 tx=0x7f7144009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.102+0000 7f7157200700 1 -- 172.21.15.57:0/3316462987 shutdown_connections 2024-02-28T23:45:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.102+0000 7f7157200700 1 --2- 172.21.15.57:0/3316462987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7150004d20 0x7f7150005190 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.103+0000 7f7157200700 1 --2- 172.21.15.57:0/3316462987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f715009a210 0x7f715009a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.103+0000 7f7157200700 1 -- 172.21.15.57:0/3316462987 >> 172.21.15.57:0/3316462987 conn(0x7f715000a090 msgr2=0x7f715000a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.103+0000 7f7157200700 1 -- 172.21.15.57:0/3316462987 shutdown_connections 2024-02-28T23:45:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.103+0000 7f7157200700 1 -- 172.21.15.57:0/3316462987 wait complete. 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 Processor -- start 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 -- start start 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 0x7f715012ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715012b8e0 con 0x7f715009a210 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7157200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715012ba20 con 0x7f7150004d20 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.104+0000 7f7155400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42408/0 (socket says 172.21.15.57:42408) 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 -- 172.21.15.57:0/1230983435 learned_addr learned my addr 172.21.15.57:0/1230983435 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155e00700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 0x7f715012ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 -- 172.21.15.57:0/1230983435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 msgr2=0x7f715012ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 0x7f715012ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 -- 172.21.15.57:0/1230983435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440095f0 con 0x7f715009a210 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155e00700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 0x7f715012ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.105+0000 7f7155400700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f714400b5c0 tx=0x7f714400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.106+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714401d070 con 0x7f715009a210 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.106+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f714400f460 con 0x7f715009a210 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.106+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7144017630 con 0x7f715009a210 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.106+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715012f810 con 0x7f715009a210 2024-02-28T23:45:40.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.106+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715012fd00 con 0x7f715009a210 2024-02-28T23:45:40.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.107+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71500044d0 con 0x7f715009a210 2024-02-28T23:45:40.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.108+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f714400f5d0 con 0x7f715009a210 2024-02-28T23:45:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.109+0000 7f714ea00700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 0x7f713c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.109+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f714408eac0 con 0x7f715009a210 2024-02-28T23:45:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.109+0000 7f7155e00700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 0x7f713c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.110+0000 7f7155e00700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 0x7f713c06fd50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7148005fd0 tx=0x7f7148005e40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.114+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7144058310 con 0x7f715009a210 2024-02-28T23:45:40.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.367+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7150130230 con 0x7f713c06d8a0 2024-02-28T23:45:40.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.370+0000 7f714ea00700 1 -- 172.21.15.57:0/1230983435 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7150130230 con 0x7f713c06d8a0 2024-02-28T23:45:40.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.372+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 msgr2=0x7f713c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.372+0000 7f7157200700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 0x7f713c06fd50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7148005fd0 tx=0x7f7148005e40 comp rx=0 tx=0).stop 2024-02-28T23:45:40.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.372+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 msgr2=0x7f715012b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.372+0000 7f7157200700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f714400b5c0 tx=0x7f714400be20 comp rx=0 tx=0).stop 2024-02-28T23:45:40.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 shutdown_connections 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f713c06d8a0 0x7f713c06fd50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7150004d20 0x7f715012ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 --2- 172.21.15.57:0/1230983435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715009a210 0x7f715012b290 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 >> 172.21.15.57:0/1230983435 conn(0x7f715000a090 msgr2=0x7f71500938c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 shutdown_connections 2024-02-28T23:45:40.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.373+0000 7f7157200700 1 -- 172.21.15.57:0/1230983435 wait complete. 2024-02-28T23:45:40.388 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:45:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 -- 172.21.15.57:0/268627613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d40730f0 msgr2=0x7f55d40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 --2- 172.21.15.57:0/268627613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d40730f0 0x7f55d40734c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009910 tx=0x7f55c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 -- 172.21.15.57:0/268627613 shutdown_connections 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 --2- 172.21.15.57:0/268627613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4073a00 0x7f55d410c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 --2- 172.21.15.57:0/268627613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d40730f0 0x7f55d40734c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.537+0000 7f55d3200700 1 -- 172.21.15.57:0/268627613 >> 172.21.15.57:0/268627613 conn(0x7f55d40fbb40 msgr2=0x7f55d40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.538+0000 7f55d3200700 1 -- 172.21.15.57:0/268627613 shutdown_connections 2024-02-28T23:45:40.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.538+0000 7f55d3200700 1 -- 172.21.15.57:0/268627613 wait complete. 2024-02-28T23:45:40.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 Processor -- start 2024-02-28T23:45:40.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 -- start start 2024-02-28T23:45:40.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 0x7f55d419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4198ad0 con 0x7f55d4073a00 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.539+0000 7f55d3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4198c40 con 0x7f55d41984c0 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42432/0 (socket says 172.21.15.57:42432) 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 -- 172.21.15.57:0/3350368810 learned_addr learned my addr 172.21.15.57:0/3350368810 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1400700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 0x7f55d419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 -- 172.21.15.57:0/3350368810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 msgr2=0x7f55d419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 0x7f55d419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 -- 172.21.15.57:0/3350368810 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40095f0 con 0x7f55d4073a00 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1400700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 0x7f55d419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:45:40.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.540+0000 7f55d1e00700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f55c400bd70 tx=0x7f55c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.541+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55c401d070 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.541+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55c400f460 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.541+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d419cee0 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.541+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55c4017630 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.541+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d419d430 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.542+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d4109a40 con 0x7f55d4073a00 2024-02-28T23:45:40.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.543+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f55c400fab0 con 0x7f55d4073a00 2024-02-28T23:45:40.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.544+0000 7f55caa00700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 0x7f55b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.544+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f55c408e900 con 0x7f55d4073a00 2024-02-28T23:45:40.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.544+0000 7f55d1400700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 0x7f55b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.545+0000 7f55d1400700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 0x7f55b806fd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005fd0 tx=0x7f55c0005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.548+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55c4058090 con 0x7f55d4073a00 2024-02-28T23:45:40.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.797+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55d4199290 con 0x7f55b806d8a0 2024-02-28T23:45:40.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.799+0000 7f55caa00700 1 -- 172.21.15.57:0/3350368810 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f55d4199290 con 0x7f55b806d8a0 2024-02-28T23:45:40.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 msgr2=0x7f55b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 0x7f55b806fd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005fd0 tx=0x7f55c0005e20 comp rx=0 tx=0).stop 2024-02-28T23:45:40.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 msgr2=0x7f55d4197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f55c400bd70 tx=0x7f55c400be50 comp rx=0 tx=0).stop 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 shutdown_connections 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f55b806d8a0 0x7f55b806fd50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d4073a00 0x7f55d4197f80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 --2- 172.21.15.57:0/3350368810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d41984c0 0x7f55d419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 >> 172.21.15.57:0/3350368810 conn(0x7f55d40fbb40 msgr2=0x7f55d4106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.802+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 shutdown_connections 2024-02-28T23:45:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.803+0000 7f55d3200700 1 -- 172.21.15.57:0/3350368810 wait complete. 2024-02-28T23:45:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 -- 172.21.15.57:0/1344974975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 msgr2=0x7efc600734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 --2- 172.21.15.57:0/1344974975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc600734c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7efc50009910 tx=0x7efc50009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 -- 172.21.15.57:0/1344974975 shutdown_connections 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 --2- 172.21.15.57:0/1344974975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 0x7efc6010c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 --2- 172.21.15.57:0/1344974975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc600734c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 -- 172.21.15.57:0/1344974975 >> 172.21.15.57:0/1344974975 conn(0x7efc600fbb60 msgr2=0x7efc600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 -- 172.21.15.57:0/1344974975 shutdown_connections 2024-02-28T23:45:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.968+0000 7efc65200700 1 -- 172.21.15.57:0/1344974975 wait complete. 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.969+0000 7efc65200700 1 Processor -- start 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.969+0000 7efc65200700 1 -- start start 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc65200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc65200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 0x7efc601983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc65200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6019c6f0 con 0x7efc600730f0 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc65200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6019c860 con 0x7efc60073a00 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc5f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc5f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42436/0 (socket says 172.21.15.57:42436) 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc5f400700 1 -- 172.21.15.57:0/3807657168 learned_addr learned my addr 172.21.15.57:0/3807657168 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.970+0000 7efc57e00700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 0x7efc601983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5f400700 1 -- 172.21.15.57:0/3807657168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 msgr2=0x7efc601983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5f400700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 0x7efc601983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5f400700 1 -- 172.21.15.57:0/3807657168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500095f0 con 0x7efc600730f0 2024-02-28T23:45:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5f400700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b5c0 tx=0x7efc5000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc5000f460 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc50017630 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.971+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6019cb40 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.972+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6019d090 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.973+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60109a60 con 0x7efc600730f0 2024-02-28T23:45:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.973+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efc5000f5d0 con 0x7efc600730f0 2024-02-28T23:45:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.974+0000 7efc5cc00700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 0x7efc4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.975+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc5008eab0 con 0x7efc600730f0 2024-02-28T23:45:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.975+0000 7efc57e00700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 0x7efc4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.975+0000 7efc57e00700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 0x7efc4406fd50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc4800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:40.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:40.979+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc50090e60 con 0x7efc600730f0 2024-02-28T23:45:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:40 smithi057 ceph-mon[89491]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:41.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.222+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efc60199140 con 0x7efc4406d8a0 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.235+0000 7efc5cc00700 1 -- 172.21.15.57:0/3807657168 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efc60199140 con 0x7efc4406d8a0 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (35m) 11m ago 49m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (45m) 11m ago 49m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (48m) 11m ago 48m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (31m) 11m ago 49m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:45:41.235 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (31m) 11m ago 47m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (34m) 11m ago 48m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (27m) 11m ago 41m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (26m) 11m ago 41m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (11m) 11m ago 41m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (26m) 11m ago 41m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (37m) 11m ago 50m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (36m) 11m ago 47m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (33m) 11m ago 51m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (32m) 11m ago 47m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (36m) 11m ago 49m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (36m) 11m ago 47m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (30m) 11m ago 46m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (30m) 11m ago 46m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (29m) 11m ago 45m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:45:41.236 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (29m) 11m ago 44m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:45:41.237 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (28m) 11m ago 44m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:45:41.237 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (27m) 11m ago 43m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:45:41.237 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (35m) 11m ago 48m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:45:41.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 msgr2=0x7efc4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 0x7efc4406fd50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc4800a560 comp rx=0 tx=0).stop 2024-02-28T23:45:41.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 msgr2=0x7efc60197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b5c0 tx=0x7efc5000be00 comp rx=0 tx=0).stop 2024-02-28T23:45:41.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 shutdown_connections 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc4406d8a0 0x7efc4406fd50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc600730f0 0x7efc60197d40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 --2- 172.21.15.57:0/3807657168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc60073a00 0x7efc601983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 >> 172.21.15.57:0/3807657168 conn(0x7efc600fbb60 msgr2=0x7efc60106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 shutdown_connections 2024-02-28T23:45:41.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.238+0000 7efc65200700 1 -- 172.21.15.57:0/3807657168 wait complete. 2024-02-28T23:45:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:40 smithi084 ceph-mon[70440]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:41.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 -- 172.21.15.57:0/163110816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298106320 msgr2=0x7f72981066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 --2- 172.21.15.57:0/163110816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298106320 0x7f72981066f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7288009910 tx=0x7f7288009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:41.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 -- 172.21.15.57:0/163110816 shutdown_connections 2024-02-28T23:45:41.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 --2- 172.21.15.57:0/163110816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298068540 0x7f72980689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 --2- 172.21.15.57:0/163110816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298106320 0x7f72981066f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 -- 172.21.15.57:0/163110816 >> 172.21.15.57:0/163110816 conn(0x7f7298075130 msgr2=0x7f7298075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:41.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 -- 172.21.15.57:0/163110816 shutdown_connections 2024-02-28T23:45:41.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.403+0000 7f729ce00700 1 -- 172.21.15.57:0/163110816 wait complete. 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.404+0000 7f729ce00700 1 Processor -- start 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.404+0000 7f729ce00700 1 -- start start 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f729ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f729ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298106320 0x7f72981985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f729ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298198c80 con 0x7f7298068540 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f729ce00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729819ca10 con 0x7f7298106320 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f7295e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f7295e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42458/0 (socket says 172.21.15.57:42458) 2024-02-28T23:45:41.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f7295e00700 1 -- 172.21.15.57:0/356405279 learned_addr learned my addr 172.21.15.57:0/356405279 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.405+0000 7f728fe00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298106320 0x7f72981985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f7295e00700 1 -- 172.21.15.57:0/356405279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298106320 msgr2=0x7f72981985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f7295e00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298106320 0x7f72981985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f7295e00700 1 -- 172.21.15.57:0/356405279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880095f0 con 0x7f7298068540 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f7295e00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f728800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728801d070 con 0x7f7298068540 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f728800f460 con 0x7f7298068540 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729819cc90 con 0x7f7298068540 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7288017630 con 0x7f7298068540 2024-02-28T23:45:41.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.406+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729819d180 con 0x7f7298068540 2024-02-28T23:45:41.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.407+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729804f2a0 con 0x7f7298068540 2024-02-28T23:45:41.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.409+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f728800f5d0 con 0x7f7298068540 2024-02-28T23:45:41.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.410+0000 7f728ea00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 0x7f727c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.410+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f728808eab0 con 0x7f7298068540 2024-02-28T23:45:41.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.410+0000 7f728fe00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 0x7f727c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.411+0000 7f728fe00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 0x7f727c06fd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7280005950 tx=0x7f72800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:41.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.415+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7288090e60 con 0x7f7298068540 2024-02-28T23:45:41.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.747+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f729819d4e0 con 0x7f7298068540 2024-02-28T23:45:41.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.749+0000 7f728ea00700 1 -- 172.21.15.57:0/356405279 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f72880581c0 con 0x7f7298068540 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:45:41.749 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:45:41.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:45:41.750 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:45:41.750 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:45:41.750 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:45:41.750 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:45:41.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 msgr2=0x7f727c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 0x7f727c06fd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7280005950 tx=0x7f72800058e0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 msgr2=0x7f7298198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f728800be20 comp rx=0 tx=0).stop 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 shutdown_connections 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f727c06d8a0 0x7f727c06fd50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7298068540 0x7f7298198060 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 --2- 172.21.15.57:0/356405279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7298106320 0x7f72981985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 >> 172.21.15.57:0/356405279 conn(0x7f7298075130 msgr2=0x7f72980fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 shutdown_connections 2024-02-28T23:45:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.752+0000 7f729ce00700 1 -- 172.21.15.57:0/356405279 wait complete. 2024-02-28T23:45:41.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 -- 172.21.15.57:0/140006198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c069140 msgr2=0x7fbd4c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 --2- 172.21.15.57:0/140006198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c069140 0x7fbd4c104eb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009910 tx=0x7fbd38009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:41.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 -- 172.21.15.57:0/140006198 shutdown_connections 2024-02-28T23:45:41.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 --2- 172.21.15.57:0/140006198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c069140 0x7fbd4c104eb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 --2- 172.21.15.57:0/140006198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 -- 172.21.15.57:0/140006198 >> 172.21.15.57:0/140006198 conn(0x7fbd4c075130 msgr2=0x7fbd4c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:41.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 -- 172.21.15.57:0/140006198 shutdown_connections 2024-02-28T23:45:41.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.917+0000 7fbd54400700 1 -- 172.21.15.57:0/140006198 wait complete. 2024-02-28T23:45:41.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.918+0000 7fbd54400700 1 Processor -- start 2024-02-28T23:45:41.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.918+0000 7fbd54400700 1 -- start start 2024-02-28T23:45:41.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd54400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd54400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c069140 0x7fbd4c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd54400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c198c80 con 0x7fbd4c0687a0 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd54400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c19ca10 con 0x7fbd4c069140 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd51a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd51a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42476/0 (socket says 172.21.15.57:42476) 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd51a00700 1 -- 172.21.15.57:0/1654887639 learned_addr learned my addr 172.21.15.57:0/1654887639 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd51000700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c069140 0x7fbd4c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.919+0000 7fbd51a00700 1 -- 172.21.15.57:0/1654887639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c069140 msgr2=0x7fbd4c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd51a00700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c069140 0x7fbd4c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:41.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd51a00700 1 -- 172.21.15.57:0/1654887639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd380095f0 con 0x7fbd4c0687a0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd51a00700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d900 tx=0x7fbd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd40004a00 con 0x7fbd4c0687a0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd400054e0 con 0x7fbd4c0687a0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c19ccf0 con 0x7fbd4c0687a0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd40009d60 con 0x7fbd4c0687a0 2024-02-28T23:45:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.920+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c19d240 con 0x7fbd4c0687a0 2024-02-28T23:45:41.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.922+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c04f2a0 con 0x7fbd4c0687a0 2024-02-28T23:45:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.923+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbd40005020 con 0x7fbd4c0687a0 2024-02-28T23:45:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.924+0000 7fbd4a000700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 0x7fbd3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.924+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd4008d800 con 0x7fbd4c0687a0 2024-02-28T23:45:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.924+0000 7fbd51000700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 0x7fbd3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:41.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.925+0000 7fbd51000700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 0x7fbd3406fd50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:41.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:41.929+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd40056f10 con 0x7fbd4c0687a0 2024-02-28T23:45:42.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:41 smithi057 ceph-mon[89491]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:42.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:41 smithi057 ceph-mon[89491]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:42.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:41 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/356405279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:45:42.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.215+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd4c19d5a0 con 0x7fbd4c0687a0 2024-02-28T23:45:42.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.217+0000 7fbd4a000700 1 -- 172.21.15.57:0/1654887639 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fbd4005a5d0 con 0x7fbd4c0687a0 2024-02-28T23:45:42.218 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:45:42.219 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:45:42.220 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:42.221 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 msgr2=0x7fbd3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 0x7fbd3406fd50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38005fd0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 msgr2=0x7fbd4c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d900 tx=0x7fbd4000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 shutdown_connections 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbd3406d8a0 0x7fbd3406fd50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd4c0687a0 0x7fbd4c198060 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 --2- 172.21.15.57:0/1654887639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd4c069140 0x7fbd4c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.220+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 >> 172.21.15.57:0/1654887639 conn(0x7fbd4c075130 msgr2=0x7fbd4c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.221+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 shutdown_connections 2024-02-28T23:45:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.221+0000 7fbd54400700 1 -- 172.21.15.57:0/1654887639 wait complete. 2024-02-28T23:45:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:45:42.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:41 smithi084 ceph-mon[70440]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:42.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:41 smithi084 ceph-mon[70440]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:42.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:41 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/356405279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 -- 172.21.15.57:0/2336159754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 msgr2=0x7faaa8073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 --2- 172.21.15.57:0/2336159754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8073460 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009940 tx=0x7faa9c009c50 comp rx=0 tx=0).stop 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 -- 172.21.15.57:0/2336159754 shutdown_connections 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 --2- 172.21.15.57:0/2336159754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 0x7faaa810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 --2- 172.21.15.57:0/2336159754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8073460 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.379+0000 7faaafc00700 1 -- 172.21.15.57:0/2336159754 >> 172.21.15.57:0/2336159754 conn(0x7faaa80fbda0 msgr2=0x7faaa80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.380+0000 7faaafc00700 1 -- 172.21.15.57:0/2336159754 shutdown_connections 2024-02-28T23:45:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.380+0000 7faaafc00700 1 -- 172.21.15.57:0/2336159754 wait complete. 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 Processor -- start 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 -- start start 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 0x7faaa81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8198d10 con 0x7faaa8073090 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaafc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8198e50 con 0x7faaa80739a0 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.381+0000 7faaad200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42504/0 (socket says 172.21.15.57:42504) 2024-02-28T23:45:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 -- 172.21.15.57:0/2333472124 learned_addr learned my addr 172.21.15.57:0/2333472124 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 -- 172.21.15.57:0/2333472124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 msgr2=0x7faaa81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaa7e00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 0x7faaa81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 0x7faaa81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 -- 172.21.15.57:0/2333472124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c0095f0 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.382+0000 7faaad200700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0055f0 tx=0x7faa9c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.383+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c01d070 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.383+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa9c00f460 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.383+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa819cc40 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.383+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c021640 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.383+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa819d130 con 0x7faaa8073090 2024-02-28T23:45:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.384+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa804f2a0 con 0x7faaa8073090 2024-02-28T23:45:42.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.385+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7faa9c00f5d0 con 0x7faaa8073090 2024-02-28T23:45:42.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.386+0000 7faaa5600700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 0x7faa9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.386+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa9c08e910 con 0x7faaa8073090 2024-02-28T23:45:42.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.387+0000 7faaa7e00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 0x7faa9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.387+0000 7faaa7e00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 0x7faa9006fd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faaa81997a0 tx=0x7faa94005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:42.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.391+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa9c058020 con 0x7faaa8073090 2024-02-28T23:45:42.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.644+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaa81097d0 con 0x7faa9006d8a0 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.647+0000 7faaa5600700 1 -- 172.21.15.57:0/2333472124 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faaa81097d0 con 0x7faa9006d8a0 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:45:42.647 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:45:42.648 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.649+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 msgr2=0x7faa9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.649+0000 7faaafc00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 0x7faa9006fd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faaa81997a0 tx=0x7faa94005d50 comp rx=0 tx=0).stop 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.649+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 msgr2=0x7faaa8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.649+0000 7faaafc00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0055f0 tx=0x7faa9c0056d0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 shutdown_connections 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faa9006d8a0 0x7faa9006fd50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaa8073090 0x7faaa8198180 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 --2- 172.21.15.57:0/2333472124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa80739a0 0x7faaa81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 >> 172.21.15.57:0/2333472124 conn(0x7faaa80fbda0 msgr2=0x7faaa8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 shutdown_connections 2024-02-28T23:45:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.650+0000 7faaafc00700 1 -- 172.21.15.57:0/2333472124 wait complete. 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 -- 172.21.15.57:0/228026145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b01004e0 msgr2=0x7fe8b01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/228026145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b01004e0 0x7fe8b01008b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe898009910 tx=0x7fe898009c20 comp rx=0 tx=0).stop 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 -- 172.21.15.57:0/228026145 shutdown_connections 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/228026145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b0100e80 0x7fe8b0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/228026145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b01004e0 0x7fe8b01008b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 -- 172.21.15.57:0/228026145 >> 172.21.15.57:0/228026145 conn(0x7fe8b00fbda0 msgr2=0x7fe8b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 -- 172.21.15.57:0/228026145 shutdown_connections 2024-02-28T23:45:42.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.817+0000 7fe8b4a00700 1 -- 172.21.15.57:0/228026145 wait complete. 2024-02-28T23:45:42.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.818+0000 7fe8b4a00700 1 Processor -- start 2024-02-28T23:45:42.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.818+0000 7fe8b4a00700 1 -- start start 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8b4a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b01004e0 0x7fe8b01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8b4a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8b4a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0198cf0 con 0x7fe8b0100e80 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8b4a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019ca80 con 0x7fe8b01004e0 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8a5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8a5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42514/0 (socket says 172.21.15.57:42514) 2024-02-28T23:45:42.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8a5400700 1 -- 172.21.15.57:0/966277076 learned_addr learned my addr 172.21.15.57:0/966277076 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:45:42.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8ade00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b01004e0 0x7fe8b01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8a5400700 1 -- 172.21.15.57:0/966277076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b01004e0 msgr2=0x7fe8b01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:42.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.819+0000 7fe8a5400700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b01004e0 0x7fe8b01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8a5400700 1 -- 172.21.15.57:0/966277076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980095f0 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8a5400700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c8f0 tx=0x7fe8a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a0007a50 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8a0007bb0 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b019cd60 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a0018630 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.820+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b019d2b0 con 0x7fe8b0100e80 2024-02-28T23:45:42.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.821+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b01045d0 con 0x7fe8b0100e80 2024-02-28T23:45:42.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.823+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe8a0007d20 con 0x7fe8b0100e80 2024-02-28T23:45:42.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.823+0000 7fe8a7400700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 0x7fe89406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:45:42.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.824+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe8a008d5a0 con 0x7fe8b0100e80 2024-02-28T23:45:42.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.824+0000 7fe8ade00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 0x7fe89406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:45:42.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.825+0000 7fe8ade00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 0x7fe89406fd50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe89801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:45:42.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:42.828+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8a0056cb0 con 0x7fe8b0100e80 2024-02-28T23:45:43.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.154+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8b004f2a0 con 0x7fe8b0100e80 2024-02-28T23:45:43.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.155+0000 7fe8a7400700 1 -- 172.21.15.57:0/966277076 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fe8a005a370 con 0x7fe8b0100e80 2024-02-28T23:45:43.155 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:45:43.155 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:45:43.155 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:45:43.155 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:45:43.156 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:45:43.157 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.157+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 msgr2=0x7fe89406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.157+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 0x7fe89406fd50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe89801a040 comp rx=0 tx=0).stop 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.157+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 msgr2=0x7fe8b0198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.157+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c8f0 tx=0x7fe8a000ccb0 comp rx=0 tx=0).stop 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 shutdown_connections 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe89406d8a0 0x7fe89406fd50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b01004e0 0x7fe8b01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 --2- 172.21.15.57:0/966277076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8b0100e80 0x7fe8b0198610 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:45:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 >> 172.21.15.57:0/966277076 conn(0x7fe8b00fbda0 msgr2=0x7fe8b00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:45:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 shutdown_connections 2024-02-28T23:45:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:45:43.158+0000 7fe8b4a00700 1 -- 172.21.15.57:0/966277076 wait complete. 2024-02-28T23:45:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:42 smithi057 ceph-mon[89491]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:42 smithi057 ceph-mon[89491]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:42 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1654887639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:45:43.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:42 smithi084 ceph-mon[70440]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:43.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:42 smithi084 ceph-mon[70440]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:43.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:42 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1654887639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:45:44.238 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:43 smithi084 ceph-mon[70440]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:44.238 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:43 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/966277076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:45:44.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:43 smithi057 ceph-mon[89491]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:45:44.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:43 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/966277076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:45:45.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:44 smithi057 ceph-mon[89491]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:44 smithi084 ceph-mon[70440]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:46 smithi057 ceph-mon[89491]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:46 smithi084 ceph-mon[70440]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:49 smithi057 ceph-mon[89491]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:49 smithi084 ceph-mon[70440]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:45:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:51 smithi057 ceph-mon[89491]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:51 smithi084 ceph-mon[70440]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:53.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:53 smithi057 ceph-mon[89491]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:53 smithi084 ceph-mon[70440]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:55 smithi057 ceph-mon[89491]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:55 smithi084 ceph-mon[70440]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:57 smithi057 ceph-mon[89491]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:57 smithi084 ceph-mon[70440]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:59.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:45:59 smithi057 ceph-mon[89491]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:45:59.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:45:59 smithi084 ceph-mon[70440]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:01.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:01 smithi057 ceph-mon[89491]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:01 smithi084 ceph-mon[70440]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:03 smithi057 ceph-mon[89491]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:03 smithi084 ceph-mon[70440]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:05 smithi057 ceph-mon[89491]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:05 smithi084 ceph-mon[70440]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:07 smithi084 ceph-mon[70440]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:07.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:07 smithi057 ceph-mon[89491]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:09.438 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:09 smithi057 ceph-mon[89491]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:09.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:09 smithi084 ceph-mon[70440]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:11 smithi057 ceph-mon[89491]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:11.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:11 smithi084 ceph-mon[70440]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:13.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 -- 172.21.15.57:0/2066598156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 msgr2=0x7f118010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 --2- 172.21.15.57:0/2066598156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118010ccc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1170009910 tx=0x7f1170009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:13.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 -- 172.21.15.57:0/2066598156 shutdown_connections 2024-02-28T23:46:13.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 --2- 172.21.15.57:0/2066598156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 0x7f1180106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 --2- 172.21.15.57:0/2066598156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118010ccc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 -- 172.21.15.57:0/2066598156 >> 172.21.15.57:0/2066598156 conn(0x7f1180076060 msgr2=0x7f1180078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:13.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 -- 172.21.15.57:0/2066598156 shutdown_connections 2024-02-28T23:46:13.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.339+0000 7f1185000700 1 -- 172.21.15.57:0/2066598156 wait complete. 2024-02-28T23:46:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.340+0000 7f1185000700 1 Processor -- start 2024-02-28T23:46:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.340+0000 7f1185000700 1 -- start start 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1185000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 0x7f118019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1185000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1185000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118019d120 con 0x7f118010c8f0 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1185000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801a0eb0 con 0x7f1180106b80 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1177e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1177e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44984/0 (socket says 172.21.15.57:44984) 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f1177e00700 1 -- 172.21.15.57:0/4248602289 learned_addr learned my addr 172.21.15.57:0/4248602289 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.341+0000 7f117de00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 0x7f118019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1177e00700 1 -- 172.21.15.57:0/4248602289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 msgr2=0x7f118019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1177e00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 0x7f118019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1177e00700 1 -- 172.21.15.57:0/4248602289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700095f0 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1177e00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f116800d8d0 tx=0x7f116800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1168009940 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1168010460 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116800f5f0 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.342+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11801a1190 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.343+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11801a16e0 con 0x7f118010c8f0 2024-02-28T23:46:13.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.344+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1180109b20 con 0x7f118010c8f0 2024-02-28T23:46:13.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.345+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f11680105d0 con 0x7f118010c8f0 2024-02-28T23:46:13.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.346+0000 7f1176a00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 0x7f116406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.346+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f116808d4c0 con 0x7f118010c8f0 2024-02-28T23:46:13.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.346+0000 7f117de00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 0x7f116406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.347+0000 7f117de00700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 0x7f116406fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1170006010 tx=0x7f117001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:13.346 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:13 smithi057 ceph-mon[89491]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:13.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.351+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1168056c50 con 0x7f118010c8f0 2024-02-28T23:46:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:13 smithi084 ceph-mon[70440]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:13.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.600+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11801a1950 con 0x7f116406d8a0 2024-02-28T23:46:13.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.602+0000 7f1176a00700 1 -- 172.21.15.57:0/4248602289 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f11801a1950 con 0x7f116406d8a0 2024-02-28T23:46:13.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 msgr2=0x7f116406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 0x7f116406fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1170006010 tx=0x7f117001a040 comp rx=0 tx=0).stop 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 msgr2=0x7f118019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f116800d8d0 tx=0x7f116800dc90 comp rx=0 tx=0).stop 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 shutdown_connections 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f116406d8a0 0x7f116406fd50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1180106b80 0x7f118019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 --2- 172.21.15.57:0/4248602289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f118010c8f0 0x7f118019ca40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 >> 172.21.15.57:0/4248602289 conn(0x7f1180076060 msgr2=0x7f1180077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 shutdown_connections 2024-02-28T23:46:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.605+0000 7f1185000700 1 -- 172.21.15.57:0/4248602289 wait complete. 2024-02-28T23:46:13.621 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:46:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 -- 172.21.15.57:0/537422966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 msgr2=0x7f8f6c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 --2- 172.21.15.57:0/537422966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c106ff0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b0a0 tx=0x7f8f5800b3b0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 -- 172.21.15.57:0/537422966 shutdown_connections 2024-02-28T23:46:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 --2- 172.21.15.57:0/537422966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c106ff0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 --2- 172.21.15.57:0/537422966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 0x7f8f6c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 -- 172.21.15.57:0/537422966 >> 172.21.15.57:0/537422966 conn(0x7f8f6c076060 msgr2=0x7f8f6c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 -- 172.21.15.57:0/537422966 shutdown_connections 2024-02-28T23:46:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.767+0000 7f8f73c00700 1 -- 172.21.15.57:0/537422966 wait complete. 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.768+0000 7f8f73c00700 1 Processor -- start 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.768+0000 7f8f73c00700 1 -- start start 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f73c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f73c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 0x7f8f6c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f73c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c19d0c0 con 0x7f8f6c106b80 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f73c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1a0e50 con 0x7f8f6c10c8f0 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f71200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f71200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44996/0 (socket says 172.21.15.57:44996) 2024-02-28T23:46:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f71200700 1 -- 172.21.15.57:0/3390860561 learned_addr learned my addr 172.21.15.57:0/3390860561 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.769+0000 7f8f6be00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 0x7f8f6c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f71200700 1 -- 172.21.15.57:0/3390860561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 msgr2=0x7f8f6c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f71200700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 0x7f8f6c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f71200700 1 -- 172.21.15.57:0/3390860561 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009d00 con 0x7f8f6c106b80 2024-02-28T23:46:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f71200700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000d8d0 tx=0x7f8f6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f60009940 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f60010460 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6000f5f0 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c1a1130 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.770+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c1a1680 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.771+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c10ecc0 con 0x7f8f6c106b80 2024-02-28T23:46:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.773+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8f600105d0 con 0x7f8f6c106b80 2024-02-28T23:46:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.774+0000 7f8f69600700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 0x7f8f5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.774+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8f6008d5d0 con 0x7f8f6c106b80 2024-02-28T23:46:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.774+0000 7f8f6be00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 0x7f8f5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:13.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.775+0000 7f8f6be00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 0x7f8f5406fd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c19dac0 tx=0x7f8f5800b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:13.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:13.779+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f60056ce0 con 0x7f8f6c106b80 2024-02-28T23:46:14.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.030+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f6c1a18a0 con 0x7f8f5406d8a0 2024-02-28T23:46:14.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.033+0000 7f8f69600700 1 -- 172.21.15.57:0/3390860561 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8f6c1a18a0 con 0x7f8f5406d8a0 2024-02-28T23:46:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 msgr2=0x7f8f5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 0x7f8f5406fd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c19dac0 tx=0x7f8f5800b7f0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 msgr2=0x7f8f6c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000d8d0 tx=0x7f8f6000dc90 comp rx=0 tx=0).stop 2024-02-28T23:46:14.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 shutdown_connections 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8f5406d8a0 0x7f8f5406fd50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19c4a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 --2- 172.21.15.57:0/3390860561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f6c10c8f0 0x7f8f6c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 >> 172.21.15.57:0/3390860561 conn(0x7f8f6c076060 msgr2=0x7f8f6c077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 shutdown_connections 2024-02-28T23:46:14.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.036+0000 7f8f73c00700 1 -- 172.21.15.57:0/3390860561 wait complete. 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.200+0000 7f6121c00700 1 -- 172.21.15.57:0/3612919920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c10c8f0 msgr2=0x7f611c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.200+0000 7f6121c00700 1 --2- 172.21.15.57:0/3612919920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c10c8f0 0x7f611c10ccc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f610c009910 tx=0x7f610c009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 -- 172.21.15.57:0/3612919920 shutdown_connections 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 --2- 172.21.15.57:0/3612919920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c106b80 0x7f611c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 --2- 172.21.15.57:0/3612919920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c10c8f0 0x7f611c10ccc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 -- 172.21.15.57:0/3612919920 >> 172.21.15.57:0/3612919920 conn(0x7f611c076060 msgr2=0x7f611c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 -- 172.21.15.57:0/3612919920 shutdown_connections 2024-02-28T23:46:14.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.201+0000 7f6121c00700 1 -- 172.21.15.57:0/3612919920 wait complete. 2024-02-28T23:46:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 Processor -- start 2024-02-28T23:46:14.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 -- start start 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 0x7f611c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c19d090 con 0x7f611c106b80 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f6121c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c19d1d0 con 0x7f611c10c8f0 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.202+0000 7f611aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45008/0 (socket says 172.21.15.57:45008) 2024-02-28T23:46:14.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 -- 172.21.15.57:0/4042367047 learned_addr learned my addr 172.21.15.57:0/4042367047 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:14.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611a000700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 0x7f611c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 -- 172.21.15.57:0/4042367047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 msgr2=0x7f611c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 0x7f611c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 -- 172.21.15.57:0/4042367047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c0095f0 con 0x7f611c106b80 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611a000700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 0x7f611c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.203+0000 7f611aa00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f610c00b5c0 tx=0x7f610c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.204+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c01d070 con 0x7f611c106b80 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.204+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f610c00f460 con 0x7f611c106b80 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.204+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c1a0fc0 con 0x7f611c106b80 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.204+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c017610 con 0x7f611c106b80 2024-02-28T23:46:14.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.204+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c1a14b0 con 0x7f611c106b80 2024-02-28T23:46:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.205+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c109b20 con 0x7f611c106b80 2024-02-28T23:46:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.206+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f610c00f5d0 con 0x7f611c106b80 2024-02-28T23:46:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.207+0000 7f6113400700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 0x7f610006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.208+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f610c08ebe0 con 0x7f611c106b80 2024-02-28T23:46:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.208+0000 7f611a000700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 0x7f610006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.209+0000 7f611a000700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 0x7f610006fd50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6108005fd0 tx=0x7f6108005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:14.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.213+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f610c0582f0 con 0x7f611c106b80 2024-02-28T23:46:14.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.454+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f611c1a1790 con 0x7f610006d8a0 2024-02-28T23:46:14.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.473+0000 7f6113400700 1 -- 172.21.15.57:0/4042367047 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f611c1a1790 con 0x7f610006d8a0 2024-02-28T23:46:14.472 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (35m) 11m ago 49m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (46m) 11m ago 50m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (48m) 11m ago 48m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (31m) 11m ago 50m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (31m) 11m ago 48m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (35m) 11m ago 49m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:46:14.473 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (27m) 11m ago 42m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (27m) 11m ago 42m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (11m) 11m ago 42m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (26m) 11m ago 42m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (38m) 11m ago 51m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (37m) 11m ago 48m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (33m) 11m ago 51m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (32m) 11m ago 48m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:46:14.474 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (36m) 11m ago 49m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (36m) 11m ago 48m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (31m) 11m ago 47m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (30m) 11m ago 46m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (30m) 11m ago 45m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (29m) 11m ago 45m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (28m) 11m ago 44m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (28m) 11m ago 44m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:46:14.475 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (36m) 11m ago 49m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 msgr2=0x7f610006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 0x7f610006fd50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6108005fd0 tx=0x7f6108005e20 comp rx=0 tx=0).stop 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 msgr2=0x7f611c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f610c00b5c0 tx=0x7f610c00be00 comp rx=0 tx=0).stop 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 shutdown_connections 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f610006d8a0 0x7f610006fd50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f611c106b80 0x7f611c19c500 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 --2- 172.21.15.57:0/4042367047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611c10c8f0 0x7f611c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 >> 172.21.15.57:0/4042367047 conn(0x7f611c076060 msgr2=0x7f611c077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:14.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.476+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 shutdown_connections 2024-02-28T23:46:14.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.477+0000 7f6121c00700 1 -- 172.21.15.57:0/4042367047 wait complete. 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.643+0000 7f4cf6c00700 1 -- 172.21.15.57:0/1435720335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0106320 msgr2=0x7f4cf01066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.643+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/1435720335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0106320 0x7f4cf01066f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009910 tx=0x7f4ce0009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 -- 172.21.15.57:0/1435720335 shutdown_connections 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/1435720335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0068540 0x7f4cf00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/1435720335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0106320 0x7f4cf01066f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 -- 172.21.15.57:0/1435720335 >> 172.21.15.57:0/1435720335 conn(0x7f4cf0075130 msgr2=0x7f4cf0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 -- 172.21.15.57:0/1435720335 shutdown_connections 2024-02-28T23:46:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.644+0000 7f4cf6c00700 1 -- 172.21.15.57:0/1435720335 wait complete. 2024-02-28T23:46:14.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.645+0000 7f4cf6c00700 1 Processor -- start 2024-02-28T23:46:14.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.645+0000 7f4cf6c00700 1 -- start start 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cf6c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cf6c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0106320 0x7f4cf01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cf6c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0198bf0 con 0x7f4cf0068540 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cf6c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0198d30 con 0x7f4cf0106320 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45030/0 (socket says 172.21.15.57:45030) 2024-02-28T23:46:14.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 -- 172.21.15.57:0/3306756404 learned_addr learned my addr 172.21.15.57:0/3306756404 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:14.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cef400700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0106320 0x7f4cf01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 -- 172.21.15.57:0/3306756404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0106320 msgr2=0x7f4cf01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0106320 0x7f4cf01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.646+0000 7f4cefe00700 1 -- 172.21.15.57:0/3306756404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00095f0 con 0x7f4cf0068540 2024-02-28T23:46:14.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cefe00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b5c0 tx=0x7f4ce000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce001d070 con 0x7f4cf0068540 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf019cb20 con 0x7f4cf0068540 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce000f460 con 0x7f4cf0068540 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce0017610 con 0x7f4cf0068540 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.647+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf019d010 con 0x7f4cf0068540 2024-02-28T23:46:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.649+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf004f2a0 con 0x7f4cf0068540 2024-02-28T23:46:14.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.650+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4ce000f5d0 con 0x7f4cf0068540 2024-02-28T23:46:14.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.651+0000 7f4cecc00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 0x7f4cd406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:14.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.651+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4ce008eb30 con 0x7f4cf0068540 2024-02-28T23:46:14.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.651+0000 7f4cef400700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 0x7f4cd406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:14.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.652+0000 7f4cef400700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 0x7f4cd406fd50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005ea0 tx=0x7f4cdc005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:14.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.657+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ce00582c0 con 0x7f4cf0068540 2024-02-28T23:46:14.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.977+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4cf01993c0 con 0x7f4cf0068540 2024-02-28T23:46:14.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.978+0000 7f4cecc00700 1 -- 172.21.15.57:0/3306756404 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4ce005b980 con 0x7f4cf0068540 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:46:14.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:14.979 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:46:14.979 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:46:14.979 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:46:14.979 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.981+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 msgr2=0x7f4cd406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.981+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 0x7f4cd406fd50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005ea0 tx=0x7f4cdc005e30 comp rx=0 tx=0).stop 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.981+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 msgr2=0x7f4cf0198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.981+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b5c0 tx=0x7f4ce000be00 comp rx=0 tx=0).stop 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 shutdown_connections 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4cd406d8a0 0x7f4cd406fd50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cf0068540 0x7f4cf0198060 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 --2- 172.21.15.57:0/3306756404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cf0106320 0x7f4cf01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 >> 172.21.15.57:0/3306756404 conn(0x7f4cf0075130 msgr2=0x7f4cf00fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:14.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 shutdown_connections 2024-02-28T23:46:14.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:14.982+0000 7f4cf6c00700 1 -- 172.21.15.57:0/3306756404 wait complete. 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 -- 172.21.15.57:0/2966167503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c0ffbd0 msgr2=0x7f1f3c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 --2- 172.21.15.57:0/2966167503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c100040 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009910 tx=0x7f1f24009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 -- 172.21.15.57:0/2966167503 shutdown_connections 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 --2- 172.21.15.57:0/2966167503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c100040 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 --2- 172.21.15.57:0/2966167503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c104130 0x7f1f3c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 -- 172.21.15.57:0/2966167503 >> 172.21.15.57:0/2966167503 conn(0x7f1f3c075130 msgr2=0x7f1f3c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:15.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 -- 172.21.15.57:0/2966167503 shutdown_connections 2024-02-28T23:46:15.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.147+0000 7f1f41600700 1 -- 172.21.15.57:0/2966167503 wait complete. 2024-02-28T23:46:15.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.148+0000 7f1f41600700 1 Processor -- start 2024-02-28T23:46:15.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.148+0000 7f1f41600700 1 -- start start 2024-02-28T23:46:15.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f41600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c10a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f41600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f41600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c10aab0 con 0x7f1f3c104130 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f41600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c10abf0 con 0x7f1f3c0ffbd0 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f32000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f32000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40066/0 (socket says 172.21.15.57:40066) 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f32000700 1 -- 172.21.15.57:0/3037878793 learned_addr learned my addr 172.21.15.57:0/3037878793 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.149+0000 7f1f3aa00700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c10a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f32000700 1 -- 172.21.15.57:0/3037878793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0ffbd0 msgr2=0x7f1f3c10a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f32000700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c10a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f32000700 1 -- 172.21.15.57:0/3037878793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240095f0 con 0x7f1f3c104130 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f32000700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400bd70 tx=0x7f1f2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2401d070 con 0x7f1f3c104130 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2400f460 con 0x7f1f3c104130 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f24017630 con 0x7f1f3c104130 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.150+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c10ae70 con 0x7f1f3c104130 2024-02-28T23:46:15.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.151+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c1a22e0 con 0x7f1f3c104130 2024-02-28T23:46:15.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.152+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c04f2a0 con 0x7f1f3c104130 2024-02-28T23:46:15.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.153+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1f2400f5d0 con 0x7f1f3c104130 2024-02-28T23:46:15.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.154+0000 7f1f33e00700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 0x7f1f2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.154+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1f2408eb00 con 0x7f1f3c104130 2024-02-28T23:46:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.155+0000 7f1f3aa00700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 0x7f1f2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.155+0000 7f1f3aa00700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 0x7f1f2006fd50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e270 tx=0x7f1f2c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.159+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f24058290 con 0x7f1f3c104130 2024-02-28T23:46:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:15 smithi057 ceph-mon[89491]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:15 smithi057 ceph-mon[89491]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:15 smithi057 ceph-mon[89491]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:15 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3306756404' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:46:15.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.437+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f3c1a25f0 con 0x7f1f3c104130 2024-02-28T23:46:15.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.438+0000 7f1f33e00700 1 -- 172.21.15.57:0/3037878793 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f1f2405b950 con 0x7f1f3c104130 2024-02-28T23:46:15.437 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:46:15.438 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:46:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:15.440 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 msgr2=0x7f1f2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 0x7f1f2006fd50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e270 tx=0x7f1f2c009040 comp rx=0 tx=0).stop 2024-02-28T23:46:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 msgr2=0x7f1f3c10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400bd70 tx=0x7f1f2400be50 comp rx=0 tx=0).stop 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 shutdown_connections 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1f2006d8a0 0x7f1f2006fd50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0ffbd0 0x7f1f3c10a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 --2- 172.21.15.57:0/3037878793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f3c104130 0x7f1f3c10a570 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 >> 172.21.15.57:0/3037878793 conn(0x7f1f3c075130 msgr2=0x7f1f3c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 shutdown_connections 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.441+0000 7f1f41600700 1 -- 172.21.15.57:0/3037878793 wait complete. 2024-02-28T23:46:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:46:15.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:15 smithi084 ceph-mon[70440]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:15.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:15 smithi084 ceph-mon[70440]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:15.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:15 smithi084 ceph-mon[70440]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:15.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:15 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3306756404' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:46:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 -- 172.21.15.57:0/3217125971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c073090 msgr2=0x7f6c7c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 --2- 172.21.15.57:0/3217125971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c073090 0x7f6c7c073460 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009940 tx=0x7f6c6c009c50 comp rx=0 tx=0).stop 2024-02-28T23:46:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 -- 172.21.15.57:0/3217125971 shutdown_connections 2024-02-28T23:46:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 --2- 172.21.15.57:0/3217125971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 --2- 172.21.15.57:0/3217125971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c073090 0x7f6c7c073460 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 -- 172.21.15.57:0/3217125971 >> 172.21.15.57:0/3217125971 conn(0x7f6c7c0fbda0 msgr2=0x7f6c7c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:15.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 -- 172.21.15.57:0/3217125971 shutdown_connections 2024-02-28T23:46:15.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.599+0000 7f6c83200700 1 -- 172.21.15.57:0/3217125971 wait complete. 2024-02-28T23:46:15.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.600+0000 7f6c83200700 1 Processor -- start 2024-02-28T23:46:15.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.600+0000 7f6c83200700 1 -- start start 2024-02-28T23:46:15.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c83200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c073090 0x7f6c7c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c83200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c83200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c198da0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c83200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19cb30 con 0x7f6c7c073090 2024-02-28T23:46:15.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c7b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c7b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40080/0 (socket says 172.21.15.57:40080) 2024-02-28T23:46:15.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c7b400700 1 -- 172.21.15.57:0/1224552875 learned_addr learned my addr 172.21.15.57:0/1224552875 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.601+0000 7f6c7be00700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c073090 0x7f6c7c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c7b400700 1 -- 172.21.15.57:0/1224552875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c073090 msgr2=0x7f6c7c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c7b400700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c073090 0x7f6c7c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c7b400700 1 -- 172.21.15.57:0/1224552875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0095f0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c7b400700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800c8a0 tx=0x7f6c6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c6800cea0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c68004830 con 0x7f6c7c0739a0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c680055b0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c19ce10 con 0x7f6c7c0739a0 2024-02-28T23:46:15.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.602+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c19d360 con 0x7f6c7c0739a0 2024-02-28T23:46:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.604+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c109ce0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.605+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6c680049a0 con 0x7f6c7c0739a0 2024-02-28T23:46:15.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.605+0000 7f6c78c00700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 0x7f6c6006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.606+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c6808d190 con 0x7f6c7c0739a0 2024-02-28T23:46:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.606+0000 7f6c7be00700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 0x7f6c6006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:15.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.606+0000 7f6c7be00700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 0x7f6c6006fd50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006010 tx=0x7f6c6c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:15.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.610+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c68056920 con 0x7f6c7c0739a0 2024-02-28T23:46:15.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.859+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c7c19d630 con 0x7f6c6006d8a0 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.862+0000 7f6c78c00700 1 -- 172.21.15.57:0/1224552875 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6c7c19d630 con 0x7f6c6006d8a0 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:46:15.861 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:46:15.862 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.864+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 msgr2=0x7f6c6006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.864+0000 7f6c83200700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 0x7f6c6006fd50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006010 tx=0x7f6c6c005a90 comp rx=0 tx=0).stop 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.864+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 msgr2=0x7f6c7c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.864+0000 7f6c83200700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800c8a0 tx=0x7f6c6800cc60 comp rx=0 tx=0).stop 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 shutdown_connections 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6c6006d8a0 0x7f6c6006fd50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c7c073090 0x7f6c7c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 --2- 172.21.15.57:0/1224552875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c7c0739a0 0x7f6c7c1986c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:15.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 >> 172.21.15.57:0/1224552875 conn(0x7f6c7c0fbda0 msgr2=0x7f6c7c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:15.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 shutdown_connections 2024-02-28T23:46:15.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:15.865+0000 7f6c83200700 1 -- 172.21.15.57:0/1224552875 wait complete. 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 -- 172.21.15.57:0/2761166155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 msgr2=0x7f7c30104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 --2- 172.21.15.57:0/2761166155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30104eb0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009910 tx=0x7f7c1c009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 -- 172.21.15.57:0/2761166155 shutdown_connections 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 --2- 172.21.15.57:0/2761166155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30104eb0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 --2- 172.21.15.57:0/2761166155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 0x7f7c30068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.024+0000 7f7c38000700 1 -- 172.21.15.57:0/2761166155 >> 172.21.15.57:0/2761166155 conn(0x7f7c30075130 msgr2=0x7f7c30075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.025+0000 7f7c38000700 1 -- 172.21.15.57:0/2761166155 shutdown_connections 2024-02-28T23:46:16.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.025+0000 7f7c38000700 1 -- 172.21.15.57:0/2761166155 wait complete. 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 Processor -- start 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 -- start start 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 0x7f7c30195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30196a50 con 0x7f7c30069140 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.026+0000 7f7c38000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019a7e0 con 0x7f7c300687a0 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40094/0 (socket says 172.21.15.57:40094) 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 -- 172.21.15.57:0/248103494 learned_addr learned my addr 172.21.15.57:0/248103494 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:16.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c35600700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 0x7f7c30195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:16.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 -- 172.21.15.57:0/248103494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 msgr2=0x7f7c30195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:16.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 0x7f7c30195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 -- 172.21.15.57:0/248103494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0095f0 con 0x7f7c30069140 2024-02-28T23:46:16.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.027+0000 7f7c34c00700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c006010 tx=0x7f7c1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:16.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.028+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1c01d070 con 0x7f7c30069140 2024-02-28T23:46:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.028+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c1c004fb0 con 0x7f7c30069140 2024-02-28T23:46:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.028+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1c022ea0 con 0x7f7c30069140 2024-02-28T23:46:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.028+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3019aa60 con 0x7f7c30069140 2024-02-28T23:46:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.028+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3019af50 con 0x7f7c30069140 2024-02-28T23:46:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.029+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30108800 con 0x7f7c30069140 2024-02-28T23:46:16.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.030+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7c1c022470 con 0x7f7c30069140 2024-02-28T23:46:16.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.031+0000 7f7c2e000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 0x7f7c1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:16.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.031+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7c1c08eab0 con 0x7f7c30069140 2024-02-28T23:46:16.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.031+0000 7f7c35600700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 0x7f7c1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:16.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.032+0000 7f7c35600700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 0x7f7c1806fd50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005fd0 tx=0x7f7c24005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:16.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.036+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c1c090e60 con 0x7f7c30069140 2024-02-28T23:46:16.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.359+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c3004f2a0 con 0x7f7c30069140 2024-02-28T23:46:16.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.360+0000 7f7c2e000700 1 -- 172.21.15.57:0/248103494 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f7c1c027020 con 0x7f7c30069140 2024-02-28T23:46:16.359 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:46:16.359 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:46:16.359 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:46:16.359 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:46:16.361 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:46:16.361 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:46:16.361 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:46:16.361 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.363+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 msgr2=0x7f7c1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 0x7f7c1806fd50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005fd0 tx=0x7f7c24005dd0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 msgr2=0x7f7c30196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c006010 tx=0x7f7c1c00be20 comp rx=0 tx=0).stop 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 shutdown_connections 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c1806d8a0 0x7f7c1806fd50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c300687a0 0x7f7c30195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 --2- 172.21.15.57:0/248103494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c30069140 0x7f7c30196370 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 >> 172.21.15.57:0/248103494 conn(0x7f7c30075130 msgr2=0x7f7c300fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:16.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 shutdown_connections 2024-02-28T23:46:16.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:16.364+0000 7f7c38000700 1 -- 172.21.15.57:0/248103494 wait complete. 2024-02-28T23:46:16.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:16 smithi057 ceph-mon[89491]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:16.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3037878793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:46:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:16 smithi084 ceph-mon[70440]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:16.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3037878793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:46:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:17 smithi057 ceph-mon[89491]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:17 smithi057 ceph-mon[89491]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:17 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/248103494' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:46:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:17 smithi084 ceph-mon[70440]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:17 smithi084 ceph-mon[70440]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:17 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/248103494' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:46:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:19 smithi057 ceph-mon[89491]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:19 smithi084 ceph-mon[70440]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:19.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:21 smithi057 ceph-mon[89491]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:21 smithi084 ceph-mon[70440]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:23 smithi057 ceph-mon[89491]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:23.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:23 smithi084 ceph-mon[70440]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:25.503 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:25 smithi084 ceph-mon[70440]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:25 smithi057 ceph-mon[89491]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:27 smithi057 ceph-mon[89491]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:27.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:27 smithi084 ceph-mon[70440]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:29.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:29 smithi057 ceph-mon[89491]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:29.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:29 smithi084 ceph-mon[70440]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:31 smithi057 ceph-mon[89491]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:31.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:31 smithi084 ceph-mon[70440]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:32.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:32 smithi057 ceph-mon[89491]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:32.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:32 smithi084 ceph-mon[70440]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:33.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:34 smithi057 ceph-mon[89491]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:34.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:34 smithi084 ceph-mon[70440]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:37.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:36 smithi057 ceph-mon[89491]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:36 smithi084 ceph-mon[70440]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:39.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:38 smithi057 ceph-mon[89491]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:39.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:38 smithi084 ceph-mon[70440]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:40.976 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:40 smithi084 ceph-mon[70440]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:40 smithi057 ceph-mon[89491]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:42 smithi057 ceph-mon[89491]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:42 smithi084 ceph-mon[70440]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:44 smithi057 ceph-mon[89491]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:44 smithi084 ceph-mon[70440]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.538+0000 7ff481800700 1 -- 172.21.15.57:0/899487705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 msgr2=0x7ff47c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.538+0000 7ff481800700 1 --2- 172.21.15.57:0/899487705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c106ff0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff468009910 tx=0x7ff468009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 -- 172.21.15.57:0/899487705 shutdown_connections 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 --2- 172.21.15.57:0/899487705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c106ff0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 --2- 172.21.15.57:0/899487705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 -- 172.21.15.57:0/899487705 >> 172.21.15.57:0/899487705 conn(0x7ff47c075e90 msgr2=0x7ff47c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:46.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 -- 172.21.15.57:0/899487705 shutdown_connections 2024-02-28T23:46:46.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.539+0000 7ff481800700 1 -- 172.21.15.57:0/899487705 wait complete. 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.540+0000 7ff481800700 1 Processor -- start 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.540+0000 7ff481800700 1 -- start start 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff481800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff481800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff481800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c19ce00 con 0x7ff47c106b80 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff481800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1a0b90 con 0x7ff47c10c660 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff47be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff47be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58358/0 (socket says 172.21.15.57:58358) 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff47be00700 1 -- 172.21.15.57:0/3989506532 learned_addr learned my addr 172.21.15.57:0/3989506532 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.541+0000 7ff47b400700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:46.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff47be00700 1 -- 172.21.15.57:0/3989506532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 msgr2=0x7ff47c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff47be00700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff47be00700 1 -- 172.21.15.57:0/3989506532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680095f0 con 0x7ff47c106b80 2024-02-28T23:46:46.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff47b400700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:46:46.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff47be00700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c8a0 tx=0x7ff46c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:46.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c00cea0 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff46c004830 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c0055b0 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.542+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c1a0e70 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.543+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c1a13c0 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.544+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c10eb40 con 0x7ff47c106b80 2024-02-28T23:46:46.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.544+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff46c0049a0 con 0x7ff47c106b80 2024-02-28T23:46:46.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.545+0000 7ff478c00700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 0x7ff46006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.545+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff46c08d190 con 0x7ff47c106b80 2024-02-28T23:46:46.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.546+0000 7ff47b400700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 0x7ff46006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:46.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.546+0000 7ff47b400700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 0x7ff46006fd50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff468005b40 tx=0x7ff46801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:46.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.551+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff46c056920 con 0x7ff47c106b80 2024-02-28T23:46:46.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.802+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff47c1a1690 con 0x7ff46006d8a0 2024-02-28T23:46:46.803 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:46 smithi057 ceph-mon[89491]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:46.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.805+0000 7ff478c00700 1 -- 172.21.15.57:0/3989506532 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff47c1a1690 con 0x7ff46006d8a0 2024-02-28T23:46:46.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 msgr2=0x7ff46006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 0x7ff46006fd50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff468005b40 tx=0x7ff46801a040 comp rx=0 tx=0).stop 2024-02-28T23:46:46.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 msgr2=0x7ff47c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c8a0 tx=0x7ff46c00cc60 comp rx=0 tx=0).stop 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 shutdown_connections 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff46006d8a0 0x7ff46006fd50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff47c106b80 0x7ff47c19c1e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 --2- 172.21.15.57:0/3989506532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c10c660 0x7ff47c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 >> 172.21.15.57:0/3989506532 conn(0x7ff47c075e90 msgr2=0x7ff47c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 shutdown_connections 2024-02-28T23:46:46.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.808+0000 7ff481800700 1 -- 172.21.15.57:0/3989506532 wait complete. 2024-02-28T23:46:46.823 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:46:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 -- 172.21.15.57:0/3112580610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec10c8f0 msgr2=0x7f94ec10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 --2- 172.21.15.57:0/3112580610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec10c8f0 0x7f94ec10ccc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009910 tx=0x7f94dc009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 -- 172.21.15.57:0/3112580610 shutdown_connections 2024-02-28T23:46:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 --2- 172.21.15.57:0/3112580610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec106b80 0x7f94ec106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 --2- 172.21.15.57:0/3112580610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec10c8f0 0x7f94ec10ccc0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 -- 172.21.15.57:0/3112580610 >> 172.21.15.57:0/3112580610 conn(0x7f94ec076060 msgr2=0x7f94ec078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:46.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 -- 172.21.15.57:0/3112580610 shutdown_connections 2024-02-28T23:46:46.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.971+0000 7f94f1400700 1 -- 172.21.15.57:0/3112580610 wait complete. 2024-02-28T23:46:46.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.972+0000 7f94f1400700 1 Processor -- start 2024-02-28T23:46:46.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.972+0000 7f94f1400700 1 -- start start 2024-02-28T23:46:46.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94f1400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94f1400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec10c8f0 0x7f94ec19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94f1400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec19d100 con 0x7f94ec106b80 2024-02-28T23:46:46.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94f1400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec1a0e90 con 0x7f94ec10c8f0 2024-02-28T23:46:46.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94ea800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:46.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94ea800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58376/0 (socket says 172.21.15.57:58376) 2024-02-28T23:46:46.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.973+0000 7f94ea800700 1 -- 172.21.15.57:0/178270856 learned_addr learned my addr 172.21.15.57:0/178270856 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94ea800700 1 -- 172.21.15.57:0/178270856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec10c8f0 msgr2=0x7f94ec19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94ea800700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec10c8f0 0x7f94ec19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94ea800700 1 -- 172.21.15.57:0/178270856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0095f0 con 0x7f94ec106b80 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94ea800700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94dc01d070 con 0x7f94ec106b80 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94dc004fb0 con 0x7f94ec106b80 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94dc022ea0 con 0x7f94ec106b80 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.974+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec1a1110 con 0x7f94ec106b80 2024-02-28T23:46:46.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.975+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1a1600 con 0x7f94ec106b80 2024-02-28T23:46:46.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.976+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec109b20 con 0x7f94ec106b80 2024-02-28T23:46:46.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.977+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f94dc022470 con 0x7f94ec106b80 2024-02-28T23:46:46.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.978+0000 7f94e3400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 0x7f94d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:46.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.978+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f94dc08e9b0 con 0x7f94ec106b80 2024-02-28T23:46:46.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.978+0000 7f94e3e00700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 0x7f94d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:46.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.979+0000 7f94e3e00700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 0x7f94d006fd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94ec19db00 tx=0x7f94d4006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:46.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:46.982+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94dc058290 con 0x7f94ec106b80 2024-02-28T23:46:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:46 smithi084 ceph-mon[70440]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:47.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.233+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94ec1a1930 con 0x7f94d006d8a0 2024-02-28T23:46:47.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.235+0000 7f94e3400700 1 -- 172.21.15.57:0/178270856 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f94ec1a1930 con 0x7f94d006d8a0 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.237+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 msgr2=0x7f94d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.237+0000 7f94f1400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 0x7f94d006fd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94ec19db00 tx=0x7f94d4006d20 comp rx=0 tx=0).stop 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.237+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 msgr2=0x7f94ec19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.237+0000 7f94f1400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc00be20 comp rx=0 tx=0).stop 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 shutdown_connections 2024-02-28T23:46:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f94d006d8a0 0x7f94d006fd50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ec106b80 0x7f94ec19c4e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 --2- 172.21.15.57:0/178270856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec10c8f0 0x7f94ec19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 >> 172.21.15.57:0/178270856 conn(0x7f94ec076060 msgr2=0x7f94ec077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 shutdown_connections 2024-02-28T23:46:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.238+0000 7f94f1400700 1 -- 172.21.15.57:0/178270856 wait complete. 2024-02-28T23:46:47.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 -- 172.21.15.57:0/1340393237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 msgr2=0x7f25281042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 --2- 172.21.15.57:0/1340393237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f25281042c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f2514009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 -- 172.21.15.57:0/1340393237 shutdown_connections 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 --2- 172.21.15.57:0/1340393237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f25281042c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 --2- 172.21.15.57:0/1340393237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f25281029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 -- 172.21.15.57:0/1340393237 >> 172.21.15.57:0/1340393237 conn(0x7f25280fddb0 msgr2=0x7f25281001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 -- 172.21.15.57:0/1340393237 shutdown_connections 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.399+0000 7f252f000700 1 -- 172.21.15.57:0/1340393237 wait complete. 2024-02-28T23:46:47.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.400+0000 7f252f000700 1 Processor -- start 2024-02-28T23:46:47.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252f000700 1 -- start start 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252f000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f2528197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528198c00 con 0x7f2528103e50 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252f000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819c990 con 0x7f25281025f0 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58396/0 (socket says 172.21.15.57:58396) 2024-02-28T23:46:47.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.401+0000 7f252d200700 1 -- 172.21.15.57:0/1956316017 learned_addr learned my addr 172.21.15.57:0/1956316017 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252dc00700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f2528197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252d200700 1 -- 172.21.15.57:0/1956316017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 msgr2=0x7f2528197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252d200700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f2528197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252d200700 1 -- 172.21.15.57:0/1956316017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140095f0 con 0x7f2528103e50 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252dc00700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f2528197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f252d200700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2514006010 tx=0x7f251400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.402+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f251401d070 con 0x7f2528103e50 2024-02-28T23:46:47.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.403+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2514004fb0 con 0x7f2528103e50 2024-02-28T23:46:47.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.403+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252819cc10 con 0x7f2528103e50 2024-02-28T23:46:47.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.403+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2514022ea0 con 0x7f2528103e50 2024-02-28T23:46:47.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.403+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252819d180 con 0x7f2528103e50 2024-02-28T23:46:47.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.404+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252810a8d0 con 0x7f2528103e50 2024-02-28T23:46:47.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.405+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2514022470 con 0x7f2528103e50 2024-02-28T23:46:47.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.406+0000 7f2526000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 0x7f251006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.406+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f251408e9b0 con 0x7f2528103e50 2024-02-28T23:46:47.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.406+0000 7f252dc00700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 0x7f251006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.407+0000 7f252dc00700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 0x7f251006fd50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f251c007900 tx=0x7f251c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:47.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.411+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2514058290 con 0x7f2528103e50 2024-02-28T23:46:47.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.639+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2528199430 con 0x7f251006d8a0 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.657+0000 7f2526000700 1 -- 172.21.15.57:0/1956316017 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2528199430 con 0x7f251006d8a0 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (36m) 12m ago 50m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (46m) 12m ago 50m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (49m) 12m ago 49m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (32m) 12m ago 50m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (32m) 12m ago 49m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (35m) 12m ago 49m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (28m) 12m ago 43m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (28m) 12m ago 42m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (12m) 12m ago 42m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:46:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (27m) 12m ago 43m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (38m) 12m ago 52m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (37m) 12m ago 48m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (34m) 12m ago 52m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (33m) 12m ago 48m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (37m) 12m ago 50m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (37m) 12m ago 48m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (31m) 12m ago 47m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (31m) 12m ago 47m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (30m) 12m ago 46m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (30m) 12m ago 45m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (29m) 12m ago 45m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (28m) 12m ago 44m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:46:47.659 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (36m) 12m ago 49m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 msgr2=0x7f251006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 0x7f251006fd50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f251c007900 tx=0x7f251c008040 comp rx=0 tx=0).stop 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 msgr2=0x7f2528198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2514006010 tx=0x7f251400be20 comp rx=0 tx=0).stop 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 shutdown_connections 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f251006d8a0 0x7f251006fd50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25281025f0 0x7f2528197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 --2- 172.21.15.57:0/1956316017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2528103e50 0x7f2528198520 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.660+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 >> 172.21.15.57:0/1956316017 conn(0x7f25280fddb0 msgr2=0x7f25280fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.661+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 shutdown_connections 2024-02-28T23:46:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.661+0000 7f252f000700 1 -- 172.21.15.57:0/1956316017 wait complete. 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 -- 172.21.15.57:0/2558502732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd4073a00 msgr2=0x7fadd410c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 --2- 172.21.15.57:0/2558502732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd4073a00 0x7fadd410c360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009910 tx=0x7fadbc009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 -- 172.21.15.57:0/2558502732 shutdown_connections 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 --2- 172.21.15.57:0/2558502732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd4073a00 0x7fadd410c360 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 --2- 172.21.15.57:0/2558502732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd40730f0 0x7fadd40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 -- 172.21.15.57:0/2558502732 >> 172.21.15.57:0/2558502732 conn(0x7fadd40fbb80 msgr2=0x7fadd40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 -- 172.21.15.57:0/2558502732 shutdown_connections 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.808+0000 7fadd3e00700 1 -- 172.21.15.57:0/2558502732 wait complete. 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 Processor -- start 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 -- start start 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4073a00 0x7fadd4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd419c7a0 con 0x7fadd40730f0 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.810+0000 7fadd3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd419c910 con 0x7fadd4073a00 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58414/0 (socket says 172.21.15.57:58414) 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 -- 172.21.15.57:0/1896049362 learned_addr learned my addr 172.21.15.57:0/1896049362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadcbe00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4073a00 0x7fadd4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 -- 172.21.15.57:0/1896049362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4073a00 msgr2=0x7fadd4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4073a00 0x7fadd4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.811+0000 7fadd2a00700 1 -- 172.21.15.57:0/1896049362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0095f0 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd2a00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b4b0 tx=0x7fadc400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc4011800 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc4011e20 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd419cbf0 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc400f2e0 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.812+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd419d140 con 0x7fadd40730f0 2024-02-28T23:46:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.813+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd4109a60 con 0x7fadd40730f0 2024-02-28T23:46:47.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.815+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fadc4010380 con 0x7fadd40730f0 2024-02-28T23:46:47.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.815+0000 7fadd8a00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 0x7fadb806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.816+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fadc408d0f0 con 0x7fadd40730f0 2024-02-28T23:46:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.816+0000 7fadcbe00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 0x7fadb806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.817+0000 7fadcbe00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 0x7fadb806fd50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b5c0 tx=0x7fadbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:47.822+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadc4056930 con 0x7fadd40730f0 2024-02-28T23:46:48.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:47 smithi057 ceph-mon[89491]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:48.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:47 smithi084 ceph-mon[70440]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:48.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.135+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fadd4062380 con 0x7fadd40730f0 2024-02-28T23:46:48.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.135+0000 7fadd8a00700 1 -- 172.21.15.57:0/1896049362 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fadc4059ff0 con 0x7fadd40730f0 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:46:48.136 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:46:48.137 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:46:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 msgr2=0x7fadb806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 0x7fadb806fd50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b5c0 tx=0x7fadbc01a040 comp rx=0 tx=0).stop 2024-02-28T23:46:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 msgr2=0x7fadd4197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b4b0 tx=0x7fadc400b870 comp rx=0 tx=0).stop 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 shutdown_connections 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fadb806d8a0 0x7fadb806fd50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadd40730f0 0x7fadd4197df0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 --2- 172.21.15.57:0/1896049362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4073a00 0x7fadd4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 >> 172.21.15.57:0/1896049362 conn(0x7fadd40fbb80 msgr2=0x7fadd4106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 shutdown_connections 2024-02-28T23:46:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.140+0000 7fadd3e00700 1 -- 172.21.15.57:0/1896049362 wait complete. 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.299+0000 7fc373c00700 1 -- 172.21.15.57:0/1370257816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c1004e0 msgr2=0x7fc36c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.299+0000 7fc373c00700 1 --2- 172.21.15.57:0/1370257816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c1004e0 0x7fc36c1008b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc360009910 tx=0x7fc360009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 -- 172.21.15.57:0/1370257816 shutdown_connections 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 --2- 172.21.15.57:0/1370257816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c100e80 0x7fc36c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 --2- 172.21.15.57:0/1370257816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c1004e0 0x7fc36c1008b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 -- 172.21.15.57:0/1370257816 >> 172.21.15.57:0/1370257816 conn(0x7fc36c0fbda0 msgr2=0x7fc36c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:48.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 -- 172.21.15.57:0/1370257816 shutdown_connections 2024-02-28T23:46:48.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.300+0000 7fc373c00700 1 -- 172.21.15.57:0/1370257816 wait complete. 2024-02-28T23:46:48.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 Processor -- start 2024-02-28T23:46:48.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 -- start start 2024-02-28T23:46:48.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c1004e0 0x7fc36c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c198dc0 con 0x7fc36c100e80 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.301+0000 7fc373c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c19cb50 con 0x7fc36c1004e0 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58432/0 (socket says 172.21.15.57:58432) 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 -- 172.21.15.57:0/2657047184 learned_addr learned my addr 172.21.15.57:0/2657047184 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc371200700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c1004e0 0x7fc36c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 -- 172.21.15.57:0/2657047184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c1004e0 msgr2=0x7fc36c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c1004e0 0x7fc36c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 -- 172.21.15.57:0/2657047184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600095f0 con 0x7fc36c100e80 2024-02-28T23:46:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.302+0000 7fc36be00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc35800b480 tx=0x7fc35800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.303+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3580118c0 con 0x7fc36c100e80 2024-02-28T23:46:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.303+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc358010380 con 0x7fc36c100e80 2024-02-28T23:46:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.303+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c19ce30 con 0x7fc36c100e80 2024-02-28T23:46:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.303+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35800f4f0 con 0x7fc36c100e80 2024-02-28T23:46:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.303+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c19d380 con 0x7fc36c100e80 2024-02-28T23:46:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.304+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c1045d0 con 0x7fc36c100e80 2024-02-28T23:46:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.305+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc3580104f0 con 0x7fc36c100e80 2024-02-28T23:46:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.306+0000 7fc369600700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 0x7fc35406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.306+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc35808d210 con 0x7fc36c100e80 2024-02-28T23:46:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.307+0000 7fc371200700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 0x7fc35406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.307+0000 7fc371200700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 0x7fc35406fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc360006010 tx=0x7fc36001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:48.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.312+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc358056920 con 0x7fc36c100e80 2024-02-28T23:46:48.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.575+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc36c19d600 con 0x7fc36c100e80 2024-02-28T23:46:48.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.576+0000 7fc369600700 1 -- 172.21.15.57:0/2657047184 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fc358059fe0 con 0x7fc36c100e80 2024-02-28T23:46:48.577 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:46:48.577 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:46:48.577 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:46:48.577 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:46:48.578 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:46:48.579 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:46:48.580 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:48.581 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:48.581 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:48.581 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 msgr2=0x7fc35406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 0x7fc35406fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc360006010 tx=0x7fc36001a040 comp rx=0 tx=0).stop 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 msgr2=0x7fc36c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc35800b480 tx=0x7fc35800b840 comp rx=0 tx=0).stop 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 shutdown_connections 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc35406d8a0 0x7fc35406fd50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c1004e0 0x7fc36c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 --2- 172.21.15.57:0/2657047184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36c100e80 0x7fc36c1986e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.579+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 >> 172.21.15.57:0/2657047184 conn(0x7fc36c0fbda0 msgr2=0x7fc36c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:48.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.580+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 shutdown_connections 2024-02-28T23:46:48.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.580+0000 7fc373c00700 1 -- 172.21.15.57:0/2657047184 wait complete. 2024-02-28T23:46:48.583 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:46:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 -- 172.21.15.57:0/1164956221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b80100e80 msgr2=0x7f6b80104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 --2- 172.21.15.57:0/1164956221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b80100e80 0x7f6b80104e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009910 tx=0x7f6b6c009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 -- 172.21.15.57:0/1164956221 shutdown_connections 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 --2- 172.21.15.57:0/1164956221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b80100e80 0x7f6b80104e50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 --2- 172.21.15.57:0/1164956221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b801004e0 0x7f6b801008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 -- 172.21.15.57:0/1164956221 >> 172.21.15.57:0/1164956221 conn(0x7f6b800fbda0 msgr2=0x7f6b800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 -- 172.21.15.57:0/1164956221 shutdown_connections 2024-02-28T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.741+0000 7f6b85800700 1 -- 172.21.15.57:0/1164956221 wait complete. 2024-02-28T23:46:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.742+0000 7f6b85800700 1 Processor -- start 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.742+0000 7f6b85800700 1 -- start start 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b85800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b85800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b80100e80 0x7f6b80073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b85800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80073ad0 con 0x7f6b801004e0 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b85800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80073c10 con 0x7f6b80100e80 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b7ea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58454/0 (socket says 172.21.15.57:58454) 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.743+0000 7f6b7ea00700 1 -- 172.21.15.57:0/3808316024 learned_addr learned my addr 172.21.15.57:0/3808316024 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b7e000700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b80100e80 0x7f6b80073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b7ea00700 1 -- 172.21.15.57:0/3808316024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b80100e80 msgr2=0x7f6b80073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b7ea00700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b80100e80 0x7f6b80073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b7ea00700 1 -- 172.21.15.57:0/3808316024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0095f0 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b7ea00700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b4b0 tx=0x7f6b7000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b70011800 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b70011e20 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b801a21c0 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7000f2e0 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.744+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b801a25d0 con 0x7f6b801004e0 2024-02-28T23:46:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.745+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b80108780 con 0x7f6b801004e0 2024-02-28T23:46:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.747+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6b70010380 con 0x7f6b801004e0 2024-02-28T23:46:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.747+0000 7f6b77400700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 0x7f6b6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.747+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6b7008d0f0 con 0x7f6b801004e0 2024-02-28T23:46:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.748+0000 7f6b7e000700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 0x7f6b6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.748+0000 7f6b7e000700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 0x7f6b6406fd50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b80074840 tx=0x7f6b6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:48.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.752+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b70056930 con 0x7f6b801004e0 2024-02-28T23:46:49.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:48.999+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b80061190 con 0x7f6b6406d8a0 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1896049362' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:49.000 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:48 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2657047184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:46:49.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.003+0000 7f6b77400700 1 -- 172.21.15.57:0/3808316024 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6b80061190 con 0x7f6b6406d8a0 2024-02-28T23:46:49.003 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:46:49.003 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:46:49.003 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:46:49.003 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:46:49.004 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:46:49.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.005+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 msgr2=0x7f6b6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.005+0000 7f6b85800700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 0x7f6b6406fd50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b80074840 tx=0x7f6b6c01a040 comp rx=0 tx=0).stop 2024-02-28T23:46:49.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.005+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 msgr2=0x7f6b80073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.005+0000 7f6b85800700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b4b0 tx=0x7f6b7000b870 comp rx=0 tx=0).stop 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 shutdown_connections 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6b6406d8a0 0x7f6b6406fd50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b801004e0 0x7f6b80073050 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 --2- 172.21.15.57:0/3808316024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b80100e80 0x7f6b80073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 >> 172.21.15.57:0/3808316024 conn(0x7f6b800fbda0 msgr2=0x7f6b800fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 shutdown_connections 2024-02-28T23:46:49.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.006+0000 7f6b85800700 1 -- 172.21.15.57:0/3808316024 wait complete. 2024-02-28T23:46:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1896049362' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:46:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:46:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:48 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2657047184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.168+0000 7fab73c00700 1 -- 172.21.15.57:0/2380153280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c103040 msgr2=0x7fab6c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.168+0000 7fab73c00700 1 --2- 172.21.15.57:0/2380153280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c103040 0x7fab6c107010 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fab58009910 tx=0x7fab58009c20 comp rx=0 tx=0).stop 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 -- 172.21.15.57:0/2380153280 shutdown_connections 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 --2- 172.21.15.57:0/2380153280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c103040 0x7fab6c107010 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 --2- 172.21.15.57:0/2380153280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c1026a0 0x7fab6c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 -- 172.21.15.57:0/2380153280 >> 172.21.15.57:0/2380153280 conn(0x7fab6c0fdf40 msgr2=0x7fab6c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 -- 172.21.15.57:0/2380153280 shutdown_connections 2024-02-28T23:46:49.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.169+0000 7fab73c00700 1 -- 172.21.15.57:0/2380153280 wait complete. 2024-02-28T23:46:49.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 Processor -- start 2024-02-28T23:46:49.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 -- start start 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c103040 0x7fab6c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c198c70 con 0x7fab6c1026a0 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.170+0000 7fab73c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c19ca00 con 0x7fab6c103040 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58480/0 (socket says 172.21.15.57:58480) 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 -- 172.21.15.57:0/970473459 learned_addr learned my addr 172.21.15.57:0/970473459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab6be00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c103040 0x7fab6c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 -- 172.21.15.57:0/970473459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c103040 msgr2=0x7fab6c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c103040 0x7fab6c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 -- 172.21.15.57:0/970473459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580095f0 con 0x7fab6c1026a0 2024-02-28T23:46:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.171+0000 7fab71200700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d8d0 tx=0x7fab6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.172+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab60009940 con 0x7fab6c1026a0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.172+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab60010460 con 0x7fab6c1026a0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.172+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6000f5f0 con 0x7fab6c1026a0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.172+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c19cce0 con 0x7fab6c1026a0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.172+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c19d230 con 0x7fab6c1026a0 2024-02-28T23:46:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.173+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c10aa10 con 0x7fab6c1026a0 2024-02-28T23:46:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.174+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fab600105d0 con 0x7fab6c1026a0 2024-02-28T23:46:49.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.175+0000 7fab69600700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 0x7fab5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:46:49.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.175+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fab6008d5d0 con 0x7fab6c1026a0 2024-02-28T23:46:49.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.175+0000 7fab6be00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 0x7fab5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:46:49.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.176+0000 7fab6be00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 0x7fab5406fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab58000c00 tx=0x7fab58005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:46:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.180+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab60056ce0 con 0x7fab6c1026a0 2024-02-28T23:46:49.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.502+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab6c04f2a0 con 0x7fab6c1026a0 2024-02-28T23:46:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.503+0000 7fab69600700 1 -- 172.21.15.57:0/970473459 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fab6005a3a0 con 0x7fab6c1026a0 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:46:49.504 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:46:49.505 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 msgr2=0x7fab5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 0x7fab5406fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab58000c00 tx=0x7fab58005fd0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 msgr2=0x7fab6c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d8d0 tx=0x7fab6000dc90 comp rx=0 tx=0).stop 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 shutdown_connections 2024-02-28T23:46:49.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fab5406d8a0 0x7fab5406fd50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab6c1026a0 0x7fab6c198050 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 --2- 172.21.15.57:0/970473459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab6c103040 0x7fab6c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:46:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 >> 172.21.15.57:0/970473459 conn(0x7fab6c0fdf40 msgr2=0x7fab6c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:46:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.506+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 shutdown_connections 2024-02-28T23:46:49.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:46:49.507+0000 7fab73c00700 1 -- 172.21.15.57:0/970473459 wait complete. 2024-02-28T23:46:49.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:49 smithi057 ceph-mon[89491]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:49.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:49 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/970473459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:46:50.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:49 smithi084 ceph-mon[70440]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:46:50.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:49 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/970473459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:46:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:50 smithi057 ceph-mon[89491]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:51.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:50 smithi084 ceph-mon[70440]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:53.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:52 smithi057 ceph-mon[89491]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:52 smithi084 ceph-mon[70440]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:55.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:54 smithi057 ceph-mon[89491]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:54 smithi084 ceph-mon[70440]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:57.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:56 smithi057 ceph-mon[89491]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:57.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:56 smithi084 ceph-mon[70440]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:46:58 smithi057 ceph-mon[89491]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:46:59.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:46:58 smithi084 ceph-mon[70440]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:01.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:00 smithi057 ceph-mon[89491]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:01.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:00 smithi084 ceph-mon[70440]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:03.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:02 smithi057 ceph-mon[89491]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:03.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:02 smithi084 ceph-mon[70440]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:04.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:04 smithi057 ceph-mon[89491]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:05.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:04 smithi084 ceph-mon[70440]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:07.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:06 smithi057 ceph-mon[89491]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:07.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:06 smithi084 ceph-mon[70440]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:09.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:08 smithi057 ceph-mon[89491]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:09.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:08 smithi084 ceph-mon[70440]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:11.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:10 smithi057 ceph-mon[89491]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:11.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:10 smithi084 ceph-mon[70440]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:13.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:12 smithi057 ceph-mon[89491]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:13.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:12 smithi084 ceph-mon[70440]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:14 smithi057 ceph-mon[89491]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:14 smithi084 ceph-mon[70440]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:16 smithi084 ceph-mon[70440]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:16 smithi057 ceph-mon[89491]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:18 smithi084 ceph-mon[70440]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:19.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:18 smithi057 ceph-mon[89491]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 -- 172.21.15.57:0/648013349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc106b80 msgr2=0x7f5ddc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 --2- 172.21.15.57:0/648013349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc106b80 0x7f5ddc106ff0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009910 tx=0x7f5dc8009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 -- 172.21.15.57:0/648013349 shutdown_connections 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 --2- 172.21.15.57:0/648013349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc106b80 0x7f5ddc106ff0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 --2- 172.21.15.57:0/648013349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 -- 172.21.15.57:0/648013349 >> 172.21.15.57:0/648013349 conn(0x7f5ddc076060 msgr2=0x7f5ddc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:19.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 -- 172.21.15.57:0/648013349 shutdown_connections 2024-02-28T23:47:19.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.685+0000 7f5de0e00700 1 -- 172.21.15.57:0/648013349 wait complete. 2024-02-28T23:47:19.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.687+0000 7f5de0e00700 1 Processor -- start 2024-02-28T23:47:19.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.687+0000 7f5de0e00700 1 -- start start 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc106b80 0x7f5ddc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5de0e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc19d060 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5de0e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc1a0df0 con 0x7f5ddc106b80 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5dd9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5dd9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49112/0 (socket says 172.21.15.57:49112) 2024-02-28T23:47:19.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5dd9400700 1 -- 172.21.15.57:0/454149049 learned_addr learned my addr 172.21.15.57:0/454149049 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:19.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.688+0000 7f5dd9e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc106b80 0x7f5ddc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:19.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd9400700 1 -- 172.21.15.57:0/454149049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc106b80 msgr2=0x7f5ddc19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:19.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd9400700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc106b80 0x7f5ddc19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd9400700 1 -- 172.21.15.57:0/454149049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80095f0 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd9400700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:19.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc801d070 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dc8004fb0 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc800f690 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.689+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc1a1070 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.690+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc1a1560 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.691+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc10ecc0 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.692+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5dc80228a0 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.693+0000 7f5dd2a00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 0x7f5dc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:19.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.693+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5dc808ede0 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.693+0000 7f5dd9e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 0x7f5dc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:19.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.694+0000 7f5dd9e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 0x7f5dc006fd50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005fd0 tx=0x7f5dcc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:19.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.698+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5dc8058570 con 0x7f5ddc10c8f0 2024-02-28T23:47:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.950+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ddc02cca0 con 0x7f5dc006d8a0 2024-02-28T23:47:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.953+0000 7f5dd2a00700 1 -- 172.21.15.57:0/454149049 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ddc02cca0 con 0x7f5dc006d8a0 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 msgr2=0x7f5dc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 0x7f5dc006fd50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005fd0 tx=0x7f5dcc00a560 comp rx=0 tx=0).stop 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 msgr2=0x7f5ddc19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc800be20 comp rx=0 tx=0).stop 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 shutdown_connections 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5dc006d8a0 0x7f5dc006fd50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ddc106b80 0x7f5ddc19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 --2- 172.21.15.57:0/454149049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ddc10c8f0 0x7f5ddc19c980 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 >> 172.21.15.57:0/454149049 conn(0x7f5ddc076060 msgr2=0x7f5ddc077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 shutdown_connections 2024-02-28T23:47:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:19.956+0000 7f5de0e00700 1 -- 172.21.15.57:0/454149049 wait complete. 2024-02-28T23:47:19.971 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 -- 172.21.15.57:0/3215395786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 msgr2=0x7f0dd410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 --2- 172.21.15.57:0/3215395786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd410ccc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009910 tx=0x7f0dc4009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 -- 172.21.15.57:0/3215395786 shutdown_connections 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 --2- 172.21.15.57:0/3215395786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 0x7f0dd4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 --2- 172.21.15.57:0/3215395786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd410ccc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 -- 172.21.15.57:0/3215395786 >> 172.21.15.57:0/3215395786 conn(0x7f0dd4076060 msgr2=0x7f0dd4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:20.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 -- 172.21.15.57:0/3215395786 shutdown_connections 2024-02-28T23:47:20.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.114+0000 7f0dd9800700 1 -- 172.21.15.57:0/3215395786 wait complete. 2024-02-28T23:47:20.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.115+0000 7f0dd9800700 1 Processor -- start 2024-02-28T23:47:20.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd9800700 1 -- start start 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd9800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 0x7f0dd419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd9800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd9800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd419d120 con 0x7f0dd410c8f0 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd9800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41a0eb0 con 0x7f0dd4106b80 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49132/0 (socket says 172.21.15.57:49132) 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.116+0000 7f0dd2000700 1 -- 172.21.15.57:0/1364101240 learned_addr learned my addr 172.21.15.57:0/1364101240 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.117+0000 7f0dd2a00700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 0x7f0dd419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.117+0000 7f0dd2000700 1 -- 172.21.15.57:0/1364101240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 msgr2=0x7f0dd419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.117+0000 7f0dd2000700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 0x7f0dd419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.117+0000 7f0dd2000700 1 -- 172.21.15.57:0/1364101240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40095f0 con 0x7f0dd410c8f0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.117+0000 7f0dd2000700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c8a0 tx=0x7f0dc000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.118+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dc000cea0 con 0x7f0dd410c8f0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.118+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dc0004830 con 0x7f0dd410c8f0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.118+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd41a1190 con 0x7f0dd410c8f0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.118+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dc00055b0 con 0x7f0dd410c8f0 2024-02-28T23:47:20.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.118+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41a16e0 con 0x7f0dd410c8f0 2024-02-28T23:47:20.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.119+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4109b20 con 0x7f0dd410c8f0 2024-02-28T23:47:20.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.120+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0dc00049a0 con 0x7f0dd410c8f0 2024-02-28T23:47:20.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.121+0000 7f0dcb400700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 0x7f0db806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.121+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0dc008d180 con 0x7f0dd410c8f0 2024-02-28T23:47:20.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.121+0000 7f0dd2a00700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 0x7f0db806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.122+0000 7f0dd2a00700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 0x7f0db806fd50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b5c0 tx=0x7f0dc401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.126+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0dc0056910 con 0x7f0dd410c8f0 2024-02-28T23:47:20.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.377+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd41a19b0 con 0x7f0db806d8a0 2024-02-28T23:47:20.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.379+0000 7f0dcb400700 1 -- 172.21.15.57:0/1364101240 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0dd41a19b0 con 0x7f0db806d8a0 2024-02-28T23:47:20.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 msgr2=0x7f0db806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 0x7f0db806fd50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b5c0 tx=0x7f0dc401a040 comp rx=0 tx=0).stop 2024-02-28T23:47:20.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 msgr2=0x7f0dd419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c8a0 tx=0x7f0dc000cc60 comp rx=0 tx=0).stop 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 shutdown_connections 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0db806d8a0 0x7f0db806fd50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd4106b80 0x7f0dd419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 --2- 172.21.15.57:0/1364101240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dd410c8f0 0x7f0dd419ca40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 >> 172.21.15.57:0/1364101240 conn(0x7f0dd4076060 msgr2=0x7f0dd4077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 shutdown_connections 2024-02-28T23:47:20.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.382+0000 7f0dd9800700 1 -- 172.21.15.57:0/1364101240 wait complete. 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.534+0000 7f3b66800700 1 -- 172.21.15.57:0/1639269215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6010c8f0 msgr2=0x7f3b6010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.534+0000 7f3b66800700 1 --2- 172.21.15.57:0/1639269215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6010c8f0 0x7f3b6010ccc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009910 tx=0x7f3b50009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 -- 172.21.15.57:0/1639269215 shutdown_connections 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 --2- 172.21.15.57:0/1639269215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b60106b80 0x7f3b60106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 --2- 172.21.15.57:0/1639269215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6010c8f0 0x7f3b6010ccc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 -- 172.21.15.57:0/1639269215 >> 172.21.15.57:0/1639269215 conn(0x7f3b60076020 msgr2=0x7f3b60078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 -- 172.21.15.57:0/1639269215 shutdown_connections 2024-02-28T23:47:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.535+0000 7f3b66800700 1 -- 172.21.15.57:0/1639269215 wait complete. 2024-02-28T23:47:20.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 Processor -- start 2024-02-28T23:47:20.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 -- start start 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b6010c8f0 0x7f3b6019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6019cff0 con 0x7f3b60106b80 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.536+0000 7f3b66800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b601a0d80 con 0x7f3b6010c8f0 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49148/0 (socket says 172.21.15.57:49148) 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 -- 172.21.15.57:0/3883554478 learned_addr learned my addr 172.21.15.57:0/3883554478 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:20.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b57e00700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b6010c8f0 0x7f3b6019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 -- 172.21.15.57:0/3883554478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b6010c8f0 msgr2=0x7f3b6019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b6010c8f0 0x7f3b6019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 -- 172.21.15.57:0/3883554478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500095f0 con 0x7f3b60106b80 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.537+0000 7f3b5f400700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006010 tx=0x7f3b5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.538+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b5001d070 con 0x7f3b60106b80 2024-02-28T23:47:20.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.538+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b50004fb0 con 0x7f3b60106b80 2024-02-28T23:47:20.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.538+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b601a1000 con 0x7f3b60106b80 2024-02-28T23:47:20.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.538+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b50022ea0 con 0x7f3b60106b80 2024-02-28T23:47:20.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.538+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b601a14f0 con 0x7f3b60106b80 2024-02-28T23:47:20.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.539+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6004f2a0 con 0x7f3b60106b80 2024-02-28T23:47:20.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.540+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3b50022470 con 0x7f3b60106b80 2024-02-28T23:47:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.541+0000 7f3b5cc00700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 0x7f3b4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.541+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3b5008eab0 con 0x7f3b60106b80 2024-02-28T23:47:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.541+0000 7f3b57e00700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 0x7f3b4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.542+0000 7f3b57e00700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 0x7f3b4406fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005fd0 tx=0x7f3b4800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.546+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b50090e60 con 0x7f3b60106b80 2024-02-28T23:47:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.781+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b601a1a00 con 0x7f3b4406d8a0 2024-02-28T23:47:20.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.814+0000 7f3b5cc00700 1 -- 172.21.15.57:0/3883554478 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3b601a1a00 con 0x7f3b4406d8a0 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (37m) 12m ago 50m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (47m) 12m ago 51m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (49m) 12m ago 49m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (33m) 12m ago 51m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (32m) 12m ago 49m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (36m) 12m ago 50m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (28m) 12m ago 43m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (28m) 12m ago 43m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (12m) 12m ago 43m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (28m) 12m ago 43m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (39m) 12m ago 52m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (38m) 12m ago 49m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:47:20.814 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (34m) 12m ago 52m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (34m) 12m ago 49m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (38m) 12m ago 50m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (37m) 12m ago 49m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (32m) 12m ago 48m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (31m) 12m ago 47m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (31m) 12m ago 47m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (30m) 12m ago 46m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (30m) 12m ago 45m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (29m) 12m ago 45m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:47:20.815 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (37m) 12m ago 50m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 msgr2=0x7f3b4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 0x7f3b4406fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005fd0 tx=0x7f3b4800a560 comp rx=0 tx=0).stop 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 msgr2=0x7f3b6019c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006010 tx=0x7f3b5000be20 comp rx=0 tx=0).stop 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 shutdown_connections 2024-02-28T23:47:20.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f3b4406d8a0 0x7f3b4406fd50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b60106b80 0x7f3b6019c3d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 --2- 172.21.15.57:0/3883554478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b6010c8f0 0x7f3b6019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 >> 172.21.15.57:0/3883554478 conn(0x7f3b60076020 msgr2=0x7f3b60077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:20.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 shutdown_connections 2024-02-28T23:47:20.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.818+0000 7f3b66800700 1 -- 172.21.15.57:0/3883554478 wait complete. 2024-02-28T23:47:20.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 -- 172.21.15.57:0/476416195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 msgr2=0x7f6298104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 --2- 172.21.15.57:0/476416195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298104e50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6280009910 tx=0x7f6280009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 -- 172.21.15.57:0/476416195 shutdown_connections 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 --2- 172.21.15.57:0/476416195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298104e50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 --2- 172.21.15.57:0/476416195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f62981008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 -- 172.21.15.57:0/476416195 >> 172.21.15.57:0/476416195 conn(0x7f62980fbda0 msgr2=0x7f62980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 -- 172.21.15.57:0/476416195 shutdown_connections 2024-02-28T23:47:20.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.978+0000 7f629ee00700 1 -- 172.21.15.57:0/476416195 wait complete. 2024-02-28T23:47:20.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 Processor -- start 2024-02-28T23:47:20.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 -- start start 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f6298195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298196b50 con 0x7f6298100e80 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.980+0000 7f629ee00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629819a8e0 con 0x7f62981004e0 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49172/0 (socket says 172.21.15.57:49172) 2024-02-28T23:47:20.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 -- 172.21.15.57:0/4020011833 learned_addr learned my addr 172.21.15.57:0/4020011833 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f6297e00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f6298195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 -- 172.21.15.57:0/4020011833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 msgr2=0x7f6298195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f6298195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f628f400700 1 -- 172.21.15.57:0/4020011833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800095f0 con 0x7f6298100e80 2024-02-28T23:47:20.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.981+0000 7f6297e00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f6298195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f628f400700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6280005b40 tx=0x7f628000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628001d070 con 0x7f6298100e80 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f628000bed0 con 0x7f6298100e80 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6280017400 con 0x7f6298100e80 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629819ab60 con 0x7f6298100e80 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.982+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629819b050 con 0x7f6298100e80 2024-02-28T23:47:20.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.983+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298108850 con 0x7f6298100e80 2024-02-28T23:47:20.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.985+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f62800050d0 con 0x7f6298100e80 2024-02-28T23:47:20.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.986+0000 7f6295600700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 0x7f627c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:20.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.986+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f628008e940 con 0x7f6298100e80 2024-02-28T23:47:20.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.986+0000 7f6297e00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 0x7f627c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:20.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.987+0000 7f6297e00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 0x7f627c06fd50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6288009fd0 tx=0x7f628800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:20.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:20.990+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62800580d0 con 0x7f6298100e80 2024-02-28T23:47:21.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:20 smithi057 ceph-mon[89491]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:21.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:20 smithi057 ceph-mon[89491]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:21.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.315+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f629804f2a0 con 0x7f6298100e80 2024-02-28T23:47:21.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.316+0000 7f6295600700 1 -- 172.21.15.57:0/4020011833 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f628005b790 con 0x7f6298100e80 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:47:21.316 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:47:21.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:21.317 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:47:21.317 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:47:21.317 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:47:21.317 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.319+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 msgr2=0x7f627c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.319+0000 7f629ee00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 0x7f627c06fd50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6288009fd0 tx=0x7f628800b040 comp rx=0 tx=0).stop 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.319+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 msgr2=0x7f6298196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.319+0000 7f629ee00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6280005b40 tx=0x7f628000f740 comp rx=0 tx=0).stop 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.319+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 shutdown_connections 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f627c06d8a0 0x7f627c06fd50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62981004e0 0x7f6298195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 --2- 172.21.15.57:0/4020011833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6298100e80 0x7f6298196470 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 >> 172.21.15.57:0/4020011833 conn(0x7f62980fbda0 msgr2=0x7f62980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 shutdown_connections 2024-02-28T23:47:21.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.320+0000 7f629ee00700 1 -- 172.21.15.57:0/4020011833 wait complete. 2024-02-28T23:47:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:20 smithi084 ceph-mon[70440]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:20 smithi084 ceph-mon[70440]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 -- 172.21.15.57:0/3262178113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25041004e0 msgr2=0x7f25041008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 --2- 172.21.15.57:0/3262178113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25041004e0 0x7f25041008b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b0a0 tx=0x7f24f400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 -- 172.21.15.57:0/3262178113 shutdown_connections 2024-02-28T23:47:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 --2- 172.21.15.57:0/3262178113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2504100e80 0x7f2504104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 --2- 172.21.15.57:0/3262178113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25041004e0 0x7f25041008b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 -- 172.21.15.57:0/3262178113 >> 172.21.15.57:0/3262178113 conn(0x7f25040fbda0 msgr2=0x7f25040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:21.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 -- 172.21.15.57:0/3262178113 shutdown_connections 2024-02-28T23:47:21.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.468+0000 7f250ac00700 1 -- 172.21.15.57:0/3262178113 wait complete. 2024-02-28T23:47:21.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 Processor -- start 2024-02-28T23:47:21.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 -- start start 2024-02-28T23:47:21.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25041004e0 0x7f2504073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504073ad0 con 0x7f2504100e80 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.469+0000 7f250ac00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504073c10 con 0x7f25041004e0 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49190/0 (socket says 172.21.15.57:49190) 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 -- 172.21.15.57:0/1855714817 learned_addr learned my addr 172.21.15.57:0/1855714817 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f2503e00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25041004e0 0x7f2504073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 -- 172.21.15.57:0/1855714817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25041004e0 msgr2=0x7f2504073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25041004e0 0x7f2504073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 -- 172.21.15.57:0/1855714817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009d00 con 0x7f2504100e80 2024-02-28T23:47:21.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.470+0000 7f24fb400700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c8f0 tx=0x7f24ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.471+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24ec007ad0 con 0x7f2504100e80 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.471+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24ec00ce90 con 0x7f2504100e80 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.471+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24ec005600 con 0x7f2504100e80 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.471+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25041a2180 con 0x7f2504100e80 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.471+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25041a2590 con 0x7f2504100e80 2024-02-28T23:47:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.472+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25041087f0 con 0x7f2504100e80 2024-02-28T23:47:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.473+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f24ec004750 con 0x7f2504100e80 2024-02-28T23:47:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.474+0000 7f2501600700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 0x7f24e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.474+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f24ec08d210 con 0x7f2504100e80 2024-02-28T23:47:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.474+0000 7f2503e00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 0x7f24e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.475+0000 7f2503e00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 0x7f24e806fd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009480 tx=0x7f24f400b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.478+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24ec0569a0 con 0x7f2504100e80 2024-02-28T23:47:21.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.758+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f250404f2a0 con 0x7f2504100e80 2024-02-28T23:47:21.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.759+0000 7f2501600700 1 -- 172.21.15.57:0/1855714817 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f24ec01d740 con 0x7f2504100e80 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:47:21.759 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:47:21.760 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:21.761 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:21.762 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:21.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 msgr2=0x7f24e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 0x7f24e806fd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009480 tx=0x7f24f400b7f0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 msgr2=0x7f2504073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c8f0 tx=0x7f24ec00ccb0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 shutdown_connections 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f24e806d8a0 0x7f24e806fd50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25041004e0 0x7f2504073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 --2- 172.21.15.57:0/1855714817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504100e80 0x7f2504073590 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.762+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 >> 172.21.15.57:0/1855714817 conn(0x7f25040fbda0 msgr2=0x7f25040fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.763+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 shutdown_connections 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.763+0000 7f250ac00700 1 -- 172.21.15.57:0/1855714817 wait complete. 2024-02-28T23:47:21.763 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:47:21.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.931+0000 7f4d35000700 1 -- 172.21.15.57:0/2333380435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 msgr2=0x7f4d300734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.931+0000 7f4d35000700 1 --2- 172.21.15.57:0/2333380435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d300734c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009910 tx=0x7f4d20009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:21.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:21 smithi057 ceph-mon[89491]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:21.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:21 smithi057 ceph-mon[89491]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:21.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:21 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4020011833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:47:21.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:21 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1855714817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.931+0000 7f4d35000700 1 -- 172.21.15.57:0/2333380435 shutdown_connections 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.931+0000 7f4d35000700 1 --2- 172.21.15.57:0/2333380435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 0x7f4d30110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.931+0000 7f4d35000700 1 --2- 172.21.15.57:0/2333380435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d300734c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.932+0000 7f4d35000700 1 -- 172.21.15.57:0/2333380435 >> 172.21.15.57:0/2333380435 conn(0x7f4d300fbda0 msgr2=0x7f4d300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.932+0000 7f4d35000700 1 -- 172.21.15.57:0/2333380435 shutdown_connections 2024-02-28T23:47:21.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.932+0000 7f4d35000700 1 -- 172.21.15.57:0/2333380435 wait complete. 2024-02-28T23:47:21.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.933+0000 7f4d35000700 1 Processor -- start 2024-02-28T23:47:21.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.933+0000 7f4d35000700 1 -- start start 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d35000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d35000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 0x7f4d3019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d35000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019d000 con 0x7f4d300730f0 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d35000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a0d90 con 0x7f4d30073a00 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49218/0 (socket says 172.21.15.57:49218) 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d2de00700 1 -- 172.21.15.57:0/2415802762 learned_addr learned my addr 172.21.15.57:0/2415802762 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.934+0000 7f4d2d400700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 0x7f4d3019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d2de00700 1 -- 172.21.15.57:0/2415802762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 msgr2=0x7f4d3019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d2de00700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 0x7f4d3019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:21.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d2de00700 1 -- 172.21.15.57:0/2415802762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200095f0 con 0x7f4d300730f0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d2de00700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2001d070 con 0x7f4d300730f0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d301a1070 con 0x7f4d300730f0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d20005dc0 con 0x7f4d300730f0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.935+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d20017400 con 0x7f4d300730f0 2024-02-28T23:47:21.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.936+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d301a15c0 con 0x7f4d300730f0 2024-02-28T23:47:21.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.937+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3010e4d0 con 0x7f4d300730f0 2024-02-28T23:47:21.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.939+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4d2000f7f0 con 0x7f4d300730f0 2024-02-28T23:47:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.940+0000 7f4d26a00700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 0x7f4d1406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.940+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4d2008e940 con 0x7f4d300730f0 2024-02-28T23:47:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.940+0000 7f4d2d400700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 0x7f4d1406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:21.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.941+0000 7f4d2d400700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 0x7f4d1406fd50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005950 tx=0x7f4d1c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:21.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:21.945+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d20058170 con 0x7f4d300730f0 2024-02-28T23:47:22.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.197+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d301a18a0 con 0x7f4d1406d8a0 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.199+0000 7f4d26a00700 1 -- 172.21.15.57:0/2415802762 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4d301a18a0 con 0x7f4d1406d8a0 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:47:22.199 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:47:22.200 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:47:22.200 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:47:22.200 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:47:22.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 msgr2=0x7f4d1406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 0x7f4d1406fd50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005950 tx=0x7f4d1c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 msgr2=0x7f4d3019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d2000be20 comp rx=0 tx=0).stop 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 shutdown_connections 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4d1406d8a0 0x7f4d1406fd50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d300730f0 0x7f4d3019c3e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 --2- 172.21.15.57:0/2415802762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d30073a00 0x7f4d3019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 >> 172.21.15.57:0/2415802762 conn(0x7f4d300fbda0 msgr2=0x7f4d30102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.203+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 shutdown_connections 2024-02-28T23:47:22.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.204+0000 7f4d35000700 1 -- 172.21.15.57:0/2415802762 wait complete. 2024-02-28T23:47:22.260 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:21 smithi084 ceph-mon[70440]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:22.260 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:21 smithi084 ceph-mon[70440]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:22.261 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:21 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4020011833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:47:22.261 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:21 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1855714817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 -- 172.21.15.57:0/2014689168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 msgr2=0x7f29ac100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 --2- 172.21.15.57:0/2014689168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac100040 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2998009910 tx=0x7f2998009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 -- 172.21.15.57:0/2014689168 shutdown_connections 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 --2- 172.21.15.57:0/2014689168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac100040 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 --2- 172.21.15.57:0/2014689168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 -- 172.21.15.57:0/2014689168 >> 172.21.15.57:0/2014689168 conn(0x7f29ac075130 msgr2=0x7f29ac075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:22.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 -- 172.21.15.57:0/2014689168 shutdown_connections 2024-02-28T23:47:22.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.369+0000 7f29b3200700 1 -- 172.21.15.57:0/2014689168 wait complete. 2024-02-28T23:47:22.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.370+0000 7f29b3200700 1 Processor -- start 2024-02-28T23:47:22.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29b3200700 1 -- start start 2024-02-28T23:47:22.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29b3200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:22.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29b3200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:22.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29b3200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac196a40 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29b3200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19a7d0 con 0x7f29ac104130 2024-02-28T23:47:22.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.371+0000 7f29abe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:22.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49234/0 (socket says 172.21.15.57:49234) 2024-02-28T23:47:22.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 -- 172.21.15.57:0/3383153456 learned_addr learned my addr 172.21.15.57:0/3383153456 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:22.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29ab400700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:22.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 -- 172.21.15.57:0/3383153456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 msgr2=0x7f29ac196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 -- 172.21.15.57:0/3383153456 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980095f0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29ab400700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.372+0000 7f29abe00700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f299c00d8d0 tx=0x7f299c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.373+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299c009940 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.373+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f299c010460 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.373+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299c00f5f0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.373+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac19aab0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.373+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac19b000 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.374+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac1086a0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.375+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f299c0105d0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.377+0000 7f29a8c00700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 0x7f299006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.377+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f299c08d4c0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.377+0000 7f29ab400700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 0x7f299006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:22.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.378+0000 7f29ab400700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 0x7f299006fd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2998006010 tx=0x7f299800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:22.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.382+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f299c056c50 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.694+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f29ac04f2a0 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.695+0000 7f29a8c00700 1 -- 172.21.15.57:0/3383153456 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f299c05a310 con 0x7f29ac0ffbd0 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:47:22.695 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:47:22.696 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:47:22.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 msgr2=0x7f299006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 0x7f299006fd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2998006010 tx=0x7f299800b540 comp rx=0 tx=0).stop 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 msgr2=0x7f29ac195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f299c00d8d0 tx=0x7f299c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 shutdown_connections 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f299006d8a0 0x7f299006fd50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac195e20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 --2- 172.21.15.57:0/3383153456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ac104130 0x7f29ac196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.698+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 >> 172.21.15.57:0/3383153456 conn(0x7f29ac075130 msgr2=0x7f29ac0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.699+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 shutdown_connections 2024-02-28T23:47:22.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:22.699+0000 7f29b3200700 1 -- 172.21.15.57:0/3383153456 wait complete. 2024-02-28T23:47:23.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:22 smithi057 ceph-mon[89491]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:23.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3383153456' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:47:23.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:22 smithi084 ceph-mon[70440]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:23.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3383153456' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:47:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:23 smithi057 ceph-mon[89491]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:24.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:23 smithi084 ceph-mon[70440]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:24 smithi057 ceph-mon[89491]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:25.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:24 smithi084 ceph-mon[70440]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:27.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:26 smithi057 ceph-mon[89491]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:27.132 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:26 smithi084 ceph-mon[70440]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:28 smithi057 ceph-mon[89491]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:29.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:28 smithi084 ceph-mon[70440]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:30 smithi057 ceph-mon[89491]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:31.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:30 smithi084 ceph-mon[70440]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:33.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:32 smithi057 ceph-mon[89491]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:32 smithi084 ceph-mon[70440]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:35.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:34 smithi057 ceph-mon[89491]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:35.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:34 smithi084 ceph-mon[70440]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:37.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:36 smithi057 ceph-mon[89491]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:37.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:36 smithi084 ceph-mon[70440]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:39.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:38 smithi057 ceph-mon[89491]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:38 smithi084 ceph-mon[70440]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:41.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:40 smithi057 ceph-mon[89491]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:40 smithi084 ceph-mon[70440]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:42 smithi057 ceph-mon[89491]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:43.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:42 smithi084 ceph-mon[70440]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:45.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:44 smithi057 ceph-mon[89491]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:45.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:44 smithi084 ceph-mon[70440]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:46 smithi057 ceph-mon[89491]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:46 smithi084 ceph-mon[70440]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:48 smithi057 ceph-mon[89491]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:48 smithi084 ceph-mon[70440]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:47:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:50 smithi057 ceph-mon[89491]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:51.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:50 smithi084 ceph-mon[70440]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 -- 172.21.15.57:0/445468549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a48106b80 msgr2=0x7f5a48106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 --2- 172.21.15.57:0/445468549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a48106b80 0x7f5a48106ff0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009910 tx=0x7f5a34009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 -- 172.21.15.57:0/445468549 shutdown_connections 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 --2- 172.21.15.57:0/445468549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a48106b80 0x7f5a48106ff0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 --2- 172.21.15.57:0/445468549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a4810c660 0x7f5a4810ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 -- 172.21.15.57:0/445468549 >> 172.21.15.57:0/445468549 conn(0x7f5a48075e90 msgr2=0x7f5a480782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 -- 172.21.15.57:0/445468549 shutdown_connections 2024-02-28T23:47:52.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.873+0000 7f5a47200700 1 -- 172.21.15.57:0/445468549 wait complete. 2024-02-28T23:47:52.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.875+0000 7f5a47200700 1 Processor -- start 2024-02-28T23:47:52.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.875+0000 7f5a47200700 1 -- start start 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a47200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a48106b80 0x7f5a4819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a47200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a47200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4819ce00 con 0x7f5a4810c660 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a47200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481a0b90 con 0x7f5a48106b80 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:52.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49968/0 (socket says 172.21.15.57:49968) 2024-02-28T23:47:52.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 -- 172.21.15.57:0/1968006504 learned_addr learned my addr 172.21.15.57:0/1968006504 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:52.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45e00700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a48106b80 0x7f5a4819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:52.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 -- 172.21.15.57:0/1968006504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a48106b80 msgr2=0x7f5a4819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a48106b80 0x7f5a4819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.876+0000 7f5a45400700 1 -- 172.21.15.57:0/1968006504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340095f0 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a45400700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3401d070 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3400f460 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a481a0e10 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a34017630 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.877+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a481a1300 con 0x7f5a4810c660 2024-02-28T23:47:52.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.879+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4810eb40 con 0x7f5a4810c660 2024-02-28T23:47:52.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.880+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5a3400f5d0 con 0x7f5a4810c660 2024-02-28T23:47:52.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.881+0000 7f5a3ea00700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 0x7f5a2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:52.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.881+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a3408ea40 con 0x7f5a4810c660 2024-02-28T23:47:52.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.881+0000 7f5a45e00700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 0x7f5a2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:52.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.882+0000 7f5a45e00700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 0x7f5a2c06fd50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005950 tx=0x7f5a380058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:52.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:52.886+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a34058150 con 0x7f5a4810c660 2024-02-28T23:47:53.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.137+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a4802cca0 con 0x7f5a2c06d8a0 2024-02-28T23:47:53.138 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:52 smithi057 ceph-mon[89491]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:53.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.140+0000 7f5a3ea00700 1 -- 172.21.15.57:0/1968006504 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a4802cca0 con 0x7f5a2c06d8a0 2024-02-28T23:47:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 msgr2=0x7f5a2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 0x7f5a2c06fd50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005950 tx=0x7f5a380058e0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 msgr2=0x7f5a4819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3400be20 comp rx=0 tx=0).stop 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 shutdown_connections 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5a2c06d8a0 0x7f5a2c06fd50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a48106b80 0x7f5a4819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 --2- 172.21.15.57:0/1968006504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a4810c660 0x7f5a4819c720 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.143+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 >> 172.21.15.57:0/1968006504 conn(0x7f5a48075e90 msgr2=0x7f5a480778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:53.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.144+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 shutdown_connections 2024-02-28T23:47:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.144+0000 7f5a47200700 1 -- 172.21.15.57:0/1968006504 wait complete. 2024-02-28T23:47:53.161 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 -- 172.21.15.57:0/3849061391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc01003e0 msgr2=0x7fadc01007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 --2- 172.21.15.57:0/3849061391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc01003e0 0x7fadc01007b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b0a0 tx=0x7fadb400b3b0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 -- 172.21.15.57:0/3849061391 shutdown_connections 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 --2- 172.21.15.57:0/3849061391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc0101c40 0x7fadc01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 --2- 172.21.15.57:0/3849061391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc01003e0 0x7fadc01007b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 -- 172.21.15.57:0/3849061391 >> 172.21.15.57:0/3849061391 conn(0x7fadc00fbbc0 msgr2=0x7fadc00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:53.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 -- 172.21.15.57:0/3849061391 shutdown_connections 2024-02-28T23:47:53.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.305+0000 7fadc6a00700 1 -- 172.21.15.57:0/3849061391 wait complete. 2024-02-28T23:47:53.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.306+0000 7fadc6a00700 1 Processor -- start 2024-02-28T23:47:53.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.306+0000 7fadc6a00700 1 -- start start 2024-02-28T23:47:53.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc6a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc01003e0 0x7fadc0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc6a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0198b00 con 0x7fadc0101c40 2024-02-28T23:47:53.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc6a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019c890 con 0x7fadc01003e0 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49990/0 (socket says 172.21.15.57:49990) 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.307+0000 7fadc4c00700 1 -- 172.21.15.57:0/2541244587 learned_addr learned my addr 172.21.15.57:0/2541244587 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.308+0000 7fadc5600700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc01003e0 0x7fadc0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.308+0000 7fadc4c00700 1 -- 172.21.15.57:0/2541244587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc01003e0 msgr2=0x7fadc0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.308+0000 7fadc4c00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc01003e0 0x7fadc0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.308+0000 7fadc4c00700 1 -- 172.21.15.57:0/2541244587 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009d00 con 0x7fadc0101c40 2024-02-28T23:47:53.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.308+0000 7fadc4c00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c8a0 tx=0x7fadac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:53.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.309+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadac00cea0 con 0x7fadc0101c40 2024-02-28T23:47:53.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.309+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadac004830 con 0x7fadc0101c40 2024-02-28T23:47:53.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.309+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadac0055b0 con 0x7fadc0101c40 2024-02-28T23:47:53.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.309+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc019cb70 con 0x7fadc0101c40 2024-02-28T23:47:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.309+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc019d140 con 0x7fadc0101c40 2024-02-28T23:47:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.310+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc0062380 con 0x7fadc0101c40 2024-02-28T23:47:53.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.311+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fadac0049a0 con 0x7fadc0101c40 2024-02-28T23:47:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.312+0000 7fadbe000700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 0x7fada806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.312+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fadac08d190 con 0x7fadc0101c40 2024-02-28T23:47:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.313+0000 7fadc5600700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 0x7fada806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.313+0000 7fadc5600700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 0x7fada806fd50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009480 tx=0x7fadb400b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:53.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.317+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadac056920 con 0x7fadc0101c40 2024-02-28T23:47:53.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:52 smithi084 ceph-mon[70440]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.569+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadc019d4f0 con 0x7fada806d8a0 2024-02-28T23:47:53.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.571+0000 7fadbe000700 1 -- 172.21.15.57:0/2541244587 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fadc019d4f0 con 0x7fada806d8a0 2024-02-28T23:47:53.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 msgr2=0x7fada806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 0x7fada806fd50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009480 tx=0x7fadb400b7f0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 msgr2=0x7fadc0198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c8a0 tx=0x7fadac00cc60 comp rx=0 tx=0).stop 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 shutdown_connections 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fada806d8a0 0x7fada806fd50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc01003e0 0x7fadc0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 --2- 172.21.15.57:0/2541244587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadc0101c40 0x7fadc0198420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.574+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 >> 172.21.15.57:0/2541244587 conn(0x7fadc00fbbc0 msgr2=0x7fadc0072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.575+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 shutdown_connections 2024-02-28T23:47:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.575+0000 7fadc6a00700 1 -- 172.21.15.57:0/2541244587 wait complete. 2024-02-28T23:47:53.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 -- 172.21.15.57:0/452590778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d301004e0 msgr2=0x7f8d301008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 --2- 172.21.15.57:0/452590778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d301004e0 0x7f8d301008b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009910 tx=0x7f8d20009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:53.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 -- 172.21.15.57:0/452590778 shutdown_connections 2024-02-28T23:47:53.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 --2- 172.21.15.57:0/452590778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d30100e80 0x7f8d30104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 --2- 172.21.15.57:0/452590778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d301004e0 0x7f8d301008b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 -- 172.21.15.57:0/452590778 >> 172.21.15.57:0/452590778 conn(0x7f8d300fbda0 msgr2=0x7f8d300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:53.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 -- 172.21.15.57:0/452590778 shutdown_connections 2024-02-28T23:47:53.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.740+0000 7f8d35000700 1 -- 172.21.15.57:0/452590778 wait complete. 2024-02-28T23:47:53.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.741+0000 7f8d35000700 1 Processor -- start 2024-02-28T23:47:53.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.741+0000 7f8d35000700 1 -- start start 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d35000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d301004e0 0x7f8d301980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d35000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d35000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30198d10 con 0x7f8d30100e80 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d35000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3019caa0 con 0x7f8d301004e0 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50004/0 (socket says 172.21.15.57:50004) 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2d400700 1 -- 172.21.15.57:0/2264985904 learned_addr learned my addr 172.21.15.57:0/2264985904 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:53.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2de00700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d301004e0 0x7f8d301980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2d400700 1 -- 172.21.15.57:0/2264985904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d301004e0 msgr2=0x7f8d301980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:53.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.742+0000 7f8d2d400700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d301004e0 0x7f8d301980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d2d400700 1 -- 172.21.15.57:0/2264985904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200095f0 con 0x7f8d30100e80 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d2d400700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c8f0 tx=0x7f8d1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c007a50 con 0x7f8d30100e80 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d1c007bb0 con 0x7f8d30100e80 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c018630 con 0x7f8d30100e80 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3019cd80 con 0x7f8d30100e80 2024-02-28T23:47:53.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.743+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3019d2d0 con 0x7f8d30100e80 2024-02-28T23:47:53.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.745+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d301045d0 con 0x7f8d30100e80 2024-02-28T23:47:53.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.746+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8d1c007d20 con 0x7f8d30100e80 2024-02-28T23:47:53.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.747+0000 7f8d26a00700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 0x7f8d1406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:53.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.747+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d1c08d4a0 con 0x7f8d30100e80 2024-02-28T23:47:53.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.747+0000 7f8d2de00700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 0x7f8d1406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:53.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.748+0000 7f8d2de00700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 0x7f8d1406fd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006010 tx=0x7f8d2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:53.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.752+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d1c056c30 con 0x7f8d30100e80 2024-02-28T23:47:53.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:53.992+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d3019d5a0 con 0x7f8d1406d8a0 2024-02-28T23:47:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.007+0000 7f8d26a00700 1 -- 172.21.15.57:0/2264985904 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8d3019d5a0 con 0x7f8d1406d8a0 2024-02-28T23:47:54.008 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (37m) 13m ago 51m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (47m) 13m ago 51m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (50m) 13m ago 50m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (33m) 13m ago 51m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (33m) 13m ago 50m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (36m) 13m ago 50m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (29m) 13m ago 44m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (29m) 13m ago 44m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (13m) 13m ago 43m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (28m) 13m ago 44m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (39m) 13m ago 53m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (38m) 13m ago 49m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:47:54.009 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (35m) 13m ago 53m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (34m) 13m ago 49m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (38m) 13m ago 51m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (38m) 13m ago 50m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (32m) 13m ago 48m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (32m) 13m ago 48m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (31m) 13m ago 47m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (31m) 13m ago 47m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (30m) 13m ago 46m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (29m) 13m ago 45m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:47:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (37m) 13m ago 50m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:47:54.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.010+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 msgr2=0x7f8d1406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.010+0000 7f8d35000700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 0x7f8d1406fd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006010 tx=0x7f8d2001a040 comp rx=0 tx=0).stop 2024-02-28T23:47:54.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 msgr2=0x7f8d30198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c8f0 tx=0x7f8d1c00ccb0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 shutdown_connections 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8d1406d8a0 0x7f8d1406fd50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d301004e0 0x7f8d301980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 --2- 172.21.15.57:0/2264985904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d30100e80 0x7f8d30198630 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 >> 172.21.15.57:0/2264985904 conn(0x7f8d300fbda0 msgr2=0x7f8d300fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 shutdown_connections 2024-02-28T23:47:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.011+0000 7f8d35000700 1 -- 172.21.15.57:0/2264985904 wait complete. 2024-02-28T23:47:54.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 -- 172.21.15.57:0/773607281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 msgr2=0x7f4b041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 --2- 172.21.15.57:0/773607281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b041028a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009910 tx=0x7f4af4009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:54.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 -- 172.21.15.57:0/773607281 shutdown_connections 2024-02-28T23:47:54.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 --2- 172.21.15.57:0/773607281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 --2- 172.21.15.57:0/773607281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b041028a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 -- 172.21.15.57:0/773607281 >> 172.21.15.57:0/773607281 conn(0x7f4b040fdd90 msgr2=0x7f4b041001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:54.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 -- 172.21.15.57:0/773607281 shutdown_connections 2024-02-28T23:47:54.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.173+0000 7f4b09000700 1 -- 172.21.15.57:0/773607281 wait complete. 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.174+0000 7f4b09000700 1 Processor -- start 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.174+0000 7f4b09000700 1 -- start start 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b09000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b09000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b09000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04198a20 con 0x7f4b041024d0 2024-02-28T23:47:54.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b09000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0419c7b0 con 0x7f4b04102e70 2024-02-28T23:47:54.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b03400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50034/0 (socket says 172.21.15.57:50034) 2024-02-28T23:47:54.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.175+0000 7f4b03400700 1 -- 172.21.15.57:0/2804752831 learned_addr learned my addr 172.21.15.57:0/2804752831 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b02a00700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b03400700 1 -- 172.21.15.57:0/2804752831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 msgr2=0x7f4b04198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b03400700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b03400700 1 -- 172.21.15.57:0/2804752831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40095f0 con 0x7f4b041024d0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b02a00700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b03400700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4af400b5c0 tx=0x7f4af400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af401d070 con 0x7f4b041024d0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af4005e00 con 0x7f4b041024d0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0419ca30 con 0x7f4b041024d0 2024-02-28T23:47:54.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af4017400 con 0x7f4b041024d0 2024-02-28T23:47:54.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.176+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0419cf20 con 0x7f4b041024d0 2024-02-28T23:47:54.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.178+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0410a7e0 con 0x7f4b041024d0 2024-02-28T23:47:54.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.179+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4af400f460 con 0x7f4b041024d0 2024-02-28T23:47:54.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.179+0000 7f4afbe00700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 0x7f4ae806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.180+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4af408ea10 con 0x7f4b041024d0 2024-02-28T23:47:54.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.180+0000 7f4b02a00700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 0x7f4ae806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.180+0000 7f4b02a00700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 0x7f4ae806fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0005f00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:54.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.184+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4af40581a0 con 0x7f4b041024d0 2024-02-28T23:47:54.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:54 smithi057 ceph-mon[89491]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:54.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:54 smithi084 ceph-mon[70440]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.512+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b04062380 con 0x7f4b041024d0 2024-02-28T23:47:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.513+0000 7f4afbe00700 1 -- 172.21.15.57:0/2804752831 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4af405b860 con 0x7f4b041024d0 2024-02-28T23:47:54.513 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:47:54.514 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:47:54.515 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 msgr2=0x7f4ae806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 0x7f4ae806fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0005f00 comp rx=0 tx=0).stop 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 msgr2=0x7f4b04197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4af400b5c0 tx=0x7f4af400be20 comp rx=0 tx=0).stop 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 shutdown_connections 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4ae806d8a0 0x7f4ae806fd50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b041024d0 0x7f4b04197e00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 --2- 172.21.15.57:0/2804752831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b04102e70 0x7f4b04198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 >> 172.21.15.57:0/2804752831 conn(0x7f4b040fdd90 msgr2=0x7f4b040ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.516+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 shutdown_connections 2024-02-28T23:47:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.517+0000 7f4b09000700 1 -- 172.21.15.57:0/2804752831 wait complete. 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 -- 172.21.15.57:0/2818392487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c10c660 msgr2=0x7f0b5c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2818392487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c10c660 0x7f0b5c10ca30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009910 tx=0x7f0b50009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 -- 172.21.15.57:0/2818392487 shutdown_connections 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2818392487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c106b80 0x7f0b5c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2818392487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c10c660 0x7f0b5c10ca30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 -- 172.21.15.57:0/2818392487 >> 172.21.15.57:0/2818392487 conn(0x7f0b5c075e90 msgr2=0x7f0b5c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.673+0000 7f0b62a00700 1 -- 172.21.15.57:0/2818392487 shutdown_connections 2024-02-28T23:47:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.674+0000 7f0b62a00700 1 -- 172.21.15.57:0/2818392487 wait complete. 2024-02-28T23:47:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 Processor -- start 2024-02-28T23:47:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 -- start start 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c10c660 0x7f0b5c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c19cea0 con 0x7f0b5c106b80 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.675+0000 7f0b62a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1a0c30 con 0x7f0b5c10c660 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50056/0 (socket says 172.21.15.57:50056) 2024-02-28T23:47:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 -- 172.21.15.57:0/2457649290 learned_addr learned my addr 172.21.15.57:0/2457649290 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b60c00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c10c660 0x7f0b5c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 -- 172.21.15.57:0/2457649290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c10c660 msgr2=0x7f0b5c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c10c660 0x7f0b5c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.676+0000 7f0b61600700 1 -- 172.21.15.57:0/2457649290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b500095f0 con 0x7f0b5c106b80 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b61600700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b500092e0 tx=0x7f0b5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5001d070 con 0x7f0b5c106b80 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c1a0eb0 con 0x7f0b5c106b80 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b50005e00 con 0x7f0b5c106b80 2024-02-28T23:47:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b50017400 con 0x7f0b5c106b80 2024-02-28T23:47:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.677+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1a13a0 con 0x7f0b5c106b80 2024-02-28T23:47:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.678+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c10eb60 con 0x7f0b5c106b80 2024-02-28T23:47:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.679+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0b5000f460 con 0x7f0b5c106b80 2024-02-28T23:47:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.680+0000 7f0b5a000700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 0x7f0b4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.680+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0b5008eb00 con 0x7f0b5c106b80 2024-02-28T23:47:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.680+0000 7f0b60c00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 0x7f0b4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:54.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.681+0000 7f0b60c00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 0x7f0b4406fd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005950 tx=0x7f0b480058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:54.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.685+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b50058210 con 0x7f0b5c106b80 2024-02-28T23:47:54.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.960+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b5c19d6f0 con 0x7f0b5c106b80 2024-02-28T23:47:54.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.961+0000 7f0b5a000700 1 -- 172.21.15.57:0/2457649290 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f0b5005b8d0 con 0x7f0b5c106b80 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:47:54.962 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:47:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:54.964 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:47:54.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.964+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 msgr2=0x7f0b4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.964+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 0x7f0b4406fd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005950 tx=0x7f0b480058e0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.964+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 msgr2=0x7f0b5c19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.964+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b500092e0 tx=0x7f0b5000be20 comp rx=0 tx=0).stop 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 shutdown_connections 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0b4406d8a0 0x7f0b4406fd50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b5c106b80 0x7f0b5c19c280 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 --2- 172.21.15.57:0/2457649290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c10c660 0x7f0b5c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 >> 172.21.15.57:0/2457649290 conn(0x7f0b5c075e90 msgr2=0x7f0b5c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 shutdown_connections 2024-02-28T23:47:54.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:54.965+0000 7f0b62a00700 1 -- 172.21.15.57:0/2457649290 wait complete. 2024-02-28T23:47:54.967 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 -- 172.21.15.57:0/4256637796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 msgr2=0x7f0bc01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/4256637796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01008b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009910 tx=0x7f0bb0009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 -- 172.21.15.57:0/4256637796 shutdown_connections 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/4256637796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 0x7f0bc0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/4256637796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01008b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 -- 172.21.15.57:0/4256637796 >> 172.21.15.57:0/4256637796 conn(0x7f0bc00fbda0 msgr2=0x7f0bc00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 -- 172.21.15.57:0/4256637796 shutdown_connections 2024-02-28T23:47:55.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.128+0000 7f0bc6c00700 1 -- 172.21.15.57:0/4256637796 wait complete. 2024-02-28T23:47:55.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.129+0000 7f0bc6c00700 1 Processor -- start 2024-02-28T23:47:55.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bc6c00700 1 -- start start 2024-02-28T23:47:55.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bc6c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bc6c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 0x7f0bc0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bc6c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0198d10 con 0x7f0bc01004e0 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bc6c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc019caa0 con 0x7f0bc0100e80 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bbfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bbfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41988/0 (socket says 172.21.15.57:41988) 2024-02-28T23:47:55.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.130+0000 7f0bbfe00700 1 -- 172.21.15.57:0/2677637774 learned_addr learned my addr 172.21.15.57:0/2677637774 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.131+0000 7f0bbf400700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 0x7f0bc0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.131+0000 7f0bbfe00700 1 -- 172.21.15.57:0/2677637774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 msgr2=0x7f0bc0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.131+0000 7f0bbfe00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 0x7f0bc0198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.131+0000 7f0bbfe00700 1 -- 172.21.15.57:0/2677637774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00095f0 con 0x7f0bc01004e0 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.131+0000 7f0bbfe00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b5c0 tx=0x7f0bb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.132+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb001d070 con 0x7f0bc01004e0 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.132+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bb0005e00 con 0x7f0bc01004e0 2024-02-28T23:47:55.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.132+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb0017400 con 0x7f0bc01004e0 2024-02-28T23:47:55.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.132+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc019cd20 con 0x7f0bc01004e0 2024-02-28T23:47:55.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.132+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc019d210 con 0x7f0bc01004e0 2024-02-28T23:47:55.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.133+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc01045d0 con 0x7f0bc01004e0 2024-02-28T23:47:55.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.134+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0bb000f460 con 0x7f0bc01004e0 2024-02-28T23:47:55.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.135+0000 7f0bbcc00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 0x7f0ba406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.135+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0bb008ea10 con 0x7f0bc01004e0 2024-02-28T23:47:55.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.135+0000 7f0bbf400700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 0x7f0ba406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.136+0000 7f0bbf400700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 0x7f0ba406fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005950 tx=0x7f0bac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:55.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.140+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bb00581a0 con 0x7f0bc01004e0 2024-02-28T23:47:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:55 smithi057 ceph-mon[89491]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:55 smithi057 ceph-mon[89491]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:55 smithi057 ceph-mon[89491]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:55 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2804752831' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:47:55.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:55 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2457649290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:47:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:55 smithi084 ceph-mon[70440]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:55 smithi084 ceph-mon[70440]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:55 smithi084 ceph-mon[70440]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:55 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2804752831' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:47:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:55 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2457649290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:47:55.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.398+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bc019d540 con 0x7f0ba406d8a0 2024-02-28T23:47:55.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.400+0000 7f0bbcc00700 1 -- 172.21.15.57:0/2677637774 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0bc019d540 con 0x7f0ba406d8a0 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:47:55.401 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:47:55.402 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:47:55.402 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:47:55.402 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:47:55.402 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.403+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 msgr2=0x7f0ba406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.403+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 0x7f0ba406fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005950 tx=0x7f0bac0058e0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.403+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 msgr2=0x7f0bc01980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.403+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b5c0 tx=0x7f0bb000be20 comp rx=0 tx=0).stop 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 shutdown_connections 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ba406d8a0 0x7f0ba406fd50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bc01004e0 0x7f0bc01980f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 --2- 172.21.15.57:0/2677637774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc0100e80 0x7f0bc0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 >> 172.21.15.57:0/2677637774 conn(0x7f0bc00fbda0 msgr2=0x7f0bc00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:55.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 shutdown_connections 2024-02-28T23:47:55.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.404+0000 7f0bc6c00700 1 -- 172.21.15.57:0/2677637774 wait complete. 2024-02-28T23:47:55.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 -- 172.21.15.57:0/1212597370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 msgr2=0x7fdef01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 --2- 172.21.15.57:0/1212597370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef01008b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009910 tx=0x7fdee0009c20 comp rx=0 tx=0).stop 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 -- 172.21.15.57:0/1212597370 shutdown_connections 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 --2- 172.21.15.57:0/1212597370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 --2- 172.21.15.57:0/1212597370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef01008b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 -- 172.21.15.57:0/1212597370 >> 172.21.15.57:0/1212597370 conn(0x7fdef00fbda0 msgr2=0x7fdef00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 -- 172.21.15.57:0/1212597370 shutdown_connections 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.564+0000 7fdef4c00700 1 -- 172.21.15.57:0/1212597370 wait complete. 2024-02-28T23:47:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.565+0000 7fdef4c00700 1 Processor -- start 2024-02-28T23:47:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.565+0000 7fdef4c00700 1 -- start start 2024-02-28T23:47:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdef4c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdef4c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdef4c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0073ad0 con 0x7fdef01004e0 2024-02-28T23:47:55.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdef4c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0073c10 con 0x7fdef0100e80 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdeede00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdeede00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42010/0 (socket says 172.21.15.57:42010) 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdeede00700 1 -- 172.21.15.57:0/397035265 learned_addr learned my addr 172.21.15.57:0/397035265 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.566+0000 7fdeed400700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdeede00700 1 -- 172.21.15.57:0/397035265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 msgr2=0x7fdef0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdeede00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdeede00700 1 -- 172.21.15.57:0/397035265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00095f0 con 0x7fdef01004e0 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdeed400700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:47:55.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdeede00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdee000f690 tx=0x7fdee000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:55.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee001d070 con 0x7fdef01004e0 2024-02-28T23:47:55.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdee00050d0 con 0x7fdef01004e0 2024-02-28T23:47:55.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef0073e90 con 0x7fdef01004e0 2024-02-28T23:47:55.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee0022470 con 0x7fdef01004e0 2024-02-28T23:47:55.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.567+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef01a23f0 con 0x7fdef01004e0 2024-02-28T23:47:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.568+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef01087f0 con 0x7fdef01004e0 2024-02-28T23:47:55.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.570+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdee0003680 con 0x7fdef01004e0 2024-02-28T23:47:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.571+0000 7fdee6a00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:47:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.571+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdee008e9d0 con 0x7fdef01004e0 2024-02-28T23:47:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.571+0000 7fdeed400700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:47:55.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.572+0000 7fdeed400700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdef0074840 tx=0x7fdedc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:47:55.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.576+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdee00580e0 con 0x7fdef01004e0 2024-02-28T23:47:55.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.890+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdef0074690 con 0x7fdef01004e0 2024-02-28T23:47:55.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.891+0000 7fdee6a00700 1 -- 172.21.15.57:0/397035265 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fdee005f0e0 con 0x7fdef01004e0 2024-02-28T23:47:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:47:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:47:55.892 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:47:55.893 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:47:55.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 msgr2=0x7fded406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdef0074840 tx=0x7fdedc009260 comp rx=0 tx=0).stop 2024-02-28T23:47:55.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 msgr2=0x7fdef0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdee000f690 tx=0x7fdee000f770 comp rx=0 tx=0).stop 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 shutdown_connections 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fded406d8a0 0x7fded406fd50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdef01004e0 0x7fdef0073050 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 --2- 172.21.15.57:0/397035265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdef0100e80 0x7fdef0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 >> 172.21.15.57:0/397035265 conn(0x7fdef00fbda0 msgr2=0x7fdef00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 shutdown_connections 2024-02-28T23:47:55.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:47:55.894+0000 7fdef4c00700 1 -- 172.21.15.57:0/397035265 wait complete. 2024-02-28T23:47:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:56 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/397035265' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:47:56.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:56 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/397035265' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:47:57.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:57 smithi057 ceph-mon[89491]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:57.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:57 smithi057 ceph-mon[89491]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:57 smithi084 ceph-mon[70440]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:47:57.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:57 smithi084 ceph-mon[70440]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:59.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:47:59 smithi057 ceph-mon[89491]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:47:59.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:47:59 smithi084 ceph-mon[70440]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:01.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:01 smithi057 ceph-mon[89491]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:01 smithi084 ceph-mon[70440]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:03 smithi057 ceph-mon[89491]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:03 smithi084 ceph-mon[70440]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:05 smithi057 ceph-mon[89491]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:05 smithi084 ceph-mon[70440]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:07 smithi057 ceph-mon[89491]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:07 smithi084 ceph-mon[70440]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:09.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:09 smithi084 ceph-mon[70440]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:09.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:09 smithi057 ceph-mon[89491]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:11 smithi057 ceph-mon[89491]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:11 smithi084 ceph-mon[70440]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:13 smithi057 ceph-mon[89491]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:13.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:13 smithi084 ceph-mon[70440]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:15.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:14 smithi057 ceph-mon[89491]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:15.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:14 smithi084 ceph-mon[70440]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:17.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:16 smithi057 ceph-mon[89491]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:17.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:16 smithi084 ceph-mon[70440]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:18.995 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:18 smithi084 ceph-mon[70440]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:18.995 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:18 smithi057 ceph-mon[89491]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:19.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:21.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:20 smithi057 ceph-mon[89491]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:21.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:20 smithi084 ceph-mon[70440]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:23.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:22 smithi057 ceph-mon[89491]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:23.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:22 smithi084 ceph-mon[70440]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:24 smithi057 ceph-mon[89491]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:25.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:24 smithi084 ceph-mon[70440]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 -- 172.21.15.57:0/508303646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 msgr2=0x7efee8100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 --2- 172.21.15.57:0/508303646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8100900 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b0a0 tx=0x7efedc00b3b0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 -- 172.21.15.57:0/508303646 shutdown_connections 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 --2- 172.21.15.57:0/508303646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 0x7efee8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 --2- 172.21.15.57:0/508303646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8100900 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 -- 172.21.15.57:0/508303646 >> 172.21.15.57:0/508303646 conn(0x7efee80fbe10 msgr2=0x7efee80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 -- 172.21.15.57:0/508303646 shutdown_connections 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.073+0000 7efef0400700 1 -- 172.21.15.57:0/508303646 wait complete. 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.074+0000 7efef0400700 1 Processor -- start 2024-02-28T23:48:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.074+0000 7efef0400700 1 -- start start 2024-02-28T23:48:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efef0400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efef0400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 0x7efee8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efef0400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8198d50 con 0x7efee8100530 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efef0400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee819cae0 con 0x7efee8100ed0 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeeda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeeda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40626/0 (socket says 172.21.15.57:40626) 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeeda00700 1 -- 172.21.15.57:0/1250086814 learned_addr learned my addr 172.21.15.57:0/1250086814 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeed000700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 0x7efee8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeeda00700 1 -- 172.21.15.57:0/1250086814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 msgr2=0x7efee8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.075+0000 7efeeda00700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 0x7efee8198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efeeda00700 1 -- 172.21.15.57:0/1250086814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009d00 con 0x7efee8100530 2024-02-28T23:48:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efeeda00700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b070 tx=0x7efedc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efedc00e040 con 0x7efee8100530 2024-02-28T23:48:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efedc003de0 con 0x7efee8100530 2024-02-28T23:48:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee819cd60 con 0x7efee8100530 2024-02-28T23:48:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efedc01d8e0 con 0x7efee8100530 2024-02-28T23:48:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.076+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee819d250 con 0x7efee8100530 2024-02-28T23:48:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.078+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee8104620 con 0x7efee8100530 2024-02-28T23:48:26.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.079+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efedc004040 con 0x7efee8100530 2024-02-28T23:48:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.080+0000 7efee6000700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 0x7efed006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.080+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efedc08ece0 con 0x7efee8100530 2024-02-28T23:48:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.080+0000 7efeed000700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 0x7efed006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.081+0000 7efeed000700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 0x7efed006fd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efed4005950 tx=0x7efed40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.085+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efedc058470 con 0x7efee8100530 2024-02-28T23:48:26.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.336+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efee819d580 con 0x7efed006d8a0 2024-02-28T23:48:26.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.340+0000 7efee6000700 1 -- 172.21.15.57:0/1250086814 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efee819d580 con 0x7efed006d8a0 2024-02-28T23:48:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 msgr2=0x7efed006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 0x7efed006fd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efed4005950 tx=0x7efed40058e0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 msgr2=0x7efee8198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b070 tx=0x7efedc00cdd0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 shutdown_connections 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efed006d8a0 0x7efed006fd50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8100530 0x7efee8198130 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 --2- 172.21.15.57:0/1250086814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8100ed0 0x7efee8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 >> 172.21.15.57:0/1250086814 conn(0x7efee80fbe10 msgr2=0x7efee80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 shutdown_connections 2024-02-28T23:48:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.343+0000 7efef0400700 1 -- 172.21.15.57:0/1250086814 wait complete. 2024-02-28T23:48:26.358 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.505+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2121784521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 msgr2=0x7f8bf410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.505+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2121784521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf410c5e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009940 tx=0x7f8bdc009c50 comp rx=0 tx=0).stop 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2121784521 shutdown_connections 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2121784521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf410c5e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2121784521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 0x7f8bf4073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2121784521 >> 172.21.15.57:0/2121784521 conn(0x7f8bf40fbda0 msgr2=0x7f8bf40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2121784521 shutdown_connections 2024-02-28T23:48:26.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.506+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2121784521 wait complete. 2024-02-28T23:48:26.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 Processor -- start 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 -- start start 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 0x7f8bf41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4198dd0 con 0x7f8bf40739a0 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.507+0000 7f8bf9c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf419cb60 con 0x7f8bf4073090 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40630/0 (socket says 172.21.15.57:40630) 2024-02-28T23:48:26.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 -- 172.21.15.57:0/2330035469 learned_addr learned my addr 172.21.15.57:0/2330035469 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:26.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bf2a00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 0x7f8bf41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 -- 172.21.15.57:0/2330035469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 msgr2=0x7f8bf41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 0x7f8bf41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 -- 172.21.15.57:0/2330035469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0095f0 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.508+0000 7f8bea000700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0055f0 tx=0x7f8bdc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.509+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc01d070 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.509+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bdc00f460 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.509+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf419cde0 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.509+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc021640 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.509+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf419d2d0 con 0x7f8bf40739a0 2024-02-28T23:48:26.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.510+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4109ce0 con 0x7f8bf40739a0 2024-02-28T23:48:26.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.511+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8bdc00fa60 con 0x7f8bf40739a0 2024-02-28T23:48:26.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.512+0000 7f8bebe00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 0x7f8bd806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.513+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8bdc08e730 con 0x7f8bf40739a0 2024-02-28T23:48:26.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.513+0000 7f8bf2a00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 0x7f8bd806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.513+0000 7f8bf2a00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 0x7f8bd806fd50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009d10 tx=0x7f8be4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.517+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bdc057ec0 con 0x7f8bf40739a0 2024-02-28T23:48:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.763+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bf4199620 con 0x7f8bd806d8a0 2024-02-28T23:48:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.765+0000 7f8bebe00700 1 -- 172.21.15.57:0/2330035469 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8bf4199620 con 0x7f8bd806d8a0 2024-02-28T23:48:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 msgr2=0x7f8bd806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 0x7f8bd806fd50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009d10 tx=0x7f8be4009040 comp rx=0 tx=0).stop 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 msgr2=0x7f8bf41986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0055f0 tx=0x7f8bdc0056d0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 shutdown_connections 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f8bd806d8a0 0x7f8bd806fd50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bf4073090 0x7f8bf41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 --2- 172.21.15.57:0/2330035469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf40739a0 0x7f8bf41986f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 >> 172.21.15.57:0/2330035469 conn(0x7f8bf40fbda0 msgr2=0x7f8bf4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 shutdown_connections 2024-02-28T23:48:26.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.768+0000 7f8bf9c00700 1 -- 172.21.15.57:0/2330035469 wait complete. 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 -- 172.21.15.57:0/1576521234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 msgr2=0x7f56cc103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 --2- 172.21.15.57:0/1576521234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc103e10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009910 tx=0x7f56b8009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 -- 172.21.15.57:0/1576521234 shutdown_connections 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 --2- 172.21.15.57:0/1576521234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc103e10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 --2- 172.21.15.57:0/1576521234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 0x7f56cc0ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.919+0000 7f56d2400700 1 -- 172.21.15.57:0/1576521234 >> 172.21.15.57:0/1576521234 conn(0x7f56cc0faa90 msgr2=0x7f56cc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:26.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.920+0000 7f56d2400700 1 -- 172.21.15.57:0/1576521234 shutdown_connections 2024-02-28T23:48:26.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.920+0000 7f56d2400700 1 -- 172.21.15.57:0/1576521234 wait complete. 2024-02-28T23:48:26.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.921+0000 7f56d2400700 1 Processor -- start 2024-02-28T23:48:26.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.921+0000 7f56d2400700 1 -- start start 2024-02-28T23:48:26.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.921+0000 7f56d2400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 0x7f56cc19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.921+0000 7f56d2400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56d2400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc19cf50 con 0x7f56cc0ff930 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56d2400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1a0ce0 con 0x7f56cc0ff020 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40644/0 (socket says 172.21.15.57:40644) 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 -- 172.21.15.57:0/3596270870 learned_addr learned my addr 172.21.15.57:0/3596270870 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 -- 172.21.15.57:0/3596270870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 msgr2=0x7f56cc19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56d1000700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 0x7f56cc19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 0x7f56cc19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.922+0000 7f56cbe00700 1 -- 172.21.15.57:0/3596270870 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80095f0 con 0x7f56cc0ff930 2024-02-28T23:48:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56cbe00700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b801d070 con 0x7f56cc0ff930 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56b8004fb0 con 0x7f56cc0ff930 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b8022ea0 con 0x7f56cc0ff930 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc1a0f60 con 0x7f56cc0ff930 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.923+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc1a1450 con 0x7f56cc0ff930 2024-02-28T23:48:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.924+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc1014a0 con 0x7f56cc0ff930 2024-02-28T23:48:26.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.925+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f56b8022470 con 0x7f56cc0ff930 2024-02-28T23:48:26.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.926+0000 7f56c9600700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 0x7f56b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:26.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.926+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f56b808ecc0 con 0x7f56cc0ff930 2024-02-28T23:48:26.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.926+0000 7f56d1000700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 0x7f56b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:26.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.927+0000 7f56d1000700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 0x7f56b406fd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006fd0 tx=0x7f56c0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:26.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:26.931+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56b8094020 con 0x7f56cc0ff930 2024-02-28T23:48:27.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:26 smithi057 ceph-mon[89491]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:27.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:26 smithi084 ceph-mon[70440]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:27.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.161+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f56cc02cca0 con 0x7f56b406d8a0 2024-02-28T23:48:27.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.194+0000 7f56c9600700 1 -- 172.21.15.57:0/3596270870 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f56cc02cca0 con 0x7f56b406d8a0 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (38m) 13m ago 51m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (48m) 13m ago 52m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (50m) 13m ago 50m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (34m) 13m ago 52m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:48:27.194 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (34m) 13m ago 50m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (37m) 13m ago 51m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (30m) 13m ago 44m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (29m) 13m ago 44m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (13m) 13m ago 44m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (29m) 13m ago 44m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (40m) 13m ago 53m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (39m) 13m ago 50m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (35m) 13m ago 54m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (35m) 13m ago 50m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (39m) 13m ago 52m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (38m) 13m ago 50m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (33m) 13m ago 49m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (32m) 13m ago 48m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (32m) 13m ago 48m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:48:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (31m) 13m ago 47m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:48:27.196 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (31m) 13m ago 47m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:48:27.196 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (30m) 13m ago 46m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:48:27.196 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (38m) 13m ago 51m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:48:27.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 msgr2=0x7f56b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 0x7f56b406fd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006fd0 tx=0x7f56c0008040 comp rx=0 tx=0).stop 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 msgr2=0x7f56cc19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b800be20 comp rx=0 tx=0).stop 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 shutdown_connections 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f56b406d8a0 0x7f56b406fd50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc0ff020 0x7f56cc19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 --2- 172.21.15.57:0/3596270870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56cc0ff930 0x7f56cc19c870 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.197+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 >> 172.21.15.57:0/3596270870 conn(0x7f56cc0faa90 msgr2=0x7f56cc110490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.198+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 shutdown_connections 2024-02-28T23:48:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.198+0000 7f56d2400700 1 -- 172.21.15.57:0/3596270870 wait complete. 2024-02-28T23:48:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 -- 172.21.15.57:0/3481893262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40108460 msgr2=0x7efc40108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 --2- 172.21.15.57:0/3481893262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40108460 0x7efc40108830 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efc30009910 tx=0x7efc30009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 -- 172.21.15.57:0/3481893262 shutdown_connections 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 --2- 172.21.15.57:0/3481893262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40102460 0x7efc401028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 --2- 172.21.15.57:0/3481893262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40108460 0x7efc40108830 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.342+0000 7efc47000700 1 -- 172.21.15.57:0/3481893262 >> 172.21.15.57:0/3481893262 conn(0x7efc400fdf60 msgr2=0x7efc40100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.343+0000 7efc47000700 1 -- 172.21.15.57:0/3481893262 shutdown_connections 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.343+0000 7efc47000700 1 -- 172.21.15.57:0/3481893262 wait complete. 2024-02-28T23:48:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 Processor -- start 2024-02-28T23:48:27.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 -- start start 2024-02-28T23:48:27.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 0x7efc401986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40198dc0 con 0x7efc40102460 2024-02-28T23:48:27.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc47000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4019cb50 con 0x7efc40108460 2024-02-28T23:48:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc3fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc3fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40658/0 (socket says 172.21.15.57:40658) 2024-02-28T23:48:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.344+0000 7efc3fe00700 1 -- 172.21.15.57:0/1129945813 learned_addr learned my addr 172.21.15.57:0/1129945813 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3f400700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 0x7efc401986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3fe00700 1 -- 172.21.15.57:0/1129945813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 msgr2=0x7efc401986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3fe00700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 0x7efc401986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3fe00700 1 -- 172.21.15.57:0/1129945813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300095f0 con 0x7efc40102460 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3f400700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 0x7efc401986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.345+0000 7efc3fe00700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b5c0 tx=0x7efc3000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.346+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3001d070 con 0x7efc40102460 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.346+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3000f460 con 0x7efc40102460 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.346+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4019cdd0 con 0x7efc40102460 2024-02-28T23:48:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.346+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc30017610 con 0x7efc40102460 2024-02-28T23:48:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.346+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4019d2c0 con 0x7efc40102460 2024-02-28T23:48:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.347+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4010a9d0 con 0x7efc40102460 2024-02-28T23:48:27.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.348+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efc3000f5d0 con 0x7efc40102460 2024-02-28T23:48:27.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.349+0000 7efc3cc00700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 0x7efc2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.350+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc3008eb30 con 0x7efc40102460 2024-02-28T23:48:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.350+0000 7efc3f400700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 0x7efc2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.350+0000 7efc3f400700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 0x7efc2406fd50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efc401997c0 tx=0x7efc2c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:27.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.354+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc300582c0 con 0x7efc40102460 2024-02-28T23:48:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.657+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efc4004f2a0 con 0x7efc40102460 2024-02-28T23:48:27.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.658+0000 7efc3cc00700 1 -- 172.21.15.57:0/1129945813 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efc3005b980 con 0x7efc40102460 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:48:27.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:48:27.659 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:48:27.659 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:48:27.659 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:48:27.659 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:48:27.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.661+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 msgr2=0x7efc2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.661+0000 7efc47000700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 0x7efc2406fd50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efc401997c0 tx=0x7efc2c008040 comp rx=0 tx=0).stop 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.661+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 msgr2=0x7efc401981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.661+0000 7efc47000700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b5c0 tx=0x7efc3000be00 comp rx=0 tx=0).stop 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 shutdown_connections 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efc2406d8a0 0x7efc2406fd50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc40102460 0x7efc401981a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 --2- 172.21.15.57:0/1129945813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc40108460 0x7efc401986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 >> 172.21.15.57:0/1129945813 conn(0x7efc400fdf60 msgr2=0x7efc400feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:27.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 shutdown_connections 2024-02-28T23:48:27.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.662+0000 7efc47000700 1 -- 172.21.15.57:0/1129945813 wait complete. 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 -- 172.21.15.57:0/15166068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4102470 msgr2=0x7fecf41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 --2- 172.21.15.57:0/15166068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4102470 0x7fecf41028e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fece0009910 tx=0x7fece0009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 -- 172.21.15.57:0/15166068 shutdown_connections 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 --2- 172.21.15.57:0/15166068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4102470 0x7fecf41028e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 --2- 172.21.15.57:0/15166068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4108470 0x7fecf4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 -- 172.21.15.57:0/15166068 >> 172.21.15.57:0/15166068 conn(0x7fecf40fdf90 msgr2=0x7fecf41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 -- 172.21.15.57:0/15166068 shutdown_connections 2024-02-28T23:48:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.813+0000 7fecfaa00700 1 -- 172.21.15.57:0/15166068 wait complete. 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 Processor -- start 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 -- start start 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4102470 0x7fecf41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4198e10 con 0x7fecf4108470 2024-02-28T23:48:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.814+0000 7fecfaa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf419cba0 con 0x7fecf4102470 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40668/0 (socket says 172.21.15.57:40668) 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 -- 172.21.15.57:0/4074042519 learned_addr learned my addr 172.21.15.57:0/4074042519 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3e00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4102470 0x7fecf41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 -- 172.21.15.57:0/4074042519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4102470 msgr2=0x7fecf41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4102470 0x7fecf41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 -- 172.21.15.57:0/4074042519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00095f0 con 0x7fecf4108470 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.815+0000 7fecf3400700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fece000b5c0 tx=0x7fece000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.816+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fece001d070 con 0x7fecf4108470 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.816+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fece000f460 con 0x7fecf4108470 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.816+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fece0017610 con 0x7fecf4108470 2024-02-28T23:48:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.816+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf419ce20 con 0x7fecf4108470 2024-02-28T23:48:27.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.816+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf419d310 con 0x7fecf4108470 2024-02-28T23:48:27.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.817+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf410a970 con 0x7fecf4108470 2024-02-28T23:48:27.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.818+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fece000f5d0 con 0x7fecf4108470 2024-02-28T23:48:27.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.819+0000 7fecf0c00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 0x7fecd806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:27.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.819+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fece008eb50 con 0x7fecf4108470 2024-02-28T23:48:27.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.819+0000 7fecf3e00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 0x7fecd806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:27.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.820+0000 7fecf3e00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 0x7fecd806fd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fece4005fd0 tx=0x7fece400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:27.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:27.823+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fece00582e0 con 0x7fecf4108470 2024-02-28T23:48:27.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:27 smithi084 ceph-mon[70440]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:27.982 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:27 smithi084 ceph-mon[70440]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:27.983 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:27 smithi084 ceph-mon[70440]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:27.983 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:27 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1129945813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:48:28.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:27 smithi057 ceph-mon[89491]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:28.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:27 smithi057 ceph-mon[89491]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:28.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:27 smithi057 ceph-mon[89491]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:28.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:27 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1129945813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:48:28.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.087+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fecf410a350 con 0x7fecf4108470 2024-02-28T23:48:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.088+0000 7fecf0c00700 1 -- 172.21.15.57:0/4074042519 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fece005b9a0 con 0x7fecf4108470 2024-02-28T23:48:28.088 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:48:28.088 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:48:28.088 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:48:28.088 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:48:28.088 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:48:28.089 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:48:28.090 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:48:28.091 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:48:28.091 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:48:28.091 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:48:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 msgr2=0x7fecd806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 0x7fecd806fd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fece4005fd0 tx=0x7fece400a560 comp rx=0 tx=0).stop 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 msgr2=0x7fecf4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fece000b5c0 tx=0x7fece000be00 comp rx=0 tx=0).stop 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 shutdown_connections 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fecd806d8a0 0x7fecd806fd50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf4102470 0x7fecf41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 --2- 172.21.15.57:0/4074042519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf4108470 0x7fecf4198730 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 >> 172.21.15.57:0/4074042519 conn(0x7fecf40fdf90 msgr2=0x7fecf40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 shutdown_connections 2024-02-28T23:48:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.092+0000 7fecfaa00700 1 -- 172.21.15.57:0/4074042519 wait complete. 2024-02-28T23:48:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 -- 172.21.15.57:0/3905972673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 msgr2=0x7fb2b8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 --2- 172.21.15.57:0/3905972673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8104e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009910 tx=0x7fb2a4009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 -- 172.21.15.57:0/3905972673 shutdown_connections 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 --2- 172.21.15.57:0/3905972673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8104e50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 --2- 172.21.15.57:0/3905972673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 -- 172.21.15.57:0/3905972673 >> 172.21.15.57:0/3905972673 conn(0x7fb2b80fbda0 msgr2=0x7fb2b80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 -- 172.21.15.57:0/3905972673 shutdown_connections 2024-02-28T23:48:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.240+0000 7fb2be200700 1 -- 172.21.15.57:0/3905972673 wait complete. 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.241+0000 7fb2be200700 1 Processor -- start 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2be200700 1 -- start start 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2be200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2be200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2be200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8198c80 con 0x7fb2b8100e80 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2be200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8198dc0 con 0x7fb2b81004e0 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2b6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40686/0 (socket says 172.21.15.57:40686) 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.242+0000 7fb2b6a00700 1 -- 172.21.15.57:0/1574278020 learned_addr learned my addr 172.21.15.57:0/1574278020 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b7400700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b6a00700 1 -- 172.21.15.57:0/1574278020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 msgr2=0x7fb2b81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b6a00700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b6a00700 1 -- 172.21.15.57:0/1574278020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a40095f0 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b7400700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2b6a00700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400bda0 tx=0x7fb2a400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.243+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a401d070 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.244+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2a4004fb0 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.244+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a400f460 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.244+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b819cbb0 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.244+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b819d050 con 0x7fb2b8100e80 2024-02-28T23:48:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.245+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b81045d0 con 0x7fb2b8100e80 2024-02-28T23:48:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.246+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb2a4022800 con 0x7fb2b8100e80 2024-02-28T23:48:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.247+0000 7fb2afe00700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 0x7fb29c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.247+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb2a408eb00 con 0x7fb2b8100e80 2024-02-28T23:48:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.247+0000 7fb2b7400700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 0x7fb29c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.248+0000 7fb2b7400700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 0x7fb29c06fd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005fd0 tx=0x7fb2a8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.252+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2a40583e0 con 0x7fb2b8100e80 2024-02-28T23:48:28.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.496+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2b8199560 con 0x7fb29c06d8a0 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.499+0000 7fb2afe00700 1 -- 172.21.15.57:0/1574278020 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2b8199560 con 0x7fb29c06d8a0 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:48:28.498 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:48:28.499 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:48:28.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 msgr2=0x7fb29c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 0x7fb29c06fd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005fd0 tx=0x7fb2a8005e20 comp rx=0 tx=0).stop 2024-02-28T23:48:28.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 msgr2=0x7fb2b8198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400bda0 tx=0x7fb2a400be80 comp rx=0 tx=0).stop 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 shutdown_connections 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb29c06d8a0 0x7fb29c06fd50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2b81004e0 0x7fb2b81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 --2- 172.21.15.57:0/1574278020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2b8100e80 0x7fb2b8198630 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 >> 172.21.15.57:0/1574278020 conn(0x7fb2b80fbda0 msgr2=0x7fb2b80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.502+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 shutdown_connections 2024-02-28T23:48:28.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.503+0000 7fb2be200700 1 -- 172.21.15.57:0/1574278020 wait complete. 2024-02-28T23:48:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 -- 172.21.15.57:0/3696184168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101b40 msgr2=0x7fc3d010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 --2- 172.21.15.57:0/3696184168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101b40 0x7fc3d010a040 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009910 tx=0x7fc3bc009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 -- 172.21.15.57:0/3696184168 shutdown_connections 2024-02-28T23:48:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 --2- 172.21.15.57:0/3696184168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101b40 0x7fc3d010a040 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 --2- 172.21.15.57:0/3696184168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101230 0x7fc3d0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 -- 172.21.15.57:0/3696184168 >> 172.21.15.57:0/3696184168 conn(0x7fc3d00faa90 msgr2=0x7fc3d00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:28.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 -- 172.21.15.57:0/3696184168 shutdown_connections 2024-02-28T23:48:28.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.659+0000 7fc3d5800700 1 -- 172.21.15.57:0/3696184168 wait complete. 2024-02-28T23:48:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 Processor -- start 2024-02-28T23:48:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 -- start start 2024-02-28T23:48:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 0x7fc3d0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0198b30 con 0x7fc3d0101230 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.660+0000 7fc3d5800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d019c8c0 con 0x7fc3d0101b40 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40704/0 (socket says 172.21.15.57:40704) 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 -- 172.21.15.57:0/234029000 learned_addr learned my addr 172.21.15.57:0/234029000 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cf400700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 0x7fc3d0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 -- 172.21.15.57:0/234029000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 msgr2=0x7fc3d0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 0x7fc3d0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cfe00700 1 -- 172.21.15.57:0/234029000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0095f0 con 0x7fc3d0101230 2024-02-28T23:48:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.661+0000 7fc3cf400700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 0x7fc3d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3cfe00700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000b4b0 tx=0x7fc3c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c0011800 con 0x7fc3d0101230 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d019cba0 con 0x7fc3d0101230 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c0011e20 con 0x7fc3d0101230 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c000f2e0 con 0x7fc3d0101230 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.662+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d019d0f0 con 0x7fc3d0101230 2024-02-28T23:48:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.663+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0109b10 con 0x7fc3d0101230 2024-02-28T23:48:28.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.665+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc3c0010380 con 0x7fc3d0101230 2024-02-28T23:48:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.665+0000 7fc3ccc00700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 0x7fc3b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.665+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc3c008d180 con 0x7fc3d0101230 2024-02-28T23:48:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.666+0000 7fc3cf400700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 0x7fc3b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.666+0000 7fc3cf400700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 0x7fc3b406fd50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005b40 tx=0x7fc3bc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:28.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:28.670+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3c0056890 con 0x7fc3d0101230 2024-02-28T23:48:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:28 smithi057 ceph-mon[89491]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:28.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:28 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4074042519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:48:29.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.005+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc3d0062380 con 0x7fc3d0101230 2024-02-28T23:48:29.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.005+0000 7fc3ccc00700 1 -- 172.21.15.57:0/234029000 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc3c0059f50 con 0x7fc3d0101230 2024-02-28T23:48:29.005 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:48:29.005 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:48:29.006 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:48:29.007 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 msgr2=0x7fc3b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 0x7fc3b406fd50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005b40 tx=0x7fc3bc01a040 comp rx=0 tx=0).stop 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 msgr2=0x7fc3d0197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000b4b0 tx=0x7fc3c000b870 comp rx=0 tx=0).stop 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 shutdown_connections 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc3b406d8a0 0x7fc3b406fd50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3d0101230 0x7fc3d0197f10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.008+0000 7fc3d5800700 1 --2- 172.21.15.57:0/234029000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3d0101b40 0x7fc3d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.009+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 >> 172.21.15.57:0/234029000 conn(0x7fc3d00faa90 msgr2=0x7fc3d00ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:29.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.009+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 shutdown_connections 2024-02-28T23:48:29.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:29.009+0000 7fc3d5800700 1 -- 172.21.15.57:0/234029000 wait complete. 2024-02-28T23:48:29.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:28 smithi084 ceph-mon[70440]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:29.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:28 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4074042519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:48:30.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:29 smithi057 ceph-mon[89491]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:30.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/234029000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:48:30.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:29 smithi084 ceph-mon[70440]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:48:30.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/234029000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:48:31.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:30 smithi057 ceph-mon[89491]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:31.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:30 smithi084 ceph-mon[70440]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:33.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:32 smithi057 ceph-mon[89491]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:33.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:32 smithi084 ceph-mon[70440]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:34.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:34.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:35.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:34 smithi057 ceph-mon[89491]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:35.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:34 smithi084 ceph-mon[70440]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:37.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:36 smithi057 ceph-mon[89491]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:37.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:36 smithi084 ceph-mon[70440]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:39.016 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:38 smithi057 ceph-mon[89491]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:39.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:38 smithi084 ceph-mon[70440]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:41.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:40 smithi057 ceph-mon[89491]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:41.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:40 smithi084 ceph-mon[70440]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:43.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:42 smithi057 ceph-mon[89491]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:43.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:42 smithi084 ceph-mon[70440]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:45.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:44 smithi057 ceph-mon[89491]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:45.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:44 smithi084 ceph-mon[70440]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:47.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:46 smithi057 ceph-mon[89491]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:47.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:46 smithi084 ceph-mon[70440]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:48 smithi057 ceph-mon[89491]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:49.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:49.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:48 smithi084 ceph-mon[70440]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:49.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:48:51.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:50 smithi057 ceph-mon[89491]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:51.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:50 smithi084 ceph-mon[70440]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:53.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:52 smithi057 ceph-mon[89491]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:53.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:52 smithi084 ceph-mon[70440]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:55.080 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:54 smithi084 ceph-mon[70440]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:54 smithi057 ceph-mon[89491]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:57.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:56 smithi084 ceph-mon[70440]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:57.122 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:56 smithi057 ceph-mon[89491]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:59.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 -- 172.21.15.57:0/3687888513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8102470 msgr2=0x7f17f81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 --2- 172.21.15.57:0/3687888513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8102470 0x7f17f81028e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009910 tx=0x7f17e4009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:59.188 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:48:58 smithi057 ceph-mon[89491]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:59.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 -- 172.21.15.57:0/3687888513 shutdown_connections 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 --2- 172.21.15.57:0/3687888513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8102470 0x7f17f81028e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 --2- 172.21.15.57:0/3687888513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8108470 0x7f17f8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 -- 172.21.15.57:0/3687888513 >> 172.21.15.57:0/3687888513 conn(0x7f17f80fdf90 msgr2=0x7f17f81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 -- 172.21.15.57:0/3687888513 shutdown_connections 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.188+0000 7f17ff600700 1 -- 172.21.15.57:0/3687888513 wait complete. 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.189+0000 7f17ff600700 1 Processor -- start 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.189+0000 7f17ff600700 1 -- start start 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17ff600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8102470 0x7f17f8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17ff600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17ff600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8198db0 con 0x7f17f8108470 2024-02-28T23:48:59.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17ff600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f819cb40 con 0x7f17f8102470 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49316/0 (socket says 172.21.15.57:49316) 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 -- 172.21.15.57:0/3855784834 learned_addr learned my addr 172.21.15.57:0/3855784834 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17fcc00700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8102470 0x7f17f8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 -- 172.21.15.57:0/3855784834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8102470 msgr2=0x7f17f8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8102470 0x7f17f8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.190+0000 7f17f7e00700 1 -- 172.21.15.57:0/3855784834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40095f0 con 0x7f17f8108470 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17f7e00700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b5c0 tx=0x7f17e400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e401d070 con 0x7f17f8108470 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17e400f460 con 0x7f17f8108470 2024-02-28T23:48:59.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f819cdc0 con 0x7f17f8108470 2024-02-28T23:48:59.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e4017610 con 0x7f17f8108470 2024-02-28T23:48:59.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.191+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f819d2b0 con 0x7f17f8108470 2024-02-28T23:48:59.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.193+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f810aa00 con 0x7f17f8108470 2024-02-28T23:48:59.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.193+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f17e400f5d0 con 0x7f17f8108470 2024-02-28T23:48:59.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.194+0000 7f17f5600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 0x7f17e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.194+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f17e408ebe0 con 0x7f17f8108470 2024-02-28T23:48:59.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.195+0000 7f17fcc00700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 0x7f17e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.195+0000 7f17fcc00700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 0x7f17e006fd50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006fd0 tx=0x7f17ec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:59.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.200+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17e40582f0 con 0x7f17f8108470 2024-02-28T23:48:59.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:48:58 smithi084 ceph-mon[70440]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:48:59.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.454+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17f802cca0 con 0x7f17e006d8a0 2024-02-28T23:48:59.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.457+0000 7f17f5600700 1 -- 172.21.15.57:0/3855784834 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f17f802cca0 con 0x7f17e006d8a0 2024-02-28T23:48:59.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.460+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 msgr2=0x7f17e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.460+0000 7f17ff600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 0x7f17e006fd50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006fd0 tx=0x7f17ec008040 comp rx=0 tx=0).stop 2024-02-28T23:48:59.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.460+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 msgr2=0x7f17f81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.460+0000 7f17ff600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b5c0 tx=0x7f17e400be00 comp rx=0 tx=0).stop 2024-02-28T23:48:59.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 shutdown_connections 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f17e006d8a0 0x7f17e006fd50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f8102470 0x7f17f8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 --2- 172.21.15.57:0/3855784834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17f8108470 0x7f17f81986d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 >> 172.21.15.57:0/3855784834 conn(0x7f17f80fdf90 msgr2=0x7f17f80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 shutdown_connections 2024-02-28T23:48:59.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.461+0000 7f17ff600700 1 -- 172.21.15.57:0/3855784834 wait complete. 2024-02-28T23:48:59.478 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 -- 172.21.15.57:0/1420787657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 msgr2=0x7f6a8c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 --2- 172.21.15.57:0/1420787657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c10ca30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009910 tx=0x7f6a7c009c20 comp rx=0 tx=0).stop 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 -- 172.21.15.57:0/1420787657 shutdown_connections 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 --2- 172.21.15.57:0/1420787657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 --2- 172.21.15.57:0/1420787657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c10ca30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 -- 172.21.15.57:0/1420787657 >> 172.21.15.57:0/1420787657 conn(0x7f6a8c075e90 msgr2=0x7f6a8c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 -- 172.21.15.57:0/1420787657 shutdown_connections 2024-02-28T23:48:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.627+0000 7f6a91600700 1 -- 172.21.15.57:0/1420787657 wait complete. 2024-02-28T23:48:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.628+0000 7f6a91600700 1 Processor -- start 2024-02-28T23:48:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.628+0000 7f6a91600700 1 -- start start 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a91600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a91600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a91600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c19cea0 con 0x7f6a8c10c660 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a91600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1a0c30 con 0x7f6a8c106b80 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49346/0 (socket says 172.21.15.57:49346) 2024-02-28T23:48:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 -- 172.21.15.57:0/1775970515 learned_addr learned my addr 172.21.15.57:0/1775970515 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:48:59.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8be00700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 -- 172.21.15.57:0/1775970515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 msgr2=0x7f6a8c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8b400700 1 -- 172.21.15.57:0/1775970515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0095f0 con 0x7f6a8c10c660 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.629+0000 7f6a8be00700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a8b400700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d8d0 tx=0x7f6a7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a78009940 con 0x7f6a8c10c660 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a78010460 con 0x7f6a8c10c660 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a7800f5f0 con 0x7f6a8c10c660 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c1a0f10 con 0x7f6a8c10c660 2024-02-28T23:48:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.630+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c1a1410 con 0x7f6a8c10c660 2024-02-28T23:48:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.631+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c10eb40 con 0x7f6a8c10c660 2024-02-28T23:48:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.633+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6a780105d0 con 0x7f6a8c10c660 2024-02-28T23:48:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.633+0000 7f6a88c00700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 0x7f6a7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:48:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.634+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6a7808d5d0 con 0x7f6a8c10c660 2024-02-28T23:48:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.634+0000 7f6a8be00700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 0x7f6a7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:48:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.635+0000 7f6a8be00700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 0x7f6a7006fd50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006010 tx=0x7f6a7c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:48:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.639+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a78056ce0 con 0x7f6a8c10c660 2024-02-28T23:48:59.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.899+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a8c1a1680 con 0x7f6a7006d8a0 2024-02-28T23:48:59.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.903+0000 7f6a88c00700 1 -- 172.21.15.57:0/1775970515 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a8c1a1680 con 0x7f6a7006d8a0 2024-02-28T23:48:59.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.905+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 msgr2=0x7f6a7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 0x7f6a7006fd50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006010 tx=0x7f6a7c00b540 comp rx=0 tx=0).stop 2024-02-28T23:48:59.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 msgr2=0x7f6a8c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:48:59.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d8d0 tx=0x7f6a7800dc90 comp rx=0 tx=0).stop 2024-02-28T23:48:59.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 shutdown_connections 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6a7006d8a0 0x7f6a7006fd50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a8c106b80 0x7f6a8c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 --2- 172.21.15.57:0/1775970515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a8c10c660 0x7f6a8c19c7c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 >> 172.21.15.57:0/1775970515 conn(0x7f6a8c075e90 msgr2=0x7f6a8c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 shutdown_connections 2024-02-28T23:48:59.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:48:59.906+0000 7f6a91600700 1 -- 172.21.15.57:0/1775970515 wait complete. 2024-02-28T23:49:00.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.071+0000 7f38b2400700 1 -- 172.21.15.57:0/117841553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac100e80 msgr2=0x7f38ac104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.071+0000 7f38b2400700 1 --2- 172.21.15.57:0/117841553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac100e80 0x7f38ac104e50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3898009910 tx=0x7f3898009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:00.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 -- 172.21.15.57:0/117841553 shutdown_connections 2024-02-28T23:49:00.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 --2- 172.21.15.57:0/117841553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac100e80 0x7f38ac104e50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 --2- 172.21.15.57:0/117841553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac1004e0 0x7f38ac1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 -- 172.21.15.57:0/117841553 >> 172.21.15.57:0/117841553 conn(0x7f38ac0fbda0 msgr2=0x7f38ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:00.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 -- 172.21.15.57:0/117841553 shutdown_connections 2024-02-28T23:49:00.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.072+0000 7f38b2400700 1 -- 172.21.15.57:0/117841553 wait complete. 2024-02-28T23:49:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.073+0000 7f38b2400700 1 Processor -- start 2024-02-28T23:49:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.073+0000 7f38b2400700 1 -- start start 2024-02-28T23:49:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.073+0000 7f38b2400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.073+0000 7f38b2400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac100e80 0x7f38ac073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38b2400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac073ad0 con 0x7f38ac1004e0 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38b2400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac073c10 con 0x7f38ac100e80 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49354/0 (socket says 172.21.15.57:49354) 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 -- 172.21.15.57:0/94604331 learned_addr learned my addr 172.21.15.57:0/94604331 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38aaa00700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac100e80 0x7f38ac073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 -- 172.21.15.57:0/94604331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac100e80 msgr2=0x7f38ac073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac100e80 0x7f38ac073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.074+0000 7f38ab400700 1 -- 172.21.15.57:0/94604331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980095f0 con 0x7f38ac1004e0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38ab400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d8d0 tx=0x7f389c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c009940 con 0x7f38ac1004e0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f389c010460 con 0x7f38ac1004e0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c00f5f0 con 0x7f38ac1004e0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac1a2100 con 0x7f38ac1004e0 2024-02-28T23:49:00.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.075+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac1a2510 con 0x7f38ac1004e0 2024-02-28T23:49:00.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.076+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac108850 con 0x7f38ac1004e0 2024-02-28T23:49:00.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.077+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f389c0105d0 con 0x7f38ac1004e0 2024-02-28T23:49:00.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.078+0000 7f38a3e00700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 0x7f389006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.078+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f389c08d4c0 con 0x7f38ac1004e0 2024-02-28T23:49:00.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.078+0000 7f38aaa00700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 0x7f389006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.079+0000 7f38aaa00700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 0x7f389006fd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f38ac074840 tx=0x7f389801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.083+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f389c056c50 con 0x7f38ac1004e0 2024-02-28T23:49:00.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.329+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ac061190 con 0x7f389006d8a0 2024-02-28T23:49:00.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.361+0000 7f38a3e00700 1 -- 172.21.15.57:0/94604331 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f38ac061190 con 0x7f389006d8a0 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (38m) 14m ago 52m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (48m) 14m ago 52m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (51m) 14m ago 51m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (34m) 14m ago 52m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (34m) 14m ago 51m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (38m) 14m ago 52m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (30m) 14m ago 45m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (30m) 14m ago 45m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (14m) 14m ago 45m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:49:00.362 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (29m) 14m ago 45m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (41m) 14m ago 54m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (40m) 14m ago 51m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (36m) 14m ago 54m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (35m) 14m ago 50m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (39m) 14m ago 52m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (39m) 14m ago 51m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (34m) 14m ago 50m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (33m) 14m ago 49m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (32m) 14m ago 48m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (32m) 14m ago 48m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (31m) 14m ago 47m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (31m) 14m ago 47m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:49:00.363 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (39m) 14m ago 51m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 msgr2=0x7f389006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 0x7f389006fd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f38ac074840 tx=0x7f389801a040 comp rx=0 tx=0).stop 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 msgr2=0x7f38ac073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d8d0 tx=0x7f389c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 shutdown_connections 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f389006d8a0 0x7f389006fd50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38ac1004e0 0x7f38ac073050 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 --2- 172.21.15.57:0/94604331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38ac100e80 0x7f38ac073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.364+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 >> 172.21.15.57:0/94604331 conn(0x7f38ac0fbda0 msgr2=0x7f38ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.365+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 shutdown_connections 2024-02-28T23:49:00.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.365+0000 7f38b2400700 1 -- 172.21.15.57:0/94604331 wait complete. 2024-02-28T23:49:00.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 -- 172.21.15.57:0/510704071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe6810c8f0 msgr2=0x7fbe6810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/510704071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe6810c8f0 0x7fbe6810ccc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009910 tx=0x7fbe58009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:00.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 -- 172.21.15.57:0/510704071 shutdown_connections 2024-02-28T23:49:00.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/510704071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe68106b80 0x7fbe68106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/510704071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe6810c8f0 0x7fbe6810ccc0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 -- 172.21.15.57:0/510704071 >> 172.21.15.57:0/510704071 conn(0x7fbe68076060 msgr2=0x7fbe68078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:00.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 -- 172.21.15.57:0/510704071 shutdown_connections 2024-02-28T23:49:00.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.529+0000 7fbe6ce00700 1 -- 172.21.15.57:0/510704071 wait complete. 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.530+0000 7fbe6ce00700 1 Processor -- start 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.530+0000 7fbe6ce00700 1 -- start start 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe6ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe6ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 0x7fbe6819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe6ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819d100 con 0x7fbe68106b80 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe6ce00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe681a0e90 con 0x7fbe6810c8f0 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49358/0 (socket says 172.21.15.57:49358) 2024-02-28T23:49:00.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 -- 172.21.15.57:0/3919228250 learned_addr learned my addr 172.21.15.57:0/3919228250 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 -- 172.21.15.57:0/3919228250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 msgr2=0x7fbe6819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65400700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 0x7fbe6819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 0x7fbe6819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.531+0000 7fbe65e00700 1 -- 172.21.15.57:0/3919228250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580095f0 con 0x7fbe68106b80 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe65400700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 0x7fbe6819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:49:00.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe65e00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5801d070 con 0x7fbe68106b80 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe58004fb0 con 0x7fbe68106b80 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe681a1110 con 0x7fbe68106b80 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5800f690 con 0x7fbe68106b80 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.532+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe681a1600 con 0x7fbe68106b80 2024-02-28T23:49:00.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.534+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe68109b20 con 0x7fbe68106b80 2024-02-28T23:49:00.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.535+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbe580228a0 con 0x7fbe68106b80 2024-02-28T23:49:00.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.536+0000 7fbe5ea00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 0x7fbe4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:00.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.536+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe5808ede0 con 0x7fbe68106b80 2024-02-28T23:49:00.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.536+0000 7fbe65400700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 0x7fbe4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:00.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.537+0000 7fbe65400700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 0x7fbe4c06fd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbe6819db00 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:00.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.540+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe58058570 con 0x7fbe68106b80 2024-02-28T23:49:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.857+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe6804f2a0 con 0x7fbe68106b80 2024-02-28T23:49:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.858+0000 7fbe5ea00700 1 -- 172.21.15.57:0/3919228250 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbe58027070 con 0x7fbe68106b80 2024-02-28T23:49:00.858 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:49:00.859 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:49:00.860 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:49:00.860 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:49:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 msgr2=0x7fbe4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 0x7fbe4c06fd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbe6819db00 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-02-28T23:49:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 msgr2=0x7fbe6819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe5800be20 comp rx=0 tx=0).stop 2024-02-28T23:49:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 shutdown_connections 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fbe4c06d8a0 0x7fbe4c06fd50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe68106b80 0x7fbe6819c4e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 --2- 172.21.15.57:0/3919228250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6810c8f0 0x7fbe6819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 >> 172.21.15.57:0/3919228250 conn(0x7fbe68076060 msgr2=0x7fbe68077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 shutdown_connections 2024-02-28T23:49:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:00.861+0000 7fbe6ce00700 1 -- 172.21.15.57:0/3919228250 wait complete. 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 -- 172.21.15.57:0/2251468784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 msgr2=0x7f94741016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 --2- 172.21.15.57:0/2251468784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f94741016c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9464009910 tx=0x7f9464009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 -- 172.21.15.57:0/2251468784 shutdown_connections 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 --2- 172.21.15.57:0/2251468784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9474101c00 0x7f947410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 --2- 172.21.15.57:0/2251468784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f94741016c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 -- 172.21.15.57:0/2251468784 >> 172.21.15.57:0/2251468784 conn(0x7f94740fac60 msgr2=0x7f94740fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 -- 172.21.15.57:0/2251468784 shutdown_connections 2024-02-28T23:49:01.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.021+0000 7f9479200700 1 -- 172.21.15.57:0/2251468784 wait complete. 2024-02-28T23:49:01.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.022+0000 7f9479200700 1 Processor -- start 2024-02-28T23:49:01.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9479200700 1 -- start start 2024-02-28T23:49:01.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9479200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9479200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9474101c00 0x7f9474196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9479200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474196c00 con 0x7f94741012f0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9479200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947419a940 con 0x7f9474101c00 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9471e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9471e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49372/0 (socket says 172.21.15.57:49372) 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.023+0000 7f9471e00700 1 -- 172.21.15.57:0/3462387543 learned_addr learned my addr 172.21.15.57:0/3462387543 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9471e00700 1 -- 172.21.15.57:0/3462387543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9474101c00 msgr2=0x7f9474196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9471e00700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9474101c00 0x7f9474196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9471e00700 1 -- 172.21.15.57:0/3462387543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640095f0 con 0x7f94741012f0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9471e00700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f946400b5c0 tx=0x7f946400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946401d070 con 0x7f94741012f0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946400f460 con 0x7f94741012f0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947419abc0 con 0x7f94741012f0 2024-02-28T23:49:01.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9464017610 con 0x7f94741012f0 2024-02-28T23:49:01.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.024+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947419b0b0 con 0x7f94741012f0 2024-02-28T23:49:01.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.026+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9474107970 con 0x7f94741012f0 2024-02-28T23:49:01.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.027+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f946400f5d0 con 0x7f94741012f0 2024-02-28T23:49:01.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.027+0000 7f946aa00700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 0x7f945806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.028+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f946408eb30 con 0x7f94741012f0 2024-02-28T23:49:01.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.028+0000 7f9471400700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 0x7f945806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.029+0000 7f9471400700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 0x7f945806fd50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f94741975b0 tx=0x7f9460006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.032+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94640582c0 con 0x7f94741012f0 2024-02-28T23:49:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.314+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f947404f2a0 con 0x7f94741012f0 2024-02-28T23:49:01.315 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:00 smithi057 ceph-mon[89491]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:01.315 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:00 smithi057 ceph-mon[89491]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:01.315 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:00 smithi057 ceph-mon[89491]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.316+0000 7f946aa00700 1 -- 172.21.15.57:0/3462387543 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f946405b980 con 0x7f94741012f0 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:49:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:49:01.318 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:49:01.319 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.318+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 msgr2=0x7f945806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.318+0000 7f9479200700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 0x7f945806fd50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f94741975b0 tx=0x7f9460006d20 comp rx=0 tx=0).stop 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.318+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 msgr2=0x7f9474195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f946400b5c0 tx=0x7f946400be00 comp rx=0 tx=0).stop 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 shutdown_connections 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f945806d8a0 0x7f945806fd50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94741012f0 0x7f9474195fe0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 --2- 172.21.15.57:0/3462387543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9474101c00 0x7f9474196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 >> 172.21.15.57:0/3462387543 conn(0x7f94740fac60 msgr2=0x7f9474104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 shutdown_connections 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.319+0000 7f9479200700 1 -- 172.21.15.57:0/3462387543 wait complete. 2024-02-28T23:49:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:49:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:00 smithi084 ceph-mon[70440]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:01.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:00 smithi084 ceph-mon[70440]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:01.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:00 smithi084 ceph-mon[70440]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:01.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 -- 172.21.15.57:0/3151373466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0108470 msgr2=0x7f0ad0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 --2- 172.21.15.57:0/3151373466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0108470 0x7f0ad0108840 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009910 tx=0x7f0ac4009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:01.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 -- 172.21.15.57:0/3151373466 shutdown_connections 2024-02-28T23:49:01.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 --2- 172.21.15.57:0/3151373466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0102470 0x7f0ad01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 --2- 172.21.15.57:0/3151373466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0108470 0x7f0ad0108840 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 -- 172.21.15.57:0/3151373466 >> 172.21.15.57:0/3151373466 conn(0x7f0ad00fdf90 msgr2=0x7f0ad01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:01.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 -- 172.21.15.57:0/3151373466 shutdown_connections 2024-02-28T23:49:01.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.478+0000 7f0ad8600700 1 -- 172.21.15.57:0/3151373466 wait complete. 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.479+0000 7f0ad8600700 1 Processor -- start 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.479+0000 7f0ad8600700 1 -- start start 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad8600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad8600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0108470 0x7f0ad0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad8600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0198e10 con 0x7f0ad0102470 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad8600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad019cba0 con 0x7f0ad0108470 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad5c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49384/0 (socket says 172.21.15.57:49384) 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad5c00700 1 -- 172.21.15.57:0/1729716324 learned_addr learned my addr 172.21.15.57:0/1729716324 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.480+0000 7f0ad5200700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0108470 0x7f0ad0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad5c00700 1 -- 172.21.15.57:0/1729716324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0108470 msgr2=0x7f0ad0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad5c00700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0108470 0x7f0ad0198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad5c00700 1 -- 172.21.15.57:0/1729716324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40095f0 con 0x7f0ad0102470 2024-02-28T23:49:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad5c00700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40092e0 tx=0x7f0ac400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac401d070 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ac4005e00 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4017400 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad019ce20 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.481+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad019d310 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.483+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad010a9e0 con 0x7f0ad0102470 2024-02-28T23:49:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.484+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0ac400f460 con 0x7f0ad0102470 2024-02-28T23:49:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.485+0000 7f0ace000700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.485+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ac408eb00 con 0x7f0ad0102470 2024-02-28T23:49:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.485+0000 7f0ad5200700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.486+0000 7f0ad5200700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005950 tx=0x7f0abc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.490+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ac4058210 con 0x7f0ad0102470 2024-02-28T23:49:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.740+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad0199690 con 0x7f0ab806d8a0 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.742+0000 7f0ace000700 1 -- 172.21.15.57:0/1729716324 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0abc0097d0 con 0x7f0ab806d8a0 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:49:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:49:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:49:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:49:01.744 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 msgr2=0x7f0ab806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005950 tx=0x7f0abc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 msgr2=0x7f0ad01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40092e0 tx=0x7f0ac400be20 comp rx=0 tx=0).stop 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 shutdown_connections 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0ab806d8a0 0x7f0ab806fd50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ad0102470 0x7f0ad01981f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 --2- 172.21.15.57:0/1729716324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad0108470 0x7f0ad0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.745+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 >> 172.21.15.57:0/1729716324 conn(0x7f0ad00fdf90 msgr2=0x7f0ad01002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.746+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 shutdown_connections 2024-02-28T23:49:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.746+0000 7f0ad8600700 1 -- 172.21.15.57:0/1729716324 wait complete. 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 -- 172.21.15.57:0/1087562754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b01026a0 msgr2=0x7f81b0102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 --2- 172.21.15.57:0/1087562754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b01026a0 0x7f81b0102a70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009910 tx=0x7f81a0009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 -- 172.21.15.57:0/1087562754 shutdown_connections 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 --2- 172.21.15.57:0/1087562754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b0103040 0x7f81b0107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 --2- 172.21.15.57:0/1087562754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b01026a0 0x7f81b0102a70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 -- 172.21.15.57:0/1087562754 >> 172.21.15.57:0/1087562754 conn(0x7f81b00fdf40 msgr2=0x7f81b0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 -- 172.21.15.57:0/1087562754 shutdown_connections 2024-02-28T23:49:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.897+0000 7f81b6c00700 1 -- 172.21.15.57:0/1087562754 wait complete. 2024-02-28T23:49:01.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.898+0000 7f81b6c00700 1 Processor -- start 2024-02-28T23:49:01.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.898+0000 7f81b6c00700 1 -- start start 2024-02-28T23:49:01.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81b6c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 0x7f81b01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81b6c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81b6c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0198dd0 con 0x7f81b0103040 2024-02-28T23:49:01.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81b6c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b019cb60 con 0x7f81b01026a0 2024-02-28T23:49:01.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49404/0 (socket says 172.21.15.57:49404) 2024-02-28T23:49:01.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 -- 172.21.15.57:0/4070375115 learned_addr learned my addr 172.21.15.57:0/4070375115 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:01.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81afe00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 0x7f81b01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 -- 172.21.15.57:0/4070375115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 msgr2=0x7f81b01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 0x7f81b01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.899+0000 7f81af400700 1 -- 172.21.15.57:0/4070375115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00095f0 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81afe00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 0x7f81b01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81af400700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d8d0 tx=0x7f819c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819c004a00 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f819c0054e0 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819c009d40 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b019ce40 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.900+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b019d390 con 0x7f81b0103040 2024-02-28T23:49:01.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.901+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b010a9b0 con 0x7f81b0103040 2024-02-28T23:49:01.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.903+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f819c005020 con 0x7f81b0103040 2024-02-28T23:49:01.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.903+0000 7f81acc00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 0x7f819406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:01.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.903+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f819c08ce30 con 0x7f81b0103040 2024-02-28T23:49:01.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.904+0000 7f81afe00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 0x7f819406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:01.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.904+0000 7f81afe00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 0x7f819406fd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f81a000b5c0 tx=0x7f81a00053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:01.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:01.908+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f819c0565c0 con 0x7f81b0103040 2024-02-28T23:49:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:01 smithi057 ceph-mon[89491]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:01 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3919228250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:02.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:01 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3462387543' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:49:02.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.220+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f81b004f2a0 con 0x7f81b0103040 2024-02-28T23:49:02.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.221+0000 7f81acc00700 1 -- 172.21.15.57:0/4070375115 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f819c020090 con 0x7f81b0103040 2024-02-28T23:49:02.221 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:49:02.221 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:49:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5zpibmq6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:49:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:49:02.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.223+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 msgr2=0x7f819406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.223+0000 7f81b6c00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 0x7f819406fd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f81a000b5c0 tx=0x7f81a00053a0 comp rx=0 tx=0).stop 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 msgr2=0x7f81b01986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d8d0 tx=0x7f819c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 shutdown_connections 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f819406d8a0 0x7f819406fd50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b01026a0 0x7f81b01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 --2- 172.21.15.57:0/4070375115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b0103040 0x7f81b01986f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 >> 172.21.15.57:0/4070375115 conn(0x7f81b00fdf40 msgr2=0x7f81b00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 shutdown_connections 2024-02-28T23:49:02.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:02.224+0000 7f81b6c00700 1 -- 172.21.15.57:0/4070375115 wait complete. 2024-02-28T23:49:02.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:01 smithi084 ceph-mon[70440]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:02.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:01 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3919228250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:02.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:01 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3462387543' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:49:03.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:02 smithi057 ceph-mon[89491]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:03.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:02 smithi057 ceph-mon[89491]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:03.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:02 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4070375115' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:49:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:02 smithi084 ceph-mon[70440]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:02 smithi084 ceph-mon[70440]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:02 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4070375115' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:49:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:04.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:04 smithi057 ceph-mon[89491]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:05.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:04 smithi084 ceph-mon[70440]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:06 smithi057 ceph-mon[89491]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:06 smithi084 ceph-mon[70440]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:08 smithi057 ceph-mon[89491]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:09.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:08 smithi084 ceph-mon[70440]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:11.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:10 smithi057 ceph-mon[89491]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:11.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:10 smithi084 ceph-mon[70440]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:12 smithi057 ceph-mon[89491]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:12 smithi084 ceph-mon[70440]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:14 smithi057 ceph-mon[89491]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:14 smithi084 ceph-mon[70440]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:16 smithi057 ceph-mon[89491]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:16 smithi084 ceph-mon[70440]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:18 smithi057 ceph-mon[89491]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:18 smithi084 ceph-mon[70440]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:20 smithi057 ceph-mon[89491]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:21.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:20 smithi084 ceph-mon[70440]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:23.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:22 smithi057 ceph-mon[89491]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:23.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:22 smithi084 ceph-mon[70440]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:24 smithi057 ceph-mon[89491]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:25.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:24 smithi084 ceph-mon[70440]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:27.232 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:26 smithi084 ceph-mon[70440]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:26 smithi057 ceph-mon[89491]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:28 smithi057 ceph-mon[89491]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:29.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:28 smithi084 ceph-mon[70440]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:31.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:30 smithi057 ceph-mon[89491]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:31.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:30 smithi084 ceph-mon[70440]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:32.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 -- 172.21.15.57:0/4195153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 msgr2=0x7f64741028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 --2- 172.21.15.57:0/4195153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f64741028a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6468009910 tx=0x7f6468009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:32.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 -- 172.21.15.57:0/4195153 shutdown_connections 2024-02-28T23:49:32.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 --2- 172.21.15.57:0/4195153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 0x7f6474106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 --2- 172.21.15.57:0/4195153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f64741028a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 -- 172.21.15.57:0/4195153 >> 172.21.15.57:0/4195153 conn(0x7f64740fdd90 msgr2=0x7f64741001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 -- 172.21.15.57:0/4195153 shutdown_connections 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.401+0000 7f647a800700 1 -- 172.21.15.57:0/4195153 wait complete. 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 Processor -- start 2024-02-28T23:49:32.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 -- start start 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 0x7f64741983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474198a80 con 0x7f64741024d0 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.402+0000 7f647a800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647419c810 con 0x7f6474102e70 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60634/0 (socket says 172.21.15.57:60634) 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 -- 172.21.15.57:0/908410966 learned_addr learned my addr 172.21.15.57:0/908410966 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6478a00700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 0x7f64741983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 -- 172.21.15.57:0/908410966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 msgr2=0x7f64741983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 0x7f64741983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.403+0000 7f6479400700 1 -- 172.21.15.57:0/908410966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680095f0 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f6479400700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f646800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f646801d070 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f646800f460 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647419ca90 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6468017630 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.404+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647419cf80 con 0x7f64741024d0 2024-02-28T23:49:32.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.405+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647410a7e0 con 0x7f64741024d0 2024-02-28T23:49:32.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.406+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f646800f5d0 con 0x7f64741024d0 2024-02-28T23:49:32.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.407+0000 7f6472000700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 0x7f645c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.408+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f646808e9a0 con 0x7f64741024d0 2024-02-28T23:49:32.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.408+0000 7f6478a00700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 0x7f645c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.409+0000 7f6478a00700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 0x7f645c06fd50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6460005950 tx=0x7f64600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:32.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.413+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6468058150 con 0x7f64741024d0 2024-02-28T23:49:32.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.663+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f647419d260 con 0x7f645c06d8a0 2024-02-28T23:49:32.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.665+0000 7f6472000700 1 -- 172.21.15.57:0/908410966 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f647419d260 con 0x7f645c06d8a0 2024-02-28T23:49:32.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 msgr2=0x7f645c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 0x7f645c06fd50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6460005950 tx=0x7f64600058e0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 msgr2=0x7f6474197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f646800be20 comp rx=0 tx=0).stop 2024-02-28T23:49:32.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 shutdown_connections 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f645c06d8a0 0x7f645c06fd50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64741024d0 0x7f6474197e60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 --2- 172.21.15.57:0/908410966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6474102e70 0x7f64741983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.668+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 >> 172.21.15.57:0/908410966 conn(0x7f64740fdd90 msgr2=0x7f6474100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.669+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 shutdown_connections 2024-02-28T23:49:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.669+0000 7f647a800700 1 -- 172.21.15.57:0/908410966 wait complete. 2024-02-28T23:49:32.685 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:49:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.840+0000 7feef6200700 1 -- 172.21.15.57:0/4037679303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 msgr2=0x7feef0102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 --2- 172.21.15.57:0/4037679303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0102a70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7feee0009910 tx=0x7feee0009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 -- 172.21.15.57:0/4037679303 shutdown_connections 2024-02-28T23:49:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 --2- 172.21.15.57:0/4037679303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 0x7feef0107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 --2- 172.21.15.57:0/4037679303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0102a70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 -- 172.21.15.57:0/4037679303 >> 172.21.15.57:0/4037679303 conn(0x7feef00fdf40 msgr2=0x7feef0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 -- 172.21.15.57:0/4037679303 shutdown_connections 2024-02-28T23:49:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.841+0000 7feef6200700 1 -- 172.21.15.57:0/4037679303 wait complete. 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 Processor -- start 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 -- start start 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 0x7feef0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0198d50 con 0x7feef01026a0 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.842+0000 7feef6200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef019cae0 con 0x7feef0103040 2024-02-28T23:49:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60648/0 (socket says 172.21.15.57:60648) 2024-02-28T23:49:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 -- 172.21.15.57:0/1280476478 learned_addr learned my addr 172.21.15.57:0/1280476478 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeeea00700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 0x7feef0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 -- 172.21.15.57:0/1280476478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 msgr2=0x7feef0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 0x7feef0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.843+0000 7feeef400700 1 -- 172.21.15.57:0/1280476478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00095f0 con 0x7feef01026a0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feeef400700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7feee00092e0 tx=0x7feee000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee001d070 con 0x7feef01026a0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feee0005e00 con 0x7feef01026a0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee0017400 con 0x7feef01026a0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef019cd60 con 0x7feef01026a0 2024-02-28T23:49:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.844+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef019d250 con 0x7feef01026a0 2024-02-28T23:49:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.845+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef010a9b0 con 0x7feef01026a0 2024-02-28T23:49:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.846+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7feee000f460 con 0x7feef01026a0 2024-02-28T23:49:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.847+0000 7feee7e00700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 0x7feed406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.847+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feee008ea30 con 0x7feef01026a0 2024-02-28T23:49:32.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.848+0000 7feeeea00700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 0x7feed406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.848+0000 7feeeea00700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 0x7feed406fd50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:32.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:32.852+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feee00581c0 con 0x7feef01026a0 2024-02-28T23:49:33.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.113+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feef019d4c0 con 0x7feed406d8a0 2024-02-28T23:49:33.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.116+0000 7feee7e00700 1 -- 172.21.15.57:0/1280476478 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feef019d4c0 con 0x7feed406d8a0 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 msgr2=0x7feed406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 0x7feed406fd50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 msgr2=0x7feef0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7feee00092e0 tx=0x7feee000be20 comp rx=0 tx=0).stop 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 shutdown_connections 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7feed406d8a0 0x7feed406fd50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feef01026a0 0x7feef0198130 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 --2- 172.21.15.57:0/1280476478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feef0103040 0x7feef0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.119+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 >> 172.21.15.57:0/1280476478 conn(0x7feef00fdf40 msgr2=0x7feef00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:33.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.120+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 shutdown_connections 2024-02-28T23:49:33.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.120+0000 7feef6200700 1 -- 172.21.15.57:0/1280476478 wait complete. 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.291+0000 7efeedc00700 1 -- 172.21.15.57:0/44927319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 msgr2=0x7efee80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.291+0000 7efeedc00700 1 --2- 172.21.15.57:0/44927319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee80689b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efed4009910 tx=0x7efed4009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 -- 172.21.15.57:0/44927319 shutdown_connections 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 --2- 172.21.15.57:0/44927319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee80689b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 --2- 172.21.15.57:0/44927319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 0x7efee81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 -- 172.21.15.57:0/44927319 >> 172.21.15.57:0/44927319 conn(0x7efee8075130 msgr2=0x7efee8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:33.292 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:32 smithi057 ceph-mon[89491]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:33.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 -- 172.21.15.57:0/44927319 shutdown_connections 2024-02-28T23:49:33.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.292+0000 7efeedc00700 1 -- 172.21.15.57:0/44927319 wait complete. 2024-02-28T23:49:33.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.293+0000 7efeedc00700 1 Processor -- start 2024-02-28T23:49:33.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.293+0000 7efeedc00700 1 -- start start 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efeedc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efeedc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 0x7efee81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efeedc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8198c90 con 0x7efee8068540 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efeedc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee819ca20 con 0x7efee8101000 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efee6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efee6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60674/0 (socket says 172.21.15.57:60674) 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efee6a00700 1 -- 172.21.15.57:0/1701453052 learned_addr learned my addr 172.21.15.57:0/1701453052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.294+0000 7efee6000700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 0x7efee81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efee6a00700 1 -- 172.21.15.57:0/1701453052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 msgr2=0x7efee81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efee6a00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 0x7efee81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efee6a00700 1 -- 172.21.15.57:0/1701453052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40095f0 con 0x7efee8068540 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efee6a00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efed800d8d0 tx=0x7efed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed8009940 con 0x7efee8068540 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efed8010460 con 0x7efee8068540 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee819cd00 con 0x7efee8068540 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.295+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed800f5f0 con 0x7efee8068540 2024-02-28T23:49:33.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.296+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee819d250 con 0x7efee8068540 2024-02-28T23:49:33.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.297+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee804f2a0 con 0x7efee8068540 2024-02-28T23:49:33.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.298+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efed80105d0 con 0x7efee8068540 2024-02-28T23:49:33.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.299+0000 7efedf400700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 0x7efecc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.299+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efed808d530 con 0x7efee8068540 2024-02-28T23:49:33.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.299+0000 7efee6000700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 0x7efecc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:33.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.300+0000 7efee6000700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 0x7efecc06fd50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efee8199690 tx=0x7efed401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:33.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.303+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efed8056ce0 con 0x7efee8068540 2024-02-28T23:49:33.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:32 smithi084 ceph-mon[70440]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:33.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.544+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efee819d580 con 0x7efecc06d8a0 2024-02-28T23:49:33.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.556+0000 7efedf400700 1 -- 172.21.15.57:0/1701453052 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efee819d580 con 0x7efecc06d8a0 2024-02-28T23:49:33.556 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:49:33.556 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (39m) 14m ago 52m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:49:33.556 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (49m) 14m ago 53m 10.3M - 18.2.0 dc2bc1663786 e89cb5856d40 2024-02-28T23:49:33.556 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (51m) 14m ago 51m 10.2M - 18.2.0 dc2bc1663786 c2942d492bb8 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (35m) 14m ago 53m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (35m) 14m ago 51m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (38m) 14m ago 52m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (31m) 14m ago 45m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (30m) 14m ago 45m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (15m) 14m ago 45m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (30m) 14m ago 45m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (41m) 14m ago 54m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (40m) 14m ago 51m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (36m) 14m ago 55m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (36m) 14m ago 51m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (40m) 14m ago 53m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (40m) 14m ago 51m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (34m) 14m ago 50m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (34m) 14m ago 49m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:49:33.557 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (33m) 14m ago 49m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:49:33.558 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (33m) 14m ago 48m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:49:33.558 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (32m) 14m ago 48m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:49:33.558 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (31m) 14m ago 47m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:49:33.558 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (39m) 14m ago 52m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.559+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 msgr2=0x7efecc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.559+0000 7efeedc00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 0x7efecc06fd50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efee8199690 tx=0x7efed401a040 comp rx=0 tx=0).stop 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.559+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 msgr2=0x7efee8198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.559+0000 7efeedc00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efed800d8d0 tx=0x7efed800dc90 comp rx=0 tx=0).stop 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 shutdown_connections 2024-02-28T23:49:33.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efecc06d8a0 0x7efecc06fd50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efee8068540 0x7efee8198070 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 --2- 172.21.15.57:0/1701453052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efee8101000 0x7efee81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 >> 172.21.15.57:0/1701453052 conn(0x7efee8075130 msgr2=0x7efee80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 shutdown_connections 2024-02-28T23:49:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.560+0000 7efeedc00700 1 -- 172.21.15.57:0/1701453052 wait complete. 2024-02-28T23:49:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 -- 172.21.15.57:0/1935510337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 msgr2=0x7f9dc00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 --2- 172.21.15.57:0/1935510337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc00734c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009910 tx=0x7f9db4009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 -- 172.21.15.57:0/1935510337 shutdown_connections 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 --2- 172.21.15.57:0/1935510337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc0073a00 0x7f9dc010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 --2- 172.21.15.57:0/1935510337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc00734c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 -- 172.21.15.57:0/1935510337 >> 172.21.15.57:0/1935510337 conn(0x7f9dc00fbb10 msgr2=0x7f9dc00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 -- 172.21.15.57:0/1935510337 shutdown_connections 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.719+0000 7f9dc6600700 1 -- 172.21.15.57:0/1935510337 wait complete. 2024-02-28T23:49:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.720+0000 7f9dc6600700 1 Processor -- start 2024-02-28T23:49:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc6600700 1 -- start start 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc6600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc6600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc0073a00 0x7f9dc019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc6600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0197b30 con 0x7f9dc00730f0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc6600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0197ca0 con 0x7f9dc0073a00 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc5200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60688/0 (socket says 172.21.15.57:60688) 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.721+0000 7f9dc5200700 1 -- 172.21.15.57:0/3149547867 learned_addr learned my addr 172.21.15.57:0/3149547867 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dc5200700 1 -- 172.21.15.57:0/3149547867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc0073a00 msgr2=0x7f9dc019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dc5200700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc0073a00 0x7f9dc019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dc5200700 1 -- 172.21.15.57:0/3149547867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40095f0 con 0x7f9dc00730f0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dc5200700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9db40092e0 tx=0x7f9db400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9db401d070 con 0x7f9dc00730f0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9db4005e00 con 0x7f9dc00730f0 2024-02-28T23:49:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9db4017400 con 0x7f9dc00730f0 2024-02-28T23:49:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.722+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc0197f80 con 0x7f9dc00730f0 2024-02-28T23:49:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.723+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc01984d0 con 0x7f9dc00730f0 2024-02-28T23:49:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.724+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0109a30 con 0x7f9dc00730f0 2024-02-28T23:49:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.725+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9db400f460 con 0x7f9dc00730f0 2024-02-28T23:49:33.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.725+0000 7f9dbd600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 0x7f9da806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:33.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.726+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9db408ea90 con 0x7f9dc00730f0 2024-02-28T23:49:33.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.726+0000 7f9dbfe00700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 0x7f9da806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:33.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.726+0000 7f9dbfe00700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 0x7f9da806fd50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005950 tx=0x7f9dac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:33.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:33.731+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9db405bcd0 con 0x7f9dc00730f0 2024-02-28T23:49:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.039+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9dc010f330 con 0x7f9dc00730f0 2024-02-28T23:49:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.040+0000 7f9dbd600700 1 -- 172.21.15.57:0/3149547867 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9db405b860 con 0x7f9dc00730f0 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:34.040 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:49:34.041 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:49:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.042+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 msgr2=0x7f9da806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.042+0000 7f9dc6600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 0x7f9da806fd50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005950 tx=0x7f9dac0058e0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 msgr2=0x7f9dc019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9db40092e0 tx=0x7f9db400be20 comp rx=0 tx=0).stop 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 shutdown_connections 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9da806d8a0 0x7f9da806fd50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dc00730f0 0x7f9dc019dab0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 --2- 172.21.15.57:0/3149547867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc0073a00 0x7f9dc019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 >> 172.21.15.57:0/3149547867 conn(0x7f9dc00fbb10 msgr2=0x7f9dc0106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 shutdown_connections 2024-02-28T23:49:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.043+0000 7f9dc6600700 1 -- 172.21.15.57:0/3149547867 wait complete. 2024-02-28T23:49:34.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 -- 172.21.15.57:0/3062354249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 msgr2=0x7f0d18106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 --2- 172.21.15.57:0/3062354249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18106e40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009910 tx=0x7f0d04009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 -- 172.21.15.57:0/3062354249 shutdown_connections 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 --2- 172.21.15.57:0/3062354249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18106e40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 --2- 172.21.15.57:0/3062354249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d181028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.199+0000 7f0d17e00700 1 -- 172.21.15.57:0/3062354249 >> 172.21.15.57:0/3062354249 conn(0x7f0d180fdd90 msgr2=0x7f0d181001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.200+0000 7f0d17e00700 1 -- 172.21.15.57:0/3062354249 shutdown_connections 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.200+0000 7f0d17e00700 1 -- 172.21.15.57:0/3062354249 wait complete. 2024-02-28T23:49:34.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 Processor -- start 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 -- start start 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d18197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18198a30 con 0x7f0d18102e70 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d17e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1819c7c0 con 0x7f0d181024d0 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.201+0000 7f0d16000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60696/0 (socket says 172.21.15.57:60696) 2024-02-28T23:49:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 -- 172.21.15.57:0/4179285430 learned_addr learned my addr 172.21.15.57:0/4179285430 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16a00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d18197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 -- 172.21.15.57:0/4179285430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 msgr2=0x7f0d18197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d18197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 -- 172.21.15.57:0/4179285430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040095f0 con 0x7f0d18102e70 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16a00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d18197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.202+0000 7f0d16000700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400bd70 tx=0x7f0d0400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.203+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d0401d070 con 0x7f0d18102e70 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.203+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d04005dc0 con 0x7f0d18102e70 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.203+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1819ca40 con 0x7f0d18102e70 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.203+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d04017400 con 0x7f0d18102e70 2024-02-28T23:49:34.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.203+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1819cf30 con 0x7f0d18102e70 2024-02-28T23:49:34.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.204+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1810a840 con 0x7f0d18102e70 2024-02-28T23:49:34.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.205+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0d0400f7f0 con 0x7f0d18102e70 2024-02-28T23:49:34.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.206+0000 7f0d0f400700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 0x7f0cfc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.206+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d0408ea10 con 0x7f0d18102e70 2024-02-28T23:49:34.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.206+0000 7f0d16a00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 0x7f0cfc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.207+0000 7f0d16a00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 0x7f0cfc06fd50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d08005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:34.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.210+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d040581a0 con 0x7f0d18102e70 2024-02-28T23:49:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:33 smithi057 ceph-mon[89491]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:33 smithi057 ceph-mon[89491]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:34.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:34.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:33 smithi084 ceph-mon[70440]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:34.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:33 smithi084 ceph-mon[70440]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:34.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:34.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.480+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d18062380 con 0x7f0d18102e70 2024-02-28T23:49:34.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.481+0000 7f0d0f400700 1 -- 172.21.15.57:0/4179285430 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f0d0405b860 con 0x7f0d18102e70 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:e33 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:49:34.481 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:18:13.072176+0000 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:49:34.482 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:up {} 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:failed 0,1 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:34.483 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-02-28T23:49:34.484 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:49:34.484 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:34.484 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{-1:34472} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:34.484 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:34.484 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{-1:44797} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 msgr2=0x7f0cfc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 0x7f0cfc06fd50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d08005ee0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 msgr2=0x7f0d18198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400bd70 tx=0x7f0d0400be50 comp rx=0 tx=0).stop 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 shutdown_connections 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0cfc06d8a0 0x7f0cfc06fd50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d181024d0 0x7f0d18197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 --2- 172.21.15.57:0/4179285430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d18102e70 0x7f0d18198350 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 >> 172.21.15.57:0/4179285430 conn(0x7f0d180fdd90 msgr2=0x7f0d18100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 shutdown_connections 2024-02-28T23:49:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.484+0000 7f0d17e00700 1 -- 172.21.15.57:0/4179285430 wait complete. 2024-02-28T23:49:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.641+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2899829296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c0810c660 msgr2=0x7f9c0810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.641+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2899829296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c0810c660 0x7f9c0810ca30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009910 tx=0x7f9bf8009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2899829296 shutdown_connections 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2899829296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c08106b80 0x7f9c08106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2899829296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c0810c660 0x7f9c0810ca30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2899829296 >> 172.21.15.57:0/2899829296 conn(0x7f9c08075e90 msgr2=0x7f9c080782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2899829296 shutdown_connections 2024-02-28T23:49:34.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.642+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2899829296 wait complete. 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 Processor -- start 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 -- start start 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 0x7f9c0819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819cea0 con 0x7f9c08106b80 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.643+0000 7f9c0ce00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081a0bc0 con 0x7f9c0810c660 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60726/0 (socket says 172.21.15.57:60726) 2024-02-28T23:49:34.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 -- 172.21.15.57:0/2583461128 learned_addr learned my addr 172.21.15.57:0/2583461128 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:34.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c06a00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 0x7f9c0819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 -- 172.21.15.57:0/2583461128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 msgr2=0x7f9c0819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 0x7f9c0819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 -- 172.21.15.57:0/2583461128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80095f0 con 0x7f9c08106b80 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c06a00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 0x7f9c0819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.644+0000 7f9c07400700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8006010 tx=0x7f9bf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.645+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf801d070 con 0x7f9c08106b80 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.645+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf800f460 con 0x7f9c08106b80 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.645+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c081a0e40 con 0x7f9c08106b80 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.645+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf8017610 con 0x7f9c08106b80 2024-02-28T23:49:34.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.645+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c081a1330 con 0x7f9c08106b80 2024-02-28T23:49:34.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.646+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0810eb60 con 0x7f9c08106b80 2024-02-28T23:49:34.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.647+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9bf800f5d0 con 0x7f9c08106b80 2024-02-28T23:49:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.648+0000 7f9bffe00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 0x7f9bec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.648+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9bf808ea00 con 0x7f9c08106b80 2024-02-28T23:49:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.648+0000 7f9c06a00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 0x7f9bec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:34.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.649+0000 7f9c06a00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 0x7f9bec06fd50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005fd0 tx=0x7f9bf4005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.653+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bf8058250 con 0x7f9c08106b80 2024-02-28T23:49:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.907+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c081a1550 con 0x7f9bec06d8a0 2024-02-28T23:49:34.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.910+0000 7f9bffe00700 1 -- 172.21.15.57:0/2583461128 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9c081a1550 con 0x7f9bec06d8a0 2024-02-28T23:49:34.910 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "16/23 daemons upgraded", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading mds daemons", 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:49:34.911 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 msgr2=0x7f9bec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 0x7f9bec06fd50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005fd0 tx=0x7f9bf4005ee0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 msgr2=0x7f9c0819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8006010 tx=0x7f9bf800be20 comp rx=0 tx=0).stop 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 shutdown_connections 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bec06d8a0 0x7f9bec06fd50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c08106b80 0x7f9c0819c280 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 --2- 172.21.15.57:0/2583461128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c0810c660 0x7f9c0819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.913+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 >> 172.21.15.57:0/2583461128 conn(0x7f9c08075e90 msgr2=0x7f9c08077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.914+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 shutdown_connections 2024-02-28T23:49:34.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:34.914+0000 7f9c0ce00700 1 -- 172.21.15.57:0/2583461128 wait complete. 2024-02-28T23:49:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 -- 172.21.15.57:0/195018872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0106200 msgr2=0x7f7ca01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 --2- 172.21.15.57:0/195018872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0106200 0x7f7ca01065d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009910 tx=0x7f7c90009c20 comp rx=0 tx=0).stop 2024-02-28T23:49:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 -- 172.21.15.57:0/195018872 shutdown_connections 2024-02-28T23:49:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 --2- 172.21.15.57:0/195018872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0068540 0x7f7ca00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 --2- 172.21.15.57:0/195018872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0106200 0x7f7ca01065d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 -- 172.21.15.57:0/195018872 >> 172.21.15.57:0/195018872 conn(0x7f7ca0075010 msgr2=0x7f7ca0075410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:35.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 -- 172.21.15.57:0/195018872 shutdown_connections 2024-02-28T23:49:35.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.073+0000 7f7ca6000700 1 -- 172.21.15.57:0/195018872 wait complete. 2024-02-28T23:49:35.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.074+0000 7f7ca6000700 1 Processor -- start 2024-02-28T23:49:35.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.074+0000 7f7ca6000700 1 -- start start 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7ca6000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7ca6000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 0x7f7ca01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7ca6000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca0198ba0 con 0x7f7ca0068540 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7ca6000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca019c930 con 0x7f7ca0106200 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c9f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:35.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c9f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41186/0 (socket says 172.21.15.57:41186) 2024-02-28T23:49:35.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c9f400700 1 -- 172.21.15.57:0/1742090376 learned_addr learned my addr 172.21.15.57:0/1742090376 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:49:35.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c97e00700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 0x7f7ca01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c9f400700 1 -- 172.21.15.57:0/1742090376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 msgr2=0x7f7ca01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.075+0000 7f7c9f400700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 0x7f7ca01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c9f400700 1 -- 172.21.15.57:0/1742090376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900095f0 con 0x7f7ca0068540 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c97e00700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 0x7f7ca01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c9f400700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b5c0 tx=0x7f7c9000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9001d070 con 0x7f7ca0068540 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9000f460 con 0x7f7ca0068540 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca019cbb0 con 0x7f7ca0068540 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.076+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c90017610 con 0x7f7ca0068540 2024-02-28T23:49:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.077+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca019d0a0 con 0x7f7ca0068540 2024-02-28T23:49:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.078+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca004f2a0 con 0x7f7ca0068540 2024-02-28T23:49:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.079+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7c9000f5d0 con 0x7f7ca0068540 2024-02-28T23:49:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.080+0000 7f7c9cc00700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 0x7f7c8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:49:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.080+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7c9008ebe0 con 0x7f7ca0068540 2024-02-28T23:49:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.080+0000 7f7c97e00700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 0x7f7c8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:49:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.081+0000 7f7c97e00700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 0x7f7c8406fd50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ca01995a0 tx=0x7f7c88008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:49:35.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.085+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c900582f0 con 0x7f7ca0068540 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3149547867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4179285430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:35.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:35 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3149547867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4179285430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:35.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:35 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:49:35.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.411+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7ca01992e0 con 0x7f7ca0068540 2024-02-28T23:49:35.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.411+0000 7f7c9cc00700 1 -- 172.21.15.57:0/1742090376 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f7c9005b9b0 con 0x7f7ca0068540 2024-02-28T23:49:35.411 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-28T23:49:35.411 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:49:35.411 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:49:35.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:49:35.412 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is degraded 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-28T23:49:35.413 INFO:teuthology.orchestra.run.smithi057.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-28T23:49:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 msgr2=0x7f7c8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 0x7f7c8406fd50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ca01995a0 tx=0x7f7c88008040 comp rx=0 tx=0).stop 2024-02-28T23:49:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 msgr2=0x7f7ca0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:49:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b5c0 tx=0x7f7c9000be00 comp rx=0 tx=0).stop 2024-02-28T23:49:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 shutdown_connections 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f7c8406d8a0 0x7f7c8406fd50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ca0068540 0x7f7ca0197f80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 --2- 172.21.15.57:0/1742090376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0106200 0x7f7ca01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.414+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 >> 172.21.15.57:0/1742090376 conn(0x7f7ca0075010 msgr2=0x7f7ca00fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.415+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 shutdown_connections 2024-02-28T23:49:35.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:49:35.415+0000 7f7ca6000700 1 -- 172.21.15.57:0/1742090376 wait complete. 2024-02-28T23:49:36.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:36.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:36.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:36.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:36.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.bgmggd"}]': finished 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.sbkaml"}]': finished 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:49:36.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.dadqfu"}]': finished 2024-02-28T23:49:36.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:49:36.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.ztfxpc"}]': finished 2024-02-28T23:49:36.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-28T23:49:36.072 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1742090376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:49:36.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:36.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.bgmggd"}]: dispatch 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.bgmggd"}]': finished 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.sbkaml"}]: dispatch 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi057.sbkaml"}]': finished 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.dadqfu"}]: dispatch 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.dadqfu"}]': finished 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.ztfxpc"}]: dispatch 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi084.ztfxpc"}]': finished 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-28T23:49:36.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1742090376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all mds 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: Upgrade: Setting filesystem cephfs Joinable 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: Standby daemon mds.cephfs.smithi057.bgmggd assigned to filesystem cephfs as rank 0 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: Standby daemon mds.cephfs.smithi057.sbkaml assigned to filesystem cephfs as rank 1 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-28T23:49:37.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:37 smithi057 ceph-mon[89491]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:replay,1=cephfs.smithi057.sbkaml=up:replay} 2 up:standby 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all mds 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: Upgrade: Setting filesystem cephfs Joinable 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: Standby daemon mds.cephfs.smithi057.bgmggd assigned to filesystem cephfs as rank 0 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: Standby daemon mds.cephfs.smithi057.sbkaml assigned to filesystem cephfs as rank 1 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-28T23:49:37.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:37 smithi084 ceph-mon[70440]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:replay,1=cephfs.smithi057.sbkaml=up:replay} 2 up:standby 2024-02-28T23:49:38.306 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:replay,1=cephfs.smithi057.sbkaml=up:replay} 2 up:standby 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.307 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:38 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:replay,1=cephfs.smithi057.sbkaml=up:replay} 2 up:standby 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:38.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:38 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:49:39.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:39 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all rgw 2024-02-28T23:49:39.311 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:39 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all rbd-mirror 2024-02-28T23:49:39.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:39 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all rgw 2024-02-28T23:49:39.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:39 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all rbd-mirror 2024-02-28T23:49:40.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:40 smithi057 ceph-mon[89491]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 0 op/s 2024-02-28T23:49:40.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:40 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:40.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:40 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:49:40.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:40 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:40.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:40 smithi084 ceph-mon[70440]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 0 op/s 2024-02-28T23:49:40.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:40 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:40.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:40 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi057", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:49:40.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:40 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:41.039 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:41 smithi057 ceph-mon[89491]: Upgrade: Updating ceph-exporter.smithi057 (1/2) 2024-02-28T23:49:41.040 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:41 smithi057 ceph-mon[89491]: Deploying daemon ceph-exporter.smithi057 on smithi057 2024-02-28T23:49:41.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:41 smithi084 ceph-mon[70440]: Upgrade: Updating ceph-exporter.smithi057 (1/2) 2024-02-28T23:49:41.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:41 smithi084 ceph-mon[70440]: Deploying daemon ceph-exporter.smithi057 on smithi057 2024-02-28T23:49:42.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:42 smithi057 ceph-mon[89491]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-02-28T23:49:42.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:42 smithi084 ceph-mon[70440]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-02-28T23:49:44.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:44 smithi057 ceph-mon[89491]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 9 op/s 2024-02-28T23:49:44.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:44 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:resolve 2024-02-28T23:49:44.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:44 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:resolve 2024-02-28T23:49:44.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:44 smithi057 ceph-mon[89491]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:resolve,1=cephfs.smithi057.sbkaml=up:resolve} 2 up:standby 2024-02-28T23:49:44.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:44 smithi084 ceph-mon[70440]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 9 op/s 2024-02-28T23:49:44.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:44 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:resolve 2024-02-28T23:49:44.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:44 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:resolve 2024-02-28T23:49:44.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:44 smithi084 ceph-mon[70440]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:resolve,1=cephfs.smithi057.sbkaml=up:resolve} 2 up:standby 2024-02-28T23:49:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: reconnect by client.24323 192.168.0.1:0/3704145563 after 0.00499992 2024-02-28T23:49:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:reconnect 2024-02-28T23:49:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: reconnect by client.14534 192.168.0.1:0/972319470 after 0.00599991 2024-02-28T23:49:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:reconnect 2024-02-28T23:49:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:reconnect,1=cephfs.smithi057.sbkaml=up:reconnect} 2 up:standby 2024-02-28T23:49:45.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: reconnect by client.14534 192.168.0.1:0/972319470 after 0.00599991 2024-02-28T23:49:45.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:45 smithi057 ceph-mon[89491]: reconnect by client.24323 192.168.0.1:0/3704145563 after 0.00699989 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: reconnect by client.24323 192.168.0.1:0/3704145563 after 0.00499992 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:reconnect 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: reconnect by client.14534 192.168.0.1:0/972319470 after 0.00599991 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:reconnect 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:reconnect,1=cephfs.smithi057.sbkaml=up:reconnect} 2 up:standby 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: reconnect by client.14534 192.168.0.1:0/972319470 after 0.00599991 2024-02-28T23:49:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:45 smithi084 ceph-mon[70440]: reconnect by client.24323 192.168.0.1:0/3704145563 after 0.00699989 2024-02-28T23:49:46.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-02-28T23:49:46.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:rejoin 2024-02-28T23:49:46.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:rejoin 2024-02-28T23:49:46.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:rejoin,1=cephfs.smithi057.sbkaml=up:rejoin} 2 up:standby 2024-02-28T23:49:46.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: daemon mds.cephfs.smithi057.sbkaml is now active in filesystem cephfs as rank 1 2024-02-28T23:49:46.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:46 smithi057 ceph-mon[89491]: daemon mds.cephfs.smithi057.bgmggd is now active in filesystem cephfs as rank 0 2024-02-28T23:49:46.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-02-28T23:49:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:rejoin 2024-02-28T23:49:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:rejoin 2024-02-28T23:49:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: fsmap cephfs:2/2 {0=cephfs.smithi057.bgmggd=up:rejoin,1=cephfs.smithi057.sbkaml=up:rejoin} 2 up:standby 2024-02-28T23:49:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: daemon mds.cephfs.smithi057.sbkaml is now active in filesystem cephfs as rank 1 2024-02-28T23:49:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:46 smithi084 ceph-mon[70440]: daemon mds.cephfs.smithi057.bgmggd is now active in filesystem cephfs as rank 0 2024-02-28T23:49:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:47 smithi084 ceph-mon[70440]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-28T23:49:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:47 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:active 2024-02-28T23:49:47.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:47 smithi084 ceph-mon[70440]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:active 2024-02-28T23:49:47.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:47 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby 2024-02-28T23:49:47.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:47 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby-replay 2024-02-28T23:49:47.451 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:47 smithi057 ceph-mon[89491]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-28T23:49:47.451 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:47 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] up:active 2024-02-28T23:49:47.451 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:47 smithi057 ceph-mon[89491]: mds.? [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] up:active 2024-02-28T23:49:47.451 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:47 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby 2024-02-28T23:49:47.451 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:47 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby-replay 2024-02-28T23:49:48.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:48 smithi084 ceph-mon[70440]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-02-28T23:49:48.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:48.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:48 smithi057 ceph-mon[89491]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-02-28T23:49:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:48.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:49:50.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:50 smithi057 ceph-mon[89491]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 170 B/s wr, 9 op/s 2024-02-28T23:49:50.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:50 smithi084 ceph-mon[70440]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 170 B/s wr, 9 op/s 2024-02-28T23:49:51.614 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:51.614 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:51 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:51.614 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:51 smithi084 ceph-mon[70440]: pgmap v1153: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 34 MiB/s rd, 938 B/s wr, 17 op/s 2024-02-28T23:49:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:51 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:51.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:51 smithi057 ceph-mon[89491]: pgmap v1153: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 34 MiB/s rd, 938 B/s wr, 17 op/s 2024-02-28T23:49:54.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:54 smithi057 ceph-mon[89491]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 12 KiB/s wr, 20 op/s 2024-02-28T23:49:54.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:54 smithi084 ceph-mon[70440]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 12 KiB/s wr, 20 op/s 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: Upgrade: Updating ceph-exporter.smithi084 (2/2) 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: Deploying daemon ceph-exporter.smithi084 on smithi084 2024-02-28T23:49:56.062 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:55 smithi084 ceph-mon[70440]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 12 KiB/s wr, 17 op/s 2024-02-28T23:49:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: Upgrade: Updating ceph-exporter.smithi084 (2/2) 2024-02-28T23:49:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:49:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi084", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-28T23:49:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:49:56.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: Deploying daemon ceph-exporter.smithi084 on smithi084 2024-02-28T23:49:56.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:55 smithi057 ceph-mon[89491]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 12 KiB/s wr, 17 op/s 2024-02-28T23:49:58.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:49:58 smithi057 ceph-mon[89491]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 12 KiB/s wr, 18 op/s 2024-02-28T23:49:58.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:49:58 smithi084 ceph-mon[70440]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 12 KiB/s wr, 18 op/s 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 12 KiB/s wr, 18 op/s 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: Health detail: HEALTH_WARN failed to probe daemons or devices 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: /usr/bin/podman: timeout after 895 seconds 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: Traceback (most recent call last): 2024-02-28T23:50:00.321 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: return _run_code(code, main_globals, None, 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: exec(code, run_globals) 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:50:00.322 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:00 smithi057 ceph-mon[89491]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:00.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 12 KiB/s wr, 18 op/s 2024-02-28T23:50:00.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: Health detail: HEALTH_WARN failed to probe daemons or devices 2024-02-28T23:50:00.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:50:00.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:50:00.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: /usr/bin/podman: timeout after 895 seconds 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: Traceback (most recent call last): 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: return _run_code(code, main_globals, None, 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: exec(code, run_globals) 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:50:00.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:50:00.333 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:00 smithi084 ceph-mon[70440]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:02.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:02 smithi057 ceph-mon[89491]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 12 KiB/s wr, 16 op/s 2024-02-28T23:50:02.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:02 smithi084 ceph-mon[70440]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 12 KiB/s wr, 16 op/s 2024-02-28T23:50:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 11 KiB/s wr, 9 op/s 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 11 KiB/s wr, 9 op/s 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 -- 172.21.15.57:0/2864517779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e0106b80 msgr2=0x7f14e0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 --2- 172.21.15.57:0/2864517779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e0106b80 0x7f14e0106ff0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009910 tx=0x7f14cc009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 -- 172.21.15.57:0/2864517779 shutdown_connections 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 --2- 172.21.15.57:0/2864517779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e0106b80 0x7f14e0106ff0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 --2- 172.21.15.57:0/2864517779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e010c8f0 0x7f14e010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 -- 172.21.15.57:0/2864517779 >> 172.21.15.57:0/2864517779 conn(0x7f14e0076060 msgr2=0x7f14e0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.592+0000 7f14e6200700 1 -- 172.21.15.57:0/2864517779 shutdown_connections 2024-02-28T23:50:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.593+0000 7f14e6200700 1 -- 172.21.15.57:0/2864517779 wait complete. 2024-02-28T23:50:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.594+0000 7f14e6200700 1 Processor -- start 2024-02-28T23:50:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.594+0000 7f14e6200700 1 -- start start 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14e6200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 0x7f14e019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14e6200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14e6200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e019d060 con 0x7f14e010c8f0 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14e6200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a0df0 con 0x7f14e0106b80 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14df400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 0x7f14e019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59544/0 (socket says 172.21.15.57:59544) 2024-02-28T23:50:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.595+0000 7f14dea00700 1 -- 172.21.15.57:0/526156760 learned_addr learned my addr 172.21.15.57:0/526156760 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14df400700 1 -- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 msgr2=0x7f14e019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14df400700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14df400700 1 -- 172.21.15.57:0/526156760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0009520 con 0x7f14e0106b80 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14dea00700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14df400700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 0x7f14e019c440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e970 tx=0x7f14d000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d000cb40 con 0x7f14e0106b80 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc0095f0 con 0x7f14e0106b80 2024-02-28T23:50:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.596+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14d000eec0 con 0x7f14e0106b80 2024-02-28T23:50:05.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.597+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d0003e00 con 0x7f14e0106b80 2024-02-28T23:50:05.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.597+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01a1490 con 0x7f14e0106b80 2024-02-28T23:50:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.598+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e010ecc0 con 0x7f14e0106b80 2024-02-28T23:50:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.599+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f14d0004000 con 0x7f14e0106b80 2024-02-28T23:50:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.600+0000 7f14d7e00700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 0x7f14c406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.600+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f14d0014070 con 0x7f14e0106b80 2024-02-28T23:50:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.600+0000 7f14dea00700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 0x7f14c406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.601+0000 7f14dea00700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 0x7f14c406fcf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b760 tx=0x7f14cc005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:05.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.610+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14d00570c0 con 0x7f14e0106b80 2024-02-28T23:50:05.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.871+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14e01a1760 con 0x7f14c406d840 2024-02-28T23:50:05.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.873+0000 7f14d7e00700 1 -- 172.21.15.57:0/526156760 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f14e01a1760 con 0x7f14c406d840 2024-02-28T23:50:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 msgr2=0x7f14c406fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 0x7f14c406fcf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b760 tx=0x7f14cc005ad0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 msgr2=0x7f14e019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 0x7f14e019c440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e970 tx=0x7f14d000ec80 comp rx=0 tx=0).stop 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 shutdown_connections 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f14c406d840 0x7f14c406fcf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14e0106b80 0x7f14e019c440 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 --2- 172.21.15.57:0/526156760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14e010c8f0 0x7f14e019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.876+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 >> 172.21.15.57:0/526156760 conn(0x7f14e0076060 msgr2=0x7f14e0077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.877+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 shutdown_connections 2024-02-28T23:50:05.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:05.877+0000 7f14e6200700 1 -- 172.21.15.57:0/526156760 wait complete. 2024-02-28T23:50:05.894 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:50:06.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 -- 172.21.15.57:0/2507201483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 msgr2=0x7f38480734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 --2- 172.21.15.57:0/2507201483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f38480734c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f383c009910 tx=0x7f383c009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 -- 172.21.15.57:0/2507201483 shutdown_connections 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 --2- 172.21.15.57:0/2507201483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 0x7f384810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 --2- 172.21.15.57:0/2507201483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f38480734c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 -- 172.21.15.57:0/2507201483 >> 172.21.15.57:0/2507201483 conn(0x7f38480fbda0 msgr2=0x7f38480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 -- 172.21.15.57:0/2507201483 shutdown_connections 2024-02-28T23:50:06.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.047+0000 7f384fa00700 1 -- 172.21.15.57:0/2507201483 wait complete. 2024-02-28T23:50:06.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.048+0000 7f384fa00700 1 Processor -- start 2024-02-28T23:50:06.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.048+0000 7f384fa00700 1 -- start start 2024-02-28T23:50:06.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384fa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384fa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 0x7f3848198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384fa00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384819c970 con 0x7f38480730f0 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384fa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384819cae0 con 0x7f3848073a00 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384d000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384d000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59562/0 (socket says 172.21.15.57:59562) 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.049+0000 7f384d000700 1 -- 172.21.15.57:0/1796258811 learned_addr learned my addr 172.21.15.57:0/1796258811 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f3847e00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 0x7f3848198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f384d000700 1 -- 172.21.15.57:0/1796258811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 msgr2=0x7f3848198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f384d000700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 0x7f3848198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f384d000700 1 -- 172.21.15.57:0/1796258811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0095f0 con 0x7f38480730f0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f384d000700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f383c00f690 tx=0x7f383c00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.050+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383c01d070 con 0x7f38480730f0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.051+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f383c0050d0 con 0x7f38480730f0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.051+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384819cd60 con 0x7f38480730f0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.051+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383c022470 con 0x7f38480730f0 2024-02-28T23:50:06.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.051+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384819d2b0 con 0x7f38480730f0 2024-02-28T23:50:06.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.052+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3848109c80 con 0x7f38480730f0 2024-02-28T23:50:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.053+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f383c003680 con 0x7f38480730f0 2024-02-28T23:50:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.054+0000 7f3845600700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 0x7f383006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.054+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f383c08e900 con 0x7f38480730f0 2024-02-28T23:50:06.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.054+0000 7f3847e00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 0x7f383006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.055+0000 7f3847e00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 0x7f383006fd50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3834009bf0 tx=0x7f3834009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.059+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f383c058130 con 0x7f38480730f0 2024-02-28T23:50:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:06 smithi057 ceph-mon[89491]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 352 KiB/s rd, 3 op/s 2024-02-28T23:50:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:06 smithi084 ceph-mon[70440]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 352 KiB/s rd, 3 op/s 2024-02-28T23:50:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.347+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3848199320 con 0x7f383006d8a0 2024-02-28T23:50:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.350+0000 7f3845600700 1 -- 172.21.15.57:0/1796258811 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f3848199320 con 0x7f383006d8a0 2024-02-28T23:50:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.352+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 msgr2=0x7f383006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.352+0000 7f384fa00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 0x7f383006fd50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3834009bf0 tx=0x7f3834009260 comp rx=0 tx=0).stop 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.352+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 msgr2=0x7f3848197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.352+0000 7f384fa00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f383c00f690 tx=0x7f383c00f770 comp rx=0 tx=0).stop 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 shutdown_connections 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f383006d8a0 0x7f383006fd50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38480730f0 0x7f3848197fc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 --2- 172.21.15.57:0/1796258811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3848073a00 0x7f3848198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 >> 172.21.15.57:0/1796258811 conn(0x7f38480fbda0 msgr2=0x7f3848106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 shutdown_connections 2024-02-28T23:50:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.353+0000 7f384fa00700 1 -- 172.21.15.57:0/1796258811 wait complete. 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 -- 172.21.15.57:0/1090246117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac106320 msgr2=0x7f80ac1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 --2- 172.21.15.57:0/1090246117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac106320 0x7f80ac1066f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 -- 172.21.15.57:0/1090246117 shutdown_connections 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 --2- 172.21.15.57:0/1090246117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac068540 0x7f80ac0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 --2- 172.21.15.57:0/1090246117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac106320 0x7f80ac1066f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 -- 172.21.15.57:0/1090246117 >> 172.21.15.57:0/1090246117 conn(0x7f80ac075130 msgr2=0x7f80ac075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:06.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 -- 172.21.15.57:0/1090246117 shutdown_connections 2024-02-28T23:50:06.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.503+0000 7f80b1a00700 1 -- 172.21.15.57:0/1090246117 wait complete. 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.504+0000 7f80b1a00700 1 Processor -- start 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.504+0000 7f80b1a00700 1 -- start start 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80b1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80b1a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac106320 0x7f80ac1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80b1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac196a90 con 0x7f80ac068540 2024-02-28T23:50:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80b1a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac19a820 con 0x7f80ac106320 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59576/0 (socket says 172.21.15.57:59576) 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80aaa00700 1 -- 172.21.15.57:0/2695377409 learned_addr learned my addr 172.21.15.57:0/2695377409 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.505+0000 7f80aa000700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac106320 0x7f80ac1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80aaa00700 1 -- 172.21.15.57:0/2695377409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac106320 msgr2=0x7f80ac1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80aaa00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac106320 0x7f80ac1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80aaa00700 1 -- 172.21.15.57:0/2695377409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c0095f0 con 0x7f80ac068540 2024-02-28T23:50:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80aaa00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f809c00b5c0 tx=0x7f809c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c01d070 con 0x7f80ac068540 2024-02-28T23:50:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f809c005e00 con 0x7f80ac068540 2024-02-28T23:50:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac19aaa0 con 0x7f80ac068540 2024-02-28T23:50:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.506+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c017400 con 0x7f80ac068540 2024-02-28T23:50:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.507+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac19af90 con 0x7f80ac068540 2024-02-28T23:50:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.508+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac108790 con 0x7f80ac068540 2024-02-28T23:50:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.508+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f809c003a60 con 0x7f80ac068540 2024-02-28T23:50:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.510+0000 7f80a3400700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 0x7f809006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.510+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f809c08ea40 con 0x7f80ac068540 2024-02-28T23:50:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.510+0000 7f80aa000700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 0x7f809006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.511+0000 7f80aa000700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 0x7f809006fd50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80ac197490 tx=0x7f809800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.515+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f809c058150 con 0x7f80ac068540 2024-02-28T23:50:06.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.756+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80ac19b1b0 con 0x7f809006d8a0 2024-02-28T23:50:06.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.797+0000 7f80a3400700 1 -- 172.21.15.57:0/2695377409 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f80ac19b1b0 con 0x7f809006d8a0 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (39m) 15m ago 53m 14.2M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 starting - - - - 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 starting - - - - 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (35m) 15m ago 53m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (35m) 15m ago 52m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:50:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (39m) 15m ago 53m 76.9M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (31m) 15m ago 46m 18.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (31m) 15m ago 46m 18.7M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (15m) 15m ago 46m 10.9M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (30m) 15m ago 46m 18.5M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (42m) 15m ago 55m 510M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (41m) 15m ago 52m 458M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (37m) 15m ago 55m 71.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (36m) 15m ago 52m 53.5M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:50:06.799 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (40m) 15m ago 53m 8342k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (40m) 15m ago 52m 8686k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (35m) 15m ago 51m 96.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (34m) 15m ago 50m 106M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (34m) 15m ago 49m 86.9M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (33m) 15m ago 49m 110M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (32m) 15m ago 48m 104M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (32m) 15m ago 48m 90.4M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:50:06.800 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (40m) 15m ago 52m 79.3M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:50:06.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 msgr2=0x7f809006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 0x7f809006fd50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80ac197490 tx=0x7f809800b410 comp rx=0 tx=0).stop 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 msgr2=0x7f80ac195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f809c00b5c0 tx=0x7f809c00bd70 comp rx=0 tx=0).stop 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 shutdown_connections 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f809006d8a0 0x7f809006fd50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80ac068540 0x7f80ac195e70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 --2- 172.21.15.57:0/2695377409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80ac106320 0x7f80ac1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 >> 172.21.15.57:0/2695377409 conn(0x7f80ac075130 msgr2=0x7f80ac0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.800+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 shutdown_connections 2024-02-28T23:50:06.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.801+0000 7f80b1a00700 1 -- 172.21.15.57:0/2695377409 wait complete. 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 -- 172.21.15.57:0/3295607524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 msgr2=0x7f88cc10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 --2- 172.21.15.57:0/3295607524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc10a230 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009910 tx=0x7f88b8009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 -- 172.21.15.57:0/3295607524 shutdown_connections 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 --2- 172.21.15.57:0/3295607524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc10a230 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 --2- 172.21.15.57:0/3295607524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.966+0000 7f88d4400700 1 -- 172.21.15.57:0/3295607524 >> 172.21.15.57:0/3295607524 conn(0x7f88cc0fac80 msgr2=0x7f88cc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:06.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.967+0000 7f88d4400700 1 -- 172.21.15.57:0/3295607524 shutdown_connections 2024-02-28T23:50:06.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.967+0000 7f88d4400700 1 -- 172.21.15.57:0/3295607524 wait complete. 2024-02-28T23:50:06.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.968+0000 7f88d4400700 1 Processor -- start 2024-02-28T23:50:06.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.968+0000 7f88d4400700 1 -- start start 2024-02-28T23:50:06.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.968+0000 7f88d4400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.968+0000 7f88d4400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d4400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc196a80 con 0x7f88cc101d30 2024-02-28T23:50:06.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d4400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc19a810 con 0x7f88cc101420 2024-02-28T23:50:06.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d1000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59592/0 (socket says 172.21.15.57:59592) 2024-02-28T23:50:06.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d1000700 1 -- 172.21.15.57:0/194786856 learned_addr learned my addr 172.21.15.57:0/194786856 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:06.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.969+0000 7f88d1a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88d1000700 1 -- 172.21.15.57:0/194786856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 msgr2=0x7f88cc195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88d1000700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88d1000700 1 -- 172.21.15.57:0/194786856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80095f0 con 0x7f88cc101d30 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88d1a00700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88d1000700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f88b800f690 tx=0x7f88b800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.970+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b801d070 con 0x7f88cc101d30 2024-02-28T23:50:06.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.971+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b80050d0 con 0x7f88cc101d30 2024-02-28T23:50:06.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.971+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b8022470 con 0x7f88cc101d30 2024-02-28T23:50:06.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.971+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc19aa90 con 0x7f88cc101d30 2024-02-28T23:50:06.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.971+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc19aea0 con 0x7f88cc101d30 2024-02-28T23:50:06.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.972+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc1078a0 con 0x7f88cc101d30 2024-02-28T23:50:06.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.973+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f88b8003780 con 0x7f88cc101d30 2024-02-28T23:50:06.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.975+0000 7f88ca000700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 0x7f88b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:06.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.975+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f88b808ea20 con 0x7f88cc101d30 2024-02-28T23:50:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.975+0000 7f88d1a00700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 0x7f88b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:06.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.976+0000 7f88d1a00700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 0x7f88b406fd50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011490 tx=0x7f88c0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:06.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:06.980+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f88b8058130 con 0x7f88cc101d30 2024-02-28T23:50:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:07 smithi057 ceph-mon[89491]: from='client.45213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.319+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f88cc04f2a0 con 0x7f88cc101d30 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.319+0000 7f88ca000700 1 -- 172.21.15.57:0/194786856 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f88b805b7f0 con 0x7f88cc101d30 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:07.320 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:50:07.321 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:50:07.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 msgr2=0x7f88b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 0x7f88b406fd50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011490 tx=0x7f88c0009260 comp rx=0 tx=0).stop 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 msgr2=0x7f88cc1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f88b800f690 tx=0x7f88b800f6c0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 shutdown_connections 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f88b406d8a0 0x7f88b406fd50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc101420 0x7f88cc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 --2- 172.21.15.57:0/194786856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88cc101d30 0x7f88cc1963a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 >> 172.21.15.57:0/194786856 conn(0x7f88cc0fac80 msgr2=0x7f88cc1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 shutdown_connections 2024-02-28T23:50:07.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.322+0000 7f88d4400700 1 -- 172.21.15.57:0/194786856 wait complete. 2024-02-28T23:50:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:07 smithi084 ceph-mon[70440]: from='client.45213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:07.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 -- 172.21.15.57:0/1373886480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 msgr2=0x7efe6c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 --2- 172.21.15.57:0/1373886480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c104e50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7efe58009910 tx=0x7efe58009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:07.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 -- 172.21.15.57:0/1373886480 shutdown_connections 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 --2- 172.21.15.57:0/1373886480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c104e50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 --2- 172.21.15.57:0/1373886480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 0x7efe6c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 -- 172.21.15.57:0/1373886480 >> 172.21.15.57:0/1373886480 conn(0x7efe6c0fbda0 msgr2=0x7efe6c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 -- 172.21.15.57:0/1373886480 shutdown_connections 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.491+0000 7efe74000700 1 -- 172.21.15.57:0/1373886480 wait complete. 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.492+0000 7efe74000700 1 Processor -- start 2024-02-28T23:50:07.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe74000700 1 -- start start 2024-02-28T23:50:07.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe74000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 0x7efe6c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe74000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe74000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c198d30 con 0x7efe6c100e80 2024-02-28T23:50:07.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe74000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19cac0 con 0x7efe6c1004e0 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe70c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe70c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59616/0 (socket says 172.21.15.57:59616) 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.493+0000 7efe70c00700 1 -- 172.21.15.57:0/4012765251 learned_addr learned my addr 172.21.15.57:0/4012765251 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.494+0000 7efe71600700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 0x7efe6c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.494+0000 7efe70c00700 1 -- 172.21.15.57:0/4012765251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 msgr2=0x7efe6c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.494+0000 7efe70c00700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 0x7efe6c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.494+0000 7efe70c00700 1 -- 172.21.15.57:0/4012765251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580095f0 con 0x7efe6c100e80 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.494+0000 7efe70c00700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efe5800bda0 tx=0x7efe5800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.495+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe5801d070 con 0x7efe6c100e80 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.495+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe58005dc0 con 0x7efe6c100e80 2024-02-28T23:50:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.495+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c19cd40 con 0x7efe6c100e80 2024-02-28T23:50:07.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.495+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe58017400 con 0x7efe6c100e80 2024-02-28T23:50:07.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.495+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c19d230 con 0x7efe6c100e80 2024-02-28T23:50:07.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.496+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c1045d0 con 0x7efe6c100e80 2024-02-28T23:50:07.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.497+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efe5800f6f0 con 0x7efe6c100e80 2024-02-28T23:50:07.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.498+0000 7efe6a000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 0x7efe5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.498+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efe5808e9c0 con 0x7efe6c100e80 2024-02-28T23:50:07.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.498+0000 7efe71600700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 0x7efe5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.499+0000 7efe71600700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 0x7efe5406fd50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efe60005950 tx=0x7efe6000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:07.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.503+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe580580d0 con 0x7efe6c100e80 2024-02-28T23:50:07.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.780+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe6c04f2a0 con 0x7efe6c100e80 2024-02-28T23:50:07.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.780+0000 7efe6a000700 1 -- 172.21.15.57:0/4012765251 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1890 (secure 0 0 0) 0x7efe5805b790 con 0x7efe6c100e80 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:e41 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 41 2024-02-28T23:50:07.781 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:49:46.183645+0000 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-28T23:50:07.782 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=34440,1=34472} 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:50:07.783 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{0:34440} state up:active seq 475 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{0:44339} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{1:34472} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{1:44797} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:50:07.784 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 msgr2=0x7efe5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 0x7efe5406fd50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efe60005950 tx=0x7efe6000b410 comp rx=0 tx=0).stop 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 msgr2=0x7efe6c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efe5800bda0 tx=0x7efe5800bdd0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 shutdown_connections 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe5406d8a0 0x7efe5406fd50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe6c1004e0 0x7efe6c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.783+0000 7efe74000700 1 --2- 172.21.15.57:0/4012765251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe6c100e80 0x7efe6c198650 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.784+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 >> 172.21.15.57:0/4012765251 conn(0x7efe6c0fbda0 msgr2=0x7efe6c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:07.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.784+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 shutdown_connections 2024-02-28T23:50:07.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.784+0000 7efe74000700 1 -- 172.21.15.57:0/4012765251 wait complete. 2024-02-28T23:50:07.786 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 41 2024-02-28T23:50:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 -- 172.21.15.57:0/2230875847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 msgr2=0x7ff39c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2230875847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c068b70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009910 tx=0x7ff38c009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 -- 172.21.15.57:0/2230875847 shutdown_connections 2024-02-28T23:50:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2230875847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2230875847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c068b70 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 -- 172.21.15.57:0/2230875847 >> 172.21.15.57:0/2230875847 conn(0x7ff39c075130 msgr2=0x7ff39c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 -- 172.21.15.57:0/2230875847 shutdown_connections 2024-02-28T23:50:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.944+0000 7ff3a2400700 1 -- 172.21.15.57:0/2230875847 wait complete. 2024-02-28T23:50:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.945+0000 7ff3a2400700 1 Processor -- start 2024-02-28T23:50:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.946+0000 7ff3a2400700 1 -- start start 2024-02-28T23:50:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.946+0000 7ff3a2400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.946+0000 7ff3a2400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.946+0000 7ff3a2400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c198cd0 con 0x7ff39c0687a0 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.946+0000 7ff3a2400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c19ca60 con 0x7ff39c069140 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.57:44176/0 (socket says 172.21.15.57:44176) 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59642/0 (socket says 172.21.15.57:59642) 2024-02-28T23:50:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 -- 172.21.15.57:0/2711300050 learned_addr learned my addr 172.21.15.57:0/2711300050 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 -- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 msgr2=0x7ff39c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.947+0000 7ff39aa00700 1 -- 172.21.15.57:0/2711300050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0095f0 con 0x7ff39c069140 2024-02-28T23:50:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff39aa00700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e8f0 tx=0x7ff38800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff388009790 con 0x7ff39c069140 2024-02-28T23:50:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c19cd40 con 0x7ff39c069140 2024-02-28T23:50:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff38800c810 con 0x7ff39c069140 2024-02-28T23:50:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff388005580 con 0x7ff39c069140 2024-02-28T23:50:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.948+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c19d290 con 0x7ff39c069140 2024-02-28T23:50:07.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.950+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39c04f2a0 con 0x7ff39c069140 2024-02-28T23:50:07.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.952+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff38800c980 con 0x7ff39c069140 2024-02-28T23:50:07.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.952+0000 7ff393e00700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 0x7ff38006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:07.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.953+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff388014070 con 0x7ff39c069140 2024-02-28T23:50:07.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.953+0000 7ff39b400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 0x7ff38006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:07.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.954+0000 7ff39b400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 0x7ff38006fd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00b760 tx=0x7ff38c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:07.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:07.966+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff388057050 con 0x7ff39c069140 2024-02-28T23:50:08.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.220+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff39c19d620 con 0x7ff38006d8a0 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='client.36054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='client.36058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 3 op/s 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/194786856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:08.221 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:08 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4012765251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:50:08.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.222+0000 7ff393e00700 1 -- 172.21.15.57:0/2711300050 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7ff39c19d620 con 0x7ff38006d8a0 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": true, 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [ 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "mon", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "osd", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "mds", 2024-02-28T23:50:08.223 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr" 2024-02-28T23:50:08.224 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-02-28T23:50:08.224 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": "17/23 daemons upgraded", 2024-02-28T23:50:08.224 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-02-28T23:50:08.224 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:50:08.224 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.225+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 msgr2=0x7ff38006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.225+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 0x7ff38006fd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00b760 tx=0x7ff38c005a90 comp rx=0 tx=0).stop 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.225+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 msgr2=0x7ff39c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.225+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e8f0 tx=0x7ff38800ec00 comp rx=0 tx=0).stop 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 shutdown_connections 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff38006d8a0 0x7ff38006fd50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39c0687a0 0x7ff39c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 --2- 172.21.15.57:0/2711300050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff39c069140 0x7ff39c1985f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 >> 172.21.15.57:0/2711300050 conn(0x7ff39c075130 msgr2=0x7ff39c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 shutdown_connections 2024-02-28T23:50:08.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.226+0000 7ff3a2400700 1 -- 172.21.15.57:0/2711300050 wait complete. 2024-02-28T23:50:08.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 -- 172.21.15.57:0/1074727311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101360 msgr2=0x7fda44101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 --2- 172.21.15.57:0/1074727311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101360 0x7fda44101730 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fda34009910 tx=0x7fda34009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 -- 172.21.15.57:0/1074727311 shutdown_connections 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 --2- 172.21.15.57:0/1074727311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101c70 0x7fda4410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 --2- 172.21.15.57:0/1074727311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101360 0x7fda44101730 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 -- 172.21.15.57:0/1074727311 >> 172.21.15.57:0/1074727311 conn(0x7fda440fab60 msgr2=0x7fda440fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 -- 172.21.15.57:0/1074727311 shutdown_connections 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.383+0000 7fda4a400700 1 -- 172.21.15.57:0/1074727311 wait complete. 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.384+0000 7fda4a400700 1 Processor -- start 2024-02-28T23:50:08.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda4a400700 1 -- start start 2024-02-28T23:50:08.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda4a400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101360 0x7fda44195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:08.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda4a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:08.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda4a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda44196870 con 0x7fda44101c70 2024-02-28T23:50:08.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda4a400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441995a0 con 0x7fda44101360 2024-02-28T23:50:08.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda3aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:08.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda3aa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59658/0 (socket says 172.21.15.57:59658) 2024-02-28T23:50:08.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda3aa00700 1 -- 172.21.15.57:0/691339556 learned_addr learned my addr 172.21.15.57:0/691339556 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:08.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.385+0000 7fda43400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101360 0x7fda44195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:08.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.386+0000 7fda3aa00700 1 -- 172.21.15.57:0/691339556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101360 msgr2=0x7fda44195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.386+0000 7fda3aa00700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101360 0x7fda44195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.386+0000 7fda3aa00700 1 -- 172.21.15.57:0/691339556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340095f0 con 0x7fda44101c70 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.386+0000 7fda3aa00700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d8d0 tx=0x7fda2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.386+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda2c0098a0 con 0x7fda44101c70 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.387+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda2c010460 con 0x7fda44101c70 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.387+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda44199880 con 0x7fda44101c70 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.387+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda2c00f5f0 con 0x7fda44101c70 2024-02-28T23:50:08.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.387+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda44199dd0 con 0x7fda44101c70 2024-02-28T23:50:08.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.388+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44107840 con 0x7fda44101c70 2024-02-28T23:50:08.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.389+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fda2c009a00 con 0x7fda44101c70 2024-02-28T23:50:08.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.391+0000 7fda40c00700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 0x7fda2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:08.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.391+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda2c08d330 con 0x7fda44101c70 2024-02-28T23:50:08.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.391+0000 7fda43400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 0x7fda2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:08.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.392+0000 7fda43400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 0x7fda2806fd50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:08.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.395+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda2c056b60 con 0x7fda44101c70 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='client.36054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='client.36058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 3 op/s 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:08.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/194786856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:08.445 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:08 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4012765251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:50:08.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.734+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fda4419a100 con 0x7fda44101c70 2024-02-28T23:50:08.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.735+0000 7fda40c00700 1 -- 172.21.15.57:0/691339556 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4081 (secure 0 0 0) 0x7fda2c05a220 con 0x7fda44101c70 2024-02-28T23:50:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_WARN failed to probe daemons or devices 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: host smithi084 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi084/config 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout:Traceback (most recent call last): 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: return _run_code(code, main_globals, None, 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: exec(code, run_globals) 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5520, in 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 5508, in main 2024-02-28T23:50:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 406, in _infer_config 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 321, in _infer_fsid 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 434, in _infer_image 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 308, in _validate_fsid 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/__main__.py", line 3301, in command_ceph_volume 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: File "/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/cephadm.8e42bbf4765eafad2b4b094655392168f9104bcdebfde51be0935be4834d689e/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-28T23:50:08.737 INFO:teuthology.orchestra.run.smithi057.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc -e NODE_NAME=smithi084 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/run/ceph:z -v /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbrqb_qb8:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc inventory --format=json-pretty --filter-for-batch 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.737+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 msgr2=0x7fda2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.737+0000 7fda4a400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 0x7fda2806fd50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda3401a040 comp rx=0 tx=0).stop 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.737+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 msgr2=0x7fda441961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.737+0000 7fda4a400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d8d0 tx=0x7fda2c00dbe0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 shutdown_connections 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda2806d8a0 0x7fda2806fd50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda44101360 0x7fda44195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 --2- 172.21.15.57:0/691339556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda44101c70 0x7fda441961e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:08.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 >> 172.21.15.57:0/691339556 conn(0x7fda440fab60 msgr2=0x7fda44104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:08.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 shutdown_connections 2024-02-28T23:50:08.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:08.738+0000 7fda4a400700 1 -- 172.21.15.57:0/691339556 wait complete. 2024-02-28T23:50:09.398 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:09 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/691339556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:50:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:09 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/691339556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:50:10.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:10 smithi057 ceph-mon[89491]: from='client.45233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:10.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:10 smithi057 ceph-mon[89491]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 3 op/s 2024-02-28T23:50:10.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:10.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:10 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:10.443 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:10 smithi084 ceph-mon[70440]: from='client.45233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:10.443 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:10 smithi084 ceph-mon[70440]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 3 op/s 2024-02-28T23:50:10.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:10.444 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:10 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:11 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:11 smithi084 ceph-mon[70440]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:11 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:11.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:11 smithi057 ceph-mon[89491]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:13.755 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:13.755 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:13 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:13.755 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:13 smithi057 ceph-mon[89491]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:13 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:13.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:13 smithi084 ceph-mon[70440]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:15.930 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:15 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:15.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:15 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:15.931 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:15 smithi057 ceph-mon[89491]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:16.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:15 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:16.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:15 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:16.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:15 smithi084 ceph-mon[70440]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:18.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:18.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:18.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:18.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:18.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:18.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:18 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:18.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:18 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-28T23:50:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:19 smithi057 ceph-mon[89491]: Cluster is now healthy 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:19.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-28T23:50:19.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-28T23:50:19.332 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:19 smithi084 ceph-mon[70440]: Cluster is now healthy 2024-02-28T23:50:20.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Setting filesystem cephfs Joinable 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby-replay 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all ceph-exporter 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi057"}]: dispatch 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi057"}]': finished 2024-02-28T23:50:20.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi084"}]: dispatch 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi084"}]': finished 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all iscsi 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all nfs 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Setting container_image for all nvmeof 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.573 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: Upgrade: Finalizing container_image settings 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-28T23:50:20.574 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.575 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:20.576 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:20 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Setting filesystem cephfs Joinable 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: fsmap cephfs:2 {0=cephfs.smithi057.bgmggd=up:active,1=cephfs.smithi057.sbkaml=up:active} 2 up:standby-replay 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all ceph-exporter 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi057"}]: dispatch 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi057"}]': finished 2024-02-28T23:50:20.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi084"}]: dispatch 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi084"}]': finished 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all iscsi 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all nfs 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Setting container_image for all nvmeof 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: Upgrade: Finalizing container_image settings 2024-02-28T23:50:20.584 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-28T23:50:20.585 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.586 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:50:20.587 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:20 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:21.129 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:21 smithi057 ceph-mon[89491]: Upgrade: Complete! 2024-02-28T23:50:21.130 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:21 smithi057 ceph-mon[89491]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:50:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:21 smithi084 ceph-mon[70440]: Upgrade: Complete! 2024-02-28T23:50:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:21 smithi084 ceph-mon[70440]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:50:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:23 smithi057 ceph-mon[89491]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:23.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:23 smithi084 ceph-mon[70440]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:25.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:24 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:25.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:24 smithi057 ceph-mon[89491]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:50:25.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:24 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:25.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:24 smithi084 ceph-mon[70440]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-28T23:50:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:26 smithi057 ceph-mon[89491]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:50:27.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:26 smithi084 ceph-mon[70440]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-28T23:50:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:28 smithi057 ceph-mon[89491]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:29.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:28 smithi084 ceph-mon[70440]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-02-28T23:50:31.071 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:30 smithi057 ceph-mon[89491]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:31.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:30 smithi084 ceph-mon[70440]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:33.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:32 smithi057 ceph-mon[89491]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:33.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:32 smithi084 ceph-mon[70440]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:50:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:35 smithi057 ceph-mon[89491]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:35.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:35 smithi084 ceph-mon[70440]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:37 smithi057 ceph-mon[89491]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:37.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:37 smithi084 ceph-mon[70440]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.916+0000 7f22a6c00700 1 -- 172.21.15.57:0/3360343518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0108470 msgr2=0x7f22a0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.916+0000 7f22a6c00700 1 --2- 172.21.15.57:0/3360343518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0108470 0x7f22a0108840 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2290009910 tx=0x7f2290009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 -- 172.21.15.57:0/3360343518 shutdown_connections 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 --2- 172.21.15.57:0/3360343518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0102470 0x7f22a01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 --2- 172.21.15.57:0/3360343518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0108470 0x7f22a0108840 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 -- 172.21.15.57:0/3360343518 >> 172.21.15.57:0/3360343518 conn(0x7f22a00fdf90 msgr2=0x7f22a01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 -- 172.21.15.57:0/3360343518 shutdown_connections 2024-02-28T23:50:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.917+0000 7f22a6c00700 1 -- 172.21.15.57:0/3360343518 wait complete. 2024-02-28T23:50:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 Processor -- start 2024-02-28T23:50:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 -- start start 2024-02-28T23:50:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 0x7f22a0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0198e30 con 0x7f22a0102470 2024-02-28T23:50:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f22a6c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a019cbc0 con 0x7f22a0108470 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.918+0000 7f229fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58308/0 (socket says 172.21.15.57:58308) 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 -- 172.21.15.57:0/1664762534 learned_addr learned my addr 172.21.15.57:0/1664762534 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f2297e00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 0x7f22a0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 -- 172.21.15.57:0/1664762534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 msgr2=0x7f22a0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 0x7f22a0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 -- 172.21.15.57:0/1664762534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900095f0 con 0x7f22a0102470 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f2297e00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 0x7f22a0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.919+0000 7f229fe00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2290006010 tx=0x7f229000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.920+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f229001d070 con 0x7f22a0102470 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.920+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2290004fb0 con 0x7f22a0102470 2024-02-28T23:50:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.920+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a019ce40 con 0x7f22a0102470 2024-02-28T23:50:38.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.920+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f229000f690 con 0x7f22a0102470 2024-02-28T23:50:38.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.920+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a019d330 con 0x7f22a0102470 2024-02-28T23:50:38.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.921+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a010a9e0 con 0x7f22a0102470 2024-02-28T23:50:38.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.922+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f22900228a0 con 0x7f22a0102470 2024-02-28T23:50:38.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.923+0000 7f229d600700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 0x7f228406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:38.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.923+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f229008ede0 con 0x7f22a0102470 2024-02-28T23:50:38.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.923+0000 7f2297e00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 0x7f228406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:38.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.924+0000 7f2297e00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 0x7f228406fd50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22a0199830 tx=0x7f2288008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:38.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:38.928+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2290058570 con 0x7f22a0102470 2024-02-28T23:50:39.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.173+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22a019d5a0 con 0x7f228406d8a0 2024-02-28T23:50:39.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.175+0000 7f229d600700 1 -- 172.21.15.57:0/1664762534 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f22a019d5a0 con 0x7f228406d8a0 2024-02-28T23:50:39.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 msgr2=0x7f228406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:39.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 0x7f228406fd50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22a0199830 tx=0x7f2288008040 comp rx=0 tx=0).stop 2024-02-28T23:50:39.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 msgr2=0x7f22a0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2290006010 tx=0x7f229000be20 comp rx=0 tx=0).stop 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 shutdown_connections 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f228406d8a0 0x7f228406fd50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22a0102470 0x7f22a0198210 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 --2- 172.21.15.57:0/1664762534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a0108470 0x7f22a0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.178+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 >> 172.21.15.57:0/1664762534 conn(0x7f22a00fdf90 msgr2=0x7f22a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.179+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 shutdown_connections 2024-02-28T23:50:39.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:39.179+0000 7f22a6c00700 1 -- 172.21.15.57:0/1664762534 wait complete. 2024-02-28T23:50:39.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:39 smithi084 ceph-mon[70440]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:39 smithi057 ceph-mon[89491]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:39.981 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch ps' 2024-02-28T23:50:40.465 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:40.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:40 smithi057 ceph-mon[89491]: from='client.36078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:40.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:40 smithi084 ceph-mon[70440]: from='client.36078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:41.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:41 smithi057 ceph-mon[89491]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:41.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:41 smithi084 ceph-mon[70440]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.592+0000 7f60d3e00700 1 -- 172.21.15.57:0/463823772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ff1d0 msgr2=0x7f60d40ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.592+0000 7f60d3e00700 1 --2- 172.21.15.57:0/463823772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ff1d0 0x7f60d40ff5a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009910 tx=0x7f60c4009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 -- 172.21.15.57:0/463823772 shutdown_connections 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 --2- 172.21.15.57:0/463823772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ffb70 0x7f60d4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 --2- 172.21.15.57:0/463823772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ff1d0 0x7f60d40ff5a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 -- 172.21.15.57:0/463823772 >> 172.21.15.57:0/463823772 conn(0x7f60d40faa90 msgr2=0x7f60d40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 -- 172.21.15.57:0/463823772 shutdown_connections 2024-02-28T23:50:42.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.593+0000 7f60d3e00700 1 -- 172.21.15.57:0/463823772 wait complete. 2024-02-28T23:50:42.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.594+0000 7f60d3e00700 1 Processor -- start 2024-02-28T23:50:42.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d3e00700 1 -- start start 2024-02-28T23:50:42.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d3e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 0x7f60d4100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:42.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:42.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d3e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4101660 con 0x7f60d40ffb70 2024-02-28T23:50:42.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d3e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41017a0 con 0x7f60d40ff1d0 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d2000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58326/0 (socket says 172.21.15.57:58326) 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.595+0000 7f60d2000700 1 -- 172.21.15.57:0/4140714222 learned_addr learned my addr 172.21.15.57:0/4140714222 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2a00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 0x7f60d4100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2000700 1 -- 172.21.15.57:0/4140714222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 msgr2=0x7f60d4100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2000700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 0x7f60d4100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2000700 1 -- 172.21.15.57:0/4140714222 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40095f0 con 0x7f60d40ffb70 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2a00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 0x7f60d4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.596+0000 7f60d2000700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d900 tx=0x7f60c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.597+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c0009900 con 0x7f60d40ffb70 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.597+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60c0010460 con 0x7f60d40ffb70 2024-02-28T23:50:42.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.597+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d41a1e50 con 0x7f60d40ffb70 2024-02-28T23:50:42.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.597+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c000f5f0 con 0x7f60d40ffb70 2024-02-28T23:50:42.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.597+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d41a2260 con 0x7f60d40ffb70 2024-02-28T23:50:42.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.598+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d41084f0 con 0x7f60d40ffb70 2024-02-28T23:50:42.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.599+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f60c00105d0 con 0x7f60d40ffb70 2024-02-28T23:50:42.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.600+0000 7f60cb400700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:42.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.600+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f60c008d430 con 0x7f60d40ffb70 2024-02-28T23:50:42.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.600+0000 7f60d2a00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:42.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.601+0000 7f60d2a00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f60c4005b40 tx=0x7f60c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:42.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.605+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60c0056b40 con 0x7f60d40ffb70 2024-02-28T23:50:42.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.852+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d402cca0 con 0x7f60b806d8a0 2024-02-28T23:50:42.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.885+0000 7f60cb400700 1 -- 172.21.15.57:0/4140714222 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f60d402cca0 con 0x7f60b806d8a0 2024-02-28T23:50:42.885 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:50:42.885 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (40m) 32s ago 54m 14.5M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:50:42.885 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (53s) 32s ago 54m 8006k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 814d9055bf9c 2024-02-28T23:50:42.885 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (40s) 35s ago 53m 7113k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 4742e02d5b02 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (36m) 32s ago 54m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (36m) 35s ago 52m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (39m) 32s ago 53m 77.0M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (32m) 32s ago 46m 61.1M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (31m) 32s ago 46m 60.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (16m) 35s ago 46m 59.6M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (31m) 35s ago 46m 60.6M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (42m) 32s ago 56m 515M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (41m) 35s ago 52m 459M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (38m) 32s ago 56m 86.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (37m) 35s ago 52m 60.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (41m) 32s ago 54m 8480k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (41m) 35s ago 52m 8430k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:50:42.886 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (35m) 32s ago 51m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (35m) 32s ago 51m 116M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (34m) 32s ago 50m 94.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (34m) 35s ago 49m 123M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (33m) 35s ago 49m 111M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (32m) 35s ago 48m 98.9M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:50:42.887 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (40m) 32s ago 53m 83.4M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 msgr2=0x7f60b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f60c4005b40 tx=0x7f60c401a040 comp rx=0 tx=0).stop 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 msgr2=0x7f60d4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d900 tx=0x7f60c000dcc0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 shutdown_connections 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f60b806d8a0 0x7f60b806fd50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d40ff1d0 0x7f60d4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 --2- 172.21.15.57:0/4140714222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d40ffb70 0x7f60d4101120 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 >> 172.21.15.57:0/4140714222 conn(0x7f60d40faa90 msgr2=0x7f60d4068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 shutdown_connections 2024-02-28T23:50:42.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:42.889+0000 7f60d3e00700 1 -- 172.21.15.57:0/4140714222 wait complete. 2024-02-28T23:50:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:43 smithi057 ceph-mon[89491]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:43 smithi084 ceph-mon[70440]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:43.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch upgrade status' 2024-02-28T23:50:44.154 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:44.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:44 smithi057 ceph-mon[89491]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:44.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:44 smithi084 ceph-mon[70440]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:45.509 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:45 smithi057 ceph-mon[89491]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:45 smithi084 ceph-mon[70440]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:46.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 -- 172.21.15.57:0/3588575384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 msgr2=0x7f11d01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 --2- 172.21.15.57:0/3588575384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01008b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009910 tx=0x7f11c0009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 -- 172.21.15.57:0/3588575384 shutdown_connections 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 --2- 172.21.15.57:0/3588575384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 0x7f11d0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 --2- 172.21.15.57:0/3588575384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01008b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 -- 172.21.15.57:0/3588575384 >> 172.21.15.57:0/3588575384 conn(0x7f11d00fbda0 msgr2=0x7f11d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 -- 172.21.15.57:0/3588575384 shutdown_connections 2024-02-28T23:50:46.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.379+0000 7f11d6000700 1 -- 172.21.15.57:0/3588575384 wait complete. 2024-02-28T23:50:46.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.380+0000 7f11d6000700 1 Processor -- start 2024-02-28T23:50:46.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.380+0000 7f11d6000700 1 -- start start 2024-02-28T23:50:46.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11d6000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:46.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11d6000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 0x7f11d0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:46.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11d6000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0198d10 con 0x7f11d01004e0 2024-02-28T23:50:46.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11d6000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d019caa0 con 0x7f11d0100e80 2024-02-28T23:50:46.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:46.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11cf400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52324/0 (socket says 172.21.15.57:52324) 2024-02-28T23:50:46.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.381+0000 7f11cf400700 1 -- 172.21.15.57:0/437318971 learned_addr learned my addr 172.21.15.57:0/437318971 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11cf400700 1 -- 172.21.15.57:0/437318971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 msgr2=0x7f11d0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11cea00700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 0x7f11d0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11cf400700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 0x7f11d0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11cf400700 1 -- 172.21.15.57:0/437318971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00095f0 con 0x7f11d01004e0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11cf400700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b5c0 tx=0x7f11c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11c001d070 con 0x7f11d01004e0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11c0005e00 con 0x7f11d01004e0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11c0017400 con 0x7f11d01004e0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d019cd20 con 0x7f11d01004e0 2024-02-28T23:50:46.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.382+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d019d210 con 0x7f11d01004e0 2024-02-28T23:50:46.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.384+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d01045d0 con 0x7f11d01004e0 2024-02-28T23:50:46.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.385+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f11c000f460 con 0x7f11d01004e0 2024-02-28T23:50:46.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.386+0000 7f11c7e00700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 0x7f11b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:46.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.386+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11c008ea10 con 0x7f11d01004e0 2024-02-28T23:50:46.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.386+0000 7f11cea00700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 0x7f11b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:46.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.387+0000 7f11cea00700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 0x7f11b406fd50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005950 tx=0x7f11bc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:46.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.391+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11c00581a0 con 0x7f11d01004e0 2024-02-28T23:50:46.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.642+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11d0199580 con 0x7f11b406d8a0 2024-02-28T23:50:46.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.644+0000 7f11c7e00700 1 -- 172.21.15.57:0/437318971 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f11d0199580 con 0x7f11b406d8a0 2024-02-28T23:50:46.643 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "target_image": null, 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "in_progress": false, 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "which": "", 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "services_complete": [], 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "progress": null, 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "message": "", 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout: "is_paused": false 2024-02-28T23:50:46.644 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:50:46.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 msgr2=0x7f11b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:46.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 0x7f11b406fd50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005950 tx=0x7f11bc0058e0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 msgr2=0x7f11d01980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:46.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b5c0 tx=0x7f11c000be20 comp rx=0 tx=0).stop 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 shutdown_connections 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f11b406d8a0 0x7f11b406fd50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11d01004e0 0x7f11d01980f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 --2- 172.21.15.57:0/437318971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11d0100e80 0x7f11d0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 >> 172.21.15.57:0/437318971 conn(0x7f11d00fbda0 msgr2=0x7f11d00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 shutdown_connections 2024-02-28T23:50:46.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:46.647+0000 7f11d6000700 1 -- 172.21.15.57:0/437318971 wait complete. 2024-02-28T23:50:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:47 smithi057 ceph-mon[89491]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:47.426 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph health detail' 2024-02-28T23:50:47.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:47 smithi084 ceph-mon[70440]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:47.827 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:48 smithi057 ceph-mon[89491]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:48.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:48 smithi084 ceph-mon[70440]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:50:48.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:50:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:49 smithi084 ceph-mon[70440]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:49.487 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:49 smithi057 ceph-mon[89491]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 -- 172.21.15.57:0/1877601243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 msgr2=0x7f9bd8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 --2- 172.21.15.57:0/1877601243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8104eb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009910 tx=0x7f9bc0009c20 comp rx=0 tx=0).stop 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 -- 172.21.15.57:0/1877601243 shutdown_connections 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 --2- 172.21.15.57:0/1877601243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8104eb0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 --2- 172.21.15.57:0/1877601243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 0x7f9bd8068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 -- 172.21.15.57:0/1877601243 >> 172.21.15.57:0/1877601243 conn(0x7f9bd8075130 msgr2=0x7f9bd8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 -- 172.21.15.57:0/1877601243 shutdown_connections 2024-02-28T23:50:49.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.975+0000 7f9bde600700 1 -- 172.21.15.57:0/1877601243 wait complete. 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.976+0000 7f9bde600700 1 Processor -- start 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.976+0000 7f9bde600700 1 -- start start 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bde600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 0x7f9bd8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bde600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bde600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd8198c70 con 0x7f9bd8069140 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bde600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd819ca00 con 0x7f9bd80687a0 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bcea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bcea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52344/0 (socket says 172.21.15.57:52344) 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bcea00700 1 -- 172.21.15.57:0/2425004166 learned_addr learned my addr 172.21.15.57:0/2425004166 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bd7400700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 0x7f9bd8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bcea00700 1 -- 172.21.15.57:0/2425004166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 msgr2=0x7f9bd8198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:49.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.977+0000 7f9bcea00700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 0x7f9bd8198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:49.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bcea00700 1 -- 172.21.15.57:0/2425004166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00095f0 con 0x7f9bd8069140 2024-02-28T23:50:49.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bcea00700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bda0 tx=0x7f9bc000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:49.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc001d070 con 0x7f9bd8069140 2024-02-28T23:50:49.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc000f460 con 0x7f9bd8069140 2024-02-28T23:50:49.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd819cc80 con 0x7f9bd8069140 2024-02-28T23:50:49.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc0017610 con 0x7f9bd8069140 2024-02-28T23:50:49.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.978+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd819d170 con 0x7f9bd8069140 2024-02-28T23:50:49.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.980+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd804f2a0 con 0x7f9bd8069140 2024-02-28T23:50:49.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.981+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9bc000fa40 con 0x7f9bd8069140 2024-02-28T23:50:49.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.982+0000 7f9bd4c00700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 0x7f9bbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:49.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.982+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9bc008e900 con 0x7f9bd8069140 2024-02-28T23:50:49.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.982+0000 7f9bd7400700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 0x7f9bbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:49.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.983+0000 7f9bd7400700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 0x7f9bbc06fd50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d10 tx=0x7f9bc8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:49.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:49.987+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bc0058130 con 0x7f9bd8069140 2024-02-28T23:50:50.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.311+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9bd81993b0 con 0x7f9bd8069140 2024-02-28T23:50:50.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.312+0000 7f9bd4c00700 1 -- 172.21.15.57:0/2425004166 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9bc005b7f0 con 0x7f9bd8069140 2024-02-28T23:50:50.312 INFO:teuthology.orchestra.run.smithi057.stdout:HEALTH_OK 2024-02-28T23:50:50.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.315+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 msgr2=0x7f9bbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:50.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.315+0000 7f9bde600700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 0x7f9bbc06fd50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d10 tx=0x7f9bc8009040 comp rx=0 tx=0).stop 2024-02-28T23:50:50.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.315+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 msgr2=0x7f9bd8198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:50.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.315+0000 7f9bde600700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bda0 tx=0x7f9bc000be80 comp rx=0 tx=0).stop 2024-02-28T23:50:50.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 shutdown_connections 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9bbc06d8a0 0x7f9bbc06fd50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bd80687a0 0x7f9bd8198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 --2- 172.21.15.57:0/2425004166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bd8069140 0x7f9bd8198590 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 >> 172.21.15.57:0/2425004166 conn(0x7f9bd8075130 msgr2=0x7f9bd8101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 shutdown_connections 2024-02-28T23:50:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:50.316+0000 7f9bde600700 1 -- 172.21.15.57:0/2425004166 wait complete. 2024-02-28T23:50:50.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:50 smithi057 ceph-mon[89491]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:50.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:50 smithi084 ceph-mon[70440]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:50:51.126 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions' 2024-02-28T23:50:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:51 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2425004166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:50:51.575 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:51.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:51 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2425004166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-28T23:50:52.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:52 smithi057 ceph-mon[89491]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:52.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:52 smithi084 ceph-mon[70440]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:50:53.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 -- 172.21.15.57:0/387376093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 msgr2=0x7f563c10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 --2- 172.21.15.57:0/387376093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c10c3c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5628009940 tx=0x7f5628009c50 comp rx=0 tx=0).stop 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 -- 172.21.15.57:0/387376093 shutdown_connections 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 --2- 172.21.15.57:0/387376093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c10c3c0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 --2- 172.21.15.57:0/387376093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 0x7f563c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 -- 172.21.15.57:0/387376093 >> 172.21.15.57:0/387376093 conn(0x7f563c0fbb60 msgr2=0x7f563c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.645+0000 7f5641800700 1 -- 172.21.15.57:0/387376093 shutdown_connections 2024-02-28T23:50:53.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.646+0000 7f5641800700 1 -- 172.21.15.57:0/387376093 wait complete. 2024-02-28T23:50:53.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 Processor -- start 2024-02-28T23:50:53.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 -- start start 2024-02-28T23:50:53.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 0x7f563c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:53.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c198b20 con 0x7f563c0739a0 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f5641800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19c8b0 con 0x7f563c073090 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.647+0000 7f563b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52352/0 (socket says 172.21.15.57:52352) 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 -- 172.21.15.57:0/60744205 learned_addr learned my addr 172.21.15.57:0/60744205 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563be00700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 0x7f563c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 -- 172.21.15.57:0/60744205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 msgr2=0x7f563c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:53.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 0x7f563c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 -- 172.21.15.57:0/60744205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56280095f0 con 0x7f563c0739a0 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.648+0000 7f563b400700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f562800b870 tx=0x7f56280056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.649+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f562801d070 con 0x7f563c0739a0 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.649+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f562800f460 con 0x7f563c0739a0 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.649+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c19cb30 con 0x7f563c0739a0 2024-02-28T23:50:53.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.649+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5628021640 con 0x7f563c0739a0 2024-02-28T23:50:53.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.649+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c19d020 con 0x7f563c0739a0 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.651+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c109ac0 con 0x7f563c0739a0 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.651+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f562800fa60 con 0x7f563c0739a0 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.652+0000 7f5638c00700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 0x7f562006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.652+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f562808e9a0 con 0x7f563c0739a0 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.653+0000 7f563be00700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 0x7f562006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:50:53.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.653+0000 7f563be00700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 0x7f562006fd50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f562c006fd0 tx=0x7f562c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:50:53.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.657+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56280580b0 con 0x7f563c0739a0 2024-02-28T23:50:53.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.981+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f563c062380 con 0x7f563c0739a0 2024-02-28T23:50:53.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.983+0000 7f5638c00700 1 -- 172.21.15.57:0/60744205 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f562805b770 con 0x7f563c0739a0 2024-02-28T23:50:53.983 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:50:53.983 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:50:53.983 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:50:53.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:50:53.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:50:53.985 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:50:53.985 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:50:53.985 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:50:53.985 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 msgr2=0x7f562006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 0x7f562006fd50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f562c006fd0 tx=0x7f562c008040 comp rx=0 tx=0).stop 2024-02-28T23:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 msgr2=0x7f563c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f562800b870 tx=0x7f56280056a0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 shutdown_connections 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f562006d8a0 0x7f562006fd50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c073090 0x7f563c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 --2- 172.21.15.57:0/60744205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f563c0739a0 0x7f563c198440 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 >> 172.21.15.57:0/60744205 conn(0x7f563c0fbb60 msgr2=0x7f563c106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 shutdown_connections 2024-02-28T23:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:50:53.986+0000 7f5641800700 1 -- 172.21.15.57:0/60744205 wait complete. 2024-02-28T23:50:54.742 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-28T23:50:55.166 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:55.316 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:54 smithi057 ceph-mon[89491]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:55.316 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:54 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/60744205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:54 smithi084 ceph-mon[70440]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:50:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:54 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/60744205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:50:57.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:56 smithi057 ceph-mon[89491]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:57.183 INFO:teuthology.orchestra.run.smithi057.stdout:wait for servicemap items w/ changing names to refresh 2024-02-28T23:50:57.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:56 smithi084 ceph-mon[70440]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:50:57.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'sleep 60' 2024-02-28T23:50:58.296 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:50:59.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:50:58 smithi057 ceph-mon[89491]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:50:59.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:50:58 smithi084 ceph-mon[70440]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:01.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:00 smithi057 ceph-mon[89491]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:01.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:00 smithi084 ceph-mon[70440]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:02 smithi057 ceph-mon[89491]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:02 smithi084 ceph-mon[70440]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:04.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:05.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:04 smithi057 ceph-mon[89491]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:05.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:04 smithi084 ceph-mon[70440]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:07 smithi057 ceph-mon[89491]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:07.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:07 smithi084 ceph-mon[70440]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:09 smithi057 ceph-mon[89491]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:09.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:09 smithi084 ceph-mon[70440]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:11.170 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:11 smithi057 ceph-mon[89491]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:11.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:11 smithi084 ceph-mon[70440]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:13.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:13 smithi057 ceph-mon[89491]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:13 smithi084 ceph-mon[70440]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:15 smithi057 ceph-mon[89491]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:15 smithi084 ceph-mon[70440]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:17 smithi057 ceph-mon[89491]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:17 smithi084 ceph-mon[70440]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:19 smithi057 ceph-mon[89491]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:19 smithi084 ceph-mon[70440]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:21 smithi057 ceph-mon[89491]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:21.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:21 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:51:21.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:21 smithi084 ceph-mon[70440]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:21.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:21 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:51:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:51:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:51:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:51:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:51:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:51:22.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:51:23.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:23 smithi084 ceph-mon[70440]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:23 smithi057 ceph-mon[89491]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:25 smithi057 ceph-mon[89491]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:25 smithi084 ceph-mon[70440]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:27.384 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:27 smithi084 ceph-mon[70440]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:27 smithi057 ceph-mon[89491]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:29 smithi057 ceph-mon[89491]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:29 smithi084 ceph-mon[70440]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:31 smithi057 ceph-mon[89491]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:31 smithi084 ceph-mon[70440]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:33.475 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:33 smithi057 ceph-mon[89491]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:33 smithi084 ceph-mon[70440]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:34.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:35.518 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:35 smithi084 ceph-mon[70440]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:35 smithi057 ceph-mon[89491]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:37 smithi057 ceph-mon[89491]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:51:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:37 smithi084 ceph-mon[70440]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:51:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:39 smithi057 ceph-mon[89491]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:51:39.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:39 smithi084 ceph-mon[70440]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:51:41.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:41 smithi057 ceph-mon[89491]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:51:41.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:41 smithi084 ceph-mon[70440]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:51:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:43 smithi057 ceph-mon[89491]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:51:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:43 smithi084 ceph-mon[70440]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:51:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:45 smithi057 ceph-mon[89491]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:51:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:45 smithi084 ceph-mon[70440]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:51:47.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:47 smithi057 ceph-mon[89491]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:51:47.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:47 smithi084 ceph-mon[70440]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:51:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:48.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:51:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:49 smithi057 ceph-mon[89491]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:49.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:49 smithi084 ceph-mon[70440]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:51 smithi084 ceph-mon[70440]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:51 smithi057 ceph-mon[89491]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:51:53.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:53 smithi057 ceph-mon[89491]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:53.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:53 smithi084 ceph-mon[70440]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:51:55.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:55 smithi057 ceph-mon[89491]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:55.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:55 smithi084 ceph-mon[70440]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:51:57.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:57 smithi057 ceph-mon[89491]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:57.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:57 smithi084 ceph-mon[70440]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:51:59.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:51:59 smithi057 ceph-mon[89491]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:51:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:51:59 smithi084 ceph-mon[70440]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:00.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:00 smithi057 ceph-mon[89491]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:00.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:00 smithi084 ceph-mon[70440]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:01.181 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph orch ps' 2024-02-28T23:52:01.630 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:52:03.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:02 smithi057 ceph-mon[89491]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:03.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:02 smithi084 ceph-mon[70440]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:04.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 -- 172.21.15.57:0/202465424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 msgr2=0x7efeb010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 --2- 172.21.15.57:0/202465424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb010ccc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efea0009910 tx=0x7efea0009c20 comp rx=0 tx=0).stop 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 -- 172.21.15.57:0/202465424 shutdown_connections 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 --2- 172.21.15.57:0/202465424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 0x7efeb0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 --2- 172.21.15.57:0/202465424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb010ccc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.050+0000 7efeb6800700 1 -- 172.21.15.57:0/202465424 >> 172.21.15.57:0/202465424 conn(0x7efeb0076060 msgr2=0x7efeb0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.051+0000 7efeb6800700 1 -- 172.21.15.57:0/202465424 shutdown_connections 2024-02-28T23:52:04.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.051+0000 7efeb6800700 1 -- 172.21.15.57:0/202465424 wait complete. 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 Processor -- start 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 -- start start 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 0x7efeb019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019d120 con 0x7efeb010c8f0 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeb6800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01a0eb0 con 0x7efeb0106b80 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.052+0000 7efeaea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49066/0 (socket says 172.21.15.57:49066) 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 -- 172.21.15.57:0/909810169 learned_addr learned my addr 172.21.15.57:0/909810169 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaf400700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 0x7efeb019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 -- 172.21.15.57:0/909810169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 msgr2=0x7efeb019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:04.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 0x7efeb019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 -- 172.21.15.57:0/909810169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efeb010c8f0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.053+0000 7efeaea00700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d8d0 tx=0x7efe9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.054+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c009940 con 0x7efeb010c8f0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.054+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe9c010460 con 0x7efeb010c8f0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.054+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c00f5f0 con 0x7efeb010c8f0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.054+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01a1190 con 0x7efeb010c8f0 2024-02-28T23:52:04.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.054+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01a16e0 con 0x7efeb010c8f0 2024-02-28T23:52:04.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.055+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0109b20 con 0x7efeb010c8f0 2024-02-28T23:52:04.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.056+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efe9c0105d0 con 0x7efeb010c8f0 2024-02-28T23:52:04.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.057+0000 7efea7e00700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 0x7efe9406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:04.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.057+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efe9c08d5d0 con 0x7efeb010c8f0 2024-02-28T23:52:04.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.057+0000 7efeaf400700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 0x7efe9406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:04.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.058+0000 7efeaf400700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 0x7efe9406fd50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efea000b5c0 tx=0x7efea0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:04.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.062+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe9c056ce0 con 0x7efeb010c8f0 2024-02-28T23:52:04.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.307+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 --> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efeb01a1970 con 0x7efe9406d8a0 2024-02-28T23:52:04.308 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:04.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:04.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.341+0000 7efea7e00700 1 -- 172.21.15.57:0/909810169 <== mgr.34104 v2:172.21.15.57:6800/667025161 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efeb01a1970 con 0x7efe9406d8a0 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:alertmanager.smithi057 smithi057 *:9093,9094 running (41m) 114s ago 55m 14.5M - 0.25.0 c8568f914cd2 369ec9c390dc 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi057 smithi057 running (2m) 114s ago 55m 8006k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 814d9055bf9c 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-exporter.smithi084 smithi084 running (2m) 117s ago 54m 7113k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 4742e02d5b02 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi057 smithi057 running (37m) 114s ago 55m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 9dbf6cde32ee 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:crash.smithi084 smithi084 running (37m) 117s ago 54m 6665k - 19.0.0-1670-g1fb5d106 2272b2c42f9b 766d79d0a893 2024-02-28T23:52:04.342 INFO:teuthology.orchestra.run.smithi057.stdout:grafana.smithi057 smithi057 *:3000 running (41m) 114s ago 55m 77.0M - 9.4.12 a0ea5e3c46d9 0c6d2827d14f 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.bgmggd smithi057 running (33m) 114s ago 48m 61.1M - 19.0.0-1670-g1fb5d106 2272b2c42f9b a47da1d547e2 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi057.sbkaml smithi057 running (33m) 114s ago 48m 60.4M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 6a230630128b 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.dadqfu smithi084 running (17m) 117s ago 48m 59.6M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 868eb4da881d 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mds.cephfs.smithi084.ztfxpc smithi084 running (32m) 117s ago 48m 60.6M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 3a298a5ad0c1 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi057.zrrldj smithi057 *:8443,9283,8765 running (44m) 114s ago 57m 515M - 19.0.0-1670-g1fb5d106 2272b2c42f9b 2ea803a52ad9 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.smithi084.fhkzis smithi084 *:8443,9283,8765 running (43m) 117s ago 54m 459M - 19.0.0-1670-g1fb5d106 2272b2c42f9b b520bce4dc59 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi057 smithi057 running (39m) 114s ago 57m 86.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b f08f19bb5712 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:mon.smithi084 smithi084 running (38m) 117s ago 54m 60.8M 2048M 19.0.0-1670-g1fb5d106 2272b2c42f9b b363cce31fa1 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi057 smithi057 *:9100 running (42m) 114s ago 55m 8480k - 1.5.0 0da6a335fe13 7fc69f895861 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:node-exporter.smithi084 smithi084 *:9100 running (42m) 117s ago 54m 8430k - 1.5.0 0da6a335fe13 1329e9a11218 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (37m) 114s ago 53m 102M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b a6b078e69c02 2024-02-28T23:52:04.343 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi057 running (36m) 114s ago 52m 116M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 3fc1bae027bf 2024-02-28T23:52:04.344 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi057 running (36m) 114s ago 51m 94.2M 1218M 19.0.0-1670-g1fb5d106 2272b2c42f9b 642db0860d86 2024-02-28T23:52:04.344 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi084 running (35m) 117s ago 51m 123M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b c438b3ff6626 2024-02-28T23:52:04.344 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi084 running (34m) 117s ago 50m 111M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b f716546fb90a 2024-02-28T23:52:04.344 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi084 running (34m) 117s ago 50m 98.9M 2241M 19.0.0-1670-g1fb5d106 2272b2c42f9b bcfd5c4a9e36 2024-02-28T23:52:04.344 INFO:teuthology.orchestra.run.smithi057.stdout:prometheus.smithi057 smithi057 *:9095 running (42m) 114s ago 54m 83.4M - 2.43.0 a07b618ecd1d 068edb9cd4c8 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 msgr2=0x7efe9406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 0x7efe9406fd50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efea000b5c0 tx=0x7efea0005960 comp rx=0 tx=0).stop 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 msgr2=0x7efeb019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d8d0 tx=0x7efe9c00dc90 comp rx=0 tx=0).stop 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 shutdown_connections 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efe9406d8a0 0x7efe9406fd50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efeb0106b80 0x7efeb019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 --2- 172.21.15.57:0/909810169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efeb010c8f0 0x7efeb019ca40 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:04.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.344+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 >> 172.21.15.57:0/909810169 conn(0x7efeb0076060 msgr2=0x7efeb0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:04.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.345+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 shutdown_connections 2024-02-28T23:52:04.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:04.345+0000 7efeb6800700 1 -- 172.21.15.57:0/909810169 wait complete. 2024-02-28T23:52:05.069 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:05 smithi057 ceph-mon[89491]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:05.158 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions' 2024-02-28T23:52:05.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:05 smithi084 ceph-mon[70440]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:52:06.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:06 smithi057 ceph-mon[89491]: from='client.36098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:52:06.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:06 smithi084 ceph-mon[70440]: from='client.36098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-28T23:52:07.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:07 smithi057 ceph-mon[89491]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:07.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:07 smithi084 ceph-mon[70440]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 -- 172.21.15.57:0/2567752644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 msgr2=0x7f599c10c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 --2- 172.21.15.57:0/2567752644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c10c5e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5988009940 tx=0x7f5988009c50 comp rx=0 tx=0).stop 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 -- 172.21.15.57:0/2567752644 shutdown_connections 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 --2- 172.21.15.57:0/2567752644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c10c5e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 --2- 172.21.15.57:0/2567752644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 -- 172.21.15.57:0/2567752644 >> 172.21.15.57:0/2567752644 conn(0x7f599c0fbda0 msgr2=0x7f599c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 -- 172.21.15.57:0/2567752644 shutdown_connections 2024-02-28T23:52:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.904+0000 7f59a1200700 1 -- 172.21.15.57:0/2567752644 wait complete. 2024-02-28T23:52:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.905+0000 7f59a1200700 1 Processor -- start 2024-02-28T23:52:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f59a1200700 1 -- start start 2024-02-28T23:52:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f59a1200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f59a1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f59a1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c198dd0 con 0x7f599c0739a0 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f59a1200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c19cb60 con 0x7f599c073090 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f5999400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f5999400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40338/0 (socket says 172.21.15.57:40338) 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.906+0000 7f5999400700 1 -- 172.21.15.57:0/1331337832 learned_addr learned my addr 172.21.15.57:0/1331337832 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999e00700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999400700 1 -- 172.21.15.57:0/1331337832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 msgr2=0x7f599c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999400700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999400700 1 -- 172.21.15.57:0/1331337832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880095f0 con 0x7f599c0739a0 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999e00700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:07.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.907+0000 7f5999400700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f598800b580 tx=0x7f59880056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.908+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598801d070 con 0x7f599c0739a0 2024-02-28T23:52:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.908+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f598800f460 con 0x7f599c0739a0 2024-02-28T23:52:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.908+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c19cde0 con 0x7f599c0739a0 2024-02-28T23:52:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.908+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5988021640 con 0x7f599c0739a0 2024-02-28T23:52:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.908+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c19d2d0 con 0x7f599c0739a0 2024-02-28T23:52:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.909+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c109ce0 con 0x7f599c0739a0 2024-02-28T23:52:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.910+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f598800fab0 con 0x7f599c0739a0 2024-02-28T23:52:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.911+0000 7f5992a00700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 0x7f598006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.911+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f598808e7e0 con 0x7f599c0739a0 2024-02-28T23:52:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.911+0000 7f5999e00700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 0x7f598006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.912+0000 7f5999e00700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 0x7f598006fd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f598c005fd0 tx=0x7f598c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:07.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:07.916+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5988057ef0 con 0x7f599c0739a0 2024-02-28T23:52:08.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.236+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f599c04f2a0 con 0x7f599c0739a0 2024-02-28T23:52:08.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.237+0000 7f5992a00700 1 -- 172.21.15.57:0/1331337832 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f598805b5b0 con 0x7f599c0739a0 2024-02-28T23:52:08.238 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-02-28T23:52:08.238 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": { 2024-02-28T23:52:08.238 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:52:08.238 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": { 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 2 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": { 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 6 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 4 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "overall": { 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)": 14 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-02-28T23:52:08.239 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 msgr2=0x7f598006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 0x7f598006fd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f598c005fd0 tx=0x7f598c005ee0 comp rx=0 tx=0).stop 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 msgr2=0x7f599c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f598800b580 tx=0x7f59880056a0 comp rx=0 tx=0).stop 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 shutdown_connections 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f598006d8a0 0x7f598006fd50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f599c073090 0x7f599c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 --2- 172.21.15.57:0/1331337832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f599c0739a0 0x7f599c1986f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:08.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.240+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 >> 172.21.15.57:0/1331337832 conn(0x7f599c0fbda0 msgr2=0x7f599c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:08.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.241+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 shutdown_connections 2024-02-28T23:52:08.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:08.241+0000 7f59a1200700 1 -- 172.21.15.57:0/1331337832 wait complete. 2024-02-28T23:52:08.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-28T23:52:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:09 smithi057 ceph-mon[89491]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:09.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:09 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1331337832' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:09.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:09 smithi084 ceph-mon[70440]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:09.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:09 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1331337832' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:09.430 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:52:11.260 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:11 smithi057 ceph-mon[89491]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:11.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:11 smithi084 ceph-mon[70440]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.439+0000 7f6b08600700 1 -- 172.21.15.57:0/3434211795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000730b0 msgr2=0x7f6b00073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.439+0000 7f6b08600700 1 --2- 172.21.15.57:0/3434211795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000730b0 0x7f6b00073480 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009940 tx=0x7f6af4009c50 comp rx=0 tx=0).stop 2024-02-28T23:52:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 -- 172.21.15.57:0/3434211795 shutdown_connections 2024-02-28T23:52:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 --2- 172.21.15.57:0/3434211795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000739c0 0x7f6b0010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 --2- 172.21.15.57:0/3434211795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000730b0 0x7f6b00073480 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 -- 172.21.15.57:0/3434211795 >> 172.21.15.57:0/3434211795 conn(0x7f6b000fbd40 msgr2=0x7f6b000fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 -- 172.21.15.57:0/3434211795 shutdown_connections 2024-02-28T23:52:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.440+0000 7f6b08600700 1 -- 172.21.15.57:0/3434211795 wait complete. 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.441+0000 7f6b08600700 1 Processor -- start 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.441+0000 7f6b08600700 1 -- start start 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.441+0000 7f6b08600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 0x7f6b00198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.441+0000 7f6b08600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b08600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00198c70 con 0x7f6b000739c0 2024-02-28T23:52:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b08600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0019ca00 con 0x7f6b000730b0 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40362/0 (socket says 172.21.15.57:40362) 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 -- 172.21.15.57:0/1722530623 learned_addr learned my addr 172.21.15.57:0/1722530623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05c00700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 0x7f6b00198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 -- 172.21.15.57:0/1722530623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 msgr2=0x7f6b00198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:11.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 0x7f6b00198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05200700 1 -- 172.21.15.57:0/1722530623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40095f0 con 0x7f6b000739c0 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.442+0000 7f6b05c00700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 0x7f6b00198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6b05200700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00d8d0 tx=0x7f6aec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aec009940 con 0x7f6b000739c0 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aec010460 con 0x7f6b000739c0 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0019cce0 con 0x7f6b000739c0 2024-02-28T23:52:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aec00f5f0 con 0x7f6b000739c0 2024-02-28T23:52:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.443+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0019d230 con 0x7f6b000739c0 2024-02-28T23:52:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.444+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b00109c80 con 0x7f6b000739c0 2024-02-28T23:52:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.445+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6aec0105d0 con 0x7f6b000739c0 2024-02-28T23:52:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.446+0000 7f6afe000700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 0x7f6ae806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.446+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6aec08d510 con 0x7f6b000739c0 2024-02-28T23:52:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.447+0000 7f6b05c00700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 0x7f6ae806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.447+0000 7f6b05c00700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 0x7f6ae806fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af4005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.451+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6aec056ca0 con 0x7f6b000739c0 2024-02-28T23:52:11.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.777+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b0019d500 con 0x7f6b000739c0 2024-02-28T23:52:11.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.778+0000 7f6afe000700 1 -- 172.21.15.57:0/1722530623 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6aec05a360 con 0x7f6b000739c0 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 msgr2=0x7f6ae806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 0x7f6ae806fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af4005ad0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 msgr2=0x7f6b00198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00d8d0 tx=0x7f6aec00dc90 comp rx=0 tx=0).stop 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 shutdown_connections 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f6ae806d8a0 0x7f6ae806fd50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b000730b0 0x7f6b00198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 --2- 172.21.15.57:0/1722530623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b000739c0 0x7f6b00198590 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:11.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.781+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 >> 172.21.15.57:0/1722530623 conn(0x7f6b000fbd40 msgr2=0x7f6b00106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.782+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 shutdown_connections 2024-02-28T23:52:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:11.782+0000 7f6b08600700 1 -- 172.21.15.57:0/1722530623 wait complete. 2024-02-28T23:52:11.798 INFO:teuthology.orchestra.run.smithi057.stdout:true 2024-02-28T23:52:12.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:12 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1722530623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:12.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:12 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1722530623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:12.507 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -e sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-28T23:52:12.929 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:52:13.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:13 smithi057 ceph-mon[89491]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:13.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:13 smithi084 ceph-mon[70440]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:15.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 -- 172.21.15.57:0/766844124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 msgr2=0x7f4b8810c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:15.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 --2- 172.21.15.57:0/766844124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8810c330 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009910 tx=0x7f4b74009c20 comp rx=0 tx=0).stop 2024-02-28T23:52:15.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 -- 172.21.15.57:0/766844124 shutdown_connections 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 --2- 172.21.15.57:0/766844124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8810c330 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 --2- 172.21.15.57:0/766844124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 0x7f4b880734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 -- 172.21.15.57:0/766844124 >> 172.21.15.57:0/766844124 conn(0x7f4b880fbb10 msgr2=0x7f4b880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 -- 172.21.15.57:0/766844124 shutdown_connections 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.045+0000 7f4b8f000700 1 -- 172.21.15.57:0/766844124 wait complete. 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.047+0000 7f4b8f000700 1 Processor -- start 2024-02-28T23:52:15.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.047+0000 7f4b8f000700 1 -- start start 2024-02-28T23:52:15.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.048+0000 7f4b8f000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 0x7f4b8819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:15.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.048+0000 7f4b8f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:15.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.048+0000 7f4b8f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88197b40 con 0x7f4b88073a00 2024-02-28T23:52:15.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.048+0000 7f4b8f000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88197cb0 con 0x7f4b880730f0 2024-02-28T23:52:15.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.048+0000 7f4b8d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:15.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57718/0 (socket says 172.21.15.57:57718) 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 -- 172.21.15.57:0/2170018516 learned_addr learned my addr 172.21.15.57:0/2170018516 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8dc00700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 0x7f4b8819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 -- 172.21.15.57:0/2170018516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 msgr2=0x7f4b8819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 0x7f4b8819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 -- 172.21.15.57:0/2170018516 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740095f0 con 0x7f4b88073a00 2024-02-28T23:52:15.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.049+0000 7f4b8d200700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4b740092e0 tx=0x7f4b7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:15.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.051+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7401d070 con 0x7f4b88073a00 2024-02-28T23:52:15.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.051+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b74005e00 con 0x7f4b88073a00 2024-02-28T23:52:15.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.051+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b88197f90 con 0x7f4b88073a00 2024-02-28T23:52:15.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.051+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b74017400 con 0x7f4b88073a00 2024-02-28T23:52:15.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.051+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881984e0 con 0x7f4b88073a00 2024-02-28T23:52:15.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.052+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b88109a30 con 0x7f4b88073a00 2024-02-28T23:52:15.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.053+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4b7400f460 con 0x7f4b88073a00 2024-02-28T23:52:15.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.054+0000 7f4b86000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 0x7f4b7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:15.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.054+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4b7408e9f0 con 0x7f4b88073a00 2024-02-28T23:52:15.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.054+0000 7f4b8dc00700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 0x7f4b7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:15.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.055+0000 7f4b8dc00700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 0x7f4b7006fd50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005950 tx=0x7f4b7c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.059+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b74090e60 con 0x7f4b88073a00 2024-02-28T23:52:15.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:15 smithi057 ceph-mon[89491]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:15.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:15 smithi084 ceph-mon[70440]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:15.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.378+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b88062380 con 0x7f4b88073a00 2024-02-28T23:52:15.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.379+0000 7f4b86000700 1 -- 172.21.15.57:0/2170018516 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4b7405bcd0 con 0x7f4b88073a00 2024-02-28T23:52:15.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 msgr2=0x7f4b7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:15.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 0x7f4b7006fd50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005950 tx=0x7f4b7c0058e0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 msgr2=0x7f4b8819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4b740092e0 tx=0x7f4b7400be20 comp rx=0 tx=0).stop 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 shutdown_connections 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4b7006d8a0 0x7f4b7006fd50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b880730f0 0x7f4b8819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 --2- 172.21.15.57:0/2170018516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b88073a00 0x7f4b8819e130 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 >> 172.21.15.57:0/2170018516 conn(0x7f4b880fbb10 msgr2=0x7f4b88106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 shutdown_connections 2024-02-28T23:52:15.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:15.382+0000 7f4b8f000700 1 -- 172.21.15.57:0/2170018516 wait complete. 2024-02-28T23:52:15.398 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph version 19.0.0-1670-g1fb5d106 (1fb5d10608dccee73edd6ef8a2c7b804a097c324) squid (dev)" 2024-02-28T23:52:16.044 DEBUG:teuthology.parallel:result is None 2024-02-28T23:52:16.044 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-28T23:52:16.060 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi057.front.sepia.ceph.com 2024-02-28T23:52:16.061 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-28T23:52:16.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:16 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2170018516' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:16.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:16 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2170018516' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-28T23:52:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/mon.smithi057/config 2024-02-28T23:52:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:17 smithi057 ceph-mon[89491]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:17 smithi084 ceph-mon[70440]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.653+0000 7f2d10c00700 1 -- 172.21.15.57:0/3313555290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 msgr2=0x7f2d0c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.653+0000 7f2d10c00700 1 --2- 172.21.15.57:0/3313555290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c106ff0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009910 tx=0x7f2cf4009c20 comp rx=0 tx=0).stop 2024-02-28T23:52:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 -- 172.21.15.57:0/3313555290 shutdown_connections 2024-02-28T23:52:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 --2- 172.21.15.57:0/3313555290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c106ff0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 --2- 172.21.15.57:0/3313555290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 0x7f2d0c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 -- 172.21.15.57:0/3313555290 >> 172.21.15.57:0/3313555290 conn(0x7f2d0c075e90 msgr2=0x7f2d0c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 -- 172.21.15.57:0/3313555290 shutdown_connections 2024-02-28T23:52:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.654+0000 7f2d10c00700 1 -- 172.21.15.57:0/3313555290 wait complete. 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 Processor -- start 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 -- start start 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 0x7f2d0c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19cf60 con 0x7f2d0c106b80 2024-02-28T23:52:18.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.655+0000 7f2d10c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a0cf0 con 0x7f2d0c10c660 2024-02-28T23:52:18.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:18.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57744/0 (socket says 172.21.15.57:57744) 2024-02-28T23:52:18.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 -- 172.21.15.57:0/2051820867 learned_addr learned my addr 172.21.15.57:0/2051820867 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:18.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d03e00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 0x7f2d0c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 -- 172.21.15.57:0/2051820867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 msgr2=0x7f2d0c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 0x7f2d0c19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.656+0000 7f2d0b400700 1 -- 172.21.15.57:0/2051820867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40095f0 con 0x7f2d0c106b80 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.657+0000 7f2d0b400700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b4b0 tx=0x7f2cfc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.657+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc011800 con 0x7f2d0c106b80 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.657+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cfc011e20 con 0x7f2d0c106b80 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.657+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc00f2e0 con 0x7f2d0c106b80 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.657+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1a0fd0 con 0x7f2d0c106b80 2024-02-28T23:52:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.658+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1a14d0 con 0x7f2d0c106b80 2024-02-28T23:52:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.659+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c10eb40 con 0x7f2d0c106b80 2024-02-28T23:52:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.660+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2cfc010380 con 0x7f2d0c106b80 2024-02-28T23:52:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.661+0000 7f2d08c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 0x7f2cf006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.661+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2cfc08d180 con 0x7f2d0c106b80 2024-02-28T23:52:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.661+0000 7f2d03e00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 0x7f2cf006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:18.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.662+0000 7f2d03e00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 0x7f2cf006fd50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c19d960 tx=0x7f2cf401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:18.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.666+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cfc056890 con 0x7f2d0c106b80 2024-02-28T23:52:18.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.951+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d0c1a17a0 con 0x7f2d0c106b80 2024-02-28T23:52:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.952+0000 7f2d08c00700 1 -- 172.21.15.57:0/2051820867 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+1890 (secure 0 0 0) 0x7f2cfc059f50 con 0x7f2d0c106b80 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout:e42 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:18.954 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 42 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-02-28T23:03:38.758886+0000 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-02-28T23:50:18.239166+0000 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 60 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-02-28T23:52:18.955 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 83 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 2 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=34440,1=34472} 2024-02-28T23:52:18.956 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 1 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.bgmggd{0:34440} state up:active seq 475 join_fscid=1 addr [v2:172.21.15.57:6826/2218205483,v1:172.21.15.57:6827/2218205483] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.ztfxpc{0:44339} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.84:6824/968233333,v1:172.21.15.84:6825/968233333] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:52:18.957 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi057.sbkaml{1:34472} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.57:6828/220807232,v1:172.21.15.57:6829/220807232] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:52:18.958 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.cephfs.smithi084.dadqfu{1:44797} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.84:6826/4290160570,v1:172.21.15.84:6827/4290160570] compat {c=[1],r=[1],i=[fff]}] 2024-02-28T23:52:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 msgr2=0x7f2cf006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 0x7f2cf006fd50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c19d960 tx=0x7f2cf401a040 comp rx=0 tx=0).stop 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 msgr2=0x7f2d0c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b4b0 tx=0x7f2cfc00b870 comp rx=0 tx=0).stop 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 shutdown_connections 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2cf006d8a0 0x7f2cf006fd50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d0c106b80 0x7f2d0c19c340 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 --2- 172.21.15.57:0/2051820867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d0c10c660 0x7f2d0c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 >> 172.21.15.57:0/2051820867 conn(0x7f2d0c075e90 msgr2=0x7f2d0c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 shutdown_connections 2024-02-28T23:52:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:18.957+0000 7f2d10c00700 1 -- 172.21.15.57:0/2051820867 wait complete. 2024-02-28T23:52:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 42 2024-02-28T23:52:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:19 smithi057 ceph-mon[89491]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:19.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:19.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:19 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2051820867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:52:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:19 smithi084 ceph-mon[70440]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:19 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2051820867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-28T23:52:19.722 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-28T23:52:19.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-28T23:52:19.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 -- 172.21.15.57:0/3590203235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c100a30 msgr2=0x7f584c100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 --2- 172.21.15.57:0/3590203235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c100a30 0x7f584c100e00 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f583c0098b0 tx=0x7f583c02bcd0 comp rx=0 tx=0).stop 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 -- 172.21.15.57:0/3590203235 shutdown_connections 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 --2- 172.21.15.57:0/3590203235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c101340 0x7f584c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 --2- 172.21.15.57:0/3590203235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c100a30 0x7f584c100e00 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 -- 172.21.15.57:0/3590203235 >> 172.21.15.57:0/3590203235 conn(0x7f584c0fa260 msgr2=0x7f584c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 -- 172.21.15.57:0/3590203235 shutdown_connections 2024-02-28T23:52:19.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.968+0000 7f5851800640 1 -- 172.21.15.57:0/3590203235 wait complete. 2024-02-28T23:52:19.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.969+0000 7f5851800640 1 Processor -- start 2024-02-28T23:52:19.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.969+0000 7f5851800640 1 -- start start 2024-02-28T23:52:19.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5851800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c100a30 0x7f584c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5851800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5851800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0ff100 con 0x7f584c101340 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5851800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0ff270 con 0x7f584c100a30 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5843e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5843e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57768/0 (socket says 172.21.15.57:57768) 2024-02-28T23:52:19.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.970+0000 7f5843e00640 1 -- 172.21.15.57:0/3471654169 learned_addr learned my addr 172.21.15.57:0/3471654169 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:19.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f584aa00640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c100a30 0x7f584c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843e00640 1 -- 172.21.15.57:0/3471654169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c100a30 msgr2=0x7f584c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843e00640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c100a30 0x7f584c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843e00640 1 -- 172.21.15.57:0/3471654169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c009560 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843e00640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f583400d6f0 tx=0x7f583400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5834009a30 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58340054e0 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c0ff4f0 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5834010460 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.971+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c1a1ac0 con 0x7f584c101340 2024-02-28T23:52:19.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.973+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c04e000 con 0x7f584c101340 2024-02-28T23:52:19.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.974+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5834009b90 con 0x7f584c101340 2024-02-28T23:52:19.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.974+0000 7f5843400640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 0x7f583006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:19.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.975+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f583408cf90 con 0x7f584c101340 2024-02-28T23:52:19.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.975+0000 7f584aa00640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 0x7f583006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:19.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.976+0000 7f584aa00640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 0x7f583006fcc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f583c006010 tx=0x7f583c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:19.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:19.980+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58340566a0 con 0x7f584c101340 2024-02-28T23:52:20.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.189+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f584c0ff4f0 con 0x7f584c101340 2024-02-28T23:52:20.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.194+0000 7f5843400640 1 -- 172.21.15.57:0/3471654169 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f584c0ff4f0 con 0x7f584c101340 2024-02-28T23:52:20.195 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:20.195 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":87,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:34:24.949532+0000","last_up_change":"2024-02-28T23:18:05.087684+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-28T23:03:37.677738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-28T23:03:38.275939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6803","nonce":1424360026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6805","nonce":1424360026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6809","nonce":1424360026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6807","nonce":1424360026}]},"public_addr":"172.21.15.57:6803/1424360026","cluster_addr":"172.21.15.57:6805/1424360026","heartbeat_back_addr":"172.21.15.57:6809/1424360026","heartbeat_front_addr":"172.21.15.57:6807/1424360026","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6811","nonce":3847484969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6813","nonce":3847484969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6817","nonce":3847484969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6815","nonce":3847484969}]},"public_addr":"172.21.15.57:6811/3847484969","cluster_addr":"172.21.15.57:6813/3847484969","heartbeat_back_addr":"172.21.15.57:6817/3847484969","heartbeat_front_addr":"172.21.15.57:6815/3847484969","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6819","nonce":3954650626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6821","nonce":3954650626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6825","nonce":3954650626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6823","nonce":3954650626}]},"public_addr":"172.21.15.57:6819/3954650626","cluster_addr":"172.21.15.57:6821/3954650626","heartbeat_back_addr":"172.21.15.57:6825/3954650626","heartbeat_front_addr":"172.21.15.57:6823/3954650626","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6801","nonce":2190333848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6803","nonce":2190333848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6807","nonce":2190333848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6805","nonce":2190333848}]},"public_addr":"172.21.15.84:6801/2190333848","cluster_addr":"172.21.15.84:6803/2190333848","heartbeat_back_addr":"172.21.15.84:6807/2190333848","heartbeat_front_addr":"172.21.15.84:6805/2190333848","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6809","nonce":890451486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6811","nonce":890451486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6815","nonce":890451486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6813","nonce":890451486}]},"public_addr":"172.21.15.84:6809/890451486","cluster_addr":"172.21.15.84:6811/890451486","heartbeat_back_addr":"172.21.15.84:6815/890451486","heartbeat_front_addr":"172.21.15.84:6813/890451486","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6817","nonce":2020554677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6819","nonce":2020554677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6823","nonce":2020554677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6821","nonce":2020554677}]},"public_addr":"172.21.15.84:6817/2020554677","cluster_addr":"172.21.15.84:6819/2020554677","heartbeat_back_addr":"172.21.15.84:6823/2020554677","heartbeat_front_addr":"172.21.15.84:6821/2020554677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6829/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.84:6824/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.84:0/3210112039":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6801/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.84:0/4232995690":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/2372541761":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/3130089202":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.84:6825/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.84:0/3528688133":"2024-02-29T23:08:14.355809+0000","172.21.15.84:6829/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.84:0/3178273795":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6826/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/4128072297":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.84:6828/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3690186012":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/4070283299":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/1810823735":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6827/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:6828/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.57:0/382831045":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/915841588":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/292614806":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/3532446948":"2024-02-29T23:12:47.945548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 msgr2=0x7f583006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 0x7f583006fcc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f583c006010 tx=0x7f583c000bc0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 msgr2=0x7f584c0febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f583400d6f0 tx=0x7f583400dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 shutdown_connections 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f583006d810 0x7f583006fcc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f584c101340 0x7f584c0febc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 --2- 172.21.15.57:0/3471654169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c100a30 0x7f584c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.197+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 >> 172.21.15.57:0/3471654169 conn(0x7f584c0fa260 msgr2=0x7f584c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:20.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.198+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 shutdown_connections 2024-02-28T23:52:20.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.198+0000 7f5851800640 1 -- 172.21.15.57:0/3471654169 wait complete. 2024-02-28T23:52:20.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-28T23:00:25.160671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-28T23:03:37.677738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-28T23:03:38.275939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-28T23:52:20.219 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-28T23:52:20.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 -- 172.21.15.57:0/3679972009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 msgr2=0x7f5728101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 --2- 172.21.15.57:0/3679972009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f5728101b90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5714009880 tx=0x7f571402bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 -- 172.21.15.57:0/3679972009 shutdown_connections 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 --2- 172.21.15.57:0/3679972009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f5728101b90 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 --2- 172.21.15.57:0/3679972009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 0x7f5728107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 -- 172.21.15.57:0/3679972009 >> 172.21.15.57:0/3679972009 conn(0x7f57280fd220 msgr2=0x7f57280ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 -- 172.21.15.57:0/3679972009 shutdown_connections 2024-02-28T23:52:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.434+0000 7f572dc00640 1 -- 172.21.15.57:0/3679972009 wait complete. 2024-02-28T23:52:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.435+0000 7f572dc00640 1 Processor -- start 2024-02-28T23:52:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.435+0000 7f572dc00640 1 -- start start 2024-02-28T23:52:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f572dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f572dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 0x7f5728197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f572dc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728197f90 con 0x7f5728101720 2024-02-28T23:52:20.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f572dc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572819bd60 con 0x7f5728107730 2024-02-28T23:52:20.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f5727e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f5727e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57794/0 (socket says 172.21.15.57:57794) 2024-02-28T23:52:20.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.436+0000 7f5727e00640 1 -- 172.21.15.57:0/991890162 learned_addr learned my addr 172.21.15.57:0/991890162 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5727400640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 0x7f5728197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5727e00640 1 -- 172.21.15.57:0/991890162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 msgr2=0x7f5728197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5727e00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 0x7f5728197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5727e00640 1 -- 172.21.15.57:0/991890162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5714009560 con 0x7f5728101720 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5727e00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f571800d8d0 tx=0x7f571800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5718009a30 con 0x7f5728101720 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5718009b90 con 0x7f5728101720 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572819c040 con 0x7f5728101720 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571800f680 con 0x7f5728101720 2024-02-28T23:52:20.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.437+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572819c620 con 0x7f5728101720 2024-02-28T23:52:20.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.439+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572804e000 con 0x7f5728101720 2024-02-28T23:52:20.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.440+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5718009d00 con 0x7f5728101720 2024-02-28T23:52:20.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.440+0000 7f5724c00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 0x7f570c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.441+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f571808d2b0 con 0x7f5728101720 2024-02-28T23:52:20.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.441+0000 7f5727400640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 0x7f570c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.442+0000 7f5727400640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 0x7f570c06fcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f571402c8b0 tx=0x7f5714005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:20.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.446+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5718056ae0 con 0x7f5728101720 2024-02-28T23:52:20.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.645+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f572819c950 con 0x7f5728101720 2024-02-28T23:52:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.647+0000 7f5724c00640 1 -- 172.21.15.57:0/991890162 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f571805a1a0 con 0x7f5728101720 2024-02-28T23:52:20.647 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 msgr2=0x7f570c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 0x7f570c06fcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f571402c8b0 tx=0x7f5714005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 msgr2=0x7f57281973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f571800d8d0 tx=0x7f571800dd90 comp rx=0 tx=0).stop 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 shutdown_connections 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f570c06d810 0x7f570c06fcc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5728107730 0x7f5728197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 --2- 172.21.15.57:0/991890162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728101720 0x7f57281973c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.649+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 >> 172.21.15.57:0/991890162 conn(0x7f57280fd220 msgr2=0x7f57280fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.650+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 shutdown_connections 2024-02-28T23:52:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.650+0000 7f572dc00640 1 -- 172.21.15.57:0/991890162 wait complete. 2024-02-28T23:52:20.670 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-28T23:52:20.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 -- 172.21.15.57:0/1532490095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a28105580 msgr2=0x7f1a28105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 --2- 172.21.15.57:0/1532490095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a28105580 0x7f1a28105950 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009880 tx=0x7f1a1802bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 -- 172.21.15.57:0/1532490095 shutdown_connections 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 --2- 172.21.15.57:0/1532490095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a280ff550 0x7f1a280ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 --2- 172.21.15.57:0/1532490095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a28105580 0x7f1a28105950 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 -- 172.21.15.57:0/1532490095 >> 172.21.15.57:0/1532490095 conn(0x7f1a280fb030 msgr2=0x7f1a280fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 -- 172.21.15.57:0/1532490095 shutdown_connections 2024-02-28T23:52:20.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.898+0000 7f1a2d800640 1 -- 172.21.15.57:0/1532490095 wait complete. 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.899+0000 7f1a2d800640 1 Processor -- start 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.899+0000 7f1a2d800640 1 -- start start 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a2d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a2d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 0x7f1a281978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a2d800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28197f70 con 0x7f1a280ff550 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a2d800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2819bd40 con 0x7f1a28105580 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57808/0 (socket says 172.21.15.57:57808) 2024-02-28T23:52:20.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 -- 172.21.15.57:0/3318489524 learned_addr learned my addr 172.21.15.57:0/3318489524 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a1fe00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 0x7f1a281978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 -- 172.21.15.57:0/3318489524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 msgr2=0x7f1a281978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 0x7f1a281978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a27e00640 1 -- 172.21.15.57:0/3318489524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009560 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.900+0000 7f1a1fe00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 0x7f1a281978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a27e00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1a18036040 tx=0x7f1a18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a180035f0 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a18003c10 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a18034440 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2819bfc0 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.901+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2819c550 con 0x7f1a280ff550 2024-02-28T23:52:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.902+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2804e000 con 0x7f1a280ff550 2024-02-28T23:52:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.903+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1a18005060 con 0x7f1a280ff550 2024-02-28T23:52:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.904+0000 7f1a25600640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 0x7f1a0c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.904+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1a180afa40 con 0x7f1a280ff550 2024-02-28T23:52:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.905+0000 7f1a1fe00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 0x7f1a0c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.905+0000 7f1a1fe00640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 0x7f1a0c06fcc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1a10006fd0 tx=0x7f1a10008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:20.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:20.909+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a18079150 con 0x7f1a280ff550 2024-02-28T23:52:21.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.111+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1a281986f0 con 0x7f1a280ff550 2024-02-28T23:52:21.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.111+0000 7f1a25600640 1 -- 172.21.15.57:0/3318489524 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1a1807c810 con 0x7f1a280ff550 2024-02-28T23:52:21.112 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 msgr2=0x7f1a0c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 0x7f1a0c06fcc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1a10006fd0 tx=0x7f1a10008040 comp rx=0 tx=0).stop 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 msgr2=0x7f1a281973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1a18036040 tx=0x7f1a18004270 comp rx=0 tx=0).stop 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 shutdown_connections 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a0c06d810 0x7f1a0c06fcc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a28105580 0x7f1a281978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 --2- 172.21.15.57:0/3318489524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a280ff550 0x7f1a281973a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.114+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 >> 172.21.15.57:0/3318489524 conn(0x7f1a280fb030 msgr2=0x7f1a280fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:21.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.115+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 shutdown_connections 2024-02-28T23:52:21.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.115+0000 7f1a2d800640 1 -- 172.21.15.57:0/3318489524 wait complete. 2024-02-28T23:52:21.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-28T23:52:21.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:21 smithi084 ceph-mon[70440]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:21.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:21 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3471654169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:21.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:21 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/991890162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:21.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 -- 172.21.15.57:0/3781321087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 msgr2=0x7f65a8107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 --2- 172.21.15.57:0/3781321087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a8107c50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f659c009880 tx=0x7f659c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 -- 172.21.15.57:0/3781321087 shutdown_connections 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 --2- 172.21.15.57:0/3781321087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 0x7f65a8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 --2- 172.21.15.57:0/3781321087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a8107c50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 -- 172.21.15.57:0/3781321087 >> 172.21.15.57:0/3781321087 conn(0x7f65a80fd370 msgr2=0x7f65a80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 -- 172.21.15.57:0/3781321087 shutdown_connections 2024-02-28T23:52:21.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.362+0000 7f65b0800640 1 -- 172.21.15.57:0/3781321087 wait complete. 2024-02-28T23:52:21.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.363+0000 7f65b0800640 1 Processor -- start 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.363+0000 7f65b0800640 1 -- start start 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65b0800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 0x7f65a81974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65b0800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65b0800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8198070 con 0x7f65a8107880 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65b0800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819be40 con 0x7f65a8101870 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57822/0 (socket says 172.21.15.57:57822) 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 -- 172.21.15.57:0/2525060922 learned_addr learned my addr 172.21.15.57:0/2525060922 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ade00640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 0x7f65a81974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 -- 172.21.15.57:0/2525060922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 msgr2=0x7f65a81974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 0x7f65a81974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.364+0000 7f65ad400640 1 -- 172.21.15.57:0/2525060922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009560 con 0x7f65a8107880 2024-02-28T23:52:21.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65ad400640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f659400d6f0 tx=0x7f659400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:21.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6594009a30 con 0x7f65a8107880 2024-02-28T23:52:21.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65940054e0 con 0x7f65a8107880 2024-02-28T23:52:21.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a819c120 con 0x7f65a8107880 2024-02-28T23:52:21.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6594010460 con 0x7f65a8107880 2024-02-28T23:52:21.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.365+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a819c700 con 0x7f65a8107880 2024-02-28T23:52:21.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.366+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a804e000 con 0x7f65a8107880 2024-02-28T23:52:21.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.367+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6594009b90 con 0x7f65a8107880 2024-02-28T23:52:21.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.368+0000 7f65a2a00640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 0x7f659006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.368+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f659408d150 con 0x7f65a8107880 2024-02-28T23:52:21.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.369+0000 7f65ade00640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 0x7f659006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.369+0000 7f65ade00640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 0x7f659006fcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f659c006010 tx=0x7f659c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:21.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.374+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6594056860 con 0x7f65a8107880 2024-02-28T23:52:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:21 smithi057 ceph-mon[89491]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:21.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:21 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3471654169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:21.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:21 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/991890162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.576+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f65a80008d0 con 0x7f65a8107880 2024-02-28T23:52:21.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.577+0000 7f65a2a00640 1 -- 172.21.15.57:0/2525060922 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6594059f20 con 0x7f65a8107880 2024-02-28T23:52:21.577 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 msgr2=0x7f659006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 0x7f659006fcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f659c006010 tx=0x7f659c000bc0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 msgr2=0x7f65a81979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f659400d6f0 tx=0x7f659400dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 shutdown_connections 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f659006d810 0x7f659006fcc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a8107880 0x7f65a81979e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 --2- 172.21.15.57:0/2525060922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65a8101870 0x7f65a81974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 >> 172.21.15.57:0/2525060922 conn(0x7f65a80fd370 msgr2=0x7f65a80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 shutdown_connections 2024-02-28T23:52:21.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.579+0000 7f65b0800640 1 -- 172.21.15.57:0/2525060922 wait complete. 2024-02-28T23:52:21.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-28T23:52:21.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 -- 172.21.15.57:0/3614980794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 msgr2=0x7f2d1c107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 --2- 172.21.15.57:0/3614980794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c107c50 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009880 tx=0x7f2d0c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 -- 172.21.15.57:0/3614980794 shutdown_connections 2024-02-28T23:52:21.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 --2- 172.21.15.57:0/3614980794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 0x7f2d1c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 --2- 172.21.15.57:0/3614980794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c107c50 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 -- 172.21.15.57:0/3614980794 >> 172.21.15.57:0/3614980794 conn(0x7f2d1c0fd330 msgr2=0x7f2d1c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 -- 172.21.15.57:0/3614980794 shutdown_connections 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.819+0000 7f2d22000640 1 -- 172.21.15.57:0/3614980794 wait complete. 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.820+0000 7f2d22000640 1 Processor -- start 2024-02-28T23:52:21.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.820+0000 7f2d22000640 1 -- start start 2024-02-28T23:52:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d22000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 0x7f2d1c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d22000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d22000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c198040 con 0x7f2d1c107880 2024-02-28T23:52:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d22000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19be10 con 0x7f2d1c101870 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57834/0 (socket says 172.21.15.57:57834) 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 -- 172.21.15.57:0/2722082616 learned_addr learned my addr 172.21.15.57:0/2722082616 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1b400640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 0x7f2d1c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 -- 172.21.15.57:0/2722082616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 msgr2=0x7f2d1c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 0x7f2d1c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.821+0000 7f2d1aa00640 1 -- 172.21.15.57:0/2722082616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c009560 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d1aa00640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800e970 tx=0x7f2d0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0800cc90 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d0800cdf0 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d08010430 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c19c0f0 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.822+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c19c6d0 con 0x7f2d1c107880 2024-02-28T23:52:21.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.824+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c04e000 con 0x7f2d1c107880 2024-02-28T23:52:21.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.825+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2d080046c0 con 0x7f2d1c107880 2024-02-28T23:52:21.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.826+0000 7f2d13e00640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 0x7f2d0006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:21.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.827+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d08014070 con 0x7f2d1c107880 2024-02-28T23:52:21.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.827+0000 7f2d1b400640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 0x7f2d0006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:21.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.828+0000 7f2d1b400640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 0x7f2d0006fcc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006010 tx=0x7f2d0c02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:21.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:21.831+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d080572a0 con 0x7f2d1c107880 2024-02-28T23:52:22.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.078+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d1c1987c0 con 0x7f2d1c107880 2024-02-28T23:52:22.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.078+0000 7f2d13e00640 1 -- 172.21.15.57:0/2722082616 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5025 (secure 0 0 0) 0x7f2d0805a960 con 0x7f2d1c107880 2024-02-28T23:52:22.079 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:22.079 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:50:18.239166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:active","state_seq":475,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:active","state_seq":468,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44339":{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44797":{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:22.079 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 42 2024-02-28T23:52:22.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 msgr2=0x7f2d0006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 0x7f2d0006fcc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006010 tx=0x7f2d0c02cf90 comp rx=0 tx=0).stop 2024-02-28T23:52:22.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 msgr2=0x7f2d1c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800e970 tx=0x7f2d0800ed70 comp rx=0 tx=0).stop 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 shutdown_connections 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d0006d810 0x7f2d0006fcc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d1c107880 0x7f2d1c1979b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 --2- 172.21.15.57:0/2722082616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d1c101870 0x7f2d1c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 >> 172.21.15.57:0/2722082616 conn(0x7f2d1c0fd330 msgr2=0x7f2d1c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 shutdown_connections 2024-02-28T23:52:22.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.082+0000 7f2d22000640 1 -- 172.21.15.57:0/2722082616 wait complete. 2024-02-28T23:52:22.103 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-02-28T23:52:22.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 -- 172.21.15.57:0/3182105809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d80fe5d0 msgr2=0x7f23d80fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 --2- 172.21.15.57:0/3182105809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d80fe5d0 0x7f23d80fea40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009880 tx=0x7f23c402bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 -- 172.21.15.57:0/3182105809 shutdown_connections 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 --2- 172.21.15.57:0/3182105809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d80fe5d0 0x7f23d80fea40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 --2- 172.21.15.57:0/3182105809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d81045b0 0x7f23d8104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 -- 172.21.15.57:0/3182105809 >> 172.21.15.57:0/3182105809 conn(0x7f23d80752e0 msgr2=0x7f23d80756e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 -- 172.21.15.57:0/3182105809 shutdown_connections 2024-02-28T23:52:22.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.318+0000 7f23dea00640 1 -- 172.21.15.57:0/3182105809 wait complete. 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.319+0000 7f23dea00640 1 Processor -- start 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23dea00640 1 -- start start 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23dea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d80fe5d0 0x7f23d8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23dea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23dea00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d8198120 con 0x7f23d81045b0 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23dea00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d819bef0 con 0x7f23d80fe5d0 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57856/0 (socket says 172.21.15.57:57856) 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.320+0000 7f23d7400640 1 -- 172.21.15.57:0/3101401608 learned_addr learned my addr 172.21.15.57:0/3101401608 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d7e00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d80fe5d0 0x7f23d8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d7400640 1 -- 172.21.15.57:0/3101401608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d80fe5d0 msgr2=0x7f23d8197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d7400640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d80fe5d0 0x7f23d8197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d7400640 1 -- 172.21.15.57:0/3101401608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4009560 con 0x7f23d81045b0 2024-02-28T23:52:22.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d7400640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009230 tx=0x7f23c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:22.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c40035f0 con 0x7f23d81045b0 2024-02-28T23:52:22.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d819c170 con 0x7f23d81045b0 2024-02-28T23:52:22.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.321+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23c4003c10 con 0x7f23d81045b0 2024-02-28T23:52:22.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.322+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c4005670 con 0x7f23d81045b0 2024-02-28T23:52:22.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.322+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d819c670 con 0x7f23d81045b0 2024-02-28T23:52:22.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.323+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d804e000 con 0x7f23d81045b0 2024-02-28T23:52:22.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.324+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f23c4004dd0 con 0x7f23d81045b0 2024-02-28T23:52:22.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.325+0000 7f23d4c00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 0x7f23bc06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.325+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23c40af9a0 con 0x7f23d81045b0 2024-02-28T23:52:22.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.326+0000 7f23d7e00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 0x7f23bc06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.326+0000 7f23d7e00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 0x7f23bc06fc70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005fd0 tx=0x7f23c8005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3318489524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:22 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2525060922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:22.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:22 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:52:22.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.334+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23c40790b0 con 0x7f23d81045b0 2024-02-28T23:52:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3318489524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:22 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2525060922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:22.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:22 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:52:22.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.579+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f23d81988a0 con 0x7f23d81045b0 2024-02-28T23:52:22.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.580+0000 7f23d4c00640 1 -- 172.21.15.57:0/3101401608 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v42) v1 ==== 107+0+4857 (secure 0 0 0) 0x7f23c407c770 con 0x7f23d81045b0 2024-02-28T23:52:22.580 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:22.580 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:06:18.679131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24291":{"gid":24291,"name":"cephfs.smithi084.dadqfu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.84:6827/2979507340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2979507340},{"type":"v1","addr":"172.21.15.84:6827","nonce":2979507340}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24299":{"gid":24299,"name":"cephfs.smithi057.bgmggd","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6827/3659715660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3659715660},{"type":"v1","addr":"172.21.15.57:6827","nonce":3659715660}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:22.581 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 18 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.583+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 msgr2=0x7f23bc06fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.583+0000 7f23dea00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 0x7f23bc06fc70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005fd0 tx=0x7f23c8005850 comp rx=0 tx=0).stop 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.583+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 msgr2=0x7f23d8197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.583+0000 7f23dea00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009230 tx=0x7f23c4004040 comp rx=0 tx=0).stop 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.583+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 shutdown_connections 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f23bc06d7c0 0x7f23bc06fc70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23d81045b0 0x7f23d8197a90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 --2- 172.21.15.57:0/3101401608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23d80fe5d0 0x7f23d8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 >> 172.21.15.57:0/3101401608 conn(0x7f23d80752e0 msgr2=0x7f23d80673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:22.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 shutdown_connections 2024-02-28T23:52:22.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.584+0000 7f23dea00640 1 -- 172.21.15.57:0/3101401608 wait complete. 2024-02-28T23:52:22.604 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-28T23:52:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 -- 172.21.15.57:0/1124741316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 msgr2=0x7f0e94074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 --2- 172.21.15.57:0/1124741316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e94074b60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009880 tx=0x7f0e8802bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 -- 172.21.15.57:0/1124741316 shutdown_connections 2024-02-28T23:52:22.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 --2- 172.21.15.57:0/1124741316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 0x7f0e9410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 --2- 172.21.15.57:0/1124741316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e94074b60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 -- 172.21.15.57:0/1124741316 >> 172.21.15.57:0/1124741316 conn(0x7f0e940fd360 msgr2=0x7f0e940ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 -- 172.21.15.57:0/1124741316 shutdown_connections 2024-02-28T23:52:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.818+0000 7f0e9c600640 1 -- 172.21.15.57:0/1124741316 wait complete. 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.819+0000 7f0e9c600640 1 Processor -- start 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.819+0000 7f0e9c600640 1 -- start start 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.819+0000 7f0e9c600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.819+0000 7f0e9c600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 0x7f0e941a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e9c600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9419b760 con 0x7f0e94074790 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e9c600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9419b8d0 con 0x7f0e940750a0 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57882/0 (socket says 172.21.15.57:57882) 2024-02-28T23:52:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 -- 172.21.15.57:0/2422717711 learned_addr learned my addr 172.21.15.57:0/2422717711 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:22.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99200640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 0x7f0e941a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 -- 172.21.15.57:0/2422717711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 msgr2=0x7f0e941a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 0x7f0e941a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.820+0000 7f0e99c00640 1 -- 172.21.15.57:0/2422717711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009560 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e99c00640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009230 tx=0x7f0e88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e880037c0 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e88005d50 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e88034440 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9419bb50 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.821+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9419c000 con 0x7f0e94074790 2024-02-28T23:52:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.822+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9404e000 con 0x7f0e94074790 2024-02-28T23:52:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.824+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0e88004dd0 con 0x7f0e94074790 2024-02-28T23:52:22.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.826+0000 7f0e92000640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 0x7f0e7c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:22.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.826+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0e880af9d0 con 0x7f0e94074790 2024-02-28T23:52:22.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.826+0000 7f0e99200640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 0x7f0e7c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:22.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.827+0000 7f0e99200640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 0x7f0e7c06fcc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005fd0 tx=0x7f0e80005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:22.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:22.831+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e880790e0 con 0x7f0e94074790 2024-02-28T23:52:23.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.076+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f0e9419cd10 con 0x7f0e94074790 2024-02-28T23:52:23.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.077+0000 7f0e92000640 1 -- 172.21.15.57:0/2422717711 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v42) v1 ==== 107+0+4858 (secure 0 0 0) 0x7f0e880056b0 con 0x7f0e94074790 2024-02-28T23:52:23.078 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:23.078 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:08:57.460981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24291":{"gid":24291,"name":"cephfs.smithi084.dadqfu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.84:6827/2979507340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2979507340},{"type":"v1","addr":"172.21.15.84:6827","nonce":2979507340}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24299":{"gid":24299,"name":"cephfs.smithi057.bgmggd","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6827/3659715660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3659715660},{"type":"v1","addr":"172.21.15.57:6827","nonce":3659715660}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:23.079 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 19 2024-02-28T23:52:23.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 msgr2=0x7f0e7c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 0x7f0e7c06fcc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005fd0 tx=0x7f0e80005850 comp rx=0 tx=0).stop 2024-02-28T23:52:23.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 msgr2=0x7f0e941a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009230 tx=0x7f0e88004270 comp rx=0 tx=0).stop 2024-02-28T23:52:23.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 shutdown_connections 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f0e7c06d810 0x7f0e7c06fcc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e940750a0 0x7f0e941a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 --2- 172.21.15.57:0/2422717711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e94074790 0x7f0e941a16f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 >> 172.21.15.57:0/2422717711 conn(0x7f0e940fd360 msgr2=0x7f0e940ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 shutdown_connections 2024-02-28T23:52:23.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.080+0000 7f0e9c600640 1 -- 172.21.15.57:0/2422717711 wait complete. 2024-02-28T23:52:23.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-28T23:52:23.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 -- 172.21.15.57:0/3225098055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 msgr2=0x7f9d800729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 --2- 172.21.15.57:0/3225098055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d800729d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009880 tx=0x7f9d7002bd90 comp rx=0 tx=0).stop 2024-02-28T23:52:23.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 -- 172.21.15.57:0/3225098055 shutdown_connections 2024-02-28T23:52:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 --2- 172.21.15.57:0/3225098055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d8010ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 --2- 172.21.15.57:0/3225098055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d800729d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 -- 172.21.15.57:0/3225098055 >> 172.21.15.57:0/3225098055 conn(0x7f9d800faff0 msgr2=0x7f9d800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 -- 172.21.15.57:0/3225098055 shutdown_connections 2024-02-28T23:52:23.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.313+0000 7f9d85000640 1 -- 172.21.15.57:0/3225098055 wait complete. 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 Processor -- start 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 -- start start 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d801a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8019b780 con 0x7f9d80072600 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.314+0000 7f9d85000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8019b8f0 con 0x7f9d80072f10 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57898/0 (socket says 172.21.15.57:57898) 2024-02-28T23:52:23.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 -- 172.21.15.57:0/1615988623 learned_addr learned my addr 172.21.15.57:0/1615988623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7ea00640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d801a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 -- 172.21.15.57:0/1615988623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 msgr2=0x7f9d801a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d801a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7f400640 1 -- 172.21.15.57:0/1615988623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70009560 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.315+0000 7f9d7ea00640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d801a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d7f400640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036040 tx=0x7f9d70004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d700035f0 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d70003c10 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7003e450 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8019bb70 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.316+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8019c0d0 con 0x7f9d80072600 2024-02-28T23:52:23.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.317+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004e000 con 0x7f9d80072600 2024-02-28T23:52:23.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.319+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9d70004fd0 con 0x7f9d80072600 2024-02-28T23:52:23.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.321+0000 7f9d77e00640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 0x7f9d6406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.321+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9d700b00c0 con 0x7f9d80072600 2024-02-28T23:52:23.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.321+0000 7f9d7ea00640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 0x7f9d6406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.322+0000 7f9d7ea00640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 0x7f9d6406fcc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006fd0 tx=0x7f9d6c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:23.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.326+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d700798f0 con 0x7f9d80072600 2024-02-28T23:52:23.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.558+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f9d8010d6b0 con 0x7f9d80072600 2024-02-28T23:52:23.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.560+0000 7f9d77e00640 1 -- 172.21.15.57:0/1615988623 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v42) v1 ==== 107+0+4857 (secure 0 0 0) 0x7f9d7007cfb0 con 0x7f9d80072600 2024-02-28T23:52:23.560 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:23.560 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:11:03.597210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24291":{"gid":24291,"name":"cephfs.smithi084.dadqfu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.84:6827/2979507340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2979507340},{"type":"v1","addr":"172.21.15.84:6827","nonce":2979507340}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24299":{"gid":24299,"name":"cephfs.smithi057.bgmggd","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6827/3659715660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3659715660},{"type":"v1","addr":"172.21.15.57:6827","nonce":3659715660}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:23.560 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.562+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 msgr2=0x7f9d6406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.562+0000 7f9d85000640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 0x7f9d6406fcc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006fd0 tx=0x7f9d6c008040 comp rx=0 tx=0).stop 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.562+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 msgr2=0x7f9d801a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.562+0000 7f9d85000640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036040 tx=0x7f9d70004040 comp rx=0 tx=0).stop 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 shutdown_connections 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f9d6406d810 0x7f9d6406fcc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80072f10 0x7f9d801a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 --2- 172.21.15.57:0/1615988623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d80072600 0x7f9d801a1710 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 >> 172.21.15.57:0/1615988623 conn(0x7f9d800faff0 msgr2=0x7f9d80101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 shutdown_connections 2024-02-28T23:52:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.563+0000 7f9d85000640 1 -- 172.21.15.57:0/1615988623 wait complete. 2024-02-28T23:52:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:23 smithi057 ceph-mon[89491]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2722082616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:52:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3101401608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-28T23:52:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:23 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2422717711' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-28T23:52:23.581 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-28T23:52:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:23 smithi084 ceph-mon[70440]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:23.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2722082616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-28T23:52:23.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3101401608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-28T23:52:23.582 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:23 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2422717711' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-28T23:52:23.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 -- 172.21.15.57:0/4279308508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 msgr2=0x7fcc90107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4279308508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90107ae0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009880 tx=0x7fcc8002bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 -- 172.21.15.57:0/4279308508 shutdown_connections 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4279308508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4279308508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90107ae0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 -- 172.21.15.57:0/4279308508 >> 172.21.15.57:0/4279308508 conn(0x7fcc900fd200 msgr2=0x7fcc900ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 -- 172.21.15.57:0/4279308508 shutdown_connections 2024-02-28T23:52:23.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.792+0000 7fcc8f200640 1 -- 172.21.15.57:0/4279308508 wait complete. 2024-02-28T23:52:23.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.793+0000 7fcc8f200640 1 Processor -- start 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.793+0000 7fcc8f200640 1 -- start start 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90197ed0 con 0x7fcc90107710 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9019bca0 con 0x7fcc90101700 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57916/0 (socket says 172.21.15.57:57916) 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 -- 172.21.15.57:0/4159387295 learned_addr learned my addr 172.21.15.57:0/4159387295 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8de00640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 -- 172.21.15.57:0/4159387295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 msgr2=0x7fcc90197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8d400640 1 -- 172.21.15.57:0/4159387295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009560 con 0x7fcc90107710 2024-02-28T23:52:23.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.794+0000 7fcc8de00640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:23.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc8d400640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00d6f0 tx=0x7fcc7c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:23.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c009a30 con 0x7fcc90107710 2024-02-28T23:52:23.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc7c0054e0 con 0x7fcc90107710 2024-02-28T23:52:23.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9019bf80 con 0x7fcc90107710 2024-02-28T23:52:23.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c010460 con 0x7fcc90107710 2024-02-28T23:52:23.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.795+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9019c560 con 0x7fcc90107710 2024-02-28T23:52:23.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.796+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9004e000 con 0x7fcc90107710 2024-02-28T23:52:23.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.797+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcc7c009b90 con 0x7fcc90107710 2024-02-28T23:52:23.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.799+0000 7fcc86a00640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 0x7fcc7406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:23.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.799+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcc7c08d150 con 0x7fcc90107710 2024-02-28T23:52:23.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.799+0000 7fcc8de00640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 0x7fcc7406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:23.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.800+0000 7fcc8de00640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 0x7fcc7406fcc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc80036040 tx=0x7fcc8002cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:23.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:23.803+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc7c056860 con 0x7fcc90107710 2024-02-28T23:52:24.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.033+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fcc9010e8f0 con 0x7fcc90107710 2024-02-28T23:52:24.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.035+0000 7fcc86a00640 1 -- 172.21.15.57:0/4159387295 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v42) v1 ==== 107+0+4856 (secure 0 0 0) 0x7fcc7c059f20 con 0x7fcc90107710 2024-02-28T23:52:24.035 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:24.035 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:11:42.560463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24291":{"gid":24291,"name":"cephfs.smithi084.dadqfu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.84:6827/2979507340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2979507340},{"type":"v1","addr":"172.21.15.84:6827","nonce":2979507340}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24299":{"gid":24299,"name":"cephfs.smithi057.bgmggd","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6827/3659715660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3659715660},{"type":"v1","addr":"172.21.15.57:6827","nonce":3659715660}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:24.036 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 21 2024-02-28T23:52:24.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.037+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 msgr2=0x7fcc7406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 0x7fcc7406fcc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc80036040 tx=0x7fcc8002cf90 comp rx=0 tx=0).stop 2024-02-28T23:52:24.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 msgr2=0x7fcc90197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00d6f0 tx=0x7fcc7c00dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 shutdown_connections 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fcc7406d810 0x7fcc7406fcc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc90107710 0x7fcc90197840 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 --2- 172.21.15.57:0/4159387295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc90101700 0x7fcc90197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 >> 172.21.15.57:0/4159387295 conn(0x7fcc900fd200 msgr2=0x7fcc900febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 shutdown_connections 2024-02-28T23:52:24.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.038+0000 7fcc8f200640 1 -- 172.21.15.57:0/4159387295 wait complete. 2024-02-28T23:52:24.057 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 -- 172.21.15.57:0/4293989579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072f10 msgr2=0x7f874810b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 --2- 172.21.15.57:0/4293989579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072f10 0x7f874810b7f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8730009880 tx=0x7f873002bd90 comp rx=0 tx=0).stop 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 -- 172.21.15.57:0/4293989579 shutdown_connections 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 --2- 172.21.15.57:0/4293989579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072f10 0x7f874810b7f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 --2- 172.21.15.57:0/4293989579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072600 0x7f87480729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 -- 172.21.15.57:0/4293989579 >> 172.21.15.57:0/4293989579 conn(0x7f87480faff0 msgr2=0x7f87480fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 -- 172.21.15.57:0/4293989579 shutdown_connections 2024-02-28T23:52:24.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.274+0000 7f874d200640 1 -- 172.21.15.57:0/4293989579 wait complete. 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.275+0000 7f874d200640 1 Processor -- start 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.275+0000 7f874d200640 1 -- start start 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f874d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f874d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072f10 0x7f87481979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f874d200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8748198030 con 0x7f8748072600 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f874d200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874819be00 con 0x7f8748072f10 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f8747400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f8747400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57934/0 (socket says 172.21.15.57:57934) 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.276+0000 7f8747400640 1 -- 172.21.15.57:0/2834654365 learned_addr learned my addr 172.21.15.57:0/2834654365 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f873ea00640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072f10 0x7f87481979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8747400640 1 -- 172.21.15.57:0/2834654365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072f10 msgr2=0x7f87481979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8747400640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072f10 0x7f87481979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8747400640 1 -- 172.21.15.57:0/2834654365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009560 con 0x7f8748072600 2024-02-28T23:52:24.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8747400640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f873800ba50 tx=0x7f873800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:24.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873800cb30 con 0x7f8748072600 2024-02-28T23:52:24.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f873800cc90 con 0x7f8748072600 2024-02-28T23:52:24.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8738007740 con 0x7f8748072600 2024-02-28T23:52:24.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874819c0e0 con 0x7f8748072600 2024-02-28T23:52:24.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.277+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874819c640 con 0x7f8748072600 2024-02-28T23:52:24.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.279+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874804e000 con 0x7f8748072600 2024-02-28T23:52:24.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.279+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8738014b90 con 0x7f8748072600 2024-02-28T23:52:24.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.280+0000 7f8744c00640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 0x7f872c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.280+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f873808ce30 con 0x7f8748072600 2024-02-28T23:52:24.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.281+0000 7f873ea00640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 0x7f872c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:24.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.281+0000 7f873ea00640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 0x7f872c06fcc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8730036040 tx=0x7f8730005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:24.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.285+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f873805a240 con 0x7f8748072600 2024-02-28T23:52:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:24 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:52:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:24 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:52:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:24 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:52:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:24 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1615988623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-28T23:52:24.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:24 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4159387295' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-28T23:52:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.516+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f874810e950 con 0x7f8748072600 2024-02-28T23:52:24.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.518+0000 7f8744c00640 1 -- 172.21.15.57:0/2834654365 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v42) v1 ==== 107+0+3284 (secure 0 0 0) 0x7f8738019cf0 con 0x7f8748072600 2024-02-28T23:52:24.518 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:24.518 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:10.809654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:24.518 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 22 2024-02-28T23:52:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 msgr2=0x7f872c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 0x7f872c06fcc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8730036040 tx=0x7f8730005a20 comp rx=0 tx=0).stop 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 msgr2=0x7f8748197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f873800ba50 tx=0x7f873800bf10 comp rx=0 tx=0).stop 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 shutdown_connections 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f872c06d810 0x7f872c06fcc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8748072f10 0x7f87481979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 --2- 172.21.15.57:0/2834654365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8748072600 0x7f8748197460 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 >> 172.21.15.57:0/2834654365 conn(0x7f87480faff0 msgr2=0x7f8748106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 shutdown_connections 2024-02-28T23:52:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.520+0000 7f874d200640 1 -- 172.21.15.57:0/2834654365 wait complete. 2024-02-28T23:52:24.539 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-02-28T23:52:24.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-28T23:52:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:24 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:52:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:24 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:52:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:24 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:52:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:24 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1615988623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-28T23:52:24.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:24 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4159387295' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-28T23:52:24.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 -- 172.21.15.57:0/3424708818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fe1a0 msgr2=0x7fe968104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 --2- 172.21.15.57:0/3424708818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fe1a0 0x7fe968104170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe950009c20 tx=0x7fe95002c1b0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 -- 172.21.15.57:0/3424708818 shutdown_connections 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 --2- 172.21.15.57:0/3424708818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fe1a0 0x7fe968104170 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 --2- 172.21.15.57:0/3424708818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9680fd800 0x7fe9680fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 -- 172.21.15.57:0/3424708818 >> 172.21.15.57:0/3424708818 conn(0x7fe9680f9200 msgr2=0x7fe9680fb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 -- 172.21.15.57:0/3424708818 shutdown_connections 2024-02-28T23:52:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.764+0000 7fe967200640 1 -- 172.21.15.57:0/3424708818 wait complete. 2024-02-28T23:52:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.765+0000 7fe967200640 1 Processor -- start 2024-02-28T23:52:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.766+0000 7fe967200640 1 -- start start 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.766+0000 7fe967200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.766+0000 7fe967200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9680fe1a0 0x7fe9680ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.766+0000 7fe967200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680ffa80 con 0x7fe9680fd800 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.766+0000 7fe967200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680ffbf0 con 0x7fe9680fe1a0 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57962/0 (socket says 172.21.15.57:57962) 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 -- 172.21.15.57:0/1202213229 learned_addr learned my addr 172.21.15.57:0/1202213229 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:24.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 -- 172.21.15.57:0/1202213229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9680fe1a0 msgr2=0x7fe9680ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9680fe1a0 0x7fe9680ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 -- 172.21.15.57:0/1202213229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009900 con 0x7fe9680fd800 2024-02-28T23:52:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.767+0000 7fe965e00640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe96000c8a0 tx=0x7fe96000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.768+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe960007b80 con 0x7fe9680fd800 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.768+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe960007ce0 con 0x7fe9680fd800 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.768+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe960018810 con 0x7fe9680fd800 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.768+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681a12f0 con 0x7fe9680fd800 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.768+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681a1660 con 0x7fe9680fd800 2024-02-28T23:52:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.769+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96804e000 con 0x7fe9680fd800 2024-02-28T23:52:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.770+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe960018a80 con 0x7fe9680fd800 2024-02-28T23:52:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.771+0000 7fe95ea00640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 0x7fe94c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.771+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe96008d790 con 0x7fe9680fd800 2024-02-28T23:52:24.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.771+0000 7fe965400640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 0x7fe94c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.772+0000 7fe965400640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 0x7fe94c06fcc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe950004af0 tx=0x7fe950004a40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:24.775+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe960056ea0 con 0x7fe9680fd800 2024-02-28T23:52:25.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.019+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fe9681079f0 con 0x7fe9680fd800 2024-02-28T23:52:25.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.021+0000 7fe95ea00640 1 -- 172.21.15.57:0/1202213229 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v42) v1 ==== 107+0+4838 (secure 0 0 0) 0x7fe96005a560 con 0x7fe9680fd800 2024-02-28T23:52:25.022 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:25.022 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34428,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/3579754700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3579754700},{"type":"v1","addr":"172.21.15.57:6827","nonce":3579754700}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:10.809654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi057.sbkaml","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.57:6829/2848794823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":2848794823},{"type":"v1","addr":"172.21.15.57:6829","nonce":2848794823}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi084.ztfxpc","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6825/2141462155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":2141462155},{"type":"v1","addr":"172.21.15.84:6825","nonce":2141462155}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:25.022 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 23 2024-02-28T23:52:25.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 msgr2=0x7fe94c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 0x7fe94c06fcc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe950004af0 tx=0x7fe950004a40 comp rx=0 tx=0).stop 2024-02-28T23:52:25.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 msgr2=0x7fe9680ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe96000c8a0 tx=0x7fe96000cd60 comp rx=0 tx=0).stop 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 shutdown_connections 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe94c06d810 0x7fe94c06fcc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9680fe1a0 0x7fe9680ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 --2- 172.21.15.57:0/1202213229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9680fd800 0x7fe9680ff000 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 >> 172.21.15.57:0/1202213229 conn(0x7fe9680f9200 msgr2=0x7fe9680fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 shutdown_connections 2024-02-28T23:52:25.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.025+0000 7fe967200640 1 -- 172.21.15.57:0/1202213229 wait complete. 2024-02-28T23:52:25.044 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-02-28T23:52:25.044 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-28T23:52:25.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 -- 172.21.15.57:0/4272520370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e4105560 msgr2=0x7fa2e4105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/4272520370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e4105560 0x7fa2e4105930 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009880 tx=0x7fa2d402bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 -- 172.21.15.57:0/4272520370 shutdown_connections 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/4272520370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e40ff530 0x7fa2e40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/4272520370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e4105560 0x7fa2e4105930 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 -- 172.21.15.57:0/4272520370 >> 172.21.15.57:0/4272520370 conn(0x7fa2e40faff0 msgr2=0x7fa2e40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 -- 172.21.15.57:0/4272520370 shutdown_connections 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.276+0000 7fa2e3e00640 1 -- 172.21.15.57:0/4272520370 wait complete. 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.277+0000 7fa2e3e00640 1 Processor -- start 2024-02-28T23:52:25.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.277+0000 7fa2e3e00640 1 -- start start 2024-02-28T23:52:25.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e4105560 0x7fa2e4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4197ea0 con 0x7fa2e40ff530 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e419bc70 con 0x7fa2e4105560 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38604/0 (socket says 172.21.15.57:38604) 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 -- 172.21.15.57:0/3059332584 learned_addr learned my addr 172.21.15.57:0/3059332584 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2000640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e4105560 0x7fa2e4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 -- 172.21.15.57:0/3059332584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e4105560 msgr2=0x7fa2e4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e4105560 0x7fa2e4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.278+0000 7fa2e2a00640 1 -- 172.21.15.57:0/3059332584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4009560 con 0x7fa2e40ff530 2024-02-28T23:52:25.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2e2a00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009230 tx=0x7fa2d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:25.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d40035f0 con 0x7fa2e40ff530 2024-02-28T23:52:25.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e419bef0 con 0x7fa2e40ff530 2024-02-28T23:52:25.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d4003c10 con 0x7fa2e40ff530 2024-02-28T23:52:25.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d4005670 con 0x7fa2e40ff530 2024-02-28T23:52:25.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.279+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e419c470 con 0x7fa2e40ff530 2024-02-28T23:52:25.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.281+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e404e000 con 0x7fa2e40ff530 2024-02-28T23:52:25.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.281+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa2d4004dd0 con 0x7fa2e40ff530 2024-02-28T23:52:25.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.282+0000 7fa2db400640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 0x7fa2c806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.283+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa2d40af9a0 con 0x7fa2e40ff530 2024-02-28T23:52:25.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.283+0000 7fa2e2000640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 0x7fa2c806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.284+0000 7fa2e2000640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 0x7fa2c806fc70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005fd0 tx=0x7fa2d0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:25.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.288+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2d40790b0 con 0x7fa2e40ff530 2024-02-28T23:52:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:25 smithi057 ceph-mon[89491]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2834654365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-28T23:52:25.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:25 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1202213229' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-28T23:52:25.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.529+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fa2e4198620 con 0x7fa2e40ff530 2024-02-28T23:52:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.530+0000 7fa2db400640 1 -- 172.21.15.57:0/3059332584 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v42) v1 ==== 107+0+3260 (secure 0 0 0) 0x7fa2d4041cd0 con 0x7fa2e40ff530 2024-02-28T23:52:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:25.530 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34428,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/3579754700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3579754700},{"type":"v1","addr":"172.21.15.57:6827","nonce":3579754700}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:25.530 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-02-28T23:52:25.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 msgr2=0x7fa2c806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 0x7fa2c806fc70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005fd0 tx=0x7fa2d0005850 comp rx=0 tx=0).stop 2024-02-28T23:52:25.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 msgr2=0x7fa2e41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009230 tx=0x7fa2d4004040 comp rx=0 tx=0).stop 2024-02-28T23:52:25.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 shutdown_connections 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fa2c806d7c0 0x7fa2c806fc70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2e4105560 0x7fa2e4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 --2- 172.21.15.57:0/3059332584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2e40ff530 0x7fa2e41972d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 >> 172.21.15.57:0/3059332584 conn(0x7fa2e40faff0 msgr2=0x7fa2e40fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 shutdown_connections 2024-02-28T23:52:25.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.533+0000 7fa2e3e00640 1 -- 172.21.15.57:0/3059332584 wait complete. 2024-02-28T23:52:25.552 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:25.552 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-02-28T23:52:25.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-28T23:52:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:25 smithi084 ceph-mon[70440]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2834654365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-28T23:52:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:25 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1202213229' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 -- 172.21.15.57:0/878922350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0107740 msgr2=0x7fd0b0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 --2- 172.21.15.57:0/878922350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0107740 0x7fd0b0107b10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009880 tx=0x7fd0a002bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 -- 172.21.15.57:0/878922350 shutdown_connections 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 --2- 172.21.15.57:0/878922350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0101730 0x7fd0b0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 --2- 172.21.15.57:0/878922350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0107740 0x7fd0b0107b10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 -- 172.21.15.57:0/878922350 >> 172.21.15.57:0/878922350 conn(0x7fd0b00fd230 msgr2=0x7fd0b00ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.775+0000 7fd0af200640 1 -- 172.21.15.57:0/878922350 shutdown_connections 2024-02-28T23:52:25.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.776+0000 7fd0af200640 1 -- 172.21.15.57:0/878922350 wait complete. 2024-02-28T23:52:25.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 Processor -- start 2024-02-28T23:52:25.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 -- start start 2024-02-28T23:52:25.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 0x7fd0b0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b0197ef0 con 0x7fd0b0101730 2024-02-28T23:52:25.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.777+0000 7fd0af200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b019bcc0 con 0x7fd0b0107740 2024-02-28T23:52:25.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38622/0 (socket says 172.21.15.57:38622) 2024-02-28T23:52:25.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 -- 172.21.15.57:0/3555627049 learned_addr learned my addr 172.21.15.57:0/3555627049 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:25.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ad400640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 0x7fd0b0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 -- 172.21.15.57:0/3555627049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 msgr2=0x7fd0b0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 0x7fd0b0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ade00640 1 -- 172.21.15.57:0/3555627049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0009560 con 0x7fd0b0101730 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.778+0000 7fd0ad400640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 0x7fd0b0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0ade00640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002c220 tx=0x7fd0a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a00035f0 con 0x7fd0b0101730 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0a0003c10 con 0x7fd0b0101730 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a0034440 con 0x7fd0b0101730 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b019bf40 con 0x7fd0b0101730 2024-02-28T23:52:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.779+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b019c4c0 con 0x7fd0b0101730 2024-02-28T23:52:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.780+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b004e000 con 0x7fd0b0101730 2024-02-28T23:52:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.781+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd0a0004ff0 con 0x7fd0b0101730 2024-02-28T23:52:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.783+0000 7fd0a6a00640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 0x7fd09406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.783+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd0a00afac0 con 0x7fd0b0101730 2024-02-28T23:52:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.783+0000 7fd0ad400640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 0x7fd09406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:25.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.784+0000 7fd0ad400640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 0x7fd09406fcc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0198930 tx=0x7fd09c005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:25.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:25.788+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0a00791d0 con 0x7fd0b0101730 2024-02-28T23:52:26.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.028+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fd0b010eb70 con 0x7fd0b0101730 2024-02-28T23:52:26.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.030+0000 7fd0a6a00640 1 -- 172.21.15.57:0/3555627049 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v42) v1 ==== 107+0+4818 (secure 0 0 0) 0x7fd0a007c890 con 0x7fd0b0101730 2024-02-28T23:52:26.030 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:26.030 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34428,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/3579754700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3579754700},{"type":"v1","addr":"172.21.15.57:6827","nonce":3579754700}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34436,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/3389712306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3389712306},{"type":"v1","addr":"172.21.15.57:6829","nonce":3389712306}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44289,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/3824521120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3824521120},{"type":"v1","addr":"172.21.15.84:6825","nonce":3824521120}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:26.031 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 msgr2=0x7fd09406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 0x7fd09406fcc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0198930 tx=0x7fd09c005d30 comp rx=0 tx=0).stop 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 msgr2=0x7fd0b0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002c220 tx=0x7fd0a0004040 comp rx=0 tx=0).stop 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 shutdown_connections 2024-02-28T23:52:26.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd09406d810 0x7fd09406fcc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107740 0x7fd0b0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 --2- 172.21.15.57:0/3555627049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0b0101730 0x7fd0b0197320 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 >> 172.21.15.57:0/3555627049 conn(0x7fd0b00fd230 msgr2=0x7fd0b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:26.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 shutdown_connections 2024-02-28T23:52:26.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.033+0000 7fd0af200640 1 -- 172.21.15.57:0/3555627049 wait complete. 2024-02-28T23:52:26.051 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:26.051 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-02-28T23:52:26.051 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-28T23:52:26.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 -- 172.21.15.57:0/123141443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 msgr2=0x7fc28c10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 --2- 172.21.15.57:0/123141443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c10c040 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009880 tx=0x7fc27c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 -- 172.21.15.57:0/123141443 shutdown_connections 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 --2- 172.21.15.57:0/123141443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 0x7fc28c1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 --2- 172.21.15.57:0/123141443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c10c040 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 -- 172.21.15.57:0/123141443 >> 172.21.15.57:0/123141443 conn(0x7fc28c0754a0 msgr2=0x7fc28c0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 -- 172.21.15.57:0/123141443 shutdown_connections 2024-02-28T23:52:26.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.275+0000 7fc291600640 1 -- 172.21.15.57:0/123141443 wait complete. 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 Processor -- start 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 -- start start 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 0x7fc28c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c1a22b0 con 0x7fc28c10bc70 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.276+0000 7fc291600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c19b820 con 0x7fc28c106090 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:26.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38638/0 (socket says 172.21.15.57:38638) 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 -- 172.21.15.57:0/3276558392 learned_addr learned my addr 172.21.15.57:0/3276558392 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc28aa00640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 0x7fc28c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 -- 172.21.15.57:0/3276558392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 msgr2=0x7fc28c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 0x7fc28c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.277+0000 7fc283e00640 1 -- 172.21.15.57:0/3276558392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c009560 con 0x7fc28c10bc70 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc283e00640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c8f0 tx=0x7fc27400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc274007b80 con 0x7fc28c10bc70 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2740054e0 con 0x7fc28c10bc70 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c19bb00 con 0x7fc28c10bc70 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc27400f450 con 0x7fc28c10bc70 2024-02-28T23:52:26.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.278+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c19c060 con 0x7fc28c10bc70 2024-02-28T23:52:26.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.279+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c04e000 con 0x7fc28c10bc70 2024-02-28T23:52:26.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.280+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc274004f60 con 0x7fc28c10bc70 2024-02-28T23:52:26.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.281+0000 7fc283400640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 0x7fc27006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.281+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc27408d3e0 con 0x7fc28c10bc70 2024-02-28T23:52:26.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.281+0000 7fc28aa00640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 0x7fc27006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:26.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.282+0000 7fc28aa00640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 0x7fc27006fcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006010 tx=0x7fc27c02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:26.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.286+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc274056af0 con 0x7fc28c10bc70 2024-02-28T23:52:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:26 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3059332584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-28T23:52:26.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:26 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3555627049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-28T23:52:26.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.528+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fc28c19cd50 con 0x7fc28c10bc70 2024-02-28T23:52:26.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.529+0000 7fc283400640 1 -- 172.21.15.57:0/3276558392 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v42) v1 ==== 107+0+4039 (secure 0 0 0) 0x7fc27405a1b0 con 0x7fc28c10bc70 2024-02-28T23:52:26.529 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:26.530 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/3389712306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3389712306},{"type":"v1","addr":"172.21.15.57:6829","nonce":3389712306}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44289,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/3824521120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3824521120},{"type":"v1","addr":"172.21.15.84:6825","nonce":3824521120}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:26.530 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2024-02-28T23:52:26.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 msgr2=0x7fc27006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 0x7fc27006fcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006010 tx=0x7fc27c02cf90 comp rx=0 tx=0).stop 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 msgr2=0x7fc28c1a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c8f0 tx=0x7fc27400cdb0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 shutdown_connections 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc27006d810 0x7fc27006fcc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc28c10bc70 0x7fc28c1a1c20 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 --2- 172.21.15.57:0/3276558392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c106090 0x7fc28c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 >> 172.21.15.57:0/3276558392 conn(0x7fc28c0754a0 msgr2=0x7fc28c0770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.532+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 shutdown_connections 2024-02-28T23:52:26.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.533+0000 7fc291600640 1 -- 172.21.15.57:0/3276558392 wait complete. 2024-02-28T23:52:26.553 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:26.554 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-02-28T23:52:26.554 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-28T23:52:26.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:26 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3059332584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-28T23:52:26.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:26 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3555627049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-28T23:52:26.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.779+0000 7efe0d400640 1 -- 172.21.15.57:0/815399711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08107730 msgr2=0x7efe08107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.779+0000 7efe0d400640 1 --2- 172.21.15.57:0/815399711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08107730 0x7efe08107b00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009c20 tx=0x7efdf802c1b0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 -- 172.21.15.57:0/815399711 shutdown_connections 2024-02-28T23:52:26.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 --2- 172.21.15.57:0/815399711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe08101720 0x7efe08101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 --2- 172.21.15.57:0/815399711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08107730 0x7efe08107b00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 -- 172.21.15.57:0/815399711 >> 172.21.15.57:0/815399711 conn(0x7efe080fd220 msgr2=0x7efe080ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:26.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 -- 172.21.15.57:0/815399711 shutdown_connections 2024-02-28T23:52:26.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.780+0000 7efe0d400640 1 -- 172.21.15.57:0/815399711 wait complete. 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 Processor -- start 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 -- start start 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe08107730 0x7efe081978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08197f40 con 0x7efe08101720 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe0d400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0819bd10 con 0x7efe08107730 2024-02-28T23:52:26.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe07e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe07e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38664/0 (socket says 172.21.15.57:38664) 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.781+0000 7efe07e00640 1 -- 172.21.15.57:0/1127477862 learned_addr learned my addr 172.21.15.57:0/1127477862 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe07e00640 1 -- 172.21.15.57:0/1127477862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe08107730 msgr2=0x7efe081978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe07e00640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe08107730 0x7efe081978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe07e00640 1 -- 172.21.15.57:0/1127477862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8009900 con 0x7efe08101720 2024-02-28T23:52:26.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe07e00640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7efdf8037040 tx=0x7efdf8032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8032ed0 con 0x7efe08101720 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdf8003e30 con 0x7efe08101720 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8004890 con 0x7efe08101720 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.782+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0819bf90 con 0x7efe08101720 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.783+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0819c490 con 0x7efe08101720 2024-02-28T23:52:26.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.784+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0804e000 con 0x7efe08101720 2024-02-28T23:52:26.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.785+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efdf803b040 con 0x7efe08101720 2024-02-28T23:52:26.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.786+0000 7efe04c00640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 0x7efdec06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:26.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.786+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efdf80b04a0 con 0x7efe08101720 2024-02-28T23:52:26.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.786+0000 7efe07400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 0x7efdec06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:26.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.787+0000 7efe07400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 0x7efdec06fcc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efdf00058c0 tx=0x7efdf0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:26.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:26.790+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efdf8079cf0 con 0x7efe08101720 2024-02-28T23:52:27.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.031+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7efe081986c0 con 0x7efe08101720 2024-02-28T23:52:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.031+0000 7efe04c00640 1 -- 172.21.15.57:0/1127477862 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v42) v1 ==== 107+0+4852 (secure 0 0 0) 0x7efdf807d3b0 con 0x7efe08101720 2024-02-28T23:52:27.032 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:27.032 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/3389712306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3389712306},{"type":"v1","addr":"172.21.15.57:6829","nonce":3389712306}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44289,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/3824521120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3824521120},{"type":"v1","addr":"172.21.15.84:6825","nonce":3824521120}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 msgr2=0x7efdec06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 0x7efdec06fcc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efdf00058c0 tx=0x7efdf0005850 comp rx=0 tx=0).stop 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 msgr2=0x7efe08197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7efdf8037040 tx=0x7efdf8032b60 comp rx=0 tx=0).stop 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 shutdown_connections 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7efdec06d810 0x7efdec06fcc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe08107730 0x7efe081978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 --2- 172.21.15.57:0/1127477862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe08101720 0x7efe08197370 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.034+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 >> 172.21.15.57:0/1127477862 conn(0x7efe080fd220 msgr2=0x7efe080feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.035+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 shutdown_connections 2024-02-28T23:52:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.035+0000 7efe0d400640 1 -- 172.21.15.57:0/1127477862 wait complete. 2024-02-28T23:52:27.052 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:27.052 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-02-28T23:52:27.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-28T23:52:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 -- 172.21.15.57:0/2493898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 msgr2=0x7f05d01066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 --2- 172.21.15.57:0/2493898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01066d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009880 tx=0x7f05c002bd10 comp rx=0 tx=0).stop 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 -- 172.21.15.57:0/2493898309 shutdown_connections 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 --2- 172.21.15.57:0/2493898309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05d0106ca0 0x7f05d010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 --2- 172.21.15.57:0/2493898309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01066d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 -- 172.21.15.57:0/2493898309 >> 172.21.15.57:0/2493898309 conn(0x7f05d00752f0 msgr2=0x7f05d0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 -- 172.21.15.57:0/2493898309 shutdown_connections 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.264+0000 7f05d4c00640 1 -- 172.21.15.57:0/2493898309 wait complete. 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.265+0000 7f05d4c00640 1 Processor -- start 2024-02-28T23:52:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.265+0000 7f05d4c00640 1 -- start start 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05d4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05d4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05d0106ca0 0x7f05d01a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05d4c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d019b770 con 0x7f05d0106300 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05d4c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d019b8e0 con 0x7f05d0106ca0 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38682/0 (socket says 172.21.15.57:38682) 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 -- 172.21.15.57:0/1535706412 learned_addr learned my addr 172.21.15.57:0/1535706412 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 -- 172.21.15.57:0/1535706412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05d0106ca0 msgr2=0x7f05d01a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05d0106ca0 0x7f05d01a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.266+0000 7f05cf400640 1 -- 172.21.15.57:0/1535706412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009560 con 0x7f05d0106300 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05cf400640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f05c0036040 tx=0x7f05c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c00037c0 con 0x7f05d0106300 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05c0003de0 con 0x7f05d0106300 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c0034440 con 0x7f05d0106300 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d019bb60 con 0x7f05d0106300 2024-02-28T23:52:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.267+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d019c060 con 0x7f05d0106300 2024-02-28T23:52:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.268+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d004e000 con 0x7f05d0106300 2024-02-28T23:52:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.269+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f05c0005060 con 0x7f05d0106300 2024-02-28T23:52:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.270+0000 7f05c7e00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 0x7f05b006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.270+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f05c00af920 con 0x7f05d0106300 2024-02-28T23:52:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.271+0000 7f05cea00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 0x7f05b006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.271+0000 7f05cea00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 0x7f05b006fcc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f05d019cbe0 tx=0x7f05b8009250 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:27.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.275+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05c0079150 con 0x7f05d0106300 2024-02-28T23:52:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:27 smithi057 ceph-mon[89491]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:27 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3276558392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-28T23:52:27.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:27 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1127477862' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-28T23:52:27.483 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:27 smithi084 ceph-mon[70440]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:27.483 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:27 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3276558392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-28T23:52:27.483 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:27 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1127477862' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-28T23:52:27.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.514+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f05d019cdc0 con 0x7f05d0106300 2024-02-28T23:52:27.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.515+0000 7f05c7e00640 1 -- 172.21.15.57:0/1535706412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v42) v1 ==== 107+0+4073 (secure 0 0 0) 0x7f05c007c810 con 0x7f05d0106300 2024-02-28T23:52:27.515 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:27.515 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44289,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/3824521120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3824521120},{"type":"v1","addr":"172.21.15.84:6825","nonce":3824521120}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:27.515 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 28 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 msgr2=0x7f05b006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 0x7f05b006fcc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f05d019cbe0 tx=0x7f05b8009250 comp rx=0 tx=0).stop 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 msgr2=0x7f05d01a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f05c0036040 tx=0x7f05c0004270 comp rx=0 tx=0).stop 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 shutdown_connections 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f05b006d810 0x7f05b006fcc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05d0106ca0 0x7f05d01a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 --2- 172.21.15.57:0/1535706412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d0106300 0x7f05d01a1700 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.517+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 >> 172.21.15.57:0/1535706412 conn(0x7f05d00752f0 msgr2=0x7f05d00769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:27.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.518+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 shutdown_connections 2024-02-28T23:52:27.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.518+0000 7f05d4c00640 1 -- 172.21.15.57:0/1535706412 wait complete. 2024-02-28T23:52:27.537 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:27.537 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-02-28T23:52:27.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 -- 172.21.15.57:0/2652358701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d8105580 msgr2=0x7fd6d8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 --2- 172.21.15.57:0/2652358701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d8105580 0x7fd6d8105950 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009880 tx=0x7fd6c802bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 -- 172.21.15.57:0/2652358701 shutdown_connections 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 --2- 172.21.15.57:0/2652358701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d80ff550 0x7fd6d80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 --2- 172.21.15.57:0/2652358701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d8105580 0x7fd6d8105950 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 -- 172.21.15.57:0/2652358701 >> 172.21.15.57:0/2652358701 conn(0x7fd6d80fb030 msgr2=0x7fd6d80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 -- 172.21.15.57:0/2652358701 shutdown_connections 2024-02-28T23:52:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.747+0000 7fd6dda00640 1 -- 172.21.15.57:0/2652358701 wait complete. 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.748+0000 7fd6dda00640 1 Processor -- start 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.748+0000 7fd6dda00640 1 -- start start 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6dda00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6dda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d8105580 0x7fd6d8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6dda00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8197e40 con 0x7fd6d80ff550 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6dda00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8197fb0 con 0x7fd6d8105580 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38700/0 (socket says 172.21.15.57:38700) 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6d7e00640 1 -- 172.21.15.57:0/22686262 learned_addr learned my addr 172.21.15.57:0/22686262 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.749+0000 7fd6d7400640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d8105580 0x7fd6d8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d7e00640 1 -- 172.21.15.57:0/22686262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d8105580 msgr2=0x7fd6d8197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d7e00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d8105580 0x7fd6d8197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d7e00640 1 -- 172.21.15.57:0/22686262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009560 con 0x7fd6d80ff550 2024-02-28T23:52:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d7e00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8036040 tx=0x7fd6c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c80035f0 con 0x7fd6d80ff550 2024-02-28T23:52:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c8003c10 con 0x7fd6d80ff550 2024-02-28T23:52:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c8034440 con 0x7fd6d80ff550 2024-02-28T23:52:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.750+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d819bde0 con 0x7fd6d80ff550 2024-02-28T23:52:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.751+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d819c340 con 0x7fd6d80ff550 2024-02-28T23:52:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.752+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d804e000 con 0x7fd6d80ff550 2024-02-28T23:52:27.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.753+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd6c8005060 con 0x7fd6d80ff550 2024-02-28T23:52:27.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.753+0000 7fd6d4c00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 0x7fd6bc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:27.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.754+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd6c80afa40 con 0x7fd6d80ff550 2024-02-28T23:52:27.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.754+0000 7fd6d7400640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 0x7fd6bc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:27.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.755+0000 7fd6d7400640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 0x7fd6bc06fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005e10 tx=0x7fd6c4005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:27.759+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6c8079150 con 0x7fd6d80ff550 2024-02-28T23:52:28.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.004+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fd6d8198650 con 0x7fd6d80ff550 2024-02-28T23:52:28.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.005+0000 7fd6d4c00640 1 -- 172.21.15.57:0/22686262 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v42) v1 ==== 107+0+4883 (secure 0 0 0) 0x7fd6c807c810 con 0x7fd6d80ff550 2024-02-28T23:52:28.006 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:28.006 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44289,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/3824521120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3824521120},{"type":"v1","addr":"172.21.15.84:6825","nonce":3824521120}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:28.006 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 29 2024-02-28T23:52:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 msgr2=0x7fd6bc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 0x7fd6bc06fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005e10 tx=0x7fd6c4005da0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 msgr2=0x7fd6d8197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8036040 tx=0x7fd6c8004270 comp rx=0 tx=0).stop 2024-02-28T23:52:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 shutdown_connections 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fd6bc06d810 0x7fd6bc06fcc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6d8105580 0x7fd6d8197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 --2- 172.21.15.57:0/22686262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6d80ff550 0x7fd6d8197300 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 >> 172.21.15.57:0/22686262 conn(0x7fd6d80fb030 msgr2=0x7fd6d80fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 shutdown_connections 2024-02-28T23:52:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.008+0000 7fd6dda00640 1 -- 172.21.15.57:0/22686262 wait complete. 2024-02-28T23:52:28.028 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:28.028 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-02-28T23:52:28.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-28T23:52:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.254+0000 7f4655000640 1 -- 172.21.15.57:0/3641820259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 msgr2=0x7f465010c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.254+0000 7f4655000640 1 --2- 172.21.15.57:0/3641820259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f465010c040 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4640009880 tx=0x7f464002bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 -- 172.21.15.57:0/3641820259 shutdown_connections 2024-02-28T23:52:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 --2- 172.21.15.57:0/3641820259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 0x7f46501064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 --2- 172.21.15.57:0/3641820259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f465010c040 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 -- 172.21.15.57:0/3641820259 >> 172.21.15.57:0/3641820259 conn(0x7f46500754a0 msgr2=0x7f46500778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 -- 172.21.15.57:0/3641820259 shutdown_connections 2024-02-28T23:52:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.255+0000 7f4655000640 1 -- 172.21.15.57:0/3641820259 wait complete. 2024-02-28T23:52:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 Processor -- start 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 -- start start 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 0x7f46501a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465019b840 con 0x7f465010bc70 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.256+0000 7f4655000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465019b9b0 con 0x7f4650106090 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38712/0 (socket says 172.21.15.57:38712) 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 -- 172.21.15.57:0/1908806102 learned_addr learned my addr 172.21.15.57:0/1908806102 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:28.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464de00640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 0x7f46501a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 -- 172.21.15.57:0/1908806102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 msgr2=0x7f46501a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 0x7f46501a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 -- 172.21.15.57:0/1908806102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640009560 con 0x7f465010bc70 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.257+0000 7f464d400640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d6f0 tx=0x7f463c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.258+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463c009a30 con 0x7f465010bc70 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.258+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463c0054e0 con 0x7f465010bc70 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.258+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463c010460 con 0x7f465010bc70 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.258+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465019bc90 con 0x7f465010bc70 2024-02-28T23:52:28.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.258+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465019c1f0 con 0x7f465010bc70 2024-02-28T23:52:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.259+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465004e000 con 0x7f465010bc70 2024-02-28T23:52:28.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.260+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f463c009b90 con 0x7f465010bc70 2024-02-28T23:52:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.261+0000 7f4646a00640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 0x7f463406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.262+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f463c08d060 con 0x7f465010bc70 2024-02-28T23:52:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.262+0000 7f464de00640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 0x7f463406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.262+0000 7f464de00640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 0x7f463406fcc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4640006010 tx=0x7f4640000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:28.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.268+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f463c0567f0 con 0x7f465010bc70 2024-02-28T23:52:28.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:28 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1535706412' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-28T23:52:28.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:28 smithi057 ceph-mon[89491]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:28.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:28 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/22686262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-28T23:52:28.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.509+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f465010e0d0 con 0x7f465010bc70 2024-02-28T23:52:28.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.511+0000 7f4646a00640 1 -- 172.21.15.57:0/1908806102 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v42) v1 ==== 107+0+4104 (secure 0 0 0) 0x7f463c020090 con 0x7f465010bc70 2024-02-28T23:52:28.511 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:28.511 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:28.511 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 30 2024-02-28T23:52:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 msgr2=0x7f463406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 0x7f463406fcc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4640006010 tx=0x7f4640000bc0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 msgr2=0x7f46501a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d6f0 tx=0x7f463c00dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 shutdown_connections 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f463406d810 0x7f463406fcc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f465010bc70 0x7f46501a1d10 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.513+0000 7f4655000640 1 --2- 172.21.15.57:0/1908806102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4650106090 0x7f46501a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.514+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 >> 172.21.15.57:0/1908806102 conn(0x7f46500754a0 msgr2=0x7f46500770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.514+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 shutdown_connections 2024-02-28T23:52:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.514+0000 7f4655000640 1 -- 172.21.15.57:0/1908806102 wait complete. 2024-02-28T23:52:28.531 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:28.531 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-02-28T23:52:28.532 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-28T23:52:28.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:28 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1535706412' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-28T23:52:28.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:28 smithi084 ceph-mon[70440]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:28.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:28 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/22686262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-28T23:52:28.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 -- 172.21.15.57:0/2070152142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948101860 msgr2=0x7f3948101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 --2- 172.21.15.57:0/2070152142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948101860 0x7f3948101cd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3934009880 tx=0x7f393402bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 -- 172.21.15.57:0/2070152142 shutdown_connections 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 --2- 172.21.15.57:0/2070152142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948101860 0x7f3948101cd0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 --2- 172.21.15.57:0/2070152142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948107870 0x7f3948107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 -- 172.21.15.57:0/2070152142 >> 172.21.15.57:0/2070152142 conn(0x7f39480fd340 msgr2=0x7f39480ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 -- 172.21.15.57:0/2070152142 shutdown_connections 2024-02-28T23:52:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.758+0000 7f394d600640 1 -- 172.21.15.57:0/2070152142 wait complete. 2024-02-28T23:52:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.759+0000 7f394d600640 1 Processor -- start 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.759+0000 7f394d600640 1 -- start start 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f394d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 0x7f3948197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f394d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f394d600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481980e0 con 0x7f3948107870 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f394d600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394819beb0 con 0x7f3948101860 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38734/0 (socket says 172.21.15.57:38734) 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 -- 172.21.15.57:0/3682969341 learned_addr learned my addr 172.21.15.57:0/3682969341 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946a00640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 0x7f3948197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 -- 172.21.15.57:0/3682969341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 msgr2=0x7f3948197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 0x7f3948197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946000640 1 -- 172.21.15.57:0/3682969341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934009560 con 0x7f3948107870 2024-02-28T23:52:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.760+0000 7f3946a00640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 0x7f3948197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f3946000640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39340035f0 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3934003c10 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394819c130 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3934005670 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.761+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394819c630 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.762+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394804e000 con 0x7f3948107870 2024-02-28T23:52:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.763+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3934004dd0 con 0x7f3948107870 2024-02-28T23:52:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.764+0000 7f393f400640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 0x7f392c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.765+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f39340af860 con 0x7f3948107870 2024-02-28T23:52:28.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.765+0000 7f3946a00640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 0x7f392c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:28.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.766+0000 7f3946a00640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 0x7f392c06fcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3938005d00 tx=0x7f3938005ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:28.770+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3934079090 con 0x7f3948107870 2024-02-28T23:52:29.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.012+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f3948198860 con 0x7f3948107870 2024-02-28T23:52:29.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.013+0000 7f393f400640 1 -- 172.21.15.57:0/3682969341 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v42) v1 ==== 107+0+4914 (secure 0 0 0) 0x7f3934081e60 con 0x7f3948107870 2024-02-28T23:52:29.014 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:29.014 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44283,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/369172854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":369172854},{"type":"v1","addr":"172.21.15.84:6827","nonce":369172854}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:29.014 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 msgr2=0x7f392c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 0x7f392c06fcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3938005d00 tx=0x7f3938005ed0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 msgr2=0x7f3948197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004040 comp rx=0 tx=0).stop 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 shutdown_connections 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f392c06d810 0x7f392c06fcc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3948107870 0x7f3948197a50 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 --2- 172.21.15.57:0/3682969341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3948101860 0x7f3948197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.016+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 >> 172.21.15.57:0/3682969341 conn(0x7f39480fd340 msgr2=0x7f39480fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.017+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 shutdown_connections 2024-02-28T23:52:29.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.017+0000 7f394d600640 1 -- 172.21.15.57:0/3682969341 wait complete. 2024-02-28T23:52:29.038 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:29.038 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-02-28T23:52:29.038 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 -- 172.21.15.57:0/1614147346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 msgr2=0x7f1a70067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 --2- 172.21.15.57:0/1614147346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a70067760 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009c20 tx=0x7f1a6002c1b0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 -- 172.21.15.57:0/1614147346 shutdown_connections 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 --2- 172.21.15.57:0/1614147346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70067d30 0x7f1a7010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 --2- 172.21.15.57:0/1614147346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a70067760 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 -- 172.21.15.57:0/1614147346 >> 172.21.15.57:0/1614147346 conn(0x7f1a70074450 msgr2=0x7f1a70074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 -- 172.21.15.57:0/1614147346 shutdown_connections 2024-02-28T23:52:29.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.260+0000 7f1a74a00640 1 -- 172.21.15.57:0/1614147346 wait complete. 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.261+0000 7f1a74a00640 1 Processor -- start 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a74a00640 1 -- start start 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a74a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a74a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70067d30 0x7f1a701a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a74a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701a2030 con 0x7f1a70067390 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a74a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7019b530 con 0x7f1a70067d30 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a6f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a6f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38740/0 (socket says 172.21.15.57:38740) 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.262+0000 7f1a6f400640 1 -- 172.21.15.57:0/2481096360 learned_addr learned my addr 172.21.15.57:0/2481096360 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a6f400640 1 -- 172.21.15.57:0/2481096360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70067d30 msgr2=0x7f1a701a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a6f400640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70067d30 0x7f1a701a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a6f400640 1 -- 172.21.15.57:0/2481096360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60009900 con 0x7f1a70067390 2024-02-28T23:52:29.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a6f400640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1a60037040 tx=0x7f1a60032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:29.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a60034820 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a60003e30 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7019b810 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a6003f440 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.263+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7019bd70 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.265+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7004e000 con 0x7f1a70067390 2024-02-28T23:52:29.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.265+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1a6003b040 con 0x7f1a70067390 2024-02-28T23:52:29.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.266+0000 7f1a67e00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 0x7f1a5406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.266+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1a600b0630 con 0x7f1a70067390 2024-02-28T23:52:29.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.267+0000 7f1a6ea00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 0x7f1a5406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.268+0000 7f1a6ea00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 0x7f1a5406fcc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a7019cd20 tx=0x7f1a58005bf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.272+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a60079e00 con 0x7f1a70067390 2024-02-28T23:52:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1908806102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-28T23:52:29.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:29 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3682969341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-28T23:52:29.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.515+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f1a701005d0 con 0x7f1a70067390 2024-02-28T23:52:29.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.517+0000 7f1a67e00640 1 -- 172.21.15.57:0/2481096360 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v42) v1 ==== 107+0+4138 (secure 0 0 0) 0x7f1a701005d0 con 0x7f1a70067390 2024-02-28T23:52:29.517 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:29.518 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:29.518 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 32 2024-02-28T23:52:29.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 msgr2=0x7f1a5406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 0x7f1a5406fcc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a7019cd20 tx=0x7f1a58005bf0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 msgr2=0x7f1a701a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1a60037040 tx=0x7f1a60032b60 comp rx=0 tx=0).stop 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 shutdown_connections 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f1a5406d810 0x7f1a5406fcc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70067d30 0x7f1a701a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 --2- 172.21.15.57:0/2481096360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a70067390 0x7f1a701a1460 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 >> 172.21.15.57:0/2481096360 conn(0x7f1a70074450 msgr2=0x7f1a7010e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 shutdown_connections 2024-02-28T23:52:29.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.520+0000 7f1a74a00640 1 -- 172.21.15.57:0/2481096360 wait complete. 2024-02-28T23:52:29.542 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:29.542 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-02-28T23:52:29.542 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-28T23:52:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1908806102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-28T23:52:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:29 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3682969341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 -- 172.21.15.57:0/2591042967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 msgr2=0x7f3404101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 --2- 172.21.15.57:0/2591042967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404101cd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009880 tx=0x7f33f002bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 -- 172.21.15.57:0/2591042967 shutdown_connections 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 --2- 172.21.15.57:0/2591042967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404101cd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 --2- 172.21.15.57:0/2591042967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 0x7f3404107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 -- 172.21.15.57:0/2591042967 >> 172.21.15.57:0/2591042967 conn(0x7f34040fd360 msgr2=0x7f34040ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 -- 172.21.15.57:0/2591042967 shutdown_connections 2024-02-28T23:52:29.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.771+0000 7f340a400640 1 -- 172.21.15.57:0/2591042967 wait complete. 2024-02-28T23:52:29.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 Processor -- start 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 -- start start 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 0x7f3404197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404198160 con 0x7f3404101860 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.772+0000 7f340a400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340419bf30 con 0x7f3404107870 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38760/0 (socket says 172.21.15.57:38760) 2024-02-28T23:52:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 -- 172.21.15.57:0/1266051793 learned_addr learned my addr 172.21.15.57:0/1266051793 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3402a00640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 0x7f3404197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 -- 172.21.15.57:0/1266051793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 msgr2=0x7f3404197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 0x7f3404197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 -- 172.21.15.57:0/1266051793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0009560 con 0x7f3404101860 2024-02-28T23:52:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.773+0000 7f3403400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d8d0 tx=0x7f33f400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.774+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f4009a30 con 0x7f3404101860 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.774+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340419c210 con 0x7f3404101860 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.774+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33f4009b90 con 0x7f3404101860 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.774+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f400f680 con 0x7f3404101860 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.774+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340419c770 con 0x7f3404101860 2024-02-28T23:52:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.775+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340404e000 con 0x7f3404101860 2024-02-28T23:52:29.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.776+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f33f4009d00 con 0x7f3404101860 2024-02-28T23:52:29.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.777+0000 7f33fbe00640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 0x7f33e806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:29.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.777+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f33f408d3d0 con 0x7f3404101860 2024-02-28T23:52:29.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.777+0000 7f3402a00640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 0x7f33e806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:29.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.778+0000 7f3402a00640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 0x7f33e806fcc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f33f0036040 tx=0x7f33f0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:29.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:29.782+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33f4056b20 con 0x7f3404101860 2024-02-28T23:52:30.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.024+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f3404109d40 con 0x7f3404101860 2024-02-28T23:52:30.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.025+0000 7f33fbe00640 1 -- 172.21.15.57:0/1266051793 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v42) v1 ==== 107+0+4951 (secure 0 0 0) 0x7f33f405a1e0 con 0x7f3404101860 2024-02-28T23:52:30.025 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:30.025 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:18:13.072176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:30.025 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 msgr2=0x7f33e806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 0x7f33e806fcc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f33f0036040 tx=0x7f33f0005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 msgr2=0x7f3404197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d8d0 tx=0x7f33f400dd90 comp rx=0 tx=0).stop 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 shutdown_connections 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f33e806d810 0x7f33e806fcc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3404107870 0x7f3404197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 --2- 172.21.15.57:0/1266051793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3404101860 0x7f3404197590 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 >> 172.21.15.57:0/1266051793 conn(0x7f34040fd360 msgr2=0x7f34040fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 shutdown_connections 2024-02-28T23:52:30.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.027+0000 7f340a400640 1 -- 172.21.15.57:0/1266051793 wait complete. 2024-02-28T23:52:30.045 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-28T23:52:30.045 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-02-28T23:52:30.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 -- 172.21.15.57:0/886516102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 msgr2=0x7f2d9c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 --2- 172.21.15.57:0/886516102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c101ba0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009880 tx=0x7f2d8402bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 -- 172.21.15.57:0/886516102 shutdown_connections 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 --2- 172.21.15.57:0/886516102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c101ba0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 --2- 172.21.15.57:0/886516102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 0x7f2d9c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 -- 172.21.15.57:0/886516102 >> 172.21.15.57:0/886516102 conn(0x7f2d9c0fd230 msgr2=0x7f2d9c0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 -- 172.21.15.57:0/886516102 shutdown_connections 2024-02-28T23:52:30.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.233+0000 7f2da1c00640 1 -- 172.21.15.57:0/886516102 wait complete. 2024-02-28T23:52:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 Processor -- start 2024-02-28T23:52:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 -- start start 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 0x7f2d9c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c197e90 con 0x7f2d9c101730 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.235+0000 7f2da1c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c19bc60 con 0x7f2d9c107740 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38776/0 (socket says 172.21.15.57:38776) 2024-02-28T23:52:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 -- 172.21.15.57:0/1487052983 learned_addr learned my addr 172.21.15.57:0/1487052983 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9b400640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 0x7f2d9c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 -- 172.21.15.57:0/1487052983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 msgr2=0x7f2d9c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 0x7f2d9c197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 -- 172.21.15.57:0/1487052983 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009560 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.236+0000 7f2d9be00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400eb40 tx=0x7f2d9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.237+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d94003970 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.237+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d94004030 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.237+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c19bf40 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.237+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d94019760 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.237+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c19c520 con 0x7f2d9c101730 2024-02-28T23:52:30.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.238+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c04e000 con 0x7f2d9c101730 2024-02-28T23:52:30.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.239+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2d940041a0 con 0x7f2d9c101730 2024-02-28T23:52:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.240+0000 7f2d98c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 0x7f2d8006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.240+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d94014070 con 0x7f2d9c101730 2024-02-28T23:52:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.241+0000 7f2d9b400640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 0x7f2d8006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.242+0000 7f2d9b400640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 0x7f2d8006fcc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2d84036040 tx=0x7f2d84005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.245+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d94091030 con 0x7f2d9c101730 2024-02-28T23:52:30.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.505+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f2d9c198610 con 0x7f2d9c101730 2024-02-28T23:52:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.506+0000 7f2d98c00640 1 -- 172.21.15.57:0/1487052983 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v42) v1 ==== 107+0+4950 (secure 0 0 0) 0x7f2d94057690 con 0x7f2d9c101730 2024-02-28T23:52:30.507 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:30.507 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:35.376941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 34 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 msgr2=0x7f2d8006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 0x7f2d8006fcc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2d84036040 tx=0x7f2d84005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 msgr2=0x7f2d9c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400eb40 tx=0x7f2d9400c2d0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 shutdown_connections 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2d8006d810 0x7f2d8006fcc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c107740 0x7f2d9c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 --2- 172.21.15.57:0/1487052983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d9c101730 0x7f2d9c1972c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 >> 172.21.15.57:0/1487052983 conn(0x7f2d9c0fd230 msgr2=0x7f2d9c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 shutdown_connections 2024-02-28T23:52:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.509+0000 7f2da1c00640 1 -- 172.21.15.57:0/1487052983 wait complete. 2024-02-28T23:52:30.523 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2481096360' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-28T23:52:30.524 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:30 smithi057 ceph-mon[89491]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:30.524 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:30 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1266051793' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-28T23:52:30.526 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-02-28T23:52:30.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-28T23:52:30.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2481096360' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-28T23:52:30.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:30 smithi084 ceph-mon[70440]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:30.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:30 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1266051793' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 -- 172.21.15.57:0/2792005528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed740ff550 msgr2=0x7fed740ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 --2- 172.21.15.57:0/2792005528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed740ff550 0x7fed740ff9c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fed60009880 tx=0x7fed6002bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 -- 172.21.15.57:0/2792005528 shutdown_connections 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 --2- 172.21.15.57:0/2792005528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed740ff550 0x7fed740ff9c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 --2- 172.21.15.57:0/2792005528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed74105580 0x7fed74105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 -- 172.21.15.57:0/2792005528 >> 172.21.15.57:0/2792005528 conn(0x7fed740fb030 msgr2=0x7fed740fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.720+0000 7fed7a400640 1 -- 172.21.15.57:0/2792005528 shutdown_connections 2024-02-28T23:52:30.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.721+0000 7fed7a400640 1 -- 172.21.15.57:0/2792005528 wait complete. 2024-02-28T23:52:30.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 Processor -- start 2024-02-28T23:52:30.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 -- start start 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed740ff550 0x7fed74197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed74198060 con 0x7fed74105580 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.722+0000 7fed7a400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7419be30 con 0x7fed740ff550 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38800/0 (socket says 172.21.15.57:38800) 2024-02-28T23:52:30.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 -- 172.21.15.57:0/3194360461 learned_addr learned my addr 172.21.15.57:0/3194360461 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:30.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 -- 172.21.15.57:0/3194360461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed740ff550 msgr2=0x7fed74197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed740ff550 0x7fed74197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 -- 172.21.15.57:0/3194360461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60009560 con 0x7fed74105580 2024-02-28T23:52:30.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.723+0000 7fed73e00640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fed60009230 tx=0x7fed60004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:30.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.724+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed600035f0 con 0x7fed74105580 2024-02-28T23:52:30.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.724+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed60003c10 con 0x7fed74105580 2024-02-28T23:52:30.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.724+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7419c0b0 con 0x7fed74105580 2024-02-28T23:52:30.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.724+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6003e460 con 0x7fed74105580 2024-02-28T23:52:30.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.724+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7419c690 con 0x7fed74105580 2024-02-28T23:52:30.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.725+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7404e000 con 0x7fed74105580 2024-02-28T23:52:30.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.727+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fed60004fd0 con 0x7fed74105580 2024-02-28T23:52:30.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.729+0000 7fed71600640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 0x7fed5c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:30.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.729+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed600afe20 con 0x7fed74105580 2024-02-28T23:52:30.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.729+0000 7fed79000640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 0x7fed5c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:30.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.730+0000 7fed79000640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 0x7fed5c06fcc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed68009710 tx=0x7fed68006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:30.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.733+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed600795b0 con 0x7fed74105580 2024-02-28T23:52:30.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.975+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fed740008d0 con 0x7fed74105580 2024-02-28T23:52:30.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.976+0000 7fed71600640 1 -- 172.21.15.57:0/3194360461 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v42) v1 ==== 107+0+5007 (secure 0 0 0) 0x7fed60042090 con 0x7fed74105580 2024-02-28T23:52:30.976 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:30.976 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:36.380077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:30.977 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 35 2024-02-28T23:52:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 msgr2=0x7fed5c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 0x7fed5c06fcc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed68009710 tx=0x7fed68006c90 comp rx=0 tx=0).stop 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 msgr2=0x7fed741979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fed60009230 tx=0x7fed60004040 comp rx=0 tx=0).stop 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 shutdown_connections 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fed5c06d810 0x7fed5c06fcc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed74105580 0x7fed741979d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 --2- 172.21.15.57:0/3194360461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed740ff550 0x7fed74197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 >> 172.21.15.57:0/3194360461 conn(0x7fed740fb030 msgr2=0x7fed740728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 shutdown_connections 2024-02-28T23:52:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:30.978+0000 7fed7a400640 1 -- 172.21.15.57:0/3194360461 wait complete. 2024-02-28T23:52:30.995 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-02-28T23:52:30.995 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-28T23:52:31.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 -- 172.21.15.57:0/4158361044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c105580 msgr2=0x7f720c105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 --2- 172.21.15.57:0/4158361044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c105580 0x7f720c105950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009880 tx=0x7f71fc02bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 -- 172.21.15.57:0/4158361044 shutdown_connections 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 --2- 172.21.15.57:0/4158361044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0ff550 0x7f720c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 --2- 172.21.15.57:0/4158361044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c105580 0x7f720c105950 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 -- 172.21.15.57:0/4158361044 >> 172.21.15.57:0/4158361044 conn(0x7f720c0fb030 msgr2=0x7f720c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 -- 172.21.15.57:0/4158361044 shutdown_connections 2024-02-28T23:52:31.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.203+0000 7f720b200640 1 -- 172.21.15.57:0/4158361044 wait complete. 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 Processor -- start 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 -- start start 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c105580 0x7f720c072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c073070 con 0x7f720c0ff550 2024-02-28T23:52:31.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.204+0000 7f720b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c0731b0 con 0x7f720c105580 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38820/0 (socket says 172.21.15.57:38820) 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 -- 172.21.15.57:0/2931765278 learned_addr learned my addr 172.21.15.57:0/2931765278 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7203e00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c105580 0x7f720c072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 -- 172.21.15.57:0/2931765278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c105580 msgr2=0x7f720c072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c105580 0x7f720c072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.205+0000 7f7209e00640 1 -- 172.21.15.57:0/2931765278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009560 con 0x7f720c0ff550 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f7209e00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009230 tx=0x7f71fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71fc0035f0 con 0x7f720c0ff550 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71fc003c10 con 0x7f720c0ff550 2024-02-28T23:52:31.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71fc03e460 con 0x7f720c0ff550 2024-02-28T23:52:31.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c0733e0 con 0x7f720c0ff550 2024-02-28T23:52:31.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.206+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c1a17a0 con 0x7f720c0ff550 2024-02-28T23:52:31.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.207+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c04e000 con 0x7f720c0ff550 2024-02-28T23:52:31.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.208+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f71fc004fd0 con 0x7f720c0ff550 2024-02-28T23:52:31.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.209+0000 7f7202a00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 0x7f71f006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.210+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f71fc0aff10 con 0x7f720c0ff550 2024-02-28T23:52:31.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.210+0000 7f7203e00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 0x7f71f006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.210+0000 7f7203e00640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 0x7f71f006fcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f720c0740d0 tx=0x7f71f4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:31.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.214+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71fc079620 con 0x7f720c0ff550 2024-02-28T23:52:31.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.458+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f720c107990 con 0x7f720c0ff550 2024-02-28T23:52:31.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.459+0000 7f7202a00640 1 -- 172.21.15.57:0/2931765278 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v42) v1 ==== 107+0+5006 (secure 0 0 0) 0x7f71fc07cce0 con 0x7f720c0ff550 2024-02-28T23:52:31.460 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:31.460 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:36.391380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:31.460 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 36 2024-02-28T23:52:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.462+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 msgr2=0x7f71f006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.462+0000 7f720b200640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 0x7f71f006fcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f720c0740d0 tx=0x7f71f4008040 comp rx=0 tx=0).stop 2024-02-28T23:52:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.462+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 msgr2=0x7f720c072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.462+0000 7f720b200640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009230 tx=0x7f71fc004040 comp rx=0 tx=0).stop 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 shutdown_connections 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f71f006d810 0x7f71f006fcc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c105580 0x7f720c072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 --2- 172.21.15.57:0/2931765278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f720c0ff550 0x7f720c072560 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 >> 172.21.15.57:0/2931765278 conn(0x7f720c0fb030 msgr2=0x7f720c0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 shutdown_connections 2024-02-28T23:52:31.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.463+0000 7f720b200640 1 -- 172.21.15.57:0/2931765278 wait complete. 2024-02-28T23:52:31.478 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:31 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1487052983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-28T23:52:31.479 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:31 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3194360461' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-28T23:52:31.482 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-02-28T23:52:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:31 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1487052983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-28T23:52:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:31 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3194360461' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-28T23:52:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 -- 172.21.15.57:0/2937787261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 msgr2=0x7fc1a4067770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/2937787261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a4067770 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc190009880 tx=0x7fc19002bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 -- 172.21.15.57:0/2937787261 shutdown_connections 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/2937787261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a4067770 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/2937787261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 0x7fc1a4104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 -- 172.21.15.57:0/2937787261 >> 172.21.15.57:0/2937787261 conn(0x7fc1a4074450 msgr2=0x7fc1a4074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 -- 172.21.15.57:0/2937787261 shutdown_connections 2024-02-28T23:52:31.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.728+0000 7fc1a8e00640 1 -- 172.21.15.57:0/2937787261 wait complete. 2024-02-28T23:52:31.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.730+0000 7fc1a8e00640 1 Processor -- start 2024-02-28T23:52:31.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.730+0000 7fc1a8e00640 1 -- start start 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.731+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.731+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 0x7fc1a40ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.731+0000 7fc1a8e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40ff970 con 0x7fc1a4067300 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.731+0000 7fc1a8e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40ffae0 con 0x7fc1a41044b0 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.731+0000 7fc1a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38832/0 (socket says 172.21.15.57:38832) 2024-02-28T23:52:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 -- 172.21.15.57:0/3385960300 learned_addr learned my addr 172.21.15.57:0/3385960300 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a2a00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 0x7fc1a40ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 -- 172.21.15.57:0/3385960300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 msgr2=0x7fc1a40ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 0x7fc1a40ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 -- 172.21.15.57:0/3385960300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc190009560 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.732+0000 7fc1a3400640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b420 tx=0x7fc19400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.733+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc194011800 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.733+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc194011e20 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.733+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a40ffd60 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.733+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19400f2e0 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.733+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a17c0 con 0x7fc1a4067300 2024-02-28T23:52:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.734+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a404e000 con 0x7fc1a4067300 2024-02-28T23:52:31.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.735+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc194010380 con 0x7fc1a4067300 2024-02-28T23:52:31.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.737+0000 7fc19be00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 0x7fc18806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:31.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.737+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc19408cce0 con 0x7fc1a4067300 2024-02-28T23:52:31.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.737+0000 7fc1a2a00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 0x7fc18806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:31.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.738+0000 7fc1a2a00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 0x7fc18806fcc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1a41006e0 tx=0x7fc190000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:31.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.740+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc194056470 con 0x7fc1a4067300 2024-02-28T23:52:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.981+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fc1a4100420 con 0x7fc1a4067300 2024-02-28T23:52:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.982+0000 7fc19be00640 1 -- 172.21.15.57:0/3385960300 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v42) v1 ==== 107+0+5012 (secure 0 0 0) 0x7fc194059b30 con 0x7fc1a4067300 2024-02-28T23:52:31.983 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:31.983 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:42.977573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:resolve","state_seq":472,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:resolve","state_seq":465,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 37 2024-02-28T23:52:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 msgr2=0x7fc18806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 0x7fc18806fcc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1a41006e0 tx=0x7fc190000bc0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 msgr2=0x7fc1a40feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b420 tx=0x7fc19400b8e0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 shutdown_connections 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fc18806d810 0x7fc18806fcc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1a41044b0 0x7fc1a40ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 --2- 172.21.15.57:0/3385960300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1a4067300 0x7fc1a40feef0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 >> 172.21.15.57:0/3385960300 conn(0x7fc1a4074450 msgr2=0x7fc1a40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 shutdown_connections 2024-02-28T23:52:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:31.985+0000 7fc1a8e00640 1 -- 172.21.15.57:0/3385960300 wait complete. 2024-02-28T23:52:32.004 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-02-28T23:52:32.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.222+0000 7f2f68400640 1 -- 172.21.15.57:0/2633278001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60101860 msgr2=0x7f2f60101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.222+0000 7f2f68400640 1 --2- 172.21.15.57:0/2633278001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60101860 0x7f2f60101cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009880 tx=0x7f2f4c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 -- 172.21.15.57:0/2633278001 shutdown_connections 2024-02-28T23:52:32.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 --2- 172.21.15.57:0/2633278001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60101860 0x7f2f60101cd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 --2- 172.21.15.57:0/2633278001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60107870 0x7f2f60107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 -- 172.21.15.57:0/2633278001 >> 172.21.15.57:0/2633278001 conn(0x7f2f600fd340 msgr2=0x7f2f600ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:32.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 -- 172.21.15.57:0/2633278001 shutdown_connections 2024-02-28T23:52:32.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.223+0000 7f2f68400640 1 -- 172.21.15.57:0/2633278001 wait complete. 2024-02-28T23:52:32.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.224+0000 7f2f68400640 1 Processor -- start 2024-02-28T23:52:32.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.224+0000 7f2f68400640 1 -- start start 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f68400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60101860 0x7f2f60197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f68400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f68400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60198100 con 0x7f2f60107870 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f68400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6019bed0 con 0x7f2f60101860 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f65000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f65000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38842/0 (socket says 172.21.15.57:38842) 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.225+0000 7f2f65000640 1 -- 172.21.15.57:0/2331009271 learned_addr learned my addr 172.21.15.57:0/2331009271 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.226+0000 7f2f65a00640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60101860 0x7f2f60197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.226+0000 7f2f65000640 1 -- 172.21.15.57:0/2331009271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60101860 msgr2=0x7f2f60197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.226+0000 7f2f65000640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60101860 0x7f2f60197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.226+0000 7f2f65000640 1 -- 172.21.15.57:0/2331009271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c009560 con 0x7f2f60107870 2024-02-28T23:52:32.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.226+0000 7f2f65000640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009230 tx=0x7f2f4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:32.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.227+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f4c0035f0 con 0x7f2f60107870 2024-02-28T23:52:32.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.227+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f4c003c10 con 0x7f2f60107870 2024-02-28T23:52:32.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.227+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6019c150 con 0x7f2f60107870 2024-02-28T23:52:32.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.227+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f4c005670 con 0x7f2f60107870 2024-02-28T23:52:32.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.227+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6019c650 con 0x7f2f60107870 2024-02-28T23:52:32.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.228+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6004e000 con 0x7f2f60107870 2024-02-28T23:52:32.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.229+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2f4c004dd0 con 0x7f2f60107870 2024-02-28T23:52:32.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.230+0000 7f2f5e000640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 0x7f2f4806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.230+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f4c0af9a0 con 0x7f2f60107870 2024-02-28T23:52:32.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.230+0000 7f2f65a00640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 0x7f2f4806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.231+0000 7f2f65a00640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 0x7f2f4806fc70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005fd0 tx=0x7f2f54005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.235+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f4c0790b0 con 0x7f2f60107870 2024-02-28T23:52:32.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.479+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f2f600008d0 con 0x7f2f60107870 2024-02-28T23:52:32.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.480+0000 7f2f5e000640 1 -- 172.21.15.57:0/2331009271 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v42) v1 ==== 107+0+5016 (secure 0 0 0) 0x7f2f4c07c770 con 0x7f2f60107870 2024-02-28T23:52:32.481 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:32.481 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:43.117851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":473,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:reconnect","state_seq":466,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:32.481 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 38 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.483+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 msgr2=0x7f2f4806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.483+0000 7f2f68400640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 0x7f2f4806fc70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005fd0 tx=0x7f2f54005850 comp rx=0 tx=0).stop 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.483+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 msgr2=0x7f2f60197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.483+0000 7f2f68400640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009230 tx=0x7f2f4c004040 comp rx=0 tx=0).stop 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 shutdown_connections 2024-02-28T23:52:32.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2f4806d7c0 0x7f2f4806fc70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f60107870 0x7f2f60197a70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 --2- 172.21.15.57:0/2331009271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f60101860 0x7f2f60197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 >> 172.21.15.57:0/2331009271 conn(0x7f2f600fd340 msgr2=0x7f2f600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:32.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 shutdown_connections 2024-02-28T23:52:32.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.484+0000 7f2f68400640 1 -- 172.21.15.57:0/2331009271 wait complete. 2024-02-28T23:52:32.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-02-28T23:52:32.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:32 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2931765278' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-28T23:52:32.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:32 smithi057 ceph-mon[89491]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:32.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:32 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3385960300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-28T23:52:32.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:32 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2931765278' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-28T23:52:32.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:32 smithi084 ceph-mon[70440]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:32.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:32 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3385960300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-28T23:52:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 -- 172.21.15.57:0/1553573212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 msgr2=0x7fb278103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 --2- 172.21.15.57:0/1553573212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278103830 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb268009880 tx=0x7fb26802bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 -- 172.21.15.57:0/1553573212 shutdown_connections 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 --2- 172.21.15.57:0/1553573212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb2780ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 --2- 172.21.15.57:0/1553573212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278103830 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 -- 172.21.15.57:0/1553573212 >> 172.21.15.57:0/1553573212 conn(0x7fb278074450 msgr2=0x7fb278074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 -- 172.21.15.57:0/1553573212 shutdown_connections 2024-02-28T23:52:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.727+0000 7fb27ca00640 1 -- 172.21.15.57:0/1553573212 wait complete. 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.728+0000 7fb27ca00640 1 Processor -- start 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.728+0000 7fb27ca00640 1 -- start start 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb278197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb27ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278197e60 con 0x7fb278103460 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb27ca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278197fd0 con 0x7fb2780fee20 2024-02-28T23:52:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38852/0 (socket says 172.21.15.57:38852) 2024-02-28T23:52:32.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 -- 172.21.15.57:0/4070817294 learned_addr learned my addr 172.21.15.57:0/4070817294 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb277400640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb278197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 -- 172.21.15.57:0/4070817294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 msgr2=0x7fb278197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb278197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb276a00640 1 -- 172.21.15.57:0/4070817294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268009560 con 0x7fb278103460 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.729+0000 7fb277400640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb276a00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c8a0 tx=0x7fb26400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb264007b80 con 0x7fb278103460 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb264007ce0 con 0x7fb278103460 2024-02-28T23:52:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb264018810 con 0x7fb278103460 2024-02-28T23:52:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27819be60 con 0x7fb278103460 2024-02-28T23:52:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.730+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27819c3c0 con 0x7fb278103460 2024-02-28T23:52:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.731+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27804e000 con 0x7fb278103460 2024-02-28T23:52:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.733+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb264018970 con 0x7fb278103460 2024-02-28T23:52:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.734+0000 7fb26fe00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 0x7fb25c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.734+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb26408d6f0 con 0x7fb278103460 2024-02-28T23:52:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.734+0000 7fb277400640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 0x7fb25c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.735+0000 7fb277400640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 0x7fb25c06fcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb268036040 tx=0x7fb26802cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:32.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.738+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb264056e80 con 0x7fb278103460 2024-02-28T23:52:32.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.976+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fb27810b820 con 0x7fb278103460 2024-02-28T23:52:32.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.978+0000 7fb26fe00640 1 -- 172.21.15.57:0/4070817294 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v42) v1 ==== 107+0+5010 (secure 0 0 0) 0x7fb26405a540 con 0x7fb278103460 2024-02-28T23:52:32.978 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:32.978 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:44.168448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":474,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:rejoin","state_seq":467,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 39 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.980+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 msgr2=0x7fb25c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.980+0000 7fb27ca00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 0x7fb25c06fcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb268036040 tx=0x7fb26802cf90 comp rx=0 tx=0).stop 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.980+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 msgr2=0x7fb278197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.980+0000 7fb27ca00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c8a0 tx=0x7fb26400cd60 comp rx=0 tx=0).stop 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 shutdown_connections 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fb25c06d810 0x7fb25c06fcc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb278103460 0x7fb278197860 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 --2- 172.21.15.57:0/4070817294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2780fee20 0x7fb278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 >> 172.21.15.57:0/4070817294 conn(0x7fb278074450 msgr2=0x7fb278067b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 shutdown_connections 2024-02-28T23:52:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:32.981+0000 7fb27ca00640 1 -- 172.21.15.57:0/4070817294 wait complete. 2024-02-28T23:52:32.998 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.212+0000 7fda9b200640 1 -- 172.21.15.57:0/1902337690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 msgr2=0x7fda9c107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.212+0000 7fda9b200640 1 --2- 172.21.15.57:0/1902337690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c107b50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009880 tx=0x7fda8c02bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 -- 172.21.15.57:0/1902337690 shutdown_connections 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 --2- 172.21.15.57:0/1902337690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 0x7fda9c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 --2- 172.21.15.57:0/1902337690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c107b50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 -- 172.21.15.57:0/1902337690 >> 172.21.15.57:0/1902337690 conn(0x7fda9c0fd2d0 msgr2=0x7fda9c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 -- 172.21.15.57:0/1902337690 shutdown_connections 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.213+0000 7fda9b200640 1 -- 172.21.15.57:0/1902337690 wait complete. 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 Processor -- start 2024-02-28T23:52:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 -- start start 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 0x7fda9c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c197f70 con 0x7fda9c107780 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda9b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c19bd40 con 0x7fda9c101770 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda99400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda99400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38872/0 (socket says 172.21.15.57:38872) 2024-02-28T23:52:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.214+0000 7fda99400640 1 -- 172.21.15.57:0/3527399319 learned_addr learned my addr 172.21.15.57:0/3527399319 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda99e00640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 0x7fda9c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda99400640 1 -- 172.21.15.57:0/3527399319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 msgr2=0x7fda9c1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda99400640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 0x7fda9c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda99400640 1 -- 172.21.15.57:0/3527399319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009560 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda99400640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b420 tx=0x7fda8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda88011800 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda88011e20 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8800f2e0 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.215+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c19c020 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.216+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c19c5b0 con 0x7fda9c107780 2024-02-28T23:52:33.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.217+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c04e000 con 0x7fda9c107780 2024-02-28T23:52:33.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.218+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fda88010380 con 0x7fda9c107780 2024-02-28T23:52:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.219+0000 7fda92a00640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 0x7fda8006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.219+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda8808cf50 con 0x7fda9c107780 2024-02-28T23:52:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.220+0000 7fda99e00640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 0x7fda8006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.220+0000 7fda99e00640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 0x7fda8006fcc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda8c036040 tx=0x7fda8c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:33.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.224+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda88056660 con 0x7fda9c107780 2024-02-28T23:52:33.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.467+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7fda9c109c50 con 0x7fda9c107780 2024-02-28T23:52:33.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.468+0000 7fda92a00640 1 -- 172.21.15.57:0/3527399319 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v42) v1 ==== 107+0+5010 (secure 0 0 0) 0x7fda88059d20 con 0x7fda9c107780 2024-02-28T23:52:33.468 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:33.468 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:45.262021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:active","state_seq":475,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:active","state_seq":468,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:33.469 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 40 2024-02-28T23:52:33.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 msgr2=0x7fda8006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 0x7fda8006fcc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda8c036040 tx=0x7fda8c005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 msgr2=0x7fda9c1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b420 tx=0x7fda8800b8e0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 shutdown_connections 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fda8006d810 0x7fda8006fcc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda9c107780 0x7fda9c1978e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 --2- 172.21.15.57:0/3527399319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c101770 0x7fda9c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 >> 172.21.15.57:0/3527399319 conn(0x7fda9c0fd2d0 msgr2=0x7fda9c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 shutdown_connections 2024-02-28T23:52:33.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.471+0000 7fda9b200640 1 -- 172.21.15.57:0/3527399319 wait complete. 2024-02-28T23:52:33.490 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-02-28T23:52:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:33 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2331009271' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-28T23:52:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:33 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/4070817294' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-28T23:52:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:33 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2331009271' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-28T23:52:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:33 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/4070817294' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-28T23:52:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 -- 172.21.15.57:0/2935634086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106300 msgr2=0x7f4a6c1066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 --2- 172.21.15.57:0/2935634086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106300 0x7f4a6c1066d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009880 tx=0x7f4a5c02bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 -- 172.21.15.57:0/2935634086 shutdown_connections 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 --2- 172.21.15.57:0/2935634086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 --2- 172.21.15.57:0/2935634086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106300 0x7f4a6c1066d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 -- 172.21.15.57:0/2935634086 >> 172.21.15.57:0/2935634086 conn(0x7f4a6c0752f0 msgr2=0x7f4a6c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 -- 172.21.15.57:0/2935634086 shutdown_connections 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.708+0000 7f4a6b200640 1 -- 172.21.15.57:0/2935634086 wait complete. 2024-02-28T23:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.709+0000 7f4a6b200640 1 Processor -- start 2024-02-28T23:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a6b200640 1 -- start start 2024-02-28T23:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 0x7f4a6c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a6b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c19b6b0 con 0x7f4a6c106ca0 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a6b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c19b820 con 0x7f4a6c106300 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.710+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38890/0 (socket says 172.21.15.57:38890) 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a63e00640 1 -- 172.21.15.57:0/3909015894 learned_addr learned my addr 172.21.15.57:0/3909015894 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a69e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 0x7f4a6c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a63e00640 1 -- 172.21.15.57:0/3909015894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 msgr2=0x7f4a6c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a63e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 0x7f4a6c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a63e00640 1 -- 172.21.15.57:0/3909015894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c009560 con 0x7f4a6c106ca0 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a69e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 0x7f4a6c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.711+0000 7f4a63e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d6d0 tx=0x7f4a5400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.712+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a540051f0 con 0x7f4a6c106ca0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.712+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a54009ce0 con 0x7f4a6c106ca0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.712+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a54004e40 con 0x7f4a6c106ca0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.712+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c19bb00 con 0x7f4a6c106ca0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.712+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c19c060 con 0x7f4a6c106ca0 2024-02-28T23:52:33.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.713+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c04e000 con 0x7f4a6c106ca0 2024-02-28T23:52:33.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.714+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a54005350 con 0x7f4a6c106ca0 2024-02-28T23:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.715+0000 7f4a62a00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 0x7f4a5006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.715+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a5408d1a0 con 0x7f4a6c106ca0 2024-02-28T23:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.715+0000 7f4a69e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 0x7f4a5006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.716+0000 7f4a69e00640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 0x7f4a5006fcc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c036040 tx=0x7f4a5c02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.720+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a54056930 con 0x7f4a6c106ca0 2024-02-28T23:52:33.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.949+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f4a6c10e020 con 0x7f4a6c106ca0 2024-02-28T23:52:33.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.950+0000 7f4a62a00640 1 -- 172.21.15.57:0/3909015894 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v42) v1 ==== 107+0+5025 (secure 0 0 0) 0x7f4a54059ff0 con 0x7f4a6c106ca0 2024-02-28T23:52:33.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:33.951 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-28T23:03:38.758886+0000","modified":"2024-02-28T23:49:46.183645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi057.bgmggd","rank":0,"incarnation":35,"state":"up:active","state_seq":475,"addr":"172.21.15.57:6827/2218205483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":2218205483},{"type":"v1","addr":"172.21.15.57:6827","nonce":2218205483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"cephfs.smithi057.sbkaml","rank":1,"incarnation":35,"state":"up:active","state_seq":468,"addr":"172.21.15.57:6829/220807232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":220807232},{"type":"v1","addr":"172.21.15.57:6829","nonce":220807232}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44339":{"gid":44339,"name":"cephfs.smithi084.ztfxpc","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6825/968233333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":968233333},{"type":"v1","addr":"172.21.15.84:6825","nonce":968233333}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44797":{"gid":44797,"name":"cephfs.smithi084.dadqfu","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6827/4290160570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":4290160570},{"type":"v1","addr":"172.21.15.84:6827","nonce":4290160570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-28T23:52:33.951 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 41 2024-02-28T23:52:33.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 msgr2=0x7f4a5006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 0x7f4a5006fcc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c036040 tx=0x7f4a5c02cf90 comp rx=0 tx=0).stop 2024-02-28T23:52:33.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 msgr2=0x7f4a6c1a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:33.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d6d0 tx=0x7f4a5400db90 comp rx=0 tx=0).stop 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 shutdown_connections 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f4a5006d810 0x7f4a5006fcc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a6c106ca0 0x7f4a6c1a1b80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 --2- 172.21.15.57:0/3909015894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a6c106300 0x7f4a6c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 >> 172.21.15.57:0/3909015894 conn(0x7f4a6c0752f0 msgr2=0x7f4a6c076970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 shutdown_connections 2024-02-28T23:52:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:33.953+0000 7f4a6b200640 1 -- 172.21.15.57:0/3909015894 wait complete. 2024-02-28T23:52:33.972 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-28T23:52:33.983 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-28T23:52:33.983 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T23:52:33.983 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-28T23:52:34.038 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T23:52:34.038 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-28T23:52:34.091 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-28T23:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 -- 172.21.15.57:0/511945787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c0ff840 msgr2=0x7f387c0ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 --2- 172.21.15.57:0/511945787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c0ff840 0x7f387c0ffcb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3868009880 tx=0x7f386802bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 -- 172.21.15.57:0/511945787 shutdown_connections 2024-02-28T23:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 --2- 172.21.15.57:0/511945787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c0ff840 0x7f387c0ffcb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 --2- 172.21.15.57:0/511945787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c105850 0x7f387c105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.315+0000 7f3884200640 1 -- 172.21.15.57:0/511945787 >> 172.21.15.57:0/511945787 conn(0x7f387c0fb320 msgr2=0x7f387c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.316+0000 7f3884200640 1 -- 172.21.15.57:0/511945787 shutdown_connections 2024-02-28T23:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.316+0000 7f3884200640 1 -- 172.21.15.57:0/511945787 wait complete. 2024-02-28T23:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.317+0000 7f3884200640 1 Processor -- start 2024-02-28T23:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.317+0000 7f3884200640 1 -- start start 2024-02-28T23:52:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3884200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c0ff840 0x7f387c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3884200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3884200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c198310 con 0x7f387c105850 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3884200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c19afe0 con 0x7f387c0ff840 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3880e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3880e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38902/0 (socket says 172.21.15.57:38902) 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3880e00640 1 -- 172.21.15.57:0/1190775913 learned_addr learned my addr 172.21.15.57:0/1190775913 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.318+0000 7f3881800640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c0ff840 0x7f387c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3880e00640 1 -- 172.21.15.57:0/1190775913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c0ff840 msgr2=0x7f387c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3880e00640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c0ff840 0x7f387c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3880e00640 1 -- 172.21.15.57:0/1190775913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3868009560 con 0x7f387c105850 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3880e00640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f386802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3868004520 con 0x7f387c105850 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3868003ab0 con 0x7f387c105850 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3868034440 con 0x7f387c105850 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c19b260 con 0x7f387c105850 2024-02-28T23:52:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.319+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c19b7e0 con 0x7f387c105850 2024-02-28T23:52:34.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.321+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c04e000 con 0x7f387c105850 2024-02-28T23:52:34.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.322+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3868003c20 con 0x7f387c105850 2024-02-28T23:52:34.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.323+0000 7f387a000640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 0x7f386406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.323+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f38680af8b0 con 0x7f387c105850 2024-02-28T23:52:34.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.323+0000 7f3881800640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 0x7f386406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.324+0000 7f3881800640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 0x7f386406fcc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3870005fd0 tx=0x7f3870005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.327+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3868079040 con 0x7f387c105850 2024-02-28T23:52:34.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.530+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f387c10ca60 con 0x7f387c105850 2024-02-28T23:52:34.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.534+0000 7f387a000640 1 -- 172.21.15.57:0/1190775913 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f386807c700 con 0x7f387c105850 2024-02-28T23:52:34.535 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:34.535 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":87,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:34:24.949532+0000","last_up_change":"2024-02-28T23:18:05.087684+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-28T23:03:37.677738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-28T23:03:38.275939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6803","nonce":1424360026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6805","nonce":1424360026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6809","nonce":1424360026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6807","nonce":1424360026}]},"public_addr":"172.21.15.57:6803/1424360026","cluster_addr":"172.21.15.57:6805/1424360026","heartbeat_back_addr":"172.21.15.57:6809/1424360026","heartbeat_front_addr":"172.21.15.57:6807/1424360026","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6811","nonce":3847484969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6813","nonce":3847484969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6817","nonce":3847484969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6815","nonce":3847484969}]},"public_addr":"172.21.15.57:6811/3847484969","cluster_addr":"172.21.15.57:6813/3847484969","heartbeat_back_addr":"172.21.15.57:6817/3847484969","heartbeat_front_addr":"172.21.15.57:6815/3847484969","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6819","nonce":3954650626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6821","nonce":3954650626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6825","nonce":3954650626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6823","nonce":3954650626}]},"public_addr":"172.21.15.57:6819/3954650626","cluster_addr":"172.21.15.57:6821/3954650626","heartbeat_back_addr":"172.21.15.57:6825/3954650626","heartbeat_front_addr":"172.21.15.57:6823/3954650626","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6801","nonce":2190333848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6803","nonce":2190333848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6807","nonce":2190333848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6805","nonce":2190333848}]},"public_addr":"172.21.15.84:6801/2190333848","cluster_addr":"172.21.15.84:6803/2190333848","heartbeat_back_addr":"172.21.15.84:6807/2190333848","heartbeat_front_addr":"172.21.15.84:6805/2190333848","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6809","nonce":890451486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6811","nonce":890451486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6815","nonce":890451486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6813","nonce":890451486}]},"public_addr":"172.21.15.84:6809/890451486","cluster_addr":"172.21.15.84:6811/890451486","heartbeat_back_addr":"172.21.15.84:6815/890451486","heartbeat_front_addr":"172.21.15.84:6813/890451486","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6817","nonce":2020554677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6819","nonce":2020554677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6823","nonce":2020554677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6821","nonce":2020554677}]},"public_addr":"172.21.15.84:6817/2020554677","cluster_addr":"172.21.15.84:6819/2020554677","heartbeat_back_addr":"172.21.15.84:6823/2020554677","heartbeat_front_addr":"172.21.15.84:6821/2020554677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6829/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.84:6824/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.84:0/3210112039":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6801/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.84:0/4232995690":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/2372541761":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/3130089202":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.84:6825/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.84:0/3528688133":"2024-02-29T23:08:14.355809+0000","172.21.15.84:6829/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.84:0/3178273795":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6826/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/4128072297":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.84:6828/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3690186012":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/4070283299":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/1810823735":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6827/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:6828/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.57:0/382831045":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/915841588":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/292614806":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/3532446948":"2024-02-29T23:12:47.945548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 msgr2=0x7f386406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 0x7f386406fcc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3870005fd0 tx=0x7f3870005850 comp rx=0 tx=0).stop 2024-02-28T23:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 msgr2=0x7f387c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f386802c850 comp rx=0 tx=0).stop 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 shutdown_connections 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f386406d810 0x7f386406fcc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f387c105850 0x7f387c197c30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 --2- 172.21.15.57:0/1190775913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f387c0ff840 0x7f387c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 >> 172.21.15.57:0/1190775913 conn(0x7f387c0fb320 msgr2=0x7f387c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 shutdown_connections 2024-02-28T23:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.537+0000 7f3884200640 1 -- 172.21.15.57:0/1190775913 wait complete. 2024-02-28T23:52:34.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-28T23:00:25.160671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-28T23:03:37.677738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-28T23:03:38.275939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-28T23:52:34.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-28T23:52:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:34 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3527399319' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-28T23:52:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:34 smithi057 ceph-mon[89491]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:34 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/3909015894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-02-28T23:52:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:34 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3527399319' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-28T23:52:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:34 smithi084 ceph-mon[70440]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:34 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/3909015894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 -- 172.21.15.57:0/4173493882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc0750a0 msgr2=0x7faafc10ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 --2- 172.21.15.57:0/4173493882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc0750a0 0x7faafc10ffc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7faaec009880 tx=0x7faaec02bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 -- 172.21.15.57:0/4173493882 shutdown_connections 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 --2- 172.21.15.57:0/4173493882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc0750a0 0x7faafc10ffc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 --2- 172.21.15.57:0/4173493882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc074790 0x7faafc074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 -- 172.21.15.57:0/4173493882 >> 172.21.15.57:0/4173493882 conn(0x7faafc0fd360 msgr2=0x7faafc0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 -- 172.21.15.57:0/4173493882 shutdown_connections 2024-02-28T23:52:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.775+0000 7fab02a00640 1 -- 172.21.15.57:0/4173493882 wait complete. 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.776+0000 7fab02a00640 1 Processor -- start 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.776+0000 7fab02a00640 1 -- start start 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7fab02a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7fab02a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc0750a0 0x7faafc1a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7fab02a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc19b700 con 0x7faafc074790 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7fab02a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc19b870 con 0x7faafc0750a0 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38934/0 (socket says 172.21.15.57:38934) 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 -- 172.21.15.57:0/2406886839 learned_addr learned my addr 172.21.15.57:0/2406886839 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafb400640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc0750a0 0x7faafc1a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 -- 172.21.15.57:0/2406886839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc0750a0 msgr2=0x7faafc1a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc0750a0 0x7faafc1a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.777+0000 7faafbe00640 1 -- 172.21.15.57:0/2406886839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009560 con 0x7faafc074790 2024-02-28T23:52:34.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7faafbe00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faae400d6f0 tx=0x7faae400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae4009a30 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faae40054e0 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc19bb50 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae4010460 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.778+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc19c0b0 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.779+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc04e000 con 0x7faafc074790 2024-02-28T23:52:34.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.780+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7faae4009b90 con 0x7faafc074790 2024-02-28T23:52:34.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.782+0000 7faaf8c00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 0x7faae006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:34.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.782+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faae408d060 con 0x7faafc074790 2024-02-28T23:52:34.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.782+0000 7faafb400640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 0x7faae006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:34.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.783+0000 7faafb400640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 0x7faae006fcc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faaec036040 tx=0x7faaec005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:34.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.787+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faae40567f0 con 0x7faafc074790 2024-02-28T23:52:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.990+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faafc10d6a0 con 0x7faafc074790 2024-02-28T23:52:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.992+0000 7faaf8c00640 1 -- 172.21.15.57:0/2406886839 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7faae4017020 con 0x7faafc074790 2024-02-28T23:52:34.992 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.994+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 msgr2=0x7faae006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.994+0000 7fab02a00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 0x7faae006fcc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faaec036040 tx=0x7faaec005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.994+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 msgr2=0x7faafc1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.994+0000 7fab02a00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faae400d6f0 tx=0x7faae400dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 shutdown_connections 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7faae006d810 0x7faae006fcc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faafc0750a0 0x7faafc1a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 --2- 172.21.15.57:0/2406886839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faafc074790 0x7faafc1a1690 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 >> 172.21.15.57:0/2406886839 conn(0x7faafc0fd360 msgr2=0x7faafc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 shutdown_connections 2024-02-28T23:52:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:34.995+0000 7fab02a00640 1 -- 172.21.15.57:0/2406886839 wait complete. 2024-02-28T23:52:35.014 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.285+0000 7f5e59000640 1 -- 172.21.15.57:0/4262673756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e540750a0 msgr2=0x7f5e5410ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.285+0000 7f5e59000640 1 --2- 172.21.15.57:0/4262673756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e540750a0 0x7f5e5410ffd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009880 tx=0x7f5e4402bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 -- 172.21.15.57:0/4262673756 shutdown_connections 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 --2- 172.21.15.57:0/4262673756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e540750a0 0x7f5e5410ffd0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 --2- 172.21.15.57:0/4262673756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e54074790 0x7f5e54074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 -- 172.21.15.57:0/4262673756 >> 172.21.15.57:0/4262673756 conn(0x7f5e540fd370 msgr2=0x7f5e540ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:35.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 -- 172.21.15.57:0/4262673756 shutdown_connections 2024-02-28T23:52:35.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.286+0000 7f5e59000640 1 -- 172.21.15.57:0/4262673756 wait complete. 2024-02-28T23:52:35.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 Processor -- start 2024-02-28T23:52:35.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 -- start start 2024-02-28T23:52:35.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 0x7f5e541a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419b730 con 0x7f5e54074790 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.287+0000 7f5e59000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419b8a0 con 0x7f5e540750a0 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41422/0 (socket says 172.21.15.57:41422) 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 -- 172.21.15.57:0/1032107753 learned_addr learned my addr 172.21.15.57:0/1032107753 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51400640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 0x7f5e541a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 -- 172.21.15.57:0/1032107753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 msgr2=0x7f5e541a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 0x7f5e541a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 -- 172.21.15.57:0/1032107753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009560 con 0x7f5e54074790 2024-02-28T23:52:35.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51400640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 0x7f5e541a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:35.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.288+0000 7f5e51e00640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000d6f0 tx=0x7f5e4000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:35.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.289+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e40009a30 con 0x7f5e54074790 2024-02-28T23:52:35.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.289+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e400054e0 con 0x7f5e54074790 2024-02-28T23:52:35.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.289+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5419bb80 con 0x7f5e54074790 2024-02-28T23:52:35.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.289+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e40010460 con 0x7f5e54074790 2024-02-28T23:52:35.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.289+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5419c0e0 con 0x7f5e54074790 2024-02-28T23:52:35.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.290+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5404e000 con 0x7f5e54074790 2024-02-28T23:52:35.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.292+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5e40009b90 con 0x7f5e54074790 2024-02-28T23:52:35.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.292+0000 7f5e4aa00640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 0x7f5e3806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.292+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e4008d150 con 0x7f5e54074790 2024-02-28T23:52:35.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.292+0000 7f5e51400640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 0x7f5e3806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.293+0000 7f5e51400640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 0x7f5e3806fcc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e44036040 tx=0x7f5e44000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:35.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.297+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e40056860 con 0x7f5e54074790 2024-02-28T23:52:35.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.505+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5e5410d6b0 con 0x7f5e54074790 2024-02-28T23:52:35.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.506+0000 7f5e4aa00640 1 -- 172.21.15.57:0/1032107753 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5e40059f20 con 0x7f5e54074790 2024-02-28T23:52:35.506 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 msgr2=0x7f5e3806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 0x7f5e3806fcc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e44036040 tx=0x7f5e44000bc0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 msgr2=0x7f5e541a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000d6f0 tx=0x7f5e4000dbb0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 shutdown_connections 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f5e3806d810 0x7f5e3806fcc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e540750a0 0x7f5e541a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 --2- 172.21.15.57:0/1032107753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e54074790 0x7f5e541a16c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 >> 172.21.15.57:0/1032107753 conn(0x7f5e540fd370 msgr2=0x7f5e540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 shutdown_connections 2024-02-28T23:52:35.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.509+0000 7f5e59000640 1 -- 172.21.15.57:0/1032107753 wait complete. 2024-02-28T23:52:35.527 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-28T23:52:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1190775913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:35 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/2406886839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:35.600 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1190775913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:35.600 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:35 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/2406886839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:35.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- 172.21.15.57:0/3737777843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d80ff4e0 msgr2=0x7f06d80ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 --2- 172.21.15.57:0/3737777843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d80ff4e0 0x7f06d80ff950 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009880 tx=0x7f06c002bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- 172.21.15.57:0/3737777843 shutdown_connections 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 --2- 172.21.15.57:0/3737777843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d80ff4e0 0x7f06d80ff950 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 --2- 172.21.15.57:0/3737777843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d81054f0 0x7f06d81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- 172.21.15.57:0/3737777843 >> 172.21.15.57:0/3737777843 conn(0x7f06d80fafe0 msgr2=0x7f06d80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- 172.21.15.57:0/3737777843 shutdown_connections 2024-02-28T23:52:35.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- 172.21.15.57:0/3737777843 wait complete. 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 Processor -- start 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.740+0000 7f06dd000640 1 -- start start 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06dd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d80ff4e0 0x7f06d81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06dd000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06dd000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8197fa0 con 0x7f06d81054f0 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06dd000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d819bd70 con 0x7f06d80ff4e0 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41440/0 (socket says 172.21.15.57:41440) 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06d6a00640 1 -- 172.21.15.57:0/614625843 learned_addr learned my addr 172.21.15.57:0/614625843 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.741+0000 7f06d7400640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d80ff4e0 0x7f06d81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06d6a00640 1 -- 172.21.15.57:0/614625843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d80ff4e0 msgr2=0x7f06d81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06d6a00640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d80ff4e0 0x7f06d81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06d6a00640 1 -- 172.21.15.57:0/614625843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009560 con 0x7f06d81054f0 2024-02-28T23:52:35.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06d6a00640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009230 tx=0x7f06c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:35.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c00035f0 con 0x7f06d81054f0 2024-02-28T23:52:35.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06c0003c10 con 0x7f06d81054f0 2024-02-28T23:52:35.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d819bff0 con 0x7f06d81054f0 2024-02-28T23:52:35.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.742+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c003e460 con 0x7f06d81054f0 2024-02-28T23:52:35.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.743+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d819c550 con 0x7f06d81054f0 2024-02-28T23:52:35.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.744+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d804e000 con 0x7f06d81054f0 2024-02-28T23:52:35.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.744+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f06c0004fd0 con 0x7f06d81054f0 2024-02-28T23:52:35.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.745+0000 7f06cfe00640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 0x7f06bc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:35.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.746+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f06c00afd80 con 0x7f06d81054f0 2024-02-28T23:52:35.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.746+0000 7f06d7400640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 0x7f06bc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:35.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.746+0000 7f06d7400640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 0x7f06bc06fcc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f06c80058c0 tx=0x7f06c800a4c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:35.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.750+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06c00795b0 con 0x7f06d81054f0 2024-02-28T23:52:35.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.951+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f06d810c7b0 con 0x7f06d81054f0 2024-02-28T23:52:35.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.954+0000 7f06cfe00640 1 -- 172.21.15.57:0/614625843 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f06c00b1020 con 0x7f06d81054f0 2024-02-28T23:52:35.953 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 msgr2=0x7f06bc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 0x7f06bc06fcc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f06c80058c0 tx=0x7f06c800a4c0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 msgr2=0x7f06d8197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009230 tx=0x7f06c0004040 comp rx=0 tx=0).stop 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 shutdown_connections 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f06bc06d810 0x7f06bc06fcc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d81054f0 0x7f06d8197910 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 --2- 172.21.15.57:0/614625843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d80ff4e0 0x7f06d81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.955+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 >> 172.21.15.57:0/614625843 conn(0x7f06d80fafe0 msgr2=0x7f06d80fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.956+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 shutdown_connections 2024-02-28T23:52:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:35.956+0000 7f06dd000640 1 -- 172.21.15.57:0/614625843 wait complete. 2024-02-28T23:52:35.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-28T23:52:36.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 -- 172.21.15.57:0/1090552515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 msgr2=0x7f2b88074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 --2- 172.21.15.57:0/1090552515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b88074b60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009880 tx=0x7f2b7802bce0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 -- 172.21.15.57:0/1090552515 shutdown_connections 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 --2- 172.21.15.57:0/1090552515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 0x7f2b8810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 --2- 172.21.15.57:0/1090552515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b88074b60 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 -- 172.21.15.57:0/1090552515 >> 172.21.15.57:0/1090552515 conn(0x7f2b880fd330 msgr2=0x7f2b880ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 -- 172.21.15.57:0/1090552515 shutdown_connections 2024-02-28T23:52:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.198+0000 7f2b8d800640 1 -- 172.21.15.57:0/1090552515 wait complete. 2024-02-28T23:52:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 Processor -- start 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 -- start start 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 0x7f2b881a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8819b730 con 0x7f2b88074790 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.199+0000 7f2b8d800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8819b8a0 con 0x7f2b880750a0 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41454/0 (socket says 172.21.15.57:41454) 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 -- 172.21.15.57:0/382126576 learned_addr learned my addr 172.21.15.57:0/382126576 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86000640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 0x7f2b881a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 -- 172.21.15.57:0/382126576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 msgr2=0x7f2b881a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 0x7f2b881a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.200+0000 7f2b86a00640 1 -- 172.21.15.57:0/382126576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b78009560 con 0x7f2b88074790 2024-02-28T23:52:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b86a00640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005ab0 tx=0x7f2b7802c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b78004520 con 0x7f2b88074790 2024-02-28T23:52:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b78003ab0 con 0x7f2b88074790 2024-02-28T23:52:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8819bb20 con 0x7f2b88074790 2024-02-28T23:52:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b78005640 con 0x7f2b88074790 2024-02-28T23:52:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.201+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8819c020 con 0x7f2b88074790 2024-02-28T23:52:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.202+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8804e000 con 0x7f2b88074790 2024-02-28T23:52:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.203+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2b78004dd0 con 0x7f2b88074790 2024-02-28T23:52:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.204+0000 7f2b7f400640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 0x7f2b6c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.204+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b780afd20 con 0x7f2b88074790 2024-02-28T23:52:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.204+0000 7f2b86000640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 0x7f2b6c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.205+0000 7f2b86000640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 0x7f2b6c06fcc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2b8819cba0 tx=0x7f2b74005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.209+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b78079430 con 0x7f2b88074790 2024-02-28T23:52:36.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.423+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b8810d670 con 0x7f2b88074790 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.424+0000 7f2b7f400640 1 -- 172.21.15.57:0/382126576 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2424 (secure 0 0 0) 0x7f2b78041990 con 0x7f2b88074790 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6829/2848794823 2024-02-29T23:18:13.072135+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6824/2141462155 2024-02-29T23:18:13.072165+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3238650208 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3210112039 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3095633985 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/448624041 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3095633985 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.424 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/4232995690 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2372541761 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3130089202 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3833211904 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2779519535 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6825/2141462155 2024-02-29T23:18:13.072165+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3528688133 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6829/1810777701 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3178273795 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6826/1292376194 2024-02-29T23:03:57.588940+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/804980878 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/4128072297 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1861782218 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1173946576 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6828/1810777701 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3003168566 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:36.425 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3690186012 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/705907232 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/4070283299 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3145473765 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/935476336 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2764732365 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/935476336 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1150451631 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/1810823735 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6827/1292376194 2024-02-29T23:03:57.588940+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6828/2848794823 2024-02-29T23:18:13.072135+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/382831045 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2764732365 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/915841588 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3145473765 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3833211904 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.426 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/292614806 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.427 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3532446948 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:36.427 INFO:teuthology.orchestra.run.smithi057.stderr:listed 41 entries 2024-02-28T23:52:36.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 msgr2=0x7f2b6c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 0x7f2b6c06fcc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2b8819cba0 tx=0x7f2b74005f90 comp rx=0 tx=0).stop 2024-02-28T23:52:36.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 msgr2=0x7f2b881a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005ab0 tx=0x7f2b7802c7c0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 shutdown_connections 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f2b6c06d810 0x7f2b6c06fcc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b880750a0 0x7f2b881a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 --2- 172.21.15.57:0/382126576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88074790 0x7f2b881a16c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 >> 172.21.15.57:0/382126576 conn(0x7f2b880fd330 msgr2=0x7f2b880fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 shutdown_connections 2024-02-28T23:52:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.427+0000 7f2b8d800640 1 -- 172.21.15.57:0/382126576 wait complete. 2024-02-28T23:52:36.445 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-02-28T23:52:36.445 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-28T23:52:36.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-28T23:52:36.587 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1032107753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:36.587 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:36 smithi057 ceph-mon[89491]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:36.587 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:36 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/614625843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 -- 172.21.15.57:0/2337916893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32541006a0 msgr2=0x7f3254100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 --2- 172.21.15.57:0/2337916893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32541006a0 0x7f3254100a70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f32480098b0 tx=0x7f324802bcd0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 -- 172.21.15.57:0/2337916893 shutdown_connections 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 --2- 172.21.15.57:0/2337916893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3254100fb0 0x7f32541094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 --2- 172.21.15.57:0/2337916893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32541006a0 0x7f3254100a70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.730+0000 7f325a800640 1 -- 172.21.15.57:0/2337916893 >> 172.21.15.57:0/2337916893 conn(0x7f32540f9eb0 msgr2=0x7f32540fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.731+0000 7f325a800640 1 -- 172.21.15.57:0/2337916893 shutdown_connections 2024-02-28T23:52:36.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.731+0000 7f325a800640 1 -- 172.21.15.57:0/2337916893 wait complete. 2024-02-28T23:52:36.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 Processor -- start 2024-02-28T23:52:36.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 -- start start 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32541006a0 0x7f3254197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254197fe0 con 0x7f3254100fb0 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.732+0000 7f325a800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325419bdb0 con 0x7f32541006a0 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41478/0 (socket says 172.21.15.57:41478) 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 -- 172.21.15.57:0/681434254 learned_addr learned my addr 172.21.15.57:0/681434254 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3259400640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32541006a0 0x7f3254197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 -- 172.21.15.57:0/681434254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32541006a0 msgr2=0x7f3254197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32541006a0 0x7f3254197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 -- 172.21.15.57:0/681434254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248009560 con 0x7f3254100fb0 2024-02-28T23:52:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.733+0000 7f3258a00640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f324000b420 tx=0x7f324000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.734+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3240011800 con 0x7f3254100fb0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.734+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3240011e20 con 0x7f3254100fb0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.734+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325419c090 con 0x7f3254100fb0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.734+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324000f2e0 con 0x7f3254100fb0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.734+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325419c5f0 con 0x7f3254100fb0 2024-02-28T23:52:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.735+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325404e850 con 0x7f3254100fb0 2024-02-28T23:52:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.737+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3240010380 con 0x7f3254100fb0 2024-02-28T23:52:36.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.739+0000 7f3252000640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 0x7f323c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:36.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.739+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f324008ce60 con 0x7f3254100fb0 2024-02-28T23:52:36.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.739+0000 7f3259400640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 0x7f323c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:36.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.740+0000 7f3259400640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 0x7f323c06fcc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3248036040 tx=0x7f3248005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:36.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.744+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32400565f0 con 0x7f3254100fb0 2024-02-28T23:52:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1032107753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:36 smithi084 ceph-mon[70440]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:36.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:36 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/614625843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.958+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f325410c730 con 0x7f3254100fb0 2024-02-28T23:52:36.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.963+0000 7f3252000640 1 -- 172.21.15.57:0/681434254 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f3240059cb0 con 0x7f3254100fb0 2024-02-28T23:52:36.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-02-28T23:52:36.963 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":87,"fsid":"3640fe00-d68c-11ee-95c3-87774f69a715","created":"2024-02-28T22:54:23.585293+0000","modified":"2024-02-28T23:34:24.949532+0000","last_up_change":"2024-02-28T23:18:05.087684+0000","last_in_change":"2024-02-28T23:01:39.963913+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-28T23:00:25.160671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-28T23:03:37.677738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-28T23:03:38.275939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"515f39d8-af53-46ca-a205-a49b706e5a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6803","nonce":1424360026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6805","nonce":1424360026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6809","nonce":1424360026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1424360026},{"type":"v1","addr":"172.21.15.57:6807","nonce":1424360026}]},"public_addr":"172.21.15.57:6803/1424360026","cluster_addr":"172.21.15.57:6805/1424360026","heartbeat_back_addr":"172.21.15.57:6809/1424360026","heartbeat_front_addr":"172.21.15.57:6807/1424360026","state":["exists","up"]},{"osd":1,"uuid":"4b6e8b7a-a0f2-4b8d-84f2-ebe895daa7d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6811","nonce":3847484969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6813","nonce":3847484969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6817","nonce":3847484969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":3847484969},{"type":"v1","addr":"172.21.15.57:6815","nonce":3847484969}]},"public_addr":"172.21.15.57:6811/3847484969","cluster_addr":"172.21.15.57:6813/3847484969","heartbeat_back_addr":"172.21.15.57:6817/3847484969","heartbeat_front_addr":"172.21.15.57:6815/3847484969","state":["exists","up"]},{"osd":2,"uuid":"a4e7051f-4c2a-482b-adef-90640ca366bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6819","nonce":3954650626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6821","nonce":3954650626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6825","nonce":3954650626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3954650626},{"type":"v1","addr":"172.21.15.57:6823","nonce":3954650626}]},"public_addr":"172.21.15.57:6819/3954650626","cluster_addr":"172.21.15.57:6821/3954650626","heartbeat_back_addr":"172.21.15.57:6825/3954650626","heartbeat_front_addr":"172.21.15.57:6823/3954650626","state":["exists","up"]},{"osd":3,"uuid":"14bf7f3a-d281-4d38-9e96-1f7a76d3d526","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6801","nonce":2190333848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6803","nonce":2190333848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6807","nonce":2190333848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2190333848},{"type":"v1","addr":"172.21.15.84:6805","nonce":2190333848}]},"public_addr":"172.21.15.84:6801/2190333848","cluster_addr":"172.21.15.84:6803/2190333848","heartbeat_back_addr":"172.21.15.84:6807/2190333848","heartbeat_front_addr":"172.21.15.84:6805/2190333848","state":["exists","up"]},{"osd":4,"uuid":"9fbb5108-2bfb-45d1-b31f-affe0d662fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6809","nonce":890451486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6811","nonce":890451486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6815","nonce":890451486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":890451486},{"type":"v1","addr":"172.21.15.84:6813","nonce":890451486}]},"public_addr":"172.21.15.84:6809/890451486","cluster_addr":"172.21.15.84:6811/890451486","heartbeat_back_addr":"172.21.15.84:6815/890451486","heartbeat_front_addr":"172.21.15.84:6813/890451486","state":["exists","up"]},{"osd":5,"uuid":"8bd03e97-0647-4187-b440-1e175d92b72e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6817","nonce":2020554677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6819","nonce":2020554677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6823","nonce":2020554677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2020554677},{"type":"v1","addr":"172.21.15.84:6821","nonce":2020554677}]},"public_addr":"172.21.15.84:6817/2020554677","cluster_addr":"172.21.15.84:6819/2020554677","heartbeat_back_addr":"172.21.15.84:6823/2020554677","heartbeat_front_addr":"172.21.15.84:6821/2020554677","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:05.406933+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T22:59:42.975601+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:20.940930+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:00:54.452374+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:01:28.137978+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-28T23:02:02.313491+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6829/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.84:6824/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.57:0/3238650208":"2024-02-29T22:55:48.976557+0000","172.21.15.84:0/3210112039":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6801/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/448624041":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6800/3095633985":"2024-02-29T23:06:53.070970+0000","172.21.15.84:0/4232995690":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/2372541761":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/3130089202":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/2779519535":"2024-02-29T22:55:48.976557+0000","172.21.15.84:6825/2141462155":"2024-02-29T23:18:13.072165+0000","172.21.15.84:0/3528688133":"2024-02-29T23:08:14.355809+0000","172.21.15.84:6829/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.84:0/3178273795":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6826/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:0/804980878":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/4128072297":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/1861782218":"2024-02-29T22:55:05.919878+0000","172.21.15.57:0/1173946576":"2024-02-29T22:57:22.445671+0000","172.21.15.84:6828/1810777701":"2024-02-29T23:08:14.355809+0000","172.21.15.57:0/3003168566":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/3690186012":"2024-02-29T23:06:53.070970+0000","172.21.15.57:0/705907232":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/4070283299":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:6801/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:6801/935476336":"2024-02-29T22:55:48.976557+0000","172.21.15.57:0/1150451631":"2024-02-29T22:55:05.919878+0000","172.21.15.84:0/1810823735":"2024-02-29T23:08:14.355809+0000","172.21.15.57:6827/1292376194":"2024-02-29T23:03:57.588940+0000","172.21.15.57:6828/2848794823":"2024-02-29T23:18:13.072135+0000","172.21.15.57:0/382831045":"2024-02-29T23:06:53.070970+0000","172.21.15.57:6800/2764732365":"2024-02-29T22:57:22.445671+0000","172.21.15.57:0/915841588":"2024-02-29T23:12:47.945548+0000","172.21.15.57:6801/3145473765":"2024-02-29T22:55:05.919878+0000","172.21.15.57:6800/3833211904":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/292614806":"2024-02-29T23:12:47.945548+0000","172.21.15.57:0/3532446948":"2024-02-29T23:12:47.945548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-28T23:52:36.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 msgr2=0x7f323c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 0x7f323c06fcc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3248036040 tx=0x7f3248005fd0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 msgr2=0x7f3254197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:36.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f324000b420 tx=0x7f324000b8e0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 shutdown_connections 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f323c06d810 0x7f323c06fcc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254100fb0 0x7f3254197950 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 --2- 172.21.15.57:0/681434254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32541006a0 0x7f3254197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 >> 172.21.15.57:0/681434254 conn(0x7f32540f9eb0 msgr2=0x7f32540feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 shutdown_connections 2024-02-28T23:52:36.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:36.966+0000 7f325a800640 1 -- 172.21.15.57:0/681434254 wait complete. 2024-02-28T23:52:36.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-28T23:00:25.160671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-28T23:03:37.677738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-28T23:03:38.275939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-28T23:52:36.984 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-28T23:52:37.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 -- 172.21.15.57:0/4051502442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458101720 msgr2=0x7f1458101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 --2- 172.21.15.57:0/4051502442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458101720 0x7f1458101b90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1444009880 tx=0x7f144402bcf0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 -- 172.21.15.57:0/4051502442 shutdown_connections 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 --2- 172.21.15.57:0/4051502442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458101720 0x7f1458101b90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 --2- 172.21.15.57:0/4051502442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458107730 0x7f1458107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 -- 172.21.15.57:0/4051502442 >> 172.21.15.57:0/4051502442 conn(0x7f14580fd220 msgr2=0x7f14580ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 -- 172.21.15.57:0/4051502442 shutdown_connections 2024-02-28T23:52:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.206+0000 7f145da00640 1 -- 172.21.15.57:0/4051502442 wait complete. 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 Processor -- start 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 -- start start 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458101720 0x7f14581972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458197ec0 con 0x7f1458107730 2024-02-28T23:52:37.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.207+0000 7f145da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145819bc90 con 0x7f1458101720 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.208+0000 7f1457400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.208+0000 7f1457e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458101720 0x7f14581972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.208+0000 7f1457400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41498/0 (socket says 172.21.15.57:41498) 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.208+0000 7f1457400640 1 -- 172.21.15.57:0/1425127479 learned_addr learned my addr 172.21.15.57:0/1425127479 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1457400640 1 -- 172.21.15.57:0/1425127479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458101720 msgr2=0x7f14581972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1457400640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458101720 0x7f14581972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1457400640 1 -- 172.21.15.57:0/1425127479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444009560 con 0x7f1458107730 2024-02-28T23:52:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1457400640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1444036040 tx=0x7f144402c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1444004520 con 0x7f1458107730 2024-02-28T23:52:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1444003ab0 con 0x7f1458107730 2024-02-28T23:52:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.209+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1444034440 con 0x7f1458107730 2024-02-28T23:52:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.210+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145819bf10 con 0x7f1458107730 2024-02-28T23:52:37.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.210+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145819c4a0 con 0x7f1458107730 2024-02-28T23:52:37.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.211+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145804e000 con 0x7f1458107730 2024-02-28T23:52:37.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.212+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1444003c20 con 0x7f1458107730 2024-02-28T23:52:37.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.213+0000 7f1454c00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 0x7f143c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.213+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f14440af830 con 0x7f1458107730 2024-02-28T23:52:37.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.213+0000 7f1457e00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 0x7f143c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.214+0000 7f1457e00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 0x7f143c06fcc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1458102860 tx=0x7f1448009670 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:37.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.217+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1444078ff0 con 0x7f1458107730 2024-02-28T23:52:37.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.423+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f145810ea30 con 0x7f1458107730 2024-02-28T23:52:37.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.424+0000 7f1454c00640 1 -- 172.21.15.57:0/1425127479 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f144407c6b0 con 0x7f1458107730 2024-02-28T23:52:37.424 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-02-28T23:52:37.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 msgr2=0x7f143c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 0x7f143c06fcc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1458102860 tx=0x7f1448009670 comp rx=0 tx=0).stop 2024-02-28T23:52:37.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 msgr2=0x7f1458197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1444036040 tx=0x7f144402c840 comp rx=0 tx=0).stop 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 shutdown_connections 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f143c06d810 0x7f143c06fcc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1458107730 0x7f1458197830 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 --2- 172.21.15.57:0/1425127479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1458101720 0x7f14581972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.427+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 >> 172.21.15.57:0/1425127479 conn(0x7f14580fd220 msgr2=0x7f14580fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.428+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 shutdown_connections 2024-02-28T23:52:37.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.428+0000 7f145da00640 1 -- 172.21.15.57:0/1425127479 wait complete. 2024-02-28T23:52:37.447 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-28T23:52:37.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 -- 172.21.15.57:0/4119241271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078107730 msgr2=0x7f1078107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 --2- 172.21.15.57:0/4119241271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078107730 0x7f1078107b00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1068009c20 tx=0x7f106802c1b0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 -- 172.21.15.57:0/4119241271 shutdown_connections 2024-02-28T23:52:37.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 --2- 172.21.15.57:0/4119241271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078101720 0x7f1078101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 --2- 172.21.15.57:0/4119241271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078107730 0x7f1078107b00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 -- 172.21.15.57:0/4119241271 >> 172.21.15.57:0/4119241271 conn(0x7f10780fd220 msgr2=0x7f10780ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:37.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 -- 172.21.15.57:0/4119241271 shutdown_connections 2024-02-28T23:52:37.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.676+0000 7f1077e00640 1 -- 172.21.15.57:0/4119241271 wait complete. 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 Processor -- start 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 -- start start 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 0x7f10781978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078197ea0 con 0x7f1078101720 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.677+0000 7f1077e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078198010 con 0x7f1078107730 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41520/0 (socket says 172.21.15.57:41520) 2024-02-28T23:52:37.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 -- 172.21.15.57:0/1038943853 learned_addr learned my addr 172.21.15.57:0/1038943853 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076000640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 0x7f10781978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 -- 172.21.15.57:0/1038943853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 msgr2=0x7f10781978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 0x7f10781978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076a00640 1 -- 172.21.15.57:0/1038943853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009900 con 0x7f1078101720 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.678+0000 7f1076000640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 0x7f10781978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f1076a00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1068032b60 tx=0x7f1068032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1068034820 con 0x7f1078101720 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1068003e30 con 0x7f1078101720 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107819be40 con 0x7f1078101720 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f106803f440 con 0x7f1078101720 2024-02-28T23:52:37.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.679+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107819c340 con 0x7f1078101720 2024-02-28T23:52:37.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.680+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107804e000 con 0x7f1078101720 2024-02-28T23:52:37.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.681+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f106803b040 con 0x7f1078101720 2024-02-28T23:52:37.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.682+0000 7f107ca00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 0x7f105c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:37.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.682+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f10680b0260 con 0x7f1078101720 2024-02-28T23:52:37.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.682+0000 7f1076000640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 0x7f105c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:37.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.683+0000 7f1076000640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 0x7f105c06fcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1078198970 tx=0x7f1060005d90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:37.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.687+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f10680799f0 con 0x7f1078101720 2024-02-28T23:52:37.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/382126576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-28T23:52:37.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:37 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/681434254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:37.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/382126576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-28T23:52:37.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:37 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/681434254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-28T23:52:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.887+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1078109c00 con 0x7f1078101720 2024-02-28T23:52:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.888+0000 7f107ca00640 1 -- 172.21.15.57:0/1038943853 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f106807d0b0 con 0x7f1078101720 2024-02-28T23:52:37.887 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.890+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 msgr2=0x7f105c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.890+0000 7f1077e00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 0x7f105c06fcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1078198970 tx=0x7f1060005d90 comp rx=0 tx=0).stop 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.890+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 msgr2=0x7f1078197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.890+0000 7f1077e00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1068032b60 tx=0x7f1068032b90 comp rx=0 tx=0).stop 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 shutdown_connections 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7f105c06d810 0x7f105c06fcc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1078107730 0x7f10781978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 --2- 172.21.15.57:0/1038943853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1078101720 0x7f1078197360 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 >> 172.21.15.57:0/1038943853 conn(0x7f10780fd220 msgr2=0x7f10780fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 shutdown_connections 2024-02-28T23:52:37.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:37.891+0000 7f1077e00640 1 -- 172.21.15.57:0/1038943853 wait complete. 2024-02-28T23:52:37.910 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-28T23:52:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 -- 172.21.15.57:0/2592564580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072f10 msgr2=0x7fe07810ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 --2- 172.21.15.57:0/2592564580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072f10 0x7fe07810ffc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe064009880 tx=0x7fe06402bd80 comp rx=0 tx=0).stop 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 -- 172.21.15.57:0/2592564580 shutdown_connections 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 --2- 172.21.15.57:0/2592564580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072f10 0x7fe07810ffc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 --2- 172.21.15.57:0/2592564580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072600 0x7fe0780729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 -- 172.21.15.57:0/2592564580 >> 172.21.15.57:0/2592564580 conn(0x7fe0780faff0 msgr2=0x7fe0780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 -- 172.21.15.57:0/2592564580 shutdown_connections 2024-02-28T23:52:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.136+0000 7fe07ec00640 1 -- 172.21.15.57:0/2592564580 wait complete. 2024-02-28T23:52:38.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.137+0000 7fe07ec00640 1 Processor -- start 2024-02-28T23:52:38.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07ec00640 1 -- start start 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07ec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 0x7fe0781a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07ec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781a1fa0 con 0x7fe078072600 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07819b530 con 0x7fe078072f10 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41532/0 (socket says 172.21.15.57:41532) 2024-02-28T23:52:38.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.138+0000 7fe07d800640 1 -- 172.21.15.57:0/1487445633 learned_addr learned my addr 172.21.15.57:0/1487445633 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07ce00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 0x7fe0781a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07d800640 1 -- 172.21.15.57:0/1487445633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 msgr2=0x7fe0781a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07d800640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 0x7fe0781a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07d800640 1 -- 172.21.15.57:0/1487445633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064009560 con 0x7fe078072600 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07ce00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 0x7fe0781a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-28T23:52:38.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07d800640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c8a0 tx=0x7fe06c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c007b80 con 0x7fe078072600 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe06c007ce0 con 0x7fe078072600 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c018810 con 0x7fe078072600 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.139+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07819b810 con 0x7fe078072600 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.140+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07819bd70 con 0x7fe078072600 2024-02-28T23:52:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.141+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07804e000 con 0x7fe078072600 2024-02-28T23:52:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.142+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe06c018a80 con 0x7fe078072600 2024-02-28T23:52:38.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.143+0000 7fe076000640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 0x7fe06006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.143+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe06c08d810 con 0x7fe078072600 2024-02-28T23:52:38.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.143+0000 7fe07ce00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 0x7fe06006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.144+0000 7fe07ce00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 0x7fe06006fcc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe06402c940 tx=0x7fe06402cc60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:38.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.147+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe06c056fd0 con 0x7fe078072600 2024-02-28T23:52:38.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.353+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe07810d6b0 con 0x7fe078072600 2024-02-28T23:52:38.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.356+0000 7fe076000640 1 -- 172.21.15.57:0/1487445633 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe06c05a690 con 0x7fe078072600 2024-02-28T23:52:38.356 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.359+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 msgr2=0x7fe06006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.359+0000 7fe07ec00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 0x7fe06006fcc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe06402c940 tx=0x7fe06402cc60 comp rx=0 tx=0).stop 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.359+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 msgr2=0x7fe0781a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.359+0000 7fe07ec00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c8a0 tx=0x7fe06c00cd60 comp rx=0 tx=0).stop 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 shutdown_connections 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7fe06006d810 0x7fe06006fcc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe078072f10 0x7fe0781a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 --2- 172.21.15.57:0/1487445633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe078072600 0x7fe0781a1460 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 >> 172.21.15.57:0/1487445633 conn(0x7fe0780faff0 msgr2=0x7fe078101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 shutdown_connections 2024-02-28T23:52:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.360+0000 7fe07ec00640 1 -- 172.21.15.57:0/1487445633 wait complete. 2024-02-28T23:52:38.378 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 -- 172.21.15.57:0/2597463119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072580 msgr2=0x7ff804072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 --2- 172.21.15.57:0/2597463119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072580 0x7ff804072950 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009c20 tx=0x7ff7f402c230 comp rx=0 tx=0).stop 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 -- 172.21.15.57:0/2597463119 shutdown_connections 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 --2- 172.21.15.57:0/2597463119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072e90 0x7ff80410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 --2- 172.21.15.57:0/2597463119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072580 0x7ff804072950 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 -- 172.21.15.57:0/2597463119 >> 172.21.15.57:0/2597463119 conn(0x7ff8040fafb0 msgr2=0x7ff8040fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 -- 172.21.15.57:0/2597463119 shutdown_connections 2024-02-28T23:52:38.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.602+0000 7ff809c00640 1 -- 172.21.15.57:0/2597463119 wait complete. 2024-02-28T23:52:38.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.603+0000 7ff809c00640 1 Processor -- start 2024-02-28T23:52:38.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.603+0000 7ff809c00640 1 -- start start 2024-02-28T23:52:38.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff809c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072580 0x7ff8041973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff809c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff809c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804197f80 con 0x7ff804072e90 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff809c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80419bd00 con 0x7ff804072580 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41556/0 (socket says 172.21.15.57:41556) 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff7fb400640 1 -- 172.21.15.57:0/785243968 learned_addr learned my addr 172.21.15.57:0/785243968 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-02-28T23:52:38.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.604+0000 7ff803e00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072580 0x7ff8041973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff7fb400640 1 -- 172.21.15.57:0/785243968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072580 msgr2=0x7ff8041973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff7fb400640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072580 0x7ff8041973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff7fb400640 1 -- 172.21.15.57:0/785243968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4009900 con 0x7ff804072e90 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff7fb400640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00b420 tx=0x7ff7ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7ec011800 con 0x7ff804072e90 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7ec011e20 con 0x7ff804072e90 2024-02-28T23:52:38.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80419bfe0 con 0x7ff804072e90 2024-02-28T23:52:38.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7ec00f2e0 con 0x7ff804072e90 2024-02-28T23:52:38.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.605+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80419c540 con 0x7ff804072e90 2024-02-28T23:52:38.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.607+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80404e850 con 0x7ff804072e90 2024-02-28T23:52:38.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.608+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 39) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff7ec010380 con 0x7ff804072e90 2024-02-28T23:52:38.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.608+0000 7ff801600640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 0x7ff7e806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-28T23:52:38.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.609+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff7ec08cf50 con 0x7ff804072e90 2024-02-28T23:52:38.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.609+0000 7ff803e00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 0x7ff7e806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-28T23:52:38.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.610+0000 7ff803e00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 0x7ff7e806fd80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4000f80 tx=0x7ff7f4034eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-28T23:52:38.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.614+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7ec056660 con 0x7ff804072e90 2024-02-28T23:52:38.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:38 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1425127479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:38.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:38 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1038943853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:38.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:38 smithi057 ceph-mon[89491]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:38.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:38 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1425127479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-28T23:52:38.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:38 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1038943853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-28T23:52:38.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:38 smithi084 ceph-mon[70440]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:38.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.834+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff80410e8e0 con 0x7ff804072e90 2024-02-28T23:52:38.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.835+0000 7ff801600640 1 -- 172.21.15.57:0/785243968 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2424 (secure 0 0 0) 0x7ff7ec059d20 con 0x7ff804072e90 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6829/2848794823 2024-02-29T23:18:13.072135+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6824/2141462155 2024-02-29T23:18:13.072165+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3238650208 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3210112039 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3095633985 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/448624041 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3095633985 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.835 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/4232995690 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2372541761 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3130089202 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3833211904 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2779519535 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6825/2141462155 2024-02-29T23:18:13.072165+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3528688133 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6829/1810777701 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/3178273795 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6826/1292376194 2024-02-29T23:03:57.588940+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/804980878 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/4128072297 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.836 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1861782218 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1173946576 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:6828/1810777701 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3003168566 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3690186012 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/705907232 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/4070283299 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3145473765 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/935476336 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2764732365 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/935476336 2024-02-29T22:55:48.976557+0000 2024-02-28T23:52:38.837 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1150451631 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.84:0/1810823735 2024-02-29T23:08:14.355809+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6827/1292376194 2024-02-29T23:03:57.588940+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6828/2848794823 2024-02-29T23:18:13.072135+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/382831045 2024-02-29T23:06:53.070970+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2764732365 2024-02-29T22:57:22.445671+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/915841588 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3145473765 2024-02-29T22:55:05.919878+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3833211904 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/292614806 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.838 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3532446948 2024-02-29T23:12:47.945548+0000 2024-02-28T23:52:38.839 INFO:teuthology.orchestra.run.smithi057.stderr:listed 41 entries 2024-02-28T23:52:38.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 msgr2=0x7ff7e806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 0x7ff7e806fd80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4000f80 tx=0x7ff7f4034eb0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 msgr2=0x7ff8041978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00b420 tx=0x7ff7ec00b8e0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 shutdown_connections 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:6800/667025161,v1:172.21.15.57:6801/667025161] conn(0x7ff7e806d8d0 0x7ff7e806fd80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff804072e90 0x7ff8041978f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 --2- 172.21.15.57:0/785243968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff804072580 0x7ff8041973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 >> 172.21.15.57:0/785243968 conn(0x7ff8040fafb0 msgr2=0x7ff804106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.838+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 shutdown_connections 2024-02-28T23:52:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-02-28T23:52:38.839+0000 7ff809c00640 1 -- 172.21.15.57:0/785243968 wait complete. 2024-02-28T23:52:38.856 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2024-02-28T23:52:38.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-28T23:52:38.856 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-28T23:52:38.960 INFO:teuthology.orchestra.run:waiting for 300 2024-02-28T23:52:39.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:39 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/1487445633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:39.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:39 smithi057 ceph-mon[89491]: from='client.? 172.21.15.57:0/785243968' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-28T23:52:39.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:39 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/1487445633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-28T23:52:39.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:39 smithi084 ceph-mon[70440]: from='client.? 172.21.15.57:0/785243968' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-28T23:52:40.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:40 smithi057 ceph-mon[89491]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:40.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:40 smithi084 ceph-mon[70440]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:43.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:42 smithi057 ceph-mon[89491]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:43.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:42 smithi084 ceph-mon[70440]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:45.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:45 smithi057 ceph-mon[89491]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:45.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:45 smithi084 ceph-mon[70440]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:47.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:47 smithi057 ceph-mon[89491]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:47.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:47 smithi084 ceph-mon[70440]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:49 smithi057 ceph-mon[89491]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:49.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:49 smithi084 ceph-mon[70440]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:52:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:51 smithi057 ceph-mon[89491]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:51 smithi084 ceph-mon[70440]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:52:53.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:53 smithi057 ceph-mon[89491]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:53.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:53 smithi084 ceph-mon[70440]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:52:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:55 smithi057 ceph-mon[89491]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:55.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:55 smithi084 ceph-mon[70440]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:52:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:57 smithi057 ceph-mon[89491]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:57.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:57 smithi084 ceph-mon[70440]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:52:59.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:52:59 smithi057 ceph-mon[89491]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:52:59.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:52:59 smithi084 ceph-mon[70440]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:01.266 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:01 smithi057 ceph-mon[89491]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:01.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:01 smithi084 ceph-mon[70440]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:03.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:03 smithi057 ceph-mon[89491]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:03 smithi084 ceph-mon[70440]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:04.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:04.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:05 smithi057 ceph-mon[89491]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:05.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:05 smithi084 ceph-mon[70440]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:07.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:07 smithi057 ceph-mon[89491]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:07.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:07 smithi084 ceph-mon[70440]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:09.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:09 smithi057 ceph-mon[89491]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:09 smithi084 ceph-mon[70440]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:11 smithi057 ceph-mon[89491]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:11 smithi084 ceph-mon[70440]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:13 smithi057 ceph-mon[89491]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:13.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:13 smithi084 ceph-mon[70440]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:15 smithi057 ceph-mon[89491]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:15.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:15 smithi084 ceph-mon[70440]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:17 smithi057 ceph-mon[89491]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:17 smithi084 ceph-mon[70440]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:19 smithi057 ceph-mon[89491]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:19 smithi084 ceph-mon[70440]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:21 smithi057 ceph-mon[89491]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:21 smithi084 ceph-mon[70440]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:23 smithi057 ceph-mon[89491]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:23 smithi084 ceph-mon[70440]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:24.330 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:24 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:53:24.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:24 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:53:25.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:25 smithi057 ceph-mon[89491]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:25.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:25 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:53:25.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:25 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:53:25.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:25 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:53:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:25 smithi084 ceph-mon[70440]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:25 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:53:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:25 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:53:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:25 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:53:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:27 smithi057 ceph-mon[89491]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:27 smithi084 ceph-mon[70440]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:29.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:29 smithi057 ceph-mon[89491]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:29 smithi084 ceph-mon[70440]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:31 smithi057 ceph-mon[89491]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:31 smithi084 ceph-mon[70440]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:33 smithi057 ceph-mon[89491]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:33.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:33 smithi084 ceph-mon[70440]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:35 smithi057 ceph-mon[89491]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:35 smithi084 ceph-mon[70440]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:37 smithi057 ceph-mon[89491]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:37 smithi084 ceph-mon[70440]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:39 smithi057 ceph-mon[89491]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:39.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:39 smithi084 ceph-mon[70440]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:41.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:41 smithi057 ceph-mon[89491]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:41.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:41 smithi084 ceph-mon[70440]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:43 smithi057 ceph-mon[89491]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:43 smithi084 ceph-mon[70440]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:45 smithi057 ceph-mon[89491]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:45 smithi084 ceph-mon[70440]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:46.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:46 smithi084 ceph-mon[70440]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:46.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:46 smithi057 ceph-mon[89491]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:49 smithi057 ceph-mon[89491]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:49.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:49 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:49 smithi084 ceph-mon[70440]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:49.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:49 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:53:51.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:51 smithi057 ceph-mon[89491]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:51.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:51 smithi084 ceph-mon[70440]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:53:53.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:53 smithi057 ceph-mon[89491]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:53.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:53 smithi084 ceph-mon[70440]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:53:55.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:55 smithi057 ceph-mon[89491]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:55.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:55 smithi084 ceph-mon[70440]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:53:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:57 smithi057 ceph-mon[89491]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:57.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:57 smithi084 ceph-mon[70440]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:53:59.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:53:59 smithi057 ceph-mon[89491]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:53:59.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:53:59 smithi084 ceph-mon[70440]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:01.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:01 smithi057 ceph-mon[89491]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:01.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:01 smithi084 ceph-mon[70440]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:03.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:03 smithi084 ceph-mon[70440]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:03.361 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:03 smithi057 ceph-mon[89491]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:04.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:04.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:05 smithi057 ceph-mon[89491]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:05.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:05 smithi084 ceph-mon[70440]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:07 smithi057 ceph-mon[89491]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:07.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:07 smithi084 ceph-mon[70440]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:09.572 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:09 smithi057 ceph-mon[89491]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:09 smithi084 ceph-mon[70440]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:11 smithi057 ceph-mon[89491]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:11.583 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:11 smithi084 ceph-mon[70440]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:13 smithi057 ceph-mon[89491]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:13.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:13 smithi084 ceph-mon[70440]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:15 smithi057 ceph-mon[89491]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:15.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:15 smithi084 ceph-mon[70440]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:17 smithi057 ceph-mon[89491]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:17 smithi084 ceph-mon[70440]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:19 smithi057 ceph-mon[89491]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:19.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:19 smithi084 ceph-mon[70440]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:21 smithi057 ceph-mon[89491]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:21.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:21 smithi084 ceph-mon[70440]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:23 smithi057 ceph-mon[89491]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:23 smithi084 ceph-mon[70440]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:25 smithi057 ceph-mon[89491]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:25 smithi084 ceph-mon[70440]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:26.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:26 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:54:26.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:26 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:54:27.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:27 smithi057 ceph-mon[89491]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:27.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:54:27.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:54:27.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:27 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:54:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:27 smithi084 ceph-mon[70440]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:54:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:54:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:27 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:54:29.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:29 smithi057 ceph-mon[89491]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:29 smithi084 ceph-mon[70440]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:31 smithi057 ceph-mon[89491]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:31 smithi084 ceph-mon[70440]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:33.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:33 smithi057 ceph-mon[89491]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:33.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:33 smithi084 ceph-mon[70440]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:35.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:35 smithi057 ceph-mon[89491]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:35 smithi084 ceph-mon[70440]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:37.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:37 smithi057 ceph-mon[89491]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:37 smithi084 ceph-mon[70440]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:39.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:39 smithi057 ceph-mon[89491]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:39.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:39 smithi084 ceph-mon[70440]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:41.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:41 smithi057 ceph-mon[89491]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:41.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:41 smithi084 ceph-mon[70440]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:43 smithi057 ceph-mon[89491]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:43 smithi084 ceph-mon[70440]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:45 smithi057 ceph-mon[89491]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:45 smithi084 ceph-mon[70440]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:47.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:47 smithi057 ceph-mon[89491]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:47.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:47 smithi084 ceph-mon[70440]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:48.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:54:49.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:49 smithi057 ceph-mon[89491]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:49.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:49 smithi084 ceph-mon[70440]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:51.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:51 smithi057 ceph-mon[89491]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:51.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:51 smithi084 ceph-mon[70440]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:54:53.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:53 smithi057 ceph-mon[89491]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:53.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:53 smithi084 ceph-mon[70440]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:54:54.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:54 smithi057 ceph-mon[89491]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:54.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:54 smithi084 ceph-mon[70440]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:54:57.285 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:57 smithi084 ceph-mon[70440]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:57.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:57 smithi057 ceph-mon[89491]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:54:59.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:54:59 smithi057 ceph-mon[89491]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:54:59.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:54:59 smithi084 ceph-mon[70440]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:01.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:01 smithi057 ceph-mon[89491]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:01.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:01 smithi084 ceph-mon[70440]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:03.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:03 smithi057 ceph-mon[89491]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:03 smithi084 ceph-mon[70440]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:04.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:04 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:04.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:04 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:05.466 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:05 smithi057 ceph-mon[89491]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:05.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:05 smithi084 ceph-mon[70440]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:07 smithi057 ceph-mon[89491]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:07.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:07 smithi084 ceph-mon[70440]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:09.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:09 smithi057 ceph-mon[89491]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:09 smithi084 ceph-mon[70440]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:11 smithi057 ceph-mon[89491]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:11 smithi084 ceph-mon[70440]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:13 smithi057 ceph-mon[89491]: pgmap v1314: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:13.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:13 smithi084 ceph-mon[70440]: pgmap v1314: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:15.070 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:14 smithi057 ceph-mon[89491]: pgmap v1315: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:15.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:14 smithi084 ceph-mon[70440]: pgmap v1315: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:17.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:17 smithi057 ceph-mon[89491]: pgmap v1316: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:17.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:17 smithi084 ceph-mon[70440]: pgmap v1316: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:19 smithi084 ceph-mon[70440]: pgmap v1317: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:19.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:19 smithi057 ceph-mon[89491]: pgmap v1317: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:21.319 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:21 smithi057 ceph-mon[89491]: pgmap v1318: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:21 smithi084 ceph-mon[70440]: pgmap v1318: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:23 smithi057 ceph-mon[89491]: pgmap v1319: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:23 smithi084 ceph-mon[70440]: pgmap v1319: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:25 smithi057 ceph-mon[89491]: pgmap v1320: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:25 smithi084 ceph-mon[70440]: pgmap v1320: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:27.385 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:27 smithi084 ceph-mon[70440]: pgmap v1321: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:27 smithi057 ceph-mon[89491]: pgmap v1321: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:28.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:28 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:55:28.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:28 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:55:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:29 smithi057 ceph-mon[89491]: pgmap v1322: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:55:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:55:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:55:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:29 smithi084 ceph-mon[70440]: pgmap v1322: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:55:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:55:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:55:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:31 smithi057 ceph-mon[89491]: pgmap v1323: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:31 smithi084 ceph-mon[70440]: pgmap v1323: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:33.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:33 smithi057 ceph-mon[89491]: pgmap v1324: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:33 smithi084 ceph-mon[70440]: pgmap v1324: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:35 smithi057 ceph-mon[89491]: pgmap v1325: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:35 smithi084 ceph-mon[70440]: pgmap v1325: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:37 smithi057 ceph-mon[89491]: pgmap v1326: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:37 smithi084 ceph-mon[70440]: pgmap v1326: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:55:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:39 smithi057 ceph-mon[89491]: pgmap v1327: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:39.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:39 smithi084 ceph-mon[70440]: pgmap v1327: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:41.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:41 smithi057 ceph-mon[89491]: pgmap v1328: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:41.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:41 smithi084 ceph-mon[70440]: pgmap v1328: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:55:43.456 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:43 smithi084 ceph-mon[70440]: pgmap v1329: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:43 smithi057 ceph-mon[89491]: pgmap v1329: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:55:45.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:45 smithi057 ceph-mon[89491]: pgmap v1330: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:45.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:45 smithi084 ceph-mon[70440]: pgmap v1330: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:55:47.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:47 smithi057 ceph-mon[89491]: pgmap v1331: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:55:47.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:47 smithi084 ceph-mon[70440]: pgmap v1331: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:55:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:48.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:55:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:49 smithi057 ceph-mon[89491]: pgmap v1332: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:55:49.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:49 smithi084 ceph-mon[70440]: pgmap v1332: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:55:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:51 smithi057 ceph-mon[89491]: pgmap v1333: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:55:51.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:51 smithi084 ceph-mon[70440]: pgmap v1333: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:55:53.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:53 smithi057 ceph-mon[89491]: pgmap v1334: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:55:53.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:53 smithi084 ceph-mon[70440]: pgmap v1334: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:55:55.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:55 smithi057 ceph-mon[89491]: pgmap v1335: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:55:55.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:55 smithi084 ceph-mon[70440]: pgmap v1335: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:55:57.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:57 smithi057 ceph-mon[89491]: pgmap v1336: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:55:57.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:57 smithi084 ceph-mon[70440]: pgmap v1336: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:55:59.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:55:59 smithi057 ceph-mon[89491]: pgmap v1337: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:55:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:55:59 smithi084 ceph-mon[70440]: pgmap v1337: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:01 smithi057 ceph-mon[89491]: pgmap v1338: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:01.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:01 smithi084 ceph-mon[70440]: pgmap v1338: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:03.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:03 smithi057 ceph-mon[89491]: pgmap v1339: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:03.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:03 smithi084 ceph-mon[70440]: pgmap v1339: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:05.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:05 smithi057 ceph-mon[89491]: pgmap v1340: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:05.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:05 smithi084 ceph-mon[70440]: pgmap v1340: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:07.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:07 smithi057 ceph-mon[89491]: pgmap v1341: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:07.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:07 smithi084 ceph-mon[70440]: pgmap v1341: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:09.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:09 smithi057 ceph-mon[89491]: pgmap v1342: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:09.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:09 smithi084 ceph-mon[70440]: pgmap v1342: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:11 smithi057 ceph-mon[89491]: pgmap v1343: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:11 smithi084 ceph-mon[70440]: pgmap v1343: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:13 smithi057 ceph-mon[89491]: pgmap v1344: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:13.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:13 smithi084 ceph-mon[70440]: pgmap v1344: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:14.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:14 smithi057 ceph-mon[89491]: pgmap v1345: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:14.831 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:14 smithi084 ceph-mon[70440]: pgmap v1345: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:17 smithi057 ceph-mon[89491]: pgmap v1346: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:17 smithi084 ceph-mon[70440]: pgmap v1346: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:19 smithi057 ceph-mon[89491]: pgmap v1347: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:19 smithi084 ceph-mon[70440]: pgmap v1347: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:21 smithi057 ceph-mon[89491]: pgmap v1348: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:21 smithi084 ceph-mon[70440]: pgmap v1348: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:23 smithi057 ceph-mon[89491]: pgmap v1349: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:23 smithi084 ceph-mon[70440]: pgmap v1349: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:25.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:25 smithi057 ceph-mon[89491]: pgmap v1350: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:25 smithi084 ceph-mon[70440]: pgmap v1350: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:27.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:27 smithi057 ceph-mon[89491]: pgmap v1351: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:27 smithi084 ceph-mon[70440]: pgmap v1351: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:29 smithi057 ceph-mon[89491]: pgmap v1352: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:29.471 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:29 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:56:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:29 smithi084 ceph-mon[70440]: pgmap v1352: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:29 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:56:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:31 smithi057 ceph-mon[89491]: pgmap v1353: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:56:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:56:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:56:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:31 smithi084 ceph-mon[70440]: pgmap v1353: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:56:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:56:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:56:33.461 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:33 smithi057 ceph-mon[89491]: pgmap v1354: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:33 smithi084 ceph-mon[70440]: pgmap v1354: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:34.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:34 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:34.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:34 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:35.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:35 smithi057 ceph-mon[89491]: pgmap v1355: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:35 smithi084 ceph-mon[70440]: pgmap v1355: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:37 smithi057 ceph-mon[89491]: pgmap v1356: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:37 smithi084 ceph-mon[70440]: pgmap v1356: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:39.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:39 smithi057 ceph-mon[89491]: pgmap v1357: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:39.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:39 smithi084 ceph-mon[70440]: pgmap v1357: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:41.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:41 smithi057 ceph-mon[89491]: pgmap v1358: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:41.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:41 smithi084 ceph-mon[70440]: pgmap v1358: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:43.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:43 smithi057 ceph-mon[89491]: pgmap v1359: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:43.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:43 smithi084 ceph-mon[70440]: pgmap v1359: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:45.474 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:45 smithi084 ceph-mon[70440]: pgmap v1360: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:45.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:45 smithi057 ceph-mon[89491]: pgmap v1360: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:47.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:47 smithi057 ceph-mon[89491]: pgmap v1361: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:47.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:47 smithi084 ceph-mon[70440]: pgmap v1361: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:48.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:48 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:48.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:48 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:56:49.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:49 smithi057 ceph-mon[89491]: pgmap v1362: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:49.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:49 smithi084 ceph-mon[70440]: pgmap v1362: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:51.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:51 smithi057 ceph-mon[89491]: pgmap v1363: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:51.580 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:51 smithi084 ceph-mon[70440]: pgmap v1363: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:56:53.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:53 smithi057 ceph-mon[89491]: pgmap v1364: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:53.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:53 smithi084 ceph-mon[70440]: pgmap v1364: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:56:55.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:55 smithi057 ceph-mon[89491]: pgmap v1365: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:55.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:55 smithi084 ceph-mon[70440]: pgmap v1365: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:56:57.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:57 smithi057 ceph-mon[89491]: pgmap v1366: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:57.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:57 smithi084 ceph-mon[70440]: pgmap v1366: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:56:59.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:56:59 smithi057 ceph-mon[89491]: pgmap v1367: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:56:59.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:56:59 smithi084 ceph-mon[70440]: pgmap v1367: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:57:01.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:01 smithi057 ceph-mon[89491]: pgmap v1368: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:01.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:01 smithi084 ceph-mon[70440]: pgmap v1368: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:03.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:03 smithi057 ceph-mon[89491]: pgmap v1369: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:03.571 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:03 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:03 smithi084 ceph-mon[70440]: pgmap v1369: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:03.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:03 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:05.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:05 smithi057 ceph-mon[89491]: pgmap v1370: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:05.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:05 smithi084 ceph-mon[70440]: pgmap v1370: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:07.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:07 smithi057 ceph-mon[89491]: pgmap v1371: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:57:07.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:07 smithi084 ceph-mon[70440]: pgmap v1371: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:57:08.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:08 smithi084 ceph-mon[70440]: pgmap v1372: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:57:08.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:08 smithi057 ceph-mon[89491]: pgmap v1372: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:57:11.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:11 smithi057 ceph-mon[89491]: pgmap v1373: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:11.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:11 smithi084 ceph-mon[70440]: pgmap v1373: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:13.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:13 smithi057 ceph-mon[89491]: pgmap v1374: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:13.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:13 smithi084 ceph-mon[70440]: pgmap v1374: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:15.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:15 smithi057 ceph-mon[89491]: pgmap v1375: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:15.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:15 smithi084 ceph-mon[70440]: pgmap v1375: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:17.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:17 smithi057 ceph-mon[89491]: pgmap v1376: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:57:17.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:17 smithi084 ceph-mon[70440]: pgmap v1376: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-28T23:57:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:19 smithi057 ceph-mon[89491]: pgmap v1377: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:57:19.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:19 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:19 smithi084 ceph-mon[70440]: pgmap v1377: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-02-28T23:57:19.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:19 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:21.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:21 smithi057 ceph-mon[89491]: pgmap v1378: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:21.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:21 smithi084 ceph-mon[70440]: pgmap v1378: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-28T23:57:23.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:23 smithi057 ceph-mon[89491]: pgmap v1379: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:23.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:23 smithi084 ceph-mon[70440]: pgmap v1379: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-28T23:57:25.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:25 smithi057 ceph-mon[89491]: pgmap v1380: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:25.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:25 smithi084 ceph-mon[70440]: pgmap v1380: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-28T23:57:27.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:27 smithi057 ceph-mon[89491]: pgmap v1381: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:57:27.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:27 smithi084 ceph-mon[70440]: pgmap v1381: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:57:29.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:29 smithi057 ceph-mon[89491]: pgmap v1382: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:57:29.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:29 smithi084 ceph-mon[70440]: pgmap v1382: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-02-28T23:57:31.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:31 smithi057 ceph-mon[89491]: pgmap v1383: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:57:31.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:31 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:57:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:31 smithi084 ceph-mon[70440]: pgmap v1383: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-28T23:57:31.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:31 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-28T23:57:32.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:32 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:57:32.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:32 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:57:32.331 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:32 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:57:32.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:32 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-28T23:57:32.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:32 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-28T23:57:32.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:32 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' 2024-02-28T23:57:33.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:33 smithi057 ceph-mon[89491]: pgmap v1384: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:57:33.570 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:33 smithi057 ceph-mon[89491]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:33 smithi084 ceph-mon[70440]: pgmap v1384: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-28T23:57:33.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:33 smithi084 ceph-mon[70440]: from='mgr.34104 172.21.15.57:0/3693230143' entity='mgr.smithi057.zrrldj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-28T23:57:35.525 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:35 smithi057 ceph-mon[89491]: pgmap v1385: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:57:35.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:35 smithi084 ceph-mon[70440]: pgmap v1385: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-28T23:57:37.569 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:37 smithi057 ceph-mon[89491]: pgmap v1386: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:57:37.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:37 smithi084 ceph-mon[70440]: pgmap v1386: 65 pgs: 65 active+clean; 257 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-28T23:57:38.029 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-28T23:57:38.030 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-28T23:57:38.032 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-28T23:57:38.046 INFO:tasks.cephadm:Teardown begin 2024-02-28T23:57:38.046 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-28T23:57:38.047 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-28T23:57:38.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-28T23:57:38.264 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-28T23:57:38.264 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-28T23:57:38.325 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-28T23:57:38.381 INFO:tasks.cephadm:Stopping all daemons... 2024-02-28T23:57:38.382 INFO:tasks.cephadm.mon.smithi057:Stopping mon.smithi057... 2024-02-28T23:57:38.382 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057 2024-02-28T23:57:38.819 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:38 smithi057 systemd[1]: Stopping Ceph mon.smithi057 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:57:39.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:57:39.063+0000 7f8651000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi057 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:57:39.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057[89487]: 2024-02-28T23:57:39.063+0000 7f8651000700 -1 mon.smithi057@0(leader) e3 *** Got Signal Terminated *** 2024-02-28T23:57:39.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 podman[146734]: 2024-02-28 23:57:39.13697369 +0000 UTC m=+0.313769696 container died f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, GIT_CLEAN=True) 2024-02-28T23:57:39.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 podman[146734]: 2024-02-28 23:57:39.666653575 +0000 UTC m=+0.843449621 container cleanup f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS) 2024-02-28T23:57:39.820 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 bash[146734]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057 2024-02-28T23:57:40.320 INFO:journalctl@ceph.mon.smithi057.smithi057.stdout:Feb 28 23:57:39 smithi057 podman[146759]: 2024-02-28 23:57:39.932685769 +0000 UTC m=+0.777724031 container remove f08f19bb57128d8fe8a520a56dff8b6ea30f3e87d44b78e318b19944cf3459d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi057, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-28T23:57:40.435 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi057.service' 2024-02-28T23:57:40.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:57:40.585 INFO:tasks.cephadm.mon.smithi057:Stopped mon.smithi057 2024-02-28T23:57:40.585 INFO:tasks.cephadm.mon.smithi084:Stopping mon.smithi084... 2024-02-28T23:57:40.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi084 2024-02-28T23:57:41.081 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:40 smithi084 systemd[1]: Stopping Ceph mon.smithi084 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:57:41.799 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:41 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084[70436]: 2024-02-28T23:57:41.403+0000 7f8164600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi084 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:57:41.799 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:41 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084[70436]: 2024-02-28T23:57:41.403+0000 7f8164600700 -1 mon.smithi084@1(peon) e3 *** Got Signal Terminated *** 2024-02-28T23:57:41.800 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:41 smithi084 podman[108414]: 2024-02-28 23:57:41.496065896 +0000 UTC m=+0.483782992 container died b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-02-28T23:57:42.055 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:42 smithi084 podman[108414]: 2024-02-28 23:57:42.035942387 +0000 UTC m=+1.023659535 container cleanup b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-28T23:57:42.055 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:42 smithi084 bash[108414]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084 2024-02-28T23:57:42.581 INFO:journalctl@ceph.mon.smithi084.smithi084.stdout:Feb 28 23:57:42 smithi084 podman[108428]: 2024-02-28 23:57:42.321074261 +0000 UTC m=+0.803760073 container remove b363cce31fa1e51886b41d6e9c155d2236329043eb115451177f6777f7f5178c (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-mon-smithi084, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-02-28T23:57:42.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@mon.smithi084.service' 2024-02-28T23:57:42.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:57:42.903 INFO:tasks.cephadm.mon.smithi084:Stopped mon.smithi084 2024-02-28T23:57:42.903 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-28T23:57:42.903 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.0 2024-02-28T23:57:43.284 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:43 smithi057 systemd[1]: Stopping Ceph osd.0 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:57:43.570 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:43 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:57:43.411+0000 7fbd93c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:57:43.570 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:43 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:57:43.411+0000 7fbd93c00700 -1 osd.0 87 *** Got signal Terminated *** 2024-02-28T23:57:43.570 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:43 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0[101820]: 2024-02-28T23:57:43.411+0000 7fbd93c00700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:57:49.069 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:48 smithi057 podman[146879]: 2024-02-28 23:57:48.806347461 +0000 UTC m=+5.531469601 container died a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222) 2024-02-28T23:57:49.569 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:49 smithi057 podman[146879]: 2024-02-28 23:57:49.22941877 +0000 UTC m=+5.954540906 container cleanup a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-28T23:57:49.570 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:49 smithi057 bash[146879]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0 2024-02-28T23:57:50.069 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:49 smithi057 podman[146897]: 2024-02-28 23:57:49.698050382 +0000 UTC m=+0.880731105 container remove a6b078e69c025fbf47169eb17dd76ec627ab65e25a56bd46453ae14b094777df (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:57:50.847 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:50 smithi057 podman[146987]: 2024-02-28 23:57:50.499059087 +0000 UTC m=+0.039706140 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:57:50.848 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:50 smithi057 podman[146987]: 2024-02-28 23:57:50.63227092 +0000 UTC m=+0.172917993 container create a46cc68f11d17a554a029db3ab52dae57ceb425759b522f2b6fa0749651ea351 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-02-28T23:57:51.321 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:50 smithi057 podman[146987]: 2024-02-28 23:57:50.886933881 +0000 UTC m=+0.427580974 container init a46cc68f11d17a554a029db3ab52dae57ceb425759b522f2b6fa0749651ea351 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-28T23:57:51.321 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:50 smithi057 podman[146987]: 2024-02-28 23:57:50.898550883 +0000 UTC m=+0.439197962 container start a46cc68f11d17a554a029db3ab52dae57ceb425759b522f2b6fa0749651ea351 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:57:51.321 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:50 smithi057 podman[146987]: 2024-02-28 23:57:50.976966971 +0000 UTC m=+0.517614084 container attach a46cc68f11d17a554a029db3ab52dae57ceb425759b522f2b6fa0749651ea351 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:57:51.701 INFO:journalctl@ceph.osd.0.smithi057.stdout:Feb 28 23:57:51 smithi057 podman[146987]: 2024-02-28 23:57:51.355697369 +0000 UTC m=+0.896344439 container died a46cc68f11d17a554a029db3ab52dae57ceb425759b522f2b6fa0749651ea351 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-28T23:57:52.065 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.0.service' 2024-02-28T23:57:52.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:57:52.199 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-28T23:57:52.199 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-28T23:57:52.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.1 2024-02-28T23:57:52.694 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:52 smithi057 systemd[1]: Stopping Ceph osd.1 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:57:53.070 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:57:52.831+0000 7fdca6800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:57:53.070 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:57:52.831+0000 7fdca6800700 -1 osd.1 87 *** Got signal Terminated *** 2024-02-28T23:57:53.070 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:52 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1[107390]: 2024-02-28T23:57:52.831+0000 7fdca6800700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:57:58.790 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:58 smithi057 podman[147092]: 2024-02-28 23:57:58.437252655 +0000 UTC m=+5.728267681 container died 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-28T23:57:59.070 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:58 smithi057 podman[147092]: 2024-02-28 23:57:58.909026112 +0000 UTC m=+6.200041182 container cleanup 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-28T23:57:59.070 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:58 smithi057 bash[147092]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1 2024-02-28T23:57:59.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:59 smithi057 podman[147116]: 2024-02-28 23:57:59.143802255 +0000 UTC m=+0.688811412 container remove 3fc1bae027bf7134237f11a3606f8944e5a0bd7873d58a5f1009c162f838a008 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True) 2024-02-28T23:58:00.250 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:57:59 smithi057 podman[147208]: 2024-02-28 23:57:59.89385213 +0000 UTC m=+0.036735650 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:58:00.250 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:58:00 smithi057 podman[147208]: 2024-02-28 23:58:00.003012069 +0000 UTC m=+0.145895617 container create 74849d56eed26fe7713588f8f72c8d0b55335ab6a49090cfafc1554852631b85 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:58:00.251 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:58:00 smithi057 podman[147208]: 2024-02-28 23:58:00.249853448 +0000 UTC m=+0.392737008 container init 74849d56eed26fe7713588f8f72c8d0b55335ab6a49090cfafc1554852631b85 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-28T23:58:00.569 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:58:00 smithi057 podman[147208]: 2024-02-28 23:58:00.263021308 +0000 UTC m=+0.405904835 container start 74849d56eed26fe7713588f8f72c8d0b55335ab6a49090cfafc1554852631b85 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:00.570 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:58:00 smithi057 podman[147208]: 2024-02-28 23:58:00.389528213 +0000 UTC m=+0.532411811 container attach 74849d56eed26fe7713588f8f72c8d0b55335ab6a49090cfafc1554852631b85 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:58:01.002 INFO:journalctl@ceph.osd.1.smithi057.stdout:Feb 28 23:58:00 smithi057 podman[147208]: 2024-02-28 23:58:00.716228813 +0000 UTC m=+0.859112331 container died 74849d56eed26fe7713588f8f72c8d0b55335ab6a49090cfafc1554852631b85 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-02-28T23:58:01.390 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.1.service' 2024-02-28T23:58:01.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:58:01.503 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-28T23:58:01.503 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-28T23:58:01.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.2 2024-02-28T23:58:02.070 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:01 smithi057 systemd[1]: Stopping Ceph osd.2 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:58:02.570 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:02 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:58:02.104+0000 7fdc80c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:58:02.570 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:02 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:58:02.104+0000 7fdc80c00700 -1 osd.2 87 *** Got signal Terminated *** 2024-02-28T23:58:02.570 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:02 smithi057 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2[113258]: 2024-02-28T23:58:02.104+0000 7fdc80c00700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:58:07.932 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:07 smithi057 podman[147323]: 2024-02-28 23:58:07.670429385 +0000 UTC m=+5.709977371 container died 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-28T23:58:08.282 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:08 smithi057 podman[147323]: 2024-02-28 23:58:08.073403678 +0000 UTC m=+6.112951690 container cleanup 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:58:08.282 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:08 smithi057 bash[147323]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2 2024-02-28T23:58:08.569 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:08 smithi057 podman[147341]: 2024-02-28 23:58:08.28283571 +0000 UTC m=+0.597916294 container remove 642db0860d86fee993484caefe44d928b92adf6069c8cd9068cc9d9d0014347a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-28T23:58:09.447 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.076300176 +0000 UTC m=+0.038009658 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:58:09.447 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.183663257 +0000 UTC m=+0.145372758 container create ebb4764183e2c9a90200a6f871c5fbfe4301f956d3e91e4b19bc075e7f46239d (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-02-28T23:58:09.819 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.445974805 +0000 UTC m=+0.407684322 container init ebb4764183e2c9a90200a6f871c5fbfe4301f956d3e91e4b19bc075e7f46239d (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:58:09.820 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.459761463 +0000 UTC m=+0.421470981 container start ebb4764183e2c9a90200a6f871c5fbfe4301f956d3e91e4b19bc075e7f46239d (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-28T23:58:09.820 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.586041165 +0000 UTC m=+0.547750709 container attach ebb4764183e2c9a90200a6f871c5fbfe4301f956d3e91e4b19bc075e7f46239d (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:10.176 INFO:journalctl@ceph.osd.2.smithi057.stdout:Feb 28 23:58:09 smithi057 podman[147427]: 2024-02-28 23:58:09.917450871 +0000 UTC m=+0.879160389 container died ebb4764183e2c9a90200a6f871c5fbfe4301f956d3e91e4b19bc075e7f46239d (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-28T23:58:10.630 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.2.service' 2024-02-28T23:58:10.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:58:10.759 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-28T23:58:10.760 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-28T23:58:10.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.3 2024-02-28T23:58:11.081 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:10 smithi084 systemd[1]: Stopping Ceph osd.3 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:58:11.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:10 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:10.730+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:11.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:11 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:11.309+0000 7fa061800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:58:11.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:11 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:11.309+0000 7fa061800700 -1 osd.3 87 *** Got signal Terminated *** 2024-02-28T23:58:11.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:11 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:11.309+0000 7fa061800700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:58:11.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:11 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:11.461+0000 7fa058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.830399+0000 front 2024-02-28T23:57:45.830487+0000 (oldest deadline 2024-02-28T23:58:11.130100+0000) 2024-02-28T23:58:12.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:11 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:11.727+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:12.697 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:12 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:12.441+0000 7fa058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.830399+0000 front 2024-02-28T23:57:45.830487+0000 (oldest deadline 2024-02-28T23:58:11.130100+0000) 2024-02-28T23:58:13.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:12 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:12.759+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:13.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:12 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:12.696+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:13.831 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:13 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:13.477+0000 7fa058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.830399+0000 front 2024-02-28T23:57:45.830487+0000 (oldest deadline 2024-02-28T23:58:11.130100+0000) 2024-02-28T23:58:13.831 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:13 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:13.723+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:13.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:13 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:13.686+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:14.762 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:14.466+0000 7fa058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.830399+0000 front 2024-02-28T23:57:45.830487+0000 (oldest deadline 2024-02-28T23:58:11.130100+0000) 2024-02-28T23:58:14.762 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:14.658+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:15.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:14 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:14.761+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:15.809 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:15 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3[79996]: 2024-02-28T23:58:15.466+0000 7fa058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.830399+0000 front 2024-02-28T23:57:45.830487+0000 (oldest deadline 2024-02-28T23:58:11.130100+0000) 2024-02-28T23:58:15.810 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:15 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:15.669+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:16.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:15 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:15.809+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:17.054 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:16 smithi084 podman[108549]: 2024-02-28 23:58:16.767245567 +0000 UTC m=+5.568211849 container died c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, io.buildah.version=1.33.5, RELEASE=main-1fb5d10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:17.055 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:16 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:16.693+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:17.055 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:16 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:16.832+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:17.331 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:17 smithi084 podman[108549]: 2024-02-28 23:58:17.17880978 +0000 UTC m=+5.979776099 container cleanup c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-02-28T23:58:17.331 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:17 smithi084 bash[108549]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3 2024-02-28T23:58:17.644 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:17 smithi084 podman[108569]: 2024-02-28 23:58:17.488616748 +0000 UTC m=+0.712945870 container remove c438b3ff662669c9f8dcfeeed24dc168e6544e70263638a6b333c19b66304046 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-28T23:58:18.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:17 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:17.644+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:18.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:17 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:17.853+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:18.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:18 smithi084 podman[108659]: 2024-02-28 23:58:18.21255629 +0000 UTC m=+0.035747931 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:58:18.581 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:18 smithi084 podman[108659]: 2024-02-28 23:58:18.322627232 +0000 UTC m=+0.145818895 container create ea8c1b8d9b9dabe5d3f1ba7e3d7ccdfaeaed1cf656e6cf2e89621b6e56bf83a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-28T23:58:19.035 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:18 smithi084 podman[108659]: 2024-02-28 23:58:18.584620933 +0000 UTC m=+0.407812603 container init ea8c1b8d9b9dabe5d3f1ba7e3d7ccdfaeaed1cf656e6cf2e89621b6e56bf83a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-02-28T23:58:19.035 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:18 smithi084 podman[108659]: 2024-02-28 23:58:18.59749387 +0000 UTC m=+0.420685563 container start ea8c1b8d9b9dabe5d3f1ba7e3d7ccdfaeaed1cf656e6cf2e89621b6e56bf83a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-28T23:58:19.035 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:18 smithi084 podman[108659]: 2024-02-28 23:58:18.674894575 +0000 UTC m=+0.498086288 container attach ea8c1b8d9b9dabe5d3f1ba7e3d7ccdfaeaed1cf656e6cf2e89621b6e56bf83a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, RELEASE=main-1fb5d10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-28T23:58:19.036 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:18 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:18.685+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:19.036 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:18 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:18.822+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:19.331 INFO:journalctl@ceph.osd.3.smithi084.stdout:Feb 28 23:58:19 smithi084 podman[108659]: 2024-02-28 23:58:19.041749814 +0000 UTC m=+0.864941440 container died ea8c1b8d9b9dabe5d3f1ba7e3d7ccdfaeaed1cf656e6cf2e89621b6e56bf83a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-28T23:58:19.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.3.service' 2024-02-28T23:58:19.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:58:19.839 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-28T23:58:19.839 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-28T23:58:19.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.4 2024-02-28T23:58:19.952 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:19 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:19.636+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:19.952 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:19 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:19.859+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:19.953 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:19 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:19.859+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:20.331 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:19 smithi084 systemd[1]: Stopping Ceph osd.4 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:58:20.676 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:20.422+0000 7f720bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:58:20.676 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:20.422+0000 7f720bc00700 -1 osd.4 87 *** Got signal Terminated *** 2024-02-28T23:58:20.676 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:20.422+0000 7f720bc00700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:58:20.677 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:20.676+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:21.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:20.890+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:21.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:20 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:20.890+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:21.931 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:21 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:21.669+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:22.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:21 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:21.931+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:22.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:21 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:21.931+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:22.961 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:22 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:22.681+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:22.961 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:22 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:22.681+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:56.643858+0000 front 2024-02-28T23:57:56.644167+0000 (oldest deadline 2024-02-28T23:58:21.943682+0000) 2024-02-28T23:58:23.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:22 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:22.961+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:23.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:22 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:22.961+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:24.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:23 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:23.917+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:24.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:23 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:23.917+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:24.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:23 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:23.725+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:24.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:23 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:23.725+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:56.643858+0000 front 2024-02-28T23:57:56.644167+0000 (oldest deadline 2024-02-28T23:58:21.943682+0000) 2024-02-28T23:58:25.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:24 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:24.875+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:25.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:24 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:24.875+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:25.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:24 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:24.754+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:46.741824+0000 front 2024-02-28T23:57:46.742017+0000 (oldest deadline 2024-02-28T23:58:12.641374+0000) 2024-02-28T23:58:25.081 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:24 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4[85354]: 2024-02-28T23:58:24.754+0000 7f7202e00700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:56.643858+0000 front 2024-02-28T23:57:56.644167+0000 (oldest deadline 2024-02-28T23:58:21.943682+0000) 2024-02-28T23:58:26.062 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:25 smithi084 podman[108778]: 2024-02-28 23:58:25.794699928 +0000 UTC m=+5.542437903 container died f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:26.062 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:25 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:25.861+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:26.062 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:25 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:25.861+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:26.331 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:26 smithi084 podman[108778]: 2024-02-28 23:58:26.189625491 +0000 UTC m=+5.937363520 container cleanup f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-28T23:58:26.331 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:26 smithi084 bash[108778]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4 2024-02-28T23:58:26.831 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:26 smithi084 podman[108796]: 2024-02-28 23:58:26.466712222 +0000 UTC m=+0.660070887 container remove f716546fb90ae2fc7fa82228c1e9a6e68af3b0dbe7093b81a8b3811294fedc80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, org.label-schema.license=GPLv2) 2024-02-28T23:58:27.279 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:26 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:26.844+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:27.280 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:26 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:26.845+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:27.581 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:27 smithi084 podman[108883]: 2024-02-28 23:58:27.185391267 +0000 UTC m=+0.035033230 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:58:27.581 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:27 smithi084 podman[108883]: 2024-02-28 23:58:27.300831056 +0000 UTC m=+0.150473057 container create 9a4ee8e7dac30fff11cfbca58f4ffdb96c966430e45cb6d8f7d402dd89aebe2a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-1fb5d10) 2024-02-28T23:58:28.067 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:27 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:27.869+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:28.067 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:27 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:27.869+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:28.067 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:27 smithi084 podman[108883]: 2024-02-28 23:58:27.621337849 +0000 UTC m=+0.470979890 container init 9a4ee8e7dac30fff11cfbca58f4ffdb96c966430e45cb6d8f7d402dd89aebe2a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-28T23:58:28.067 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:27 smithi084 podman[108883]: 2024-02-28 23:58:27.633327455 +0000 UTC m=+0.482969462 container start 9a4ee8e7dac30fff11cfbca58f4ffdb96c966430e45cb6d8f7d402dd89aebe2a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-28T23:58:28.067 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:27 smithi084 podman[108883]: 2024-02-28 23:58:27.711746682 +0000 UTC m=+0.561388730 container attach 9a4ee8e7dac30fff11cfbca58f4ffdb96c966430e45cb6d8f7d402dd89aebe2a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:28.331 INFO:journalctl@ceph.osd.4.smithi084.stdout:Feb 28 23:58:28 smithi084 podman[108883]: 2024-02-28 23:58:28.067768458 +0000 UTC m=+0.917410492 container died 9a4ee8e7dac30fff11cfbca58f4ffdb96c966430e45cb6d8f7d402dd89aebe2a (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-4-deactivate, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:28.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.4.service' 2024-02-28T23:58:28.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:58:28.963 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-28T23:58:28.963 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-28T23:58:28.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.5 2024-02-28T23:58:29.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:28 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:28.824+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:29.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:28 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:28.824+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:29.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:28 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:28.824+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:29.533 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 systemd[1]: Stopping Ceph osd.5 for 3640fe00-d68c-11ee-95c3-87774f69a715... 2024-02-28T23:58:29.831 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.533+0000 7f782da00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-28T23:58:29.831 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.533+0000 7f782da00700 -1 osd.5 87 *** Got signal Terminated *** 2024-02-28T23:58:29.831 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.533+0000 7f782da00700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-28T23:58:30.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.856+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:30.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.856+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:30.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:29 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:29.856+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:31.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:30 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:30.842+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:31.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:30 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:30.842+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:31.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:30 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:30.842+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:32.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:31.860+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:32.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:31.860+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:32.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:31 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:31.860+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:33.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:32 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:32.820+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:33.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:32 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:32.820+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:33.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:32 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:32.820+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:34.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:33 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:33.775+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-02-28T23:57:45.755064+0000 front 2024-02-28T23:57:45.755197+0000 (oldest deadline 2024-02-28T23:58:10.454621+0000) 2024-02-28T23:58:34.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:33 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:33.775+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6814 osd.1 since back 2024-02-28T23:57:53.356091+0000 front 2024-02-28T23:57:53.356023+0000 (oldest deadline 2024-02-28T23:58:19.255759+0000) 2024-02-28T23:58:34.082 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:33 smithi084 ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5[90273]: 2024-02-28T23:58:33.775+0000 7f7829c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.57:6822 osd.2 since back 2024-02-28T23:58:03.957387+0000 front 2024-02-28T23:58:03.957432+0000 (oldest deadline 2024-02-28T23:58:28.056980+0000) 2024-02-28T23:58:35.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:35 smithi084 podman[108990]: 2024-02-28 23:58:35.080638307 +0000 UTC m=+5.676344171 container died bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-1fb5d10) 2024-02-28T23:58:35.785 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:35 smithi084 podman[108990]: 2024-02-28 23:58:35.508876706 +0000 UTC m=+6.104582606 container cleanup bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-28T23:58:35.785 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:35 smithi084 bash[108990]: ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5 2024-02-28T23:58:36.081 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:35 smithi084 podman[109009]: 2024-02-28 23:58:35.784903552 +0000 UTC m=+0.699474064 container remove bcfd5c4a9e3692e920ff50b7c4f2d497f07095244d14a800c1c08ecca3bf82b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-28T23:58:36.990 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:36 smithi084 podman[109097]: 2024-02-28 23:58:36.558671732 +0000 UTC m=+0.032670626 image pull 2272b2c42f9babde922baaabf6efc9b50088cc9d3a128524b9d525dcb1eddc6e quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc 2024-02-28T23:58:36.991 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:36 smithi084 podman[109097]: 2024-02-28 23:58:36.694069433 +0000 UTC m=+0.168068364 container create 3a586b9bd5fee8bc10268525d3fe0b03835f7a5edcfc3f3f6783f3a97a713031 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-28T23:58:36.991 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:36 smithi084 podman[109097]: 2024-02-28 23:58:36.989852948 +0000 UTC m=+0.463851885 container init 3a586b9bd5fee8bc10268525d3fe0b03835f7a5edcfc3f3f6783f3a97a713031 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-1fb5d10, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-28T23:58:37.331 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:37 smithi084 podman[109097]: 2024-02-28 23:58:37.001400507 +0000 UTC m=+0.475399465 container start 3a586b9bd5fee8bc10268525d3fe0b03835f7a5edcfc3f3f6783f3a97a713031 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-1fb5d10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-28T23:58:37.332 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:37 smithi084 podman[109097]: 2024-02-28 23:58:37.096681539 +0000 UTC m=+0.570680479 container attach 3a586b9bd5fee8bc10268525d3fe0b03835f7a5edcfc3f3f6783f3a97a713031 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, RELEASE=main-1fb5d10, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-02-28T23:58:37.786 INFO:journalctl@ceph.osd.5.smithi084.stdout:Feb 28 23:58:37 smithi084 podman[109097]: 2024-02-28 23:58:37.426347496 +0000 UTC m=+0.900346460 container died 3a586b9bd5fee8bc10268525d3fe0b03835f7a5edcfc3f3f6783f3a97a713031 (image=quay.ceph.io/ceph-ci/ceph@sha256:7051beed95d8dfc8c2b1de37206abb8f79d596c447bea39002f2bbaf581330cc, name=ceph-3640fe00-d68c-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-1fb5d10, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-28T23:58:38.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3640fe00-d68c-11ee-95c3-87774f69a715@osd.5.service' 2024-02-28T23:58:38.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-28T23:58:38.301 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-28T23:58:38.302 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-28T23:58:38.602 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:58:45.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[67132]: fuse finished with error 0 and tester_r 0 2024-02-28T23:59:12.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-28T23:59:13.270 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-28T23:59:32.625 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-28T23:59:32.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-28T23:59:32.838 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-28T23:59:32.841 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi057/crash 2024-02-28T23:59:32.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash -- . 2024-02-28T23:59:32.944 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-28T23:59:32.944 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2024-02-28T23:59:32.950 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi084/crash 2024-02-28T23:59:32.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash -- . 2024-02-28T23:59:33.039 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-28T23:59:33.040 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-02-28T23:59:33.043 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-28T23:59:33.043 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-28T23:59:33.151 INFO:teuthology.orchestra.run.smithi057.stdout:2024-02-28T23:40:00.000188+0000 mon.smithi057 (mon.0) 895 : cluster 4 [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-28T23:59:33.152 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-28T23:59:33.152 INFO:tasks.cephadm:Compressing logs... 2024-02-28T23:59:33.153 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-28T23:59:33.156 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-28T23:59:33.230 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-02-28T23:59:33.230 INFO:teuthology.orchestra.run.smithi057.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-28T23:59:33.231 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-02-28T23:59:33.232 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.sbkaml.log 2024-02-28T23:59:33.233 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-28T23:59:33.234 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.sbkaml.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.bgmggd.log 2024-02-28T23:59:33.236 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi057.log 2024-02-28T23:59:33.238 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.bgmggd.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.1.log 2024-02-28T23:59:33.240 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi057.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi057.zrrldj.log 2024-02-28T23:59:33.242 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.1.log: /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi057.zrrldj.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi057.log 2024-02-28T23:59:33.245 INFO:teuthology.orchestra.run.smithi057.stderr: 91.2% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-28T23:59:33.245 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi057.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.0.log 2024-02-28T23:59:33.247 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-28T23:59:33.248 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-02-28T23:59:33.248 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-28T23:59:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi084.log 2024-02-28T23:59:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-28T23:59:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi084.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.4.log 2024-02-28T23:59:33.251 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.0.log: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-28T23:59:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi084.fhkzis.log 2024-02-28T23:59:33.253 INFO:teuthology.orchestra.run.smithi084.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-28T23:59:33.253 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log: 85.7% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-28T23:59:33.253 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log 2024-02-28T23:59:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.5.log 2024-02-28T23:59:33.255 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi084.fhkzis.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.3.log 2024-02-28T23:59:33.256 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.ztfxpc.log 2024-02-28T23:59:33.258 INFO:teuthology.orchestra.run.smithi084.stderr: 91.3% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-28T23:59:33.258 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.dadqfu.log 2024-02-28T23:59:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-28T23:59:33.260 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.2.log 2024-02-28T23:59:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.ztfxpc.log: /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.dadqfu.log: /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log: 85.6% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-28T23:59:33.263 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi084.log 2024-02-28T23:59:33.264 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-28T23:59:33.264 INFO:teuthology.orchestra.run.smithi057.stderr: 89.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-28T23:59:33.266 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi084.log: gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log 2024-02-28T23:59:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.2.log: 94.0% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi057.log.gz 2024-02-28T23:59:33.271 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-28T23:59:33.275 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log: /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-02-28T23:59:33.275 INFO:teuthology.orchestra.run.smithi084.stderr: 94.5% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-client.ceph-exporter.smithi084.log.gz 2024-02-28T23:59:33.280 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.1.log: 89.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-28T23:59:33.298 INFO:teuthology.orchestra.run.smithi084.stderr: 91.3% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-28T23:59:33.307 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log: 91.5% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-28T23:59:33.325 INFO:teuthology.orchestra.run.smithi084.stderr: 89.9% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi084.fhkzis.log.gz 2024-02-28T23:59:34.030 INFO:teuthology.orchestra.run.smithi057.stderr: 89.3% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mgr.smithi057.zrrldj.log.gz 2024-02-28T23:59:34.255 INFO:teuthology.orchestra.run.smithi084.stderr: 92.5% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi084.log.gz 2024-02-28T23:59:36.067 INFO:teuthology.orchestra.run.smithi057.stderr: 89.8% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mon.smithi057.log.gz 2024-02-28T23:59:40.255 INFO:teuthology.orchestra.run.smithi084.stderr: 93.6% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.5.log.gz 2024-02-28T23:59:40.598 INFO:teuthology.orchestra.run.smithi084.stderr: 93.5% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.3.log.gz 2024-02-28T23:59:40.728 INFO:teuthology.orchestra.run.smithi084.stderr: 93.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.4.log.gz 2024-02-28T23:59:41.229 INFO:teuthology.orchestra.run.smithi057.stderr: 93.3% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.2.log.gz 2024-02-28T23:59:41.755 INFO:teuthology.orchestra.run.smithi057.stderr: 93.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.0.log.gz 2024-02-28T23:59:42.596 INFO:teuthology.orchestra.run.smithi057.stderr: 93.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-osd.1.log.gz 2024-02-28T23:59:43.090 INFO:teuthology.orchestra.run.smithi084.stderr: 95.2% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.dadqfu.log.gz 2024-02-28T23:59:43.728 INFO:teuthology.orchestra.run.smithi057.stderr: 95.1% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.bgmggd.log.gz 2024-02-28T23:59:49.205 INFO:teuthology.orchestra.run.smithi057.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-02-28T23:59:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-28T23:59:50.278 INFO:teuthology.orchestra.run.smithi084.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-02-29T00:00:21.677 INFO:teuthology.orchestra.run.smithi084.stderr: 93.5% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi084.ztfxpc.log.gz 2024-02-29T00:00:21.681 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-02-29T00:00:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m48.473s 2024-02-29T00:00:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:user 1m31.833s 2024-02-29T00:00:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m6.406s 2024-02-29T00:00:25.570 INFO:teuthology.orchestra.run.smithi057.stderr: 93.4% -- replaced with /var/log/ceph/3640fe00-d68c-11ee-95c3-87774f69a715/ceph-mds.cephfs.smithi057.sbkaml.log.gz 2024-02-29T00:00:25.576 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-02-29T00:00:25.576 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m52.384s 2024-02-29T00:00:25.576 INFO:teuthology.orchestra.run.smithi057.stderr:user 1m40.939s 2024-02-29T00:00:25.576 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m7.418s 2024-02-29T00:00:25.577 INFO:tasks.cephadm:Archiving logs... 2024-02-29T00:00:25.577 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi057/log 2024-02-29T00:00:25.578 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-29T00:00:38.791 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi084/log 2024-02-29T00:00:38.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-29T00:00:51.347 INFO:tasks.cephadm:Removing cluster... 2024-02-29T00:00:51.348 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 --force 2024-02-29T00:00:51.657 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-29T00:00:52.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3640fe00-d68c-11ee-95c3-87774f69a715 --force 2024-02-29T00:00:52.834 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 3640fe00-d68c-11ee-95c3-87774f69a715 2024-02-29T00:00:53.742 INFO:tasks.cephadm:Removing cephadm ... 2024-02-29T00:00:53.742 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-29T00:00:53.802 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-29T00:00:53.859 INFO:tasks.cephadm:Teardown complete 2024-02-29T00:00:53.859 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-29T00:00:53.934 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-29T00:00:53.935 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-29T00:00:53.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-29T00:00:53.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-29T00:00:54.093 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-29T00:00:54.127 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-29T00:00:54.127 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-29T00:00:54.135 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-29T00:00:54.185 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-02-29T00:00:54.188 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:^- hv01.front.sepia.ceph.com 2 6 377 11 +2220us[+2220us] +/- 157ms 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -1234us[-1221us] +/- 57ms 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +1517us[+1530us] +/- 68ms 2024-02-29T00:00:54.196 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +1882us[+1879us] +/- 157ms 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1577us[-1580us] +/- 57ms 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +1183us[+1179us] +/- 68ms 2024-02-29T00:00:54.198 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-29T00:00:54.199 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-29T00:00:54.446 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-29T00:00:54.447 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-29T00:00:54.505 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-29T00:00:54.772 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-29T00:00:55.003 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-29T00:00:55.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-29T00:00:55.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-29T00:00:55.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-29T00:00:55.221 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-29T00:00:55.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-29T00:00:55.282 INFO:teuthology.task.internal:Duration was 4609.126544 seconds 2024-02-29T00:00:55.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-29T00:00:55.326 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-29T00:00:55.326 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-29T00:00:55.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-29T00:00:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-29T00:00:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-29T00:00:55.912 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-29T00:00:55.912 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2024-02-29T00:00:55.912 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-29T00:00:55.974 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-02-29T00:00:55.974 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-29T00:00:56.044 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-29T00:00:56.044 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-29T00:00:56.049 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-29T00:00:56.139 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-29T00:00:56.147 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-29T00:00:56.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-29T00:00:57.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-29T00:00:57.569 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-29T00:00:57.570 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-29T00:00:57.661 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-29T00:00:57.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-29T00:00:57.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-29T00:00:57.782 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-29T00:00:57.883 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2024-02-29T00:00:57.896 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-02-29T00:00:57.952 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-29T00:00:58.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-29T00:00:58.004 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-29T00:00:58.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-29T00:00:58.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-29T00:00:58.070 INFO:teuthology.task.internal:Transferring archived files... 2024-02-29T00:00:58.071 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi057 2024-02-29T00:00:58.071 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-29T00:00:58.214 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603/remote/smithi084 2024-02-29T00:00:58.214 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-29T00:00:58.323 INFO:teuthology.task.internal:Removing archive directory... 2024-02-29T00:00:58.323 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-29T00:00:58.331 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-29T00:00:58.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-29T00:00:58.458 INFO:teuthology.task.internal:Not uploading archives. 2024-02-29T00:00:58.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-29T00:00:58.528 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-29T00:00:58.528 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-29T00:00:58.536 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-29T00:00:58.588 INFO:teuthology.orchestra.run.smithi057.stdout: 395173 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 29 00:00 /home/ubuntu/cephtest 2024-02-29T00:00:58.588 INFO:teuthology.orchestra.run.smithi057.stdout: 917879 4 d--------- 2 ubuntu ubuntu 4096 Feb 28 23:04 /home/ubuntu/cephtest/mnt.0 2024-02-29T00:00:58.589 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-29T00:00:58.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-29T00:00:58.592 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-29T00:00:58.592 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1fb5d10608dccee73edd6ef8a2c7b804a097c324/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi057 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-29T00:00:58.593 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-29T00:00:58.606 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-29T00:00:58.713 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-29T00:00:58.713 INFO:teuthology.nuke:Checking targets against current locks 2024-02-29T00:00:58.739 DEBUG:teuthology.nuke:shortname: smithi057 2024-02-29T00:00:58.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-29T00:00:58.763 DEBUG:teuthology.nuke:shortname: smithi084 2024-02-29T00:00:58.763 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-29T00:00:58.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-28 22:33:06.857282', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPyUeAHE+pp+qsi9PwfFSyTqcvIpf+MMHyBwtJQS5LT0oRTI8OnDc5En1PxVOxfZmoOK0RXw+Ryr8x9ys2OdmY='} 2024-02-29T00:00:58.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-28 22:33:06.855821', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYrpinHrdS6obIeVRroouFicW2iMRUs07L2SfDCXTt8b/FteHe2nSgbFWebkSSZsXWqLRD9kdEZo3L9LzMCNLk='} 2024-02-29T00:00:58.820 INFO:teuthology.orchestra.console.smithi057:Power off 2024-02-29T00:00:58.820 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-29T00:00:58.849 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2024-02-29T00:00:58.849 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:00:58.863 INFO:teuthology.orchestra.console.smithi084:Power off 2024-02-29T00:00:58.863 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-29T00:00:58.874 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2024-02-29T00:00:58.884 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2024-02-29T00:00:58.884 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:00:58.906 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-02-29T00:01:02.879 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:01:02.907 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:01:02.989 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2024-02-29T00:01:03.016 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-02-29T00:01:06.994 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:01:07.016 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-29T00:01:07.104 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2024-02-29T00:01:07.124 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-02-29T00:01:07.205 INFO:teuthology.orchestra.console.smithi057:Power off completed 2024-02-29T00:01:07.225 INFO:teuthology.orchestra.console.smithi084:Power off completed 2024-02-29T00:01:07.329 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4609.126544237137 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: bd7d00fb724897dda560070d01bc6e5c80881f15 host.b-kernel-sha1: bd7d00fb724897dda560070d01bc6e5c80881f15 owner: scheduled_vshankar@teuthology status: fail success: false 2024-02-29T00:01:07.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-29T00:01:07.415 INFO:teuthology.run:FAIL